Control method, audio playing method, related device, system and vehicle

Abstract

A control method, an audio playing method, and a related apparatus, system, vehicle, applicable to a vehicle equipped with a display device (220), the method comprising: acquiring audio data to be played; controlling a display effect of a display device (220) of the vehicle according to characteristics of the audio data; features of the audio data include one or more of a location and a type of sound. The invention further comprises a controller (210), a player, a computer readable storage medium and a computer program product, the display effect of the display device (220) of the vehicle is controlled based on the audio data to be played, the display effect can correspond to the sound played in the vehicle more accurately, the linkage effect between the display effect and the sound played in the vehicle is better, and the user experience is improved. The use experience of a user can be improved.

Description

Control method, audio playback method and related device, system, and vehicle Technical Field

[0001] The present application relates to the field of intelligent control technology, and in particular to a control method, an audio playback method, and related devices, systems, and vehicles. Background Art

[0002] With the advancement of technology and improvements in living standards, people are increasingly valuing comfort. For example, vehicles are often equipped with music players, which people use to relax while riding. Many vehicles also feature ambient lighting, which creates visual effects with shifting colors and brightness, enhancing the passenger experience.

[0003] With the development of intelligent vehicles, it is now possible to control the rhythm of ambient lighting at the corresponding location based on the location of the sound collected from the vehicle. However, the linkage between the rhythm of the ambient lighting and the sound presented by this solution is not very effective. For example, due to the complex sound environment and poor sound reception conditions inside the vehicle, it is difficult to accurately locate the location of the sound based on the sound collected from the vehicle, resulting in poor linkage between the sound and the ambient lighting.

[0004] Summary of the Invention

[0005] The present application provides a control method, an audio playback method, and related devices, systems, and vehicles, in order to achieve a better linkage effect between the display effect and sound of the vehicle.

[0006] In a first aspect, the present application provides a control method, which includes: obtaining audio data to be played; controlling the display effect of a vehicle's display device based on characteristics of the audio data; the characteristics of the audio data include one or more of the location and type of sound.

[0007] The audio data to be played is audio data that is decoded by a player or controller and played by a speaker. Based on the audio data to be played, characteristics of the audio data can be determined, such as the location and / or type of the sound corresponding to the audio data, and the display effect of the display device configured in the vehicle can be controlled accordingly.

[0008] Based on the above, the display effect of the vehicle's display device can be controlled based on the characteristics of the audio data to be played. Determining the characteristics of the audio data based on the audio data to be played, rather than based on the sound collected in the vehicle, can make the determined audio data characteristics more accurate, thereby enabling more accurate control of the display effect of the display device. In this way, when the sound corresponding to the audio data is played in the vehicle, the correspondence between the display effect of the display device and the played sound can be made more accurate, which can provide users with a better user experience.

[0009] The above method can be applied to a controller configured in a vehicle. The controller can be, for example, a processor, a chip, or a chip system configured in the vehicle, and this application does not impose any restrictions on this.

[0010] Optionally, the position of the sound includes the position of a sound source included in the sound.

[0011] The characteristics of the audio data may include the location of the sound, which may include the location of the sound source included in the sound. The sound source refers to the sound source that emitted the sound when the audio data was collected. The display effect of the display device can be controlled based on the location of the sound source.

[0012] Optionally, the position of the sound includes the playback position of the sound.

[0013] The characteristics of the audio data may include the location of the sound, which may include the playback location of the sound, such as the location of the speaker used to play the sound. The display effect of the display device can be controlled based on the playback location of the sound.

[0014] Optionally, the type of sound includes the type of sound source.

[0015] The characteristics of the audio data may include the type of sound, which may include the type of sound source that emits the sound when the audio data is collected. The display effect of the display device may be controlled based on the type of sound.

[0016] In combination with the first aspect, in some possible implementations of the first aspect, the display device of the vehicle includes at least one of an ambient light and a display, and the display effect includes the luminous position, brightness, color, saturation, luminous time of the ambient light on the vehicle and at least one of the image and brightness of the display.

[0017] The display device configured in the vehicle may include an ambient light, a display, or both. The display effect presented by the ambient light is the lighting effect of the ambient light, which can be the lighting state (including brightness, color, or saturation, etc.) of the ambient light at the same or different locations at different times. The display effect presented by the display includes images presented at different times at the same or different locations on the display, or changes in the brightness of the display, etc.

[0018] Based on the characteristics of the audio data to be played, the lighting effects of the ambient light and the display effects of the display can be controlled. Based on the audio data to be played, the characteristics of the audio data can be accurately determined, and the lighting effects of the ambient light and / or the display effects of the display can be accurately controlled. When the sound corresponding to the audio data is played, the lighting effects of the ambient light or the display effects of the display can be accurately linked to the sound, thereby improving the user experience.

[0019] The display effect of the display device can be adapted to the characteristics of the audio data.

[0020] In a possible implementation, the display effect of the display device is adapted to the position of the sound source.

[0021] Features of the audio data may include one or more sound source locations, and the display effects of the vehicle's display device may be adapted to the locations of the sound sources. For example, the vehicle may be equipped with one or more display devices, each of which may correspond to a sound source location, or multiple areas on a display device may each correspond to a sound source location. The vehicle's display device may be an ambient light that illuminates at locations corresponding to the sound source locations; the vehicle's display device may also be a display that displays images in areas corresponding to the sound source locations.

[0022] In this way, when the characteristics of the audio data include the position of the sound source, the display effect of the display device can also reflect the position of the sound source. When the position of the sound source changes, the light-emitting position of the atmosphere light or the display area of ​​the display also changes accordingly, thereby realizing the linkage between the display effect and the position of the sound source.

[0023] In a possible implementation, the display effect of the display device is adapted to the playback position of the sound.

[0024] Features of the audio data may include the playback location of the sound, such as the location of the speaker playing the sound. The display effects of the vehicle's display device may be adapted to the playback location of the sound. For example, the vehicle may be equipped with one or more display devices, each of which may correspond to a playback location, or multiple areas on a display device may each correspond to a playback location. The vehicle's display device may be an ambient light that illuminates at a location corresponding to the playback location of the sound; the vehicle's display device may also be a display that displays an image at a location corresponding to the playback location of the sound.

[0025] In this way, when the characteristics of the audio data include the playback position of the sound, the display effect of the display device can also correspond to the playback position. For example, it can be displayed in the area at the same position as the speaker that plays the sound. When the playback position of the sound changes, the light-emitting position of the atmosphere light or the display area of ​​the display also changes accordingly, thereby realizing the linkage between the display effect and the playback position.

[0026] In a possible implementation, the display effect of the display device is adapted to the type of the sound source.

[0027] Features of audio data may include the type of sound, such as the type of sound source. The display effects of a vehicle's display device may be adapted to the type of sound source. For example, a vehicle may be equipped with one or more display devices, each corresponding to a specific sound source type, or multiple areas on a display device may each correspond to a specific sound source type. A vehicle's display device may be an ambient light that illuminates at a location corresponding to the type of sound source; or a display device may be a monitor that displays an image at a location corresponding to the type of sound source.

[0028] The color, brightness or saturation of the display device can also correspond to the type of the sound source. For example, when the sound source is a human, the display device can appear red, and when the sound source is an animal, the display device can appear green.

[0029] When the display device is a monitor, the image in the monitor may also correspond to the type of the sound source. For example, when the sound source is a person, the image in the monitor may be an image of a person; when the sound source is a bird, the image in the monitor may be an image of a bird.

[0030] In this way, when the characteristics of the audio data include the type of sound source, the display effect of the display device can correspond to the type of sound source. When the type of sound source changes, the display effect can change accordingly, thereby realizing the linkage between the display effect and the type of sound source.

[0031] The characteristics of the audio data may further include one or more of the loudness, beat, or fundamental frequency of the sound. The display effect of the display device may be controlled based on one or more of the loudness, beat, or fundamental frequency of the sound.

[0032] Optionally, the brightness of the display device corresponds to the loudness of the sound. For example, the loudness value of the sound can be determined based on the audio data, and the brightness value of the display device can be obtained according to a predefined correspondence, thereby controlling the brightness of the display device.

[0033] In this way, the display device can express the loudness change of the sound through the change of brightness. When the sound is played, the brightness and loudness are linked. For example, when the loudness of the sound increases, the brightness of the display device also increases.

[0034] Optionally, the color of the display device corresponds to the beat of the sound. For example, the beat value of the sound can be determined based on the audio data, and the color value of the display device can be obtained according to a predefined correspondence, thereby controlling the color of the display device.

[0035] In this way, the display device can express the rhythm changes of the sound through color changes, and realize the linkage between color and rhythm when the sound is played.

[0036] Optionally, the saturation of the display device corresponds to the fundamental frequency of the sound. For example, the fundamental frequency value of the sound can be determined based on the audio data, and the saturation value of the display device can be obtained according to a predefined corresponding relationship, thereby controlling the saturation of the display device.

[0037] In this way, the display device can express the change of the fundamental frequency of the sound through the change of saturation, and realize the linkage between saturation and fundamental frequency when the sound is played.

[0038] Optionally, the display device is an ambient light. The lighting effects of the ambient light can correspond to characteristics of the audio data, for example, the brightness of the ambient light can correspond to the loudness of the sound, the color of the ambient light can correspond to the beat of the sound, and the saturation of the ambient light can correspond to the fundamental frequency of the sound.

[0039] It should be understood that the display effects, characteristics of audio data, or the correspondence between the display effects of the display device and the characteristics of audio data in the above content are only illustrations given in this application. This application does not limit the content included in the display effects of the display device, nor does it limit the content included in the characteristics of audio data, nor does it limit the specific correspondence between the display effects and the characteristics of audio data.

[0040] Based on the above technical content, the display effect of the vehicle's display device can be controlled based on the audio data to be played. The characteristics determined from the audio data to be played (such as the position, type, loudness, beat, fundamental frequency, etc. of the sound) are more accurate, so the display effect of the display device can be controlled more accurately. When the sound corresponding to the audio data is played, the display effect and the sound can be accurately corresponded, presenting a better linkage effect, and bringing a better user experience.

[0041] In a second aspect, the present application provides an audio playback method, the method comprising: obtaining audio data to be played; playing a sound corresponding to the audio data, wherein the playback position of the sound is determined by the type of the sound.

[0042] According to the audio data to be played, the type of sound can be determined, and based on the type of sound, the position of the speaker corresponding to the type of sound is determined, and the speaker at the corresponding position is controlled to play the sound.

[0043] The type of sound may be, for example, the type of the sound source that emits the sound. For example, the sounds corresponding to the audio data include piano and violin sounds. The speakers in front of the vehicle may be controlled to play the sound of the piano, and the speakers at the rear of the vehicle may be controlled to play the sound of the violin.

[0044] Based on the above technical solution, speakers at different positions in the car can play different types of sounds, making the played sounds have a sense of space, which can improve the user experience.

[0045] The above method can be applied to vehicles, for example, executed by a player configured in the vehicle, which can be implemented by software, hardware, or a combination of software and hardware, for example, it can include a processor and a speaker; the method can also be executed by a controller configured in the vehicle, which can be implemented by software, hardware, or a combination of software and hardware, for example, the controller can be a chip or chip system configured in the vehicle.

[0046] In a possible implementation, the audio data carries type information of the sound, where the type information indicates the type of the sound.

[0047] The audio data may carry type information, which may indicate the type of the sound, thereby determining the position of the speaker playing the sound.

[0048] In a possible implementation, the type of the sound is obtained based on separation of the audio data.

[0049] Even if the audio data does not carry type information, the sound type can be separated from the audio data through the sound source separation algorithm, and the position of the speaker playing the sound can be controlled accordingly, so that the played sound has a sense of space.

[0050] In a third aspect, the present application provides a controller comprising modules or units for implementing the method in the first aspect and any possible implementation of the first aspect. It should be understood that each module or unit can implement the corresponding function by executing a computer program.

[0051] In a fourth aspect, the present application provides a controller comprising at least one processor, wherein the at least one processor is used to implement the control method in the first aspect and any possible implementation manner of the first aspect.

[0052] The controller may further include a memory for storing instructions and data. The memory is coupled to the processor, and when the processor executes the instructions stored in the memory, the method of the first aspect and any possible implementation of the first aspect may be implemented. The controller may further include a communication interface for communicating between the controller and other devices. Exemplarily, the communication interface may be a transceiver, circuit, bus, module, or other type of communication interface.

[0053] In a fifth aspect, the present application provides a player, comprising modules or units for implementing the method in the second aspect and any possible implementation of the second aspect. It should be understood that each module or unit can implement the corresponding function by executing a computer program.

[0054] In a sixth aspect, the present application provides a player comprising a processor and a speaker, wherein the processor is used to control the speaker to execute the audio playback method described in the second aspect and any possible implementation of the second aspect.

[0055] The player may further include a memory for storing instructions and data. The memory is coupled to the processor, and when the processor executes the instructions stored in the memory, the method described in the second aspect and any possible implementation of the second aspect may be implemented. The apparatus may further include a communication interface for communicating with other devices. Exemplarily, the communication interface may be a transceiver, circuit, bus, module, or other type of communication interface.

[0056] In a seventh aspect, the present application provides a control system comprising a controller, a speaker, and a display device. The controller is configured to control the speaker and / or the display device to implement the method of the first aspect and any possible implementation of the first aspect, and / or to implement the method of the second aspect and any possible implementation of the second aspect. The speaker is configured to play sound in response to control by the controller; and the display device is configured to present a display effect in response to control by the controller.

[0057] In an eighth aspect, the present application provides a vehicle comprising at least one of the controller described in the third or fourth aspect, the player described in the fifth or sixth aspect, and the control system described in the seventh aspect.

[0058] In a ninth aspect, the present application provides a chip system comprising at least one processor for supporting the implementation of the functions involved in the above aspects, for example, processing one or more of the data and information involved in the methods involved in the above aspects.

[0059] In one possible design, the chip system further includes a memory, which is used to store program instructions and data, and the memory is located inside or outside the processor.

[0060] The chip system can be composed of chips, or can include chips and other discrete devices.

[0061] In a tenth aspect, the present application provides a computer-readable storage medium, comprising a computer program, which, when executed on a computer, enables the computer to implement the methods involved in the above aspects.

[0062] In an eleventh aspect, the present application provides a computer program product, which includes: a computer program (also referred to as code, or instruction), which enables the methods involved in the above aspects to be executed when the computer program is run.

[0063] It should be understood that the third to eleventh aspects of the present application correspond to the technical solutions of the first and second aspects of the present application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar and will not be repeated. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] FIG1 is a schematic diagram of an application scenario applicable to an embodiment of the present application;

[0065] FIG2 is a schematic diagram of a control system provided in an embodiment of the present application;

[0066] FIG3 is a schematic flow chart of a control method provided in an embodiment of the present application;

[0067] FIG4 is a schematic diagram of a cabin space division method according to an embodiment of the present application;

[0068] 5 and 6 are schematic diagrams of the light-emitting position of the ambient light and the position of the sound source according to an embodiment of the present application;

[0069] FIG7 is a schematic diagram of a correspondence between characteristics of audio data and display effects of a display screen provided in an embodiment of the present application;

[0070] FIG8 is a schematic flow chart of an audio playback method provided in an embodiment of the present application;

[0071] 9 and 10 are schematic diagrams of a cabin space division method according to an embodiment of the present application;

[0072] FIG11 is a schematic diagram of the positions of the sound source and the loudspeaker provided in an embodiment of the present application;

[0073] FIG12 is a schematic block diagram of a controller provided in an embodiment of the present application;

[0074] FIG13 is another schematic block diagram of a controller provided in an embodiment of the present application;

[0075] FIG14 is a schematic block diagram of a player provided in an embodiment of the present application;

[0076] FIG15 is another schematic block diagram of the player provided in an embodiment of the present application. DETAILED DESCRIPTION

[0077] The technical solution in this application will be described below with reference to the accompanying drawings.

[0078] The method provided in the embodiments of the present application can be applied to a vehicle equipped with a display device. The vehicle includes but is not limited to a vehicle, an airplane, or a ship; the display device includes but is not limited to an ambient light or a display, such as a device capable of emitting light or displaying an image.

[0079] In an embodiment of the present application, the display device can present a display effect based on predefined control parameters.

[0080] Exemplarily, when the display device is a display, the display effect may be presenting the same or different images at the same or different positions in the display, including the type of image presented, and changes in the type and position of the image over time. The change in the type of image over time may be manifested as the movement of the image in the display, and the change in the position of the image over time may be manifested as the movement of the image in the display. When multiple displays are configured, the display effect also includes presenting the same or different images in displays at different positions. The display effect of the display also includes the brightness of the display. For example, different areas of the display may present the same or different brightness, and the brightness of the display may change over time. When multiple displays are configured, different displays may also present the same or different brightness.

[0081] When the display device is an ambient light, the display effect can be the ambient light's lighting effect. This ambient light effect includes the time-dependent changes in the lighting state of one or more ambient lights, either individually or collectively, including but not limited to gradual changes in the color of the ambient light, changes in the brightness of the ambient light, or periodic changes in the lighting state of ambient lights at different locations. It should be understood that an ambient light can have multiple lighting states, such as color, brightness, or saturation, and the ambient light's lighting effect can also be a change in multiple lighting states. The ambient light's lighting effect can also be the same lighting state exhibited by ambient lights in the same or different locations at different times.

[0082] In the embodiments of the present application, the atmosphere light may have various forms, for example, it may be in the form of lamp beads or strip-shaped light strips, etc., and the present application does not impose any restrictions on this.

[0083] The vehicle may also be provided with a loudspeaker, which may play sound based on audio data. In an embodiment of the present application, the data used to store sound in a storage medium is referred to as an audio file, and the data used to play the audio file after decoding is referred to as audio data. To achieve the acquisition and storage of sound, it is necessary to sample, quantize and encode the sound, and use a binary number to represent the amplitude of the sound signal, thereby achieving the storage of the sound. This process is called encoding the sound. There may be a variety of encoding formats for sound, such as pulse code modulation (PCM) format, adaptive differential pulse code modulation (ADPCM) format, advanced audio coding (AAC) format, etc. For audio files obtained using different encoding formats, audio data that can be played by a loudspeaker may be obtained through a corresponding decoding process.

[0084] Figure 1 is a schematic diagram of an application scenario applicable to an embodiment of the present application. As shown in Figure 1, the scenario shows a vehicle cabin 100 equipped with ambient lights and speakers, wherein ambient lights 110a to 110f and speakers 120a to 120f are installed in the cabin 100.

[0085] Each of the ambient lights 110a to 110f can exhibit the same or different lighting states over time, for example, periodic brightness changes, periodic color changes, or both brightness and color changes. In this embodiment of the present application, ambient lights located in different locations in the cabin can exhibit different lighting states at the same time, and different parts of a single ambient light can also exhibit different lighting states.

[0086] The speakers 120a to 120f can each independently emit sound, or can emit the same or different sounds simultaneously. The speakers can cooperate with each other to emit stereo sound.

[0087] The lighting state of the ambient light can be linked to the sound emitted by the speaker. For example, when the speaker plays sounds with different characteristics, the ambient light can display different lighting states. For example, the sound characteristics may include the loudness and rhythm of the sound. When the loudness of the sound changes, the saturation of the ambient light can change accordingly; when the rhythm of the sound changes, the brightness of the ambient light can change accordingly.

[0088] It should be understood that the scene shown in Figure 1 is only an example in which the display device is an ambient light. The present application does not limit the form and number of ambient lights or speakers configured in the cabin, nor does it limit the position of each ambient light or each speaker in the cabin. It should also be understood that the display device installed in the cabin shown in Figure 1 may also be a display, and the present application does not limit this.

[0089] At present, to achieve the linkage between ambient lights and sounds inside the car, it is usually necessary to collect the sounds inside the car, then perform feature extraction and positioning, and on this basis determine the parameters for controlling the ambient lights and control the lighting status of the ambient lights at the corresponding positions.

[0090] However, this solution does not provide a good linkage effect between the lighting effects of the ambient light and the sound. For example, to accurately capture sound, and then accurately identify and locate the sound based on the captured sound, specialized sound reception conditions are required, such as multiple microphone arrays. Such conditions are not usually available in cars, and the sound environment inside a car is complex. Therefore, the sound captured based on the sound played inside the car is easily distorted, making it difficult to accurately determine the location of the sound inside the car based on the captured sound. If the determined sound location is inaccurate, the position of the ambient light determined based on this location will not accurately correspond to the location of the sound inside the car, resulting in a poor linkage effect between the ambient light and the sound inside the car.

[0091] To this end, an embodiment of the present application provides a control method, which obtains audio data to be played and controls the display effect of the vehicle's display device (such as the lighting effect of the ambient light) according to the characteristics of the audio data, so that the display effect of the display device and the sound played according to the audio data have a better linkage effect.

[0092] For ease of understanding, the following explanation is first given:

[0093] In this application, "at least one" means one or more, and "more" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b and c can mean: a, or b, or c, or a and b, or a and c, or b and c, or a, b and c, where a, b, c can be single or multiple.

[0094] In this application, "predefinition" or "preconfiguration" can be implemented by pre-saving corresponding codes, tables or other methods that can be used to indicate relevant information in a device (for example, including a controller and a player), and this application does not limit its specific implementation method. Among them, "saving" can mean saving in one or more memories. The one or more memories can be set separately or integrated in a controller, player, processor, or terminal. The one or more memories can also be partially set separately and partially integrated in a controller, player, processor, or terminal. The type of memory can be any form of storage medium, and this application does not limit it.

[0095] Next, the system, method, and device provided in the embodiments of the present application are described in detail with reference to the accompanying drawings.

[0096] Figure 2 is a schematic diagram of a control system provided in an embodiment of the present application, and the control system 200 can be configured in the cockpit of a vehicle for implementing the control method provided in the present application. As shown in Figure 2, the control system 200 may include: a controller 210, a display device 220 and a speaker 230. The devices can communicate with each other through an in-vehicle communication link. As an example and not a limitation, the in-vehicle communication link may include: a controller area network (CAN) bus, a CAN bus with a flexible data rate (FD) (abbreviated as CAN FD), a local interconnect network (LIN) bus, a flex ray bus, a media oriented systems transport (MOST) bus, an in-vehicle Ethernet, etc.

[0097] The controller 210 may be a processor or an onboard terminal configured on the vehicle, and may obtain the audio data to be played via an in-vehicle communication link. The controller 210 may also obtain the audio data to be played via a mobile cellular network, etc. The controller 210 may also obtain an audio file and decode the audio file to obtain the audio data to be played. The controller 210 may control the speaker 230 to play sound or control the display device 220 to present a display effect via the in-vehicle communication link. The controller 210 may control the speaker 230 to play sound and the display device 220 to present a display effect simultaneously or independently.

[0098] The display device 220 may be a device capable of emitting light or displaying images, such as an ambient light or a display. The display device 220 may present a display effect in response to the control of the controller 210 .

[0099] The speaker 230 may emit sound in response to the control of the controller 210 .

[0100] In an embodiment of the present application, the controller 210 can control the speaker 230 to play the sound corresponding to the audio data based on the characteristics of the acquired audio data to be played, and can also control the display device 220 to present a display effect based on the characteristics of the audio data. The characteristics of the audio data in the embodiment of the present application include one or more of the type of sound played based on the audio data and the position of the sound. The position of the sound includes the position of the sound source and the playback position of the sound. The sound source refers to the sound source that emits the sound when the audio data is collected. The position of the sound source includes the position of the sound source that emits the sound when the sound is collected. The playback position of the sound includes the position of the speaker that plays the sound when the sound is played. The type of sound includes the type of sound source that emits the sound when the audio data is collected. The characteristics of the audio data can also include audio characteristics (such as loudness, beat or fundamental frequency, etc.) of the sound played based on the audio data.

[0101] Optionally, the control system 200 may further include a player for acquiring audio data and playing sounds corresponding to the audio data.

[0102] It should be understood that the embodiments of the present application do not limit the number of controllers 210, display devices 220 or speakers 230 in the control system 200. For example, the control system 200 may include multiple display devices (such as the ambient lights 110a to 110f in the cockpit 100), and the controller 210 may control the display effect of each display device respectively; for another example, the control system 200 may include multiple speakers (such as the speakers 120a to 120f in the cockpit 100), and the controller 210 may control each speaker to play sound respectively; for another example, the control system 200 may include multiple controllers, one or more of which can be used to control the display effect of the display device, and one or more can be used to control the speaker to play sound.

[0103] In some embodiments, the controller 210 and the display device 220 may be integrated, and the controller 210 and the speaker 230 may also be integrated.

[0104] Next, the method executed by the controller is described in detail with reference to the accompanying drawings.

[0105] FIG3 is a schematic flow chart of a control method provided in an embodiment of the present application. The method shown in FIG3 can be executed by a controller, for example, the controller shown in FIG2. The controller can be implemented by software, hardware, or a combination of software and hardware. The controller can be configured in a vehicle, for example, a chip or chip system configured in the vehicle, and can be part of the vehicle's control system.

[0106] The control method shown in FIG3 is described below with the controller as the execution subject.

[0107] As shown in Figure 3, the control method 300 may include step 310 and step 320. Each step in the method 300 is described in detail below.

[0108] In step 310 , audio data to be played in the vehicle is obtained.

[0109] In one possible implementation, the controller obtains audio data to be played in the vehicle from a player. The vehicle may be equipped with a player that, under the control of a predefined program or user instructions, decodes an audio file from a storage medium to obtain audio data, and controls the speakers to play sound based on the audio data. The controller can obtain the audio data obtained by the player from decoding the audio file.

[0110] In one possible implementation, the controller may obtain an audio file from a storage medium, and decode the audio file based on a format of the audio file to obtain audio data to be played.

[0111] In an embodiment of the present application, the controller may also replace the player and control the speaker to play sound based on the audio data.

[0112] In step 320 , the controller controls the display effect of the vehicle's display device according to the characteristics of the audio data.

[0113] The characteristics of the audio data include one or more of the location and type of the sound.

[0114] The location of a sound may include one or more of the location of the sound source emitting the sound and the location where the sound is played.

[0115] When audio data is played simultaneously by two or more speakers, the sounds played by each speaker can be coordinated, allowing the listener to perceive the sound as emanating from a corresponding sound source in the surrounding space. This type of sound is called spatial audio. The location of the sound source is the location specified when the spatial audio source is created, or the location of the sound source emitting the sound when the spatial audio data is collected. It should be understood that the location of the sound source can be fixed or variable, for example, it can change over time.

[0116] The playback location of a sound includes the location of the speakers used to play the sound.

[0117] The type of sound includes the type of sound source that produces the sound when the audio data is collected, or a specified sound source type. For example, the sound corresponding to the audio data may include the sound of a piano, the sound of a violin, and the sound of a person.

[0118] The type of sound may also include the frequency band of the sound. For example, the sound may be divided into a low-frequency sound, a mid-frequency sound, and a high-frequency sound. For example, a sound with a frequency between 20 Hz and 160 Hz may be divided into a low-frequency sound, a sound with a frequency between 160 Hz and 2500 Hz may be divided into a mid-frequency sound, and a sound with a frequency between 2500 Hz and 20,000 Hz may be divided into a high-frequency sound. The sound corresponding to the audio data may include a low-frequency sound, a mid-frequency sound, and a high-frequency sound.

[0119] It should be understood that the examples of the types of sounds in the embodiments of the present application are only given for ease of understanding, and the present application does not limit the way of classifying the types of sounds.

[0120] The characteristics of the audio data may be indicated by indication information (such as position information or type information) included in the audio data, or may be separated from the audio data by a controller.

[0121] In some embodiments, the audio data may include one or more of sound type information and position information. The position of the sound may be indicated by the sound position information, and the type of the sound may be indicated by the sound type information.

[0122] For example, the position information of a sound may indicate the position of a speaker used to play the sound. The position information may also indicate the position of the sound source. When the position of the sound source changes over time, the position information may further indicate the position of the sound source at different moments.

[0123] The type information of the sound can indicate the type of the sound. For example, when the sound corresponding to the audio data includes the sound of a piano, the sound of a violin, and the sound of a human, the type information can indicate that the sound is produced by a piano, a violin, or a human; when the sound corresponding to the audio data includes the sound of a low frequency band, the sound of a mid-frequency band, and the sound of a high frequency band, the type information can indicate that the sound belongs to the low frequency band, the mid-frequency band, or the high frequency band.

[0124] The audio data may include one or more of type information and position information.

[0125] Exemplarily, the audio data may include data from multiple channels, and the data from each channel may correspond to the location of a different sound, or may correspond to a different type of sound. The audio data may include one or more of position information and type information. The position information or type information of the sound corresponding to the data of each channel may be included in the data of each channel, or may exist independently of the data of each channel. One or more of the above-mentioned position information and type information may be, for example, metadata carried in the audio data.

[0126] For example, the audio data may be 5.1 surround sound audio data, including 6 channels of data (5 full-band channels and one low-frequency effect channel), each channel corresponding to a speaker. When the audio data is played, each speaker plays sound based on the data of its corresponding channel.

[0127] The audio data may also be three-dimensional vivid audio data, in which different types of sounds are stored in different channels, and each channel also includes metadata for carrying position information and type information of the sound corresponding to the channel.

[0128] In some embodiments, the controller may separate data corresponding to the features of the audio data from the audio data using a predefined sound source separation algorithm.

[0129] The controller may separate data corresponding to different sound types from the audio data.

[0130] The controller can separate different types of sounds from the audio data through a sound source separation method. For example, it can use a deep neural network model to identify different types of sounds in the audio data and separate the audio data into multiple channels according to the type of sound, or separate the data corresponding to different types of sounds in the audio data through a separation method based on non-negative matrix factorization (NMF).

[0131] The controller may also separate data corresponding to different sound source positions from the audio data.

[0132] When the audio data acquired by the controller includes data from multiple channels picked up by multiple microphones, the controller may process the data from the multiple channels using a sound source separation method to identify the location of the sound source when the sound was collected. For example, the location of the sound source corresponding to the audio data may be determined using a time difference of arrival (TDOA) method. The controller may separate the audio data into data corresponding to different sound source locations.

[0133] Regarding the method by which the controller separates data corresponding to the features of the audio data from the audio data, reference may be made to the methods in the prior art, which will not be described in detail here.

[0134] In this way, when the audio data acquired by the controller does not carry information indicating the characteristics of the audio data, the controller can still obtain the characteristics of the audio data, and then control the display effect of the display device accordingly.

[0135] The display effect of the display device configured in the vehicle can be adapted to the location of the sound, such as the location of the sound source, or the location where the sound is played; the display effect can also be adapted to the type of sound, such as the type of sound source that emits the sound.

[0136] The display effect is adapted to the location of the sound, including the display device being able to display at a location corresponding to the location of the sound, such as an ambient light being able to emit light at a corresponding light-emitting location, or a display being able to display an image at a corresponding display location. The correspondence between the light-emitting location or display location and the location of the sound can be predefined or user-specified, and this application does not limit this.

[0137] The display effect is adapted to the type of sound, and the display device can display at a position corresponding to the type of sound. The correspondence between the light-emitting position or display position and the type of sound can be predefined or user-specified, and this application does not limit this.

[0138] The display effect is adapted to the type of sound, and also includes the display device being able to present corresponding brightness, color or saturation based on the type of sound and the predefined correspondence between the type and brightness, color or saturation.

[0139] The display device configured in the vehicle may include one or more of an ambient light or a display. When the display device is an ambient light, the display effect may include one or more of the ambient light's lighting position, brightness, color, saturation, or lighting duration. When the display device is a display, the display effect may include one or more of the display's brightness, color, or saturation, and may also include the image displayed on the display.

[0140] The features of audio data may also include audio features of the sound, such as the loudness, beat or fundamental frequency of the sound. The display effect of the display device may correspond to the loudness of the sound, for example, the brightness, color or saturation of the display device may correspond to the loudness of the sound; the display effect of the display device may also correspond to the beat of the sound, for example, the brightness, color or saturation of the display device may correspond to the beat of the sound; the display effect of the display device may correspond to the fundamental frequency of the sound, for example, the brightness, color or saturation of the display device may correspond to the fundamental frequency of the sound.

[0141] In this way, the characteristics of the audio data can be expressed through the display effects of the display device, and the linkage between the display effects and the sound can be achieved when the sound is played.

[0142] It should be understood that the embodiments of the present application do not limit the content included in the display effect of the display device, nor do they limit the content included in the characteristics of the audio data, nor do they limit the specific correspondence between the display effect and the characteristics of the audio data.

[0143] Next, taking the display devices as atmosphere lights and monitors as examples, the method for controlling the display effects of the display device based on the characteristics of audio data provided by this application is described in detail.

[0144] In some embodiments, the display device is an ambient light configured in the vehicle, and the display effect of the vehicle can be the lighting effect presented by the ambient light of the vehicle, including but not limited to the lighting position of the ambient light on the vehicle, the color, brightness, saturation and lighting time of the ambient light, etc.

[0145] The controller can determine the light-emitting position of the atmosphere light based on the type or position of the sound corresponding to the audio data, and determine the control parameters of the atmosphere light corresponding to the light-emitting position based on the audio characteristics of the sound corresponding to the corresponding type or position, so as to control the lighting state of the atmosphere light and make the atmosphere light present a lighting effect.

[0146] In a possible implementation, the lighting position of the ambient light may be determined by the type of sound corresponding to the audio data, and the lighting position of the ambient light and the type of sound may satisfy a predefined correspondence.

[0147] For example, the lighting position of the ambient light may correspond to the type of sound source emitting the sound.

[0148] Taking cockpit 100 as an example, the sounds corresponding to the audio data to be played may include sounds from different sound sources, such as piano, violin, male voice, and female voice. The lighting state of ambient light 110a may correspond to the audio characteristics of the piano sound, the lighting state of ambient light 110d may correspond to the audio characteristics of the violin sound, the lighting states of ambient lights 110b and 110c may correspond to the audio characteristics of a male voice, and the lighting states of ambient lights 110e and 110f may correspond to the audio characteristics of a female voice.

[0149] It should be understood that the present application does not limit the correspondence between the position of the ambient light and the type of sound.

[0150] The controller can also divide the space into multiple areas, and control the lighting effects of the atmosphere lights in the corresponding areas based on the correspondence between the areas and the types of sounds.

[0151] Taking cockpit 100 as an example, the sounds corresponding to the audio data to be played may include sounds from different types of sound sources, such as the sound of a piano, the sound of a violin, a male voice, and a female voice. The controller may divide cockpit 100 into zones 4A-4D as shown in FIG4 . The lighting state of the ambient lights in zone 4A (including ambient lights 110f and 110e) may correspond to the audio characteristics of the sound of a piano, the lighting state of the ambient lights in zone 4B (including ambient light 110a) may correspond to the audio characteristics of the sound of a violin, the lighting state of the ambient lights in zone 4C (including ambient lights 110b and 110c) may correspond to the audio characteristics of a male voice, and the lighting state of the ambient lights in zone 4D (including ambient light 110d) may correspond to the audio characteristics of a female voice.

[0152] It should be understood that the area division shown in Figure 4 is only an example in the embodiments of this application. This application does not limit the number of areas into which the controller divides the space, nor does it limit the division method. The controller can divide the space into any number of areas. In some embodiments, different parts of an atmosphere light can belong to different areas, and different parts of the atmosphere light can present different lighting effects.

[0153] It should be understood that there can be different ways to classify the types of sounds, such as including the frequency bands of the sounds, etc. The correspondence between the light-emitting positions and other types of sounds can refer to the correspondence between the light-emitting positions and the types of sound sources, and will not be repeated here.

[0154] The correspondence between the type of sound and the area where the ambient light is located may also be other predefined correspondences, and the correspondence may also change over time, and the correspondence may also be set by the user.

[0155] In a possible implementation, the lighting position of the ambient light may be determined by the position of the sound corresponding to the audio data, and the lighting position of the ambient light and the position of the sound may satisfy a predefined correspondence.

[0156] For example, the lighting position of the ambient light can be determined by the sound source position of the sound.

[0157] Based on a preset reference point, the controller can divide the cabin horizontally into multiple zones with equal or different angles. Based on the audio characteristics of the sound source, the controller can control the lighting state of the ambient lighting in the same zone as the sound source. The preset reference point can be the center point of the entire cabin, the center point of the front seats, the center point of the rear seats, or any other arbitrary location. This application does not limit this, nor does it limit the method of dividing the zones or the number of zones.

[0158] 5 , taking cabin 100 as an example, cabin 100 is equipped with ambient lights 110a-110f. The controller can obtain audio data to be played corresponding to the chirping of birds. The controller can divide the cabin into six areas with the center of the cabin as a reference point.

[0159] In the scenario shown in Figure 5, the location of the sound source corresponds to the location of the bird. Figure 5 schematically illustrates positions 5a-5f of the bird. Based on the bird's location, the controller can determine the lighting position of the ambient light, thereby controlling the lighting state of the ambient light in the same area as the bird. For example, when the bird is in position 5a, the controller can control the lighting state of ambient light 110a based on the audio characteristics of the bird's chirping sound; when the bird is in position 5b, the controller can control the lighting state of ambient light 110b based on the audio characteristics of the bird's chirping sound; and when the bird is in position 5c, the controller can control the lighting state of ambient light 110c based on the audio characteristics of the bird's chirping sound.

[0160] The controller can also control the lighting state of the atmosphere lamp in a different area from the sound source based on the audio characteristics of the sound corresponding to the sound source. For example, in the scene shown in Figure 5, when the bird is at position 5a, the controller can control the lighting state of the atmosphere lamp 110d based on the audio characteristics of the bird's call; when the bird is at position 5d, the controller can control the lighting state of the atmosphere lamp 110a based on the audio characteristics of the bird's call.

[0161] When controlling the illumination of the ambient light, the controller may divide the space in a manner that differs from the representation of the sound source location indicated by the location information carried in the audio data, or from the representation of the sound source location determined by the controller using a sound source separation algorithm. The controller may determine the illumination position of the ambient light based on a predefined correspondence.

[0162] Exemplarily, referring to FIG6 , the position of the sound source may be expressed as the angle of the sound source on the horizontal plane. For example, the position information may indicate that the sound source is in a direction of -15° to 15°, or in a direction of 15° to 60°, etc., directly in front of the center point of the cabin. When determining the light-emitting position of the ambient light, the controller may divide the cabin space into seven areas: front center, front left, front right, left, right, left rear, and right rear, based on the center point of the cabin. The correspondence between the predefined light-emitting positions and the positions of the sound sources can be, for example, that the ambient lights in the front center area correspond to the sound sources in the area of ​​-15° to 15°, the ambient lights in the front left area correspond to the sound sources in the area of ​​-60° to -15°, the ambient lights in the front right area correspond to the sound sources in the area of ​​15° to 60°, the ambient lights in the left area correspond to the sound sources in the area of ​​-90° to -65°, the ambient lights in the right area correspond to the sound sources in the area of ​​65° to 90°, the ambient lights in the left rear area correspond to the sound sources in the area of ​​-180° to -90°, and the ambient lights in the right rear area correspond to the sound sources in the area of ​​90° to 180°.

[0163] This application does not specifically limit the correspondence between the light-emitting position of the atmosphere light and the position of the sound source.

[0164] For another example, the lighting position of the ambient light may be determined by the playback position of the sound corresponding to the audio data, that is, the lighting position of the ambient light and the position of the speaker that plays the sound may satisfy a predefined correspondence.

[0165] Taking the cockpit 100 as an example, the cockpit 100 is equipped with ambient lights 110a-110f and speakers 120a-120f. The controller can control the lighting state of the ambient light 110b based on the audio characteristics of the sound to be played through the speaker 120a; can control the lighting state of the ambient light 110c based on the audio characteristics of the sound to be played through the speaker 120b; and can control the lighting state of the ambient light 110e based on the audio characteristics of the sound to be played through the speaker 120e.

[0166] It should be understood that the present application does not specifically limit the correspondence between the position of the ambient light and the position of the speaker. The method of determining the light position by the sound playback position can refer to the method of determining the light position by the position of the sound source, which will not be repeated here.

[0167] The lighting state of the ambient light at each lighting position can be related to the audio characteristics of the sound. The controller can extract the audio characteristics from the audio data and determine the control parameters for controlling the lighting state of the ambient light accordingly, thereby controlling the ambient light to present the corresponding lighting state.

[0168] For example, the audio features of the sound corresponding to the audio data may include information such as the loudness, tempo, and fundamental frequency of the sound. The controller may extract the loudness, tempo, and fundamental frequency of the sound from the audio data using a predefined algorithm, and determine control parameters for controlling the lighting state of the ambient light based on the correspondence between the predefined audio features and the lighting state of the ambient light.

[0169] The method of extracting the audio features of the sound from the audio data can be carried out with reference to the existing technology and will not be described in detail here.

[0170] As an example, the brightness of the ambient light may correspond to the loudness of the sound, the color of the ambient light may correspond to the beat of the sound, and the saturation of the ambient light may correspond to the fundamental frequency of the sound.

[0171] The control parameters for controlling the ambient light may include brightness, color, and saturation. The corresponding relationship between the control parameters and the audio features may be, for example, that the brightness is proportional to the loudness, the color is proportional to the tempo, and the saturation is proportional to the fundamental frequency. The controller may determine the brightness, color, and saturation values ​​based on predefined corresponding relationships and control the lighting state of the ambient light accordingly.

[0172] The control parameters for controlling the atmosphere light may also include intensity values ​​of the three colors red, green and blue. The controller may convert the determined brightness value, color value and saturation value into intensity values ​​of the three colors red, green and blue, and control the lighting state of the atmosphere light accordingly.

[0173] The method of converting the brightness value, color value and saturation value into the intensity values ​​of the three colors red, green and blue can be carried out with reference to the existing technology and will not be described in detail here.

[0174] It should be understood that the above content is merely illustrative, and this application does not limit the specific correspondence between audio features and lighting states. For example, the brightness of the ambient light can correspond to the tempo of a sound, and the color of the ambient light can correspond to the loudness of the sound. The method for controlling the lighting state of the ambient light based on audio features can also refer to existing technologies.

[0175] The controller can also determine the control parameters for controlling the ambient light based on the type of sound, or determine the control parameters for controlling the ambient light based on the type of sound and the audio characteristics of the sound, thereby controlling the ambient light to present a corresponding lighting state.

[0176] For example, part of the control parameters of the ambient light can correspond to the type of sound, and part can correspond to the audio characteristics of the sound. For example, the color of the ambient light can correspond to the type of sound, and the brightness and saturation of the ambient light can correspond to the audio characteristics of the sound. This application does not impose any restrictions on this.

[0177] The controller can also control the lighting time of the atmosphere light based on the characteristics of the audio data. For example, when the sound type is a high-frequency sound, the atmosphere light is controlled to remain lit; when the sound type is a low-frequency sound, the atmosphere light is controlled not to light up; when the sound type is a medium-frequency sound, the atmosphere light is periodically controlled to light up and turn off, such as lighting up for 1 second and then turning off for 1 second.

[0178] It should be understood that the method of controlling the lighting time of the atmosphere light based on the characteristics of the audio data can be carried out with reference to the existing technology, and this application does not limit this.

[0179] Based on this solution, the vehicle's ambient lighting effects can be controlled based on the characteristics of the audio data being played, allowing the lighting effects to mimic the rhythm and spatial sense of the sound. When the sound corresponding to the audio data is played, the lighting effects can be linked to the sound. The ambient lighting position can also correspond to different sound locations or types, providing a better user experience.

[0180] In some embodiments, the display device is a display configured in a vehicle, and the display effect of the vehicle can be the display effect presented by the display, including but not limited to the brightness, color, saturation, etc. of the display, and can also include the image presented in the display.

[0181] The controller may determine the display position of the image in the display based on the type or position of the sound in the audio data, and may control the type of the image in the display based on the audio characteristics of the sound.

[0182] The method for controlling the display position of an image in a display based on the type or position of sound may refer to the aforementioned method for determining the light-emitting position of the ambient light in the cabin based on the type or position of sound, and will not be repeated here.

[0183] The controller may determine the type of image on the display based on one or more of the loudness, tempo, or fundamental frequency of the sound. For example, the type of image may be determined by the loudness of the sound, or by the tempo of the sound, or by a combination of loudness, tempo, and fundamental frequency. The type of image may also be determined by the type of sound.

[0184] The controller may determine an image to be displayed in the display from a predefined image set based on a predefined correspondence, and control the display to display the image at a display position determined by the type or position of the sound.

[0185] As the audio characteristics of the sound or the type of the sound changes, the type of the image on the display may also change accordingly, that is, it is manifested as the movement of the image on the display.

[0186] For example, referring to FIG7 , the audio data may correspond to the sound of a bird. The controller may determine, based on the audio characteristics or type of the sound, whether to present an image of the bird on the display. The location of the sound source corresponds to the location of the bird. If the bird's location in the sound circles the cabin from 7a through 7b, 7c, 7d, 7e, and finally 7f, the image on the display may appear in a circle, as shown in FIG7 .

[0187] It should be understood that the present application does not limit the type of image in the display, nor the specific correspondence between the image and the sound corresponding to the audio data. The image in the display may also be an image of a virtual character, and the display effect may be that the virtual character beats toward the corresponding position according to the position of the sound.

[0188] The controller can also control one or more of the display's brightness, color, or saturation based on the characteristics of the audio data. For example, the display's brightness can correspond to the loudness of the sound, such as being proportional to the loudness; the display's brightness can also correspond to the sound's fundamental frequency; and the display's brightness can also correspond to the type of sound, such as the type of sound source. For example, if the audio data corresponds to a human voice, the controller can control the display to display 100% brightness. If the audio data corresponds to an instrument, the controller can control the display to display 80% brightness.

[0189] It should be understood that the method of controlling the brightness, color or saturation of the display based on the characteristics of audio data can refer to the aforementioned method of controlling the brightness, color or saturation of the atmosphere light based on the characteristics of audio data, or can refer to the existing technology, and this application is not limited to this.

[0190] In this way, the vehicle display's display effects can be controlled based on the characteristics of the audio data being played, allowing the display to reflect the rhythm and spatial sense of the sound. When the sound corresponding to the audio data is played, the display effects can be linked to the sound playing inside the vehicle. The position of the image on the display can also correspond to the location or type of the sound, enhancing the user experience.

[0191] In some embodiments, the vehicle may be configured with ambient lighting and a display at the same time, and the controller may simultaneously control the lighting effects of the ambient lighting and the display effects of the display based on the audio data to be played.

[0192] Based on the above technical content, the controller can control the display effect of the vehicle's display device according to the characteristics of the audio data to be played. The characteristics of the audio data are determined based on the audio data to be played, rather than based on the sound collected in the car. This can make the characteristics of the determined audio data more accurate, thereby being able to more accurately control the display effect of the display device. When the sound corresponding to the audio data is played in the car, the correspondence between the display effect of the display device and the played sound can be made more accurate; different positions of the vehicle can also present different display effects to correspond to different types or positions of sound. In this way, there is a better linkage effect between the display effect presented by the vehicle and the sound played in the car, which can bring a better user experience to the user.

[0193] The above content introduces a method in which a controller controls a vehicle's display device to present a display effect based on audio data to be played. Next, the audio playback method provided in this application is introduced in conjunction with the accompanying drawings.

[0194] Referring to FIG8 , FIG8 is a schematic flow chart of an audio playback method provided in an embodiment of the present application. The method shown in FIG8 can be executed by a controller in the control system 200 or by another type of controller. The controller can be implemented using software, hardware, or a combination of software and hardware. The controller can be configured in the vehicle, for example, as a chip or chip system configured in the vehicle, or as a player configured in the vehicle, or as a processing module within the player.

[0195] As shown in FIG. 8 , the audio playing method 800 may include step 810 and step 820 .

[0196] Step 810: Acquire audio data to be played by the vehicle.

[0197] The controller can obtain the audio data to be played by the vehicle. The specific implementation method of step 810 can refer to the aforementioned step 310 and will not be repeated here.

[0198] Step 820: Play the sound corresponding to the audio data.

[0199] The controller can determine the position of the speaker for playing the sound based on the characteristics of the acquired audio data to be played (including one or more of the type and position of the sound), and control each speaker to play the sound. The information indicating the type or position of the sound can be carried in the audio data, or determined by the controller through a predefined sound source separation method. The method for the controller to determine the type of sound or the position of the sound is shown in the above-mentioned step 320 and will not be repeated here.

[0200] One possible implementation is that the controller controls speakers at different locations to play sounds based on the type of sound.

[0201] As an example, the controller can divide the space into multiple areas and control the speakers in different areas to play sounds based on the type of sound.

[0202] Taking cockpit 100 as an example, the sounds corresponding to the audio data to be played can include sounds from different types of sound sources, such as piano, violin, male voice, and female voice. The controller can divide cockpit 100 into zones 9A-9D as shown in FIG9 . The controller can control the speakers in zone 9A (including speaker 120a) to play piano sounds, the speakers in zone 9B (including speakers 120b and 120c) to play violin sounds, the speakers in zone 9C (including speakers 120d and 120e) to play male voices, and the speakers in zone 9D (including speaker 120f) to play female voices.

[0203] The controller can also divide the cockpit 100 into areas 10A-10F as shown in Figure 10, and can control the speakers located in areas 10A and 10C (including speakers 120a and 120c) to play the sound of the piano, control the speakers located in areas 10D and 10F (including speakers 120d and 120f) to play the sound of the violin, and control the speakers located in areas 10B and 10E (including speakers 120b and 120e) to play both male and female voices.

[0204] The controller can also obtain the position of each speaker in the cabin and divide the area based on this.

[0205] It should be understood that the area division shown in Figures 9 and 10 is only an example in the embodiment of the present application. The present application does not limit the number of areas into which the controller divides the space, nor does it limit the method of division. The controller can divide the space into any number of areas, and the correspondence between the type of sound and the area where the speaker is located can also be other predefined correspondences. The correspondence can also change over time, and the correspondence can also be set by the user.

[0206] The controller can also obtain information about each speaker in the cabin and control the corresponding speakers to play sounds based on predefined correspondences. Taking cabin 100 as an example, the sounds corresponding to the audio data obtained by the controller may include, for example, piano sounds, violin sounds, male voices, and female voices. The controller can obtain information about each speaker in the cabin and determine that speakers 120a-120f are configured within the cabin. The controller can then control speaker 120a to play piano sounds, speaker 120f to play violin sounds, speaker 120b to play male voices, speaker 120e to play female voices, and speakers 120c and 120e to remain silent.

[0207] It should be understood that the correspondence between the sound type and the speaker may also be other predefined correspondences, the correspondence may also change over time, and the correspondence may also be set by the user.

[0208] One possible implementation is that the controller can determine the position of the speaker for playing the sound based on the position of the sound, such as the position of the sound source or the playback position of the sound, and control the speaker at the corresponding position to play the sound.

[0209] As an example, the controller can obtain the location of speakers within the cabin and, based on the location indicated by the position information carried in the audio data, control the speakers at the corresponding locations to play sound using a predefined correspondence. For example, the audio data can be 5.1 surround sound audio data, which includes six channels of data. The controller can control speakers at different locations to play different channels of data based on the predefined correspondence.

[0210] The controller can also determine the speaker used to play the sound based on the location of the sound source and control multiple speakers to play the sound in coordination.

[0211] Referring to Figure 11 , taking cabin 100 as an example, cabin 100 is equipped with speakers 120a-120f. The audio data to be played, acquired by the controller, can correspond to the chirping of a bird. In the scenario shown in Figure 11 , the location of the sound source corresponds to the location of the bird. Figure 11 schematically illustrates bird positions 11a-11f. Based on the bird's location, the controller can control speakers at different locations to play the sound. For example, when the bird is in position 11a, the controller can control speakers 120a and 120f to play the sound simultaneously; when the bird is in position 11b, the controller can control speaker 120a to play the sound; and when the bird is in position 11c, the controller can control speaker 120b to play the sound.

[0212] Regarding the method in which the controller controls multiple speakers to play sounds collaboratively based on the location of the sound source, thereby enabling the user to feel the location of the sound source based on the sound played in the cabin, reference can be made to the existing technology and will not be repeated here.

[0213] It should be understood that the present application does not limit the specific correspondence between the position of a sound and the position of a speaker used to play the sound.

[0214] Through the solution of step 800, speakers at different locations can play the same or different sounds based on the type or location of the sound, so that the sound played by each speaker presents a sense of space, which can improve the user experience.

[0215] In the embodiment of the present application, the audio playback method 800 and the control method 300 can be performed simultaneously or separately. The audio playback method 800 and the control method 300 can be implemented by the same controller or by different controllers. For example, after obtaining audio data, a controller can simultaneously control the display effect of the display device and control the playback of sound.

[0216] At the same time, based on the same audio data, the controller can simultaneously control the speaker to play sound and control the display effect of the vehicle's display device, avoiding the process of collecting and processing sound. The vehicle's display effect and the sound played by the speaker can be synchronized, which can improve the user experience.

[0217] The method provided by the embodiment of the present application is described above in conjunction with the accompanying drawings. The device provided by the embodiment of the present application will be introduced below.

[0218] Figure 12 is a schematic block diagram of a controller provided in an embodiment of the present application. Controller 1200 can be configured on a vehicle and, through its components, can perform the functions of the controller described in method 300 or method 800. Controller 1200 can, for example, be the controller in control system 200. For example, as shown in Figure 12, controller 1200 can include an acquisition module 1210 and a control module 1220.

[0219] For example, when the controller 1200 is used to implement the functions of the controller in method 300, the acquisition module 1210 is used to acquire audio data to be played, and the control module 1220 is used to control the display effect of the vehicle's display device based on the characteristics of the audio data. The characteristics of the audio data include one or more of the location and type of the sound.

[0220] Optionally, the display device includes at least one of an ambient light and a display.

[0221] When the controller 1200 is used to implement the functions involved in the controller in the method 800, the acquisition module 1210 is used to obtain audio data to be played, and the control module 1220 is used to control the speaker to play sound according to the audio data.

[0222] It should be understood that the division of modules in the embodiments of the present application is illustrative and is merely a logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional modules in the various embodiments of the present application may be integrated into a single processor, or may exist physically as separate modules, or two or more modules may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or software functional modules.

[0223] FIG13 is another schematic block diagram of a controller provided in an embodiment of the present application. As shown in FIG13 , controller 1300 may include at least one processor 1310, configured to implement the functions of the controller in method 300 and / or the functions of the controller in method 800. Controller 1300 may be configured on a vehicle, and may be, for example, a controller in control system 200.

[0224] The controller 1300 may also include at least one memory 1320 for storing one or more of program instructions and data. The memory 1320 is coupled to the processor 1310. The coupling in the embodiments of the present application is an indirect coupling or communication connection between devices, units or modules, which can be electrical, mechanical or other forms, and is used for information exchange between devices, units or modules. The processor 1310 can operate in conjunction with the memory 1320. The processor 1310 can execute program instructions stored in the memory 1320. At least one of the at least one memory may be included in the processor.

[0225] The controller 1300 may also include an interface 1330 for communicating with other devices via a transmission medium, thereby enabling the controller 1300 to communicate with other devices, such as a memory, a display device, or a speaker. The interface 1330 may be, for example, a transceiver, an interface, a bus, a circuit, or a device capable of performing transceiver functions. The processor 1310 may utilize the interface 1330 to transmit and receive data and / or information, and to implement the method in the embodiment corresponding to FIG. 3 and / or the method in the embodiment corresponding to FIG. 8 .

[0226] For example, when the controller 1300 is used to implement the functions involved in the controller in the aforementioned method 300, the processor 1310 can be used to control the interface 1330 to obtain audio data to be played, and can also be used to control the display effect of the vehicle's display device based on the characteristics of the audio data; wherein the characteristics of the audio data include one or more of the location and type of the sound.

[0227] When the controller 1300 is used to implement the functions involved in the controller in the aforementioned method 300, the processor 1310 can be used to control the interface 1330 to obtain audio data to be played, and can also be used to control the speaker to play sound according to the audio data.

[0228] 14 is a schematic block diagram of a player provided in an embodiment of the present application. The player 1400 can be configured on a vehicle and, through its components, can execute the steps in the aforementioned method 800. For example, the player 1400 can include a control module 1410 and a playback module 1420.

[0229] Among them, the control module 1410 is used to obtain audio data to be played in the vehicle; and control the playback module to play the sound corresponding to the audio data, and the playback position of the sound is determined by the type of sound; the playback module 1420 is used to play the sound.

[0230] FIG15 is another schematic block diagram of a player provided in an embodiment of the present application. As shown in FIG15 , the player 1500 may include at least one processor 1510 and at least one speaker 1520 for implementing the steps in the aforementioned method 800 .

[0231] The player 1500 may also include at least one memory 1530 for storing program instructions and / or data. The memory 1530 is coupled to the processor 1520. Coupling in the embodiments of the present application refers to an indirect coupling or communication connection between devices, units, or modules, which may be electrical, mechanical, or other forms, and is used for information exchange between the devices, units, or modules. The processor 1510 can operate in conjunction with the memory 1530. The processor 1510 can execute program instructions stored in the memory 1530. At least one of the at least one memory may be included in the processor.

[0232] The player 1500 may also include an interface 1540 for communicating with other devices via a transmission medium, thereby enabling the player 1500 to communicate with other devices. The other devices may be, for example, a memory or a controller. The interface 1540 may be, for example, a transceiver, an interface, a bus, a circuit, or a device capable of performing transceiver functions. The processor 1510 may utilize the interface 1540 to transmit and receive data and / or information, and to implement the methods involved in the embodiment corresponding to FIG. 8 . For example, the player 1500 may obtain audio data via the interface 1540.

[0233] The present application also provides a control system, including a controller, a speaker and a display device, wherein the controller is used to execute the method executed by the controller in the aforementioned method 300, and / or is used to execute the method executed by the controller in the aforementioned method 800; the speaker is used to respond to the control of the controller and play sound; and the display device is used to present a display effect in response to the control of the controller.

[0234] The present application also provides a terminal, which may be a vehicle, and includes the above-mentioned control system.

[0235] Optionally, the terminal may further include the aforementioned player 1400 or player 1500 .

[0236] The present application also provides a computer-readable storage medium storing a computer program (also referred to as code or instruction). When the computer program is executed, the functions of the controller in the above method embodiment are implemented.

[0237] The present application also provides a computer program product, which includes: a computer program (also referred to as code, or instruction), which, when executed, enables the functions involved in the controller in the above method embodiment to be implemented.

[0238] It should be noted that the above method embodiments can be applied to or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method embodiments can be completed by hardware integrated logic circuits in the processor or by software instructions.

[0239] The processor may be a general purpose processor, a DSP, an ASIC, an FPGA or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component or any combination thereof. The general purpose processor may be a microprocessor or any conventional processor.

[0240] The steps of the method disclosed in the embodiments of this application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method.

[0241] The memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus RAM (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0242] The methods provided in the above embodiments can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, they can be implemented in whole or in part in the form of a computer program product. The computer program product may include one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic disk), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state disk (SSD)).

[0243] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0244] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0245] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0246] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of the solution of this embodiment according to actual needs.

[0247] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0248] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk, or an optical disk.

[0249] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A control method, characterized in that: include: Get the audio data to be played; The display effect of the display device of the vehicle is controlled according to the characteristics of the audio data; the characteristics of the audio data include one or more of the location and type of the sound.

2. The method according to claim 1, characterized in that The position of the sound includes the position of a sound source included in the sound.

3. The method according to claim 2, characterized in that The display effect of the display device is adapted to the position of the sound source.

4. The method according to any one of claims 1 to 3, characterized in that The location of the sound includes the playback location of the sound.

5. The method according to claim 4, characterized in that The display effect of the display device is adapted to the playback position.

6. The method according to any one of claims 1 to 5, characterized in that The type of sound includes the type of sound source.

7. The method according to claim 6, characterized in that The display effect of the display device is adapted to the type of the sound source.

8. The method according to any one of claims 1 to 7, characterized in that The display device of the vehicle includes at least one of an ambient light and a display, and the display effect includes at least one of the light-emitting position, brightness, color, saturation, and light-emitting time of the ambient light on the vehicle and the image and brightness of the display.

9. The method according to any one of claims 1 to 8, characterized in that The characteristics of the audio data also include one or more of the loudness, beat or fundamental frequency of the sound.

10. The method according to claim 9, characterized in that The brightness of the display device corresponds to the loudness of the sound.

11. The method according to claim 9 or 10, characterized in that The color of the display device corresponds to the beat of the sound.

12. The method according to any one of claims 9 to 11, characterized in that The saturation of the display device corresponds to the fundamental frequency of the sound.

13. The method according to any one of claims 1 to 12, characterized in that The display device is an atmosphere light.

14. An audio playing method, characterized in that: include: Get the audio data to be played; The sound corresponding to the audio data is played, and the playing position of the sound is determined by the type of the sound.

15. The method according to claim 14, characterized in that The audio data carries type information of the sound, where the type information indicates the type of the sound.

16. The method according to claim 14, characterized in that The type of the sound is obtained based on the separation of the audio data.

17. A controller, characterized in that: include: An acquisition module, used to acquire audio data to be played; A control module is used to control the display effect of the vehicle's display device according to the characteristics of the audio data; the characteristics of the audio data include one or more of the location and type of the sound.

18. The controller according to claim 17, characterized in that The position of the sound includes the position of a sound source included in the sound.

19. The controller according to claim 18, characterized in that The display effect of the display device is adapted to the position of the sound source.

20. The controller according to any one of claims 17 to 19, characterized in that: The location of the sound includes the playback location of the sound.

21. The controller according to claim 20, characterized in that The display effect of the display device is adapted to the playback position.

22. The controller according to any one of claims 17 to 21, characterized in that The type of sound includes the type of sound source.

23. The controller according to claim 22, characterized in that The display effect of the display device is adapted to the type of the sound source.

24. The controller according to any one of claims 17 to 23, characterized in that: The display device of the vehicle includes at least one of an ambient light and a display, and the display effect includes at least one of the light-emitting position, brightness, color, saturation, and light-emitting time of the ambient light on the vehicle and the image and brightness of the display.

25. The controller according to any one of claims 17 to 24, characterized in that The characteristics of the audio data also include one or more of the loudness, beat and fundamental frequency of the sound.

26. The controller according to claim 25, characterized in that The brightness of the display device corresponds to the loudness of the sound.

27. The controller according to claim 25 or 26, characterized in that: The color of the display device corresponds to the beat of the sound.

28. A controller according to any one of claims 25 to 27, characterized in that The saturation of the display device corresponds to the fundamental frequency of the sound.

29. A player, characterized in that: include: A control module, used for acquiring audio data to be played in the vehicle; Controlling the playing module to play the sound corresponding to the audio data, wherein the playing position of the sound is determined by the type of the sound; The playing module is used for playing the sound.

30. The player according to claim 29, characterized in that The audio data carries type information of the sound, where the type information indicates the type of the sound.

31. The player according to claim 29, characterized in that The type of the sound is obtained based on the separation of the audio data.

32. A control system, characterized in that: include: controller, speaker and display device; wherein, The controller is used to obtain audio data to be played, and is also used to control the speaker to play the sound corresponding to the audio data and control the display effect of the display device based on the characteristics of the audio data; the characteristics of the audio data include one or more of the position and type of the sound; The speaker is used to play the sound; The display device is used to present the display effect.

33. A vehicle, characterized in that: The vehicle comprises a controller as claimed in any one of claims 17-28 or a player as claimed in any one of claims 29-31 or a control system as claimed in claim 32.

34. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed, the method according to any one of claims 1 to 16 is performed.

35. A computer program product, characterized in that A computer program is included which, when executed, causes the method according to any one of claims 1 to 16 to be performed.

Images