Control method and apparatus, and vehicle

Abstract

Provided in the present application are a control method and apparatus, and a vehicle. The control method may be applied to the field of intelligent cockpits. The method comprises: acquiring a plurality of sound elements corresponding to audio data (S1410); and on the basis of the type of the audio data, controlling each sound-producing apparatus among a plurality of sound-producing apparatuses in a vehicle cockpit to play some of the plurality of sound elements (S1420). The present application can be applied to an intelligent vehicle or an electric vehicle, is conducive to improving the auditory experience of a user in a cockpit, and is also conductive to improving the degree of intellectualization of a vehicle.

Description

Control methods, devices and vehicles

[0001] This application claims priority to Chinese Patent Application No. 202410705350.1, filed with the China National Intellectual Property Administration on May 31, 2024, entitled "Control Method, Apparatus and Vehicle", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of smart cockpits, and more specifically, to a control method, device, and vehicle. Background Technology

[0003] As users' auditory needs continue to improve, more and more speakers (or audio systems) are being installed in vehicles. Currently, vehicles can play audio data to users through multiple speakers, and the interaction between users and the audio data is limited to adjusting the volume. This relatively singular playback mode results in a poor auditory experience for users within the cabin. Summary of the Invention

[0004] This application provides a control method, device, and vehicle that helps improve the user's auditory experience in the cabin and also helps improve the vehicle's intelligence level.

[0005] In a first aspect, a control method is provided, the method comprising: acquiring a plurality of sound elements corresponding to audio data; and controlling each of a plurality of sound-emitting devices in a vehicle cabin to play a portion of the plurality of sound elements according to the type of the audio data.

[0006] Based on the above technical solution, controlling the playback of some sound elements from multiple sound elements by each sound-producing device through the type of audio data helps to improve the user's auditory experience for different types of audio data, giving the user a better sense of immersion; at the same time, it also helps to improve the intelligence level of the vehicle.

[0007] In some possible implementations, the audio data can be of various types, including audio from crosstalk, stand-up comedy, concerts, radio broadcasts, music, and videos.

[0008] In some possible implementations, when the type of the audio data is crosstalk, the multiple sound elements include the sound elements of each of the one or more crosstalk performers and the sound elements of the audience.

[0009] In some possible implementations, when the type of the audio data is stand-up comedy, the multiple sound elements include the sound elements of each of the stand-up comedians and the sound elements of the audience.

[0010] In some possible implementations, when the type of the audio data is a concert, the multiple sound elements include the sound elements of each of one or more singers and the sound elements of the audience.

[0011] In some possible implementations, when the type of the audio data is broadcast sound, the multiple sound elements include the sound element of each of one or more announcers.

[0012] In some possible implementations, when the audio data is music or audio from a video, the multiple sound elements include at least two of the following: male voice, female voice, chorus, instrumental sound, and background sound. The background sound can be any sound element in the audio data other than male voice, female voice, chorus, and instrumental sound. When the audio data is music, the multiple sound elements can also be referred to as multiple music elements.

[0013] In some possible implementations, obtaining multiple sound elements corresponding to audio data includes: obtaining the parsed multiple sound elements. For example, a vehicle may have one or more pre-set audio data sets, and multiple sound elements corresponding to each audio data set.

[0014] In some possible implementations, obtaining multiple sound elements corresponding to audio data includes parsing the audio data to obtain these multiple sound elements. Parsing audio data via a vehicle reduces dependence on the original sound source; even if multiple sound elements are not parsed from the audio data, they can still be obtained through vehicle-based audio parsing, achieving a good auditory effect. For example, parsing crosstalk audio can obtain the sound elements of the first performer (the straight man), the second performer (the comedian), and the audience. Similarly, parsing concert audio can obtain the singer's sound elements and the audience's sound elements.

[0015] In some possible implementations, acquiring multiple sound elements corresponding to audio data includes: acquiring these multiple sound elements sent by the audio parsing module in the vehicle. The audio parsing module can be a module within the vehicle's infotainment system, or it can be a module used to parse audio, independent of the vehicle's infotainment system.

[0016] In some possible implementations, multiple sound elements corresponding to audio data can be acquired, including acquiring these multiple sound elements sent by a cloud server. By acquiring these multiple sound elements from a cloud server, the demand for vehicle computing power can be reduced, thus decreasing the overhead of computing resources.

[0017] The multiple sound-generating devices in the vehicle cabin mentioned above can be all the sound-generating devices in the cabin, or they can be some of the sound-generating devices among all the sound-generating devices in the cabin.

[0018] In conjunction with the first aspect, in some implementations of the first aspect, controlling each of the multiple sound-emitting devices in the vehicle cabin to play a portion of the multiple sound elements according to the type of the audio data includes: determining the playback area of ​​each of the multiple sound elements in the cabin according to the type of the audio data; and controlling each sound-emitting device to play a portion of the multiple sound elements according to the playback area of ​​each of the multiple sound elements in the cabin.

[0019] Based on the above technical solution, for different types of audio data, after parsing and obtaining multiple sound elements, the playback area of ​​each sound element within the cabin is first determined, and then the sound-emitting device in the corresponding playback area is controlled to play the corresponding sound element. This helps improve the user's auditory experience with different types of audio data, giving the user a better sense of immersion; at the same time, it also helps improve the vehicle's level of intelligence.

[0020] In some possible implementations, determining the playback area of ​​each of the multiple sound elements within the cockpit based on the type of the audio data includes: determining the playback area of ​​each sound element within the cockpit based on the type of the audio data and a preset association relationship, wherein the preset association relationship includes the association relationship between the type of audio data and the playback area of ​​the sound element within the cockpit.

[0021] In some possible implementations, determining the playback area of ​​each of the plurality of sound elements within the cabin based on the type of the audio data includes: when the type of the audio data is a first performance type and the audio data includes the sound elements of a first performer and a second performer, determining that the sound-emitting device on the first side of the cabin plays the sound element of the first performer and determining that the sound-emitting device on the second side of the cabin plays the sound element of the second performer; or, when the type of the audio data is a second performance type and the audio data includes the sound elements of a third performer and the sound elements of the audience, determining that the sound-emitting device on the third side of the cabin plays the sound element of the third performer and determining that the sound-emitting device on the fourth side of the cabin plays the sound element of the audience, wherein the first side and the second side are opposite sides, and the third side and the fourth side are opposite sides.

[0022] For example, a first performance type may include two main performers and an audience, with one performer on the left and the other on the right, and the audience members beside or behind them. For instance, the first performance type could be a crosstalk performance, with the two performers' voices positioned to the left and right front, respectively; or it could be an interview program, where the interviewee and interviewer are positioned to the left and right front, respectively, or vice versa.

[0023] For example, the second performance type may include a performer and an audience, wherein the performer is located at the front, and the audience is located beside or behind them. For instance, the second performance type could be a stand-up comedy routine, where the comedian is at the front and the audience is at the back or beside the comedian, or vice versa. As another example, the second performance type could be a concert, where the singer is at the front and the audience is at the back or beside the singer, or vice versa.

[0024] In conjunction with the first aspect, in certain implementations of the first aspect, depending on the type of the audio data, controlling each of the multiple sound-emitting devices in the vehicle cabin to play a portion of the multiple sound elements includes: when the type of the audio data is a first performance type, controlling a first sound-emitting device located in the left-hand area of ​​the cabin to play the sound element of a first performer and controlling a second sound-emitting device located in the right-hand area of ​​the cabin to play the sound element of a second performer, wherein the multiple sound-emitting devices include the first sound-emitting device and the second sound-emitting device; or, when the type of the audio data is a second performance type, controlling a third sound-emitting device located in the front-hand area of ​​the cabin to play the sound element of a performer and controlling a fourth sound-emitting device located in the rear-hand area of ​​the cabin to play the sound element of an audience member, wherein the multiple sound-emitting devices include the third sound-emitting device and the fourth sound-emitting device.

[0025] Based on the above technical solution, for different types of audio data, the sound-emitting devices in different areas of the cockpit can be controlled to play sound elements. For example, when the audio data is of the first performance type (e.g., crosstalk), the sound-emitting devices in the left and right areas of the cockpit can play the voices of the first performer (e.g., the straight man) and the second performer (e.g., the lead man). As another example, when the audio data is of the second performance type (e.g., stand-up comedy or a concert), the sound-emitting devices in the front and rear areas of the cockpit can play the voices of the performers and the audience, respectively. This gives the user a feeling of being immersed in the performance, enhancing the user's sense of immersion and also contributing to the vehicle's level of intelligence.

[0026] In some possible implementations, depending on the type of the audio data, each of the multiple sound-emitting devices in the vehicle cabin is controlled to play a portion of the multiple sound elements, including: when the type of the audio data is a first performance type, controlling the first sound-emitting device located in the left-hand area of ​​the cabin to play the sound element of the first performer, controlling the sound-emitting device in the middle area of ​​the cabin to play the sound of the third performer, and controlling the second sound-emitting device in the right-hand area of ​​the cabin to play the sound element of the second performer.

[0027] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: when the type of the audio data is the first performance type and the audio data includes the audience's voice elements, controlling the sound-emitting device in the rear area of ​​the cockpit to play the audience's voice elements.

[0028] Based on the above technical solution, when the audio data type is the first performance type (such as crosstalk), the audience's voice can also be played through a sound-emitting device in the rear area of ​​the cabin. This can give users a feeling of being immersed in the performance, helping to enhance the user's sense of immersion.

[0029] In some possible implementations, the method further includes: when the audio data is of broadcast type and includes the voices of a first announcer and a second announcer, controlling a sound-emitting device on a first side of the cabin to play the voice of the first announcer and controlling a sound-emitting device on a second side of the cabin to play the voice of the second announcer, wherein the first side and the second side are opposite sides of the cabin. For example, the first side is the left side of the cabin, and the second side is the right side of the cabin.

[0030] In conjunction with the first aspect, in some implementations of the first aspect, controlling each of the multiple sound-emitting devices in the vehicle cabin to play a portion of the multiple sound elements according to the type of the audio data includes: controlling each sound-emitting device to play a portion of the multiple sound elements according to the type of the audio data and the area where the user is located in the cabin.

[0031] For example, when the audio data type is crosstalk and the area where the user is located in the cockpit is the driver's seat area, the sound device located on the left side of the driver's seat area can be controlled to play the voice of the lead comedian, and the sound device located on the right side of the driver's seat area can be controlled to play the voice of the supporting comedian.

[0032] For example, when the type of audio data is a concert and the area where the user is located in the cockpit is the driver's seat area, the sound device located in front of the driver's seat area can be controlled to play the singer's voice and the sound device located behind the driver's seat area can be controlled to play the audience's voice.

[0033] Based on the above technical solutions, users can be provided with an immersive auditory experience in the area where they are located, depending on the type of audio data. This helps to improve the user's driving experience and also enhances the vehicle's intelligence.

[0034] In some possible implementations, depending on the type of audio data and the area where the user is located in the cabin, each sound-emitting device is controlled to play a portion of the multiple sound elements, including: when it is detected that a user in the boss's seat (or, princess's seat) in the cabin is playing a crosstalk, and when there are users in the driver's seat area and the boss's seat area, the sound-emitting device located to the left of the boss's seat area is controlled to play the voice of the supporting actor, and the sound-emitting device located to the right of the boss's seat area is controlled to play the voice of the lead actor.

[0035] In conjunction with the first aspect, in some implementations of the first aspect, according to the type of the audio data, each of the multiple sound-emitting devices in the vehicle cabin is controlled to play a portion of the multiple sound elements, including: when the type of the audio data is audio data in music or video, each sound-emitting device is controlled to play a portion of the multiple sound elements according to the area where the user is located in the cabin.

[0036] In some possible implementations, each sound-emitting device is controlled to play a portion of the multiple sound elements based on the area where the user is located in the cockpit. This includes controlling each of the multiple sound-emitting devices in the front area of ​​the cockpit to play a portion of the multiple sound elements when a user is detected in the driver's seat area and the passenger seat area of ​​the cockpit.

[0037] In some possible implementations, each sound-emitting device is controlled to play a portion of the multiple sound elements based on the area where the user is located in the cabin. This includes: when it is detected that only the driver's area in the cabin has a user, controlling each of the multiple sound-emitting devices in the driver's area of ​​the cabin to play a portion of the multiple sound elements.

[0038] Based on the above technical solution, the sound-generating devices and sound elements can be intelligently matched according to the distribution of users in the cabin, so that all users in the cabin can have a better immersive auditory experience.

[0039] In conjunction with the first aspect, in some implementations of the first aspect, controlling each sound-emitting device to play a portion of the multiple sound elements based on the area where the user is located in the cockpit includes: controlling each sound-emitting device to play a portion of the multiple sound elements based on the area where the user is located in the cockpit and the information of each sound element in the multiple sound elements; wherein the information of each sound element includes the volume information of each sound element and / or the playback duration of each sound element in the audio data.

[0040] Based on the above technical solution, intelligent matching of sound-generating devices and sound elements by the distribution of users in the cabin and the information of each sound element helps to further enhance the user's immersive auditory experience.

[0041] In some possible implementations, the plurality of sound elements includes a first sound element and a second sound element. Based on the area where the user is located in the cabin and information about each of the plurality of sound elements, each sound-emitting device is controlled to play a portion of the plurality of sound elements. This includes: determining a first set of sound-emitting devices and a second set of sound-emitting devices based on the area where the user is located in the cabin, with the sound-emitting devices in the first set surrounding the area where the user is located in the cabin; when the volume of the first sound element is greater than the volume of the second sound element, controlling the sound-emitting devices in the first set to play the first sound element and controlling the sound-emitting devices in the second set to play the second sound element; or, when the playback duration of the first sound element in the audio data is greater than the playback duration of the second sound element in the audio data, controlling the sound-emitting devices in the first set to play the first sound element and controlling the sound-emitting devices in the second set to play the second sound element.

[0042] For example, based on the area where the user is located in the cabin and the information of each of the multiple sound elements, each sound-emitting device is controlled to play a portion of the multiple sound elements, including: when a user is detected in the driver's seat area and the passenger seat area in the cabin and the volume of the first sound element is greater than the volume of the second sound element, each of the multiple sound-emitting devices in the front area of ​​the cabin is controlled to play the first sound element, and each of the multiple sound-emitting devices in the rear area of ​​the cabin is controlled to play the second sound element.

[0043] The multiple sound-emitting devices in the forward area of ​​the cockpit can be all the sound-emitting devices in the forward area, or they can be some of the sound-emitting devices in the forward area. The multiple sound-emitting devices in the rear area of ​​the cockpit can be all the sound-emitting devices in the rear area, or they can be some of the sound-emitting devices in the rear area.

[0044] In conjunction with the first aspect, in some implementations of the first aspect, the plurality of sound-emitting devices includes a fifth sound-emitting device and a sixth sound-emitting device, and the plurality of sound elements includes a first sound element and a second sound element. Controlling each of the plurality of sound-emitting devices in the vehicle cabin to play a portion of the plurality of sound elements includes: controlling the fifth sound-emitting device to play the first sound element and controlling the sixth sound-emitting device to play the second sound element; wherein, the method further includes: when a user's first gesture and / or first limb movement is detected, controlling the fifth sound-emitting device to play the second sound element and controlling the sixth sound-emitting device to play the first sound element.

[0045] Based on the above technical solution, by detecting the user's preset gestures or body movements, sound elements can be switched to different sound-producing devices for playback. This allows the user to become a "music conductor" within the cabin, enhancing the user's driving experience.

[0046] For example, if a user in the front area gives a first gesture and / or a first body movement, the sound elements in the driver's area and the passenger's area can be swapped, or the sound elements played by a certain sound-producing device in the driver's area and a certain sound-producing device in the passenger's area can be swapped.

[0047] In conjunction with the first aspect, in some implementations of the first aspect, when a user's first hand gesture and / or first limb movement is detected, controlling the fifth sound-emitting device to play the second sound element and controlling the sixth sound-emitting device to play the first sound element includes: when the user's first hand gesture and / or first limb movement toward the fifth sound-emitting device and the sixth sound-emitting device is detected, controlling the fifth sound-emitting device to play the second sound element and controlling the sixth sound-emitting device to play the first sound element.

[0048] Based on the above technical solution, after detecting the user's action of clearly pointing at the fifth and sixth sound-emitting devices, the sound elements played by the fifth and sixth sound-emitting devices can be swapped.

[0049] For example, the first gesture could be the user pointing the fingers of both hands to the fifth and sixth sound-emitting devices respectively.

[0050] In conjunction with the first aspect, in some implementations of the first aspect, the plurality of sound-emitting devices includes a seventh sound-emitting device, the plurality of sound elements includes a third sound element, and controlling each of the plurality of sound-emitting devices in the vehicle cabin to play a portion of the plurality of sound elements includes: controlling the seventh sound-emitting device to play the third sound element; wherein, the method further includes: when a second hand gesture and / or second limb movement of a user is detected toward the seventh sound-emitting device, controlling the seventh sound-emitting device to stop playing the third sound element.

[0051] Based on the above technical solution, when a user makes a second hand gesture and / or a second limb movement towards a sound-emitting device, the sound-emitting device can be stopped from playing sound elements. This allows users to conveniently control the sound-emitting device to stop playing the corresponding sound elements.

[0052] In some possible implementations, the second gesture and / or second limb movement could be the user extending their arm and turning their palm toward the seventh sound-emitting device.

[0053] In some possible implementations, the second hand gesture and / or second limb movement could be the user extending their arm and repeatedly patting the seventh sound-emitting device.

[0054] In some possible implementations, the method further includes controlling the seventh sound device to play the third sound element when the user's second gesture and / or second limb movement is detected again.

[0055] In conjunction with the first aspect, in some implementations of the first aspect, the plurality of sound elements does not include a fourth sound element, and the method further includes: controlling an eighth sound-emitting device in the cockpit to play the fourth sound element when a third hand gesture and / or third limb movement of the user is detected.

[0056] Based on the above technical solution, when the user makes a third gesture and / or a third limb movement again, a certain sound-emitting device can be controlled to play a new sound element.

[0057] In some possible implementations, the plurality of sound elements does not include a fourth sound element, and the method further includes: controlling an eighth sound-emitting device in the cockpit to play the fourth sound element when a user's preset facial expression is detected or when a user is detected laughing.

[0058] For example, the preset emoticon is a happy emoticon.

[0059] For example, the fourth sound element is a DJ turn, a scratching sound, or a rubbing sound.

[0060] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: adjusting the volume of at least some of the multiple sound-emitting devices when a user's fourth gesture and / or fourth limb movement is detected.

[0061] Based on the above technical solution, when a user makes a fourth gesture and / or a fourth limb movement, the volume of a single sound-emitting device can be increased, or the volume of several sound-emitting devices can be adjusted. This allows for control of the volume of one or more sound-emitting devices.

[0062] In conjunction with the first aspect, in some implementations of the first aspect, the plurality of sound-emitting devices includes a ninth sound-emitting device and a tenth sound-emitting device, and the plurality of sound elements includes a ninth sound element and a tenth sound element. Controlling each of the plurality of sound-emitting devices in the vehicle cabin to play a portion of the plurality of sound elements includes: controlling the ninth sound-emitting device to play the fifth sound element and controlling the tenth sound-emitting device to play the sixth sound element; wherein the method further includes: controlling a display device to display the association relationship between the plurality of sound-emitting devices and the plurality of sound elements; when a first touch command from a user is detected regarding the fifth and sixth sound elements, controlling the ninth sound-emitting device to play the sixth sound element and controlling the tenth sound-emitting device to play the fifth sound element; or, when a second touch command from a user is detected regarding the fifth sound element, controlling the ninth sound-emitting device to stop playing the fifth sound element.

[0063] Based on the above technical solution, the sound elements can be displayed in sections via an interface. Users can control the sound elements to switch sections or stop playing certain sound elements by using touch commands on the display device.

[0064] In some possible implementations, the first touch instruction is a touch action of dragging the fifth sound element to the sixth sound element; or, the first touch instruction can be a touch action of the user clicking the fifth sound element and the sixth sound element with two fingers respectively.

[0065] In conjunction with the first aspect, in some implementations of the first aspect, the plurality of sound-emitting devices includes an eleventh sound-emitting device, the sound element played by the eleventh sound-emitting device being the seventh sound element among the plurality of sound elements, the eleventh sound-emitting device being located in a first area in the vehicle, and the method further includes: controlling the ambient light in the first area to operate based on information from the seventh sound element.

[0066] Based on the above technical solution, the ambient lights in the area can be controlled simultaneously with the sound-emitting device playing the sound element, based on the information of the area where the sound-emitting device is located and the sound element played by the sound-emitting device. This achieves linkage between the sound-emitting device and the ambient lights, further enhancing the user's driving experience.

[0067] Secondly, a control method is provided, comprising: acquiring a plurality of sound elements corresponding to audio data; controlling each of a plurality of sound-emitting devices in a vehicle cabin to play a portion of the plurality of sound elements; and adjusting the sound elements played by at least a portion of the sound-emitting devices according to at least one of a user's gestures, body movements, and facial expressions.

[0068] In conjunction with the second aspect, in some implementations of the second aspect, the plurality of sound elements includes a first sound element and a second sound element. Controlling each of the plurality of sound-emitting devices in the vehicle cabin to play a portion of the plurality of sound elements includes: controlling the first sound-emitting device to play the first sound element and controlling the second sound-emitting device to play the second sound element; wherein, adjusting the sound elements played by at least a portion of the plurality of sound-emitting devices according to at least one of the user's gestures, body movements, and facial expressions includes: when a user's first gesture and / or first body movement is detected, controlling the first sound-emitting device to play the second sound element and controlling the second sound-emitting device to play the first sound element.

[0069] In conjunction with the second aspect, in some implementations of the second aspect, when a user's first gesture and / or first limb movement is detected, controlling the first sound-emitting device to play the second sound element and controlling the second sound-emitting device to play the first sound element includes: when the user's first gesture and / or first limb movement toward the first sound-emitting device and the second sound-emitting device is detected, controlling the first sound-emitting device to play the second sound element and controlling the second sound-emitting device to play the first sound element.

[0070] In conjunction with the second aspect, in some implementations of the second aspect, the plurality of sound elements includes a third sound element. Controlling each of the plurality of sound-emitting devices in the vehicle cabin to play a portion of the plurality of sound elements includes: controlling the third sound-emitting device to play the third sound element; wherein, adjusting the sound element played by at least a portion of the plurality of sound-emitting devices based on at least one of the user's gestures, body movements, and facial expressions includes: controlling the third sound-emitting device to stop playing the third sound element when a second gesture and / or second body movement of the user toward the third sound-emitting device is detected.

[0071] In conjunction with the second aspect, in some implementations of the second aspect, the plurality of sound elements does not include a fourth sound element. Based on at least one of the user's gestures, body movements, and facial expressions, the sound elements played by at least some of the multiple sound-emitting devices are adjusted, including: when a user's preset facial expression is detected, when a user's laughter is detected, when a third gesture or third body movement is detected, the fourth sound-emitting device in the cockpit is controlled to play the fourth sound element.

[0072] In conjunction with the second aspect, in some implementations of the second aspect, adjusting the sound elements played by at least some of the multiple sound-emitting devices based on at least one of the user's gestures, body movements, and facial expressions includes: adjusting the volume of at least some of the multiple sound-emitting devices when a fourth gesture and / or fourth body movement of the user is detected.

[0073] Thirdly, a control device is provided, comprising: an acquisition unit for acquiring a plurality of sound elements corresponding to audio data; and a control unit for controlling each of a plurality of sound-emitting devices in a vehicle cabin to play a portion of the plurality of sound elements according to the type of the audio data.

[0074] In conjunction with the third aspect, in some implementations of the third aspect, the control unit is specifically used to: determine the playback area of ​​each sound element in the cockpit according to the type of the audio data; and control each sound-emitting device to play a portion of the multiple sound elements according to the playback area of ​​each sound element in the cockpit.

[0075] In conjunction with the third aspect, in some implementations of the third aspect, the control unit is specifically configured to: when the type of the audio data is a first performance type, control a first sound-emitting device located in the left-hand area of ​​the cabin to play the sound elements of the first performer and control a second sound-emitting device located in the right-hand area of ​​the cabin to play the sound elements of the second performer, the plurality of sound-emitting devices including the first sound-emitting device and the second sound-emitting device; or, when the type of the audio data is a second performance type, control a third sound-emitting device located in the front-hand area of ​​the cabin to play the sound elements of the performer and control a fourth sound-emitting device located in the rear-hand area of ​​the cabin to play the sound elements of the audience, the plurality of sound-emitting devices including the third sound-emitting device and the fourth sound-emitting device.

[0076] In conjunction with the third aspect, in some implementations of the third aspect, the control unit is further configured to: control a sound-emitting device in the rear area of ​​the cockpit to play the audience's sound elements when the type of the audio data is the first performance type and the audio data includes the audience's voice elements.

[0077] In conjunction with the third aspect, in some implementations of the third aspect, the control unit is specifically used to: when the type of the audio data is audio data in music or video, control each sound-emitting device to play a portion of the multiple sound elements according to the area where the user is located in the cockpit.

[0078] In conjunction with the third aspect, in some implementations of the third aspect, the control unit is specifically used to: control each sound-emitting device to play a portion of the multiple sound elements based on the area where the user is located in the cockpit and the information of each sound element among the multiple sound elements; wherein the information of each sound element includes the volume information of each sound element and / or the playback duration of each sound element in the audio data.

[0079] In conjunction with the third aspect, in some implementations of the third aspect, the plurality of sound-emitting devices include a fifth sound-emitting device and a sixth sound-emitting device, the plurality of sound elements include a first sound element and a second sound element, and the control unit is specifically configured to: control the fifth sound-emitting device to play the first sound element and control the sixth sound-emitting device to play the second sound element; wherein, the control device is further configured to, upon detecting a user's first gesture and / or first limb movement, control the fifth sound-emitting device to play the second sound element and control the sixth sound-emitting device to play the first sound element.

[0080] In conjunction with the third aspect, in some implementations of the third aspect, the control unit is specifically configured to: when detecting the user's first gesture and / or first limb movement toward the fifth and sixth sound-emitting devices, control the fifth sound-emitting device to play the second sound element and control the sixth sound-emitting device to play the first sound element.

[0081] In conjunction with the third aspect, in some implementations of the third aspect, the plurality of sound-emitting devices include a seventh sound-emitting device, the plurality of sound elements include a third sound element, and the control unit is specifically configured to: control the seventh sound-emitting device to play the third sound element; wherein, the control device is further configured to control the seventh sound-emitting device to stop playing the third sound element when a second hand gesture and / or second limb movement of a user is detected toward the seventh sound-emitting device.

[0082] In conjunction with the third aspect, in some implementations of the third aspect, the plurality of sound elements does not include a fourth sound element, and the control device is also used to control an eighth sound-emitting device in the cockpit to play the fourth sound element when a third hand gesture and / or third limb movement of the user is detected.

[0083] In conjunction with the third aspect, in some implementations of the third aspect, the control device is also used to control the volume increase of at least some of the multiple sound-emitting devices when a user's fourth hand gesture and / or fourth limb movement is detected.

[0084] In conjunction with the third aspect, in some implementations of the third aspect, the plurality of sound-emitting devices include a ninth sound-emitting device and a tenth sound-emitting device, and the plurality of sound elements include a fifth sound element and a sixth sound element. The control unit is specifically configured to: control the ninth sound-emitting device to play the fifth sound element and control the tenth sound-emitting device to play the sixth sound element; wherein, the control device is further configured to: control the display device to display the association relationship between the plurality of sound-emitting devices and the plurality of sound elements; when a first touch command from the user for the fifth sound element and the sixth sound element is detected, control the ninth sound-emitting device to play the sixth sound element and control the tenth sound-emitting device to play the fifth sound element; or, when a second touch command from the user for the fifth sound element is detected, control the ninth sound-emitting device to stop playing the fifth sound element.

[0085] In conjunction with the third aspect, in some implementations of the third aspect, the plurality of sound-emitting devices includes an eleventh sound-emitting device, the sound element played by the eleventh sound-emitting device being the seventh sound element among the plurality of sound elements, the eleventh sound-emitting device being located in a first area in the vehicle, and the control device being further configured to: control the ambient light in the first area to operate based on information from the seventh sound element.

[0086] Fourthly, a control device is provided, comprising: an acquisition unit for acquiring a plurality of sound elements corresponding to audio data; a control unit for controlling each of a plurality of sound-emitting devices in a vehicle cabin to play a portion of the plurality of sound elements; and an adjustment unit for adjusting the sound elements played by at least a portion of the plurality of sound-emitting devices according to at least one of a user's gestures, body movements, and facial expressions.

[0087] In conjunction with the fourth aspect, in some implementations of the fourth aspect, the plurality of sound elements includes a first sound element and a second sound element. The control unit is specifically configured to: control the first sound-emitting device to play the first sound element and control the second sound-emitting device to play the second sound element; wherein, the adjustment unit is specifically configured to: control the first sound-emitting device to play the second sound element and control the second sound-emitting device to play the first sound element when a user's first gesture and / or first limb movement is detected.

[0088] In conjunction with the fourth aspect, in some implementations of the fourth aspect, the control unit is specifically configured to: when detecting the user's first gesture and / or first limb movement toward the first sound-emitting device and the second sound-emitting device, control the first sound-emitting device to play the second sound element and control the second sound-emitting device to play the first sound element.

[0089] In conjunction with the fourth aspect, in some implementations of the fourth aspect, the plurality of sound elements includes a third sound element, and the control unit is specifically configured to: control the third sound-emitting device to play the third sound element; wherein the adjustment unit is specifically configured to: control the third sound-emitting device to stop playing the third sound element when a second hand gesture and / or second limb movement of the user is detected toward the third sound-emitting device.

[0090] In conjunction with the fourth aspect, in some implementations of the fourth aspect, the plurality of sound elements does not include the fourth sound element, wherein the adjustment unit is specifically used to: control the fourth sound-emitting device in the cockpit to play the fourth sound element when at least one of the following is detected: a user's preset facial expression, a user's laughter, a third gesture, or a third limb movement is detected.

[0091] In conjunction with the fourth aspect, in some implementations of the fourth aspect, the adjustment unit is specifically used to: adjust the volume of at least some of the multiple sound-emitting devices when a user's fourth gesture and / or fourth limb movement is detected.

[0092] Fifthly, a control device is provided, comprising a memory and a processor, the memory for storing a computer program and the processor for executing the computer program in the memory, such that the control device can implement the method in any possible implementation of the first or second aspect described above.

[0093] In a sixth aspect, a control system is provided, the braking system including a plurality of sound-generating devices and a control device as described in any one of the third to fifth aspects above.

[0094] In a seventh aspect, a vehicle is provided that includes a control device possible in any of the third to fifth aspects described above, or includes a control system as described in the sixth aspect.

[0095] The term "vehicle" in this application is used in a broad sense and can refer to means of transportation (such as commercial vehicles, passenger cars, motorcycles, flying cars, trains, etc.), industrial vehicles (such as forklifts, trailers, tractors, etc.), engineering vehicles (such as excavators, bulldozers, cranes, etc.), agricultural equipment (such as lawnmowers, harvesters, etc.), amusement equipment, toy vehicles, etc. The embodiments of this application do not specifically limit the type of vehicle.

[0096] Eighthly, a computer program product is provided, comprising: computer program code, which, when executed on a computer, causes the computer to perform the method in any possible implementation of the first or second aspect.

[0097] Ninth aspect, a computer-readable storage medium is provided, the computer-readable medium storing a computer program that, when the computer program is run on a computer, causes the computer to perform the method in any possible implementation of the first or second aspect.

[0098] In a tenth aspect, a chip is provided, the chip including circuitry for performing the method in any possible implementation of the first or second aspect described above. Attached Figure Description

[0099] Figure 1 is a functional block diagram of the vehicle provided in an embodiment of this application.

[0100] Figure 2 is a schematic diagram of a sound element played through a sound-emitting device in the cockpit, according to an embodiment of this application.

[0101] Figure 3 is a schematic diagram of another method of playing sound elements through a sound-emitting device in the cockpit, provided in an embodiment of this application.

[0102] Figure 4 is a schematic diagram of another method of playing sound elements through a sound-emitting device in the cockpit, provided in an embodiment of this application.

[0103] Figure 5 is a schematic diagram of another method of playing sound elements through a sound-emitting device in the cockpit, provided in an embodiment of this application.

[0104] Figure 6 is a schematic diagram of another method of playing sound elements through a sound-emitting device in the cockpit, provided in an embodiment of this application.

[0105] Figure 7 is a schematic diagram of another method of playing sound elements through a sound-emitting device in the cockpit, provided in an embodiment of this application.

[0106] Figure 8 is a schematic diagram of another method of playing sound elements through a sound-emitting device in the cockpit, provided in an embodiment of this application.

[0107] Figure 9 is a schematic diagram of another method of playing sound elements through a sound-emitting device in the cockpit, provided by an embodiment of this application.

[0108] Figure 10 is a schematic diagram of another method of playing sound elements through a sound-emitting device in the cockpit, provided by an embodiment of this application.

[0109] Figure 11 is a schematic diagram of another method of playing sound elements through a sound-emitting device in the cockpit, provided in an embodiment of this application.

[0110] Figure 12 is a schematic diagram of another method of playing sound elements through a sound-emitting device in the cockpit, provided in an embodiment of this application.

[0111] Figure 13 is a schematic diagram of another method of playing sound elements through a sound-emitting device in the cockpit, provided in an embodiment of this application.

[0112] Figure 14 is a schematic flowchart of the control method provided in an embodiment of this application.

[0113] Figure 15 is a schematic block diagram of the control device provided in an embodiment of this application. Detailed Implementation

[0114] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; the term "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone.

[0115] The prefixes such as "first" and "second" used in this application embodiment are merely for distinguishing different descriptive objects and do not limit the position, order, priority, quantity, or content of the described objects. The use of ordinal numbers and other prefixes used to distinguish descriptive objects in this application embodiment does not constitute a limitation on the described objects. The description of the described objects is given in the claims or the context of the embodiments, and should not constitute unnecessary restrictions due to the use of such prefixes. Furthermore, in the description of this embodiment, unless otherwise stated, "multiple" means two or more.

[0116] Figure 1 is a functional block diagram of a vehicle 100 provided in an embodiment of this application. The vehicle 100 may include a sensing system 110, a computing platform 120, a display device 130, and a sound-emitting device 140. The sensing system 110 may include one or more sensors for sensing information about the environment surrounding the vehicle 100. For example, the sensing system 110 may include a positioning system, which may be a Global Positioning System (GPS), a BeiDou Navigation Satellite System, or another positioning system. The sensing system 110 may also include one or more of the following: an inertial measurement unit (IMU), a lidar, a millimeter-wave radar, an ultrasonic radar, and a camera device.

[0117] Some or all of the functions of vehicle 100 can be controlled by computing platform 120. Computing platform 120 may include one or more processors, such as processors 121 to 12n (n being a positive integer). A processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction read and execute capabilities, such as a central processing unit (CPU), microprocessor, graphics processing unit (GPU) (which can be understood as a type of microprocessor), or digital signal processor (DSP). In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. These logical relationships are fixed or reconfigurable. For example, the processor may be a hardware circuit implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as a field-programmable gate array (FPGA). In reconfigurable hardware circuits, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the process of the processor loading instructions to implement some or all of the functions of the aforementioned units. Furthermore, the processor can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), tensor processing unit (TPU), deep learning processing unit (DPU), etc. In addition, the computing platform 120 may also include a memory for storing instructions. Some or all of the processors 121 to 12n can call and execute the instructions in the memory to achieve the corresponding functions.

[0118] The in-cabin display devices 130 are mainly divided into two categories: the first is the in-vehicle display screen; the second is the projection display screen, such as the head-up display (HUD). An in-vehicle display screen is a physical display screen and an important component of the in-vehicle infotainment system. Multiple displays can be installed in the cabin, such as the digital instrument cluster display, the central control screen, the display screen in front of the front passenger (also known as the passenger in the forward area), the display screen in front of the left rear passenger, the display screen in front of the right rear passenger, and even the car window can be used as a display screen. A head-up display, also known as a head-up display system, is mainly used to display driving information such as speed and navigation on a display device in front of the driver (such as the windshield). This reduces the driver's eye-shifting time, avoids pupil changes caused by eye-shifting, and improves driving safety and comfort. Examples of HUDs include combiner-HUD (C-HUD) systems, windshield-HUD (W-HUD) systems, and augmented reality HUD (AR-HUD) systems. It should be understood that HUDs can also evolve into other types of systems as technology progresses, and this application does not limit them.

[0119] The sound-generating device 140 can be a loudspeaker, audio system, or horn, etc.

[0120] As users' auditory needs continue to improve, more and more speakers (or audio systems) are being installed in vehicles. Currently, vehicles can play audio data to users through multiple speakers, but the interaction between users and the audio data is limited to volume adjustment. This relatively singular playback mode results in a poor auditory experience for users within the cabin.

[0121] In this embodiment, the vehicle can play sound elements from audio data through different sound-producing devices. For example, intelligent matching of multiple sound-producing devices and multiple sound elements helps improve the user's auditory experience with different types of audio data, providing a better sense of immersion. For example, through the user's gestures, body movements, and facial expressions, the sound elements can be controlled, allowing the user to become a "music conductor" within the cabin, thus enhancing the user's driving experience.

[0122] For example, Figure 2 shows a schematic diagram of playing sound elements through a sound-generating device in the cockpit, according to an embodiment of this application.

[0123] As shown in Figure 2(a), the vehicle's cabin includes sound-emitting devices 1-6. When the vehicle detects that a user has launched a music application to play "Song A", it can control the sound-emitting devices 1-6 to play "Song A".

[0124] As shown in Figure 2(b), when the vehicle detects the user issuing the voice command "Xiao A Xiao A, audio track separation", it can analyze "Song A". For example, analyzing the audio data corresponding to "Song A" at the current moment can yield male voice, female voice, chorus, piano sound, guitar sound, and background sound. At this time, the vehicle can control sound device 1 to play male voice, control sound device 2 to play female voice, control sound device 3 to play piano sound, control sound device 4 to play guitar sound, control sound device 5 to play background sound, and control sound device 6 to play chorus.

[0125] All other sound elements in the above audio data, excluding male voices, female voices, choral voices, piano sounds, and guitar sounds, can be collectively referred to as background sounds.

[0126] The above-mentioned sound-generating devices 1-6 can be all the sound-generating devices in the cockpit, or sound-generating devices 1-6 can be a part of all the sound-generating devices in the cockpit.

[0127] In one embodiment, after parsing "Song A" to obtain multiple sound elements, each of the multiple sound elements can be randomly associated with a sound-producing device, thereby controlling the sound-producing device to play the corresponding sound element.

[0128] In one embodiment, the vehicle may maintain associations between the types of sound-generating devices and sound elements. For example, Table 1 illustrates the associations between speakers and sound element types within the vehicle cabin.

[0129] Table 1

[0130] The relationships described in Table 1 above can be pre-set in the vehicle at the factory or can be user-defined. This application embodiment does not specifically limit this.

[0131] In one embodiment, the example in Figure 2(b) above illustrates the parsing of audio data triggered by a voice command, but this embodiment is not limited to this. For example, the parsing of audio data and the control of the sound-producing device to play corresponding sound elements can also be triggered by preset gestures, clicking virtual buttons on the display screen, etc.

[0132] For example, Figure 3 shows a schematic diagram of another method of playing sound elements through a sound-generating device in the cockpit, as provided in an embodiment of this application.

[0133] As shown in Figure 3(a), the vehicle detects that the user extends his arm and controls his palm to swing back and forth.

[0134] The user's outstretched arm movement can be considered a physical action, while the movement of controlling the palm to swing back and forth can be considered a hand gesture.

[0135] For example, the vehicle detects that the user extends his arm and controls his palm to swing back and forth, including: the vehicle determines that the user extends his arm and controls his palm to swing back and forth based on data collected by an in-cabin camera or radar.

[0136] For example, the in-cabin camera could be a driver monitor system (DMS) camera.

[0137] As shown in Figure 3(b), in response to detecting that the user extends his arm and controls his palm to swing back and forth, the vehicle can switch from playing a male voice from the control sound device 1 and playing a chorus from the control sound device 6 to playing a chorus from the control sound device 1 and playing a male voice from the control sound device 6.

[0138] In one embodiment, the vehicle can switch from controlling the sound-emitting device 1 to playing a male voice and controlling the sound-emitting device 6 to playing a chorus, depending on the user's location.

[0139] For example, when a user in the driver's seat area extends their arm and controls their hand to swing back and forth, the sound elements played by the sound-emitting device 1 and the sound-emitting device 6 in the driver's seat area can be interchanged.

[0140] For example, when a user in the rear area extends their arm and controls their hand to swing back and forth, the sound elements played by the sound-emitting device 3 and the sound-emitting device 4 in the rear area can be interchanged.

[0141] In one embodiment, the vehicle can switch from controlling sound device 1 to play male voice and control sound device 6 to play chorus, and then switch back to controlling sound device 1 to play chorus and control sound device 6 to play male voice. This includes: determining sound device 1 and sound device 6 based on the start and end positions of the user's hand swing; and switching from controlling sound device 1 to play male voice and control sound device 6 to play chorus and control sound device 6 to play male voice. For example, at the start position of the hand swing, four fingers point to sound device 1. At the end position of the hand swing, four fingers point to sound device 6.

[0142] The direction of the palm above can be understood as the direction of the four fingers shown in Figure 3(a).

[0143] Figure 3 above illustrates the example of detecting a user extending their arm and controlling their hand to swing back and forth; however, this application is not limited to this. For example, when it is detected that a finger of one hand is pointing at sound-emitting device 1 and a finger of the other hand is pointing at sound-emitting device 6, the sound elements played by sound-emitting device 1 and sound-emitting device 6 can be interchanged. For example, when the vehicle detects that a user's finger first points at sound-emitting device 1 and then at sound-emitting device 6, or when it detects that a user's fingers are simultaneously pointing at sound-emitting device 1 and sound-emitting device 6, the sound elements played by sound-emitting device 1 and sound-emitting device 6 can be interchanged.

[0144] Based on the above technical solution, by detecting the user's hand gestures or body movements, sound elements can be switched to different sound-producing devices. This allows the user to become a "music conductor" within the cabin, enhancing the user's driving experience.

[0145] For example, Figure 4 shows a schematic diagram of another method of playing sound elements through a sound-generating device in the cockpit, as provided in an embodiment of this application.

[0146] As shown in Figure 4(a), the vehicle detects the user extending their arm and controls their palm to face outwards.

[0147] The above actions of extending the user's arm can be considered as the user's physical movements, while controlling the palm to face outwards can be considered as the user's hand gestures.

[0148] As shown in Figure 4(b), in response to detecting that the user extends their arm and controls their palm to face outward, the vehicle can control the sound-emitting device 6 with the palm facing outward to stop playing the chorus.

[0149] Based on the above technical solution, by detecting the user's preset gestures or body movements, it is possible to stop a certain sound-producing device from playing a corresponding sound element. This allows the user to become a "music conductor" within the cabin, contributing to an enhanced driving experience.

[0150] For example, Figure 5 shows a schematic diagram of another method of playing sound elements through a sound-generating device in the cockpit, as provided in an embodiment of this application.

[0151] As shown in Figure 5(a), the vehicle detects that the user extends their arm and controls both hands to clap.

[0152] The above actions of extending the user's arm can be considered as the user's physical movements, while the actions of controlling both hands to clap can be considered as the user's hand gestures.

[0153] As shown in Figure 5(b), in response to detecting that the user extends his arm and controls both hands to clap, the vehicle can control the sound device 7 to play a DJ sound.

[0154] The above-mentioned DJing sound can also be understood as a friction sound or a scratching sound.

[0155] In one embodiment, in response to a user in the driver's area extending their arm and controlling both hands to clap, the vehicle can control the sound device 8 in the driver's area to play a new sound element (e.g., a DJ turn).

[0156] In one embodiment, the vehicle may also control a sound-emitting device in a certain area to play a new sound element (e.g., a DJing sound) when it detects a happy expression of a user in a certain area or when it detects a user laughing.

[0157] The above description uses the example of playing a DJ sound through a sound-emitting device 7 that has not previously played sound elements when at least one of the user's gestures, body movements, or facial expressions is detected. The embodiments of this application are not limited to this. For example, when at least one of the user's gestures, body movements, or facial expressions is detected, one or more of the sound-emitting devices 1-6 can also be controlled to play a DJ sound.

[0158] Based on the above technical solution, by detecting the user's preset gestures, body movements, or facial expressions, it is possible to control a certain sound-emitting device to play sound elements not included in the audio data. This allows the user to experience excitement by having new sound elements played through the in-cabin sound-emitting device, providing feedback to the user's excitement and enhancing the driving experience.

[0159] For example, Figure 6 shows a schematic diagram of another method of playing sound elements through a sound-generating device in the cockpit, as provided in an embodiment of this application.

[0160] As shown in Figure 6(a), while the vehicle is playing sound elements through the sound-emitting device, controls for music element zones can be added to the display screen. When the vehicle detects that the user clicks on the control for the music element zone, it can control the display screen to show the graphical user interface (GUI) as shown in Figure 6(b).

[0161] As shown in Figure 6(b), in response to detecting that the user clicks on the control of the music element section, the vehicle can control the display screen to show the association between the sound-generating device and the music element.

[0162] The above musical elements can be one type of sound element. For example, when the type of audio data is music, the result of parsing the audio data corresponding to that music can include multiple musical elements, such as male voices, female voices, chorus voices, piano sounds, guitar sounds, and background sounds.

[0163] As shown in Figure 6(c), in response to detecting that the user drags the piano tone zone to the guitar tone zone, the vehicle can control the sound-emitting device 3 to switch from playing piano sounds to playing guitar sounds and control the sound-emitting device 4 to switch from playing guitar sounds to playing piano sounds.

[0164] In one embodiment, in response to detecting a user clicking on the piano and guitar sound zones, the vehicle can control the sound-emitting device 3 to switch from playing piano sounds to playing guitar sounds and control the sound-emitting device 4 to switch from playing guitar sounds to playing piano sounds.

[0165] Based on the above technical solution, when parsing audio data and controlling the sound-producing device to play corresponding sound elements, the control for music element partitioning can also be displayed on the screen. This allows users to easily view and adjust the music element partitioning results on the screen.

[0166] The above, in conjunction with Figures 3 to 6, describes the process by which users adjust sound elements through gestures, body movements, facial expressions, or touch commands. The following, in conjunction with Figures 7 and 8, describes the process of intelligent matching between sound elements and sound-producing devices.

[0167] For example, Figure 7 shows a schematic diagram of another method for playing sound elements through a sound-generating device in the cockpit, as provided in an embodiment of this application.

[0168] As shown in Figure 7(a), when "Song B" is played to 00 minutes and 03 seconds, "Song B" is analyzed to obtain male voice, piano sound and guitar sound. When the volume of male singer C is detected to be the largest among multiple sound elements and users are detected in the driver's seat area and passenger seat area of ​​the cabin, the sound-emitting devices 1, 2, 5 and 6 in the front area of ​​the cabin can be controlled to play the voice of male singer C, the sound-emitting device 3 in the rear area can be controlled to play the piano sound and the sound-emitting device 4 in the rear area can be controlled to play the guitar sound.

[0169] In one embodiment, the vehicle can pre-analyze the audio data corresponding to "Song B" to obtain information about each sound element in "Song B". For example, if the male vocals play for the longest duration within the 4 minutes and 56 seconds of "Song B" and there are users in the driver's and passenger's seats, then the sound emitters in the front area can be controlled to play the male vocals while the sound emitters in the rear area can be controlled to play piano and guitar sounds.

[0170] As shown in Figure 7(b), when "Song B" is played at 02 minutes and 23 seconds, the song is analyzed to obtain female vocals, piano sounds, and guitar sounds. When the volume of female singer D is detected to be the largest among the multiple sound elements and users are detected in the driver's seat area and passenger seat area of ​​the cabin, the sound-emitting devices 1, 2, 5, and 6 in the front area of ​​the cabin can be controlled to play the sound element of female singer D, the sound-emitting device 3 in the rear area can be controlled to play the piano sound, and the sound-emitting device 4 in the rear area can be controlled to play the guitar sound.

[0171] Based on the above technical solution, after analyzing and obtaining multiple sound elements, intelligent matching can be performed between the sound-generating device and the music elements according to the information of each sound element (e.g., volume information) and the area where the user is located in the cockpit, which helps to improve the user's auditory experience.

[0172] For example, Figure 8 shows a schematic diagram of another method of playing sound elements through a sound-generating device in the cockpit, as provided in an embodiment of this application.

[0173] As shown in Figure 8(a), when "Song C" is played to 00 minutes and 01 seconds, "Song C" is analyzed to obtain male voice and guitar sound. When the volume of male singer E is detected to be the largest among multiple sound elements and users are detected in the driver's area, passenger area, left side area of ​​the second row and right side area of ​​the second row, the sound-emitting devices 1, 2, 3, 4 and 6 around the driver's area, passenger area, left side area of ​​the second row and right side area of ​​the second row can be controlled to play the voice of male singer E, and the sound-emitting device 5 located on the ceiling can be controlled to play the guitar sound.

[0174] In one embodiment, the vehicle can pre-analyze the audio data corresponding to "Song C" to obtain information about each sound element in "Song C". For example, if the male voice plays for the longest duration during the 4 minutes and 56 seconds of "Song C" and there are users in the driver's seat area, passenger seat area, left side area of ​​the second row, and right side area of ​​the second row, then the sound-emitting devices 1, 2, 3, 4, and 6 surrounding the users can be controlled to play the male voice, while other sound-emitting devices in the cabin can be controlled to play the female voice and guitar sound.

[0175] As shown in Figure 8(b), when "Song C" is played to 01 minutes and 22 seconds, the female voice and guitar sound are obtained by parsing "Song C". When the volume of the female singer F is detected to be the largest among multiple sound elements and users are detected in the driver's area, passenger area, left side area of ​​the second row and right side area of ​​the second row, the sound-emitting devices 1, 2, 3, 4 and 6 around the driver's area, passenger area, left side area of ​​the second row and right side area of ​​the second row can be controlled to play the sound element of the female singer F, and the sound-emitting device 5 located on the ceiling can be controlled to play the guitar sound.

[0176] Based on the above technical solution, after analyzing and obtaining multiple sound elements, intelligent matching can be performed between the sound-generating device and the music elements according to the information of each sound element (e.g., volume information) and the area where the user is located in the cockpit, which helps to improve the user's auditory experience.

[0177] The process of each sound-generating device playing one sound element has been described above with reference to Figures 2 to 8. This application does not impose specific limitations on this process. The following describes, with reference to Figures 9 and 10, the process of playing two or more sound elements from multiple sound elements using a single sound-generating device.

[0178] For example, Figure 9 shows a schematic diagram of another method of playing sound elements through a sound-generating device in the cockpit, as provided in an embodiment of this application.

[0179] As shown in Figure 9, the vehicle can analyze "Song A". For example, analyzing the audio data corresponding to "Song A" at the current moment can yield male voice, female voice, chorus, piano sound, guitar sound, and background sound. At this time, the vehicle can control sound device 1 to play the male voice, control sound device 2 to play the female voice, control sound device 3 to play the piano and guitar sounds, control sound device 4 to play the piano and guitar sounds, control sound device 5 to play the background sound, and control sound device 6 to play the chorus.

[0180] For example, Figure 10 shows a schematic diagram of another method of playing sound elements through a sound-generating device in the cockpit, as provided in an embodiment of this application.

[0181] As shown in Figure 10, the vehicle can analyze "Song C". For example, analyzing the audio data corresponding to "Song C" at the current moment yields male voice, female voice, piano sound, and guitar sound. The vehicle can then categorize these multiple sound elements, such as classifying male and female voices as human voices and piano and guitar sounds as instrument sounds. The vehicle can control sound device 1 to play human voices (including male and female voices), control sound device 2 to play human voices, control sound device 3 to play instrument sounds (including piano and guitar sounds), control sound device 4 to play instrument sounds, control sound device 5 to play human voices, and control sound device 6 to play human voices.

[0182] The above, combined with Figures 7 and 8, describes the process of intelligent matching between the sound-generating device and sound elements when the audio data type is music. The following, combined with Figures 11 and 12, describes the process of intelligent matching between the sound-generating device and sound elements when the audio data type is crosstalk, stand-up comedy, etc.

[0183] For example, Figure 11 shows a schematic diagram of another method of playing sound elements through a sound-generating device in the cockpit, as provided in an embodiment of this application.

[0184] As shown in Figure 11(a), the vehicle is playing a crosstalk performance. At 3 minutes and 2 seconds into the performance, the analysis yields two sound elements: the voice of the lead comedian 1 and the voice of the supporting comedian 2. The vehicle can control the sound-emitting devices 1 and 3 on the left side of the cabin to play the voice of the lead comedian 1, and control the sound-emitting devices 2 and 4 on the right side of the cabin to play the voice of the supporting comedian 2.

[0185] As shown in Figure 11(b), when the crosstalk performance reaches 03 minutes and 12 seconds, the analysis yields three sound elements: the voice of the lead comedian 1, the voice of the supporting comedian 2, and the voice of the audience. The vehicle can control the sound-emitting device 1 in the left area of ​​the cabin to play the voice of the lead comedian 1, control the sound-emitting device 2 in the right area of ​​the cabin to play the voice of the supporting comedian 2, and control the sound-emitting device 7 in the rear area of ​​the cabin to play the voice of the audience.

[0186] For example, at a certain moment, the analysis of "Xiangsheng" yields two sound elements, including the voice of the lead comedian 1 and the voice of the audience. The vehicle can control the sound-emitting device 1 in the left area of ​​the cabin to play the voice of the lead comedian 1 and control the sound-emitting device 7 in the rear area of ​​the cabin to play the voice of the audience.

[0187] In one embodiment, for a group crosstalk performance (e.g., a group crosstalk performance typically includes three or more performers), taking a three-person group crosstalk performance as an example, the audio data can be parsed to obtain three sound elements, such as the voice of the first performer (e.g., the lead comedian), the voice of the second performer (e.g., the supporting comedian), and the voice of the third performer (e.g., the filler comedian). For example, sound-emitting device 1 can be controlled to play the voice of the lead comedian, sound-emitting device 6 can be controlled to play the voice of the supporting comedian, and sound-emitting device 2 can be controlled to play the voice of the filler comedian.

[0188] In one embodiment, audio data can be input into a big data model to obtain the voices of the lead comedian, the supporting comedian, and the clown comedian.

[0189] For example, a big data model can include voiceprint information of actor 1, actor 2, and actor 3; information of lead comedian 1 (e.g., generally the lead comedian in a crosstalk performance); information of supporting comedian 2 (e.g., generally the supporting comedian in a crosstalk performance); information of lead comedian 3 (e.g., generally the supporting comedian in a crosstalk performance and occasionally the supporting comedian); and information of supporting comedian 4. When the audio data includes voiceprint information of actor 1 and actor 2, it can be determined that actor 1's voice is the lead comedian's voice and actor 2's voice is the supporting comedian's voice. When the audio data includes voiceprint information of actor 1, actor 2, and actor 3, it can be determined that actor 1's voice is the lead comedian's voice, actor 2's voice is the supporting comedian's voice, and actor 3's voice is the supporting comedian's voice. When the audio data includes voiceprint information of actor 1 and actor 3, it can be determined that actor 1's voice is the lead comedian's voice and actor 3's voice is the supporting comedian's voice.

[0190] In this way, the voice of the lead comedian is played through the sound device on the left side of the cabin, the voice of the supporting comedian is played through the sound device on the right side of the cabin, and the voice of the audience is played through the sound device in the rear of the cabin. This gives the users in the cabin the feeling of being in a crosstalk performance, which helps to enhance the immersion of the users when listening to crosstalk.

[0191] For example, Figure 12 shows a schematic diagram of another method for playing sound elements through a sound-generating device in the cockpit, as provided in an embodiment of this application.

[0192] As shown in Figure 12, the vehicle is playing a stand-up comedy routine. At 00:22, the stand-up comedy routine is analyzed, yielding two sound elements: the voice of the stand-up comedian 3 and the voice of the audience. The vehicle can control the sound-emitting devices 1 and 2 in the front area of ​​the cabin to play the voice of the stand-up comedian 3, and control the sound-emitting device 7 in the rear area of ​​the cabin to play the voice of the audience.

[0193] For example, Figure 13 shows a schematic diagram of another method of playing sound elements through a sound-generating device in the cockpit, as provided in an embodiment of this application.

[0194] As shown in Figure 13, users can customize the association between the sound-generating device and the music elements. For example, the vehicle can display a prompt message on the screen: "Please drag the music elements to different speakers for a trial listening." When the vehicle detects that the user has dragged different music elements to different speakers and clicks the trial listening control, it can control the speakers to play the corresponding music elements. Users can then test the music elements played by the speakers. If they believe that the current binding relationship between the speakers and the music elements provides a good audio-visual experience, they can click to establish the binding control.

[0195] When a user clicks the "Create Binding Control" button, the vehicle can establish a binding relationship between the audio system and music elements. When the vehicle plays audio data, the audio data can be parsed to obtain multiple music elements. Based on the previously established binding relationship, the vehicle can control different audio systems in the cabin to play the corresponding music elements.

[0196] Figure 14 shows a schematic flowchart of the control method 1400 provided in an embodiment of this application. The method 1400 can be executed by the vehicle 100; or, the method 1400 can also be executed by the computing platform 120 in the vehicle 100; or, the method 1400 can also be executed by a processor, chip, or circuit in the computing platform 120. The method 1400 includes:

[0197] S1410, acquire multiple sound elements corresponding to the audio data.

[0198] Optionally, the type of audio data includes audio from crosstalk, stand-up comedy, concerts, radio, music, and videos.

[0199] Optionally, when the type of the audio data is crosstalk, the multiple sound elements include the sound elements of each of the crosstalk performers and the sound elements of the audience.

[0200] Optionally, when the type of the audio data is stand-up comedy, the multiple sound elements include the sound elements of each of the stand-up comedians and the sound elements of the audience.

[0201] Optionally, when the type of the audio data is a concert, the multiple sound elements include the sound elements of each of one or more singers and the sound elements of the audience.

[0202] Optionally, when the audio data is music or audio from a video, the plurality of sound elements includes at least two of the following: male voice, female voice, chorus, instrumental sound, and background sound. The background sound can be any sound element in the audio data other than male voice, female voice, chorus, and instrumental sound.

[0203] The male voices, female voices, choral voices, instrumental sounds, and background sounds mentioned above can also be called musical elements. Musical elements can be a specific form of expression of sound elements.

[0204] Optionally, obtaining multiple sound elements corresponding to the audio data includes: parsing the audio data to obtain the multiple sound elements.

[0205] Optionally, acquiring multiple sound elements corresponding to the audio data includes: acquiring the multiple sound elements sent by the audio parsing module in the vehicle. Taking method 1400 executed by processor 121 in the aforementioned computing platform 120 as an example, the audio parsing module in the vehicle can be processor 122. Processor 122 can parse the audio data to obtain multiple sound elements. Processor 121 can acquire the multiple sound elements sent by processor 122.

[0206] Optionally, acquiring multiple sound elements corresponding to audio data includes: acquiring the multiple sound elements sent by a cloud server.

[0207] Alternatively, the multiple sound elements can be pre-parsed sound elements.

[0208] For example, the vehicle can obtain multiple sound elements corresponding to the audio data of a song from a cloud server in advance. When it detects that a user has opened Song A through the in-vehicle audio application, it can control the sound-producing device to play the pre-analyzed sound elements.

[0209] S1420, depending on the type of the audio data, control each of the multiple sound-emitting devices in the vehicle cabin to play a portion of the multiple sound elements.

[0210] The multiple sound-generating devices in the vehicle cabin mentioned above can be all the sound-generating devices in the cabin, or they can be some of the sound-generating devices in the cabin.

[0211] Optionally, depending on the type of the audio data, each of the multiple sound-emitting devices in the vehicle cabin is controlled to play a portion of the multiple sound elements, including: determining the playback area of ​​each of the multiple sound elements in the cabin according to the type of the audio data; and controlling each sound-emitting device to play a portion of the multiple sound elements according to the playback area of ​​each of the multiple sound elements in the cabin.

[0212] Optionally, based on the type of the audio data, determining the playback area of ​​each of the plurality of sound elements within the cockpit includes: when the type of the audio data is a first performance type and the audio data includes the sound elements of a first performer and the sound elements of a second performer, determining that the sound-emitting device on the first side of the cockpit plays the sound element of the first performer and determining that the sound-emitting device on the second side of the cockpit plays the sound element of the second performer, wherein the first side and the second side are opposite sides.

[0213] For example, a first performance type may include two main performers and an audience, with one performer on the left and the other on the right, and the audience members beside or behind them. For instance, the first performance type could be a crosstalk performance, with the two performers' voices positioned to the left and right front, respectively; or it could be an interview program, where the interviewee and interviewer are positioned to the left and right front, respectively, or vice versa.

[0214] For example, the first side is the left side area inside the cabin, and the second side is the right side area inside the cabin. According to the type of the audio data, each of the multiple sound-emitting devices in the vehicle cabin is controlled to play a portion of the multiple sound elements, including: when the type of the audio data is a first performance type (e.g., crosstalk), the first sound-emitting device located in the left side area of ​​the cabin is controlled to play the sound element of the first performer (e.g., the lead comedian), and the second sound-emitting device in the right side area of ​​the cabin is controlled to play the sound element of the second performer (e.g., the supporting comedian). The multiple sound-emitting devices include the first sound-emitting device and the second sound-emitting device.

[0215] For example, as shown in Figure 11(a), when the vehicle is playing a crosstalk performance, the sound-emitting devices 1 and 3 on the left side of the cabin can be controlled to play the voice of the lead comedian 1, while the sound-emitting devices 2 and 4 on the right side of the cabin can be controlled to play the voice of the supporting comedian 2. In this way, by playing the lead comedian's voice through the sound-emitting devices on the left side of the cabin and controlling the sound-emitting devices on the right side of the cabin to play the supporting comedian's voice, the user can feel immersed in the performance, thus enhancing the user's sense of immersion.

[0216] Optionally, the method 1400 further includes: when the type of the audio data is the first performance type and the audio data includes the audience's voice elements, controlling the sound-emitting device in the rear area of ​​the cockpit to play the audience's voice elements.

[0217] For example, as shown in Figure 11(b), when the audience's voice is included at 03 minutes and 02 seconds of the "Crosstalk" performance, the sound-emitting device 7 in the rear area can be controlled to play the audience's voice.

[0218] Optionally, depending on the type of the audio data, the playback area of ​​each of the plurality of sound elements in the cabin is determined, including: when the type of the audio data is a second performance type and the audio data includes the sound elements of a third performer and the sound elements of the audience, determining that the sound-emitting device on the third side of the cabin plays the sound elements of the third performer and determining that the sound-emitting device on the fourth side of the cabin plays the sound elements of the audience, wherein the third side and the fourth side are opposite sides.

[0219] For example, the second performance type may include a performer and an audience, wherein the performer is located at the front, and the audience is located beside or behind them. For instance, the second performance type could be a stand-up comedy routine, where the comedian is at the front and the audience is at the back or beside the comedian, or vice versa. As another example, the second performance type could be a concert, where the singer is at the front and the audience is at the back or beside the singer, or vice versa.

[0220] For example, the third side is the front area of ​​the cabin, and the fourth side is the rear area of ​​the cabin. Depending on the type of the audio data, each of the multiple sound-emitting devices in the vehicle cabin is controlled to play a portion of the multiple sound elements. This includes: when the type of the audio data is a second performance type (e.g., a stand-up comedy or a concert), controlling the third sound-emitting device located in the front area of ​​the cabin to play the performer's (e.g., a stand-up comedian or singer) sound elements and controlling the fourth sound-emitting device in the rear area of ​​the cabin to play the audience's sound elements. The multiple sound-emitting devices include the third sound-emitting device and the fourth sound-emitting device.

[0221] For example, as shown in Figure 12, when a stand-up comedy show is playing in the vehicle, the sound-emitting devices 1 and 2 in the front area of ​​the cabin can be controlled to play the voice of the stand-up comedian 3, while the sound-emitting device 7 in the rear area of ​​the cabin can play the voice of the audience. This can give users the feeling of being immersed in a stand-up comedy show, helping to enhance the user's sense of immersion.

[0222] For example, Table 2 shows the correlation between different types of audio data and the playback area in the cockpit.

[0223] Table 2

[0224] The correspondence between the types of audio data, sound elements and playback areas in the cockpit shown in Table 2 above is merely illustrative and is not specifically limited in this embodiment.

[0225] Optionally, depending on the type of the audio data, each of the multiple sound-emitting devices in the vehicle cabin is controlled to play a portion of the multiple sound elements, including: when the type of the audio data is audio data from music or video, each sound-emitting device is controlled to play a portion of the multiple sound elements according to the area where the user is located in the cabin.

[0226] Optionally, depending on the area where the user is located in the cabin, each sound-emitting device is controlled to play a portion of the multiple sound elements, including: when a user is detected in the driver's seat area and the passenger seat area in the cabin, each of the multiple sound-emitting devices in the front area of ​​the cabin is controlled to play a portion of the multiple sound elements.

[0227] Optionally, depending on the area where the user is located in the cabin, each sound-emitting device is controlled to play a portion of the multiple sound elements, including: when it is detected that only the driver's area in the cabin has a user, controlling each of the multiple sound-emitting devices in the driver's area of ​​the cabin to play a portion of the multiple sound elements.

[0228] For example, when it is detected that a user has opened "Song A" through an in-vehicle audio application and multiple musical elements (e.g., including piano and guitar sounds) corresponding to the audio data of "Song A" are obtained, the following can be implemented: When a user is detected in the driver's seat area and the passenger seat area, the sound-emitting device 1 in the front area can be controlled to play piano sounds and the sound-emitting device 2 can be controlled to play guitar sounds; or, when a user is detected in only the driver's seat area, the two sound-emitting devices in the driver's seat area can be controlled to play piano sounds and guitar sounds respectively; or, when a user is detected in the driver's seat area, the passenger seat area, the left side area of ​​the second row, and the right side area of ​​the second row, the sound-emitting device in the front area can be controlled to play piano sounds and the sound-emitting device in the rear area can be controlled to play guitar sounds.

[0229] Optionally, based on the area where the user is located in the cockpit, each sound-emitting device is controlled to play a portion of the multiple sound elements, including: based on the area where the user is located in the cockpit and the information of each sound element in the multiple sound elements, each sound-emitting device is controlled to play a portion of the multiple sound elements; wherein, the information of each sound element includes the volume information of each sound element, and / or, the playback duration of each sound element in the audio data.

[0230] For example, the multiple sound elements include a first sound element and a second sound element. Depending on the area where the user is located in the cabin, each sound-emitting device is controlled to play a portion of the multiple sound elements, including: when a user is detected in the driver's seat area and the passenger seat area in the cabin and the volume of the first sound element is greater than the volume of the second sound element, each sound-emitting device in the front area of ​​the cabin is controlled to play the first sound element, and each sound-emitting device in the rear area of ​​the cabin is controlled to play the second sound element.

[0231] For example, as shown in Figure 7(a), when the volume of male singer C is detected to be the largest among multiple sound elements and a user is detected in the driver's seat area and passenger seat area of ​​the cabin, the sound-emitting devices 1, 2, 5 and 6 in the front area of ​​the cabin can be controlled to play the sound element of male singer C, the sound-emitting device 3 in the rear area can be controlled to play piano music and the sound-emitting device 4 in the rear area can be controlled to play guitar music.

[0232] For example, as shown in Figure 8(b), when the volume of male singer E is detected to be the largest among multiple sound elements and a user is detected in the driver's area, passenger area, left side of the second row, and right side of the second row, the sound-emitting devices 1, 2, 3, 4, and 6 surrounding the driver's area, passenger area, left side of the second row, and right side of the second row can be controlled to play the sound element of male singer E, and the sound-emitting device 5 located on the ceiling can be controlled to play guitar sounds.

[0233] In this way, by intelligently matching the sound-generating devices and sound elements based on the distribution of users in the cabin and the information of each sound element, the process of intelligently matching sound elements with sound-generating devices helps to further enhance the immersive auditory experience of users in the cabin.

[0234] Optionally, the plurality of sound-emitting devices includes a fifth sound-emitting device and a sixth sound-emitting device, and the plurality of sound elements includes a first sound element and a second sound element. Controlling each of the plurality of sound-emitting devices in the vehicle cabin to play a portion of the plurality of sound elements includes: controlling the fifth sound-emitting device to play the first sound element and controlling the sixth sound-emitting device to play the second sound element; wherein, the method further includes: when a user's first gesture and / or first limb movement is detected, controlling the fifth sound-emitting device to play the second sound element and controlling the sixth sound-emitting device to play the first sound element.

[0235] For example, as shown in Figures 3(a) and (b), when the user extends his arm and controls his palm to swing back and forth, the vehicle can switch from playing a male voice from the control sound device 1 and playing a chorus from the control sound device 6 to playing a chorus from the control sound device 1 and playing a male voice from the control sound device 6.

[0236] Optionally, when the user's first gesture and / or first limb movement is detected, the fifth sound-emitting device is controlled to play the second sound element and the sixth sound-emitting device is controlled to play the first sound element, including: when the user's first gesture and / or first limb movement towards the fifth sound-emitting device and the sixth sound-emitting device is detected, the fifth sound-emitting device is controlled to play the second sound element and the sixth sound-emitting device is controlled to play the first sound element.

[0237] Thus, upon detecting a user's explicit instruction to point at the fifth and sixth sound-emitting devices, the sound elements played by the fifth and sixth sound-emitting devices can be switched. For example, this first gesture could be the user pointing the fingers of both hands at the fifth and sixth sound-emitting devices respectively.

[0238] Optionally, the plurality of sound-emitting devices includes a seventh sound-emitting device, and the plurality of sound elements includes a third sound element. Controlling each of the plurality of sound-emitting devices in the vehicle cabin to play a portion of the plurality of sound elements includes: controlling the seventh sound-emitting device to play the third sound element; wherein, the method 1400 further includes: when a second hand gesture and / or second limb movement of a user is detected toward the seventh sound-emitting device, controlling the seventh sound-emitting device to stop playing the third sound element.

[0239] For example, as shown in Figures 4(a) and (b), when the vehicle detects that the user has extended his arm and controls his palm to face outward, it can control the sound-emitting device 6 with the palm facing outward to stop playing the chorus.

[0240] Optionally, the second hand gesture and / or second limb movement can be the user extending their arm and continuously patting the seventh sound-emitting device.

[0241] Optionally, the method 1400 further includes: controlling the seventh sound device to play the third sound element when the user's second gesture and / or second limb movement is detected again towards the seventh sound device.

[0242] Optionally, the plurality of sound elements does not include a fourth sound element, and the method 1400 further includes: controlling an eighth sound-emitting device in the cockpit to play the fourth sound element when a third hand gesture and / or third limb movement of the user is detected.

[0243] For example, the fourth sound element is a DJ turn, a scratching sound, or a rubbing sound.

[0244] For example, as shown in Figures 5(a) and (b), when the vehicle detects that the user has extended his arm and is clapping with both hands, it can control the sound-generating device 7 to play a DJ sound.

[0245] Optionally, the plurality of sound elements does not include the fourth sound element, and the method 1400 further includes: controlling an eighth sound-emitting device in the cockpit to play the fourth sound element when a user's preset facial expression is detected or when a user is detected laughing.

[0246] For example, the preset emoticon is a happy emoticon.

[0247] Optionally, the method 1400 further includes: adjusting the volume of at least some of the multiple sound-emitting devices when a user's fourth gesture and / or fourth limb movement is detected.

[0248] Optionally, the plurality of sound-emitting devices includes a ninth sound-emitting device and a tenth sound-emitting device, and the plurality of sound elements includes a fifth sound element and a sixth sound element. Controlling each of the plurality of sound-emitting devices in the vehicle cabin to play a portion of the plurality of sound elements includes: controlling the ninth sound-emitting device to play the fifth sound element and controlling the tenth sound-emitting device to play the sixth sound element; wherein, the method 1400 further includes: controlling a display device to display the association relationship between the plurality of sound-emitting devices and the plurality of sound elements; when a first touch command from a user is detected regarding the fifth and sixth sound elements, controlling the ninth sound-emitting device to play the sixth sound element and controlling the tenth sound-emitting device to play the fifth sound element; or, when a second touch command from a user is detected regarding the fifth sound element, controlling the ninth sound-emitting device to stop playing the fifth sound element.

[0249] For example, the first touch instruction is a touch action of dragging the fifth sound element to the sixth sound element; or, the first touch instruction can be a touch action of the user tapping the fifth sound element and the sixth sound element with two fingers.

[0250] For example, as shown in Figures 6(a)-(c), when the vehicle detects a user dragging the piano tone zone to the guitar tone zone, it can control the sound-emitting device 3 to switch from playing piano sounds to playing guitar sounds, and control the sound-emitting device 4 to switch from playing guitar sounds to playing piano sounds. In this way, the sound element zones can be displayed on the interface. Through touch commands on the display device, the sound element zones can be swapped, or certain sound elements can be stopped from playing.

[0251] Optionally, the plurality of sound-emitting devices includes an eleventh sound-emitting device, the sound element played by the eleventh sound-emitting device being the seventh sound element among the plurality of sound elements, the eleventh sound-emitting device being located in a first area in the vehicle, and the method further includes: controlling the ambient light in the first area to operate based on information from the seventh sound element.

[0252] For example, as shown in Figure 2(b), the vehicle can control the sound-emitting device 5 to play background sounds. When the background sounds include the sound of leaves rustling, the ambient lights on the vehicle roof can be controlled to display green.

[0253] For example, multiple sound elements include the sound of drums. While controlling the sound-emitting device 8 in the passenger area to play the sound of drums, the vehicle can also control the ambient lights in the passenger area to rhythmically move in accordance with the drumbeats.

[0254] Based on the above technical solution, the system can control the ambient lighting in the area to play the sound element, based on the area where the sound-emitting device is located and the information of the sound element played by the sound-emitting device, thereby further enhancing the user experience.

[0255] Figure 15 shows a schematic block diagram of a control device 1500 provided in an embodiment of this application. The control device 1500 includes: an acquisition unit 1510 for acquiring a plurality of sound elements corresponding to audio data; and a control unit 1520 for controlling each of a plurality of sound-emitting devices in a vehicle cabin to play a portion of the plurality of sound elements according to the type of the audio data.

[0256] Optionally, the control unit 1520 is specifically used to: determine the playback area of ​​each sound element in the cockpit according to the type of the audio data; and control each sound-emitting device to play a portion of the multiple sound elements according to the playback area of ​​each sound element in the cockpit.

[0257] Optionally, the control unit 1520 is specifically configured to: when the type of the audio data is a first performance type, control a first sound-emitting device located in the left-side area of ​​the cabin to play the sound elements of the first performer and control a second sound-emitting device located in the right-side area of ​​the cabin to play the sound elements of the second performer, wherein the plurality of sound-emitting devices includes the first sound-emitting device and the second sound-emitting device; or, when the type of the audio data is a second performance type, control a third sound-emitting device located in the front area of ​​the cabin to play the sound elements of the performer and control a fourth sound-emitting device located in the rear area of ​​the cabin to play the sound elements of the audience, wherein the plurality of sound-emitting devices includes the third sound-emitting device and the fourth sound-emitting device.

[0258] Optionally, the control unit 1520 is further configured to: control the sound-emitting device in the rear area of ​​the cockpit to play the audience's sound elements when the type of the audio data is the first performance type and the audio data includes the audience's sound elements.

[0259] Optionally, the control unit 1520 is specifically used to: when the type of audio data is music or audio data in a video, control each sound-emitting device to play a portion of the multiple sound elements according to the area where the user is located in the cockpit.

[0260] Optionally, the control unit 1520 is specifically configured to: control each sound-emitting device to play a portion of the multiple sound elements based on the area where the user is located in the cockpit and the information of each sound element among the multiple sound elements; wherein the information of each sound element includes the volume information of each sound element and / or the playback duration of each sound element in the audio data.

[0261] Optionally, the plurality of sound-emitting devices includes a fifth sound-emitting device and a sixth sound-emitting device, and the plurality of sound elements includes a first sound element and a second sound element. The control unit 1520 is specifically used to: control the fifth sound-emitting device to play the first sound element and control the sixth sound-emitting device to play the second sound element; wherein, the control device is further used to control the fifth sound-emitting device to play the second sound element and control the sixth sound-emitting device to play the first sound element when a user's first gesture and / or first limb movement is detected.

[0262] Optionally, the control unit 1520 is specifically configured to: when detecting the user's first gesture and / or first limb movement toward the fifth and sixth sound-emitting devices, control the fifth sound-emitting device to play the second sound element and control the sixth sound-emitting device to play the first sound element.

[0263] Optionally, the plurality of sound-emitting devices includes a seventh sound-emitting device, and the plurality of sound elements includes a third sound element. The control unit 1520 is specifically configured to: control the seventh sound-emitting device to play the third sound element; wherein, the control device 1520 is further configured to control the seventh sound-emitting device to stop playing the third sound element when a second hand gesture and / or second limb movement of a user is detected toward the seventh sound-emitting device.

[0264] Optionally, the plurality of sound elements does not include a fourth sound element, and the control device 1520 is also used to control an eighth sound-emitting device in the cockpit to play the fourth sound element when a third hand gesture and / or third limb movement of the user is detected.

[0265] Optionally, the control device 1520 is also configured to control the volume increase of at least some of the multiple sound-emitting devices when a user's fourth hand gesture and / or fourth limb movement is detected.

[0266] Optionally, the plurality of sound-emitting devices includes a ninth sound-emitting device and a tenth sound-emitting device, and the plurality of sound elements includes a ninth sound element and a tenth sound element. The control unit 1520 is specifically configured to: control the ninth sound-emitting device to play the fifth sound element and control the tenth sound-emitting device to play the sixth sound element; wherein, the control device 1520 is further configured to: control the display device to display the association relationship between the plurality of sound-emitting devices and the plurality of sound elements; when a user's first touch command for the fifth sound element and the sixth sound element is detected, control the ninth sound-emitting device to play the sixth sound element and control the tenth sound-emitting device to play the fifth sound element; or, when a user's second touch command for the fifth sound element is detected, control the ninth sound-emitting device to stop playing the fifth sound element.

[0267] Optionally, the plurality of sound-emitting devices includes an eleventh sound-emitting device, the sound element played by the eleventh sound-emitting device being the seventh sound element among the plurality of sound elements, the eleventh sound-emitting device being located in a first area in the vehicle, and the control device 1520 being further configured to: control the ambient light in the first area to operate based on information from the seventh sound element.

[0268] The functions implemented by the acquisition unit 1510 can be implemented by the processor, chip, or circuit in the aforementioned computing platform. For example, the functions implemented by the acquisition unit 1510 can be implemented by the processor 121. The processor 121 can parse the audio data to obtain multiple sound elements; or, the processor 121 can receive multiple sound elements sent from a cloud server.

[0269] The functions implemented by the control unit 1520 can be implemented by the processor, chip, or circuit in the aforementioned computing platform. For example, the functions implemented by the control unit 1520 can be implemented by the processor 122. The processor 122 can control each of the multiple sound-emitting devices in the cockpit to play a portion of multiple sound elements according to the type of audio data; or, the processor 122 can adjust the sound elements played by the sound-emitting devices according to at least one of the user's gestures, body movements, or facial expressions.

[0270] The functions implemented by the acquisition unit 1510 and the control unit 1520 can be implemented by the same processor or by different processors. This application embodiment does not specifically limit this.

[0271] It should be understood that the division of units in the above device is only a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, the units in the device can be implemented by a processor calling software; for example, the device includes a processor connected to memory, which stores instructions. The processor calls the instructions stored in memory to implement any of the above methods or to implement the functions of each unit in the device. The processor can be, for example, a general-purpose processor, such as a CPU or microprocessor, and the memory can be internal or external to the device. Alternatively, the units in the device can be implemented as hardware circuits. The functions of some or all units can be implemented through the design of the hardware circuits, which can be understood as one or more processors. For example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all units are implemented through the design of the logical relationships between the components within the circuit. In another implementation, the hardware circuit can be implemented using a PLD, such as an FPGA, which can include a large number of logic gates. The connection relationships between the logic gates are configured through configuration files, thereby implementing the functions of some or all units. All units of the above devices can be implemented entirely through processor calling software, or entirely through hardware circuits, or partially through processor calling software with the remaining parts implemented through hardware circuits.

[0272] In this application embodiment, a processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction reading and execution capabilities, such as a CPU, microprocessor, GPU, or DSP. In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. These logical relationships are fixed or reconfigurable. For example, the processor may be a hardware circuit implemented as an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the processor loading instructions to implement the functions of some or all of the above units. Furthermore, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, or DPU.

[0273] As can be seen, each unit in the above device can be one or more processors (or processing circuits) configured to implement the above methods, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0274] Furthermore, the units in the above devices can be integrated in whole or in part, or they can be implemented independently. In one implementation, these units are integrated together as a System-on-a-Chip (SoC). The SoC may include at least one processor for implementing any of the above methods or implementing the functions of the units in the device. The at least one processor may be of different types, such as CPU and FPGA, CPU and AI processor, CPU and GPU, etc.

[0275] This application also provides a control device, which includes a processing unit and a storage unit. The storage unit is used to store instructions, and the processing unit executes the instructions stored in the storage unit to cause the device to perform the methods or steps described in the above embodiments.

[0276] Optionally, if the control device is located in a vehicle, the processing unit may be one or more of the processors 121-12n shown in FIG1.

[0277] This application also provides a control system, which includes the control device 1500 and a plurality of sound-generating devices.

[0278] This application also provides a vehicle that may include the control device 1500 or the control system described above.

[0279] This application also provides a computer program product, which includes computer program code that, when run on a computer, causes the computer to perform the methods described in the above embodiments.

[0280] This application also provides a computer-readable medium storing program code that, when run on a computer, causes the computer to perform the methods described in the above embodiments.

[0281] This application also provides a chip, which includes a circuit for performing the methods described in the above embodiments.

[0282] In implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software. The method disclosed in the embodiments of this application can be directly implemented by a hardware processor, or by a combination of hardware and software modules within the processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, power-on erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, detailed descriptions are omitted here.

[0283] It should be understood that in the embodiments of this application, the memory may include read-only memory and random access memory, and provides instructions and data to the processor.

[0284] It should also be understood that, in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0285] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0286] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0287] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0288] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0289] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0290] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0291] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be covered. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A control method characterized by, The method comprises: acquiring a plurality of sound elements corresponding to audio data; controlling each sound emitting device in a vehicle cabin to play part of the plurality of sound elements according to a type of the audio data.

2. The method of claim 1, wherein, The controlling each sound emitting device in the vehicle cabin to play part of the plurality of sound elements according to the type of the audio data comprises: determining a playing area of each sound element in the cabin according to the type of the audio data; controlling each sound emitting device to play part of the plurality of sound elements according to the playing area of each sound element in the cabin.

3. The method according to claim 1 or 2, characterized in that, The controlling each sound emitting device in the vehicle cabin to play part of the plurality of sound elements according to the type of the audio data comprises: when the type of the audio data is a first performance type, controlling a first sound emitting device located in a left side area in the cabin to play sound elements of a first performer and controlling a second sound emitting device located in a right side area in the cabin to play sound elements of a second performer, the plurality of sound elements comprising the sound elements of the first performer and the sound elements of the second performer, and the plurality of sound emitting devices comprising the first sound emitting device and the second sound emitting device; or when the type of the audio data is a second performance type, controlling a third sound emitting device located in a front area in the cabin to play sound elements of a performer and controlling a fourth sound emitting device located in a rear area in the cabin to play sound elements of an audience, the plurality of sound elements comprising the sound elements of the performer and the sound elements of the audience, and the plurality of sound emitting devices comprising the third sound emitting device and the fourth sound emitting device.

4. The method of claim 3, wherein, The method further comprises: when the type of the audio data is the first performance type and the audio data comprises sound elements of an audience, controlling a sound emitting device located in a rear area in the cabin to play the sound elements of the audience.

5. The method of claim 1, wherein, The controlling each sound emitting device in the vehicle cabin to play part of the plurality of sound elements according to the type of the audio data comprises: when the type of the audio data is music or audio data in a video, controlling each sound emitting device to play part of the plurality of sound elements according to an area in the cabin where a user is located.

6. The method of claim 5, wherein, The controlling each sound emitting device to play part of the plurality of sound elements according to the area in the cabin where the user is located comprises: controlling each sound emitting device to play part of the plurality of sound elements according to the area in the cabin where the user is located and information of each sound element in the plurality of sound elements; wherein the information of each sound element comprises volume information of each sound element, and / or a playing duration of each sound element in the audio data.

7. The method according to any one of claims 1 to 6, characterized in that, The plurality of sound emitting devices comprises a fifth sound emitting device and a sixth sound emitting device, and the plurality of sound elements comprises a first sound element and a second sound element, and the method comprises: controlling the fifth sound emitting device to play the first sound element and controlling the sixth sound emitting device to play the second sound element. The method further comprises: controlling the fifth sound emitting device to play the second sound element and controlling the sixth sound emitting device to play the first sound element when a first gesture action and / or a first limb action of a user is detected.

8. The method of claim 7, wherein, The controlling the fifth sound emitting device to play the second sound element and controlling the sixth sound emitting device to play the first sound element when a first gesture action and / or a first limb action of a user is detected comprises: controlling the fifth sound emitting device to play the second sound element and controlling the sixth sound emitting device to play the first sound element when the first gesture action and / or the first limb action of the user directed to the fifth sound emitting device and the sixth sound emitting device is detected.

9. The method according to any one of claims 1 to 8, characterized in that, The plurality of sound emitting devices comprises a seventh sound emitting device, and the plurality of sound elements comprises a third sound element, and the method comprises: controlling the seventh sound emitting device to play the third sound element. The method further comprises: controlling the seventh sound emitting device to stop playing the third sound element when a second gesture action and / or a second limb action of a user directed to the seventh sound emitting device is detected.

10. The method according to any one of claims 1 to 9, characterized in that, The plurality of sound elements does not comprise a fourth sound element, and the method further comprises: controlling an eighth sound emitting device in the vehicle cabin to play the fourth sound element when a third gesture action and / or a third limb action of a user is detected.

11. The method according to any one of claims 1 to 10, characterized in that, The plurality of sound elements does not comprise a fourth sound element, and the method further comprises: controlling an eighth sound emitting device in the vehicle cabin to play the fourth sound element when a happy expression of a user is detected or when laughter of a user is detected.

12. The method according to any one of claims 1 to 11, characterized in that, The method further comprises: adjusting a volume of at least part of the plurality of sound emitting devices when a fourth gesture action and / or a fourth limb action of a user is detected.

13. The method according to any one of claims 1 to 12, characterized in that, The plurality of sound emitting devices comprises a ninth sound emitting device and a tenth sound emitting device, and the plurality of sound elements comprises a fifth sound element and a sixth sound element, and the method comprises: controlling the ninth sound emitting device to play the fifth sound element and controlling the tenth sound emitting device to play the sixth sound element. The method further comprises: controlling a display device to display an association relationship between the plurality of sound emitting devices and the plurality of sound elements. controlling the ninth sound emitting device to play the sixth sound element and controlling the tenth sound emitting device to play the fifth sound element when a first touch instruction of a user directed to the fifth sound element and the sixth sound element is detected; or, When a second touch instruction of the user for the fifth sound element is detected, the ninth sound emitting device is controlled to stop playing the fifth sound element.

14. The method according to any one of claims 1 to 13, characterized in that, The plurality of sound emitting devices includes an eleventh sound emitting device, a sound element played by the eleventh sound emitting device is a seventh sound element in the plurality of sound elements, the eleventh sound emitting device is located in a first region in the vehicle, and the method further includes: According to information of the seventh sound element, the ambient light in the first region is controlled to work.

15. A control device characterized by comprising: Comprise: An acquisition unit is configured to acquire a plurality of sound elements corresponding to audio data; A control unit is configured to control each sound emitting device in a plurality of sound emitting devices in a vehicle cabin to play part of a plurality of sound elements according to a type of the audio data.

16. The apparatus of claim 15, wherein, The control unit is specifically configured to: According to the type of the audio data, determine a playing region of each sound element in the cabin; According to the playing region of each sound element in the cabin, control each sound emitting device to play part of the plurality of sound elements.

17. The apparatus of claim 15 or 16, wherein, The control unit is specifically configured to: When the type of the audio data is a first performance type, control a first sound emitting device located in a left side region in the cabin to play a sound element of a first performer and control a second sound emitting device located in a right side region in the cabin to play a sound element of a second performer, the plurality of sound elements include the sound element of the first performer and the sound element of the second performer, and the plurality of sound emitting devices include the first sound emitting device and the second sound emitting device; Or, When the type of the audio data is a second performance type, control a third sound emitting device located in a front region in the cabin to play a sound element of a performer and control a fourth sound emitting device located in a rear region in the cabin to play a sound element of an audience, the plurality of sound elements include the sound element of the performer and the sound element of the audience, and the plurality of sound emitting devices include the third sound emitting device and the fourth sound emitting device.

18. The apparatus of claim 17, wherein The control unit is further configured to, when the type of the audio data is the first performance type and the audio data includes a sound element of an audience, control a sound emitting device located in a rear region in the cabin to play the sound element of the audience.

19. The apparatus of claim 15, wherein, The control unit is specifically configured to: When the type of the audio data is music or audio data in a video, according to a region in the cabin where a user is located, control each sound emitting device to play part of the plurality of sound elements.

20. The apparatus of claim 19, wherein, The control unit is specifically configured to: According to a region in the cabin where a user is located and information of each sound element in the plurality of sound elements, control each sound emitting device to play part of the plurality of sound elements; The information of each sound element includes volume information of the each sound element, and / or a playing duration of the each sound element in the audio data.

21. The apparatus of any one of claims 15-20, wherein, The plurality of sound production devices comprises a fifth sound production device and a sixth sound production device, and the plurality of sound elements comprises a first sound element and a second sound element, The control unit is specifically configured to control the fifth sound production device to play the first sound element and control the sixth sound production device to play the second sound element. The control device is further configured to control the fifth sound production device to play the second sound element and control the sixth sound production device to play the first sound element when a first gesture action and / or a first limb action of a user is detected.

22. The apparatus of claim 21, wherein, The control unit is specifically configured to: control the fifth sound production device to play the second sound element and control the sixth sound production device to play the first sound element when the first gesture action and / or the first limb action of the user for the fifth sound production device and the sixth sound production device is detected.

23. The apparatus according to any one of claims 15 to 22, characterized in that, The plurality of sound production devices comprises a seventh sound production device, and the plurality of sound elements comprises a third sound element, The control unit is specifically configured to control the seventh sound production device to play the third sound element. The control device is further configured to control the seventh sound production device to stop playing the third sound element when a second gesture action and / or a second limb action of a user for the seventh sound production device is detected.

24. The apparatus of any of claims 15 to 23, wherein, The plurality of sound elements does not comprise a fourth sound element, The control device is further configured to control an eighth sound production device in the cabin to play the fourth sound element when a third gesture action and / or a third limb action of a user is detected.

25. The apparatus of any one of claims 15-23, wherein, The plurality of sound elements does not comprise a fourth sound element, The control device is further configured to control an eighth sound production device in the cabin to play the fourth sound element when a happy expression of a user is detected or when the user laughs.

26. The apparatus of any one of claims 15 to 25, wherein The control device is further configured to adjust the volume of at least part of the plurality of sound production devices when a fourth gesture action and / or a fourth limb action of a user is detected.

27. The apparatus of any one of claims 15-26, wherein, The plurality of sound production devices comprises a ninth sound production device and a tenth sound production device, and the plurality of sound elements comprises a fifth sound element and a sixth sound element, The control unit is specifically configured to control the ninth sound production device to play the fifth sound element and control the tenth sound production device to play the sixth sound element. The control device is further configured to: control a display device to display an association relationship between the plurality of sound production devices and the plurality of sound elements; control the ninth sound production device to play the sixth sound element and control the tenth sound production device to play the fifth sound element when a first touch control instruction of a user for the fifth sound element and the sixth sound element is detected; or control the ninth sound production device to stop playing the fifth sound element when a second touch control instruction of a user for the fifth sound element is detected.

28. The apparatus of any of claims 15 to 27, wherein, The plurality of sound production devices comprises an eleventh sound production device, a sound element played by the eleventh sound production device is a seventh sound element in the plurality of sound elements, the eleventh sound production device is located in a first area in the vehicle, The control device is further configured to control an atmosphere lamp in the first area to work according to information of the seventh sound element.

29. A control device characterized by comprising: Comprising: a memory for storing a computer program; a processor for executing the computer program stored in the memory, so that the device executes the method as claimed in any one of claims 1 to 14.

30. A control system characterized by, The control system comprises a computing platform and a plurality of sound production devices, and the computing platform comprises the control device as claimed in any one of claims 15 to 29.

31. A vehicle characterized by comprise the device as claimed in any one of claims 15 to 29, or comprise the system as claimed in claim 30.

32. A computer-readable storage medium, comprising: instructions stored thereon, which, when executed by a processor, cause the processor to implement the method as claimed in any one of claims 1 to 14.

33. A computer program product, characterised in that, The computer program product comprises computer program code which, when executed on a computer, causes the computer to implement the method as claimed in any one of claims 1 to 14.

34. A chip, characterized by The chip comprises a circuit for executing the method as claimed in any one of claims 1 to 14.

Images