Vehicle control method, control device, vehicle and computer readable storage medium

Abstract

The invention discloses a vehicle control method. The method comprises the steps that information of passengers left in a vehicle cabin is obtained; under the condition that the retained passenger information meets a preset condition, acquiring emotional state information of the retained passenger; under the condition that the remaining passenger is in the preset emotional state, voice information of the remaining passenger is acquired; obtaining matched target music according to the voice information; and controlling the vehicle to play the target music. In the application, according to the information of the remaining passengers in the cabin, the emotional state of the remaining passengers is recognized under the condition that the preset condition is met, and music matching is performed according to the related information in the voice sent by the user at the moment under the condition that the related passengers are in some special emotional states, so that emotional assistance is provided for the remaining passengers through the music, and the experience of the passengers is improved. The vehicle can become an emotion space of the user, so that the vehicle is endowed with more roles, and the vehicle use experience of the user is improved.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and in particular to a vehicle control method, a control device, a vehicle and a computer-readable storage medium. Background Art

[0002] With the development of vehicle technology, vehicles are no longer just a means of transportation for people. How to combine the intelligent functions of vehicles to make them no longer just a single means of transportation, so as to provide users with richer vehicle use scenarios and driving experience, has become an urgent problem to be solved. Summary of the invention

[0003] Embodiments of the present application provide a vehicle control method, a control device, a vehicle, and a computer-readable storage medium.

[0004] The vehicle control method of the embodiment of the present application includes:

[0005] Obtaining the information of the passengers retained in the vehicle cabin;

[0006] When the information of the retained occupant meets a preset condition, obtaining the emotional state information of the retained occupant;

[0007] When the remaining occupant is in a predetermined emotional state, acquiring voice information of the remaining occupant;

[0008] Acquire matching target music according to the voice information;

[0009] The vehicle is controlled to play the target music.

[0010] Thus, in this application, based on the information of retained passengers in the cabin, when the preset conditions are met, the emotional state of the retained passengers is identified, and when the relevant passengers are in certain special emotional states, music matching is performed based on the relevant information in the voice uttered by the user at this time, thereby providing emotional assistance to the retained personnel through music, enabling the vehicle to become an emotional space for the user, thereby giving the vehicle more roles and enhancing the user's car experience.

[0011] In some embodiments, obtaining the information of the retained occupant in the vehicle cabin includes:

[0012] Acquiring the information on the number of remaining passengers according to occupancy signals of each seat in the vehicle cabin; and / or

[0013] The information on the number of remaining passengers is obtained based on the image information of each seat in the vehicle cabin.

[0014] In this way, the number of remaining passengers can be determined through the occupancy signals of each seat or the image information at the seat, or a combination of the two, providing an information basis for the implementation of subsequent steps of the plan.

[0015] In some embodiments, obtaining the information of the retained occupant in the vehicle cabin includes:

[0016] The age information of the retained occupant is obtained based on the image information obtained in the vehicle cabin.

[0017] In this way, by using the images obtained inside the vehicle cabin, the age information of the remaining occupants can be determined, providing an information basis for the implementation of the subsequent steps of the plan.

[0018] In some embodiments, when the retained occupant information meets a preset condition, obtaining the emotional state information of the retained occupant includes:

[0019] When the number of the remaining passengers is 1 and / or the age of the remaining passengers is not greater than a preset age threshold, acquiring facial image information of the remaining passengers;

[0020] The emotional state information of the retained occupant is obtained based on the facial image information.

[0021] In this way, based on the set number and / or age information of retained occupants as preset conditions, when the preset conditions are met, facial image information of the retained occupants can be obtained, and the emotional state information of the retained occupants can be determined based on the facial image information, thereby providing an emotional state information basis for subsequent execution of emotional assistance.

[0022] In some implementations, acquiring matching target music according to the voice information includes:

[0023] Get the voice message of preset duration;

[0024] Performing natural language processing on the voice information of the preset duration to obtain corresponding text information;

[0025] The text information is used as lyrics to perform music matching processing to obtain the target music.

[0026] In this way, relevant information is obtained from the voice uttered by the retained occupants under preset emotions, and then lyrics are matched according to the text information to determine the target music, so that the retained occupants can be relieved of their emotions through music.

[0027] In some implementations, acquiring matching target music according to the voice information includes:

[0028] Get the voice message of preset duration;

[0029] Music matching processing is performed based on the melody accompanying the voice information to obtain the target music.

[0030] In this way, relevant information is obtained from the voice uttered by the retained occupants under preset emotions, and then the target music is matched according to the hummed melody, so that the retained occupants can be relieved from their emotions through music.

[0031] In some implementations, controlling the vehicle to play the target music includes:

[0032] The vehicle is controlled to play the target music with a predetermined playing effect, wherein the predetermined playing effect includes playing the music with a preset initial volume gradually increasing to a target volume.

[0033] In this way, emotional assistance can be provided by playing music that the retained passengers actively hum, providing a sense of companionship, and playing the target music with a predetermined playback effect can reduce the distress that the playing music may cause to the passengers.

[0034] In some implementations, controlling the vehicle to play the target music includes:

[0035] When the target music is played, the vehicle is controlled to continue playing music tracks of the same type as the target music.

[0036] In this way, after the target music is played, other songs of the same type can be played to continue to relieve the emotions of the retained passengers and improve the user experience.

[0037] The vehicle control device according to the embodiment of the present application includes:

[0038] A retained information acquisition module, used to acquire the retained passenger information in the vehicle cabin;

[0039] An emotional information acquisition module, used to acquire emotional state information of the retained occupant when the retained occupant information meets a preset condition;

[0040] A voice acquisition module, used for acquiring voice information of the retained occupant when the retained occupant is in a predetermined emotional state;

[0041] A matching module, used for acquiring matching target music according to the voice information;

[0042] A control module is used to control the vehicle to play the target music.

[0043] The vehicle of the embodiment of the present application includes a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the above-mentioned vehicle control method is implemented.

[0044] The computer-readable storage medium of the embodiment of the present application stores a computer program, and when the computer program is executed by one or more processors, the above-mentioned vehicle control method is implemented.

[0045] Additional aspects and advantages of the embodiments of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0047] Figure 1 is a flow chart of a vehicle control method according to an embodiment of the present application;

[0048] Figure 2 is a flow chart of a vehicle control method according to an embodiment of the present application;

[0049] Figure 3 is a schematic diagram of a vehicle system architecture of an implementation method of the present application;

[0050] Figure 4 is a flow chart of a vehicle control method according to an embodiment of the present application;

[0051] Figure 5 is a flow chart of a vehicle control method according to an embodiment of the present application;

[0052] Figure 6 It is a flow chart of the vehicle control method according to the embodiment of the present application. DETAILED DESCRIPTION

[0053] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions from beginning to end. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the embodiments of the present application, and cannot be understood as limiting the embodiments of the present application.

[0054] See also Figure 1 , the vehicle control method of the embodiment of the present application includes:

[0055] 01: Get the number of passengers remaining in the vehicle cabin;

[0056] 02: When the number of remaining passengers in the vehicle cabin is a preset number, obtaining emotional state information of the remaining passengers;

[0057] 03: When the remaining occupant is in a predetermined emotional state, obtain the voice information of the remaining occupant;

[0058] 04: Obtain matching target music based on voice information;

[0059] 05: Control the vehicle to play the target music.

[0060] The embodiment of the present application provides a control device for a vehicle. The control method of the vehicle of the embodiment of the present application can be implemented by the control device of the vehicle of the embodiment of the present application. Specifically, the control device of the vehicle includes a retained information acquisition module, an emotional information acquisition module, a voice acquisition module, a matching module and a control module. Among them, the retained information acquisition module is used to obtain the retained occupant information in the vehicle cabin. The emotional information acquisition module is used to obtain the emotional state information of the retained occupant when the retained occupant information meets the preset conditions. The voice acquisition module is used to obtain the voice information of the retained occupant when the retained occupant is in a predetermined emotional state. The matching module is used to obtain matching target music according to the voice information. The control module is used to control the vehicle to play the target music.

[0061] The embodiment of the present application also provides a vehicle, which includes a memory and a processor. The vehicle control method of the present application can be implemented by the control device of the present application. Specifically, a computer program is stored in the memory, and the processor is used to obtain the information of the retained occupants in the vehicle cabin, and to obtain the emotional state information of the retained occupants when the retained occupants information meets the preset conditions, and to obtain the voice information of the retained occupants when the retained occupants are in a predetermined emotional state, and to obtain matching target music according to the voice information, and to control the vehicle to play the target music.

[0062] Specifically, with the development of vehicle technology, vehicles are no longer just a simple means of transportation for people. On the basis of transportation, vehicles can also provide users with a lifestyle, a resting place or an emotional space, etc. For example, users drive vehicles to go camping in the wild to enrich their life experience. For another example, users use corresponding accessories to set the vehicle cabin as a rest space or entertainment space. For another example, using the audio and video entertainment system in the vehicle cabin, entertainment activities such as watching movies can be carried out in the vehicle cabin. Therefore, with the development of vehicle intelligence, vehicles can provide users with more added value on the basis of their role as a means of transportation.

[0063] In the control method of the implementation mode of the present application, combined with the information of the retained passengers in the cabin and the emotion recognition of the retained passengers, when the relevant passengers are in certain special emotional states, such as sadness, loss, frustration and other negative emotions, music matching is performed according to the relevant information in the voice uttered by the user in this situation, and emotional assistance is provided to the retained personnel through music, so that the vehicle can become the user's emotional space, thereby giving the vehicle more roles and enhancing the user's car experience.

[0064] In the implementation mode of the present application, it is first necessary to obtain the information of the remaining passengers in the vehicle cabin, and the information of the remaining passengers may include the number of passengers, age information, etc. It can be understood that, under normal circumstances, if there are a large number of passengers in the vehicle cabin, if there are passengers who are in a bad mood, the passengers can comfort each other through words, etc. Therefore, the implementation mode of the present application is applicable to specific circumstances, for example, when the number of remaining passengers in the cabin is small, for example, when the number is 1, the specific number is not limited and can be set according to actual needs. For another example, the remaining passengers in the cabin are certain special groups of people, such as children. It can be understood that children usually have a weaker ability to relieve their emotions, especially in a closed cabin space, and there is a greater need to use external forces to relieve emotions.

[0065] The preset conditions that need to be met to retain the passenger information are also the conditions for implementing the subsequent music matching action. As mentioned above, when setting the preset conditions, the number and age of the retained passengers can be considered as preset conditions.

[0066] When the preset conditions are met, the emotional state information of the retained occupants is obtained, for example, it can be judged through facial expressions, voice and other information, and the specific method is not limited.

[0067] When it is confirmed that the retained occupant is in a predetermined emotional state, the voice pickup function will be started, for example, the voice assistant of the vehicle system can be automatically controlled to start, pick up the user's voice, and match the target music through the voice information, such as the keywords in it. The predetermined emotional state may include such as loss and depression, and of course, it may also include emotions such as happiness, which are not limited here.

[0068] It is understandable that when a person is alone, he or she may express or relieve his or her emotions by talking to himself or herself, humming, etc. In the implementation of the present application, music matching is performed through the picked-up voice information. For example, if the user is humming a specific song, the song matching can be performed through the obtained lyrics or melody, etc. For another example, if the user is talking to himself or herself, the target music can be matched according to some keywords in the song.

[0069] Finally, the matched target music is played to provide emotional care to the retained passengers through music.

[0070] In summary, in this application, based on the information of retained passengers in the cabin, when the preset conditions are met, the emotional state of the retained passengers is identified, and when the relevant passengers are in certain special emotional states, music is matched according to the relevant information in the voice uttered by the user at this time, thereby providing emotional assistance to the retained personnel through music, enabling the vehicle to become an emotional space for the user, thereby giving the vehicle more roles and enhancing the user's car experience.

[0071] See also Figure 2 In some embodiments, step 01 includes:

[0072] 010: Obtaining information on the number of remaining passengers according to occupancy signals of each seat in the vehicle cabin; and / or

[0073] 011: Obtain information on the number of remaining passengers based on the image information of each seat in the vehicle cabin.

[0074] In some embodiments, the retention information acquisition module is used to obtain the information on the number of retained passengers based on the occupancy signals of each seat in the vehicle cabin; and / or to obtain the information on the number of retained passengers based on the image information obtained at each seat in the vehicle cabin.

[0075] In some embodiments, the processor is used to obtain information on the number of remaining passengers based on occupancy signals of each seat in the vehicle cabin; and / or to obtain information on the number of remaining passengers based on image information obtained at each seat in the vehicle cabin.

[0076] Specifically, see Figure 3 In this embodiment, the information of the retained passengers is the number of retained passengers, which can be obtained through the occupancy signal in the vehicle cabin. The seats in the vehicle cabin can be divided into the driver's seat, the front passenger seat, the rear left seat, the rear right seat, etc. Each seat corresponds to a seat module, for example, the driver's seat corresponds to the SMD (Driver Seat Module) module, the front passenger seat corresponds to the SMP (Passenger Seat Module) module, the rear left seat corresponds to the SMBL (Left Back Seat Module) module, and the rear right seat corresponds to the SMBR (Right Back Seat Module) module. These modules are connected to the CEM (Central Electronic Module) of the vehicle through the CAN bus, so that the electrical signal generated at the seat is sent to the CEM. In some examples, a pressure sensor unit or an occupancy sensor based on the pressure sensor unit can be arranged at the seat. When an occupant is located on the seat, the sensor will generate a detection signal, and the CEM determines whether there are occupants retained in each seat based on the received detection signal.

[0077] It is understandable that in some cases, there may be no occupants in the seats, but other items may be retained. In this case, a detection signal may still be generated at the corresponding seat, which will lead to a misjudgment of whether there is a person retained at the seat. In this case, the image information in the cockpit can be combined to determine whether there is really an occupant or other object at the seat position where the signal is generated. The image can be acquired through an imaging device such as a camera in the cockpit. Of course, the image information can also be directly obtained by using the camera to obtain the retained occupant information, that is to say, in the process of obtaining the retained occupant information, the image information can be used alone without being combined with the seat occupancy information.

[0078] In some other implementations, considering the software and hardware costs of image processing, an infrared sensor may be used instead of an image sensor to perform liveness detection, and combined with an occupancy signal to determine the number of remaining passengers.

[0079] Of course, due to cost and other factors, some vehicle cabins may not be equipped with imaging devices and infrared sensors. In this case, the number of remaining passengers can be roughly determined only by the seat occupancy signals.

[0080] In this way, the number of remaining passengers can be determined through the occupancy signals of each seat or the image information at the seat, or a combination of the two, providing an information basis for the implementation of subsequent steps of the plan.

[0081] In some embodiments, step 01 includes:

[0082] The age information of the retained occupants is obtained based on the image information obtained in the vehicle cabin.

[0083] In some embodiments, the retained information acquisition module is used to acquire the age information of the retained occupants based on the acquired image information in the vehicle cabin.

[0084] In certain embodiments, the processor is configured to obtain age information of the remaining occupants based on the image information obtained within the vehicle cabin.

[0085] Specifically, in this embodiment, the retained occupant information is the age information of the retained occupants, and the age information of the retained occupants can be determined by the image information obtained in the vehicle cabin. Specifically, in the actual processing process, face recognition can be performed based on the image in the vehicle cabin obtained. After the face in the image is detected, the feature points, feature areas, etc. in the face are extracted, and then the age of the object in the image is predicted by the relevant algorithm of the age detector. Among them, face recognition detection can be achieved through Haar cascades, HOG+ linear SVM, single-pole detector, etc. After the face area is identified, the boundary of the face is determined, and the feature points therein are extracted, thereby ignoring other parts, so that the age detector can focus on the face rather than other "noise" parts. Furthermore, the age detector can be an age detector generated by a deep learning model based on a convolutional neural network framework.

[0086] The image processing related models can be deployed on the vehicle's cloud server to reduce the processing burden on the vehicle side. Of course, the large cloud model can also be processed into a lightweight model through distillation and other methods and deployed on the vehicle side, thereby providing faster processing and response.

[0087] Of course, in some embodiments, the car owner can establish a user profile for frequent passengers in the vehicle system, and the user profile may include relevant biometric information and age information. During the implementation process, after the user's identity is determined through relevant biometric information, such as facial information, fingerprint information, voiceprint information, etc., the age information of the passenger can be determined.

[0088] In this way, the images obtained inside the vehicle cabin can be used to determine the age information of the remaining occupants, providing an information basis for the implementation of subsequent steps of the plan.

[0089] See also Figure 4 In some embodiments, step 02 includes:

[0090] 021: when the number of the remaining passengers is 1 and / or the age of the remaining passengers is not greater than a preset age threshold, obtaining facial image information of the remaining passengers;

[0091] 022: Obtain emotional status information of retained occupants based on facial image information.

[0092] In certain embodiments, the emotion information acquisition module is used to acquire facial image information of the retained occupants when the number of retained occupants is 1 and / or when the age of the retained occupants meets a preset age threshold, and to acquire emotional state information of the retained occupants based on the facial image information.

[0093] In certain embodiments, the processor is used to obtain facial image information of the retained occupants when the number of retained occupants is 1 and / or when the age of the retained occupants meets a preset age threshold and to obtain emotional state information of the retained occupants based on the facial image information.

[0094] Specifically, as mentioned above, when there are usually a large number of passengers remaining in the cabin, the passengers will usually take the initiative to relieve abnormal emotions through communication and other means. In this case, emotional relief through the vehicle-mounted system may disturb the user and cause discomfort to the user. Therefore, it is possible to consider setting the number of remaining passengers as one of the preset conditions for obtaining the emotional state information of the remaining passengers. In the implementation manner of the present application, the subsequent emotional comfort of the user is set to be carried out when the number of remaining passengers in the cabin is small, for example, the number of remaining passengers is 1. That is, when there are more than one remaining passengers in the cabin, the subsequent steps will no longer be executed.

[0095] Understandably, considering that adults usually have stronger psychological adjustment abilities, the need for emotional relief through the vehicle system may not be strong. Relatively speaking, younger users, such as children, may need other auxiliary means to help relieve emotions. Therefore, it is possible to consider setting the age of the remaining passengers as a preset condition for obtaining the emotional state information of the remaining passengers. In the implementation of the present application, the subsequent emotional comfort of the user is set to be carried out when the age of the remaining passengers in the cabin is relatively young, for example, the remaining passengers are not older than 12 years old. That is, if the remaining passengers in the cabin are older than 12 years old, the subsequent steps will no longer be executed.

[0096] In some examples, both may be used as preset conditions together, that is, there are fewer passengers remaining in the cabin and the remaining passengers are relatively young, for example, the number of remaining passengers is 1 and the remaining person is not older than 12 years old.

[0097] In addition, whether to execute subsequent steps for retained personnel who do not meet the age requirements (over 12 years old) can be processed according to historical records. That is to say, if before the current execution, in a similar scenario, a user (over 12 years old) chooses to use the vehicle system to help relieve emotions. Then, in the current scenario, the subsequent steps will be executed directly for the user. In this implementation, it is necessary to confirm the user's identity information. In one example, the user can set whether to trigger subsequent emotional relief in the vehicle system, or the vehicle system provides feedback through relevant methods, such as text, icons, voice, etc. when performing the emotional relief operation for the first time, and executes it after the user confirms.

[0098] When it is confirmed that the retained occupant information in the cockpit meets the corresponding preset conditions, the facial image information of the retained occupant will be further obtained, such as an image or a frame image captured from a video. And the emotional state information of the occupant is determined through the facial image information. Specifically, the facial image of the occupant obtained by the camera can be used to identify features such as eyes, nose, eyebrows and mouth through image recognition related algorithms, and the occupant's expression can be further confirmed based on the attributes of the features. It can be understood that under different emotional states, the facial features of the occupant will present different states, such as the current morphological changes of the facial features, and the position changes between the various features. The current expression of the occupant can be confirmed through relevant algorithms, so that emotion recognition can be performed based on the expression.

[0099] Please refer again Figure 3 The above-mentioned functions, including turning on the camera, confirming the number of retained passengers, their age, and emotion recognition, can be realized through the DHU (Display Head Unit, driving information and entertainment host) in the vehicle. The DHU can be connected to the CEM of the vehicle through the FlexRay bus and send relevant information to the CEM.

[0100] In this way, based on the set number and / or age information of retained occupants as preset conditions, when the preset conditions are met, facial image information of the retained occupants can be obtained, and the emotional state information of the retained occupants can be determined based on the facial image information, thereby providing an emotional state information basis for subsequent execution of emotional assistance.

[0101] See also Figure 5 In some embodiments, step 04 includes:

[0102] 041: Get the voice message of preset duration;

[0103] 042: Perform natural language processing on the voice information of the preset duration to obtain corresponding text information;

[0104] 043: Use text information as lyrics to perform music matching processing to obtain target music.

[0105] In some embodiments, the matching module is used to obtain voice information of a preset length, to perform natural language processing on the voice information of the preset length to obtain corresponding text information, and to perform music matching processing using the text information as lyrics to obtain target music.

[0106] In some embodiments, the processor is used to obtain voice information of a preset length, to perform natural language processing on the voice information of the preset length to obtain corresponding text information, and to perform music matching processing using the text information as lyrics to obtain target music.

[0107] Specifically, the predetermined emotional state may include relatively negative emotions such as frustration, loss, sadness, etc. When it is determined that the retained occupant is in the above emotional states, the voice assistant of the vehicle-mounted system will automatically start to pick up the voice emitted by the user, thereby obtaining voice information. It is understandable that, in general, when in a negative emotion, the user may relieve emotions by talking to himself, humming songs, etc. In the implementation manner of the present application, the relevant content of the user's self-emotional relief is matched, and then the emotional relief is assisted by the vehicle-mounted system, and highly relevant content is provided to help the user further relieve emotions.

[0108] In the specific implementation process, when it is detected that the retained occupant is in a predetermined emotional state, the voice assistant of the vehicle system can be controlled to turn on to detect the voice activities that may occur in the background during the user's autonomous emotional relief process. In some examples, the voice assistant may remain active in the background when the retained occupant is in a predetermined emotional state. Or, remain active before obtaining voice information at least once. In the process of obtaining voice information, it can be continuously turned on to obtain voice information. It is also possible to pause the detection within a predetermined time, such as 120s, if no voice activity is detected, and turn it on again after pausing for 30s, and repeat this cycle until the entire process is completed (the retained occupant leaves the car or the user's voice activity is detected).

[0109] When the user's voice activity is detected, the preset duration of the action will be continuously acquired, such as 45s, 60s, etc. Further, natural speech processing is performed on the voice information acquired within the predetermined duration. Specifically, it may include converting the voice information into text, and then identifying the slots, extracting entities in the text, and using the extracted entities as keywords to match the lyrics to obtain the target music. In some examples, in order to provide users with the most accurate target music possible, after the acquisition action is completed, the target music can be matched and acquired based on all the keywords acquired.

[0110] In other examples, in order to speed up the matching of the target music, streaming processing can be used, that is, lyrics matching starts after a keyword is processed. It can be understood that at this time, due to the small number of keywords, multiple candidates may be matched. In this case, voice can be picked up and text information can be obtained to match lyrics until the only candidate, that is, the target music, is matched. Generally speaking, there are few cases where multiple consecutive keywords are the same in different songs. By using streaming processing, fewer frames of voice can be used to quickly match the target music, thereby providing corresponding services to the retained passengers more quickly.

[0111] In this way, relevant information is obtained from the voice uttered by the retained occupants under preset emotions, and then lyrics are matched according to the text information to determine the target music, so that the retained occupants can be relieved of their emotions through music.

[0112] See also Figure 6 In some embodiments, step 04 includes:

[0113] 044: Get the voice message of preset duration;

[0114] 045: Perform music matching processing based on the melody accompanying the voice information to obtain the target music.

[0115] In some embodiments, the matching module is used to obtain voice information of a preset duration, and to perform music matching processing based on the melody accompanying the voice information to obtain target music.

[0116] In some embodiments, the processor is used to obtain voice information of a preset duration, and to perform music matching processing based on the melody accompanying the voice information to obtain target music.

[0117] Specifically, in the above implementation, when the user hums the lyrics, the lyrics can be matched according to the text information in the voice to determine the target music. In some scenarios, when the user hums a song, he may only hum the melody without humming the lyrics. For such scenarios, in this implementation, the target music will be matched according to the melody hummed by the user. In actual operation, the music application of the vehicle system can be opened in the background, and similar functions such as song recognition by listening and humming recognition can be enabled, and the target music can be matched according to the melody of the passenger during the humming process.

[0118] The realization of similar functions such as song recognition by listening is based on extracting the audio fingerprint of the melody hummed by the user, and then matching the audio fingerprint to locate similar tracks in the audio database. Audio fingerprints are similar to human fingerprints, which are unique and deterministic feature information extracted from audio signals.

[0119] In this way, relevant information is obtained from the voice uttered by the retained occupants under preset emotions, and then the target music is matched according to the hummed melody, so that the retained occupants can be relieved from their emotions through music.

[0120] In some embodiments, step 05 includes:

[0121] The vehicle is controlled to play the target music with a predetermined playing effect, wherein the predetermined playing effect includes playing at a preset initial volume and gradually increasing to a target volume.

[0122] In some embodiments, the control module is used to control the vehicle to play the target music with a predetermined playing effect, where the predetermined playing effect includes playing with a preset initial volume gradually increasing to a target volume.

[0123] In some embodiments, the processor is used to control the vehicle to play the target music with a predetermined playback effect, where the predetermined playback effect includes playing at a preset initial volume and gradually increasing to a target volume.

[0124] Specifically, it is understandable that suddenly playing music in a quiet cabin may cause a certain degree of disturbance to the occupants. The occupants may also mistakenly believe that there is a malfunction in the relevant functions of the vehicle system, thereby increasing the occupants' distress. In order to alleviate this distress, in the implementation mode of the present application, the target music is controlled to be played with a predetermined playback effect. Specifically, the target music can be played with a gradual entry effect, that is, the preset initial volume that is lower than the target volume is gradually increased until the target volume is reached. In this way, emotional assistance is provided by playing the music that the occupants actively hum, providing a sense of companionship, and at the same time, it will not cause distress to the occupants due to overly abrupt intervention.

[0125] In some examples, considering that in the implementation mode of the present application, music is mainly used for emotional assistance rather than the passengers actively listening to songs, the determined target music can be used as the background music for the current cabin environment, and the volume can be lower than the volume set for daily music playing.

[0126] In this way, emotional assistance can be provided by playing music that the retained passengers actively hum, providing a sense of companionship, and playing the target music with a predetermined playback effect can reduce the distress that the playing music may cause to the passengers.

[0127] In some embodiments, step 05 includes:

[0128] When the target music is played, the vehicle is controlled to continue playing music tracks of the same type as the target music.

[0129] In some embodiments, the control module is used to control the vehicle to continue playing music tracks of the same type as the target music when the target music is finished playing.

[0130] In some embodiments, the processor is used to control the vehicle to continue playing music tracks of the same type as the target music when the target music is finished playing.

[0131] Specifically, during the playback of the target music, if the remaining occupants do not actively intervene, such as choosing to stop playing music or switch to other playback content, after the target music is played, the music tracks of the same type as the target music will continue to be played. The music type of the target music can be determined based on the classification of the target music in the music application. In some examples, after the target music is played, other music in the song list or song list where the target music is located can also be played. The song list or song list is generated by sharing by users of the music application. It can be understood that music of the same music type or added to the same song list or list usually has similar emotions, and can have an emotional relief effect similar to the target music to a certain extent, thereby continuously providing emotional assistance to users.

[0132] Please refer again Figure 3 The above-mentioned voice collection of retained passengers and music playback can be achieved through the AUD (Audio System) in the vehicle. The AUD can be connected to the DHU of the vehicle through the CAN bus and send relevant information to the DHU. The DHU is also used to match music according to the information reported by the AUD and return the target music.

[0133] In this way, after the target music is played, other songs of the same type can be played to continue to relieve the emotions of retained passengers and improve the user experience.

[0134] In some embodiments, during the process of playing music for emotional relief, the emotional detection of the remaining passengers will continue. When it is continuously detected that the user is no longer in a negative emotional state within a certain period of time, such as 10 minutes, the music can be automatically stopped.

[0135] In certain embodiments, after obtaining the user's voice information, natural language generation can be performed based on the result of natural speech processing of the voice information and the emotional state information of the retained occupant, and then the emotional soothing strategy is fed back in the vehicle cabin based on the result of the natural language generation. After receiving the confirmation request for the emotional soothing strategy from the retained occupant, the vehicle is controlled to execute the emotional soothing strategy.

[0136] For example, in one example, the voice message of the retained occupant contains negative words such as scolding, and at this time, the retained occupant is in a negative emotional state, then the emotional soothing strategy can be fed back to the user through natural language generation, such as "Dear car owner, you are in an abnormal mood now. There are thousands of roads, and safety is the first. Do you need soothing music?" After the emotional soothing strategy is fed back to the retained occupant, the retained occupant can confirm the emotional soothing strategy. The retained occupant can reply with affirmative voice requests such as "OK", and after confirming the emotional soothing strategy, the vehicle can be controlled to execute the corresponding emotional soothing strategy.

[0137] The present application also provides a computer-readable storage medium storing a computer program, which implements the above-mentioned vehicle control method when the computer program is executed by one or more processors.

[0138] In the description of this specification, the description with reference to the terms "specifically", "similarly" and the like means that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms are not intended to refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0139] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0140] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A vehicle control method, characterized in that: The method comprises: Obtaining the information of the passengers retained in the vehicle cabin; When the information of the retained occupant meets a preset condition, acquiring emotional state information of the retained occupant; When the remaining occupant is in a predetermined emotional state, acquiring voice information of the remaining occupant; Acquire matching target music according to the voice information; The vehicle is controlled to play the target music.

2. The method according to claim 1, characterized in that The obtaining of the retained passenger information in the vehicle cabin includes: Acquiring the information on the number of remaining passengers according to occupancy signals of each seat in the vehicle cabin; and / or The information on the number of remaining passengers is obtained based on the image information of each seat in the vehicle cabin.

3. The method according to claim 1, characterized in that The obtaining of the retained passenger information in the vehicle cabin includes: The age information of the retained occupant is obtained based on the image information obtained in the vehicle cabin.

4. The method according to claim 1, characterized in that: When the information of the retained occupant meets a preset condition, obtaining the emotional state information of the retained occupant includes: When the number of the remaining passengers is 1 and / or the age of the remaining passengers is not greater than a preset age threshold, acquiring facial image information of the remaining passengers; The emotional state information of the retained occupant is obtained based on the facial image information.

5. The method according to claim 1, characterized in that: The step of acquiring matching target music according to the voice information comprises: Get the voice message of preset duration; Performing natural language processing on the voice information of the preset duration to obtain corresponding text information; The text information is used as lyrics to perform music matching processing to obtain the target music.

6. The method according to claim 1, characterized in that The step of acquiring matching target music according to the voice information comprises: Get the voice message of preset duration; Music matching processing is performed based on the melody accompanying the voice information to obtain the target music.

7. The method according to claim 1, characterized in that The controlling the vehicle to play the target music includes: The vehicle is controlled to play the target music with a predetermined playing effect, wherein the predetermined playing effect includes playing the music with a preset initial volume gradually increasing to a target volume.

8. The method according to claim 1, characterized in that: Controlling the vehicle to play the target music includes: When the target music is played, the vehicle is controlled to continue playing music tracks of the same type as the target music.

9. A vehicle control device, characterized in that: include: A retained information acquisition module, used to acquire the retained passenger information in the vehicle cabin; An emotional information acquisition module, used to acquire emotional state information of the retained occupant when the retained occupant information meets a preset condition; A voice acquisition module, used for acquiring voice information of the retained occupant when the retained occupant is in a predetermined emotional state; A matching module, used for acquiring matching target music according to the voice information; A control module is used to control the vehicle to play the target music.

10. A vehicle, characterized in that: The vehicle comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the method described in claims 1-8 is implemented.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by one or more processors, the method according to any one of claims 1 to 8 is implemented.

Images