Human-vehicle interaction method, device and system, vehicle and storage medium
By using virtual voice images in vehicles based on user status feedback, the problem of poor voice interaction experience in the prior art by using virtual voice images is solved, the user's sense of participation and experience is improved, and a more emotional and focused interaction process is achieved. .
Patent Information
- Application Number
- CN202411033041.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-30
- Publication Date
- 2024-11-15
AI Technical Summary
During the voice interaction between human-vehicles and vehicles, the existing vehicle voice recognition system has a low user experience and lacks an effective feedback mechanism, which affects the user's interactive experience.
By awakening the virtual voice image on the on-board display screen, the matching virtual voice image is determined based on the user's current status (including facial features, voice features and emotional levels), and the feedback results are output to enhance the user's sense of participation and experience, and The image display is adjusted to adapt to the weather light intensity on the external interactive screen of the car.
It improves users' sense of participation and experience in the voice interaction process of human-vehicles. Through emotional resonance and visual feedback, the user interaction process is more focused and comfortable, and enhances the vehicle usage experience.
Smart Images

Figure CN118968979A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of vehicles, and more specifically, to a method, device, system, vehicle and storage medium for human-vehicle interaction in the field of vehicles. Background Art
[0002] At present, vehicle voice recognition mainly refers to the user interacting with the vehicle through voice, and the vehicle's voice recognition system recognizes and responds to the user's voice commands. For example, in some scenarios, the user can control the vehicle to open the trunk or play music through voice.
[0003] However, in the existing technology, the user experience during the human-vehicle voice interaction process needs to be improved.
[0004] Therefore, there is an urgent need for a method of human-vehicle interaction to feed back the results of the vehicle's response to voice commands to the user, thereby improving the user's experience during the human-vehicle voice interaction process. Summary of the invention
[0005] The present application provides a method, device, system, vehicle and storage medium for human-vehicle interaction, which can enhance the experience of the target object during the human-vehicle voice interaction process.
[0006] In a first aspect, a method for human-vehicle interaction is provided, the method comprising: when an on-board display screen of a vehicle is awakened, in response to voice information of a target object, obtaining a current state of the target object; based on the current state, determining a virtual voice image corresponding to the on-board display screen; and outputting a feedback result in response to the voice information through the virtual voice image.
[0007] In the above technical solution, after the vehicle display screen is awakened, the method can respond to the voice information of the target object (user) interacting with the vehicle, and determine the virtual voice image corresponding to the vehicle display screen based on the current state of the user. Furthermore, the feedback result of responding to the voice information is output through the virtual voice image. The virtual voice image is related to the user himself. In the process of interaction between the user and the vehicle, the user can feel a sense of intimacy by responding to the voice information through the virtual voice image. In addition, the virtual voice image is related to the current state of the user. When the virtual voice image responds to the voice information, the virtual voice phenomenon can reflect the current state of the user, so that the user can be more focused when interacting with the vehicle and pay more attention to his current state. Therefore, this method can enhance the sense of participation and experience of the target object in the process of human-vehicle voice interaction.
[0008] In combination with the first aspect, in some possible implementations, based on the current state, the virtual voice image corresponding to the vehicle display screen is determined, including: based on the current state, determining the current emotional level of the target object; and determining the voice image that matches the emotional level as the virtual voice image.
[0009] In the above technical solution, the current state of the target object (user) can reflect the current emotion of the user. Therefore, the method determines the current emotion level of the user based on the current state of the target object (user). Furthermore, the method determines the voice image that matches the emotion level as the virtual voice image. In this way, during the interaction between the user and the vehicle, not only can the voice information be responded to, but also the voice information can be responded to with a virtual voice image that matches the user's emotion level, which can empathize with the user or make the user feel comfortable.
[0010] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, based on the current state, the current emotional level of the target object is determined, including any one of the following: when the current state indicates that the facial feature of the target object is a first preset feature, the emotional level is determined as a happy emotion, and the first preset feature is a facial feature related to happiness; when the current state indicates that the facial feature is a second preset feature, the emotional level is determined as a sad emotion, and the second preset feature is a facial feature related to sadness; when the current state indicates that the target object's speaking tone is calm, the emotional level is determined as a calm emotion; when the current state indicates that the voice feature of the target object is a preset voice feature, the emotional level is determined as an angry emotion, and the preset voice feature is a voice feature related to anger.
[0011] In the above technical solution, when the facial features of the target object (user) are facial features related to happiness, the user is happy. Therefore, when the current state indicates that the facial features of the target object are facial features related to happiness, the method determines the user's emotional level as happy emotion. When the facial features of the user are facial features related to sadness, the user is sad. Therefore, when the current state indicates that the facial features of the target object are facial features related to sadness, the method determines the user's emotional level as sad emotion. When the voice features of the user are voice features related to anger, the user is angry. Therefore, when the current state indicates that the voice features of the target object are voice features related to anger, the method determines the user's emotional level as angry emotion. When the user's current emotion is relatively stable (peaceful), his speech tone is peaceful. Therefore, when the current state indicates that the target object's speech tone is peaceful, the method determines the user's emotional level as peaceful emotion. The method specifically analyzes the current state and can accurately determine the target object's emotional level through different user features.
[0012] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, a voice image that matches the emotion level is determined as the virtual voice image, including any of the following: when the emotion level is a happy emotion, the virtual voice image is determined as a voice image used to convey a happy emotion; when the emotion level is a sad emotion or an angry emotion, the virtual voice image is determined as a voice image with a gentle speaking tone; when the emotion level is a peaceful emotion, the virtual voice image is determined as a voice image used to create a comfortable voice atmosphere.
[0013] In the above technical solution, when the target object's emotional level is happy, the virtual voice image is determined as a voice image used to convey happy emotions, which can not only enhance the target object's sense of participation and experience in the process of human-vehicle voice interaction, but also empathize with the target object, prompting the target object to continue to maintain a happy emotional state, and enhance the target object's car experience. When the emotional level is sad or angry, the virtual voice image is determined as a voice image with a gentle speaking tone, which can prompt the target object's emotional state from bad to good, allowing the target object to maintain a good emotional state and enhance the target object's car experience. When the emotional level is peaceful, the virtual voice image is determined as a voice image used to create a comfortable voice atmosphere, which can empathize with the target object and allow the target object to maintain a stable emotional state.
[0014] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, the feedback result in response to the voice information is outputted by the virtual voice image, including: based on the voice information, locating the sound source of the target object, and determining the position of the target object relative to the vehicle display screen; displaying an image frame including the virtual voice image on a display area corresponding to the position in the vehicle display screen; and outputting the feedback result through the virtual voice image in the image frame.
[0015] In the above technical solution, based on the voice information of the target object, the sound source of the target object is located, and the specific position of the target object (relative to the position of the vehicle display screen) can be clarified. In this way, the feedback information can be displayed in the display area corresponding to the target object on the vehicle display screen, so that the target object can clearly know the feedback information, and improve the target object's experience in the process of human-vehicle voice interaction.
[0016] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, the vehicle-mounted display screen includes an outside-vehicle interactive screen, and an image frame including the virtual voice image is displayed on a display area corresponding to the position in the vehicle-mounted display screen, including: when the light intensity of the current weather environment exceeds a preset range, the contrast and details of the initial image frame including the virtual voice image are enhanced to obtain the image frame and display it on the display area; and / or; contour extraction is performed on the initial image frame to obtain a first image frame; based on the size of the display area, the size of the first image frame is adjusted to obtain the image frame and display it on the display area.
[0017] In the above technical solution, the vehicle-mounted display screen in the vehicle can be installed outside the vehicle, which is called an outside-vehicle interactive screen, so that the outside-vehicle user (the target object outside the vehicle) can interact with the vehicle (or the vehicle user in the vehicle). Usually, the size of the outside-vehicle interactive screen is small. Therefore, in order to fully display the information in the image frame on the outside-vehicle interactive screen, the method extracts the contour of the initial image frame including the virtual voice image, and resizes the image frame after the contour extraction (the first image frame), and finally displays the resized image frame on the display area. In addition, the image displayed on the outside-vehicle interactive screen is easily affected by the light intensity of the external environment, resulting in poor visual effects. In order to enable the outside-vehicle user to clearly know the image frame, the method enhances the contrast and details of the initial image frame including the virtual voice image based on the light intensity of the current weather environment, and displays the enhanced image frame on the display area.
[0018] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, an image frame including the virtual voice image is displayed on a display area corresponding to the position in the vehicle-mounted display screen, including: determining a target state of the virtual voice image, the target state including a behavioral state, a skin state, and a corresponding subject state; and displaying an image frame including the virtual voice image in the target state on the display area.
[0019] In the above technical solution, the target state of the virtual voice image is further discussed. In this way, during the interaction between the target object (user) and the vehicle, the virtual voice image can respond to the voice information in the target state, which helps to attract the attention of the target object and make the user more focused when interacting with the vehicle.
[0020] In a second aspect, a device for human-vehicle interaction is provided, which includes: an acquisition module for acquiring the current state of the target object in response to the voice information of the target object when the vehicle's on-board display screen is awakened; a determination module for determining a virtual voice image corresponding to the on-board display screen based on the current state; and an output module for outputting a feedback result in response to the voice information through the virtual voice image.
[0021] In conjunction with the second aspect, in some possible implementations, the determination module is specifically used to: determine the current emotion level of the target object based on the current state; and determine a voice image that matches the emotion level as the virtual voice image.
[0022] In combination with the second aspect and the above-mentioned implementation methods, in some possible implementation methods, the determination module is specifically further used for any one of the following: when the current state indicates that the facial feature of the target object is a first preset feature, the emotion level is determined as a happy emotion, and the first preset feature is a facial feature related to happiness; when the current state indicates that the facial feature is a second preset feature, the emotion level is determined as a sad emotion, and the second preset feature is a facial feature related to sadness; when the current state indicates that the target object's speaking tone is calm, the emotion level is determined as a calm emotion; when the current state indicates that the voice feature of the target object is a preset voice feature, the emotion level is determined as an angry emotion, and the preset voice feature is a voice feature related to anger.
[0023] In combination with the second aspect and the above-mentioned implementation methods, in some possible implementation methods, the determination module is specifically used for any one of the following: when the emotion level is happy emotion, the virtual voice image is determined as a voice image used to convey happy emotion; when the emotion level is sad emotion or angry emotion, the virtual voice image is determined as a voice image with a gentle speaking tone; when the emotion level is peaceful emotion, the virtual voice image is determined as a voice image used to create a comfortable voice atmosphere.
[0024] In combination with the second aspect and the above-mentioned implementation methods, in some possible implementation methods, the determination module is specifically used to locate the sound source of the target object based on the voice information, and determine the position of the target object relative to the vehicle-mounted display screen; the output module is specifically used to: display an image frame including the virtual voice image on a display area corresponding to the position in the vehicle-mounted display screen; and output the feedback result through the virtual voice image in the image frame.
[0025] In combination with the second aspect and the above-mentioned implementation methods, in some possible implementation methods, the vehicle-mounted display screen includes an outdoor interactive screen, and the output module is specifically used to: when the light intensity of the current weather environment exceeds a preset range, enhance the contrast and details of the initial image frame including the virtual voice image, obtain the image frame and display it on the display area; and / or; extract the contour of the initial image frame to obtain a first image frame; based on the size of the display area, adjust the size of the first image frame to obtain the image frame and display it on the display area.
[0026] In combination with the second aspect and the above-mentioned implementation methods, in some possible implementation methods, the determination module is specifically used to determine the target state of the virtual voice image, which target state includes the behavioral state, skin state and the subject state to which it belongs; the output module is specifically used to display an image frame of the virtual voice image including the target state on the display area.
[0027] In a third aspect, a system for human-vehicle interaction is provided, the system comprising: an external display screen module, an external display screen control module, a sound module and an internal display screen module, the internal display screen module comprising a voice recognition module, a broadcast module and a voice image module, the external display screen module is connected to the internal display screen module via the external display screen control module, the sound module is connected to the internal display screen module, and the broadcast module is connected to the voice recognition module and the voice image module respectively;
[0028] The voice image module is used to embed multiple virtual voice images, the voice recognition module is used to recognize the voice information of the target object, and the external display screen control module is used to:
[0029] Obtaining the on / off state of the virtual voice image displayed on the external display screen module of the vehicle;
[0030] When the switch is in the on state, obtaining a virtual voice image determined by the in-vehicle display screen module from the multiple virtual voice images based on the current state of the target object outside the vehicle;
[0031] Displaying the virtual voice image, and outputting a feedback result in response to the voice information through the virtual voice image based on the external display screen module or the sound module or the broadcast module;
[0032] When the switch is in the closed state, the exterior display screen module is controlled to maintain the previously set state.
[0033] In a fourth aspect, a vehicle is provided, comprising a memory and a processor. The memory is used to store executable program code, and the processor is used to call and run the executable program code from the memory, so that the vehicle executes the method in the first aspect or any possible implementation of the first aspect.
[0034] In a fifth aspect, a computer-readable storage medium is provided, which stores an executable program code. When the executable program code runs on a computer, the computer executes the method in the above-mentioned first aspect or any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] Figure 1 is a schematic diagram of a vehicle used according to an embodiment of the present application;
[0036] Figure 2 is a schematic flow chart of a method for human-vehicle interaction provided in an embodiment of the present application;
[0037] Figure 3 is a schematic diagram of displaying an image frame provided by an embodiment of the present application;
[0038] Figure 4 It is a structural schematic diagram of a human-vehicle interaction device provided in an embodiment of the present application;
[0039] Figure 5 is a structural diagram of a human-vehicle interaction system provided in an embodiment of the present application;
[0040] Figure 6 It is a structural schematic diagram of a vehicle provided in an embodiment of the present application. DETAILED DESCRIPTION
[0041] The technical solution in the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B: "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.
[0042] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features.
[0043] Figure 1 It is a schematic diagram of a vehicle used according to an embodiment of the present application.
[0044] It should be understood that vehicle voice recognition mainly refers to the user interacting with the vehicle through voice, and the vehicle's voice recognition system recognizes and responds to the user's voice information. Figure 1 In the scenario shown, vehicle A can recognize the voice command of the user outside the vehicle to query the weather, and display the weather query results through the interactive screen outside the vehicle; or, the user inside the vehicle can control the display screen inside the vehicle to play music through voice.
[0045] However, in the prior art, the user experience during the human-vehicle voice interaction process is not high.
[0046] Therefore, the present application provides a method for human-vehicle interaction to feed back the result of the vehicle's response to the voice command to the user, thereby improving the user's experience during the human-vehicle voice interaction process.
[0047] Figure 2 It is a schematic flowchart of a method for human-vehicle interaction provided in an embodiment of the present application.
[0048] It should be understood that the method for human-vehicle interaction provided in the embodiment of the present application is applied to a vehicle (eg Figure 1 Vehicle A), or, the method of human-vehicle interaction can also be applied to a target controller in the vehicle, which is a vehicle controller or a controller of an on-board display screen.
[0049] For example, Figure 2 As shown, the method 200 includes:
[0050] Step 201 , when the vehicle-mounted display screen of the vehicle is awakened, the controller obtains the current state of the target object in response to the voice information of the target object.
[0051] It should be understood that the "vehicle display screen" in the above step 201 includes an external interactive screen and an internal display screen. In some embodiments, the internal display screen includes at least one of a central control screen, a head-up display screen, a co-pilot entertainment screen, a headrest screen, and a starry sky ceiling. The "vehicle display screen is awakened" in the above step 201 means that the internal system of the vehicle display screen has been started, and the internal system has been restored from a low-power sleep state to an active state, and can receive and execute instructions, or respond to the operation of the target object to output feedback information.
[0052] It should also be understood that the target object can display a welcome pattern on the external interactive screen by entering the vehicle's welcome and farewell mode to achieve a welcoming effect; the vehicle can also display a pattern of the current vehicle being charged and the current remaining power on the external interactive screen by entering the vehicle's charging display mode; the target object can also display a pattern or text used to express mood on the external interactive screen to express the current mood.
[0053] It should also be understood that the "target object" in the above step 201 refers to the object that interacts with the vehicle. In some embodiments, the target object is a user in the vehicle (target object in the vehicle) or a user outside the vehicle. Normally, the target object can query information through the vehicle display screen or control vehicle components through the vehicle display screen or have a simple voice conversation with the vehicle display screen. In some embodiments, weather information is queried through the vehicle display screen, and the vehicle display screen displays the weather query results; or, the voice playback device in the vehicle is controlled by the vehicle display screen to play music; or, the target object and the vehicle display screen tell each other "goodbye". Therefore, the interaction between the target object and the vehicle is not only achieved by voice commands, but also includes other methods (such as simple greetings such as goodbye). Therefore, the above step 201 is the voice information of the target object.
[0054] It should also be understood that the "current state of the target object" in the above step 201 is used to indicate the state of the target object in the current time period. The state includes behavioral state, emotional state, clothing state and position state, etc. Among them, the emotional state indicates the comprehensive expression of the intensity, tension and persistence of the emotional experience experienced by the target object in a specific situation; the behavioral state indicates what the target object is currently doing; the clothing state indicates the style and color of the target object's current clothes; and the position state indicates the current location of the target object.
[0055] In some embodiments, the method for determining whether the vehicle's onboard display screen is awakened in step 201 includes: a controller receives a voice wake-up word output by a target object; the controller compares the voice wake-up word with a target wake-up word used to wake up the vehicle display screen, and determines the similarity between the voice wake-up word and the target wake-up word; when the similarity is greater than a preset similarity, the controller wakes up the vehicle display screen to determine that the vehicle display screen is awakened; or, the controller wakes up the vehicle display screen in response to a preset wake-up operation of the target object on the vehicle display screen to determine that the vehicle display screen is awakened.
[0056] In some embodiments, the preset wake-up operation includes continuously pressing the vehicle-mounted display screen for a preset time or performing a preset gesture.
[0057] In some embodiments, the controller in step 201 obtains the current state of the target object in response to the voice information of the target object, including at least one of the following: when the current state includes a behavioral state, the controller detects the hand area of the target object; the controller determines the behavioral state of the target object based on the items held in the hand area; or, when the current state includes an emotional state, the controller detects the facial area of the target object; the controller determines the emotional state of the target object based on the facial features on the facial area; or, when the current state includes a dressing state, the controller detects the torso area of the target object; the controller determines the dressing state of the target object based on the clothing features on the torso area; or, the controller determines the position state of the target object through a vehicle-mounted positioning device.
[0058] It should be understood that in the above solution, the emotional state of the target object may also be obtained in other ways, for example, by determining the emotional state of the target object through the tone of speech of the target object.
[0059] In some embodiments, the controller detects the hand region of the target object, including: the controller captures an upper body image of the target object through a vehicle-mounted camera; the controller determines target key points in the upper body image, the target key points include wrists, palms and finger joints; the controller determines the boundaries of the hand based on the target key points; the controller crops an image region surrounded by the boundaries of the hand in the upper body image, and determines the image region as the hand region.
[0060] In some embodiments, the controller determines the target key point in the upper body image, including: the controller aligns the upper body image to adjust the upper body image to a target size and a target direction; the controller extracts features from the aligned upper body image to obtain a first hand feature, which is used to describe the hand feature of the upper body; and the vehicle locates the target key point in the aligned upper body image based on the first hand feature.
[0061] It should be understood that the controller can obtain the hand features of the target object by performing feature extraction on the aligned upper body image. The hand features may include the positions of the wrist, palm and finger joints on the upper body image respectively. Therefore, the controller can locate the target key points in the aligned upper body image based on the hand features.
[0062] In some embodiments, the controller determines the behavior state of the target object based on the object held in the hand area, including any one of the following: when the object is a mobile phone, the controller determines that the target object is playing with the mobile phone; when the object is a water cup, the controller determines that the target object is drinking water; when the object is a cigarette, the controller determines that the target object is smoking; when the object is chopsticks, the controller determines that the target object is eating.
[0063] In some embodiments, the controller determines the dressing state of the target object based on the clothing features on the torso area, including: when the color feature included in the clothing feature is a first color, the controller determines that the dressing state includes wearing the first color; when the clothing feature is a loose feature, the controller determines that the dressing state includes wearing for casual occasions; when the clothing feature is a slim feature, the controller determines that the dressing state includes wearing for formal occasions.
[0064] In some embodiments, the vehicle-mounted positioning device includes at least one of a radar sensor, a camera, and a Global Positioning System (GPS) locator.
[0065] Step 202: The controller determines a virtual voice image corresponding to the vehicle-mounted display screen based on the current state.
[0066] It should be understood that the "virtual voice image corresponding to the vehicle display screen" in the above step 202 refers to a voice image with visual expression and voice interaction capabilities created using artificial intelligence technology, where "virtual" means that the voice image does not exist in the real world, but is a digital product that exists in computer space. These virtual voice images usually combine multimodal interaction technologies such as speech synthesis, facial animation, and body movements, and can simulate human communication methods and interact with users. The virtual voice image can interact with the target object through technologies such as speech recognition, natural language processing, and speech-to-text; the target object can interact with the virtual voice image through voice, and the virtual voice image can output feedback information in response to the target object's voice information in voice or through the display of the vehicle display screen.
[0067] Outputting feedback information in the form of voice means that the virtual voice image uses text-to-speech technology to convert the feedback information into natural and smooth voice for output. This enables the virtual voice image to speak like a real person, providing an auditory interactive experience. Outputting feedback information through the display mode of the vehicle display screen means that the virtual voice image displays the feedback information on the vehicle display screen. In addition, regardless of the voice mode or the display mode, the virtual voice image is displayed on the vehicle display screen.
[0068] In some embodiments, the virtual voice image can replace the work of online customer service and answer questions for vehicle users; the virtual voice image can act as an idol and interact with vehicle users; and the virtual voice image can be controlled through voice commands to query vehicle information or weather information.
[0069] In a possible implementation, step 202 includes: the controller determines the current emotion level of the target object based on the current state; and the controller determines a voice image that matches the emotion level as the virtual voice image.
[0070] It should be understood that the "emotional level" in the above scheme is used to quantify the intensity, tension and persistence of emotional experience, and is a specific division of emotional states in terms of levels. Generally, the emotional level includes happiness, peace, initiative, disgust, fear, sadness, anger and indifference.
[0071] In the above technical solution, the current state of the target object (user) can reflect the current emotion of the user. Therefore, the method determines the current emotion level of the user based on the current state of the target object (user). Furthermore, the method determines the voice image that matches the emotion level as the virtual voice image. In this way, during the interaction between the user and the vehicle, not only can the voice information be responded to, but also the voice information can be responded to with a virtual voice image that matches the user's emotion level, which can empathize with the user or make the user feel comfortable.
[0072] In one possible implementation, the controller determines the current emotion level of the target object based on the current state, including any one of the following: when the current state indicates that the facial feature of the target object is a first preset feature, the controller determines the emotion level as a happy emotion, and the first preset feature is a facial feature related to happiness; when the current state indicates that the facial feature is a second preset feature, the controller determines the emotion level as a sad emotion, and the second preset feature is a facial feature related to sadness; when the current state indicates that the target object's speaking tone is calm, the controller determines the emotion level as a calm emotion; when the current state indicates that the voice feature of the target object is a preset voice feature, the controller determines the emotion level as an angry emotion, and the preset voice feature is a voice feature related to anger.
[0073] It should be understood that the happy emotion level is referred to as happy emotion, the sad emotion level is referred to as sad emotion, and the angry emotion level is referred to as angry emotion. Generally, the peaceful emotion level is referred to as peaceful emotion, which is usually accompanied by a steady speech speed, a moderate volume, and a gentle tone. Therefore, when the current state indicates that the target object has a peaceful speech tone, the emotion level is determined to be a peaceful emotion.
[0074] In the above technical solution, when the facial features of the target object (user) are facial features related to happiness, the user is happy. Therefore, when the current state indicates that the facial features of the target object are facial features related to happiness, the method determines the user's emotional level as happy emotion. When the facial features of the user are facial features related to sadness, the user is sad. Therefore, when the current state indicates that the facial features of the target object are facial features related to sadness, the method determines the user's emotional level as sad emotion. When the voice features of the user are voice features related to anger, the user is angry. Therefore, when the current state indicates that the voice features of the target object are voice features related to anger, the method determines the user's emotional level as angry emotion. When the user's current emotion is relatively stable (peaceful), his speech tone is peaceful. Therefore, when the current state indicates that the target object's speech tone is peaceful, the method determines the user's emotional level as peaceful emotion. The method specifically analyzes the current state and can accurately determine the target object's emotional level through different user features.
[0075] In some embodiments, the first preset feature includes raised corners of the mouth, exposed teeth, raised corners of the eyes, raised eyebrows, dilated pupils, rosy skin and round cheeks.
[0076] In some embodiments, the second preset feature includes wrinkles between the eyebrows, drooping eye corners, tearful eyes, downward curved corners of the mouth, and lack of vitality on the face.
[0077] In some embodiments, the preset voice features include increased volume, increased pitch, faster speaking speed, pauses, repetition of target vocabulary, and rapid tone.
[0078] In one possible implementation, the controller determines the voice image that matches the emotion level as the virtual voice image, including any of the following: when the emotion level is happy, the controller determines the virtual voice image as a voice image used to convey happy emotions; when the emotion level is sad or angry, the controller determines the virtual voice image as a voice image with a gentle speaking tone; when the emotion level is peaceful, the controller determines the virtual voice image as a voice image used to create a comfortable voice atmosphere.
[0079] It should be understood that the “voice image used to convey happy emotions” in the above scheme refers to a voice image with a soft and cute appearance, a lively and warm personality, and a voice image that brings happiness to the target object. Normally, when the target object is sad or angry, comfort should be given. Therefore, in the above scheme, when the emotional level is sad or angry, the controller determines the virtual voice image as a voice image with a gentle speaking tone. The voice image with a gentle speaking tone refers to a voice image with a friendly appearance, a patient and calm personality, and a voice image that can channel the negative emotions of the target object. The “voice image used to create a comfortable voice atmosphere” in the above scheme refers to a voice image with a confident and steady appearance, a calm and composed personality, and the ability to empathize with the target object.
[0080] In the above technical solution, when the target object's emotional level is happy, the virtual voice image is determined as a voice image used to convey happy emotions, which can not only enhance the target object's sense of participation and experience in the process of human-vehicle voice interaction, but also empathize with the target object, prompting the target object to continue to maintain a happy emotional state, and enhance the target object's car experience. When the emotional level is sad or angry, the virtual voice image is determined as a voice image with a gentle speaking tone, which can prompt the target object's emotional state from bad to good, allowing the target object to maintain a good emotional state and enhance the target object's car experience. When the emotional level is peaceful, the virtual voice image is determined as a voice image used to create a comfortable voice atmosphere, which can empathize with the target object and allow the target object to maintain a stable emotional state.
[0081] Step 203: The controller outputs a feedback result in response to the voice information through the virtual voice image.
[0082] It should be understood that the "feedback information" in step 203 is strongly related to the "voice information". In some embodiments, when the voice information is used to instruct the vehicle to query the required information through the target function, the virtual voice image displays the feedback information on the vehicle display screen.
[0083] Optionally, when the voice information is used to instruct the vehicle to query the weather information of the target date through the weather forecast function, the virtual voice image displays the temperature, UV index, humidity, wind speed and precipitation of the target date on the vehicle display screen or outputs it through voice. Optionally, when the voice information is used to instruct the vehicle to query multiple songs of the target type through the music query function, the virtual voice image displays the song titles, singers and playing time of the multiple songs on the vehicle display screen or outputs it through voice.
[0084] It should be understood that the target object's voice information is continuously output, so the controller can receive the voice information in a preset period of time. Therefore, in some embodiments, during the process of receiving the target object's voice information, the controller displays a virtual voice image that is concentrating on listening on the vehicle display screen.
[0085] In other embodiments, during the process of receiving voice information from a target object, the controller detects whether the voice information is interrupted due to the target object ceasing to output; when it is detected that the voice information is interrupted due to the target object ceasing to output, the controller displays a virtual voice image that is thinking on the vehicle-mounted display screen.
[0086] In one possible implementation, step 203 includes: the controller locates the sound source of the target object based on the voice information, and determines the position of the target object relative to the vehicle-mounted display screen; the controller displays an image frame including the virtual voice image on a display area corresponding to the position in the vehicle-mounted display screen; and the controller outputs the feedback result through the virtual voice image in the image frame.
[0087] It should be understood that the "sound source localization" in the above solution refers to analyzing the voice signal corresponding to the voice information to determine the position of the sound source (target object) used to output the voice signal in space.
[0088] In the above technical solution, based on the voice information of the target object, the sound source of the target object is located, and the specific position of the target object (relative to the position of the vehicle display screen) can be clarified. In this way, the feedback information can be displayed in the display area corresponding to the target object on the vehicle display screen, so that the target object can clearly know the feedback information, and improve the target object's experience in the process of human-vehicle voice interaction.
[0089] In some embodiments, the controller locates the sound source of the target object based on the voice information, and determines the position of the target object relative to the vehicle-mounted display screen, including: the controller obtains the voice signal corresponding to the voice information through multiple microphones in the vehicle, and the multiple microphones are located in different positions in the vehicle; the controller converts the voice signal corresponding to each microphone into an electrical signal, and performs feature extraction on the electrical signal corresponding to each microphone to obtain the target feature corresponding to each microphone, and the target feature includes a signal delay difference; the controller determines the target position of the target object relative to a target microphone among the multiple microphones based on the signal delay difference corresponding to each microphone; the controller determines the position of the target object relative to the vehicle-mounted display screen based on the position of the target microphone relative to the vehicle-mounted display screen and the target position.
[0090] It should be understood that the microphones are located in different positions, so the time at which the microphones receive the voice signal corresponding to the voice information is different.
[0091] In some embodiments, the multiple microphones include a first microphone and a second microphone, and the controller determines the target position of the target object relative to a target microphone among the multiple microphones based on the signal delay difference corresponding to each microphone, including: the controller determines the first distance between the target object and the first microphone as the product of the signal delay difference corresponding to the first microphone and the transmission speed of the voice signal; the controller determines the second distance between the target object and the second microphone as the product of the signal delay difference corresponding to the second microphone and the transmission speed of the voice signal; the controller constructs a target triangle based on the target object, the first microphone and the second microphone; in the target triangle, the controller determines the target position based on the first distance, the second distance and the target distance between the first microphone and the second microphone, and the target microphone is the first microphone or the second microphone.
[0092] In some embodiments, the controller determines the target position based on the first distance, the second distance, and the target distance between the first microphone and the second microphone, including: when the target microphone is the first microphone, the controller determines the target position of the target object relative to the first microphone based on the following formula (1), the target position including the direction and distance of the target object relative to the first microphone;
[0093]
[0094] Among them, cosθ is the cosine value of the first angle, the first angle is the first angle between the line segment corresponding to the first distance and the line segment corresponding to the target distance, the first angle is used to determine the direction of the target object relative to the first microphone, the distance of the target object relative to the first microphone is the first distance, b is the second distance, c is the target distance, and a is the first distance.
[0095] In some other embodiments, the controller determines the target position based on the first distance, the second distance, and the target distance between the first microphone and the second microphone, including: when the target microphone is the second microphone, the controller determines the target position of the target object relative to the second microphone based on the following formula (2), the target position including the direction and distance of the target object relative to the second microphone;
[0096]
[0097] in, is the cosine value of the second angle, the second angle is the second angle between the line segment corresponding to the second distance and the line segment corresponding to the target distance, the second angle is used to determine the direction of the target object relative to the second microphone, and the distance of the target object relative to the second microphone is the second distance.
[0098] In some embodiments, the vehicle-mounted display screen is a headrest screen, and the controller displays an image frame including the virtual voice image on a display area corresponding to the position in the vehicle-mounted display screen, including: when the position is the left side of the headrest screen, the controller displays the image frame including the virtual voice image on the left half of the display area in the headrest screen; when the position is the right side of the headrest screen, the controller displays the image frame including the virtual voice image on the right half of the display area in the headrest screen.
[0099] Figure 3 It is a schematic diagram of displaying an image frame provided in an embodiment of the present application.
[0100] For example, the vehicle display screen can display the following information in a certain order within a preset time. Figure 3 The frame-by-frame images shown in (a) (one image frame is displayed at a time) in FIG. 1 include a virtual voice image corresponding to the target state. Figure 3 From (a) in FIG. 1 , it can be seen that the behavior state of the virtual voice image in each image frame is different.
[0101] like Figure 3 As shown in (b), when the target object is located on the left side of the vehicle-external interactive screen, the controller displays the left half of the vehicle-external interactive screen ( Figure 3 In the case where the target object is located on the right side of the vehicle-external interactive screen, the controller displays an image frame including a virtual voice image on the display area corresponding to the "left sound source" shown in (b) of the vehicle-external interactive screen; when the target object is located on the right side of the vehicle-external interactive screen, the controller displays an image frame including a virtual voice image on the display area corresponding to the "left sound source" shown in (b) of the vehicle-external interactive screen; Figure 3 An image frame including a virtual voice image is displayed on the display area corresponding to the "right left sound source" shown in (b) in the figure.
[0102] In one possible implementation, the vehicle-mounted display screen includes an outside-vehicle interactive screen, and the controller displays an image frame including the virtual voice image on a display area corresponding to the position in the vehicle-mounted display screen, including: when the light intensity of the current weather environment exceeds a preset range, the controller enhances the contrast and details of the initial image frame including the virtual voice image, obtains the image frame, and displays it on the display area; and / or; the controller extracts the contour of the initial image frame to obtain a first image frame; the controller adjusts the size of the first image frame based on the size of the display area, obtains the image frame, and displays it on the display area.
[0103] It should be understood that the "external interactive screen" in the above solution is usually a light emitting diode (LED) display screen, which uses light emitting diodes as pixels. The pixels of the LED display screen are composed of red, green and blue LED lamp beads. The in-car display screen generally uses liquid crystal display (LCD) or organic light emitting diode (OLED) technology, which has a higher resolution. In addition, the size of the external interactive screen is relatively small, which is 960*320mm.
[0104] In the above technical solution, the vehicle-mounted display screen in the vehicle can be installed outside the vehicle, which is called an outside-vehicle interactive screen, so that users outside the vehicle can interact with the vehicle (or vehicle users in the vehicle). Usually, the size of the outside-vehicle interactive screen is small. Therefore, in order to fully display the information in the image frame on the outside-vehicle interactive screen, the method extracts the contour of the initial image frame including the virtual voice image, and resizes the image frame after contour extraction (the first image frame), and finally displays the resized image frame on the display area. In addition, the image displayed on the outside-vehicle interactive screen is easily affected by the light intensity of the external environment, resulting in poor visual effects. In order to enable users outside the vehicle to clearly know the image frame, the method enhances the contrast and details of the initial image frame including the virtual voice image based on the light intensity of the current weather environment, and displays the enhanced image frame on the display area.
[0105] In some embodiments, when the light intensity of the current weather environment exceeds a preset range, the controller enhances the contrast and details of the initial image frame including the virtual voice image, obtains the image frame and displays it on the display area, including: when the light intensity of the current weather environment exceeds the preset range, the controller determines whether the light intensity is greater than the maximum light intensity in the preset range; when the light intensity is greater than the maximum light intensity, the controller determines an image area from the initial image frame whose brightness is greater than a first preset brightness value; the controller reduces the brightness value of the pixel points in the image area; or, when the light intensity is less than the maximum light intensity, the controller determines an image area from the initial image frame whose brightness is less than a second preset brightness value; the controller increases the brightness value of the pixel points in the image area, and the first preset brightness value is greater than or equal to the second preset brightness value.
[0106] It should be understood that the "preset range" in the above solution is a range of light intensity suitable for the human eye.
[0107] In some embodiments, the controller adjusts the brightness value of the pixel points in the image area to be smaller, including: for any pixel point, the controller adjusts the brightness value of the pixel point to the first preset brightness value; or, the controller determines the intensity difference between the light intensity and the target light intensity, and determines the ratio of the intensity difference to the target light intensity as a reduction coefficient, and the target light intensity is the light intensity corresponding to the light adaptation area; the controller determines the product of the brightness value of the pixel point and the reduction coefficient as the reduction brightness value; the controller determines the difference between the brightness value and the reduction brightness value as the target brightness value, and adjusts the brightness value of the pixel point to the target brightness value.
[0108] In some embodiments, the controller performs contour extraction on the initial image frame to obtain a first image frame, including: the controller grays the initial image frame to obtain a gray image frame; the controller binarizes the gray image frame to obtain a binary image; the controller performs contour detection on the binary image through a target detection model to obtain an image coordinate set for characterizing the image frame contour; the controller draws a contour on a blank image frame based on the image coordinate set through a target drawing model to obtain the first image frame.
[0109] It should be understood that the "target detection model" in the above scheme can be regarded as the cv2.findContours() function, and the "target drawing model" in the above scheme can be regarded as the cv2.drawContours() function.
[0110] In some embodiments, the controller adjusts the size of the first image frame based on the size of the display area to obtain the image frame and display it on the display area, including: the controller determines the difference between the length of the display area and a preset length as a first length; the controller determines the difference between the width of the display area and a preset width as a second width; the controller adjusts the length of the first image frame to the first length, and adjusts the width of the first image frame to the second width, and displays the resized image frame on the display area.
[0111] In one possible implementation, the controller displays an image frame including the virtual voice image on a display area corresponding to the position in the vehicle-mounted display screen, including: the controller determines a target state of the virtual voice image, the target state including a behavioral state, a skin state, and a corresponding subject state; the controller displays an image frame including the virtual voice image in the target state on the display area.
[0112] It should be understood that the "behavior state" in the above solution indicates what the virtual voice image is currently doing; the "skin state" indicates the current appearance style of the virtual voice image; and the "theme state" indicates the interface theme of the vehicle display screen where the virtual voice image is currently located. It should also be understood that the image frames of the virtual voice image including the target state are displayed on the display area frame by frame. In addition, the target state also includes the position state.
[0113] In the above technical solution, the target state of the virtual voice image is further discussed. In this way, during the interaction between the target object (user) and the vehicle, the virtual voice image can respond to the voice information in the target state, which helps to attract the attention of the target object and make the user more focused when interacting with the vehicle.
[0114] In some embodiments, the controller determines a target state of the virtual voice image, including: the controller determines the target state based on a current state of the target object.
[0115] It should be understood that the current state of the target object includes behavioral state, emotional state, clothing state and position state. In some embodiments, the controller can determine virtual voice images with differences in behavioral state, emotional state, skin state and position state based on the current state. For example, a virtual voice image that is outputting weather query results, a virtual voice image that is concentrating on listening, a voice image used to convey happy emotions, a virtual voice image wearing blue armor, and a virtual voice image displayed on the left half of the display area of the vehicle display screen.
[0116] In some embodiments, the controller determines the target state based on the current state of the target object, including: the controller determines the behavior state of the target object as the behavior state of the virtual voice image; and / or, the controller determines the skin state of the virtual voice image based on the dressing state of the target object; and / or, the controller determines the theme state of the virtual voice image based on the color in the dressing state and the appearance style; and / or, the controller determines the position state of the virtual voice image based on the position state of the target object.
[0117] In some embodiments, the controller determines the skin state of the virtual voice image based on the dressing state of the target object, including at least one of the following: the controller determines the color in the dressing state as the color in the skin state; the controller determines the appearance style in the skin state based on the style of the clothes in the dressing state.
[0118] In some embodiments, the controller determines the appearance style in the skin state based on the style of the clothes in the dressing state, including: when the style is a sports series, the controller determines the appearance style as a loose and comfortable appearance style; when the style is an evening dress series, the controller determines the appearance style as an elegant and noble appearance style; when the style is a print series, the controller determines the appearance style as a fresh and cute appearance style; when the style is a fashion series, the controller determines the appearance style as a simple and elegant appearance style.
[0119] In some embodiments, the controller determines the theme state of the virtual voice image based on the color in the dressing state and the appearance style, including: the controller determines the color in the dressing state as the color of the interface theme of the vehicle display screen; the controller determines the theme style of the interface theme based on the appearance style.
[0120] In some embodiments, the controller determines the theme style of the interface theme based on the appearance style, including: when the appearance style is a loose and comfortable appearance style, the controller determines the theme style as a minimalist blank design style; when the appearance style is an elegant and noble appearance style, the controller determines the theme style as a magazine style; when the appearance style is a fresh and cute appearance style, the controller determines the theme style as a neon color gradient style; when the appearance style is a simple and elegant appearance style, the controller determines the theme style as a glass simulation style.
[0121] It should be understood that the "minimalist blank design style" in the above scheme is a visual design style that emphasizes simplicity and blankness. It reduces unnecessary elements and reserves enough space to make the interface design appear clean, clear, and highlight the core content. The "magazine style" in the above scheme is a visual design style that emphasizes superimposed typesetting and personalization, and it has a strong artistic flavor. The "neon color gradient style" in the above scheme mainly emphasizes the combination of abstract graphics or textures with shiny colors. It uses gradient functions to enrich colors and increase the vitality of the interface. The "glass simulation style" in the above scheme creates a light and fashionable visual effect through blurred backgrounds and translucent page elements.
[0122] In some embodiments, the controller determines the position state of the virtual voice image based on the position state of the target object, including: the controller determines the target position of the target object relative to the vehicle-mounted display screen; the controller determines that the virtual voice image is in a display area of the vehicle-mounted display screen corresponding to the target position, and the target position is on the left or right side.
[0123] It should be understood that when the target position is on the left side, the display area corresponding to the target position is the left half of the display area in the vehicle display screen; when the target position is on the right side, the display area corresponding to the target position is the right half of the display area in the vehicle display screen.
[0124] It should also be understood that in some scenarios, the voice interaction between the target object outside the vehicle and the vehicle is not in a quiet environment. The ambient noise outside the vehicle is often relatively loud. At present, the existing vehicle voice recognition system can effectively eliminate the ambient noise during the voice interaction outside the vehicle, but the user's human ear cannot eliminate it. Therefore, the feedback result of the controller in the vehicle in response to the voice information should be output in a display manner. In some embodiments, the controller displays the feedback result in response to the voice information on the vehicle display screen through the virtual voice image.
[0125] It should also be understood that the virtual voice image in the above scheme is strongly related to the current state of the target object, making the target object feel friendly. In the process of human-vehicle voice interaction, the in-vehicle display screen is no longer a cold machine. The existence of the virtual voice image makes the interaction process full of warmth and more delicate and rich in emotion, and enhances the overall experience of the human-vehicle voice interaction process through visual feedback. Therefore, this method can enhance the target object's sense of participation and experience in the process of human-vehicle voice interaction.
[0126] Figure 4 It is a structural schematic diagram of a human-vehicle interaction device provided in an embodiment of the present application.
[0127] For example, Figure 4 As shown, the device 400 includes:
[0128] Acquisition module 401: used for acquiring the current state of the target object in response to the voice information of the target object when the vehicle-mounted display screen of the vehicle is awakened;
[0129] Determination module 402: used to determine the virtual voice image corresponding to the vehicle-mounted display screen based on the current state;
[0130] Output module 403: used to output the feedback result in response to the voice information through the virtual voice image.
[0131] Optionally, the determination module 402 is specifically used to: determine the current emotion level of the target object based on the current state; and determine a voice image that matches the emotion level as the virtual voice image.
[0132] Optionally, the determination module 402 is also specifically used for any one of the following: when the current state indicates that the facial feature of the target object is a first preset feature, the emotion level is determined as a happy emotion, and the first preset feature is a facial feature related to happiness; when the current state indicates that the facial feature is a second preset feature, the emotion level is determined as a sad emotion, and the second preset feature is a facial feature related to sadness; when the current state indicates that the target object's speaking tone is calm, the emotion level is determined as a calm emotion; when the current state indicates that the voice feature of the target object is a preset voice feature, the emotion level is determined as an angry emotion, and the preset voice feature is a voice feature related to anger.
[0133] Optionally, the determination module 402 is also specifically used for any one of the following: when the emotion level is happy emotion, the virtual voice image is determined as a voice image used to convey happy emotion; when the emotion level is sad emotion or angry emotion, the virtual voice image is determined as a voice image with a gentle speaking tone; when the emotion level is peaceful emotion, the virtual voice image is determined as a voice image used to create a comfortable voice atmosphere.
[0134] Optionally, the determination module 402 is further specifically used to locate the sound source of the target object based on the voice information, and determine the position of the target object relative to the vehicle-mounted display screen; the output module 403 is specifically used to: display an image frame including the virtual voice image on a display area corresponding to the position in the vehicle-mounted display screen; and output the feedback result through the virtual voice image in the image frame.
[0135] Optionally, the vehicle-mounted display screen includes an outdoor interactive screen, and the output module 403 is specifically used to: when the light intensity of the current weather environment exceeds a preset range, enhance the contrast and details of the initial image frame including the virtual voice image, obtain the image frame and display it on the display area; and / or; extract the contour of the initial image frame to obtain a first image frame; based on the size of the display area, adjust the size of the first image frame to obtain the image frame and display it on the display area.
[0136] Optionally, the determination module 402 is further used to determine the target state of the virtual voice image, which target state includes behavioral state, skin state and subject state; the output module 403 is further used to display an image frame of the virtual voice image including the target state on the display area.
[0137] Figure 5 It is a structural diagram of a human-vehicle interaction system provided in an embodiment of the present application.
[0138] For example, Figure 5 As shown, the system 500 includes: an external display screen module 501, an external display screen control module 502, a sound module 503 and an internal display screen module 504, the internal display screen module 504 includes a voice recognition module 505, a broadcast module 506 and a voice image module 507, the external display screen module 501 is connected to the internal display screen module 504 through the external display screen control module 502, the sound module 503 is connected to the internal display screen module 504, and the broadcast module 506 is connected to the voice recognition module 505 and the voice image module 507 respectively;
[0139] The voice image module 507 is used to embed multiple virtual voice images, the voice recognition module 505 is used to recognize the voice information of the target object, and the external display screen control module 502 is used to:
[0140] Obtaining the switch state of the virtual voice image displayed on the vehicle exterior display screen module 501;
[0141] When the switch is in the on state, obtaining a virtual voice image determined by the in-vehicle display screen module 504 from the multiple virtual voice images based on the current state of the target object outside the vehicle;
[0142] Display the virtual voice image, and output a feedback result in response to the voice information through the virtual voice image based on the external display screen module 501 or the sound module 503 or the broadcast module 506;
[0143] When the switch is in the closed state, the exterior display screen module 501 is controlled to maintain the previously set state.
[0144] In some embodiments, the voice recognition module 505 includes voice characteristics such as wake-up, recognition, sound source localization, and semantic understanding, and the overall human-vehicle voice interaction link is completed through the voice recognition module 505; the broadcast module 506, as the end of the link in the voice recognition process, broadcasts feedback on the execution, status judgment, interaction, etc. of the voice information; the voice image module 507 can display the corresponding actions on the vehicle display screen in combination with the voice recognition process; the external display screen control module 502 obtains the switch state of the virtual voice image displayed on the external display screen module 501 by transmitting data with the in-vehicle display screen module 505; the sound module 503 is used to broadcast the feedback results of the virtual voice image output in response to the voice information of the target object.
[0145] In some embodiments, when in the on state, the external vehicle display screen control module 502 controls the external vehicle display screen module 501 to display image frames of the currently set virtual voice image one by one.
[0146] In some embodiments, the external vehicle display screen module 501 is an external vehicle interactive screen, the external vehicle display screen control module 502 is a controller for controlling the external vehicle interactive screen, and the sound module 503 is an external vehicle voice playback device.
[0147] In some embodiments, the in-vehicle display screen module 504 is an in-vehicle display screen, and the broadcast module 506 is an in-vehicle voice playback device.
[0148] In some embodiments, the in-vehicle display screen is a heads-up display screen.
[0149] It should be understood that the head-up display integrates a voice recognition module, a broadcast module and a voice image module, and the head-up display realizes the switch setting of displaying the virtual voice image on other display screens in the car and displaying the virtual voice image on the interactive screen outside the car.
[0150] Figure 6 It is a structural schematic diagram of a vehicle provided in an embodiment of the present application.
[0151] For example, Figure 6 As shown, the vehicle 600 includes: a memory 601 and a processor 602, wherein the memory 601 stores an executable program code 603, and the processor 602 is used to call and execute the executable program code 603 to perform a method of human-vehicle interaction.
[0152] In addition, an embodiment of the present application also protects a device, which may include a memory and a processor, wherein the memory stores an executable program code, and the processor is used to call and execute the executable program code to execute a method of human-vehicle interaction provided by an embodiment of the present application.
[0153] In this embodiment, the functional modules of the device can be divided according to the above method example. For example, each functional module can be corresponded, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is schematic and is only a logical function division. There may be other division methods in actual implementation.
[0154] In the case of dividing each functional module according to each function, the device may further include an acquisition module, a determination module, an output module, etc. It should be noted that all relevant contents involved in the above method embodiment can be referred to the functional description of the corresponding functional module, which will not be repeated here.
[0155] It should be understood that the device provided in this embodiment is used to execute the above-mentioned method of human-vehicle interaction, and thus can achieve the same effect as the above-mentioned implementation method.
[0156] In the case of an integrated unit, the device may include a processing module and a storage module. When the device is applied to a vehicle, the processing module may be used to control and manage the actions of the vehicle. The storage module may be used to support the vehicle in executing related executable program codes, etc.
[0157] The processing module may be a processor or a controller, which may implement or execute various exemplary logic blocks, modules and circuits shown in conjunction with the disclosure of the present application. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module may be a memory.
[0158] In addition, the device provided in the embodiments of the present application may specifically be a chip, a component or a module, and the chip may include a connected processor and a memory; wherein the memory is used to store instructions, and when the processor calls and executes the instructions, the chip can execute a method of human-vehicle interaction provided in the above embodiments.
[0159] This embodiment also provides a computer-readable storage medium, in which an executable program code is stored. When the executable program code is run on a computer, the computer executes the above-mentioned related method steps to implement a method for human-vehicle interaction provided in the above embodiment.
[0160] This embodiment also provides a computer program product. When the computer program product is run on a computer, the computer is caused to execute the above-mentioned related steps to implement a method for human-vehicle interaction provided in the above embodiment.
[0161] Among them, the device, computer-readable storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.
[0162] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.
[0163] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0164] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A method for human-vehicle interaction, characterized in that: The method comprises: When the vehicle-mounted display screen of the vehicle is awakened, in response to the voice information of the target object, obtaining the current state of the target object; Based on the current state, determining a virtual voice image corresponding to the vehicle-mounted display screen; The feedback result in response to the voice information is outputted by the virtual voice image.
2. The method according to claim 1, characterized in that The step of determining the virtual voice image corresponding to the vehicle-mounted display screen based on the current state includes: Based on the current state, determining the current emotion level of the target object; The voice image that matches the emotion level is determined as the virtual voice image.
3. The method according to claim 2, characterized in that Determining the current emotion level of the target object based on the current state includes any one of the following: In the case where the current state indicates that the facial feature of the target object is a first preset feature, determining the emotion level as a happy emotion, the first preset feature being a facial feature associated with happiness; In a case where the current state indicates that the facial feature is a second preset feature, determining the emotion level as sadness, the second preset feature being a facial feature associated with sadness; In the case where the current state indicates that the target object has a peaceful speech tone, determining the emotion level as a peaceful emotion; In a case where the current state indicates that the voice feature of the target object is a preset voice feature, the emotion level is determined to be anger, and the preset voice feature is a voice feature related to anger.
4. The method according to claim 2, characterized in that: The step of determining the voice image that matches the emotion level as the virtual voice image includes any one of the following: In the case where the emotion level is a happy emotion, determining the virtual voice image as a voice image for conveying a happy emotion; When the emotion level is sadness or anger, the virtual voice image is determined to be a voice image with a gentle speaking tone; When the emotion level is a peaceful emotion, the virtual voice image is determined as a voice image for creating a comfortable voice atmosphere.
5. The method according to claim 1, characterized in that The feedback result of responding to the voice information by outputting the virtual voice image includes: Based on the voice information, localize the sound source of the target object to determine the position of the target object relative to the vehicle-mounted display screen; Displaying an image frame including the virtual voice image on a display area corresponding to the position in the vehicle-mounted display screen; The feedback result is outputted through the virtual voice image in the image frame.
6. The method according to claim 5, characterized in that The vehicle-mounted display screen includes an external vehicle interactive screen, and the image frame including the virtual voice image is displayed on a display area corresponding to the position in the vehicle-mounted display screen, including: When the light intensity of the current weather environment exceeds a preset range, the contrast and details of the initial image frame including the virtual voice image are enhanced to obtain the image frame and display it on the display area; and / or; Performing contour extraction on the initial image frame to obtain a first image frame; Based on the size of the display area, the size of the first image frame is adjusted to obtain the image frame and display it on the display area.
7. The method according to claim 5, characterized in that The step of displaying an image frame including the virtual voice image on a display area corresponding to the position in the vehicle-mounted display screen comprises: Determining a target state of the virtual voice image, wherein the target state includes a behavior state, a skin state, and a state of a corresponding theme; An image frame including a virtual voice image of the target state is displayed on the display area.
8. A device for human-vehicle interaction, characterized in that: The device comprises: An acquisition module, configured to acquire a current state of a target object in response to voice information of the target object when the vehicle-mounted display screen of the vehicle is awakened; A determination module, used to determine the virtual voice image corresponding to the vehicle-mounted display screen based on the current state; The output module is used to output the feedback result in response to the voice information through the virtual voice image.
9. A human-vehicle interaction system, characterized in that: The system comprises: An external display screen module, an external display screen control module, a sound module and an internal display screen module, wherein the internal display screen module includes a voice recognition module, a broadcast module and a voice image module, the external display screen module is connected to the internal display screen module via the external display screen control module, the sound module is connected to the internal display screen module, and the broadcast module is connected to the voice recognition module and the voice image module respectively; The voice image module is used to embed multiple virtual voice images, the voice recognition module is used to recognize the voice information of the target object, and the external display screen control module is used to: Acquiring the on / off state of the virtual voice image displayed on the external vehicle display screen module; When the switch state is in the on state, obtaining a virtual voice image determined by the in-vehicle display screen module from the multiple virtual voice images based on the current state of the target object; Displaying the virtual voice image, and outputting a feedback result in response to the voice information through the virtual voice image based on the external vehicle display screen module or the sound module or the broadcast module; When the switch state is the closed state, the exterior display screen module is controlled to maintain a previously set state.
10. A vehicle, characterized in that: The vehicle comprises: A memory for storing executable program codes; A processor, configured to call and run the executable program code from the memory, so that the vehicle executes the method according to any one of claims 1 to 7.
11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores an executable program code, and when the executable program code is executed, the method according to any one of claims 1 to 7 is implemented.
