A human-computer interaction method, device, equipment and vehicle

Abstract

This application relates to the field of assisted driving, specifically providing a human-computer interaction method, device, equipment, and vehicle. The human-computer interaction method includes: acquiring information expressed by a user; acquiring the user's experience intent based on the expressed information; acquiring multiple recommended strategy actions based on the vehicle's system status information and the user's experience intent; the strategy actions are used to control the vehicle's system and are configured with scores; adjusting the scores of the multiple strategy actions based on the user's strategy preference information; determining the strategy action to control the vehicle's system based on the adjusted scores, and executing the control of the vehicle's system. This application can provide strategy actions that conform to the current scenario and user preferences based on the user's expression habits, achieving the user's expectations for human-computer interaction and improving the user's experience evaluation.

Description

Technical Field

[0001] This application relates to the field of human-computer interaction technology, and in particular to a human-computer interaction method, apparatus, equipment and vehicle. Background Technology

[0002] In current in-vehicle mission-oriented human-machine interaction systems, user expressions and system actions are usually tied together, which cannot adapt to different user expression habits, dynamic spatial environments, and equipment conditions.

[0003] like Figure 1 As shown, in an existing in-vehicle task-oriented human-machine interaction system, user expression information can be obtained through multiple modal information from various sensors. Related user intentions are then derived through pattern matching, and corresponding system strategies are provided based on these intentions. Finally, the system action is executed based on system feedback. Throughout the pattern matching process, the relevant features of user expression, user intention, and system strategy are matched accordingly. This is because current in-vehicle human-machine interaction design primarily revolves around inferring user expression from service functions, rather than recommending service functions based on user expression. The advantage of this design is that it can easily convert all service functions into corresponding user expressions based on downstream service interface commands and parameters, achieving direct voice interaction between user expression and service functions. However, this design's drawback is that it ignores the diversity of subjective expression and the ambiguity of subjective demands, especially when users are unfamiliar with device operation. User expressions are often subjective expressions of experiential demands rather than objective descriptions of device operation. Due to empathy, when human-machine interaction strategies are implemented through voice, human expectations for vehicle system feedback increase. Therefore, when users' subjective expressions fail to receive feedback from the vehicle system, or when the vehicle system's feedback strategy fails, user evaluation of the interaction system will be significantly reduced. Summary of the Invention

[0004] In view of this, this application provides a human-computer interaction method, apparatus, device, and vehicle that can provide strategic actions that conform to the current scenario and user preferences based on the user's expression habits, thereby meeting the user's expectations for human-computer interaction and improving the user's experience evaluation.

[0005] It should be understood that in the solution provided in this application, the human-computer interaction method can be executed by a human-computer interaction device or some components within that device. The human-computer interaction device can be a human-machine interface (HMI), an in-vehicle infotainment system (IVI), a vehicle, a server, etc. Some components within the human-computer interaction device can be processing chips, processing circuits, processors, etc.

[0006] The first aspect of this application provides a human-computer interaction method, comprising: acquiring information expressed by a user; acquiring the user's experience intent based on the expressed information; acquiring multiple recommended strategy actions based on the vehicle's system status information and the user's experience intent, the strategy actions being used to control the vehicle's system and configured with scores; adjusting the scores of the multiple strategy actions based on the user's strategy preference information; determining a strategy action to control the vehicle's system based on the adjusted score, and executing the control of the vehicle's system.

[0007] Therefore, in the human-computer interaction method of this application, the user end can express their experience intent through voice, gestures, facial expressions, etc., while the device end can collect the information expressed by the user, perform intent recognition according to a preset intent recognition model or intent recognition algorithm, and obtain the user experience intent corresponding to the information expressed by the user. Based on the recognized user experience intent and the vehicle system's status information, multiple recommended strategy actions are generated, and the scores of the recommended multiple strategy actions are adjusted according to the user's strategy preference information. Finally, the strategy action is determined based on the adjusted score to control the vehicle system. This method is applicable to different user expression habits and different vehicle system states, and can dynamically recommend appropriate strategy actions to users, thereby meeting users' expectations for human-computer interaction and improving user experience evaluation.

[0008] In one possible implementation of the first aspect, determining the strategy action to control the vehicle system based on the adjusted score includes: when determining from two or more strategy actions, and the difference between the scores of the two or more strategy actions is greater than a threshold, determining the strategy action to control the vehicle system based on the highest score.

[0009] Based on the adjusted scores, the strategy action to control the vehicle system can be determined from the recommended strategy actions according to the scores. If the difference between the scores of two or more strategy actions with higher scores is greater than the threshold, it can be considered that the strategy action with the highest score is more in line with the user's experience intention. In this case, the strategy action with the highest score can be directly selected to control the vehicle system.

[0010] In one possible implementation of the first aspect, the method further includes: when determining from two or more policy actions, and the difference between the scores of the two or more policy actions is less than a threshold, prompting the user with the two or more policy actions; receiving the user's selection information, and determining the policy action to control the vehicle system based on the user's selection information.

[0011] Based on the adjusted scores, the system can select a strategy action from multiple recommended strategies to control the vehicle system. If the difference between the scores of two or more strategies with higher scores is less than a threshold, it can be considered that the two or more strategies are more in line with the user's experience intention. At this time, the two or more strategies can be prompted to the user, and the user's selection information can be received to determine the strategy action that matches the user's preference to control the vehicle system.

[0012] One possible implementation of the first aspect also includes: updating the user's policy preference information based on the user's selection information.

[0013] Therefore, when presenting a user with two or more policy actions for selection, the user's policy preference information can be updated based on the user's selection to better align with their usage habits. In this application, the user's policy preference information may specifically be the user's preference for or frequency of use of a particular policy action. This policy preference information can be further refined to better suit the user's individual needs as the user's human-computer interaction method is used.

[0014] In one possible implementation of the first aspect, it further includes: updating the correlation between any two or more of the information expressed by the user, the user's experience intent, the multiple recommended strategy actions, and the user's strategy preference information, based on the user's selection information.

[0015] Therefore, when two or more policy actions are presented to the user for selection, the correlation between the user's expressed information, the user's experience intent, the recommended policy actions, and the user's policy preference information can be updated based on the user's selection information. This makes the human-computer interaction of this application more in line with the user's preferences, and can quickly determine the policy actions that match the user's preferences in subsequent interactions, thereby improving the user experience.

[0016] In one possible implementation of the first aspect, determining the strategy action to control the vehicle system based on the adjusted score includes: determining two or more strategy actions that are higher than a set value as the strategy actions to control the vehicle system based on the adjusted score.

[0017] Based on the adjusted scores, two or more recommended strategies can be selected from the multiple strategies to determine the strategies for controlling the vehicle system, thereby achieving better control performance.

[0018] In one possible implementation of the first aspect, adjusting the scores of the multiple policy actions based on the user's policy preference information includes: performing a weighted calculation of the scores of the multiple policy actions based on the user's policy preference information.

[0019] Therefore, when adjusting the scores of multiple strategy actions, the scores can be weighted by setting weights for the user's strategy preference information, and the scores of multiple strategy actions can be adjusted based on the weighted calculation results.

[0020] In one possible implementation of the first aspect, the information expressed by the user includes one or more of the user's voice information, action information, and facial expression information.

[0021] In one possible implementation of the first aspect, when obtaining the user's experience intent based on the information expressed by the user, it is also obtained based on the user's vital signs information.

[0022] Therefore, when obtaining information expressed by users, information can be obtained by observing how users express themselves through voice, gestures, body movements, or facial expressions, or by monitoring the user's skin temperature, heart rate, etc.

[0023] In one possible implementation of the first aspect, when obtaining the user's experience intent based on the information expressed by the user, it is also obtained based on the session history information; the session history information includes previously determined policy actions for controlling the vehicle system.

[0024] As shown above, the human-computer interaction process of this method can be multi-round. After each round of human-computer interaction, the previously determined strategy actions for controlling the vehicle can be used as the session history and applied to the next round of human-computer interaction to obtain a more accurate understanding of the user's experience intent.

[0025] In one possible implementation of the first aspect, the vehicle's status information includes one or more of the vehicle's device status information and the environmental status information obtained by the vehicle.

[0026] As mentioned above, the status information of the vehicle's infotainment system can specifically include the status of devices such as air conditioning, seats, speakers, displays, lights, sunroof, and windows, as well as environmental status information such as the temperature, humidity, and air quality inside or outside the vehicle.

[0027] One possible implementation of the first aspect further includes: obtaining the user's identity information and obtaining the user's policy preference information based on the user's identity information.

[0028] As described above, this method can store user policy preference information through local storage or cloud storage. When the user's identity information is obtained, the corresponding policy preference information can be obtained based on the user's identity information, thereby making the human-computer interaction process of this method more in line with the user's personalized needs.

[0029] A second aspect of this application provides a human-computer interaction device, comprising: a recognition module for acquiring information expressed by a user and obtaining the user's experience intent based on the expressed information; a recommendation module for acquiring multiple recommended strategy actions based on the vehicle's system status information and the user's experience intent, the strategy actions being used to control the vehicle's system and configured with scores; an adjustment module for adjusting the scores of the multiple strategy actions based on the user's strategy preference information; and an output module for determining the strategy action to control the vehicle's system based on the adjusted scores and executing the control of the vehicle's system.

[0030] In one possible implementation of the second aspect, the output module is specifically used to: determine the strategy action to control the vehicle system based on the highest score when the strategy action is determined from two or more strategy actions and the difference between the scores of the two or more strategy actions is greater than a threshold.

[0031] In one possible implementation of the second aspect, the output module is further configured to: when determined from two or more policy actions, and the difference in the scores of the two or more policy actions is less than a threshold, prompt the user with the two or more policy actions; receive the user's selection information, and determine the policy action to control the vehicle system based on the user's selection information.

[0032] In one possible implementation of the second aspect, the adjustment module is also used to: update the user's policy preference information based on the user's selection information.

[0033] In one possible implementation of the second aspect, it further includes: a training module, used to update the correlation between any two or more of the information expressed by the user, the user's experience intent, the multiple recommended strategy actions, and the user's strategy preference information, based on the user's selection information.

[0034] In one possible implementation of the second aspect, the output module is specifically used to: determine two or more policy actions that are higher than the set value as policy actions to control the vehicle system based on the adjusted score.

[0035] In one possible implementation of the second aspect, the adjustment module is specifically used to: perform a weighted calculation of the scores of the multiple policy actions based on the user's policy preference information.

[0036] In one possible implementation of the second aspect, the information expressed by the user includes one or more of the user's voice information, action information, and facial expression information.

[0037] In one possible implementation of the second aspect, when the identification module obtains the user's experience intent based on the information expressed by the user, it also obtains it based on the user's vital signs information.

[0038] In one possible implementation of the second aspect, when the identification module obtains the user's experience intent based on the information expressed by the user, it also obtains it based on the session history information; the session history information includes previously determined policy actions for controlling the vehicle system.

[0039] In one possible implementation of the second aspect, the vehicle's status information includes one or more of the vehicle's device status information and the environmental status information obtained by the vehicle.

[0040] In one possible implementation of the second aspect, it further includes: an identity authentication module, used to obtain the user's identity information and obtain the user's policy preference information based on the user's identity information.

[0041] A third aspect of this application provides a system comprising:

[0042] A multi-dimensional sensing device is used to collect user expression data and generate information expressed by the user.

[0043] The vehicle-mounted system management device is used to collect the device status data and environmental status data of the vehicle-mounted system, and generate the status information of the vehicle-mounted system.

[0044] Such as the human-computer interaction devices provided in the second aspect and the various optional implementation methods mentioned above.

[0045] In one possible implementation of the third aspect, the human-computer interaction device obtains the user's experience intent based on the user expression information generated by the multi-dimensional sensing device. The human-computer interaction device obtains multiple recommended strategy actions based on the vehicle's system status information and the user's experience intent, adjusts the scores of these multiple strategy actions according to the user's strategy preference information, and determines the strategy action to control the vehicle based on the adjusted scores, thereby executing control of the vehicle.

[0046] In one possible implementation of the third aspect, when determining the strategy action to control the vehicle system based on the adjusted score, the human-machine interaction device can select from two or more strategy actions. When the difference in scores between the two or more strategy actions is greater than a threshold, the strategy action to control the vehicle system is determined based on the highest score. When the difference in scores between the two or more strategy actions is less than the threshold, the two or more strategy actions are presented to the user, the user's selection information is received, and the strategy action to control the vehicle system is determined based on the user's selection information. The human-machine interaction device can also, based on the adjusted score, determine two or more strategy actions with scores higher than a set value as the strategy action to control the vehicle system.

[0047] In one possible implementation of the third aspect, the human-computer interaction device can update the user's strategy preference information based on the user's selection information. It can also update the correlation between any two or more of the information expressed by the user, the user's experience intent, the recommended multiple strategy actions, and the user's strategy preference information based on the user's selection information.

[0048] In one possible implementation of the third aspect, the human-computer interaction device or the multi-dimensional sensing device can also be used to obtain the user's identity information and obtain the user's strategy preference information based on the user's identity information.

[0049] In one possible implementation of the third aspect, the system is a vehicle.

[0050] In one possible implementation of the third aspect, the multi-dimensional sensing device may include one or more of a microphone module, a camera module, and a human body sensor module disposed in the vehicle.

[0051] In one possible implementation of the third aspect, the vehicle management device may include one or more of the following: air conditioning controller, seat controller, speaker controller, display controller, lighting controller, sunroof controller, window controller, curtain controller, and environmental sensor module, all located in the vehicle.

[0052] A fourth aspect of this application provides an electronic device comprising:

[0053] A processor and an interface circuit, wherein the processor accesses a memory through the interface circuit, the memory storing program instructions, which, when executed by the processor, cause the processor to perform a human-computer interaction method among the various technical solutions provided by the first aspect and the various alternative implementations described above.

[0054] In one possible implementation of the fourth aspect, when a user expresses themselves through voice, gestures, facial expressions, or other collectable means, the processor can obtain acquisition instructions from the memory via the interface circuit to collect information such as the user's voice, gestures, facial expressions, or body movements, and then organize and generate the information expressed by the user. The processor can also obtain recognition instructions from the memory via the interface circuit to identify the user's experiential intent based on the information expressed by the user. Furthermore, the processor can obtain processing instructions from the memory via the interface circuit to obtain multiple recommended strategy actions based on the vehicle's status information and the user's experiential intent, and simultaneously adjust the scores of these strategy actions based on the user's strategy preference information. Finally, the processor can obtain output instructions from the memory via the interface circuit to output strategy actions to control the vehicle based on the adjusted scores, thereby achieving vehicle control.

[0055] In one possible implementation of the fourth aspect, the electronic device is a human-computer interaction interface or an in-vehicle infotainment system.

[0056] In one possible implementation of the fourth aspect, the electronic device is a vehicle.

[0057] In one possible implementation of the fourth aspect, the electronic device is one of the vehicle's infotainment system or onboard computer.

[0058] A fifth aspect of this application provides a computing device, comprising:

[0059] processor, and

[0060] A memory storing program instructions that, when executed by the processor, cause the processor to perform human-computer interaction methods as provided in the first aspect and the various alternative implementations described above.

[0061] In one possible implementation of the fifth aspect, when a user expresses themselves through voice, gestures, facial expressions, or other collectable means, the processor can collect information such as the user's voice, gestures, facial expressions, or body movements by executing acquisition instructions in the memory, and then organize and generate the information expressed by the user. The processor can also identify the user's experiential intent based on the information expressed by the user by executing recognition instructions in the memory. Furthermore, the processor can obtain multiple recommended strategy actions based on the vehicle's status information and the user's experiential intent by executing processing instructions in the memory, and adjust the scores of these strategy actions based on the user's strategy preference information. Finally, the processor can output strategy actions to control the vehicle based on the adjusted scores by executing output instructions in the memory, thereby achieving vehicle control.

[0062] In one possible implementation of the fifth aspect, the computing device is a human-computer interaction interface or an in-vehicle infotainment system.

[0063] In one possible implementation of the fifth aspect, the computing device is a vehicle.

[0064] In one possible implementation of the fifth aspect, the computing device is one of the vehicle's infotainment system or onboard computer.

[0065] A sixth aspect of this application provides a computer-readable storage medium having program instructions stored thereon, which, when executed by a computer, cause the computer to perform a human-computer interaction method as provided in the first aspect and various alternative implementations described above.

[0066] The seventh aspect of this application provides a computer program product including program instructions that, when executed by a computer, cause the computer to perform a human-computer interaction method as provided in the first aspect and the various alternative implementations described above.

[0067] In summary, the human-computer interaction method, apparatus, device, and vehicle provided in this application acquire user expression information to identify user experience intentions. Based on the user experience intentions and the vehicle's status information, multiple recommended strategy actions are generated, which are used to control the vehicle's system. Since each user has different strategy preferences, the scores of multiple strategy actions can be adjusted based on the user's strategy preference information, and the strategy action to control the vehicle's system is determined based on the adjusted scores. Specifically, the strategy action to control the vehicle's system can be determined directly based on the highest score, or the user can be prompted with two or more strategy actions with similar scores, and the final decision can be made based on the user's selection information. In other implementations of this application, the user's selection information can also be used to update the user's strategy preference information, as well as update the correlation between the user's expression information, user experience intentions, multiple recommended strategy actions, and the user's strategy preference information, so that the human-computer interaction is more in line with the user's personalized preferences. The human-computer interaction method, apparatus, device, and vehicle provided in this application are applicable to different user groups and do not require complex learning costs. They can provide strategy actions that conform to the current scenario and user preferences based on the user's expression habits, achieving the user's expectations for human-computer interaction and improving the user's experience evaluation. Attached Figure Description

[0068] Figure 1 This is an interaction flowchart for an in-vehicle mission-oriented human-machine interaction system.

[0069] Figure 2 A schematic diagram illustrating an application scenario of a human-computer interaction method provided in an embodiment of this application;

[0070] Figure 3 A framework diagram illustrating an application scenario of a human-computer interaction method provided in this application embodiment;

[0071] Figure 4 A schematic diagram of the architecture of the state processing unit provided in an embodiment of this application;

[0072] Figure 5 This is a schematic diagram of the reward time distribution provided in an embodiment of this application;

[0073] Figure 6 A flowchart illustrating a human-computer interaction method provided in an embodiment of this application;

[0074] Figure 7 A schematic diagram illustrating a strategy generation and clarification process provided in an embodiment of this application;

[0075] Figure 8A A schematic diagram of the strategy generation process for a general platform;

[0076] Figure 8B A schematic diagram illustrating the strategy generation and clarification process provided in an embodiment of this application;

[0077] Figure 9 An architectural diagram of a human-computer interaction device provided in an embodiment of this application;

[0078] Figure 10 An architecture diagram of a computing device provided in an embodiment of this application;

[0079] Figure 11 An architectural diagram of an electronic device provided in this application embodiment;

[0080] Figure 12 This is an architecture diagram of a human-computer interaction system provided in an embodiment of this application.

[0081] It should be understood that the dimensions and shapes of the blocks in the above structural diagrams are for reference only and should not constitute an exclusive interpretation of the embodiments of the present invention. The relative positions and inclusion relationships between the blocks presented in the structural diagrams are only schematic representations of the structural relationships between the blocks, and are not intended to limit the physical connection methods of the embodiments of the present invention. Detailed Implementation

[0082] The technical solutions provided in this application will be further described below with reference to the accompanying drawings and embodiments. It should be understood that the system architecture and business scenarios provided in the embodiments of this application are mainly for illustrating possible implementations of the technical solutions of this application and should not be construed as the sole limitation on the technical solutions of this application. Those skilled in the art will recognize that the technical solutions provided in this application are equally applicable to similar technical problems as system architectures evolve and new business scenarios emerge.

[0083] It should be understood that the human-computer interaction solutions provided in the embodiments of this application include human-computer interaction methods and devices, equipment, and vehicles. Since these technical solutions solve problems based on the same or similar principles, some repetitive details may not be repeated in the following descriptions of specific embodiments. However, it should be considered that these specific embodiments have references to each other and can be combined with each other.

[0084] This application provides a human-computer interaction method, apparatus, device, and vehicle. It can provide strategic actions that conform to the current scenario and user preferences based on the user's expression habits, thereby meeting the user's expectations for human-computer interaction and improving the user's experience. This application can be applied to various human-computer interaction technical fields and can be adapted to various spatial scenarios, such as intelligent driving scenarios and smart home scenarios. The device (i.e., the "machine" in human-computer interaction) that interacts with the user (i.e., the "person" in human-computer interaction) can be a vehicle, robot, smart home device, etc. The following is a detailed description of this application.

[0085] The following is a reference first. Figure 2 and Figure 3 This embodiment provides a brief overview of the application scenarios involved. For example... Figure 2 As shown, the application scenario of this embodiment specifically involves a vehicle 100, which can be a passenger car, a truck, or a special vehicle such as an ambulance, fire truck, police car, or emergency rescue vehicle. In the human-computer interaction method of this embodiment, the interaction objects can be a user and a vehicle. The user can be the driver or a passenger of the vehicle 100. The user can express their experience intentions through voice, gestures, facial expressions, or other collectable means. The vehicle 100 can collect the information expressed by the user to identify the user's experience intentions and provide corresponding strategy actions based on those intentions. The vehicle 100 can be equipped with relevant devices supporting the human-computer interaction method of this embodiment, either externally or inside the vehicle cabin. Then, based on the collected information expressed by the driver or passenger, the vehicle generates, recommends, and executes strategy actions to satisfy the user's experience intentions.

[0086] like Figure 3 As shown, the vehicle 100 may be equipped with a multi-dimensional sensing unit 110, an intent recognition unit 120, a device management unit 130, a state processing unit 140, a system strategy unit 150, and an action execution unit 160 for interactive purposes. Specifically,

[0087] The multi-dimensional sensing unit 110 is mainly used to collect information such as the user's voice, gestures, facial expressions, or body movements. This multi-dimensional sensing unit 110 can be implemented as an integrated unit, such as an integrated unit including a microphone module 111, a camera module 112, and a human body sensor module 113, to collect information such as the user's voice, gestures, facial expressions, or body movements. Alternatively, it can be implemented using multiple independent units or modules, such as one or more separately configured microphone modules 111, camera modules 112, and human body sensor modules 113. The microphone module 111 can be installed inside the vehicle cabin to detect the user's voice information; there can be one or more. The camera module 112 can be installed on the A-pillar, B-pillar, or the side of the rearview mirror facing the user in the vehicle cabin; it can also be installed near the steering wheel, center console, or above the display screen behind the seat, etc. It can be used to collect the user's gestures, facial expressions, or body movements, and can also be used for facial recognition for user authentication. The human body sensor module 113 can be used to detect the user's human vital signs, such as body temperature and heart rate. Correspondingly, the human body sensor module 113 can be a temperature sensor, heart rate sensor, or other sensor device that can detect body temperature and heart rate.

[0088] In this embodiment, the microphone module 111, camera module 112, and human body sensor module 113 can be pre-installed in the vehicle cabin or installed retrofitted into the vehicle cabin. The multi-dimensional perception unit 110 collects the user's voice data, gestures, facial expressions, or physical characteristics through the microphone module 111, camera module 112, and human body sensor module 113. It obtains semantic features by transcribing the voice data collected by the microphone module 111 through speech recognition, obtains gesture sequence features by transcribing the user's gestures collected by the camera module 112 through gesture recognition, obtains emotional features by transcribing the user's facial expressions collected by the camera module 112 through facial expression recognition, and obtains human physical characteristics by transcribing the human physical characteristics collected by the human body sensor module 113 through heart rate signal processing. The semantic features, gesture sequence features, emotional features, and human physical characteristics are then combined to generate a human signal bundle.

[0089] The intent recognition unit 120 can be an electronic device, specifically an in-vehicle processing device such as a vehicle infotainment system or in-vehicle computer, a chip or processor within the vehicle infotainment system or in-vehicle computer, a conventional chip processor such as a central processing unit (CPU) or microcontroller (MCU), or a terminal device such as a mobile phone or tablet. The intent recognition unit 120 can have a preset intent recognition model, or it can acquire a preset intent recognition model from other devices within the vehicle. It can recognize the received human signal bundle containing semantic features, gesture sequence features, emotional features, and human physical characteristics to identify the user's experience intent, which can specifically be the experience the user expects to achieve within the vehicle. In this embodiment, during intent recognition, some features can be designated as core features and others as auxiliary features based on their weights. For example, the user's semantic features can be designated as core features with higher weights, while other gesture sequence features, emotional features, and human physical characteristics can be designated as auxiliary features with lower weights. The user's experience intent is determined based on each feature and its weight. In this embodiment, the intent recognition model can be a mainstream model in the field of machine translation. By using this intent recognition model to encode and decode the input human body signal bundle, the user's experience intent corresponding to the human body signal bundle can be translated. Specifically, the user's experience intent can include the user's subjective feelings and objective intent. Subjective feelings mainly refer to the user's description of their own feelings, such as "I am hot" or "I am cold". Objective intent mainly refers to the user's operational intent on the device or service that is equivalent to the user's subjective feelings. For example, the service operation intent equivalent to "I am hot" is "lower the perceived temperature", and the equivalent device operation intent is "turn on the air conditioner", "lower the air conditioner temperature", or "turn on the seat fan". Or, the service operation intent equivalent to "I am cold" is "increase the perceived temperature", and the equivalent device operation intent is "turn off the air conditioner", "increase the air conditioner temperature", or "turn off the seat fan".

[0090] The device management unit 130 can be various sensors or controllers installed on the vehicle, mainly used to detect the vehicle's equipment status and environmental status. Specifically, it can include an air conditioning controller 131, seat controller 132, speaker controller 133, display controller 134, lighting controller 135, sunroof controller 136, window controller 137, curtain controller 138, and an environmental sensor module 139, etc. Each controller can be installed in the central control area of ​​the vehicle's cabin, or it can be installed at the device it is connected to, to perform control and status detection of its corresponding device. The environmental sensor module 139 can be a light sensor, temperature sensor, humidity sensor, air quality meter, etc., to detect the environmental status inside and outside the vehicle. The device management unit 130 collects the vehicle's equipment status data and environmental status data, and organizes them to generate a cabin signal bundle.

[0091] The state processing unit 140 can be a chip or processor in the vehicle's infotainment system or onboard computer, or a conventional chip processor such as a CPU or MCU, or a software module with processing capabilities on the chip or processor. It is used to store the device state data and environmental state data in the cockpit signal beam as corresponding feature parameters, and maintain and generate the vehicle's state pool. Specifically, for example... Figure 4 As shown, each device, when installed on the vehicle, needs to be registered with relevant software definitions. For example, this registration may include registering the vehicle's device status data, generating device status information, and registering the corresponding actions and action-related parameters required to change the device's status, thus generating device action characteristics. Simultaneously, the vehicle's environmental status data also needs to be registered to generate environmental status information.

[0092] The system strategy unit 150 can also be an electronic device, specifically an in-vehicle processing device such as a vehicle infotainment system or in-vehicle computer, a chip or processor in the vehicle infotainment system or in-vehicle computer, a conventional chip processor such as a CPU or MCU, or a terminal device such as a mobile phone or tablet. The system strategy unit 150 can determine the strategy actions for controlling the vehicle based on the user's experience intent, the vehicle's device status information, and environmental status information. In this embodiment, the system strategy unit 150 specifically includes a strategy knowledge base 151, a strategy sorting module 152, and a strategy clarification module 153. The strategy knowledge base 151, strategy sorting module 152, and strategy clarification module 153 can be specific hardware modules in the system strategy unit 150, or software modules with processing capabilities within the system strategy unit 150.

[0093] The strategy knowledge base 151 is used to generate recommended strategy sequences based on the user's experience intent, vehicle device status information, and environmental status information. These strategy sequences include different strategy actions for different controls of the vehicle's infotainment system and their initial scores. The user's experience intent is E. m The vehicle's current state is s (including the current device state S). dev and environmental state S env User experience intent E m The desired vehicle state is s′ (including the desired equipment state S). dev and environmental state S env Then, a trained Markov Decision Process (MDP) model can be used to generate policy actions. i To perform an initial score, specifically, the action to execute this strategy can be... i / a The state transition probability of the vehicle from its current state s to its destination state s′ when controlling the vehicle, which is the action of this policy. i initial rating For example, the state transition probability relationship in this MDP model can be:

[0094]

[0095]

[0096] This state transition probability relationship is mainly a reflection of the current state s and the action Action. i The vehicle's target state s′ is a function of the vehicle's current state s and the policy action Action. Based on this state transition probability relationship, it can be seen that the vehicle's target state s′ is only related to the vehicle's current state s and the policy action Action. i Regarding this, r is the reward element for this state transition probability relationship. Based on the obtained state transition probability P(s′|s,Action) i This allows us to determine the strategy action (Action). i initial rating

[0097] In this embodiment, the training process of the above MDP model mainly requires obtaining information about the reward r(s, Action). i The distribution of ) is used to train the MDP model. Specifically, the reward r(s, Action) can be modeled and fitted in the following way. i Distribution of )

[0098] The user's experience intent is defined based on the user's expressed information. In this embodiment, the user's expressed information may include physiological characteristics and psychological characteristics. Physiological characteristics mainly refer to the user's physiological perception of the external environment. When the in-vehicle environment exceeds the user's comfort range, the user will expect to perform appropriate strategic actions to adjust the in-vehicle environment to achieve their experience intent. For example, as shown in the table below, common physiological characteristic types include sound comfort, light comfort, temperature comfort, and air comfort. The human organs used to perceive the environment are the ears, eyes, skin, and nose. The user perceives the in-vehicle environment based on these organs. For example, based on the ears' perception of sound, the description is "noisy" or "quiet"; based on the eyes' perception of light, the description is "bright" or "dark"; based on the skin's perception of temperature, the description is "hot" or "cold"; and based on the nose's perception of odor, the description is "turbid" or "clear".

[0099] Physiological characteristic types organ Expression description type Sound comfort Ear Noisy or quiet Lighting comfort Head Bright, dark Temperature comfort skin Hot, cold Air comfort nose Turbid, clear

[0100] Psychological characteristics mainly refer to the user's psychological state, which is usually the user's subjective experience. For example, as shown in the table below, common psychological characteristic types include emotions such as happiness, anger, sadness, fear, fatigue, and tension. The corresponding experience descriptions include "annoyed" and "calm", "angry" and "furious", "bitter" and "painful", "fearful" and "terrified", "tired" and "weak", and "excited".

[0101] Psychological characteristic types Expression description type happy Annoyance, tranquility angry anger sad pain Fear Fear, terror exhausted Tired, weak nervous Excited …… ……

[0102] Based on the aforementioned physiological and psychological characteristics of users, corresponding strongly related strategy actions can be defined. For example, when a user expresses "I'm so hot" via voice, the corresponding expression description is "hot," and the corresponding physiological characteristic type is "temperature comfort." In this case, the strategy action "lower the air conditioning temperature" can be initially considered as the equivalent experiential intention of "I'm so hot." Then, adjacent strategy actions such as "turn on the seat fan" and "increase the air conditioning fan speed" can all be considered as related strategy actions of the user's equivalent experiential intention of "I'm so hot." Based on these related strategy actions, the relevance of each strategy action to the experiential intention can be scored. For example, when the strategy action Action... i The closer the time is to "lowering the air conditioner temperature", the more likely the strategy action can be considered to be active. i The higher the correlation score with "temperature comfort," the more likely the in-vehicle environment state after a period of time can be considered as the target state s′ corresponding to the user's experiential intent. Specifically, for example... Figure 5 As shown, the strategy action is... i The reward r is negatively exponentially correlated with the distribution of the time interval Δt, i.e.

[0103]

[0104] As shown in the formula above, the longer the time interval, the lower the reward *r* of the strategy action. This method allows us to obtain the correlation between each of the aforementioned strategy actions and the user's experience intent. By obtaining the correlation between the reward and experience intent for each strategy action, the MDP model can be trained, thereby quickly filtering out multiple strategy actions related to the user's experience intent when acquiring that intent.

[0105] Based on the selected multiple strategy actions, the current experience intent E can be obtained. m Under the following conditions, the selected strategy action a is adopted to make the current state s (the current device state S) change. dev and environmental state S env Adjust to the target state s′ (the desired device state S) dev and environmental state S env The transition probability of ′), i.e.

[0106]

[0107] The strategy ranking module 152 is used to weight and rank multiple generated strategy actions based on the user's strategy preference information. The user's strategy preference information can be reflected by the frequency of the user's use of strategy actions. To smooth the influence of the user's strategy preference information and initial scores on the final generated strategy actions, weights can be set to perform a weighted calculation on the initial score of each strategy action. The calculation method is as follows:

[0108]

[0109] Where w1 represents the user's policy preference information. u_i The weights, w2, are the generated policy actions (Action) from the policy knowledge base 151. i initial rating (i.e., through this strategy action) i The weight of the transition probability from the current state s to the target state s′ can be used to weight multiple policy actions in the generated policy sequence. The policy actions can be sorted according to the weighted scores, and one or more policy actions at the top of the sort can be considered as the policy actions that best match the current experience intent, device state and environment state.

[0110] The policy clarification module 153 can be used to clarify one or more policy actions in the ranking head to select the final policy action, thereby updating the scoring model and recording policy preference information. For example, it can be used to clarify the two policy actions in the ranking head. i and Action j The weighted scores are used to calculate the difference.

[0111] Δ=|Score i -Score j |(Score i >Score j )

[0112] Furthermore, based on the calculated difference Δ and the set threshold δ, the system selects the highest-scoring strategy action or initiates a clarification. This threshold δ can be manually set or modified according to the user's preference for the strategy clarification action. For example, if the user prefers the selection of the strategy action in the human-computer interaction process to be determined personally, the threshold δ can be set to a larger value. When the difference Δ obtained by calculating the weighted scores of two strategy actions is less than the threshold δ, both strategy actions can be provided to the user for clarification. Conversely, if the user prefers the selection of the strategy action in the human-computer interaction process to be automated, the threshold δ can be set to a smaller value. When the difference Δ obtained by calculating the weighted scores of two strategy actions is greater than the threshold δ, the highest-scoring strategy action can be automatically selected as the final strategy action.

[0113]

[0114] When the difference Δ is greater than the threshold δ, there is no need to initiate clarification; the highest-scoring strategy action can be selected directly. i As the final policy action, when the difference Δ is less than the threshold δ, a clarification needs to be initiated, and the two policy actions will be changed. i and Action j Provide clarification to the user. Select the final strategy action based on the user's clarification, and update the user's strategy preference information accordingly.

[0115] The difference calculation is only one implementation method in this embodiment. In some embodiments, when clarifying the strategy for multiple strategy actions in the sorting head, variance calculation can also be used, and the strategy clarification can be performed based on the variance calculation result to select the final strategy action.

[0116] In some embodiments, the strategy clarification module 153 can also directly select multiple strategy actions at the beginning of the strategy sequence and combine them to generate the final strategy action. Specifically, a set score can be used as a threshold when selecting strategy actions. This set score can be set manually or determined based on the weighted scores of strategy actions frequently selected by the user. For example, the weighted scores of strategy actions frequently selected by the user can be used as the set score when filtering strategy actions. By filtering multiple strategy actions in the strategy sequence whose weighted scores exceed the set score, multiple strategy actions with the same or similar execution effects can be combined to generate the final strategy action, thereby achieving better strategy recommendation and execution effects.

[0117] In some embodiments, when the final strategy action is selected based on the user's clarification result, the correlation between any two or more of the information expressed by the user, the user's experience intent, the recommended multiple strategy actions, and the user's strategy preference information can be updated based on the strategy action selected by the user. This updates the strategy recommendation model in the strategy knowledge base 151, the strategy ranking model in the strategy ranking module 152, and forms a corpus pair between the information expressed by the user input to the intent recognition unit 120 and the final strategy action, thereby training and optimizing the intent recognition algorithm or intent recognition model in the intent recognition unit 120.

[0118] The action execution unit 160 can be an Electronic Control Unit (ECU) in the vehicle, or a component of the ECU, such as an interface circuit or actuator. It is used to execute corresponding policy actions to control the vehicle based on the clarification status bit of the policy clarification module 153. When the clarification status bit is "request", it queries the user for the desired action. i and Action j Based on user preferences and feedback, the system executes the final policy action. When the clarification status bit is "execute," the corresponding policy action is executed directly, or a policy action generated by combining the above actions is executed. The action execution unit 160 can also update the vehicle's equipment and environmental status in real time based on the policy action execution information until the user's experience intent is met.

[0119] The multi-dimensional sensing unit 110, intent recognition unit 120, device management unit 130, state processing unit 140, system strategy unit 150, and action execution unit 160 can communicate data or commands via wired communication (such as interface circuits) or wireless communication (such as Bluetooth or Wi-Fi). Through this structure, the vehicle 100 in this embodiment can collect information expressed by the user, perform intent recognition according to a preset intent recognition model or algorithm to obtain the user experience intent corresponding to the user's expressed information, generate multiple recommended strategy actions based on the identified user experience intent and vehicle state information, adjust the scores of the recommended multiple strategy actions according to the user's strategy preference information, and determine the strategy action based on the adjusted score to control the vehicle system. The vehicle 100 in this embodiment is suitable for different user expression habits and can dynamically recommend appropriate strategy actions to the user, thereby meeting the user's expectations for human-computer interaction and improving the user's experience evaluation.

[0120] Figure 6 A flowchart illustrating a human-computer interaction method provided in an embodiment of this application is shown. This method can be executed by a human-computer interaction device or some components within that device, such as a vehicle, a vehicle interaction system, or a processor. The processor can be the processor of the human-computer interaction device, or it can be the processor of an in-vehicle processing device such as a vehicle infotainment system or an in-vehicle computer. Specifically, it involves acquiring information expressed by the user, performing intent recognition based on a preset intent recognition model or algorithm to obtain the user's experience intent, generating multiple recommended strategy actions based on the user's experience intent and the vehicle infotainment system's state information, and determining the final strategy action based on the user's strategy preference information to control the vehicle infotainment system. Figure 6 As shown, the human-computer interaction method includes:

[0121] S610: Obtain information expressed by the user, and obtain the user's experience intent based on the information expressed by the user.

[0122] In this embodiment, the information expressed by the user can be obtained through the above... Figure 3The system utilizes a microphone module 111, a camera module 112, and a human body sensor module 113 to collect data. The microphone module 111 detects the user's speech and transcribes the speech data collected by the microphone module 111 using speech recognition to obtain semantic features. The camera module 112 collects the user's gestures, facial expressions, and body movements. Gesture recognition transcribes the user's gestures collected by the camera module 112 to obtain gesture sequence features, and facial expression recognition transcribes the user's facial expressions collected by the camera module 112 to obtain emotional features. The human body sensor module 113 detects the user's vital signs, such as body temperature and heart rate. Heart rate signal processing transcribes the vital signs collected by the human body sensor module 113 to obtain vital sign features. These semantic features, gesture sequence features, emotional features, and vital sign features are then processed to generate the user's expressed information. The intent recognition unit 120 uses an intent recognition model or algorithm to identify the user's intended experience within the vehicle. Specifically, the user's intended experience can be the experience the user desires within the vehicle.

[0123] S620: Based on the vehicle system's status information and the user's experience intent, obtain multiple recommended strategy actions.

[0124] The vehicle's status information specifically includes the device status information and environmental status information. By inputting the user's experience intent, device status information, and environmental status information into the policy knowledge base 151, a recommended policy sequence can be generated. This policy sequence includes multiple policy actions and their initial scores. The initial score for each policy action can be the probability that, when the policy action is used to control the vehicle, the current device status and environmental status will be adjusted to the target status corresponding to the user's experience intent.

[0125] S630: Adjust the scores of the multiple policy actions based on the user's policy preference information.

[0126] User strategy preference information can be reflected by the frequency of user use of strategy actions. To smooth the impact of user strategy preference information and the initial scores of each strategy action on the generated final strategy action, the initial scores of each strategy action can be weighted by setting weights, and multiple strategy actions can be sorted according to the calculated weighted scores. One or more strategy actions at the top of the sort can be considered as the strategy actions that best match the current user's experience intent, device status and environment status.

[0127] S640: Determine the strategy action for controlling the vehicle system based on the adjusted score, and execute the control of the vehicle system.

[0128] Based on the adjusted scores, the control strategy for the vehicle system can be determined from a ranking of multiple strategy actions. Specifically, the weighted scores of the top-ranked strategy actions can be used to determine whether to initiate user clarification. For example, the weighted scores of at least two top-ranked strategy actions can be compared to calculate the difference. Based on the difference and a set threshold, the strategy action with the highest score can be selected to control the vehicle system, or a clarification process can be initiated. When the difference is greater than the threshold, no clarification is needed, and the strategy action with the highest score can be directly selected as the final strategy action to control the vehicle system. When the difference is less than the threshold, clarification is required. The two strategy actions are presented to the user for selection and clarification. The user's selection information is received, and the strategy action that matches the user's preference is determined to control the vehicle system.

[0129] In some embodiments, the correlation between any two or more of the user's expressed information, user's experience intent, recommended multiple strategy actions, and user's strategy preference information can be updated according to the strategy action selected by the user. This can optimize and train the strategy recommendation model in the strategy knowledge base 151 and the strategy ranking model in the strategy ranking module 152. Additionally, the user's expressed information input to the intent recognition unit 120 can be combined with the final strategy action to form a corpus pair, thereby training and optimizing the intent recognition algorithm or intent recognition model in the intent recognition unit 120.

[0130] In some embodiments, multiple policy actions at the beginning of the policy sequence can be directly selected and combined to generate the final policy action. For example, a set score can be used as a threshold when selecting policy actions. By filtering multiple policy actions in the policy sequence whose weighted scores exceed the set score, multiple policy actions with the same or similar execution effects can be combined to generate the final policy action, thereby achieving better policy recommendation and execution results.

[0131] In some embodiments, before performing step S610, this method can also obtain the user's identity information through the microphone module 111 or the camera module 112, and obtain the pre-stored user intent preferences and user strategy preferences based on the user's identity information, so as to use the user intent preferences to identify the user's experience intent, and use the user's strategy preferences to adjust the scores of multiple generated strategy actions, so that the final strategy actions are more in line with the user's usage habits.

[0132] Figure 7 This illustration shows a schematic diagram of a strategy generation and clarification process provided in an embodiment of this application, with reference to... Figure 7 This section provides a systematic explanation of the specific execution process of the aforementioned human-computer interaction methods. For example... Figure 7 As shown, the strategy generation and clarification process includes:

[0133] Based on the user's expressed information, the intent recognition unit identifies the user's intent. For example, when the user's first expression is "I'm so hot," accompanied by a "fanning" gesture, and also by physical signs such as sweating, reddening of the skin, increased heart rate, and increased body temperature, the intent recognition unit identifies the user's experiential intent through a pre-stored intent recognition model, such as the need to lower the perceived temperature or to be soothed.

[0134] Based on the vehicle's current device and environmental states, as well as the user's intended experience, a recommended strategy sequence is generated using a strategy knowledge base. This strategy sequence includes multiple strategy actions and initial scores for each action. Specifically, the strategy sequence may include different strategy actions that control the vehicle's infotainment system, such as reducing seat heating, increasing seat fan speed, enhancing air conditioning cooling, increasing air conditioning fan speed, opening and closing windows, playing quiet music, and adjusting ambient lighting. The initial score for each strategy action is generated based on its relevance to the user's intended experience. For example, the initial scores for each strategy action in this strategy sequence are as follows:

[0135] Seat heating reduced by 0.01, seat airflow increased by 0.94, air conditioning cooling strengthened by 0.95, air conditioning airflow increased by 0.01, windows opened by 0.93, windows closed by 0.01, playing quiet music by 0.23, ambient lighting cool light by 0.01.

[0136] Based on user strategy preference information and the initial score of each strategy action, the strategy ranking module performs a weighted calculation on each strategy action in the strategy sequence generated by the strategy knowledge base, and then ranks them according to the weighted scores. In this weighted calculation, the user's strategy preference information can specifically be the frequency of the user's use of a strategy action; the higher the frequency of use of a strategy action, the higher the user's preference for that strategy action. By assigning weights to the user's strategy preference information and initial score for each strategy action, and recalculating the initial score of the strategy action according to the set weights, a weighted score for each strategy action is generated. Based on the weighted scores, multiple strategy actions in the strategy sequence can be ranked. For example, the weighted scores and the ranked strategy actions are as follows:

[0137] Seat air conditioning 0.97, window opening 0.93, air conditioning cooling enhanced 0.93, playing quiet music 0.23, seat heating reduced 0.01, air conditioning airflow increased 0.01, window closed 0.01, ambient light cool light 0.01.

[0138] Based on the weighted scores of at least two policy sequences at the top of the ranking, policy clarification can be initiated through the policy clarification module. In this embodiment, when selecting at least two policy actions at the top of the ranking, it is usually to select two policy actions with the same or similar control effects at the top of the policy sequence. For example, "seat blowing" and "air conditioning cooling enhancement" at the top of the ranking can be selected for user clarification. By calculating the difference between the weighted scores of "seat blowing" and "air conditioning cooling enhancement" (0.97-0.93=0.04), and based on the result of the difference calculation (0.04) and the size of the set threshold, the policy action with the highest score can be directly selected or clarification can be initiated for both policy actions. For example, when the set threshold is 0.03, the difference (0.04) is greater than the threshold (0.03), so no clarification is needed, and the "seat air conditioning" with the highest weighted score between the two strategy actions can be directly selected as the final strategy action; or, when the set threshold is 0.05, the difference (0.04) is less than the threshold (0.05), so clarification needs to be initiated, and the two strategy actions "seat air conditioning" and "enhanced air conditioning cooling" are provided to the user for clarification. The final strategy action is selected based on the user's clarification result, for example, by asking the user "Do you want to lower the air conditioning temperature or turn on the seat air conditioning?", and based on the user's choice (e.g., the user chooses to lower the air conditioning temperature), the final strategy action "enhanced air conditioning cooling" is selected to control the vehicle system.

[0139] In some embodiments, the user's strategy preference information and the initial score of the strategy action "enhanced air conditioning cooling" in the strategy knowledge base can be updated based on the user's selected strategy action "enhanced air conditioning cooling". At the same time, the information expressed by the user and the user's selected strategy action "enhanced air conditioning cooling" can be used to form a sample to train the intent recognition unit, so as to improve the intent recognition, strategy recommendation and the relevance between the user's strategy preference information and the user's expression, making the generated strategy action more accurate.

[0140] In some embodiments, the final policy action can also be executed by the action execution unit, and the device state and environment state of the vehicle system can be updated according to the execution information. Since the purpose of executing the final policy action is to adjust the current device state or environment state to the target state corresponding to the user's experience intent, when the final policy action can no longer be executed or the execution fails, this embodiment can use the previously generated final policy action as the session history, and generate an alternative policy action based on the information expressed by the user in the second round. The alternative policy action has the same or similar execution effect as the final policy action. For example, when the air conditioning temperature has been adjusted to the lowest level according to the policy action "enhanced air conditioning cooling" selected by the user, but the user still expresses "I'm so hot" with a "fanning" gesture, "adjusted the air conditioning temperature to the lowest level" can be used as the session history. Based on the second round of information expressed by the user, a second round of intent recognition, policy recommendation, and policy clarification can be performed to select an alternative policy action. At this time, the alternative policy action has the same or similar control effect as the policy action in the first round. For example, the policy action selected in the second round can be "seat air conditioning", thereby realizing full-scene policy recommendation centered on user experience.

[0141] The following is based on Figures 8A-8B The policy generation process of the general platform is compared and described with the policy generation and clarification process of the embodiments of this application to demonstrate the beneficial effects of the embodiments of this application.

[0142] like Figure 8AThis is a schematic diagram of a strategy generation process for a general platform. In this platform, users need to express themselves according to a pre-configured dialogue strategy, and the content of their expression must be accurate in order to recognize the user's intention. This intention recognition can identify the user's objective intention and subjective feelings. Objective intention is the main interaction method of this general platform, while subjective feelings are an auxiliary method. Specifically, the objective intent can be the user's expressed intention to operate the vehicle's equipment. For example, if the user's semantics are "turn on the air conditioning," the corresponding matching strategy is to turn on the air conditioning. However, the user needs to express this command accurately; otherwise, it may result in the inability to recognize the intent or the failure of the matching strategy. Subjective feelings can be the user's description of their own feelings. For example, if the user's semantics are the pre-configured "feeling hot," and the equivalent objective intent corresponding to "feeling hot" is "turn on the air conditioning," then the corresponding matching strategy is to turn on the air conditioning. If the user's semantics do not have a pre-configured equivalent objective intent, it may result in the inability to recognize the intent or the failure of the matching strategy. For the generated strategy action, a strategy check can be performed to verify whether the generated strategy action is invalid. If it is not invalid, the generated strategy action can be executed. If it is invalid, the configured fallback strategy will be triggered, indicating that the generated strategy action cannot be executed and no other alternative strategy action can be recommended, resulting in the current interaction process failing to meet the user's experience intent.

[0143] According to this Figure 8A As shown, in the strategy generation scenario of a general platform, user intent and strategy action need to be configured in advance. This configuration is usually statically bound, and one user intent can usually only correspond to one strategy action, which is very easy to miss other feasible strategy actions and lacks full-scenario strategy recommendation centered on user experience. At the same time, users must also learn specific expressions to trigger specific pre-configured scenarios. When the user's expression is incorrect or too personalized, it will directly lead to the inability to trigger the specific pre-configured scenario, which directly reduces the user experience.

[0144] Compared to the strategy generation process of general platforms, such as Figure 8BThis is a schematic diagram of the strategy generation and clarification process according to an embodiment of this application. In this embodiment, there is no need for specific pre-configuration of user intent and strategy actions. It is only necessary to register the user's usual expression characteristics and the status information of related devices. Then, through an intent recognition model or intent recognition algorithm, the user's expressed information is identified, thereby recognizing the user's subjective feelings and objective intentions. Since this embodiment of the application adopts a more intelligent intent recognition model or intent recognition algorithm, users can directly describe their feelings or needs according to their own expression habits. It can also quickly identify the user's experience intent. The user's experience intent and the obtained device status and environmental status characteristics are input into the strategy knowledge base for strategy generation and strategy recommendation. This generates multiple strategy actions related to the user's experience intent, and each strategy action is scored and ranked. When there are multiple strategy actions to choose from, strategy clarification can also be performed, providing multiple strategy actions to the user for selection, and finally executing the final strategy action.

[0145] According to this Figure 8B As shown, this embodiment of the application can realize strategy recommendation based on the user's subjective feelings and objective intentions, and does not specifically limit the user's expression. It has a more intelligent intention recognition model or intention recognition algorithm. Users can directly describe their own feelings or needs without needing to understand device operation knowledge, and the vehicle system can provide appropriate feedback, lowering the user experience threshold. At the same time, it can also dynamically adjust the generated strategy actions according to changes in device status and environmental status. For failed strategy actions, the score of the failed strategy action can be reduced, and strategy actions with the same experience can be recommended in a timely manner, thereby improving the user's experience evaluation. In addition, this embodiment can also dynamically update the user's strategy preference information and dynamically update the matching relationship between user expression and strategy actions, adapting to user expression habits and realizing dynamic recognition of user habits.

[0146] As a modified embodiment of the human-computer interaction method provided in this application, another embodiment of this application relates to a method for interaction between a user and a smart home device.

[0147] With the increasing popularity of smart homes, many smart home devices offer human-computer interaction functions and execute relevant control measures based on strategies generated by human-computer interaction. For example, users can control smart home devices through voice, gestures, facial expressions, and body movements. Therefore, this embodiment provides a method for effective human-computer interaction between users and smart home devices.

[0148] Many existing smart home devices are equipped with microphone modules, camera modules, and human body sensor modules that can interact with users. For example, the microphone module of a smart speaker can be used to acquire the user's voice information and can also serve as a control medium to connect with other smart home devices via local area network or Bluetooth. By recognizing the intent of the acquired voice information and generating policy actions to send to the corresponding smart home devices, it is possible to control other smart home devices. For example, controlling devices such as air conditioners, televisions, water heaters, and curtain motors based on user commands. Alternatively, home cameras can be used to acquire the user's gestures. Image information such as facial expressions and body movements can be used to generate policy actions by performing intent recognition on the acquired image information and sending them to the corresponding smart home devices. This enables the control of other smart home devices. For example, based on the user's expression of "fanning," or based on the human body sensor's detection of the user's elevated body temperature and facial sweating, the system can identify the user's subjective feeling of being hot and needing to lower the perceived temperature, as well as the pre-set objective intent to control the smart home devices equivalent to this subjective feeling. This can generate policy actions such as turning on the air conditioner and closing the curtains, and send them to the corresponding air conditioner and curtain motors for execution to meet the user's experience intent.

[0149] In the description of the above embodiments, vehicles and smart home devices were used as examples of target devices. However, the human-computer interaction technology of this application can also be applied to other scenarios where interactive control is performed through voice, gestures, body movements, etc., or in scenarios that require identity authentication.

[0150] In summary, the human-computer interaction method provided in this application can achieve strategy recommendation based on the user's subjective feelings and objective intentions. It can also dynamically adjust the generated strategy actions according to changes in device and environmental states. Furthermore, when multiple suitable strategy actions are available, user clarification can be initiated, and the optimal strategy action that matches the current valid experience intention can be generated based on the user's selection, or multiple strategy actions that match the user's valid experience intention can be combined. The strategy knowledge base involved in this application can be designed according to the user's experience intentions, ensuring that the generated strategy actions match the user's experience intentions. Based on this strategy knowledge base, users do not need to understand complex device operation knowledge; they only need to intuitively describe their feelings or experience intentions to achieve intent recognition and generate appropriate strategy actions, thus lowering the user's usage threshold. Moreover, this application employs machine learning algorithms, which can learn and optimize based on the user's expression habits and the results of strategy clarification, dynamically updating the user's strategy preference information, improving the hit rate of generated strategy actions, and reducing unnecessary user interaction rounds.

[0151] Figure 9This is an architectural diagram of a human-computer interaction device according to an embodiment of this application. This human-computer interaction device can be used to implement various optional embodiments of the above-described human-computer interaction methods. The human-computer interaction device can be an in-vehicle device or terminal, such as a vehicle, an in-vehicle infotainment system, or an in-vehicle computer, or a terminal device such as a mobile phone or tablet. The human-computer interaction device can also be a chip or chip system inside the terminal, such as a chip or processor in an in-vehicle infotainment system or a mobile phone or tablet, or a conventional chip processor such as a CPU or MCU. The human-computer interaction device can also be a hardware and software system inside the terminal, such as a human-computer interaction interface in the vehicle cabin, an in-vehicle infotainment system, or other systems with both software and hardware. Figure 9 As shown, the human-computer interaction device has a recognition module 910, a recommendation module 920, an adjustment module 930, and an output module 940.

[0152] The recognition module 910 is used to execute step S610 of the above-described human-computer interaction method and examples thereof. For example, the recognition module 910 may have a preset intent recognition model, or may obtain a preset intent recognition model from other devices in the vehicle, and use the intent recognition module to recognize the information expressed by the user to obtain the user's experience intent. The recommendation module 920 is used to execute step S620 of the above-described human-computer interaction method and examples thereof. For example, the recommendation module 920 may use a strategy knowledge base to obtain multiple recommended strategy actions based on the vehicle system's status information and the user's experience intent. The adjustment module 930 is used to execute step S630 of the above-described human-computer interaction method and examples thereof. For example, the adjustment module 930 may be used to adjust the ratings of multiple strategy actions based on the user's strategy preference information. The output module 940 is used to execute step S640 of the above-described human-computer interaction method and examples thereof. For example, the output module 940 may determine the strategy action to control the vehicle system based on the adjusted rating and execute control of the vehicle system. For details, please refer to the detailed description in the method embodiments, which will not be repeated here.

[0153] In some embodiments, the human-computer interaction device may further include a training module 950, which can implement the training optimization part of the above-described human-computer interaction method. For example, it can update the correlation between any two or more of the information expressed by the user, the user's experience intent, the recommended multiple strategy actions, and the user's strategy preference information based on the user's selection information, thereby achieving optimized training of the intent recognition algorithm or intent recognition model, strategy recommendation model, strategy ranking model, and other algorithms or models, making them more in line with the user's personal preferences and improving the accuracy of human-computer interaction.

[0154] In some embodiments, the human-computer interaction device may also have an identity authentication module 960, which can implement the identity authentication part of the above-described human-computer interaction method. For example, the identity authentication module 960 can obtain the user's identity information and obtain the user's strategy preference information or intent preference information based on the user's identity information, so that the human-computer interaction device can better meet the user's customized needs.

[0155] The human-computer interaction device provided in this embodiment can collect information expressed by the user, perform intent recognition according to a preset intent recognition model or algorithm to obtain the user experience intent corresponding to the information expressed by the user, generate multiple recommended strategy actions based on the identified user experience intent and the vehicle system's status information, adjust the scores of the recommended multiple strategy actions according to the user's strategy preference information, and determine the strategy action based on the adjusted score to control the vehicle system. This human-computer interaction device is applicable to different user expression habits and different vehicle system states, and can dynamically recommend appropriate strategy actions to the user, thereby meeting the user's expectations for human-computer interaction and improving the user experience evaluation.

[0156] It should be understood that the human-computer interaction device in this embodiment can be implemented by software, for example, by a computer program or instruction with the above-mentioned functions. The corresponding computer program or instruction can be stored in the memory inside the terminal, and the above functions can be implemented by the processor reading the corresponding computer program or instruction in the memory. Alternatively, the human-computer interaction device in this embodiment can also be implemented by hardware. For example, the recognition module 910 can be implemented by an intent recognition unit on the vehicle, such as an in-vehicle processing device like a vehicle infotainment system or in-vehicle computer, or a chip or processor in the vehicle infotainment system or in-vehicle computer. The recommendation module 920, adjustment module 930, and output module 940 can be implemented by a system strategy unit on the vehicle. For example, the recommendation module 920 can be implemented by a strategy knowledge base in the system strategy unit, the adjustment module 930 can be implemented by a strategy sorting module in the system strategy unit, and the output module 940 can be implemented by a strategy clarification module in the system strategy unit. In this embodiment, the system strategy unit can specifically be an in-vehicle processing device like a vehicle infotainment system or in-vehicle computer, or a chip or processor in the vehicle infotainment system or in-vehicle computer, or a conventional chip processor such as a CPU or MCU. The training module 950 can be implemented by a system policy unit on the vehicle, for example, by one or more of the policy knowledge base, policy ranking module, and policy clarification module within the system policy unit, or by the interface circuit between the policy knowledge base, policy ranking module, and policy clarification module. The identity authentication module 960 can be implemented by a multi-dimensional perception unit on the vehicle, for example, by one or more of a camera module, microphone module, or human body sensor module, to authenticate the user's identity using facial recognition, voice recognition, or body feature recognition. Alternatively, the human-computer interaction device in this embodiment can also be implemented by a combination of a processor and software modules.

[0157] It should be understood that the processing details of the apparatus or module in the embodiments of this application can be found by referring to... Figures 2-8B The descriptions of the embodiments and related extended embodiments shown will not be repeated in this application.

[0158] Furthermore, this application embodiment also provides a vehicle equipped with the aforementioned human-machine interaction device. This vehicle can be a passenger car, a truck, or a special vehicle such as an ambulance, fire truck, police car, or emergency rescue vehicle. The modules and devices within the aforementioned human-machine interaction device can be pre-installed or retrofitted into the vehicle system. The modules can interact with each other via the vehicle's bus or interface circuits, or, with the development of wireless technology, they can also interact wirelessly to eliminate the inconvenience of wiring. In addition, the human-machine interaction device of this embodiment can also be combined with the vehicle's infotainment system, onboard computer, ECU, and other devices to be installed on the vehicle as an in-vehicle device, thereby achieving a better human-machine interaction effect.

[0159] Figure 10 This is an architectural diagram of a computing device 1000 provided in an embodiment of this application. This computing device can serve as a human-computer interaction device, executing various optional embodiments of the aforementioned human-computer interaction methods. The computing device can be an in-vehicle device or terminal, such as a vehicle, an in-vehicle infotainment system, or an in-vehicle computer, or a terminal device such as a mobile phone or tablet. The computing device can also be a chip or chip system within a terminal, such as a chip or processor in an in-vehicle infotainment system or computer, a chip or processor in a mobile phone or tablet, or a conventional chip processor such as a CPU or MCU. The computing device can also be a hardware and software system within a terminal, such as a human-computer interaction interface in a vehicle cabin, an in-vehicle infotainment system, or other systems with both software and hardware. Figure 10 As shown, the computing device 1000 includes a processor 1010 and a memory 1020.

[0160] It should be understood that Figure 10 The computing device 1000 shown may also include a communication interface 1030, which can be used to communicate with other devices, and may specifically include one or more transceiver circuits or interface circuits.

[0161] The processor 1010 can be connected to the memory 1020. The memory 1020 can be used to store the program code and data. Therefore, the memory 1020 can be a storage module inside the processor 1010, an external storage module independent of the processor 1010, or a component that includes both the storage module inside the processor 1010 and the external storage module independent of the processor 1010.

[0162] The computing device 1000 may also include a bus. The memory 1020 and communication interface 1030 can be connected to the processor 1010 via the bus. The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 10 The symbol is represented by only one line, but this does not mean that there is only one bus or one type of bus.

[0163] It should be understood that in the embodiments of this application, the processor 1010 may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor. Alternatively, the processor 1010 may employ one or more integrated circuits to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0164] The memory 1020 may include read-only memory and random access memory, and provides instructions and data to the processor 1010. A portion of the processor 1010 may also include non-volatile random access memory. For example, the processor 1010 may also store device type information.

[0165] When the computing device 1000 is running, the processor 1010 executes computer execution instructions stored in the memory 1020 to perform any operation step of the above-described human-computer interaction method and any optional embodiment thereof. For example, the processor 1010 can execute computer execution instructions stored in the memory 1020 to perform... Figure 6In the corresponding embodiment of the human-computer interaction method, when a user expresses themselves through voice, gestures, facial expressions, or other collectable means, the processor 1010 executes the acquisition instructions in the memory 1020 to control the vehicle's multi-dimensional perception unit to collect information such as the user's voice, gestures, facial expressions, or body movements, and then organizes and generates the information expressed by the user. The processor 1010 executes the recognition instructions in the memory 1020 to enable the vehicle's intent recognition unit to recognize the user's experience intent based on the information expressed by the user. The processor 1010 executes the processing instructions in the memory 1020 to enable the vehicle's system strategy unit to obtain multiple recommended strategy actions based on the vehicle's system status information and the user's experience intent, and also adjusts the scores of these multiple strategy actions based on the user's strategy preference information. The processor 1010 executes the output instructions in the memory 1020 to output control actions for the vehicle based on the adjusted scores, causing the vehicle's action execution unit to execute the strategy actions, thereby achieving vehicle control.

[0166] It should be understood that the computing device 1000 according to the embodiments of this application can correspond to the corresponding subject in executing the methods according to the various embodiments of this application, and the above and other operations and / or functions of each module in the computing device 1000 are respectively for implementing the corresponding processes of the methods of this embodiment. For the sake of brevity, they will not be described in detail here.

[0167] Figure 11 This is an architectural diagram of an electronic device 1100 provided in an embodiment of this application. The electronic device 1100 can serve as a human-computer interaction device, executing various optional embodiments of the aforementioned human-computer interaction methods. The electronic device can be an in-vehicle device or terminal, such as a vehicle, an in-vehicle infotainment system, or an in-vehicle computer, or a mobile phone, tablet, or other terminal device. It can also be a chip or chip system within a terminal, such as a chip or processor in an in-vehicle infotainment system or a mobile phone, tablet, or other terminal device, or a conventional chip processor such as a CPU or MCU. The electronic device can also be a hardware and software system within a terminal, such as a human-computer interaction interface in a vehicle cabin, an in-vehicle infotainment system, or other systems with both software and hardware. Figure 11 The electronic device 1100 includes a processor 1110 and an interface circuit 1120. The processor 1110 accesses a memory through the interface circuit 1120. The memory stores program instructions, which, when executed by the processor, cause the processor to perform any operation step of the aforementioned human-computer interaction method and any optional embodiment thereof. For example, the processor 1110 can obtain computer execution instructions from the memory through the interface circuit 1120 to execute... Figure 6In the corresponding embodiment of the human-computer interaction method, when a user expresses themselves through voice, gestures, facial expressions, or other collectable means, the processor 1110 obtains the acquisition instructions from the memory through the interface circuit 1120, controlling the vehicle's multi-dimensional perception unit to collect information such as the user's voice, gestures, facial expressions, or body movements, and organizes and generates the information expressed by the user. The processor 1110 obtains the recognition instructions from the memory through the interface circuit 1120, enabling the vehicle's intent recognition unit to recognize the user's experience intent based on the information expressed by the user. The processor 1110 obtains the processing instructions from the memory through the interface circuit 1120, enabling the vehicle's system strategy unit to obtain multiple recommended strategy actions based on the vehicle's system status information and the user's experience intent, and also adjusts the scores of multiple strategy actions based on the user's strategy preference information. The processor 1110 obtains the output instructions from the memory through the interface circuit 1120, and outputs the controlled strategy actions of the vehicle based on the adjusted scores, causing the vehicle's action execution unit to execute the strategy actions, thereby achieving vehicle control.

[0168] In addition, the electronic device may also include a communication interface, a bus, etc., for details please refer to Figure 10 The descriptions in the illustrated embodiments will not be repeated here.

[0169] Figure 12 This is an architecture diagram of a human-computer interaction system provided in an embodiment of this application. This system can execute various optional embodiments of the aforementioned human-computer interaction methods. The system can be a terminal, such as a vehicle, a vehicle-mounted infotainment system, or an in-vehicle computer, or a mobile phone, tablet, or other terminal device. It can also be a chip or chip system within the terminal, such as a chip or processor in a vehicle-mounted infotainment system or computer, a chip or processor in a mobile phone, tablet, or other terminal device, or a conventional chip processor such as a CPU or MCU. Furthermore, the system can be a hardware and software system within the terminal, such as a human-computer interface in a vehicle cabin, an in-vehicle infotainment system, or other systems with both software and hardware. Figure 12 The human-computer interaction system 1200 includes a multi-dimensional sensing device 1210, a vehicle management device 1220, and a human-computer interaction device 1230.

[0170] The multi-dimensional sensing device 1210 can be derived from Figure 2 The multi-dimensional sensing unit 110 in the illustrated embodiment can be implemented by, for example, one or more of a microphone module, a camera module, and a human body sensor module, or by... Figure 2 The interface circuit between the multi-dimensional sensing unit 110 and the intent recognition unit 120 in the illustrated embodiment is implemented. The vehicle management device 1220 can be implemented by... Figure 2The device management unit 130 in the illustrated embodiment can be implemented, for example, by one or more of the following: air conditioning controller, seat controller, speaker controller, display controller, lighting controller, sunroof controller, window controller, curtain controller, and environmental sensor module. It can also be implemented by... Figure 2 The state processing unit 140 in the illustrated embodiment is implemented, but it can also be implemented by the interface circuit between the state processing unit 140 and the system strategy unit 150. The human-computer interaction device 1230 can be implemented by... Figure 2 The implementation of one or more of the intent recognition unit 120, system strategy unit 150, and action execution unit 160 in the illustrated embodiment can also be provided by Figure 9 The human-computer interaction device 900 in the illustrated embodiment can be implemented, for example, by one or more of the following modules: recognition module 910, recommendation module 920, adjustment module 930, output module 940, training module 950, and identity authentication module 960. Alternatively, it can be implemented by... Figure 10 The computing device 1000 in the illustrated embodiment can also be implemented by... Figure 11 The electronic device 1100 in the illustrated embodiment is used for implementation. Alternatively, the human-computer interaction system in the embodiments of this application can also be implemented by a combination of a processor and software modules.

[0171] In this embodiment, the human-computer interaction system 1200 can execute... Figure 6 The human-computer interaction method in the illustrated embodiment and its optional embodiments, for example, can collect user expression data through the multi-dimensional sensing device 1210, generate user expression information, and provide it to the human-computer interaction device 1230. The vehicle management device 1220 can collect vehicle system device status data and vehicle system environmental status data, generate vehicle system status information, and provide it to the human-computer interaction device 1230. The human-computer interaction device 1230 obtains the user's experience intent based on the user's expression information. Based on the vehicle system status information and the user's experience intent, the human-computer interaction device 1230 obtains multiple recommended strategy actions, adjusts the scores of these multiple strategy actions according to the user's strategy preference information, and then determines the strategy action to control the vehicle system based on the adjusted scores to execute control of the vehicle system. The multi-dimensional sensing device 1210, the vehicle management device 1220, and the human-computer interaction device 1230 can communicate data or commands via wired means (e.g., interface circuits) or wireless means (e.g., Bluetooth, Wi-Fi).

[0172] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0173] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0174] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0175] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0176] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0177] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0178] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, performs a human-computer interaction method, including at least one of the solutions described in the above embodiments.

[0179] The computer storage medium in this application embodiment can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. For example, a computer-readable storage medium can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0180] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, wherein computer-readable program code is attached. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which can transmit, propagate, or transfer programs for use by or in connection with an instruction execution system, apparatus, or device.

[0181] The program code contained on a computer-readable medium may be transmitted using any suitable medium, including, but not limited to, wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0182] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0183] It should be noted that the embodiments described in this application are merely some embodiments, not all embodiments. The components of the embodiments of this application typically described and shown in the accompanying drawings can be arranged and designed in various different configurations. Therefore, the above detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0184] The terms "first, second, third, etc." or similar terms such as module A, module B, module C, etc., used in the specification and claims are only used to distinguish similar objects and do not represent a specific ordering of objects. It is understood that a specific order or sequence may be interchanged where permitted so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0185] In the above description, the step numbers involved, such as S110, S120, etc., do not mean that this step will necessarily be executed. It may also include intermediate steps or be replaced by other steps. Where permissible, the order of the preceding and following steps may be interchanged or executed simultaneously.

[0186] The term "comprising" as used in the specification and claims should not be construed as limiting itself to what follows; it does not exclude other elements or steps. Therefore, it should be interpreted as specifying the presence of the mentioned feature, integral, step, or component, but does not exclude the presence or addition of one or more other features, integrals, steps, or components, or groups thereof. Thus, the statement "device comprising means A and B" should not be limited to a device consisting solely of components A and B.

[0187] The terms "an embodiment" or "an embodiment" as used in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in at least one embodiment of this application. Therefore, the terms "in one embodiment" or "in an embodiment" appearing throughout this specification do not necessarily refer to the same embodiment, but may refer to the same embodiment. Furthermore, in the various embodiments of this application, unless otherwise specified or in case of logical conflict, the terminology and / or descriptions between different embodiments are consistent and can be mutually referenced. Technical features in different embodiments can be combined to form new embodiments based on their inherent logical relationships.

[0188] Note that the above are merely preferred embodiments and the technical principles employed in this application. Those skilled in the art will understand that this application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of this application. Therefore, although this application has been described in detail through the above embodiments, this application is not limited to the above embodiments. Many other equivalent embodiments may be included without departing from the concept of this application, all of which fall within the scope of protection of this application.

Claims

1. A human-machine interaction method, characterized in that, include: The system obtains user-expressed information and, based on this information, obtains the user's experience intent. The user-expressed information includes physiological characteristics, which are used to describe the user's physiological perception of the external environment. The expressive information does not include policy actions for controlling the vehicle system. Based on the vehicle system's status information and the user's experience intent, multiple recommended strategy actions are obtained; these strategy actions are used to control the vehicle system and are configured with a rating. Adjust the scores of the multiple strategy actions based on the user's strategy preference information; When a strategy action is determined from two or more strategy actions, and the difference between the scores of the two or more strategy actions is greater than a threshold, the strategy action to control the vehicle system is determined based on the highest score. When a decision is made from two or more policy actions, and the difference in the scores of the two or more policy actions is less than a threshold, the two or more policy actions are prompted to the user, the user's selection information is received, the policy action to control the vehicle system is determined based on the user's selection information, the user's policy preference information is updated based on the user's selection information, and the correlation between any two or more of the information expressed by the user, the user's experience intent, the recommended multiple policy actions, and the user's policy preference information; The vehicle system is controlled according to the strategy action described above; When the policy action can no longer be executed or fails to execute, the previously generated policy action is used as the session history, and an alternative policy action is generated based on the information expressed by the user in the second round. The meaning of the information expressed in the second round is the same as or similar to the meaning of the information expressed in the previous round, and the alternative policy action has the same or similar execution effect as the policy action. The strategy action is used to respond to the physiological characteristics.

2. The method of claim 1, wherein, The method further includes: When the first round of information expressed by the user indicates that the user feels hot, a first strategy action is determined, which is to enhance the air conditioning cooling. When the second round of information expressed by the user indicates that the user feels hot, a second strategy action is determined, which is a seat air blower.

3. The method according to claim 1 or 2, characterized in that, The method further includes: Based on the adjusted scores, two or more policy actions with scores higher than the set value are identified as policy actions to control the vehicle system.

4. The method according to claim 1 or 2, characterized in that, The method further includes: The scores of the multiple policy actions are weighted and calculated based on the user's policy preference information.

5. The method according to claim 1 or 2, characterized in that, The information expressed by the user includes one or more of the user's voice information, action information, and facial expression information.

6. The method of claim 1 or 2, wherein, When obtaining the user's experience intent based on the information expressed by the user, the user's vital signs information is also obtained.

7. The method according to claim 1 or 2, characterized in that, When obtaining the user's experience intent based on the information expressed by the user, the user also obtains it based on the session history information; the session history information includes previously determined policy actions for controlling the vehicle system.

8. The method of claim 1 or 2, wherein, The vehicle system's status information includes one or more of the vehicle system's device status information and the environmental status information obtained by the vehicle system.

9. The method of claim 1 or 2, wherein, Also includes: Obtain the user's identity information, and obtain the user's strategy preference information based on the user's identity information.

10. A human-machine interaction device, characterized in that, include: The recognition module is used to obtain information expressed by the user and to obtain the user's experience intention based on the information expressed by the user. The information expressed by the user includes physiological characteristics, which are used to indicate the description of the user's physiological perception of the outside world. The information expressed does not include strategy actions for controlling the vehicle system. The recommendation module is used to obtain multiple recommended strategy actions based on the vehicle system's status information and the user's experience intent; the strategy actions are used to control the vehicle system and are configured with a rating. The adjustment module is used to adjust the scores of the multiple strategy actions based on the user's strategy preference information; The output module is configured to: determine the control strategy for the vehicle system based on the highest score when the strategy is determined from two or more strategy actions and the difference between the scores of the two or more strategy actions is greater than a threshold; and prompt the user with the two or more strategy actions when the strategy is determined from two or more strategy actions and the difference between the scores of the two or more strategy actions is less than a threshold; and receive the user's selection information and determine the control strategy for the vehicle system based on the user's selection information. The adjustment module is also used to update the user's strategy preference information based on the user's selection information; The training module is used to update the correlation between any two or more of the information expressed by the user, the user's experience intent, the recommended multiple strategy actions, and the user's strategy preference information based on the user's selection information. The output module is also used to execute control of the vehicle system according to the strategy action; The recommendation module is further configured to, when the strategy action can no longer be executed or the execution fails, use the previously generated strategy action as the session history, and generate an alternative strategy action based on the information expressed by the user in the second round, wherein the meaning of the information expressed in the second round is the same as or similar to the meaning of the information expressed in the previous round, and the alternative strategy action has the same or similar execution effect as the strategy action; wherein the strategy action is used to respond to the physiological characteristics.

11. The apparatus of claim 10, wherein, The recommendation module is specifically used for: When the first round of information expressed by the user indicates that the user feels hot, a first strategy action is determined, which is to enhance the air conditioning cooling. When the second round of information expressed by the user indicates that the user feels hot, a second strategy action is determined, which is a seat air blower.

12. The apparatus of claim 10 or 11, wherein, The output module is specifically used for: Based on the adjusted scores, two or more policy actions with scores higher than the set value are identified as policy actions to control the vehicle system.

13. The apparatus of claim 10 or 11, wherein, The adjustment module is specifically used for: The scores of the multiple policy actions are weighted and calculated based on the user's policy preference information.

14. The apparatus of claim 10 or 11, wherein, The information expressed by the user includes one or more of the user's voice information, action information, and facial expression information.

15. The apparatus according to claim 10 or 11, characterized in that, When the recognition module obtains the user's experience intent based on the information expressed by the user, it also obtains it based on the user's vital signs information.

16. The apparatus of claim 10 or 11, wherein, When the identification module obtains the user's experience intent based on the information expressed by the user, it also obtains it based on the session history information; the session history information includes previously determined policy actions for controlling the vehicle system.

17. The apparatus of claim 10 or 11, wherein, The vehicle system's status information includes one or more of the vehicle system's device status information and the environmental status information obtained by the vehicle system.

18. The apparatus of claim 10 or 11, wherein, Also includes: The identity authentication module is used to obtain the user's identity information and, based on the user's identity information, obtain the user's policy preference information.

19. A vehicle characterized by comprising: include: A multi-dimensional sensing device is used to collect user expression data and generate information expressed by the user. The vehicle-mounted system management device is used to collect the device status data and environmental status data of the vehicle-mounted system, and generate the status information of the vehicle-mounted system. The human-computer interaction device as described in any one of claims 10 to 18.

20. An electronic device, comprising: include: A processor and an interface circuit, wherein the processor accesses a memory through the interface circuit, the memory storing program instructions that, when executed by the processor, cause the processor to perform the human-computer interaction method as described in any one of claims 1 to 9.

21. A computing device, comprising: include: processor, and A memory having stored program instructions that, when executed by the processor, cause the processor to perform the human-computer interaction method as described in any one of claims 1 to 9.

22. A computer-readable storage medium, characterized in that, It stores program instructions that, when executed by a computer, cause the computer to perform the human-computer interaction method as described in any one of claims 1 to 9.

23. A computer program product, characterised in that, It includes program instructions that, when executed by a computer, cause the computer to perform the human-computer interaction method as described in any one of claims 1 to 9.

Images