Air outlet control method, system and equipment based on intention recognition and medium

Abstract

The invention discloses an air outlet control method, system and device based on intention recognition and a medium, and the method comprises the steps that face image information and interaction information of a target passenger are obtained, a driving mode of a target vehicle is obtained, eye movement tracking and emotion sensing processing are conducted on the face image information to obtain sensing information, and the sensing information is sent to the target passenger; and performing voice recognition and gesture recognition processing on the interaction information to obtain instruction information, inputting the sensing information, the instruction information and the driving mode into an intention recognition model for intention recognition processing to obtain an adjustment intention, generating a control instruction according to the adjustment intention, and controlling the air outlet of the target vehicle through the control instruction. According to the embodiment of the invention, the intention of the target passenger to adjust the air outlet can be recognized in time, so that accurate adjustment is performed according to the specific requirements of the passenger, the user experience is improved, and the method can be widely applied to the technical field of vehicle control.

Description

Technical Field

[0001] This application relates to the technical field of vehicle control, and particularly to an air outlet control method, system, device, and medium based on intention recognition. Background Art

[0002] With the development of automotive technology, the requirements for the functions of electric air outlets are also constantly increasing. In related technologies, passengers in different positions inside the vehicle have different feelings about temperature and wind speed, but the related electric air outlets cannot be precisely adjusted according to the specific needs of passengers and can only be roughly adjusted manually, which cannot meet the pursuit of users for a comfortable driving experience. In summary, the technical problems existing in related technologies need to be improved. Summary of the Invention

[0003] The main purpose of the embodiments of this application is to propose an air outlet control method, system, device, and medium based on intention recognition, which can improve the user experience.

[0004] To achieve the above object, on the one hand, an embodiment of this application proposes an air outlet control method based on intention recognition, and the method includes:

[0005] Obtain the face image information and interaction information of the target occupant, and obtain the driving mode of the target vehicle;

[0006] Perform eye movement tracking and emotion perception processing on the face image information to obtain perception information;

[0007] Perform speech recognition and gesture recognition processing on the interaction information to obtain command information;

[0008] Input the perception information, the command information, and the driving mode into an intention recognition model for intention recognition processing to obtain an adjustment intention;

[0009] Generate a control command according to the adjustment intention, and control the air outlet of the target vehicle through the control command.

[0010] In some embodiments, the obtaining the face image information and interaction information of the target occupant, and obtaining the driving mode of the target vehicle includes the following steps:

[0011] Obtain the face image information through an in-vehicle camera;

[0012] Collect voice data through a microphone, and obtain gesture action data through a gesture sensor to obtain the interaction information;

[0013] Obtain the driving mode through the vehicle system.

[0014] In some embodiments, the process of performing eye movement tracking and emotion perception on the face image information to obtain perception information includes the following steps:

[0015] Perform eye image extraction processing on the face image information to obtain an eye image;

[0016] Perform eye movement tracking processing on the eye image to obtain attention information;

[0017] Perform facial expression recognition processing on the face image information to obtain a facial expression;

[0018] Perform emotion analysis and body temperature prediction processing on the facial expression to obtain body temperature information;

[0019] Determine the perception information according to the attention information and the body temperature information.

[0020] In some embodiments, the process of performing eye movement tracking processing on the eye image to obtain attention information includes the following steps:

[0021] Perform feature extraction processing on the eye image to obtain pupil features and corneal features;

[0022] Perform eyeball modeling processing on the pupil features and corneal features to obtain an eyeball model;

[0023] Perform coordinate calculation processing on the eyeball model according to eyeball geometric parameters to obtain fixation point coordinates;

[0024] Perform real space coordinate system mapping processing on the fixation point coordinates to obtain mapping coordinates;

[0025] Obtain air outlet related coordinates, and perform visual attention prediction processing according to the mapping coordinates and the air outlet related coordinates to obtain the attention information.

[0026] In some embodiments, the process of performing emotion analysis and body temperature prediction processing on the facial expression to obtain body temperature information includes the following steps:

[0027] Perform emotion recognition processing on the facial expression to obtain emotion information;

[0028] Perform body temperature prediction processing on the emotion information through a regression model to obtain the body temperature information.

[0029] In some embodiments, the process of performing speech recognition and gesture recognition processing on the interaction information to obtain instruction information includes the following steps:

[0030] Obtain speech data and gesture action data according to the interaction information;

[0031] Perform text conversion processing on the voice data, and perform adjustment intention extraction processing through natural language processing technology to obtain a voice command;

[0032] Perform action recognition and gesture matching processing on the gesture action data to obtain an action command;

[0033] Determine the instruction information according to the voice command and the action command.

[0034] In some embodiments, the generating a control command according to the adjustment intention and controlling the air outlet of the target vehicle through the control command includes the following steps:

[0035] Perform control parameter matching processing on the adjustment intention according to a preset control strategy to obtain the control command;

[0036] Control the air outlet of the target vehicle through the control command, and perform state acquisition processing on the air outlet of the target vehicle to obtain air outlet state information;

[0037] Perform deviation comparison processing on the control command according to the air outlet state information to obtain deviation information;

[0038] Perform update and adjustment processing on the control command according to the deviation information, and return to controlling the air outlet of the target vehicle through the control command.

[0039] To achieve the above object, another aspect of the embodiments of the present application proposes an air outlet control system based on intention recognition, and the system includes:

[0040] A first module, configured to obtain the face image information and interaction information of a target occupant, and obtain the driving mode of a target vehicle;

[0041] A second module, configured to perform eye movement tracking and emotion perception processing on the face image information to obtain perception information;

[0042] A third module, configured to perform speech recognition and gesture recognition processing on the interaction information to obtain instruction information;

[0043] A fourth module, configured to input the perception information, the instruction information, and the driving mode into a pre-trained intention recognition model to obtain an adjustment intention;

[0044] A fifth module, configured to generate a control command according to the adjustment intention, and control the air outlet of the target vehicle through the control command.

[0045] To achieve the above object, on the other hand, an embodiment of the present application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the foregoing method is implemented.

[0046] To achieve the above object, on the other hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the foregoing method is implemented.

[0047] The embodiments of the present application at least include the following beneficial effects: The present application provides a method, system, device and medium for controlling an air outlet based on intention recognition. This solution obtains the face image information and interaction information of a target occupant, and obtains the driving mode of a target vehicle. By performing eye movement tracking and emotion perception processing on the face image information, perception information is obtained. By performing speech recognition and gesture recognition processing on the interaction information, instruction information is obtained. Moreover, the perception information, instruction information and driving mode are input into an intention recognition model for intention recognition processing to obtain an adjustment intention, a control instruction is generated according to the adjustment intention, and the air outlet of the target vehicle is controlled by the control instruction. By performing intention recognition on the perception information, instruction information and driving mode, the present application can timely recognize the intention of the target occupant to adjust the air outlet, so as to perform precise adjustment according to the specific needs of the passenger, improving the user experience. Description of the Drawings

[0048] Figure 1 is a flowchart of a method for controlling an air outlet based on intention recognition provided by an embodiment of the present application;

[0049] Figure 2 is a schematic structural diagram of a system for controlling an air outlet based on intention recognition provided by an embodiment of the present application;

[0050] Figure 3 is a schematic hardware structure diagram of an electronic device provided by an embodiment of the present application. Detailed Embodiments

[0051] In order to make the purpose, technical solution and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are only examples of systems and methods consistent with some aspects of the embodiments of the present application described in detail in the appended claims.

[0052] It can be understood that the terms "first", "second", etc. used in this application may be used herein to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, words such as "if" and "when" used herein may be interpreted as "when...", "while...", or "in response to determining".

[0053] The terms "at least one", "a plurality of", "each", "any one", etc. used in this application, "at least one" includes one, two or more than two, "a plurality of" includes two or more than two, "each" refers to each one of the corresponding plurality, and "any one" refers to any one of the plurality.

[0054] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0055] In the related art, passengers in different positions in the vehicle have different feelings about temperature and wind speed, but the related electric air vents cannot be accurately adjusted according to the specific needs of passengers and can only be roughly adjusted manually, which cannot meet the pursuit of users for a comfortable driving experience.

[0056] Exemplarily, for example, during driving, the driver needs to adjust the electric air vent, such as opening and closing the air vent, adjusting the air volume, and changing the air direction. During this process, the driver needs to manually adjust the air vent, which causes the driver to be distracted and prone to driving safety problems. At the same time, users cannot enjoy more intelligent and personalized services during use.

[0057] In view of this, an air outlet control method, system, device and medium based on intention recognition are provided in the embodiments of the present application. This solution obtains the face image information and interaction information of the target occupant, and obtains the driving mode of the target vehicle. The perception information is obtained by performing eye movement tracking and emotion perception processing on the face image information, and the instruction information is obtained by performing speech recognition and gesture recognition processing on the interaction information. Moreover, the perception information, instruction information and driving mode are input into an intention recognition model for intention recognition processing to obtain an adjustment intention, a control instruction is generated according to the adjustment intention, and the air outlet of the target vehicle is controlled by the control instruction. By performing intention recognition on the perception information, instruction information and driving mode, the present application can timely recognize the intention of the target occupant to adjust the air outlet, so as to perform precise adjustment according to the specific needs of the passenger, improving the user experience.

[0058] An air outlet control method based on intention recognition provided by an embodiment of the present application relates to the technical field of vehicle control. The air outlet control method based on intention recognition provided by an embodiment of the present application can be applied to a terminal, can also be applied to a server, or can also be software running on a terminal or a server. In some embodiments, the terminal can be an in-vehicle terminal, a regional controller, etc., but is not limited thereto; the server side can be configured as an independent physical server, can also be configured as a server cluster or distributed system composed of multiple physical servers, and can also be configured as a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network; the software can be an application for implementing the air outlet control method based on intention recognition, etc., but is not limited to the above forms.

[0059] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, small computers, large computers, distributed computing environments including any of the above systems or devices, and so on. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in a distributed computing environment where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0060] It should be noted that in each specific embodiment of the present application, when it comes to relevant processing based on data related to the user's identity or characteristics, such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first. Moreover, the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiments of the present application need to obtain the user's sensitive personal information, the user's separate permission or separate consent will be obtained through methods such as pop-up windows or jumping to a confirmation page. After clearly obtaining the user's separate permission or separate consent, the necessary user-related data for the normal operation of the embodiments of the present application will be obtained.

[0061] Figure 1 FIG. is an alternative flowchart of a method for controlling an air outlet based on intent recognition provided by an embodiment of the present application. Figure 1 The method in may include but is not limited to steps S101 to S105.

[0062] Step S101, obtain the face image information and interaction information of the target occupant, and obtain the driving mode of the target vehicle;

[0063] Step S102, perform eye movement tracking and emotion perception processing on the face image information to obtain perception information;

[0064] Step S103, perform speech recognition and gesture recognition processing on the interaction information to obtain instruction information;

[0065] Step S104, input the perception information, the instruction information, and the driving mode into an intent recognition model for intent recognition processing to obtain an adjustment intent;

[0066] Step S105, generate a control instruction according to the adjustment intent, and control the air outlet of the target vehicle through the control instruction.

[0067] Steps S101 to S105 illustrated in the embodiments of the present application obtain the facial image information and interaction information of the target occupant, and obtain the driving mode of the target vehicle. Among them, the interaction information may include voice information and gesture action information, and the driving mode may include a sports mode, a comfort mode, etc. The embodiments of the present application analyze the facial image information and interaction information, and combine the current driving mode of the vehicle to automatically identify the intention of the target occupant to control the air outlet, so as to automatically adjust the air outlet, and can provide more intelligent and personalized services during use. The embodiments of the present application perform eye movement tracking and emotion perception processing on the facial image information to obtain perception information, can obtain the operation intention of the user through eye movement analysis and perform corresponding parameter adjustment in combination with the emotion. And the embodiments of the present application perform voice recognition and gesture recognition processing on the interaction information to obtain the corresponding command information. Then, the perception information, command information, and driving mode are input into the intention recognition model for intention recognition processing to obtain the adjustment intention of the target occupant for the air outlet. Finally, a corresponding control command is generated according to the adjustment intention, and the air outlet of the target vehicle is controlled through the control command, which can reduce the complexity of manual operation and provide intelligent services.

[0068] One of the above technical solutions has the following advantages or beneficial effects: The embodiments of the present application can timely identify the intention of the target occupant to adjust the air outlet by performing intention recognition on the perception information, command information, and driving mode, so as to perform precise adjustment according to the specific needs of the passenger, improving the user experience.

[0069] In step S101 of some embodiments, the obtaining of the facial image information and interaction information of the target occupant, and the obtaining of the driving mode of the target vehicle include the following steps:

[0070] Obtain the facial image information through an in-vehicle camera;

[0071] Collect voice data through a microphone and obtain gesture action data through a gesture sensor to obtain the interaction information;

[0072] Obtain the driving mode through the vehicle system.

[0073] In the embodiments of the present application, the face image information of the occupant can be obtained through an in-vehicle camera, and the voice data of the occupant can be collected in combination with an in-vehicle microphone. The gesture action data can also be obtained through a gesture sensor. By using the voice data and the gesture action data as interaction information, the interaction instruction of the occupant can be obtained through the interaction information. For example, for voice instructions such as "Make the wind stronger" spoken by the occupant, or action instructions such as the occupant fanning with their hand, the intention of the user to adjust the air outlet can be recognized based on these interaction instructions, providing a basis for subsequent intention analysis. Moreover, in the embodiments of the present application, the driving mode is obtained through the vehicle system, and thus the corresponding application scenario can be recognized according to the driving mode. For example, in different driving modes, the vehicle conditions and user needs are different, and the electric air outlet can be automatically adjusted according to the corresponding driving mode in combination with the face image information and the interaction instructions. For example, in the sport driving mode, the vehicle has strong power output, and the user may be more concerned about the driving experience. At this time, the zone controller will automatically adjust the electric air outlet, mainly blowing the air towards the front windshield and the driver's position to keep the driving vision clear, and at the same time create a cool and comfortable driving environment for the driver to enhance the driving experience; while in the comfort driving mode, the vehicle focuses on smoothness and comfort, and the zone controller will adjust the electric air outlet to make the air volume and temperature more uniform at each position in the vehicle, providing a comfortable riding environment for all passengers in the vehicle.

[0074] It can be conceived that in the embodiments of the present application, by integrating the control function of the electric air outlet in the domain controller (CDC), the electric air outlet only serves as an actuator, and the domain controller and the electric air outlet communicate with each other using the ISO standard communication matrix to ensure accurate and stable data transmission. In terms of data processing and control, the zone controller has strong data processing capabilities, can obtain the data of multiple sensors such as in-vehicle cameras, microphones, and gesture sensors, and obtain the driving mode through the vehicle system, so as to perform intention recognition on the obtained data, analyze the control intention of the user for the air outlet, and generate corresponding control instructions. The control instructions are sent by the domain controller to the air outlet for precise control of the opening and closing of the air outlet, the air volume, the air direction, etc.

[0075] In step S102 of some embodiments, the process of performing eye movement tracking and emotion perception processing on the face image information to obtain perception information includes the following steps:

[0076] Perform eye image interception processing on the face image information to obtain an eye image;

[0077] Perform eye movement tracking processing on the eye image to obtain attention information;

[0078] Perform facial expression recognition processing on the face image information to obtain a facial expression;

[0079] Perform emotion analysis and body temperature prediction processing on the facial expression to obtain body temperature information;

[0080] Determine the perception information according to the attention information and the body temperature information.

[0081] In the embodiments of the present application, perception information can be obtained by performing eye movement tracking and emotion perception on the facial image information. According to this perception information, the control intention of the user for the air outlet can be analyzed better. Specifically, by performing eye image intercepting processing on the facial image information, an eye image is obtained, and eye movement tracking processing is performed on the eye image to obtain attention information. This attention information is the user's line-of-sight attention information. For example, when the user feels that the temperature inside the vehicle is relatively hot and thus wants to adjust the air outlet, the user generally subconsciously looks at the position of the air outlet or the position of the corresponding air outlet control device. At this time, the intention of the user to adjust the air outlet can be recognized. The embodiments of the present application also perform facial expression recognition processing on the facial image information to obtain the facial expression, so that emotion analysis and body temperature prediction processing can be performed on the facial expression to obtain body temperature information. By performing emotion analysis on the facial expression, the current emotion of the user can be obtained, and the emotional state will cause changes in the human body temperature. The body temperature can be predicted according to the corresponding emotion. Among them, the emotional state can affect the activities of the autonomic nervous system, thereby affecting body temperature regulation. For example, when the human body feels nervous, frightened or excited, the activities of the sympathetic nervous system increase, which will lead to an increase in sweating. Through the evaporation effect, the evaporation of sweat will extract heat, resulting in a decrease in body temperature. The perception information can be obtained by combining the attention information and the body temperature information, that is, the perception information can include the attention information and the body temperature information. The attention information is used to represent the user's attention to the adjustment of the air outlet, and the body temperature information is used to represent the body temperature prediction of the user. The embodiments of the present application can analyze the intention of the user to adjust the air outlet by analyzing the attention information, and can better analyze how the user wants to adjust the air outlet by combining the body temperature information, so that the air volume of the air outlet can be adjusted, improving the intelligent control degree of the air outlet.

[0082] In some embodiments, the performing eye movement tracking processing on the eye image to obtain attention information includes the following steps:

[0083] Perform feature extraction processing on the eye image to obtain pupil features and corneal features;

[0084] Perform eyeball modeling processing on the pupil features and the corneal features to obtain an eyeball model;

[0085] Perform coordinate calculation processing on the eyeball model according to the eyeball geometric parameters to obtain the fixation point coordinates;

[0086] Perform a mapping process on the fixation point coordinates in the real - space coordinate system to obtain the mapped coordinates;

[0087] Obtain the coordinates related to the air outlet, and perform visual attention prediction processing based on the mapped coordinates and the coordinates related to the air outlet to obtain the attention information.

[0088] In the embodiment of the present application, the neural network model can be used to extract features from the eye image to obtain pupil features and corneal features. Thus, a three - dimensional model of the user's eyeball can be constructed according to the pupil features and corneal features. By performing feature - space mapping and positioning on the pupil features and corneal features, a corresponding three - dimensional coordinate system is constructed. The eyeball model is generated by parametric modeling in the three - dimensional coordinate system. Then, coordinate calculation is performed on the eyeball model according to the geometric parameters of the eyeball, and the corresponding fixation point coordinates can be obtained. The fixation point coordinates are mapped to the real space to obtain the mapped coordinates in the real - space coordinate system. And the coordinates related to the air outlet are obtained. The coordinates related to the air outlet can be the coordinates of the air - outlet device or the coordinates of the air - outlet adjustment device. In the embodiment of the present application, the attention prediction model can perform visual attention prediction processing according to the mapped coordinates and the coordinates related to the air outlet to obtain the attention information, so that the control intention of the user for the air outlet can be better recognized according to the attention information.

[0089] In some embodiments, the performing emotion analysis and body - temperature prediction processing on the facial expression to obtain the body - temperature information includes the following steps:

[0090] Perform emotion recognition processing on the facial expression to obtain emotion information;

[0091] Perform body - temperature prediction processing on the emotion information through a regression model to obtain the body - temperature information.

[0092] In the embodiment of the present application, the neural network model can be used to recognize emotions from facial expressions. By performing pre - processing, feature extraction, feature classification, etc. on the facial - expression image, emotion information such as sadness, fear, disgust, happiness, anger, surprise, and neutrality can be obtained. Then, the regression model is used to predict the body temperature based on the emotion information. For example, a set of body - temperature data of participants in different emotional states is collected, and then the regression model is trained to predict the body - temperature change. The regression model can adopt linear regression, random forest, or neural network, etc. It can be imagined that in the embodiment of the present application, the body - temperature, humidity sensor, heart - rate sensor, and blood - pressure sensor, etc. set on the vehicle seat can also be used to obtain the metabolic rate, skin blood flow, or sweating rate, etc. for body - temperature calculation.

[0093] In step S103 of some embodiments, the performing speech recognition and gesture recognition processing on the interaction information to obtain the instruction information includes the following steps:

[0094] Obtain voice data and gesture action data according to the interaction information;

[0095] Perform text conversion processing on the voice data, and perform adjustment intention extraction processing through natural language processing technology to obtain a voice command;

[0096] Perform action recognition and gesture matching processing on the gesture action data to obtain an action command;

[0097] Determine the instruction information according to the voice command and the action command.

[0098] In the embodiment of the present application, voice data and gesture action data can be obtained through interaction information, so that the voice data can be converted into text, and the converted text can be subjected to intention recognition through natural language processing technology, and the corresponding adjustment keyword is extracted as a voice command. The embodiment of the present application also performs action recognition and gesture matching processing on the gesture action data. For example, when the user makes a gesture of fanning with the hand, by performing action recognition and matching the corresponding gesture, the action command can be obtained as a command to reduce the temperature, so that the control intention of the user to adjust the air outlet can be recognized according to the action command. The voice command and the action command are determined to obtain the instruction information. It can be imagined that the embodiment of the present application can also perform emotion recognition processing on the voice data, recognize the emotion of the user from the tone of the user, etc., so as to better recognize the control intention of the user.

[0099] In step S104 of some embodiments, the perception information, the instruction information, and the driving mode are input into an intention recognition model for intention recognition processing to obtain an adjustment intention.

[0100] In an embodiment of the present application, the method further includes pre-training an intent recognition model, which is used to recognize the intent of the user to control or adjust the air outlet. Specifically, the intent recognition model can be a Transformer model, an LSTM model, etc. Taking the Transformer model as an example, the Transformer model includes an encoder and a decoder. The training process of the language model includes: obtaining perception information, instruction information, and driving mode for training as a training data set, and inputting the training data into the Transformer model. The training data is encoded by the encoder of the Transformer model to obtain a training data hidden feature vector, and the training data hidden feature vector is decoded by the decoder of the Transformer model, so that the training data hidden feature vector is matrix-multiplied with the reference word vector to obtain a training intent recognition result. The similarity between the training intent recognition result and the label of the training data is calculated by the loss function of the Transformer model, the loss function of the Transformer model is optimized according to the similarity, the model loss of the loss function is backpropagated, and the model parameters are continuously adjusted until the similarity is greater than or equal to the similarity threshold, and the optimization of the Transformer model is stopped to obtain a compliant intent recognition model.

[0101] In step S105 of some embodiments, the generating a control instruction according to the adjustment intent and controlling the air outlet of the target vehicle through the control instruction includes the following steps:

[0102] Performing control parameter matching processing on the adjustment intent according to a preset control strategy to obtain the control instruction;

[0103] Controlling the air outlet of the target vehicle through the control instruction, and performing state acquisition processing on the air outlet of the target vehicle to obtain air outlet state information;

[0104] Performing deviation comparison processing on the control instruction according to the air outlet state information to obtain deviation information;

[0105] Updating and adjusting the control instruction according to the deviation information, and returning to controlling the air outlet of the target vehicle through the control instruction.

[0106] In the embodiment of the present application, the control parameter matching process is performed on the adjustment intention according to a preset control strategy. The preset control strategy includes control parameters for opening and closing the air outlet, air volume size, air direction, etc., and corresponding control instructions are obtained. The corresponding control parameters are matched according to the specific intention recognition result. For example, if the intention recognition result is that the user wants to lower the temperature of the air outlet, the control parameter of increasing the air volume can be matched, and the size of the parameter can also be adjusted according to the user's emotion in the intention recognition result or the specific adjustment parameter mentioned in the instruction, etc. And the air outlet of the target vehicle is controlled for corresponding opening and closing, air volume size, and air direction adjustment through the control instruction. The embodiment of the present application also performs a state acquisition process on the air outlet of the target vehicle to obtain the air outlet state information. The deviation information is obtained by performing a deviation comparison process on the control instruction according to the air outlet state information, and the control instruction is updated and adjusted according to the deviation information, and then returned to control the air outlet of the target vehicle through the control instruction. By continuously looping the above steps, if it is found that there is a deviation between the actual execution situation and the instruction, the domain controller will adjust the subsequent control instructions to ensure that the operating state of the electric air outlet always meets the expectation.

[0107] Next, in combination with specific application examples, the solution of the embodiment of the present application will be introduced and described in detail:

[0108] The embodiments of the present application can be applied in the field of vehicle control technology, specifically in the application scenario of controlling air outlets, such as intelligent automatic control of the electric air outlets of a vehicle. The embodiments of the present application use the domain controller (CDC) as the control core and obtain information from multiple data sources. On the one hand, it receives data collected by various in-vehicle sensors, such as user expression image data captured by a camera, voice data collected by a microphone, gesture action data obtained by a gesture sensor, etc. These data are used to analyze the potential adjustment needs of the user for the electric air outlet; on the other hand, the domain controller receives vehicle system information, such as the current driving mode (sport, comfort, economy, etc.). At the same time, the domain controller also stores pre-set electric air outlet control strategies and parameters, including the optimal air direction, air volume settings, etc. in different scenarios. The domain controller uses the neural network model built-in to analyze and process the user-related data collected. Taking the user voice command as an example, the voice recognition algorithm first converts the voice data into text information, and then natural language processing technology understands and analyzes the text to extract key adjustment intentions such as "increase air volume" and "adjust air direction to the left". Combining the current driving mode information of the vehicle, the domain controller generates control commands for the electric air outlet according to the pre-set control strategy. For example, in the sport driving mode, if it is detected that the driver is sweating, the domain controller will generate a command to increase the air volume blowing towards the driver. These commands contain specific control parameters, such as the motor speed (corresponding to the air volume size), the blade angle (corresponding to the air direction), and the on / off state of the air outlet, etc. The generated control commands are encoded through the ISO standard communication matrix to convert the command information into a data packet format that conforms to the communication specification. The data packet contains the command type, the target electric air outlet identifier, control parameters, etc. Then, the data packet is transmitted to the network node where the electric air outlet is located through the in-vehicle network (such as Ethernet or CAN bus, etc.). During the transmission process, the error checking and retransmission mechanism compliant with the ISO standard is followed to ensure the accuracy and integrity of the data. If an error occurs in the data packet during the transmission process, the receiving party will request the sending party to retransmit the data packet according to the check information. The electric air outlet is equipped with a dedicated communication receiving module, which is responsible for receiving the data packet from the domain controller. After receiving the data packet, it first performs a decoding operation to restore the data packet to the original control command. Then, according to the target identifier in the command, it determines whether the command is for itself. If so, the drive module of the electric air outlet drives the motor to operate and the blade to move according to the control parameters in the command. For example, if the command requires an increase in air volume, the drive module will increase the motor speed to allow more air to pass through the air outlet; if the command requires an adjustment of the air direction, the drive module will control the motor to drive the transmission gear, and then adjust the blade angle to achieve a change in the air direction. After the electric air outlet executes the command, it will feedback its own status information to the domain controller. These status information include the actual motor speed, the actual blade angle, the current on / off state of the air outlet, etc.Similarly, this feedback information is encoded and transmitted according to the ISO standard communication matrix. After the domain controller receives the feedback information, it compares it with the preset instructions. If a deviation is found between the actual execution and the instructions, the domain controller will adjust the subsequent control instructions to ensure that the operating state of the electric air outlet always meets the expectations.

[0109] Please refer to Figure 2 , this embodiment of the application also provides an air outlet control system based on intention recognition, which can implement the above-mentioned air outlet control method based on intention recognition. The system includes:

[0110] The first module 201 is used to obtain the face image information and interaction information of the target occupant, and obtain the driving mode of the target vehicle;

[0111] The second module 202 is used to perform eye movement tracking and emotion perception processing on the face image information to obtain perception information;

[0112] The third module 203 is used to perform speech recognition and gesture recognition processing on the interaction information to obtain instruction information;

[0113] The fourth module 204 is used to input the perception information, the instruction information, and the driving mode into a pre-trained intention recognition model to obtain an adjustment intention;

[0114] The fifth module 205 is used to generate a control instruction according to the adjustment intention, and control the air outlet of the target vehicle through the control instruction.

[0115] It can be understood that the content in the above method embodiments is applicable to the system embodiments of the present application. The functions specifically implemented by the system embodiments of the present application are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those of the above method embodiments.

[0116] This embodiment of the application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the above-mentioned air outlet control method based on intention recognition. The electronic device can be any intelligent terminal including a tablet computer, an in-vehicle computer, etc.

[0117] It can be understood that the content in the above method embodiments is applicable to the device embodiments of the present application. The functions specifically implemented by the device embodiments of the present application are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those of the above method embodiments.

[0118] Please refer to Figure 3 , Figure 3 schematically shows the hardware structure of an electronic device in another embodiment. The electronic device includes:

[0119] The processor 301 can be implemented in ways such as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0120] The memory 302 can be implemented in forms such as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 302 can store an operating system and other application programs. When implementing the technical solutions provided in the embodiments of this specification through software or firmware, the relevant program codes are stored in the memory 302 and are called by the processor 301 to execute a method for controlling an air outlet based on intention recognition in the embodiments of the present application;

[0121] The input / output interface 303 is used to implement information input and output;

[0122] The communication interface 304 is used to implement communication interaction between this device and other devices, and can implement communication through a wired method (such as USB, network cable, etc.) or through a wireless method (such as a mobile network, WIFI, Bluetooth, etc.);

[0123] The bus 305 transmits information between the various components of the device (such as the processor 301, the memory 302, the input / output interface 303, and the communication interface 304);

[0124] Among them, the processor 301, the memory 302, the input / output interface 303, and the communication interface 304 achieve communication connections with each other inside the device through the bus 305.

[0125] The embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned method for controlling an air outlet based on intention recognition is implemented.

[0126] It can be understood that the content in the above method embodiments is applicable to the embodiments of this storage medium. The functions specifically implemented by the embodiments of this storage medium are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those of the above method embodiments.

[0127] As a non-transitory computer-readable storage medium, the memory can be used to store non-transitory software programs and non-transitory computer-executable programs. In addition, the memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory optionally includes a memory remotely disposed relative to the processor, and these remote memories can be connected to the processor through a network. Examples of the above networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0128] An air outlet control method, system, device and medium based on intention recognition provided by an embodiment of the present application. The solution obtains the face image information and interaction information of a target occupant, and obtains the driving mode of the target vehicle. The perception information is obtained by performing eye movement tracking and emotion perception processing on the face image information, and the instruction information is obtained by performing speech recognition and gesture recognition processing on the interaction information. Moreover, the perception information, the instruction information and the driving mode are input into an intention recognition model for intention recognition processing to obtain an adjustment intention, a control instruction is generated according to the adjustment intention, and the air outlet of the target vehicle is controlled by the control instruction. By performing intention recognition on the perception information, the instruction information and the driving mode, the present application can timely recognize the intention of the target occupant to adjust the air outlet, so as to perform precise adjustment according to the specific needs of the passenger, improving the user experience.

[0129] The embodiments described in the embodiments of the present application are for more clearly illustrating the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art will know that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of the present application are equally applicable to similar technical problems.

[0130] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or combine certain steps, or different steps.

[0131] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0132] Those of ordinary skill in the art can understand that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices can be implemented as software, firmware, hardware, and appropriate combinations thereof.

[0133] In the description of the present application and the above-mentioned drawings, terms such as "first", "second", "third", "fourth", etc. (if any) are used to distinguish similar objects and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0134] It should be understood that in the present application, "at least one (item)" means one or more, and "a plurality" means two or more. "And / or" is used to describe the association relationship of associated objects and indicates that there can be three relationships. For example, "A and / or B" can mean: only A exists, only B exists, and both A and B exist at the same time. Among them, A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. "At least one (one) of the following" or its similar expression refers to any combination of these items, including any combination of single items (ones) or plural items (ones). For example, at least one (one) of a, b, or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0135] In several embodiments provided by the present application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the above-mentioned division of units is only a logical function division, and there can be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some interfaces, and the indirect coupling or communication connection of systems or units can be in electrical, mechanical, or other forms.

[0136] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0137] In addition, in each embodiment of the present application, the functional units can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0138] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in each embodiment of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store programs.

[0139] The preferred embodiments of the embodiments of the present application have been described above with reference to the accompanying drawings. However, this does not limit the scope of the rights of the embodiments of the present application. Any modifications, equivalent replacements, and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall fall within the scope of the rights of the embodiments of the present application.

Claims

1. An air outlet control method based on intent recognition, characterized in that, The method includes the following steps: Obtain the face image information and interaction information of the target occupant, and obtain the driving mode of the target vehicle; Perform eye movement tracking and emotion perception processing on the face image information to obtain perception information; Perform speech recognition and gesture recognition processing on the interaction information to obtain instruction information; Input the perception information, the instruction information, and the driving mode into an intention recognition model for intention recognition processing to obtain an adjustment intention; Generate a control instruction according to the adjustment intention, and control the air outlet of the target vehicle through the control instruction.

2. The method according to claim 1, wherein The obtaining of the face image information and interaction information of the target occupant, and the obtaining of the driving mode of the target vehicle include the following steps: Obtain the face image information through an in-vehicle camera; Collect voice data through a microphone, and obtain gesture action data through a gesture sensor to obtain the interaction information; Obtain the driving mode through the vehicle system.

3. The method according to claim 1, characterized in that, The performing of eye movement tracking and emotion perception processing on the face image information to obtain perception information includes the following steps: Perform eye image intercepting processing on the face image information to obtain an eye image; Perform eye movement tracking processing on the eye image to obtain attention information; Perform facial expression recognition processing on the face image information to obtain a facial expression; Perform emotion analysis and body temperature prediction processing on the facial expression to obtain body temperature information; Determine the perception information according to the attention information and the body temperature information.

4. The method according to claim 3, characterized in that, The performing of eye movement tracking processing on the eye image to obtain attention information includes the following steps: Perform feature extraction processing on the eye image to obtain pupil features and corneal features; Perform eyeball modeling processing on the pupil features and corneal features to obtain an eyeball model; Perform coordinate calculation processing on the eyeball model according to eyeball geometric parameters to obtain fixation point coordinates; Perform real-space coordinate system mapping processing on the fixation point coordinates to obtain mapped coordinates; Obtain air outlet-related coordinates, and perform visual attention prediction processing according to the mapped coordinates and the air outlet-related coordinates to obtain the attention information.

5. The method according to claim 3, characterized in that, The performing of emotion analysis and body temperature prediction processing on the facial expression to obtain body temperature information includes the following steps: Perform emotion recognition processing on the facial expression to obtain emotion information; Perform body temperature prediction processing on the emotion information through a regression model to obtain the body temperature information.

6. The method according to claim 1, wherein The performing of speech recognition and gesture recognition processing on the interaction information to obtain instruction information includes the following steps: Obtain voice data and gesture action data according to the interaction information; Perform text conversion processing on the voice data, and perform adjustment intention extraction processing through natural language processing technology to obtain a voice instruction; Perform action recognition and gesture matching processing on the gesture action data to obtain an action instruction; Determine the instruction information according to the voice instruction and the action instruction.

7. The method according to any one of claims 1 to 6, characterized in that, The generating of a control instruction according to the adjustment intention, and the controlling of the air outlet of the target vehicle through the control instruction include the following steps: Match control parameters for the adjustment intention according to a preset control strategy to obtain the control instruction; Control the air outlet of the target vehicle through the control instruction, and perform state acquisition processing on the air outlet of the target vehicle to obtain air outlet state information; Perform deviation comparison processing on the control instruction according to the air outlet state information to obtain deviation information; Update and adjust the control instruction according to the deviation information, and return to controlling the air outlet of the target vehicle through the control instruction.

8. An air outlet control system based on intention recognition, characterized in that, The system includes: A first module, configured to obtain the face image information and interaction information of a target occupant, and obtain the driving mode of the target vehicle; A second module, configured to perform eye movement tracking and emotion perception processing on the face image information to obtain perception information; A third module, configured to perform speech recognition and gesture recognition processing on the interaction information to obtain instruction information; A fourth module, configured to input the perception information, the instruction information, and the driving mode into a pre-trained intention recognition model to obtain an adjustment intention; A fifth module, configured to generate a control instruction according to the adjustment intention, and control the air outlet of the target vehicle through the control instruction.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, the method according to any one of claims 1 to 7 is implemented.

Images