Voice control method and device for vehicle-mounted air conditioner, electronic equipment and medium

Abstract

The invention provides a voice control method and device for a vehicle-mounted air conditioner, electronic equipment and a medium, and solves the problem that the effect of an existing vehicle in actual use is poor due to various limitations of a voice instruction mode on the existing vehicle. The method comprises the steps that voice data and environment data sent by a passenger in a vehicle are acquired in real time through a pre-configured multi-mode input module, and a vehicle machine is controlled to extract voiceprint features in the voice data; matching the voiceprint features to the passenger identity, converting and analyzing the voice data based on the passenger identity to obtain the intention of the passenger, and calling the historical preference data of the passenger from the constructed preference database; fusing the environment data, the passenger intention, the historical preference data and the vehicle data to generate an optimal control strategy, and generating a control instruction based on the optimal control strategy; and a control instruction is safely issued to an air conditioner controller, so that the air conditioner controller responds to the control instruction and controls the vehicle-mounted air conditioner in the target area.

Description

Technical Field

[0001] This application relates to the field of speech recognition technology, and more specifically, to a voice control method, device, electronic device, and medium for vehicle air conditioning. Background Technology

[0002] With the development of science and technology and the continuous improvement of people's living standards, the intelligent development of automobiles has become a trend. Among them, voice interaction in in-vehicle systems has gradually become a major human-computer interaction method. In recent years, the application of in-vehicle voice recognition technology has continued to develop, and its functions have become increasingly rich. The application of voice recognition technology in in-vehicle infotainment systems is also becoming more and more widespread, including the control of in-vehicle multimedia, Bluetooth phone control, and vehicle body control, including air conditioning, windows, etc.

[0003] Current in-vehicle air conditioning controls mainly rely on physical buttons or touchscreens. However, manual operation distracts the driver and increases driving risks. While some high-end models support voice command modes, which simplify physical button or touchscreen operation to some extent, they also have limitations, resulting in unsatisfactory performance in actual use. Limited instruction set: It only supports fixed syntax commands, cannot understand contextual commands, such as "temperature 25 degrees", and cannot understand natural language; Poor environmental adaptability: In-vehicle noise interference leads to low recognition rate; Lack of intelligent decision-making: It cannot automatically adjust based on the in-vehicle environment, such as PM2.5, humidity, and temperature. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a voice control method, device, electronic device and medium for vehicle air conditioning, which solves the problem that the existing voice command mode in vehicles has many limitations, resulting in poor performance in actual use.

[0005] In a first aspect, embodiments of this application provide a voice control method for an in-vehicle air conditioner, applicable to a control system, wherein the control system includes a multimodal input module, an air conditioner controller, and an in-vehicle infotainment system, and the method includes: The vehicle system acquires voice data and environmental data emitted by passengers in real time through a pre-configured multimodal input module, and controls the vehicle system to extract voiceprint features from the voice data; the voice data and environmental data are pre-processed. The occupant's identity is matched using the voiceprint features, and the occupant's intention is obtained by converting and parsing the voice data based on the occupant's identity, as well as by retrieving the occupant's historical preference data from the constructed preference database; the occupant's intention is used for adjusting the mode of the vehicle's air conditioning. The optimal control strategy is generated by integrating the environmental data, the occupant's intention, the historical preference data, and the vehicle data, and control commands are generated based on the optimal control strategy; the optimal control strategy includes a target area on the vehicle. The control command is safely sent to the air conditioning controller so that the air conditioning controller responds to the control command and controls the vehicle air conditioning in the target area.

[0006] In conjunction with the first aspect, this application provides a first possible implementation of the first aspect, wherein the step of generating an optimal control strategy by fusing the environmental data, the occupant's intention, the historical preference data, and the vehicle data includes: An initial strategy for adjusting the vehicle's air conditioning mode is generated based on the occupant's intentions in the target area, the environmental data, and the vehicle data. The initial strategy is optimized based on the historical preference data to obtain the optimal control strategy, and an interpretable text of the optimal control strategy is generated.

[0007] In conjunction with the first aspect, this application provides a second possible implementation of the first aspect, wherein optimizing the initial strategy based on the historical preference data to obtain the optimal control strategy includes: Match the occupant's intention with the environmental data at the corresponding time, and determine whether there is a conflict between the occupant's intention and the environmental data; If so, the conflict level is determined, and the optimal control strategy is obtained based on the conflict negotiation strategy corresponding to the conflict level.

[0008] In conjunction with the first aspect, this application provides a third possible implementation of the first aspect, wherein the control system further includes a camera; The initial strategy for adjusting the vehicle's air conditioning mode, generated based on occupant intentions in the target area, environmental data, and vehicle data, includes: The target image corresponding to the occupant's intention is obtained from the real-time image captured by the camera, and the area where the occupant is located in the target image is extracted; Based on the occupant's intention, the target area is selected from the area where the occupant is located to obtain an initial strategy based on vehicle air conditioning zone control.

[0009] In conjunction with the first aspect, this application provides a fourth possible implementation of the first aspect, wherein the step of matching the occupant identity through the voiceprint features includes: Standard voice data of multiple identified occupants in the vehicle are collected in advance, and corresponding standard voiceprint features are extracted to construct a voiceprint feature library. The similarity between the voiceprint features and the standard voiceprint features in the voiceprint feature library is calculated to determine the identity of the passenger.

[0010] In conjunction with the first aspect, this application provides a fifth possible implementation of the first aspect, wherein the conversion and parsing of the voice data to obtain the occupant's intention includes: The speech data is converted into text data, and at least one keyword is extracted from the text data; the keyword includes entities and relationships; Based on the keywords and the voice data collected from the occupant in the previous collection time, the occupant's intention is determined.

[0011] In conjunction with the first aspect, this application provides a sixth possible implementation of the first aspect, wherein the secure transmission of the control command to the air conditioner controller includes: For the issuance of the control commands, multiple security dimensions are set; different security dimensions correspond to different security control methods. The vehicle control system and the air conditioning controller are controlled to execute various safety control methods corresponding to different safety dimensions to ensure safe delivery.

[0012] Secondly, embodiments of this application provide a voice control device for an in-vehicle air conditioner, applicable to a control system. The control system includes a multimodal input module, an air conditioner controller, and an in-vehicle infotainment system. The device includes: The configuration module is used to acquire, in real time, the voice data and environmental data emitted by the occupants inside the vehicle through a pre-configured multimodal input module, and control the vehicle system to extract the voiceprint features from the voice data; the voice data and environmental data are both pre-processed. The parsing module is used to match the occupant's identity through the voiceprint features, and to convert and parse the voice data based on the occupant's identity to obtain the occupant's intention and retrieve the occupant's historical preference data from the constructed preference database; the occupant's intention is used for the mode adjustment of the vehicle air conditioning. The generation module is used to integrate the environmental data, the occupant's intention, the historical preference data, and the vehicle data to generate an optimal control strategy, and to generate control commands based on the optimal control strategy; the optimal control strategy includes a target area on the vehicle. The response module is used to securely send the control command to the air conditioning controller, so that the air conditioning controller responds to the control command and controls the vehicle air conditioning in the target area.

[0013] Thirdly, embodiments of this application provide an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform the steps of any one of the voice control methods for a vehicle air conditioner described above.

[0014] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of any one of the voice control methods for an in-vehicle air conditioner.

[0015] This application provides a voice control method for an in-vehicle air conditioner, applicable to a control system. The control system includes a multimodal input module, an air conditioner controller, and an in-vehicle system. The method first acquires, in real-time, voice data and environmental data emitted by occupants inside the vehicle through a pre-configured multimodal input module, and controls the in-vehicle system to extract voiceprint features from the voice data. Both the voice data and environmental data are pre-processed. Next, the occupant's identity is matched using the voiceprint features. Based on the occupant's identity, the voice data is converted and parsed to obtain the occupant's intention and historical preference data is retrieved from a constructed preference database. The occupant's intention is used for adjusting the in-vehicle air conditioner's mode. Then, the environmental data, the occupant's intention, the historical preference data, and vehicle data are fused to generate an optimal control strategy, and a control command is generated based on the optimal control strategy. The optimal control strategy includes a target area on the vehicle. Finally, the control command is securely sent to the air conditioner controller, causing the air conditioner controller to respond to the control command and control the in-vehicle air conditioner in the target area. Based on the above methods, the automatic adjustment of the vehicle air conditioning can be achieved based on the natural voice data of the occupants, avoiding the phenomenon of single command. At the same time, it also achieves the effect of intelligent decision-making based on the occupants' voices, avoiding the effect of no intelligent decision-making. By preprocessing the voice data, the phenomenon of low recognition rate caused by in-vehicle noise interference is avoided, improving the accuracy of voice data. At the same time, personalized adjustment effect is achieved based on environmental data and historical preference data, ensuring the user experience for occupants. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This application provides a schematic flowchart of a voice control method for an in-vehicle air conditioner according to an embodiment of the present application. Figure 2 This application provides another flowchart illustrating a voice control method for an in-vehicle air conditioner. Figure 3 The illustration shows a flowchart of an embodiment of this application providing a method for obtaining the optimal control strategy; Figure 4 This application provides a structural block diagram of a voice control device for an in-vehicle air conditioner according to an embodiment of the present application. Figure 5 A structural block diagram of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0019] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0020] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0021] Current in-vehicle air conditioning controls mainly rely on physical buttons or touchscreens. Some high-end models support voice command modes, which simplify physical button or touchscreen operations to some extent, but also have limitations such as limited commands, poor environmental adaptability, and lack of intelligent decision-making, resulting in poor performance in actual use.

[0022] Based on this, the present application provides a voice control method, device, electronic device and medium for vehicle air conditioning, which will be described below through embodiments.

[0023] Example 1 To facilitate understanding of this embodiment, a voice control method for a vehicle air conditioner disclosed in this application will first be described in detail. For example... Figure 1 The diagram shown is a flowchart of a voice control method for a vehicle air conditioner. Figure 2 The diagram shows another flowchart of a voice control method for an in-vehicle air conditioner. This application provides a voice control method for an in-vehicle air conditioner, applicable to a control system, which includes a multimodal input module, an air conditioner controller, and an in-vehicle infotainment system. The method includes: S101. The vehicle system acquires voice data and environmental data emitted by passengers in the vehicle in real time through a pre-configured multimodal input module, and controls the vehicle system to extract voiceprint features from the voice data; the voice data and environmental data are pre-processed. S102. The occupant's identity is matched through the voiceprint features, and the occupant's intention is obtained by converting and parsing the voice data based on the occupant's identity, as well as the occupant's historical preference data is retrieved from the constructed preference database; the occupant's intention is used for the mode adjustment of the vehicle air conditioning. S103. The optimal control strategy is generated by integrating the environmental data, the occupant's intention, the historical preference data, and the vehicle data, and control commands are generated based on the optimal control strategy; the optimal control strategy includes a target area on the vehicle. S104. Securely send the control command to the air conditioning controller, so that the air conditioning controller responds to the control command and controls the vehicle air conditioning in the target area.

[0024] In this application, the control system includes a multimodal input module, an intelligent semantic parsing engine, a dynamic decision control unit, and an execution feedback module. The intelligent semantic parsing engine and the dynamic decision control unit are located in the vehicle infotainment system. The multimodal input module includes a microphone array, an environmental sensor, and a camera. The vehicle infotainment system communicates with the multimodal input module and the air conditioning controller via a CAN bus.

[0025] In step S101, this application acquires in real-time voice data and environmental data emitted by the occupant inside the vehicle through a pre-configured multimodal input module. The voice data is collected by a microphone array within the multimodal input module. This microphone array is highly sensitive and pre-configured, specifically consisting of six high-sensitivity MEMS microphones arranged in a ring around the vehicle ceiling. The sampling rate is 16kHz, supporting a 48kHz high-fidelity mode, and the sound source localization accuracy is ±15° (horizontal direction). The environmental data includes, but is not limited to, temperature and humidity data, CO2, and... The concentration data corresponding to PM2.5 are collected by the corresponding and configured sensors. Table 1 shows the configuration table of the sensors corresponding to the environmental data. The vehicle control unit converts and parses the voice data to obtain the passenger's intention. The passenger's intention is used to adjust the mode of the vehicle air conditioner. The voice data and environmental data are obtained by preprocessing after being received by the vehicle control unit. The preprocessing of the voice data includes multimodal fusion, beamforming, echo cancellation and noise suppression. Multimodal fusion is to fuse the voice data with the image captured by the camera based on formula (1): (1); Where A is the audio feature sequence, V is the visual feature sequence, Align(A,V) is the name of the loss function, indicating the alignment of audio and visual features, α=0.8 is the audio weight, and β=0.2 is the visual weight. At For the first t Audio features of frames Vt and Vt 1 represents the first t Frame and the t Visual features of a single frame T Total number of frames Io ( U , Vt 1) is the loss term for the visual part, CTC( AtThe following parameters represent the connection timing classification loss: beamforming refers to determining the direction of the sound source by calculating the time difference of each microphone signal; echo cancellation refers to using an adaptive filtering algorithm to eliminate the echo generated by the car audio system, and comparing the speaker output and microphone input in real time to eliminate correlation; noise suppression refers to using spectral subtraction to identify and suppress steady-state noise such as engine and wind noise, thereby maintaining a recognition rate of over 85% even at high speeds (>80km / h). After preprocessing, the multimodal input module inputs the voice data and environmental data to the vehicle's infotainment system via the CAN bus. Upon receiving the voice data, the vehicle's infotainment system extracts the voiceprint features from the voice data based on the voiceprint feature extraction (MFCC+GMM model).

[0026] Table 1 Sensor Configuration Table

[0027] In step S102, the vehicle system of this application pre-builds a passenger identity database. Passenger identities are matched against the voiceprint features in this database. If a matching voiceprint feature exists, the matched passenger identity can be determined; otherwise, no match is found. At this point, the speaker is invoked to broadcast content similar to "A new user has been detected. Do you wish to receive the new user's voice command?" After determining the passenger identity, the voice data is converted and parsed based on the passenger identity to obtain the passenger's intention and retrieve the passenger's historical preference data from the constructed preference database. The passenger intention is used for adjusting the vehicle's air conditioning mode. If the identified passenger intention is for adjusting the seat, the execution of the method provided in this application ends. The preference database and the passenger identity database are interconnected, establishing a mapping relationship between passenger identities and historical preference data. Querying the preference database based on the passenger identity determines the corresponding passenger's historical preference data. The preference database and the passenger identity database can also be stored in the cloud connected to the vehicle system for use during operation or offline, thus ensuring the vehicle system's performance. The historical preference data mentioned above is also collected from multiple dimensions, including explicit preferences of parameters directly set by users, implicit behaviors of frequently manually adjusted patterns, and contextual correlations of preference differences under different times and weather conditions. It also identifies temporary preference changes based on the adjustment behavior recorded in a single trip and optimizes the control strategy of the current trip in real time. It also performs long-term preference modeling to establish personalized user preference profiles, including identifying stable preference patterns such as a constant 24°C preference and distinguishing between seasonal and contextual preferences. It can also predict user needs based on historical data, proactively recommend settings that match preferences, reduce repetitive user input, and continuously update the preference model to identify preference change trends and balance personalization and universality.

[0028] In a specific implementation of step S102, one embodiment is as follows: matching the occupant's identity through the voiceprint features includes; S10211. Collect standard voice data of multiple identified occupants in the vehicle in advance, and extract the corresponding standard voiceprint features to construct a voiceprint feature library. S10212. Calculate the similarity between the voiceprint features and the standard voiceprint features in the voiceprint feature library to determine the identity of the passenger.

[0029] In steps S10211-S10212, the vehicle system pre-collects standard voice data from multiple identified occupants in the vehicle. Each occupant needs to record 3-5 different voice recordings, and the corresponding standard voiceprint features are extracted to construct a voiceprint feature library. The standard voiceprint features include physiological features and behavioral features. The physiological features are inherent properties based on vocal cord vibration and oral cavity structure, such as fundamental frequency (the pitch reference for different people's speech, approximately 80-200Hz for men and 200-450Hz for women), and formant frequencies (the characteristic frequencies formed by oral cavity resonance). Behavioral features are dynamic attributes of an individual's speaking habits, such as speech rate (e.g., the number of words uttered per second), pause intervals (e.g., the interval between saying "turn on" and "air conditioning"), and stress distribution (e.g., emphasizing "turn on" or "air conditioning"). The extracted standard voiceprint features are converted into a unified format digital vector, such as 128. A 3D feature vector is used to facilitate subsequent calculation of the matching degree, thereby constructing a voiceprint feature library containing standard voice data and standard voiceprint features of multiple identified occupants. The voiceprint feature library can be stored in the vehicle's infotainment system or in the cloud that communicates with the vehicle's infotainment system, and also supports offline use to ensure the performance of the vehicle's infotainment system. The vehicle's infotainment system uses a Gaussian mixture model (GMM) to calculate the similarity between the voiceprint features of the voice data and the standard voiceprint features in the voiceprint feature library to determine the occupant's identity. The voiceprint feature library is implemented based on a finite state machine (FSM) of dialogue states, supporting two modes: short-term memory (such as the most recent 3 or more rounds of dialogue records) and long-term memory (such as user preference profiles). If the highest similarity is greater than or equal to a preset similarity threshold, the match is considered successful, and the occupant's identity is confirmed, such as being identified as occupant A. If all similarities are less than the similarity threshold, the match is considered unsuccessful, and the speaker is invoked to broadcast content similar to "A new user has been detected. Do you want to receive the new user's voice command?" If the voiceprint features are changed due to a cold, the similarity threshold can be temporarily lowered, such as from 85 points to 75 points, to avoid misjudgment.

[0030] In a specific implementation of step S102, another embodiment exists whereby the conversion and parsing of the voice data to obtain the occupant's intention includes: S10221. Convert the speech data to obtain text data, and extract at least one keyword from the text data; the keyword includes entities and relationships; S10122. Based on the keywords and the voice data collected by the occupant in the previous collection time, determine the occupant's intention.

[0031] In steps S10221-S10222, the vehicle system also needs to process the original audio. Specifically, this includes noise reduction to filter out noise from the vehicle engine, wind, etc., followed by frame segmentation of the sound data collected by the microphone array based on an intelligent semantic parsing engine, thus cutting the continuous audio into short segments. Then, feature extraction is performed, such as extracting Mel-frequency features to capture the frequency and energy information of the speech. Through deep learning models such as CNN+RNN and Transformer, the audio features are mapped into a probability sequence of pinyin or text, outputting candidate text such as "It's so hot in the car". The candidate text is then corrected and optimized for fluency, finally obtaining standardized text data. The text data is also preprocessed, specifically by splitting the text data into word units and labeling them with parts of speech, identifying keywords in the text, and extracting at least one keyword from the text data. The keywords include entities and relations. The entity is the object of the intent, such as action entities: turn on, turn off, turn up, turn down; device entities such as air conditioner; attribute entities such as temperature 25℃, airflow 3. The relationships mentioned above refer to the logical relationships between entities. For example, "opening" and "air conditioning" have an operation-object relationship. Based on the intent recognition classifier, the keywords are identified into four types: temperature adjustment ({hot, cold, temperature, heating up, cooling down}); airflow control ({high airflow, low airflow, ventilation, stuffy}); mode switching ({defogging, defrosting, automatic, energy saving}); and comprehensive commands ({comfortable, poor air quality, stuffy}). For example, if a user says, "The glass is fogged up, turn up the fan," the identified intent is: defogging + airflow adjustment. If the same occupant's voice data was collected in the previous collection time of the microphone array, and the previous voice data indicated that the airflow was too low, then based on the keywords and the occupant's voice data collected in the previous collection time, the occupant's intent is determined to be defogging + airflow adjustment, thus completing or correcting the occupant's intent.

[0032] In step S103, after obtaining the occupant's intention, the vehicle system integrates the environmental data, the occupant's intention, the historical preference data, and the vehicle data through various fusion methods to generate an optimal control strategy, and generates control commands based on the optimal control strategy. The vehicle data, such as air conditioning data or vehicle speed data, is obtained by the vehicle system based on the CAN bus. According to the hardware capabilities of the vehicle air conditioning, the strategy parameters are converted into a command format supported by the device. For example, the air conditioning controller may only support integer temperatures such as 25℃ instead of 25.5℃, and the fan speed may be divided into 6 levels, with level 4 corresponding to medium fan speed. The strategy parameters need to be mapped to a code that the device can recognize, such as the external circulation corresponding to mode code 0x03. The optimal control strategy also needs to balance comfort, energy consumption, and response speed, and dynamically adjust the weights according to the degree of urgency to achieve the best control effect and ensure passenger safety. If the optimal control strategy involves step-by-step execution, such as gradual cooling, the parameters and intervals of each stage need to be clearly defined, such as maintaining 26°C for 5 minutes in stage 1 and 25°C in stage 2. The optimal control strategy includes target areas on the vehicle, which can refer to the seats on the vehicle for all passengers or some passengers, thereby realizing zoned control of the vehicle air conditioning, further improving the user experience, and better reflecting the intelligent control of the vehicle air conditioning.

[0033] If the occupant's text data is instruction-type, there is no need to determine the occupant's intention. In this case, the optimal control strategy is obtained based on the text data, the environmental data, the vehicle data, and the historical preference data.

[0034] In the specific implementation of step S103, one embodiment is as follows: Figure 3 As shown, the process of generating an optimal control strategy by integrating the environmental data, the occupant's intentions, the historical preference data, and the vehicle data includes: S1031. Generate an initial strategy for adjusting the vehicle air conditioning mode based on the occupant's intentions in the target area, the environmental data, and the vehicle data. S1032. Optimize the initial strategy based on the historical preference data to obtain the optimal control strategy, and generate an interpretable text of the optimal control strategy.

[0035] In steps S1031-S1032, the vehicle system first achieves multi-source data synchronization and alignment through the dynamic decision control unit. This involves adding a unified timestamp to the voice data, environmental data, and vehicle data (such as air conditioning data), where the environmental and vehicle data are the extracted entity data. A 200ms time window is set for data alignment processing, resolving the issue of asynchronous time among multi-source data. This provides a time-series correlation basis for strategy decisions, such as selecting appropriate airflow and temperature, and also lays the foundation for the fusion of voice data, environmental data, and vehicle data, such as associating voice commands with vehicle status and performing time-series analysis of sensor data. The system is generated based on the occupant's intent in the target area, the environmental data, and the vehicle data. The initial strategy for adjusting the vehicle's air conditioning mode includes setting operational boundaries based on different vehicle states and high-speed driving conditions: prohibiting direct window opening to avoid strong airflow affecting driving safety, limiting sudden temperature drops to prevent passenger discomfort or window fogging, ensuring the safety of adjustment behavior, and using sensor data to determine the balance between air freshness and the risk of introducing pollutants. Based on the historical preference data, the initial strategy is optimized to obtain the optimal control strategy, reflecting the system's adaptability to individual passenger habits, thereby ensuring the accuracy of the optimal control strategy. An interpretable text of the optimal control strategy is generated, and the speaker is invoked to broadcast the interpretable text to improve passenger trust and acceptability.

[0036] In a specific implementation of step S1031, one embodiment is as follows: the control system further includes a camera; The initial strategy for adjusting the vehicle's air conditioning mode, generated based on occupant intentions in the target area, environmental data, and vehicle data, includes: A1. Obtain the target image corresponding to the occupant's intention from the real-time image captured by the camera, and extract the area where the occupant is located in the target image; A2. Based on the occupant's intention, select the target area from the area where the occupant is located to obtain an initial strategy based on vehicle air conditioning zone control.

[0037] In steps A1-A2, the control system of this application is also pre-configured for the camera. The hardware configuration includes setting an infrared camera to support low-light environments, a resolution of 640×480, a frame rate of 30fps, and a built-in CNN-based occupant detection algorithm to detect the faces of occupants and identify the number of occupants. When in use, the target image corresponding to the occupant's intention is obtained from the real-time image captured by the camera. That is, based on the aligned user intention, the real-time image at the corresponding moment is retrieved to obtain the target image. The area where the occupant is located in the target image is extracted, that is, the location of the occupant is determined. If there is specific location information in the occupant's intention, the target area is selected from the area where the occupant is located, and the vehicle air conditioner in the target area is determined to obtain an initial strategy based on the vehicle air conditioner zoning control. For example, if the occupant's intention is to adjust the rear vehicle air conditioner to child mode, the optimal control strategy is to activate child mode to avoid direct airflow, maintain a moderate temperature (24-26℃), and provide gentle airflow.

[0038] In a specific implementation of step S1032, one embodiment is as follows: optimizing the initial strategy based on the historical preference data to obtain the optimal control strategy includes: B1. Match the occupant's intention with the environmental data at the corresponding time, and determine whether there is a conflict between the occupant's intention and the environmental data; B2. If so, the conflict level is determined, and the optimal control strategy is obtained based on the conflict negotiation strategy corresponding to the conflict level.

[0039] In steps B1-B2, after aligning the multimodal data, the dynamic decision control unit matches the occupant's intention with the corresponding environmental data based on the time of the voice data. It then determines whether there is a conflict between the occupant's intention and the environmental data. If a conflict exists, such as when the occupant's intention is to open the window, but PM2.5 > 150 μg / m³ is detected (indicating severe pollution), or when the user's intention is to activate the maximum airflow external circulation, but the vehicle speed is detected to be > 120 km / h, activating the maximum airflow external circulation may conflict with driving stability or occupant comfort requirements, the conflict level is determined based on the occupant's intention, environmental data, or vehicle data. The conflict levels are categorized as high-risk, medium-risk, or low-risk. High-risk conflicts directly threaten safety, such as opening windows in smoggy weather leading to health risks, or opening windows at high speeds causing vehicle instability. Medium-risk conflicts affect comfort or equipment lifespan, such as turning on the cooling mode in sub-zero temperatures. Low-risk conflicts only deviate from user habits, such as preferring 20°C but currently setting the temperature to 22°C. Different conflict levels correspond to different conflict negotiation strategies. For high-risk conflicts, the original instruction is directly rejected, with a clear reason output: the window opening operation cannot be performed because the current external PM2.5 concentration is 210μg / m³, exceeding the safety threshold. For medium-risk conflicts, partial execution is performed with additional constraints. When the occupant intends to adjust the airflow to maximum but the external noise is >70 decibels, the airflow is adjusted to 70%+ and the window on the same side is closed. The compromise logic is clearly stated: to reduce noise interference, the airflow has been adjusted to a suitable level, and a safe alternative is triggered simultaneously: automatically switching to internal circulation + air purification mode. For low-risk conflicts, the execution right of the initial strategy is retained, while optimization suggestions are provided: the current setting is 20℃, and the detected humidity inside the vehicle is 80%. It is recommended to adjust it to 22℃ to reduce the risk of condensation. Whether to continue adjusting is requested, and the user is asked to confirm again. If the timeout occurs, the safety option is executed by default. If there is no conflict, the initial strategy for adjusting the vehicle air conditioning mode is directly generated based on the occupant's intention in the target area and the environmental data.

[0040] In step S104, after generating the control command, the vehicle infotainment system securely sends the control command to the air conditioning controller via the CAN bus. The vehicle infotainment system and the air conditioning controller also execute a security mechanism for the control command to ensure the security of the control command execution. The system also controls the air conditioning controller to respond to the control command through the feedback adjustment module, controls the vehicle air conditioning in the target area, and controls the speaker to provide voice TTS feedback, broadcasting the user's operation results. The system also performs closed-loop learning, recording user operation behavior to optimize strategies. The intelligent feedback timing includes immediate feedback, delayed feedback, and proactive reminders. Immediate feedback is used after the user has clearly executed the command, such as when the temperature has been set. Delayed feedback is used when the system automatically adjusts, such as when poor air quality is detected and the internal circulation has been activated. Proactive reminders are used for abnormal state warnings, such as when the PM2.5 concentration is high and it is recommended to close the windows. This achieves a multi-module collaborative control mechanism for the vehicle air conditioning.

[0041] In the specific implementation of step S104, one embodiment is as follows: Figure 3 As shown, the safe transmission of the control command to the air conditioner controller includes: S1041. For the issuance of the control command, multiple security dimensions are set; different security dimensions correspond to different security control methods; S1042. Control the vehicle system and the air conditioning controller to execute various safety control methods corresponding to different safety dimensions to ensure safe delivery.

[0042] In steps S1041-S1042, the vehicle infotainment system and the air conditioning controller, through the security mechanism, set multiple security dimensions for the issuance of control commands. These security dimensions include a verification dimension and a retransmission dimension. Different security dimensions correspond to different security control methods. Specifically, the verification dimension involves controlling the vehicle infotainment system to calculate the CRC value of the control command and fill it into the frame end, and incrementing the sequence number (mod 256) for each command. The air conditioning controller maintains the most recently executed sequence number. After receiving the control command, the air conditioning controller recalculates the CRC value and compares it with the frame end value. If they do not match, the command is discarded and a retransmission is requested. Alternatively, if the air conditioning controller confirms that the received sequence number is less than or equal to the executed value, the command is discarded. To prevent replay attacks, the specific security control method for the retransmission dimension is as follows: after the vehicle's infotainment system sends a control command, it starts a 500ms timer. If an ACK confirmation frame from the air conditioning controller is received within the timer, the transmission is completed. If no ACK is received within the timeout, the command is retransmitted, with a maximum of 3 retries. After 3 failed retries, an error is reported to the vehicle's HMI, prompting the user with the result "Air conditioning control failed". The vehicle's infotainment system and the air conditioning controller execute various security control methods corresponding to different security dimensions to achieve secure issuance of control commands. This constructs a highly reliable vehicle air conditioning control command transmission logic, effectively defending against the risk of misoperation under data tampering and replay attacks while ensuring real-time performance.

[0043] Example 2 This application also provides a voice control device for a vehicle air conditioner, such as... Figure 4 The diagram shows a block diagram of a voice control device for an in-vehicle air conditioner. The functions implemented by this device correspond to the steps of executing a voice control method for an in-vehicle air conditioner on a terminal device as described above. This device can be understood as a server component including a processor. The voice control device for an in-vehicle air conditioner described in this application is applicable to a control system, which includes a multimodal input module, an air conditioning controller, and an in-vehicle infotainment system. The device includes: The configuration module 401 is used to acquire, in real time, the voice data and environmental data emitted by the occupant in the vehicle through a pre-configured multimodal input module, and control the vehicle unit to extract the voiceprint features from the voice data; the voice data and environmental data are both pre-processed. The parsing module 402 is used to match the occupant identity through the voiceprint features, and to convert and parse the voice data based on the occupant identity to obtain the occupant's intention and retrieve the occupant's historical preference data from the constructed preference database; the occupant's intention is used for the mode adjustment of the vehicle air conditioning. The generation module 403 is used to integrate the environmental data, the occupant intention, the historical preference data, and the vehicle data to generate an optimal control strategy, and to generate control commands based on the optimal control strategy; the optimal control strategy includes a target area on the vehicle. The response module 404 is used to securely send the control command to the air conditioning controller, so that the air conditioning controller responds to the control command and controls the vehicle air conditioning in the target area.

[0044] In one feasible implementation, the generation module includes: The adjustment module is used to generate an initial strategy for adjusting the vehicle air conditioning mode based on the occupant's intentions in the target area, the environmental data, and the vehicle data. The optimization module is used to optimize the initial strategy based on the historical preference data to obtain the optimal control strategy, and generate an interpretable text of the optimal control strategy.

[0045] In one feasible implementation, the generation module further includes: The matching module is used to match the occupant's intention with the environmental data at the corresponding time, and to determine whether there is a conflict between the occupant's intention and the environmental data; The determination module is used to determine the conflict level if the conflict is true, and then obtain the optimal control strategy based on the conflict negotiation strategy corresponding to the conflict level.

[0046] In one feasible implementation, the generation module also includes: The acquisition module is used to acquire the target image corresponding to the occupant's intention from the real-time image captured by the camera, and extract the area where the occupant is located in the target image; The selection module is used to select the target area from the area where the occupant is located based on the occupant's intention, so as to obtain an initial strategy based on vehicle air conditioning zoning control.

[0047] In one feasible implementation, the configuration module includes: The collection module is used to collect standard voice data of multiple identified occupants in the vehicle in advance, and extract the corresponding standard voiceprint features to build a voiceprint feature library. The calculation module is used to calculate the similarity between the voiceprint features and the standard voiceprint features in the voiceprint feature library in order to determine the identity of the passenger.

[0048] In one feasible implementation, the parsing module includes: A conversion module is used to convert the speech data into text data and extract at least one keyword from the text data; the keyword includes entities and relationships; The acquisition module is used to determine the passenger's intention based on the keywords and the voice data acquired by the passenger in the previous acquisition time.

[0049] In one feasible implementation, the response module includes: The configuration module is used to set multiple security dimensions for the issuance of the control commands; different security dimensions correspond to different security control methods. The control module is used to control the vehicle system and the air conditioning controller to execute various safety control methods corresponding to different safety dimensions for secure delivery.

[0050] Example 3 This application also provides an electronic device, such as Figure 5 As shown, it includes: a processor 501, a memory 502, and a bus 503. The memory 502 stores machine-readable instructions that can be executed by the processor 501. When the electronic device is running, the processor 501 and the memory 502 communicate through the bus 503. When the machine-readable instructions are executed by the processor 501, the steps of any one of the voice control methods for a vehicle air conditioner are performed.

[0051] Example 4 This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of any one of the methods for voice control of an in-vehicle air conditioner.

[0052] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some communication interfaces; the indirect coupling or communication connection of devices or modules can be electrical, mechanical, or other forms.

[0053] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0054] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0055] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a platform server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0056] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A voice control method for a vehicle air conditioner, characterized in that, The method is applicable to a control system, which includes a multimodal input module, an air conditioning controller, and a vehicle infotainment system, and includes: The vehicle system acquires voice data and environmental data emitted by passengers in real time through a pre-configured multimodal input module, and controls the vehicle system to extract voiceprint features from the voice data; the voice data and environmental data are pre-processed. The occupant's identity is matched using the voiceprint features, and the occupant's intention is obtained by converting and parsing the voice data based on the occupant's identity, as well as by retrieving the occupant's historical preference data from the constructed preference database; the occupant's intention is used for adjusting the mode of the vehicle's air conditioning. The optimal control strategy is generated by integrating the environmental data, the occupant's intention, the historical preference data, and the vehicle data, and control commands are generated based on the optimal control strategy; the optimal control strategy includes a target area on the vehicle. The control command is safely sent to the air conditioning controller so that the air conditioning controller responds to the control command and controls the vehicle air conditioning in the target area.

2. The method according to claim 1, characterized in that, The process of generating an optimal control strategy by integrating the environmental data, occupant intentions, historical preference data, and vehicle data includes: An initial strategy for adjusting the vehicle's air conditioning mode is generated based on the occupant's intentions in the target area, the environmental data, and the vehicle data. The initial strategy is optimized based on the historical preference data to obtain the optimal control strategy, and an interpretable text of the optimal control strategy is generated.

3. The method according to claim 2, characterized in that, The process of optimizing the initial strategy based on the historical preference data to obtain the optimal control strategy includes: Match the occupant's intention with the environmental data at the corresponding time, and determine whether there is a conflict between the occupant's intention and the environmental data; If so, the conflict level is determined, and the optimal control strategy is obtained based on the conflict negotiation strategy corresponding to the conflict level.

4. The method according to claim 2, characterized in that, The control system also includes a camera; The initial strategy for adjusting the vehicle's air conditioning mode, generated based on occupant intentions in the target area, environmental data, and vehicle data, includes: The target image corresponding to the occupant's intention is obtained from the real-time image captured by the camera, and the area where the occupant is located in the target image is extracted; Based on the occupant's intention, the target area is selected from the area where the occupant is located to obtain an initial strategy based on vehicle air conditioning zone control.

5. The method according to claim 1, characterized in that, The process of matching passenger identity using the voiceprint features includes: Standard voice data of multiple identified occupants in the vehicle are collected in advance, and corresponding standard voiceprint features are extracted to construct a voiceprint feature library. The similarity between the voiceprint features and the standard voiceprint features in the voiceprint feature library is calculated to determine the identity of the passenger.

6. The method according to claim 1, characterized in that, The process of converting and parsing the speech data to obtain the occupant's intention includes: The speech data is converted into text data, and at least one keyword is extracted from the text data; the keyword includes entities and relationships; Based on the keywords and the voice data collected from the occupant in the previous collection time, the occupant's intention is determined.

7. The method according to claim 1, characterized in that, The secure transmission of the control command to the air conditioner controller includes: For the issuance of the control commands, multiple security dimensions are set; different security dimensions correspond to different security control methods. The vehicle control system and the air conditioning controller are controlled to execute various safety control methods corresponding to different safety dimensions to ensure safe delivery.

8. A voice control device for a vehicle air conditioner, characterized in that, Suitable for control systems, the control system including a multimodal input module, an air conditioning controller, and a vehicle infotainment system, the device includes: The configuration module is used to acquire, in real time, the voice data and environmental data emitted by the occupants inside the vehicle through a pre-configured multimodal input module, and control the vehicle system to extract the voiceprint features from the voice data; the voice data and environmental data are both pre-processed. The parsing module is used to match the occupant's identity through the voiceprint features, and to convert and parse the voice data based on the occupant's identity to obtain the occupant's intention and retrieve the occupant's historical preference data from the constructed preference database; the occupant's intention is used for the mode adjustment of the vehicle air conditioning. The generation module is used to integrate the environmental data, the occupant's intention, the historical preference data, and the vehicle data to generate an optimal control strategy, and to generate control commands based on the optimal control strategy; the optimal control strategy includes a target area on the vehicle. The response module is used to securely send the control command to the air conditioning controller, so that the air conditioning controller responds to the control command and controls the vehicle air conditioning in the target area.

9. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform the steps of a voice control method for an in-vehicle air conditioner as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of a voice control method for an in-vehicle air conditioner as described in any one of claims 1 to 7.

Images