Air conditioner multi-scene AI deep dialogue control method and system

Abstract

The application relates to the field of intelligent household appliances, and provides an air conditioner multi-scene AI deep dialogue control method and system. The method comprises the following steps: receiving control input information sent by a user through a preset mode; performing intent recognition on the received control input information, and analyzing and obtaining corresponding control requirements; performing voiceprint recognition on voice input, and matching pre-stored user preference parameters; inputting the multiple control requirements obtained through analysis and the matched user preference parameters into a pre-trained user preference model, to generate air conditioner control instructions and cross-device linkage instructions; sending the air conditioner control instructions to an air conditioner execution mechanism, to control the air conditioner to perform temperature regulation and mode switching operations; sending the cross-device linkage instructions to associated intelligent devices, to control the associated intelligent devices to perform corresponding scene operations; and sending personalized voice feedback to a user terminal corresponding to the user, to determine a corresponding control execution state. The method improves the convenience and flexibility of user operation.

Description

Technical Field

[0001] This application relates to the field of smart home appliance technology, and in particular to a method and system for multi-scenario AI deep dialogue control of air conditioners. Background Technology

[0002] With the rapid development of the smart home industry, air conditioners, as the core equipment for indoor environmental regulation, have evolved from traditional physical button control to voice control. Currently, smart air conditioner products on the market generally suffer from the following technical deficiencies: First, existing voice control systems can only recognize preset, simple commands and cannot parse multiple independent control intentions contained in the user's natural language. When a user issues a command containing complex needs, the control system can only respond to one intention, requiring the user to send commands multiple times to complete the entire operation, resulting in low interaction efficiency.

[0003] Secondly, the existing control methods of smart air conditioners are fragmented. Although some products support voice, gesture or mobile application control at the same time, each control channel operates independently, which cannot achieve unified parsing and collaborative processing of multimodal input information. Users can only choose a single control method, which severely limits the flexibility of operation.

[0004] Third, the existing system lacks user identification and personalized adaptation capabilities, with all users sharing the same set of default operating parameters. In multi-member families or multi-person office scenarios, it cannot automatically match the personalized needs of different users such as temperature, fan speed, and operating mode, making it difficult to meet differentiated comfort experiences.

[0005] Fourth, existing air conditioning control systems operate in isolation, unable to establish data communication and command linkage with other smart home appliances. Users need to operate multiple devices such as air conditioners, lights, and curtains separately, and cannot trigger the coordinated operation of multiple devices with a single command, thus failing to create a complete smart scene experience.

[0006] Fifth, existing control strategies operate with fixed logic and lack self-learning and adaptive capabilities. The system cannot automatically optimize operating parameters based on users' historical usage habits and changes in environmental parameters, requiring frequent manual adjustments by users. This results in insufficient intelligence and difficulty in achieving precise energy consumption control.

[0007] Therefore, a method is urgently needed to solve at least one of the above problems. Summary of the Invention

[0008] This application provides a multi-scenario AI deep dialogue control method and system for air conditioners, aiming to solve the problems of existing voice control systems that can only recognize preset single simple commands and cannot parse multiple independent control intentions contained in the user's natural language. When the user issues a command containing complex needs, the control system can only respond to one intention, and the user needs to send commands multiple times to complete all operations, resulting in low interaction efficiency.

[0009] In a first aspect, embodiments of this application provide an AI-powered deep dialogue control method for air conditioning in multiple scenarios, the method comprising: Receive control input information sent by the user through preset methods; the preset methods include voice input, gesture input, and mobile application input; perform intent recognition on the received control input information and parse it to obtain the corresponding control requirements; Voice input is processed by voiceprint recognition and matched with pre-stored user preference parameters. Multiple control requirements and matching user preference parameters are then input into a pre-trained user preference model to generate air conditioning control commands and cross-device linkage commands. Air conditioning control commands are sent to the air conditioning actuator to control the air conditioning to perform temperature adjustment and mode switching operations. Cross-device linkage commands are sent to associated smart devices to control the associated smart devices to perform corresponding scene operations. Send personalized voice feedback to the user's corresponding user terminal to determine the corresponding control execution status; record the user's current control behavior and environmental parameters, and update the user preference model.

[0010] In some embodiments, receiving control input information sent by the user in a preset manner includes: acquiring the user's voice input signal through a microphone array and performing noise reduction and enhancement processing on the voice input signal; acquiring the user's gesture input image through a camera and performing frame extraction and feature preprocessing on the gesture input image; receiving digital control signals sent by a mobile application through a wireless communication module; and uniformly converting the processed voice input signal, gesture input image, and digital control signal into control input information in a standard format.

[0011] In some embodiments, the step of performing intent recognition and parsing on the received control input information to obtain the corresponding control requirements includes: performing natural language processing and visual semantic parsing on the control input information; extracting multiple independent control intents contained in the control input information; associating historical interaction context information to complete the missing parameters of each control intent; verifying the executability of each control intent and filtering invalid control intents; and generating a structured list of control requirements containing all valid control intents.

[0012] In some embodiments, the voiceprint recognition of the voice input and matching of pre-stored user preference parameters includes: extracting voiceprint feature vectors from the voice input signal; comparing the voiceprint feature vectors with a locally stored voiceprint feature library one by one; when the similarity reaches a preset threshold, obtaining the user preference parameters associated with the corresponding voiceprint feature; when the similarity does not reach the preset threshold, using the system's default user preference parameters; and uploading new unmatched voiceprint features to the cloud for storage and subsequent registration.

[0013] In some embodiments, the step of inputting the parsed multiple control requirements and matching user preference parameters into a pre-trained user preference model to generate air conditioning control instructions and cross-device linkage instructions includes: determining the corresponding scenario type based on the structured control requirement list; adjusting the basic operating parameters of the corresponding scenario type in combination with the user preference parameters; generating air conditioning control instructions containing execution order and execution time; and generating cross-device linkage instructions that match the scenario type, wherein the cross-device linkage instructions contain the target device identifier and the corresponding operation instructions.

[0014] In some embodiments, sending air conditioning control commands to the air conditioning actuator to control the air conditioning to perform temperature adjustment and mode switching operations includes: the air conditioning actuator receiving the air conditioning control commands; adjusting the operating temperature and operating mode of the air conditioning according to the air conditioning control commands; collecting indoor temperature and humidity data through temperature and humidity sensors; collecting indoor air quality data through air quality sensors; fine-tuning the air conditioning operating parameters in real time based on the collected temperature and humidity data and air quality data; and sending the real-time operating status data of the air conditioning to the system coordination module.

[0015] In some embodiments, sending cross-device linkage instructions to associated smart devices to control the associated smart devices to perform corresponding scenario operations includes: sending corresponding operation instructions to each associated smart device according to the target device identifier in the cross-device linkage instruction; receiving execution status information returned by each associated smart device; summarizing the execution status information of all associated smart devices; and sending the summarized execution status information to the system collaboration module for end-to-end synchronization.

[0016] In some embodiments, recording the user's current control behavior and environmental parameters, and updating the user preference model, includes: recording the occurrence time, control request content, and final execution result of the user's current control behavior; recording indoor temperature and humidity data, indoor air quality data, and outdoor environmental data during the current control process; inputting all recorded data into the user preference model; adjusting the weight parameters of corresponding features in the user preference model; generating an updated user preference model; and synchronously storing the updated user preference model on the local device and the cloud server.

[0017] In some embodiments, before inputting the parsed multiple control requirements and matching user preference parameters into the pre-trained user preference model, the method further includes: collecting historical control behavior data and corresponding environmental parameter data of multiple users; cleaning the collected data to remove abnormal and duplicate data; labeling the cleaned data, including control requirements, scenario types, and user satisfaction; dividing the labeled data into training and validation sets; using the training set to iteratively train the initial neural network model; using the validation set to validate the accuracy of the trained model; when the validation accuracy reaches a preset standard, generating the pre-trained user preference model; and distributing the pre-trained user preference model to all connected local devices.

[0018] Secondly, this application provides an AI-powered deep dialogue control system for multiple scenarios of air conditioning, the system comprising: An information receiving unit is used to receive control input information sent by a user through a preset method; the preset method includes voice input, gesture input, and mobile application input; the received control input information is subjected to intent recognition and parsing to obtain the corresponding control requirements; The voiceprint recognition unit is used to perform voiceprint recognition on voice input and match it with pre-stored user preference parameters; it inputs the parsed multiple control requirements and the matched user preference parameters into a pre-trained user preference model to generate air conditioning control commands and cross-device linkage commands; it sends air conditioning control commands to the air conditioning actuator to control the air conditioning to perform temperature adjustment and mode switching operations; it sends cross-device linkage commands to associated smart devices to control the associated smart devices to perform corresponding scene operations. The model update unit is used to send personalized voice feedback to the user's corresponding user terminal to determine the corresponding control execution status; record the user's current control behavior and environmental parameters, and update the user preference model.

[0019] This application achieves unified processing of multimodal control input, supporting collaborative interaction of voice, gesture, and mobile application control methods, significantly improving the convenience and flexibility of user operation. It also possesses multi-intent recognition capabilities, accurately parsing complex control needs in the user's natural language and executing multiple related operations simultaneously, eliminating the need for users to repeat commands repeatedly and significantly improving interaction efficiency. Through voiceprint recognition technology, it automatically matches user-specific preference parameters, providing personalized air conditioning control services for different users, perfectly adapting to the differentiated needs of multi-member families and multi-person office scenarios. It supports cross-device scenario linkage, simultaneously generating air conditioning control commands and linkage commands with other smart devices, enabling the coordinated operation of multiple devices with a single command, building a complete smart scenario experience. It has continuous self-learning capabilities, dynamically updating the user preference model based on user control behavior and real-time environmental parameters, automatically optimizing control strategies, improving user comfort while reducing air conditioning energy consumption.

[0020] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic flowchart illustrating the steps of an air conditioning multi-scenario AI deep dialogue control method provided in an embodiment of this application; Figure 2 This is a schematic diagram of a scenario of an AI deep dialogue control method for air conditioning in multiple scenarios provided in an embodiment of this application; Figure 3 This is a schematic block diagram of the structure of an air conditioning multi-scenario AI deep dialogue control system provided in one embodiment of this application; Figure 4 This is a schematic block diagram of the structure of a computer device provided in an embodiment of this application.

[0023] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0026] It should be understood that, in order to clearly describe the technical solutions of the embodiments of the present invention, the terms "first" and "second" are used in the embodiments of the present invention to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and the terms "first" and "second" are not necessarily different.

[0027] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0028] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0029] With the rapid development of the smart home industry, air conditioners, as the core equipment for indoor environmental regulation, have evolved from traditional physical button control to voice control. Currently, smart air conditioner products on the market generally suffer from the following technical deficiencies: First, existing voice control systems can only recognize preset, simple commands and cannot parse multiple independent control intentions contained in the user's natural language. When a user issues a command containing complex needs, the control system can only respond to one intention, requiring the user to send commands multiple times to complete the entire operation, resulting in low interaction efficiency.

[0030] Secondly, the existing control methods of smart air conditioners are fragmented. Although some products support voice, gesture or mobile application control at the same time, each control channel operates independently, which cannot achieve unified parsing and collaborative processing of multimodal input information. Users can only choose a single control method, which severely limits the flexibility of operation.

[0031] Third, the existing system lacks user identification and personalized adaptation capabilities, with all users sharing the same set of default operating parameters. In multi-member families or multi-person office scenarios, it cannot automatically match the personalized needs of different users such as temperature, fan speed, and operating mode, making it difficult to meet differentiated comfort experiences.

[0032] Fourth, existing air conditioning control systems operate in isolation, unable to establish data communication and command linkage with other smart home appliances. Users need to operate multiple devices such as air conditioners, lights, and curtains separately, and cannot trigger the coordinated operation of multiple devices with a single command, thus failing to create a complete smart scene experience.

[0033] Fifth, existing control strategies operate with fixed logic and lack self-learning and adaptive capabilities. The system cannot automatically optimize operating parameters based on users' historical usage habits and changes in environmental parameters, requiring frequent manual adjustments by users. This results in insufficient intelligence and difficulty in achieving precise energy consumption control.

[0034] Therefore, a method is urgently needed to solve at least one of the above problems.

[0035] To solve the above problem, please refer to Figure 1 and Figure 2 This application provides a multi-scenario AI deep dialogue control method for air conditioning, applied to computer equipment. The computer equipment can be deployed on a single server or server cluster. It can also be deployed on handheld terminals, laptops, wearable devices, or robots, etc. It should be noted that every piece of information involved in the method provided in this application is extracted with the authorization of the relevant user and in accordance with relevant regulations, and will not infringe on user privacy.

[0036] The provided air conditioning multi-scenario AI deep dialogue control method includes steps S101 to S103. Details are as follows: Step S101. Receive control input information sent by the user through a preset method; the preset method includes voice input, gesture input and mobile application input; perform intent recognition on the received control input information and parse it to obtain the corresponding control requirements.

[0037] Specifically, this step is the input entry point and intent parsing stage of the entire control method, responsible for collecting the user's multimodal control input and converting it into structured control requirements that the system can understand.

[0038] The system supports three preset control input methods: voice input, gesture input, and mobile application input. These three input methods can be used independently or in combination.

[0039] Voice input collects the user's voice signal through the microphone array on the air conditioner itself or associated smart devices. The collection range covers the entire indoor space where the air conditioner is located and can recognize normal volume voice within a five-meter range of the air conditioner itself.

[0040] Gesture input captures images of the user's hand gestures via a camera on the air conditioner itself or a connected smart device. The capture angle covers the user's frontal hand gestures in the main indoor activity areas and can recognize both static and dynamic gestures.

[0041] Mobile application input receives digital control signals sent by the user through a dedicated application installed on the user's mobile terminal such as a smartphone or tablet, and supports three interaction methods: text input, button click, and swipe operation.

[0042] The system monitors the signals from the three input channels in real time. When a valid input is detected on any channel, the subsequent preprocessing and parsing process is immediately triggered.

[0043] By performing targeted preprocessing on the raw input information collected from different channels, the accuracy of subsequent intent recognition can be improved.

[0044] The system performs noise reduction on the voice input signal to remove background noise, echoes, and interference from the environment; it also performs enhancement processing to amplify the strength of the user's voice signal; and it performs endpoint detection to accurately identify the start and end points of the voice.

[0045] Frame extraction is performed on the gesture input image to extract keyframes containing complete gesture actions from a continuous video stream; image preprocessing is performed, including grayscale conversion, normalization, and noise reduction; gesture region segmentation is performed to separate the gesture action region from the background.

[0046] The system performs format verification on digital control signals input from mobile applications to confirm their integrity and validity; it also performs protocol conversion to transform control signals of different formats sent by different mobile terminals into a unified signal format within the system.

[0047] Deep semantic parsing is performed on the preprocessed control input information to extract the user's control intent and generate structured control requirements.

[0048] Natural language processing is performed on voice input information, including word segmentation, part-of-speech tagging, syntactic analysis, and semantic understanding, to extract keywords and semantic information from the speech.

[0049] Visual semantic analysis is performed on gesture input information to extract features such as the shape, direction, trajectory, and speed of the gesture, and these features are mapped to corresponding control semantics.

[0050] The system parses input information from mobile applications to extract user-selected control functions and set parameter values.

[0051] Identify multiple independent control intentions contained in the input information and break down complex requirements into multiple individual control intentions.

[0052] By associating historical interaction context information between the user and the system, missing parameters in each control intent can be supplemented. For example, when the user says "turn it up a little", the historical context can be used to determine whether to increase the temperature or the fan speed.

[0053] Verify the executability of each control intent, determine whether the current air conditioning status and equipment environment support the execution of the control intent, and filter out invalid control intents that cannot be executed.

[0054] Generate a structured list of control requirements containing all valid control intents. Each entry in the list includes information such as control intent type, control parameters, execution priority, and execution time.

[0055] Step S102. Perform voiceprint recognition on the voice input and match it with pre-stored user preference parameters; input the parsed multiple control requirements and the matched user preference parameters into the pre-trained user preference model to generate air conditioning control commands and cross-device linkage commands; send air conditioning control commands to the air conditioning actuator to control the air conditioning to perform temperature adjustment and mode switching operations; send cross-device linkage commands to the associated smart devices to control the associated smart devices to perform corresponding scene operations.

[0056] Specifically, this step is the core processing and execution stage of the entire control method, responsible for identifying the user, generating personalized control commands, and controlling the air conditioner and other smart devices to perform corresponding operations.

[0057] When the control input includes voice input, the system performs voiceprint recognition on the voice input to determine the user's identity and match the corresponding personalized preference parameters.

[0058] Voiceprint features are extracted from the preprocessed speech input signal. These features include the user's timbre, pitch, speech rate, formant frequencies, and speech rhythm, which can uniquely identify the user.

[0059] The extracted voiceprint features are compared one by one with all voiceprint features in the locally stored voiceprint feature library to calculate the similarity between the two voiceprint features.

[0060] When the similarity reaches a preset threshold, a successful match is determined, and the user preference parameters associated with that voiceprint feature are obtained. These user preference parameters include the user's preferred temperature, wind speed, operating mode, wind direction, timing, and scene preferences.

[0061] If the similarity does not reach the preset threshold, the match is deemed to have failed, and the default user preference parameters preset by the system are used.

[0062] New, unmatched voiceprint features are uploaded to a cloud server for storage. Once the user completes voiceprint registration, these features are then associated with the user's preference parameters.

[0063] The structured control requirements list generated in step S101 and the matched user preference parameters are input into the pre-trained user preference model to generate air conditioning control commands and cross-device linkage commands.

[0064] The user preference model is a pre-trained neural network model that can output the optimal control strategy based on the user's control needs and preference parameters.

[0065] First, based on the control intent type and content in the structured control requirements list, determine the corresponding scenario type. Scenario types include morning mode, movie-watching mode, sleep mode, work mode, and meeting mode, etc.

[0066] Then, by combining user preference parameters, the basic operating parameters for the corresponding scenario type are adjusted to generate a combination of operating parameters that meet the user's personalized needs.

[0067] Generate air conditioning control commands, which include information such as command type, target parameter value, execution order, execution time, and execution duration. Command types include temperature adjustment, mode switching, fan speed adjustment, fan direction adjustment, and timer setting.

[0068] Generate cross-device linkage instructions that match the scene type. These instructions include information such as the target device identifier, operation commands, execution order, and execution time. Target devices include, but are not limited to, smart lights, smart curtains, smart coffee machines, and smart TVs.

[0069] The generated air conditioning control commands are sent to the air conditioning actuator to control the air conditioning to perform the corresponding operations.

[0070] After receiving the air conditioning control command, the air conditioning actuator parses the command content and performs the corresponding operation according to the command requirements.

[0071] When the command is a temperature adjustment command, adjust the air conditioner's set temperature to the target temperature value.

[0072] When the command is a mode switching command, the air conditioner's operating mode is switched to the target mode. The operating modes include cooling mode, heating mode, dehumidification mode, air supply mode, and automatic mode.

[0073] When the command is a fan speed adjustment command, adjust the air conditioner's fan speed to the target fan speed level.

[0074] When the command is a wind direction adjustment command, adjust the angle of the air conditioner's air deflector to the target wind direction position.

[0075] When the instruction is a timed setting instruction, it sets the time for the air conditioner to turn on or off at a set time.

[0076] During the operation of the air conditioner, temperature and humidity data of the room are collected in real time by temperature and humidity sensors installed on the air conditioner body, and air quality data of the room are collected in real time by air quality sensors. The air quality data includes PM2.5 concentration, carbon dioxide concentration and volatile organic compound concentration.

[0077] Based on real-time collected temperature, humidity, and air quality data, the operating parameters of the air conditioner are fine-tuned in real time to maintain the indoor environment in a comfortable state preferred by the user.

[0078] The real-time operating status data of the air conditioner is sent to the system collaboration module. The operating status data includes the current set temperature, actual indoor temperature, operating mode, fan speed, air direction, and running time.

[0079] Send generated cross-device linkage commands to associated smart devices to control the associated smart devices to perform corresponding scene operations.

[0080] The system establishes communication connections with other smart devices in the home or office environment through a wireless communication module. The wireless communication methods include, but are not limited to, wireless LAN communication and Bluetooth communication.

[0081] Based on the target device identifier in the cross-device linkage command, send the corresponding operation command to each associated smart device individually.

[0082] Receive execution status information returned by each associated smart device. The execution status information includes three states: execution successful, execution failed, and execution in progress.

[0083] Summarize the execution status information of all associated smart devices and generate a complete scene execution status report.

[0084] The aggregated scene execution status information is sent to the system collaboration module to synchronize the status of the air conditioner with other smart devices.

[0085] Step S103. Send personalized voice feedback to the user's corresponding user terminal to determine the corresponding control execution status; record the user's current control behavior and environmental parameters, and update the user preference model.

[0086] Specifically, this step is the feedback and iteration phase of the entire control method. It is responsible for providing feedback on the control execution results to the user and optimizing the model performance through self-learning. Based on the execution status of the air conditioner and other smart devices, personalized voice feedback is generated and sent to the user's corresponding user terminal.

[0087] The voice feedback includes the execution results of control commands, the current operating status of the air conditioner, the linkage status of other smart devices, and indoor environmental parameters.

[0088] Based on the user's identity and preferences, the timbre, speed, and tone of the voice feedback are adjusted to generate personalized voice feedback that matches the user's habits.

[0089] Voice feedback is played to the user through the speaker on the air conditioner itself or the user's mobile device to inform the user of the execution status of the control operation.

[0090] Record user behavior data and environmental parameter data during this control process to provide data support for model updates.

[0091] Record the time of the user's control action, the control input method, the content of the control request, the result of the control command execution, and the user's feedback on the execution result.

[0092] Record indoor environmental parameters during this control process, including indoor temperature, indoor humidity, PM2.5 concentration, carbon dioxide concentration, and volatile organic compound concentration.

[0093] Record the outdoor environmental parameters during this control process, including outdoor temperature, outdoor humidity, weather conditions, and air quality index.

[0094] All recorded data is organized and stored according to a preset format, with storage locations including local devices and cloud servers, ensuring data security and reliability.

[0095] The user preference model is updated and optimized using recorded user behavior data and environmental parameter data.

[0096] The processed user behavior data and environmental parameter data are input into the user preference model.

[0097] Adjust the weight parameters of each feature in the user preference model to enable the model to more accurately predict users' control needs and preferences.

[0098] Generate an updated user preference model to replace the old model.

[0099] The updated user preference model is synchronized and stored on local devices and cloud servers to ensure that users receive a consistent personalized experience when using the device on different devices.

[0100] Regularly retrain the user preference model comprehensively, using historical data over a longer period to further improve the model's accuracy and adaptability.

[0101] In some embodiments, receiving control input information sent by the user in a preset manner includes: acquiring the user's voice input signal through a microphone array and performing noise reduction and enhancement processing on the voice input signal; acquiring the user's gesture input image through a camera and performing frame extraction and feature preprocessing on the gesture input image; receiving digital control signals sent by a mobile application through a wireless communication module; and uniformly converting the processed voice input signal, gesture input image, and digital control signal into control input information in a standard format.

[0102] The user's voice input signal is collected by a six-microphone array mounted on the top of the air conditioner. The six microphones are arranged in a ring to achieve 360-degree omnidirectional voice acquisition. The collected voice input signal undergoes adaptive noise cancellation processing to remove background noise such as noise generated by the air conditioner itself, television sound, and conversations; voice enhancement processing is performed to improve the signal-to-noise ratio of the voice signal; endpoint detection is performed, using a dual-threshold method to accurately identify the start and end positions of the voice and remove silent segments.

[0103] The system captures user gesture input images using a high-definition camera mounted on the front of the air conditioner unit. The camera has a resolution of 1920×1080 and a frame rate of 30 frames per second. Frames are extracted from the captured continuous video stream, one frame every 0.1 seconds. The extracted images are then converted to grayscale. Normalization is applied to resize the images to 224×224 pixels. Gaussian filtering is used to remove noise. Finally, a skin color detection algorithm is used for gesture region segmentation, separating the gesture action area from the background.

[0104] The system receives digital control signals sent by a user's mobile application via a wireless LAN communication module. The mobile application supports both Android and Apple operating systems. The received digital control signals undergo format verification, checking whether the signal length, checksum, and version number conform to preset standards. Protocol conversion is then performed, converting the Hypertext Transfer Protocol (HTTP) signals used by the mobile application into binary control signals used internally by the system.

[0105] The processed voice input signals, gesture input images, and digital control signals are uniformly converted into control input information in the system's internal standard format. The standard format includes four fields: input type, input time, input content, and input source.

[0106] In some embodiments, the step of performing intent recognition and parsing on the received control input information to obtain the corresponding control requirements includes: performing natural language processing and visual semantic parsing on the control input information; extracting multiple independent control intents contained in the control input information; associating historical interaction context information to complete the missing parameters of each control intent; verifying the executability of each control intent and filtering invalid control intents; and generating a structured list of control requirements containing all valid control intents.

[0107] Natural language processing is performed on the voice input information. The Chinese word segmentation algorithm is used to segment the text after voice conversion into individual words; part-of-speech tagging is performed to label the part of speech of each word; syntactic analysis is performed to analyze the grammatical structure of the sentence; and semantic understanding is performed to extract the core semantics and keywords in the sentence.

[0108] Visual semantic parsing is performed on gesture input information, and feature vectors of gesture images are extracted using convolutional neural networks. The feature vectors are then input into a pre-trained gesture classification model to identify the type of gesture. Based on the gesture type and motion trajectory, the gesture is mapped to the corresponding control semantics, such as waving an upward gesture corresponding to raising the temperature and waving a downward gesture corresponding to lowering the temperature.

[0109] The system parses input information from mobile applications to extract the control functions corresponding to the buttons clicked by the user and the parameter values ​​entered by the user.

[0110] The multi-intent classification model is used to identify multiple independent control intents contained in the input information, and to break down a complex need such as "I want to sleep, the room is a bit hot" into two independent control intents: "turn on sleep mode" and "turn down the temperature".

[0111] By associating the user's historical interaction context with the system over the past 24 hours, missing parameters in each control intent can be supplemented. For example, when the user says "turn on timer", the user's preferred timer duration can be determined to be 8 hours based on the historical context.

[0112] Verify the executability of each control intent. For example, when the air conditioner is off, filter out invalid control intents such as "increase fan speed"; when the indoor temperature is already lower than the air conditioner's minimum set temperature, filter out invalid control intents such as "continue to lower temperature".

[0113] Generate a structured list of control requirements containing all valid control intentions. Each entry in the list contains four fields: control intention type, control parameters, execution priority, and execution time. The execution priority is set according to the priority explicitly specified by the user and the urgency of the control intention.

[0114] In some embodiments, the voiceprint recognition of the voice input and matching of pre-stored user preference parameters includes: extracting voiceprint feature vectors from the voice input signal; comparing the voiceprint feature vectors with a locally stored voiceprint feature library one by one; when the similarity reaches a preset threshold, obtaining the user preference parameters associated with the corresponding voiceprint feature; when the similarity does not reach the preset threshold, using the system's default user preference parameters; and uploading new unmatched voiceprint features to the cloud for storage and subsequent registration.

[0115] Mel frequency cepstral coefficient features, linear prediction cepstral coefficient features, and fundamental frequency features are extracted from the preprocessed speech input signal, and these features are combined into a 128-dimensional voiceprint feature vector.

[0116] The extracted voiceprint feature vector is compared one by one with all voiceprint feature vectors in the locally stored voiceprint feature library, and the cosine similarity between the two vectors is calculated.

[0117] The preset similarity threshold is 0.85. When the calculated cosine similarity is greater than or equal to 0.85, it is determined to be a successful match. The user preference parameters associated with the voiceprint feature are obtained from the local database. The user preference parameters include the user's preferred cooling temperature of 26 degrees Celsius, heating temperature of 22 degrees Celsius, fan speed of automatic, airflow direction of up and down, and temperature increase of 0.5 degrees Celsius per hour in sleep mode.

[0118] When the calculated cosine similarity is less than 0.85, the match is deemed to have failed, and the system's default user preference parameters are used. The system's default parameters include a cooling temperature of 26 degrees Celsius, a heating temperature of 20 degrees Celsius, a medium fan speed, and an automatic fan direction.

[0119] New, unmatched voiceprint feature vectors and corresponding speech segments are uploaded to a cloud server for encrypted storage. When a user subsequently completes voiceprint registration in the mobile application, the voiceprint feature vector is associated with the user's registered preference parameters and synchronized to the local voiceprint feature library.

[0120] In some embodiments, the step of inputting the parsed multiple control requirements and matching user preference parameters into a pre-trained user preference model to generate air conditioning control instructions and cross-device linkage instructions includes: determining the corresponding scenario type based on the structured control requirement list; adjusting the basic operating parameters of the corresponding scenario type in combination with the user preference parameters; generating air conditioning control instructions containing execution order and execution time; and generating cross-device linkage instructions that match the scenario type, wherein the cross-device linkage instructions contain the target device identifier and the corresponding operation instructions.

[0121] The structured control requirements list and user preference parameters are input into a pre-trained deep neural network user preference model. The input layer of the model contains seven features: control intent type, control parameters, user preferred temperature, user preferred wind speed, indoor temperature, indoor humidity, and outdoor temperature. The hidden layer contains three fully connected layers, each with 256 neurons. The output layer contains two parts: air conditioning control parameters and cross-device linkage instructions.

[0122] First, based on the control intent content in the structured control requirements list, determine the corresponding scenario type. For example, when the control requirement includes "I want to watch a movie", determine the scenario type as movie viewing mode.

[0123] Then, based on user preference parameters, the basic operating parameters of the movie viewing mode are adjusted. The basic operating parameters of the movie viewing mode are a temperature of 26 degrees Celsius, a low wind speed, and a horizontal wind direction. Based on the user preference temperature of 25 degrees Celsius, the final operating temperature is adjusted to 25 degrees Celsius.

[0124] Generate an air conditioning control command. The command content is: switch the air conditioning operation mode to cooling mode, set the temperature to 25 degrees Celsius, adjust the fan speed to low speed, adjust the airflow direction to horizontal sweep, execute immediately, and execute for 2 hours.

[0125] Generate cross-device linkage commands that match the viewing mode. The commands are as follows: send a command to the smart lights to dim the brightness to 30%, send a command to the smart curtains to close the curtains, and send a command to the smart TV to turn on the power. All commands are executed immediately.

[0126] In some embodiments, sending air conditioning control commands to the air conditioning actuator to control the air conditioning to perform temperature adjustment and mode switching operations includes: the air conditioning actuator receiving the air conditioning control commands; adjusting the operating temperature and operating mode of the air conditioning according to the air conditioning control commands; collecting indoor temperature and humidity data through temperature and humidity sensors; collecting indoor air quality data through air quality sensors; fine-tuning the air conditioning operating parameters in real time based on the collected temperature and humidity data and air quality data; and sending the real-time operating status data of the air conditioning to the system coordination module.

[0127] The air conditioning main control unit receives the generated air conditioning control commands through the internal bus and parses the various parameters in the commands.

[0128] First, switch the air conditioner's operating mode from the current fan mode to cooling mode, and start the compressor and outdoor fan.

[0129] Then adjust the air conditioner's set temperature from the current 28 degrees Celsius to 25 degrees Celsius, control the opening of the electronic expansion valve, and adjust the refrigerant flow to gradually lower the indoor temperature to 25 degrees Celsius.

[0130] Next, adjust the indoor fan speed of the air conditioner from the current medium speed to low speed, and adjust the angle of the air guide plate to the horizontal direction to achieve horizontal air sweeping.

[0131] During air conditioning operation, indoor temperature and humidity data are collected every 10 seconds by a temperature and humidity sensor installed at the air inlet of the air conditioner, and indoor PM2.5 and carbon dioxide concentration data are collected every minute by an air quality sensor.

[0132] When the indoor temperature is below 24.5 degrees Celsius, reduce the opening of the electronic expansion valve to lower the cooling power; when the indoor temperature is above 25.5 degrees Celsius, increase the opening of the electronic expansion valve to increase the cooling power, so that the indoor temperature is stabilized within the range of 25 degrees Celsius ± 0.5 degrees Celsius.

[0133] When the indoor PM2.5 concentration is higher than 75 micrograms per cubic meter, the air purification function of the air conditioner will be turned on automatically; when the carbon dioxide concentration is higher than 1000 ppm, the fresh air function will be turned on automatically.

[0134] Every 30 seconds, the real-time operating status data of the air conditioner is sent to the system collaboration module. The operating status data includes the current set temperature, actual indoor temperature, operating mode, fan speed, air direction, compressor operating frequency, and air quality data.

[0135] In some embodiments, sending cross-device linkage instructions to associated smart devices to control the associated smart devices to perform corresponding scenario operations includes: sending corresponding operation instructions to each associated smart device according to the target device identifier in the cross-device linkage instruction; receiving execution status information returned by each associated smart device; summarizing the execution status information of all associated smart devices; and sending the summarized execution status information to the system collaboration module for end-to-end synchronization.

[0136] The system collaboration module establishes communication connections with smart lights, smart curtains, and smart TVs in the home through the wireless LAN communication module, and each smart device has a unique device identifier.

[0137] Based on the target device identifier in the cross-device linkage command, corresponding operation commands are sent to the smart lights, smart curtains, and smart TV respectively.

[0138] Send a command to the smart light with device identifier LIGHT001 to dim its brightness to 30%. After receiving the command, the smart light will gradually dim its brightness from the current 100% to 30% and return a status message indicating successful execution to the system.

[0139] Send a command to the smart curtain with device identifier CURTAIN001 to close the curtain. After receiving the command, the smart curtain starts the motor to gradually close the curtain from the current open state to the fully closed state, and returns a status message of successful execution to the system.

[0140] A power-on command is sent to the smart TV with device identifier TV001. After receiving the command, the smart TV powers on, enters standby mode, and returns a status message indicating successful execution to the system.

[0141] The system receives execution status information returned by all smart devices, summarizes and generates a scene execution status report, which includes the device identifier, operation instructions and execution status of each device.

[0142] The aggregated scenario execution status report is sent to the system collaboration module, which then synchronizes the report to the cloud server and the user's mobile application, achieving end-to-end status synchronization.

[0143] In some embodiments, recording the user's current control behavior and environmental parameters, and updating the user preference model, includes: recording the occurrence time, control request content, and final execution result of the user's current control behavior; recording indoor temperature and humidity data, indoor air quality data, and outdoor environmental data during the current control process; inputting all recorded data into the user preference model; adjusting the weight parameters of corresponding features in the user preference model; generating an updated user preference model; and synchronously storing the updated user preference model on the local device and the cloud server.

[0144] The user's control action was recorded as occurring at 8:30 PM on May 7, 2026. The control input method was voice input, the control request was "I want to watch a movie", and the control command was executed successfully. The user did not raise any objections to the execution result.

[0145] Record the indoor environmental parameters during this control process: the initial indoor temperature was 28 degrees Celsius, the final stable temperature was 25 degrees Celsius, the initial indoor humidity was 60%, the final stable humidity was 50%, the PM2.5 concentration was 35 micrograms per cubic meter, and the carbon dioxide concentration was 800 ppm.

[0146] Record the outdoor environmental parameters during this control process: outdoor temperature is 32 degrees Celsius, outdoor humidity is 70%, weather conditions are sunny, and air quality index is 50.

[0147] All recorded data is organized into JSON format and stored in an embedded database on the local device, while simultaneously being synchronized to a distributed database on a cloud server via encrypted transmission.

[0148] Every day at 2 AM, the system automatically performs an incremental update to the user preference model using all user behavior data and environmental parameter data recorded that day. The processed data is then input into the user preference model, and the weight parameters of each neuron in the model are adjusted using the backpropagation algorithm, with the learning rate set to 0.001 and the number of iterations set to 100.

[0149] The updated user preference model is generated, replacing the old model on local devices and the cloud server. After the update is complete, the system will automatically verify the model to ensure that the updated model can operate normally and that its accuracy is no lower than that of the original model.

[0150] In some embodiments, before inputting the parsed multiple control requirements and matching user preference parameters into the pre-trained user preference model, the method further includes: collecting historical control behavior data and corresponding environmental parameter data of multiple users; cleaning the collected data to remove abnormal and duplicate data; labeling the cleaned data, including control requirements, scenario types, and user satisfaction; dividing the labeled data into training and validation sets; using the training set to iteratively train the initial neural network model; using the validation set to validate the accuracy of the trained model; when the validation accuracy reaches a preset standard, generating the pre-trained user preference model; and distributing the pre-trained user preference model to all connected local devices.

[0151] Historical control behavior data and corresponding environmental parameter data were collected from 10,000 users of different ages, genders, regions and lifestyles. The data collection period spanned 6 months, with a total data volume of over 10 million records.

[0152] The collected raw data is cleaned to remove outlier data with values ​​outside the reasonable range, such as temperature values ​​of -10 degrees Celsius or 50 degrees Celsius; duplicate records are removed; and data with missing key fields are removed.

[0153] The cleaned data is manually labeled, including the control requirements, scenario type, and user satisfaction for each control behavior. User satisfaction is divided into five levels: very satisfied, satisfied, average, dissatisfied, and very dissatisfied.

[0154] The labeled dataset was randomly divided into a training set and a validation set in an 8:2 ratio. The training set contained 8 million data points and was used for model training; the validation set contained 2 million data points and was used for model validation.

[0155] Construct an initial deep neural network model with the following structure: input layer - hidden layer 1 - hidden layer 2 - hidden layer 3 - output layer. The input layer has 7 neurons, hidden layer 1 has 256 neurons, hidden layer 2 has 128 neurons, hidden layer 3 has 64 neurons, and the output layer has 16 neurons. The ReLU activation function and the mean squared error loss function are used.

[0156] The initial model was iteratively trained using the training set, with a batch size of 64, the Adam optimizer, a learning rate of 0.001, and a maximum number of iterations of 1000. During training, the model's accuracy was validated using the validation set every 10 iterations.

[0157] Training is stopped when the model achieves an accuracy of over 95% on the validation set and the accuracy no longer improves after 50 consecutive iterations, and a pre-trained user preference model is generated.

[0158] The pre-trained user preference model is quantized and compressed to reduce its size, enabling it to run on resource-constrained terminal devices. The compressed model is then distributed to local devices on all connected systems, completing model deployment.

[0159] In some embodiments, to address the technical problem of system response chaos caused by multiple users simultaneously sending control commands in multi-member family or multi-person office scenarios, a multi-user conflict handling method based on voiceprint identity, user priority, and command urgency is provided. This method is a combined application of step S101 (multi-input reception and intent recognition) and step S102 (voiceprint recognition and command generation).

[0160] The system simultaneously activates six independent voice acquisition channels, corresponding to the six pickup directions of the microphone array. Each channel undergoes independent noise reduction, enhancement, and endpoint detection processing. When two or more valid voice inputs are detected within a one-second time window, a multi-user conflict handling process is triggered, storing each voice signal and marking its corresponding pickup direction and timestamp.

[0161] For each isolated speech signal, a 128-dimensional voiceprint feature vector is extracted and compared in parallel with the local voiceprint feature database to identify the identity of each speaker. For unfamiliar users whose voiceprints are not matched, a temporary user identifier and default priority are automatically assigned.

[0162] The system has preset user priority levels, divided into Level 1 (elderly and children), Level 2 (adult family members), and Level 3 (visitors). Users can customize and adjust the priority through the mobile application. Simultaneously, each user's control intent is scored based on its urgency, ranging from 0 to 10. Emergency commands (such as "It's too cold" or "It's too hot") are rated 8-10 points; Common adjustment commands (such as "increase by one degree" or "switch to dehumidification mode") score 4-7 points; Non-urgent commands (such as "set the computer to start at 7 a.m. tomorrow") are rated 0-3 points; The system calculates the overall weight of each instruction based on the formula "user priority × 60% + instruction urgency × 40%", and selects the instruction with the highest overall weight as the primary instruction to be executed. For instructions with the second highest weight, if they do not conflict with the primary instruction (e.g., one adjusts the temperature, and the other adjusts the fan speed), they are executed simultaneously; if they conflict with the primary instruction (e.g., two users adjust the temperature at the same time), the primary instruction is executed, and a voice feedback is sent to the user with the lower weight: "Dad's instruction has been executed first, and the temperature has been adjusted to 26 degrees. Do you need me to adjust it for you?"

[0163] When multiple users coexist in the same space for an extended period, the system records the preference parameters of all users and calculates a weighted average as the default operating parameter for that space. For example, if the father prefers 26 degrees Celsius, the mother prefers 25 degrees Celsius, and the child prefers 27 degrees Celsius, the system automatically sets the default temperature to 26 degrees Celsius and dynamically adjusts the weights based on the usage time of different users.

[0164] The system sends personalized execution status feedback to each user, records the conflict resolution results and user feedback of this multi-user interaction, updates the user priority model and conflict decision model, and improves the accuracy of subsequent conflict resolution.

[0165] In some embodiments, to address the pain point of users needing to manually switch on and off and adjust the air conditioners in multiple rooms when moving between different rooms, a user-following air conditioner control method based on indoor positioning is provided, which is a full-process application of steps S101 (multimodal input), S102 (device collaboration and instruction generation), and S103 (model update).

[0166] By deploying Bluetooth beacons and WiFi access points in each room, and enabling human detection in the air conditioner's built-in camera, a multimodal indoor positioning system combining Bluetooth, WiFi, and vision is constructed, achieving a positioning accuracy of up to 0.5 meters. The system collects real-time data on the user's mobile terminal's Bluetooth signal strength, WiFi fingerprint, and the camera's human detection results, using a fusion algorithm to determine the user's real-time location.

[0167] The system determines the user's activity status based on the user's location change trajectory and dwell time: If a user stays in a room for more than 5 minutes, it is considered that the user has entered the room. A user is considered to have left a room if they leave the room for more than 10 minutes and do not return. When a user moves quickly between multiple rooms, it is considered a temporary passage and the air conditioning switch is not triggered. Automatic air conditioning switching process: When the system detects that a user has moved from room A to room B: Send a standby command to the air conditioner in room A to set it to low-power standby mode while retaining user preference parameters; Send a power-on command to the air conditioner in room B, and automatically load the user's preferred parameters (temperature, fan speed, mode, etc.). If room B is already in use by other users, then the preference parameters of multiple users are merged according to the multi-user conflict handling method in Example 9. Send a push notification to the user's mobile device: "The bedroom air conditioner has been turned on for you, and the temperature is 26 degrees Celsius."

[0168] The system handles simultaneous movement of multiple users by prioritizing them according to their user priority when it detects multiple users moving from room A to room B. It loads the preference parameters of higher-priority users first, records the presence of all users, and dynamically adjusts the air conditioning parameters. For example, when an elderly person and a child enter the room simultaneously, the system automatically raises the temperature by 1 degree and sets the fan speed to the lowest setting.

[0169] Energy-saving control for unoccupied rooms automatically shuts off the air conditioning and associated smart devices (such as lights and televisions) in a room when the system detects that no user has been present for 30 consecutive minutes, thus saving energy. Users can set the minimum and maximum temperatures for unoccupied rooms via a mobile application to prevent the room temperature from becoming too high or too low.

[0170] User habit learning and optimization involves recording users' daily activity patterns and air conditioning usage habits to predict their movement routes and usage needs, and then turning on the air conditioning in the target room in advance. For example, if the system learns that a user returns to their bedroom from the living room at 10 p.m. every night, it will automatically turn on the bedroom air conditioning at 9:55 p.m. and adjust the temperature to the user's preferred sleep temperature.

[0171] In some embodiments, to address the problem that the system cannot automatically adjust the control strategy when abnormal situations such as filter blockage, sensor drift, or compressor failure occur during air conditioner operation, resulting in decreased comfort and increased energy consumption, an abnormal state detection and adaptive control method is provided, which is an application of steps S102 (instruction execution and device coordination) and S103 (model update).

[0172] The system collects the air conditioner's operating parameters and environmental parameters in real time, including compressor operating frequency, fan speed, electronic expansion valve opening, inlet and outlet air temperature, inlet and outlet air pressure difference, indoor temperature and humidity, and outdoor temperature and humidity, etc., at a frequency of once per second.

[0173] The collected operational data is input into a pre-trained anomaly detection model, which is a time-series anomaly detection model based on a long short-term memory network, capable of identifying various abnormal states during air conditioner operation. Filter clogging: This is determined by changes in the inlet and outlet air pressure difference and the fan speed. When the inlet and outlet air pressure difference exceeds a preset threshold and the fan speed increases abnormally, it is determined that the filter is clogged.

[0174] Temperature and humidity sensor drift: The sensor drift is determined by comparing the indoor temperature and humidity sensor data with the outdoor temperature and humidity data and the temperature and humidity data of adjacent rooms. When the difference exceeds a preset threshold, it is determined to be sensor drift.

[0175] Compressor malfunction: It is determined by changes in compressor operating frequency, current and inlet and outlet air temperature. When the compressor operating frequency is abnormal but the inlet and outlet air temperature difference does not change, it is determined to be a compressor malfunction.

[0176] When an abnormal state is detected, the system automatically adjusts the control strategy to reduce the risk of equipment damage while ensuring basic comfort. Filter clogging: Automatically reduces fan speed and airflow while increasing fresh airflow to improve indoor air quality; automatically activates self-cleaning function every hour to alleviate filter clogging.

[0177] Temperature and humidity sensor drift: Automatically switch to redundant sensor data. If there is no redundant sensor, use temperature and humidity data from adjacent rooms and historical data to estimate and adjust the air conditioning operating parameters.

[0178] Compressor malfunction: Automatically switches to air supply mode, shuts off the compressor, and simultaneously activates the fresh air function to maintain indoor air circulation.

[0179] The system sends tiered early warning information based on the severity of the abnormal state: Level 1 Warning (Minor Anomaly): Informs the user via voice feedback, such as "The air conditioning filter is detected to be dirty. We recommend that you clean it in time."

[0180] Level 2 Warning (Moderate Abnormality): Notify users via voice feedback and mobile application push, such as "The temperature and humidity sensor has drifted and has switched to standby mode. We suggest you contact after-sales service for inspection."

[0181] Level 3 Warning (Severe Anomaly): Notifies the user via voice feedback, mobile application push, and SMS, and automatically shuts off the air conditioner, such as "A compressor malfunction has been detected, and the air conditioner has been automatically shut off. Please contact after-sales service immediately."

[0182] The system records the occurrence time, operational data, and processing results of all abnormal states and uploads them to the cloud server. The cloud server periodically retrains the anomaly detection model using fault data to improve the accuracy of anomaly detection. Simultaneously, the system optimizes adaptive control strategies based on the processing effectiveness of different abnormal states.

[0183] In some embodiments, to address the problem that a single modality cannot accurately express complex control requirements, a composite instruction parsing method that integrates voice and vision in real time is provided. This method can understand complex instructions that include indicator pronouns and spatial directions, and is a core application of step S101 (multimodal input and intent recognition).

[0184] The system synchronously collects the user's voice signals and gesture image signals. Through hardware clock synchronization technology, it aligns the timestamps of the voice signals and image signals, achieving a time synchronization accuracy of less than 10 milliseconds.

[0185] Natural language processing is performed on speech signals to extract keywords, verbs, demonstrative pronouns (such as "here", "this", "that"), and quantifiers.

[0186] Visual semantic analysis is performed on gesture images to extract the pointing direction, fingertip position, gesture type, and movement trajectory.

[0187] Object detection algorithms are used to identify the position and status of objects such as air conditioner panels, thermometers, and remote controls.

[0188] The system integrates speech and visual semantics, associating demonstrative pronouns in speech with target locations in vision to parse out accurate control parameters. When a user says "Set the temperature here" while pointing to the 24-degree mark on the wall thermometer, the system recognizes that the fingertip is pointing to the 24-degree mark on the thermometer and generates a control command to "Set the air conditioner temperature to 24 degrees".

[0189] When a user says "Turn on this mode" while pointing to the "Sleep Mode" button on the air conditioner panel, the system recognizes the button being pointed to as the sleep mode button and generates a control command to "Turn on air conditioner sleep mode".

[0190] When a user says "Turn the fan speed up to this level" while gesturing with their hand, the system adjusts the fan speed to the corresponding level based on the size of the gesture.

[0191] When ambiguity arises in multimodal semantic fusion, the system confirms to the user via voice: "Do you want to adjust the temperature to 24 degrees?" The user can confirm with a simple "yes" or "no", or issue the command again.

[0192] The system supports multimodal composite commands that include timing relationships. For example, if a user says, "Lower the temperature by two degrees first, and then adjust it back to 26 degrees in 10 minutes," while pointing to the 26-degree position, the system will parse out two control intentions: immediately lower the temperature by two degrees, and then adjust the temperature back to 26 degrees in 10 minutes, executing them sequentially according to the timing sequence.

[0193] The system records users' multimodal interaction data and feedback results, updates the multimodal semantic fusion model, and improves its adaptability to different users' gesture habits and voice expression methods.

[0194] In some embodiments, to address the issues of high energy consumption and high electricity costs associated with air conditioning, an intelligent energy consumption scheduling method that combines peak and off-peak electricity prices and user habits is provided. This method minimizes electricity costs while ensuring user comfort and is an application of steps S102 (instruction generation) and S103 (model update).

[0195] The system automatically obtains the peak-valley electricity pricing periods and pricing standards of the local power grid through a cloud server, supporting peak-valley electricity pricing policies in all provinces and cities across the country. Users can also manually input or adjust peak-valley electricity pricing information through a mobile application.

[0196] The system records users' air conditioner usage data for the past 30 days, including daily start-up time, shutdown time, set temperature, operating mode, and indoor and outdoor environmental parameters, to build a user habit model and predict users' air conditioner usage needs and comfort requirements at different times.

[0197] The system generates the optimal air conditioning operation scheduling plan based on the peak and valley electricity price information of the power grid and the user usage habit model: During off-peak electricity hours (such as 10:00 PM to 8:00 AM the next day), appropriately lower the air conditioner's set temperature (cooling mode) or raise the set temperature (heating mode) to store cooling or heating energy in advance.

[0198] During peak electricity pricing periods (such as 8:00-22:00 during the day), appropriately increase the air conditioner's set temperature (cooling mode) or decrease the set temperature (heating mode) to reduce the compressor's running time and utilize stored cold or heat energy to maintain the indoor temperature.

[0199] When it is predicted that users will use the air conditioner, turn it on in advance during off-peak electricity prices and adjust the indoor temperature to the user's preferred temperature.

[0200] The system sets comfort constraint thresholds, ensuring that the difference between the indoor temperature and the user's preferred temperature does not exceed ±1 degree Celsius, thus guaranteeing the user's basic comfort. Users can adjust the comfort priority through a mobile application. When the comfort priority is set to the highest level, the system will prioritize comfort and then consider energy costs.

[0201] The system dynamically adjusts the scheduling plan based on real-time indoor and outdoor environmental parameters and actual user usage. For example, when the outdoor temperature suddenly rises, the system automatically increases the compressor's running time to ensure stable indoor temperature. Users can intervene in the air conditioner's operation at any time via voice or mobile application, and the system will adjust the subsequent scheduling plan based on the user's intervention.

[0202] The system calculates daily electricity consumption and costs for air conditioners, generating an energy consumption report that is sent to the user's mobile application. The report includes peak and off-peak electricity consumption percentages, energy-saving effect analysis, and energy-saving suggestions to help users understand their air conditioner's energy consumption and develop energy-saving habits.

[0203] Please see Figure 3 As shown, Figure 3 This is a schematic diagram of the structure of an air conditioning multi-scene AI deep dialogue control system 200 provided in this application embodiment. The air conditioning multi-scene AI deep dialogue control system 200 is used to execute the steps of the air conditioning multi-scene AI deep dialogue control method shown in the above embodiments. The air conditioning multi-scene AI deep dialogue control system 200 can be a single server or a server cluster, or it can be a terminal, such as a handheld terminal, laptop computer, wearable device, or robot.

[0204] like Figure 3 As shown, the air conditioning multi-scenario AI deep dialogue control system 200 includes: Information receiving unit 201 is used to receive control input information sent by the user through a preset method; the preset method includes voice input, gesture input and mobile application input; the received control input information is subjected to intent recognition and parsing to obtain the corresponding control requirements; The voiceprint recognition unit 202 is used to perform voiceprint recognition on voice input and match pre-stored user preference parameters; input the parsed multiple control requirements and the matched user preference parameters into a pre-trained user preference model to generate air conditioning control commands and cross-device linkage commands; send air conditioning control commands to the air conditioning actuator to control the air conditioning to perform temperature adjustment and mode switching operations; and send cross-device linkage commands to associated smart devices to control the associated smart devices to perform corresponding scene operations. The model update unit 203 is used to send personalized voice feedback to the user terminal corresponding to the user to determine the corresponding control execution status; record the user's current control behavior and environmental parameters, and update the user preference model.

[0205] In some embodiments, receiving control input information sent by the user in a preset manner includes: acquiring the user's voice input signal through a microphone array and performing noise reduction and enhancement processing on the voice input signal; acquiring the user's gesture input image through a camera and performing frame extraction and feature preprocessing on the gesture input image; receiving digital control signals sent by a mobile application through a wireless communication module; and uniformly converting the processed voice input signal, gesture input image, and digital control signal into control input information in a standard format.

[0206] In some embodiments, the step of performing intent recognition and parsing on the received control input information to obtain the corresponding control requirements includes: performing natural language processing and visual semantic parsing on the control input information; extracting multiple independent control intents contained in the control input information; associating historical interaction context information to complete the missing parameters of each control intent; verifying the executability of each control intent and filtering invalid control intents; and generating a structured list of control requirements containing all valid control intents.

[0207] In some embodiments, the voiceprint recognition of the voice input and matching of pre-stored user preference parameters includes: extracting voiceprint feature vectors from the voice input signal; comparing the voiceprint feature vectors with a locally stored voiceprint feature library one by one; when the similarity reaches a preset threshold, obtaining the user preference parameters associated with the corresponding voiceprint feature; when the similarity does not reach the preset threshold, using the system's default user preference parameters; and uploading new unmatched voiceprint features to the cloud for storage and subsequent registration.

[0208] In some embodiments, the step of inputting the parsed multiple control requirements and matching user preference parameters into a pre-trained user preference model to generate air conditioning control instructions and cross-device linkage instructions includes: determining the corresponding scenario type based on the structured control requirement list; adjusting the basic operating parameters of the corresponding scenario type in combination with the user preference parameters; generating air conditioning control instructions containing execution order and execution time; and generating cross-device linkage instructions that match the scenario type, wherein the cross-device linkage instructions contain the target device identifier and the corresponding operation instructions.

[0209] In some embodiments, sending air conditioning control commands to the air conditioning actuator to control the air conditioning to perform temperature adjustment and mode switching operations includes: the air conditioning actuator receiving the air conditioning control commands; adjusting the operating temperature and operating mode of the air conditioning according to the air conditioning control commands; collecting indoor temperature and humidity data through temperature and humidity sensors; collecting indoor air quality data through air quality sensors; fine-tuning the air conditioning operating parameters in real time based on the collected temperature and humidity data and air quality data; and sending the real-time operating status data of the air conditioning to the system coordination module.

[0210] In some embodiments, sending cross-device linkage instructions to associated smart devices to control the associated smart devices to perform corresponding scenario operations includes: sending corresponding operation instructions to each associated smart device according to the target device identifier in the cross-device linkage instruction; receiving execution status information returned by each associated smart device; summarizing the execution status information of all associated smart devices; and sending the summarized execution status information to the system collaboration module for end-to-end synchronization.

[0211] In some embodiments, recording the user's current control behavior and environmental parameters, and updating the user preference model, includes: recording the occurrence time, control request content, and final execution result of the user's current control behavior; recording indoor temperature and humidity data, indoor air quality data, and outdoor environmental data during the current control process; inputting all recorded data into the user preference model; adjusting the weight parameters of corresponding features in the user preference model; generating an updated user preference model; and synchronously storing the updated user preference model on the local device and the cloud server.

[0212] In some embodiments, before inputting the parsed multiple control requirements and matching user preference parameters into the pre-trained user preference model, the method further includes: collecting historical control behavior data and corresponding environmental parameter data of multiple users; cleaning the collected data to remove abnormal and duplicate data; labeling the cleaned data, including control requirements, scenario types, and user satisfaction; dividing the labeled data into training and validation sets; using the training set to iteratively train the initial neural network model; using the validation set to validate the accuracy of the trained model; when the validation accuracy reaches a preset standard, generating the pre-trained user preference model; and distributing the pre-trained user preference model to all connected local devices.

[0213] It should be noted that, for the sake of convenience and brevity, the specific working processes of the air conditioning multi-scenario AI deep dialogue control system and its modules described above can be found in the corresponding contents of the various embodiments of the air conditioning multi-scenario AI deep dialogue control method, and will not be repeated here.

[0214] The aforementioned multi-scenario AI deep dialogue control method for air conditioners can be implemented as a computer program, which can be used in, for example... Figure 3 It runs on the device shown.

[0215] Please see Figure 4 , Figure 4 This is a schematic block diagram of the structure of a computer device provided in an embodiment of this application. The computer device includes a processor, a memory, and a network interface connected via a device bus, wherein the memory may include a storage medium and internal memory.

[0216] The storage medium can store operating devices and computer programs. The computer program includes program instructions that, when executed, cause the processor to perform any multi-scenario AI deep dialogue control method for air conditioning.

[0217] The processor provides computing and control capabilities, supporting the operation of the entire computer device.

[0218] The internal memory provides an environment for the execution of computer programs in non-volatile storage media. When the computer program is executed by the processor, it enables the processor to execute any kind of air conditioning multi-scenario AI deep dialogue control method.

[0219] This network interface is used for network communication, such as sending assigned tasks. Those skilled in the art will understand that... Figure 4The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the terminal to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0220] It should be understood that the processor can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among these, a general-purpose processor can be a microprocessor or any conventional processor.

[0221] In one embodiment, the processor is configured to run a computer program stored in memory to perform the following steps: Receive control input information sent by the user through preset methods; the preset methods include voice input, gesture input, and mobile application input; perform intent recognition on the received control input information and parse it to obtain the corresponding control requirements; Voice input is processed by voiceprint recognition and matched with pre-stored user preference parameters. Multiple control requirements and matching user preference parameters are then input into a pre-trained user preference model to generate air conditioning control commands and cross-device linkage commands. Air conditioning control commands are sent to the air conditioning actuator to control the air conditioning to perform temperature adjustment and mode switching operations. Cross-device linkage commands are sent to associated smart devices to control the associated smart devices to perform corresponding scene operations. Send personalized voice feedback to the user's corresponding user terminal to determine the corresponding control execution status; record the user's current control behavior and environmental parameters, and update the user preference model.

[0222] In some embodiments, receiving control input information sent by the user in a preset manner includes: acquiring the user's voice input signal through a microphone array and performing noise reduction and enhancement processing on the voice input signal; acquiring the user's gesture input image through a camera and performing frame extraction and feature preprocessing on the gesture input image; receiving digital control signals sent by a mobile application through a wireless communication module; and uniformly converting the processed voice input signal, gesture input image, and digital control signal into control input information in a standard format.

[0223] In some embodiments, the step of performing intent recognition and parsing on the received control input information to obtain the corresponding control requirements includes: performing natural language processing and visual semantic parsing on the control input information; extracting multiple independent control intents contained in the control input information; associating historical interaction context information to complete the missing parameters of each control intent; verifying the executability of each control intent and filtering invalid control intents; and generating a structured list of control requirements containing all valid control intents.

[0224] In some embodiments, the voiceprint recognition of the voice input and matching of pre-stored user preference parameters includes: extracting voiceprint feature vectors from the voice input signal; comparing the voiceprint feature vectors with a locally stored voiceprint feature library one by one; when the similarity reaches a preset threshold, obtaining the user preference parameters associated with the corresponding voiceprint feature; when the similarity does not reach the preset threshold, using the system's default user preference parameters; and uploading new unmatched voiceprint features to the cloud for storage and subsequent registration.

[0225] In some embodiments, the step of inputting the parsed multiple control requirements and matching user preference parameters into a pre-trained user preference model to generate air conditioning control instructions and cross-device linkage instructions includes: determining the corresponding scenario type based on the structured control requirement list; adjusting the basic operating parameters of the corresponding scenario type in combination with the user preference parameters; generating air conditioning control instructions containing execution order and execution time; and generating cross-device linkage instructions that match the scenario type, wherein the cross-device linkage instructions contain the target device identifier and the corresponding operation instructions.

[0226] In some embodiments, sending air conditioning control commands to the air conditioning actuator to control the air conditioning to perform temperature adjustment and mode switching operations includes: the air conditioning actuator receiving the air conditioning control commands; adjusting the operating temperature and operating mode of the air conditioning according to the air conditioning control commands; collecting indoor temperature and humidity data through temperature and humidity sensors; collecting indoor air quality data through air quality sensors; fine-tuning the air conditioning operating parameters in real time based on the collected temperature and humidity data and air quality data; and sending the real-time operating status data of the air conditioning to the system coordination module.

[0227] In some embodiments, sending cross-device linkage instructions to associated smart devices to control the associated smart devices to perform corresponding scenario operations includes: sending corresponding operation instructions to each associated smart device according to the target device identifier in the cross-device linkage instruction; receiving execution status information returned by each associated smart device; summarizing the execution status information of all associated smart devices; and sending the summarized execution status information to the system collaboration module for end-to-end synchronization.

[0228] In some embodiments, recording the user's current control behavior and environmental parameters, and updating the user preference model, includes: recording the occurrence time, control request content, and final execution result of the user's current control behavior; recording indoor temperature and humidity data, indoor air quality data, and outdoor environmental data during the current control process; inputting all recorded data into the user preference model; adjusting the weight parameters of corresponding features in the user preference model; generating an updated user preference model; and synchronously storing the updated user preference model on the local device and the cloud server.

[0229] In some embodiments, before inputting the parsed multiple control requirements and matching user preference parameters into the pre-trained user preference model, the method further includes: collecting historical control behavior data and corresponding environmental parameter data of multiple users; cleaning the collected data to remove abnormal and duplicate data; labeling the cleaned data, including control requirements, scenario types, and user satisfaction; dividing the labeled data into training and validation sets; using the training set to iteratively train the initial neural network model; using the validation set to validate the accuracy of the trained model; when the validation accuracy reaches a preset standard, generating the pre-trained user preference model; and distributing the pre-trained user preference model to all connected local devices.

[0230] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to implement the steps of the air conditioning multi-scene AI deep dialogue control method provided in any embodiment of this application.

[0231] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiments, such as the hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, SmartMedia Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the computer device.

[0232] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for multi-scenario AI deep dialogue control of air conditioners, characterized in that, include: Receive control input information sent by the user through a preset method; The preset methods include voice input, gesture input, and mobile application input; the received control input information is subjected to intent recognition and parsing to obtain the corresponding control requirements; Voice input is processed by voiceprint recognition and matched with pre-stored user preference parameters. Multiple control requirements and matching user preference parameters are then input into a pre-trained user preference model to generate air conditioning control commands and cross-device linkage commands. Air conditioning control commands are sent to the air conditioning actuator to control the air conditioning to perform temperature adjustment and mode switching operations. Cross-device linkage commands are sent to associated smart devices to control the associated smart devices to perform corresponding scene operations. Send personalized voice feedback to the user's corresponding user terminal to determine the corresponding control execution status; record the user's current control behavior and environmental parameters, and update the user preference model.

2. The method according to claim 1, characterized in that, The receiving of control input information sent by the user through a preset method includes: The system collects the user's voice input signal through a microphone array and performs noise reduction and enhancement processing on the voice input signal. The system captures user gesture input images using a camera, and then performs frame extraction and feature preprocessing on these images. The wireless communication module receives digital control signals sent by mobile applications. The processed voice input signals, gesture input images, and digital control signals are uniformly converted into control input information in a standard format.

3. The method according to claim 1, characterized in that, The process of performing intent recognition on the received control input information and parsing it to obtain the corresponding control requirements includes: Perform natural language processing and visual semantic parsing on control input information; Extract multiple independent control intentions contained in the control input information; Associate historical interaction context information to complete the missing parameters for each control intent; Verify the executability of each control intent and filter out invalid control intents; Generate a structured list of control requirements containing all valid control intentions.

4. The method according to claim 1, characterized in that, The process of performing voiceprint recognition on the voice input and matching it with pre-stored user preference parameters includes: Extract the voiceprint feature vector from the speech input signal; The voiceprint feature vector is compared one by one with the locally stored voiceprint feature database. When the similarity reaches a preset threshold, the user preference parameters associated with the corresponding voiceprint features are obtained. When the similarity comparison does not reach the preset threshold, the system's default user preference parameters are used. New unmatched voiceprint features are uploaded to the cloud for storage and subsequent registration.

5. The method according to claim 1, characterized in that, The process involves inputting the parsed multiple control requirements and matching user preference parameters into a pre-trained user preference model to generate air conditioning control commands and cross-device linkage commands, including: Determine the corresponding scenario type based on the structured control requirements list; Adjust the basic operating parameters for the corresponding scenario type based on user preference parameters; Generate air conditioning control instructions containing execution order and execution time; Generate cross-device linkage instructions that match the scene type. The cross-device linkage instructions include the target device identifier and the corresponding operation instructions.

6. The method according to claim 1, characterized in that, Sending air conditioning control commands to the air conditioning actuator to control the air conditioning to perform temperature adjustment and mode switching operations includes: The air conditioning actuator receives air conditioning control commands; Adjust the air conditioner's operating temperature and mode according to the air conditioner control commands; Indoor temperature and humidity data are collected using temperature and humidity sensors; Indoor air quality data is collected using air quality sensors; The air conditioning operating parameters are fine-tuned in real time based on the collected temperature, humidity and air quality data; The real-time operating status data of the air conditioner is sent to the system collaboration module.

7. The method according to claim 1, characterized in that, Sending cross-device linkage commands to associated smart devices to control them to perform corresponding scenario operations includes: Based on the target device identifier in the cross-device linkage command, send the corresponding operation command to each associated smart device; Receive execution status information returned by each associated smart device; Summarize the execution status information of all associated smart devices; The aggregated execution status information is sent to the system collaboration module for end-to-end synchronization.

8. The method according to claim 1, characterized in that, The process of recording the user's current control behavior and environmental parameters, and updating the user preference model, includes: Record the time of occurrence, content of control request, and final execution result of the user's control action; Record indoor temperature and humidity data, indoor air quality data, and outdoor environmental data during this control process; Input all recorded data into the user preference model; Adjust the weight parameters of the corresponding features in the user preference model; Generate an updated user preference model; The updated user preference model is synchronized and stored on local devices and cloud servers.

9. The method according to claim 1, characterized in that, Before inputting the parsed multiple control requirements and matched user preference parameters into the pre-trained user preference model, the method further includes: Collect historical control behavior data and corresponding environmental parameter data from multiple users; The collected data is cleaned to remove abnormal and duplicate data. The cleaned data is labeled, including control requirements, scenario type, and user satisfaction. The labeled data is divided into a training set and a validation set; The initial neural network model is trained iteratively using the training set; Use the validation set to validate the accuracy of the trained model; When the verification accuracy reaches the preset standard, a pre-trained user preference model is generated; The pre-trained user preference model is distributed to all connected local devices.

10. An air conditioning multi-scenario AI deep dialogue control system, used to implement the method as described in any one of claims 1-9, characterized in that, include: The information receiving unit is used to receive control input information sent by the user in a preset manner; The preset methods include voice input, gesture input, and mobile application input; the received control input information is subjected to intent recognition and parsing to obtain the corresponding control requirements; The voiceprint recognition unit is used to perform voiceprint recognition on voice input and match it with pre-stored user preference parameters; it inputs the parsed multiple control requirements and the matched user preference parameters into a pre-trained user preference model to generate air conditioning control commands and cross-device linkage commands; it sends air conditioning control commands to the air conditioning actuator to control the air conditioning to perform temperature adjustment and mode switching operations; it sends cross-device linkage commands to associated smart devices to control the associated smart devices to perform corresponding scene operations. The model update unit is used to send personalized voice feedback to the user's corresponding user terminal to determine the corresponding control execution status; record the user's current control behavior and environmental parameters, and update the user preference model.

Images