Smart home cooperative control system and method based on multi-mode interactive charging pile

Abstract

The invention relates to the technical field of intelligent Internet of Things, and discloses a smart home cooperative control system and method based on a multi-mode interactive charging pile. According to the system, by integrating a multi-mode interaction module and intelligent monitoring, the household comprehensive energy efficiency is improved by about 40%, the valley electricity utilization rate exceeds 90%, the charging loss is reduced to below 5%, the energy use efficiency is optimized, the recognition accuracy of abnormal events is remarkably improved through an intelligent sensor and real-time video analysis in the aspect of safety protection, the response time is controlled within 30 seconds, and the safety protection effect is good. Meanwhile, the system has the instruction recognition accuracy as high as 95% and the multi-device response time faster than 2 seconds, the user experience is greatly improved, complex semantic analysis is supported, the naturalness and flexibility of interaction are enhanced, seamless connection between devices of different manufacturers is achieved through the protocol conversion module, the communication problem caused by a heterogeneous protocol is solved, and the system is suitable for popularization and application. The efficient cooperation of the charging pile and the smart home equipment is ensured, and the deep integration of the smart home and the electric vehicle charging is promoted.

Description

Technical Field

[0001] The present invention relates to the field of intelligent Internet of Things technology, and specifically to a smart home collaborative control system and method based on a multimodal interactive charging pile. Background Art

[0002] With the popularity of electric vehicles and the development of smart homes, the demand for collaboration between charging piles and smart home devices is growing. However, many problems currently limit their coordinated development. First, charging piles usually operate independently and cannot effectively interact with household electrical devices. As a result, during peak household electricity consumption, charging piles may be used simultaneously with other high-power appliances, causing power load conflicts, affecting household power stability, and even causing electrical failures. Secondly, in a dynamic electricity pricing environment, existing charging strategies lack intelligent scheduling capabilities. Users are unable to rationally schedule charging times based on real-time electricity price information, making it difficult to fully utilize low-price off-peak periods for charging. This results in high charging costs and prevents efficient and economical energy utilization. In addition, smart home devices and charging piles often come from different manufacturers and use multiple heterogeneous protocols, which makes communication and collaboration between devices face huge challenges. For example, the OCPP protocol used by charging piles is quite different from the Zigbee protocol commonly used in smart home devices. Protocol conversion is difficult, which seriously hinders the interconnection and collaboration between multiple devices. Summary of the Invention

[0003] In response to the shortcomings of the existing technology, the present invention provides a smart home collaborative control system based on multimodal interactive charging piles, which can realize three-dimensional collaborative control of energy scheduling, safety protection, and human-computer interaction through intelligent linkage between smart homes and charging piles.

[0004] To achieve the above objectives, the present invention provides the following technical solutions: a smart home collaborative control system based on a multimodal interactive charging pile, comprising a multimodal interaction module, a security protection module, a protocol conversion module, an energy scheduling module, a privacy processing module, and an integrated control module; The multimodal interaction module collects user voice commands through a voice interaction device and converts them into text data through voice recognition technology. At the same time, it collects real-time video streams of charging scenes through smart cameras and analyzes them together with the current waveform data of the charging piles, using computer vision technology to identify charging status, user behavior, and environmental changes. The safety protection module uses smart sensors to monitor the status parameters of the charging pile and its surrounding environment in real time, analyzes the charging scene video collected by the smart camera in real time, and uses image processing algorithms to determine whether there are abnormal conditions such as electrical fire, illegal intrusion, and equipment leakage in the charging scene. According to different safety risks, a three-level response mechanism is established. When an abnormality is detected, the local sound and light alarm, dynamic adjustment of charging power, and V2H emergency power supply mode are activated in sequence; The protocol conversion module effectively converts data formats between different communication protocols by establishing a data conversion mechanism for input and output protocols, and automatically identifies and adapts to the communication protocol of the newly connected device through a dynamic protocol discovery mechanism using a reinforcement learning algorithm; The energy scheduling module is used to obtain real-time information on grid load, real-time electricity prices, and the operating status of household electrical appliances, and evaluate users' charging needs through data analysis. By applying a rolling horizon optimization algorithm, it dynamically adjusts the charging power to optimize users' charging costs, and evaluates the operating status of interruptible household appliances through a household load elasticity classification model. The privacy processing module is used to perform privacy protection processing on the data generated by charging and the user's voice commands; The integrated control module connects the multimodal interaction module, security protection module, protocol conversion module, energy scheduling module and privacy processing module through the home network to form a unified smart home management and scheduling platform.

[0005] Preferably, the conversion of the user's voice command into text data is performed by the following formula:

[0006] In the formula, Represents the converted text data, Represents the collected voice signal, represents a deep neural network for speech recognition, Indicates recognition error.

[0007] Preferably, the formula for analyzing the video stream of the charging scene collected in real time by the smart camera is as follows:

[0008] In the formula, represents the output of the joint visual analysis, Indicates that the video stream is based on and current waveform The output of the joint model is used to identify charging status, anomalies or environmental changes, Indicates the deviation that occurs during the multimodal fusion process.

[0009] Preferably, the safety protection module performs safety risk assessment of the charging scenario by applying the following formula:

[0010] In the formula, Indicates the probability of detecting an anomaly, between 0 and 1. express activation function, Represents the sensor state vector The corresponding weight vector The dot product of Indicates the abnormality score and corresponding weight obtained by video analysis, Represents the bias term, which is used to adjust the output threshold of the model.

[0011] Preferably, the safety protection module also establishes a safety protection system, applying a three-level fuse control logic: Level 1 response: local sound and light alarm (response time < 1s); Secondary response: dynamic power regulation (gradient load reduction speed: 10kw / s); Level 3 response: V2H emergency power supply switching (switching time < 200ms).

[0012] Preferably, the protocol conversion module performs dynamic data conversion using the following formula:

[0013] In the formula, Indicates the converted protocol data. Indicates that based on input protocol data and model parameters Complete data format conversion, Indicates the error generated during the conversion process.

[0014] Preferably, the protocol conversion module automatically identifies the new protocol using the following formula:

[0015] In the formula, Indicates the updated new protocol identification and adaptation model parameters, Represents a historical protocol sample set As input, through the reinforcement learning model The learned strategy is used to identify and adapt to new protocol changes.

[0016] Preferably, the energy scheduling module evaluates user charging demand using the following formula:

[0017] In the formula, represents the charging demand assessment value, Indicates the current grid load. Indicates the maximum load of the power grid, represents the current electricity price, represents the maximum electricity price, Indicates the operating status of the home elastic device, expressed as 0-1. 、 The corresponding weight coefficient is automatically assigned by the system.

[0018] Preferably, the dynamic adjustment of charging power is achieved by applying the following formula:

[0019] In the formula, Indicates the adjusted charging power, Indicates the maximum allowed charging power, Indicates the maximum load of the power grid, Indicates the grid load safety threshold, Indicates the current actual grid load. Indicates the dynamic adjustment coefficient of charging power.

[0020] Preferably, the privacy processing module establishes a privacy protection mechanism, and performs user data privacy processing by applying a data desensitization solution, wherein user charging data is processed Anonymization processing: users' voice commands are desensitized by differential privacy noise injection.

[0021] The present invention provides a smart home collaborative control method based on a multimodal interactive charging pile, comprising the following steps: The user's voice commands are collected through voice interaction devices and converted into text data through voice recognition technology. At the same time, the smart camera collects real-time video streams of the charging scene and combines them with the current waveform data of the charging pile for analysis. Computer vision technology is used to identify charging status, user behavior, and environmental changes. Smart sensors monitor the status parameters of charging piles and their surroundings in real time, analyze videos of charging scenes captured by smart cameras, and apply image processing algorithms to determine whether there are any abnormalities such as electrical fires, illegal intrusions, or equipment leakage. A three-level response mechanism is established for different security risks. When an anomaly is detected, local sound and light alarms, dynamic charging power adjustment, and V2H emergency power supply mode are activated in sequence. By establishing a data conversion mechanism for input and output protocols, the data formats between different communication protocols are effectively converted. Through a dynamic protocol discovery mechanism, reinforcement learning algorithms are used to automatically identify and adapt to the communication protocols of newly connected devices. Obtain real-time information on grid load, real-time electricity prices, and the operating status of household electrical appliances, and evaluate users' charging needs through data analysis. By applying a rolling time domain optimization algorithm, the charging power is dynamically adjusted to optimize users' charging costs. The operating status of interruptible household appliances is evaluated through a household appliance load elasticity grading model. Privacy protection is performed on charging data and user voice commands.

[0022] Compared with the existing technology, the present invention provides a smart home collaborative control system based on multimodal interactive charging piles, which has the following beneficial effects: By integrating multimodal interaction modules with intelligent monitoring, the present invention improves the overall energy efficiency of households by about 40%, and the utilization rate of off-peak electricity exceeds 90%, and the charging loss is reduced to less than 5%, which greatly optimizes the energy utilization efficiency. In terms of safety protection, through intelligent sensors and real-time video analysis, the accuracy of identifying abnormal events is significantly improved, and the response time is controlled within 30 seconds to ensure family safety. At the same time, the system has a command recognition accuracy of up to 95% and a multi-device response time of faster than 2 seconds, which greatly improves the user experience, supports complex semantic analysis, and enhances the naturalness and flexibility of interaction. The protocol conversion module realizes seamless connection between devices from different manufacturers, solves the communication problems brought by heterogeneous protocols, ensures the efficient collaboration between charging piles and smart home devices, and promotes the deep integration of smart homes and electric vehicle charging. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 The figure is a flow chart of a smart home collaborative control system based on a multimodal interactive charging pile in an embodiment of the present invention.

[0024] Figure 2 The figure is a flow chart of a smart home collaborative control method based on a multimodal interactive charging pile in an embodiment of the present invention. DETAILED DESCRIPTION

[0025] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0026] The current independent operation of charging piles leads to conflicts with household electricity loads, lacks intelligent scheduling strategies under dynamic electricity prices, and the heterogeneity of charging devices makes protocol conversion difficult, making it impossible to solve the problem of multi-device collaboration. Therefore, a smart home collaborative control system based on multimodal interactive charging piles is proposed. Figure 1 ,The system includes a multimodal interaction module, a security protection module, a protocol conversion module, an energy scheduling module, a privacy processing module, and an integrated control module; The multimodal interaction module is a core component of smart charging facilities. By integrating voice interaction devices and advanced computer vision technology, it achieves efficient perception and understanding of user needs and the charging environment. The module first uses highly sensitive voice acquisition equipment to collect user voice commands. With the help of deep learning-driven speech recognition technology, the user's voice data is converted into text information, greatly enhancing the natural interaction experience between people and the charging system. At the same time, the intelligent camera continuously monitors the charging scene. By collecting real-time video streams, combining them with the current waveform data of the charging pile, and using computer vision models for joint analysis, it can achieve accurate recognition of multi-dimensional information such as charging status, user behavior, and environmental changes. Specifically, the visual model will identify whether the user is operating the charging equipment correctly, detect any abnormal behavior or safety hazards (such as fire or intrusion), and capture dynamic changes in the external environment. The results of this joint analysis process provide structured multi-source information required for system decision-making, enhancing scene perception capabilities. At the algorithm level, the above visual analysis model is expressed as:

[0027] in, represents a deep learning model that integrates multimodal information, This reflects the errors or deviations that may exist in the model during the fusion and recognition process. The greatest advantage of this joint processing of multi-source information is that it can fully understand the status of the charging site from different angles and dimensions, significantly improving the accuracy of anomaly detection and behavior recognition, and providing a solid data foundation for subsequent safety warnings, intelligent scheduling and other links. Ultimately, it realizes the naturalization of human-computer interaction, comprehensive scene perception and intelligent charging system. Through this integrated perception mechanism, not only the safety and reliability of the system are improved, but also the user experience and operational efficiency are enhanced, laying a solid technical foundation for the widespread application of smart charging infrastructure in the future. The safety protection module integrates high-precision intelligent sensors to monitor the charging pile and its surrounding environment in real time, including key status parameters such as temperature, current, voltage, and vibration, to ensure the safe operation of the system under various operating conditions. At the same time, the visual monitoring system uses advanced image processing and target recognition algorithms to detect abnormal behaviors in the scene in real time, such as electrical fires, illegal intrusions, or equipment leakage, and other potential risks. When identifying these anomalies, deep learning and image analysis models (such as image classification and target detection algorithms) are applied. Its core formula can be expressed as follows:

[0028] in, and Represents the parameters obtained from model training, corresponding to the weights of sensor data and video anomaly scores, is the bias term, The activation function (Sigmoid) outputs a risk probability value. When a risk probability higher than a preset threshold (e.g., 70%) is detected, the system will gradually initiate response measures according to a pre-designed three-level response mechanism to ensure the safety of the charging environment. Specifically, the first response is a fast-response local audible and visual alarm (response time less than 1 second) to quickly alert on-site personnel to potential dangers. The second response is dynamic adjustment of charging power. Its adjustment strategy uses a gradient load reduction method (load reduction rate of approximately 10kW / s) to gradually reduce equipment load and alleviate potential electrical hazards or overload risks. The third response is V2H (Vehicle-to-Home) emergency power supply switching. When the output device detects a serious danger, it can quickly switch to the backup power source within 200 milliseconds to ensure the continuous power supply and safety of the entire system and user devices. The safety system also establishes a three-level fuse control logic (fuse mechanism) to prevent accidents from escalating: the first-level response uses local audible and visual alarms to ensure extremely fast reaction speeds (less than 1 second). The second-level regulation uses gradient load reduction control to reduce equipment load and mitigate electrical risks (load reduction rate is approximately 10kW per second). The third-level switching provides emergency power supply, ensuring that in severe abnormal situations such as fire and electrical failures, the system can switch to backup power in an extremely short time (less than 200ms), effectively preventing the spread of accidents and equipment damage. This complete multi-level, multi-link safety protection system effectively improves the safety level of charging venues and the overall reliability of the system through multimodal perception, intelligent analysis and rapid response mechanisms, providing users with a safer, more stable and intelligent charging experience. The protocol conversion module plays a key role in the smart charging infrastructure. It ensures that devices from different manufacturers and with different protocol standards can interconnect and work seamlessly by setting up a flexible and efficient data conversion mechanism between input and output protocols. The system uses deep learning or reinforcement learning algorithms to dynamically learn and optimize protocol conversion strategies, achieving automated and intelligent protocol mapping, thereby greatly reducing manual debugging and maintenance costs. At the same time, in order to meet the actual needs of continuous protocol evolution and continuous device access, the protocol conversion module also introduces a dynamic protocol discovery mechanism. It uses reinforcement learning to continuously accumulate historical protocol samples, identify potential protocol changes, and automatically adapt to the access of new protocols. The core formula of this mechanism can be expressed as:

[0029] in, Represents the parameters The protocol mapping model generated by the learned conversion strategy, This formula reflects the error or deviation in data conversion. It reflects the ability to continuously optimize protocol conversion through learning models, reduce human intervention, and improve the system's adaptability and scalability. It introduces a dynamic protocol discovery mechanism and uses reinforcement learning to automatically identify and adapt to new protocol formats, avoiding the limitations of traditional static rules and significantly enhancing the system's intelligence level. Overall, this dynamic and automated protocol conversion solution not only greatly improves device interoperability and system compatibility, but also ensures stable system operation and future expansion in an environment where device protocols are rapidly evolving, providing a solid technical guarantee for the popularization and intelligent upgrade of smart charging infrastructure. The energy scheduling module plays a core regulatory role in the smart charging system. It can formulate scientific and reasonable charging strategies for users by real-time monitoring of key parameters such as grid load, dynamic electricity prices, and the operating status of household electrical appliances. Through highly integrated data analysis technology, this module evaluates users' charging needs from complex multi-source data, taking into account current load, cost, and equipment flexibility, providing a basis for subsequent scheduling. In terms of specific algorithms, a rolling time domain optimization algorithm is used to dynamically adjust the charging power based on real-time data and forecast information to ensure that while meeting user charging needs, electricity expenses and load impact on the grid are minimized. This optimization process can be expressed by the following formula:

[0030] in, is the maximum load capacity of the grid (e.g. megawatt level), is the safe load threshold (to ensure safe operation of the system), and It is a dynamically adjusted optimization coefficient that changes dynamically based on time, load trends, and forecast results. In order to reasonably evaluate the scheduling potential of various flexible devices in the home, the module also introduces a household appliance load elasticity grading model. According to the elasticity level of different devices (0-1), the operating status of interruptible devices is graded and evaluated, thereby achieving flexible load regulation and improving overall energy efficiency and scheduling adaptability. Through the above multi-factor and multi-dimensional scheduling strategy, the energy scheduling module can not only effectively improve the load balance of the power grid and reduce peak load pressure, but also reduce user charging costs and improve energy utilization efficiency, ultimately achieving the goal of a smart, green, and economical charging ecosystem. This dynamic optimization mechanism, combined with real-time monitoring and elastic regulation, provides solid technical support for the smooth and efficient operation of future smart grids. The privacy processing module plays a key role in protecting user data security and privacy. It introduces a multi-level and multi-faceted privacy protection mechanism to ensure that all types of data generated during the charging process and the user's voice commands will not be accessed or leaked without authorization. Specifically, a data desensitization solution is used for the user's charging data (such as identity information, location, charging time, and power consumption). First, anonymization is implemented to remove or obfuscate identifiable user identity information and eliminate direct correlation, thereby improving data privacy security. At the same time, for the user's voice commands, differential privacy technology is used to inject designed noise into the original voice data to meet the differential privacy standard ( -Differential privacy) ensures that even if leaked, it is difficult to reversely infer user sensitive information. Specific technical implementations include applying privacy protection algorithms before data is uploaded or stored, such as:

[0031] Among them, noise Generated according to Laplace or Gaussian distributions, with parameters adjusted to balance privacy protection strength and data practicality, this module also integrates access control and encrypted transmission measures (such as TLS / SSL protocols) to provide multi-layered security for user data during communication and storage. This integration of a series of technical measures not only significantly reduces the risk of data leakage and abuse, but also meets the strict privacy requirements of data protection regulations (such as GDPR and LGPD), providing users with a safe and reliable charging experience while supporting the system's compliant operation in a big data environment. As the core hub of the smart home management system, the integrated control module mainly realizes the seamless connection and coordination of the multimodal interaction module, security protection module, protocol conversion module, energy scheduling module and privacy processing module through efficient communication protocols based on the home network (such as Wi-Fi, Zigbee, Z-Wave or Ethernet). The module adopts industrial-grade LAN technology, IoT communication protocols and intelligent gateway devices (such as embedded controllers and edge computing nodes) to ensure efficient and reliable data transmission and command control between various subsystems. Through an integrated software platform (such as a cloud management platform or a local IoT control center), it realizes the unified scheduling, monitoring and collaboration of various modules in the home environment. For example, the integrated control module will dispatch energy resources, trigger security warnings, manage user interactions, and protect user data privacy according to real-time control instructions, and coordinate the compatibility issues of protocol conversion. It also embeds intelligent algorithms to realize multiple functions such as equipment status monitoring, fault diagnosis, and energy efficiency optimization. As the core of the overall system, this integrated control module effectively integrates the efficient human-computer interaction experience of multimodal interaction, the solid guarantee of security protection mechanism, the flexible compatibility of protocol conversion, the intelligent optimization of energy scheduling and the data security of privacy protection, forming an efficient, reliable and intelligent home management and scheduling platform. This platform can not only realize the coordinated operation and intelligent control of various devices and modules in the home environment, but also continuously improve the system performance and user experience through data analysis and algorithm optimization, and meet the multiple requirements of future smart home systems for security, privacy protection, energy efficiency and interactive experience. Overall, this integrated control system uses the integration of software and hardware and data-driven methods to build a highly integrated, intelligent and scalable home ecosystem, providing home users with a safe, convenient, energy-saving and personalized living environment, and promoting the popularization and upgrading of smart homes to a higher level of intelligence.

[0032] The present invention provides a smart home collaborative control method based on a multimodal interactive charging pile, comprising the following steps: The user's voice commands are collected through voice interaction devices and converted into text data through voice recognition technology. At the same time, the smart camera collects real-time video streams of the charging scene and combines them with the current waveform data of the charging pile for analysis. Computer vision technology is used to identify charging status, user behavior, and environmental changes. Smart sensors monitor the status parameters of charging piles and their surroundings in real time, analyze videos of charging scenes captured by smart cameras, and apply image processing algorithms to determine whether there are any abnormalities such as electrical fires, illegal intrusions, or equipment leakage. A three-level response mechanism is established for different security risks. When an anomaly is detected, local sound and light alarms, dynamic charging power adjustment, and V2H emergency power supply mode are activated in sequence. By establishing a data conversion mechanism for input and output protocols, the data formats between different communication protocols are effectively converted. Through a dynamic protocol discovery mechanism, reinforcement learning algorithms are used to automatically identify and adapt to the communication protocols of newly connected devices. Obtain real-time information on grid load, real-time electricity prices, and the operating status of household electrical appliances, and evaluate users' charging needs through data analysis. By applying a rolling time domain optimization algorithm, the charging power is dynamically adjusted to optimize users' charging costs. The operating status of interruptible household appliances is evaluated through a household appliance load elasticity grading model. Privacy protection is performed on charging data and user voice commands.

[0033] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A smart home collaborative control system based on a multimodal interactive charging pile, characterized by: It includes multimodal interaction module, security protection module, protocol conversion module, energy scheduling module, privacy processing module and integrated control module; The multimodal interaction module collects user voice commands through a voice interaction device and converts them into text data through voice recognition technology. At the same time, it collects real-time video streams of charging scenes through smart cameras and analyzes them together with the current waveform data of the charging piles, using computer vision technology to identify charging status, user behavior, and environmental changes. The safety protection module uses smart sensors to monitor the status parameters of the charging pile and its surrounding environment in real time, analyzes the charging scene video collected by the smart camera in real time, and uses image processing algorithms to determine whether there are abnormal conditions such as electrical fire, illegal intrusion, and equipment leakage in the charging scene. According to different safety risks, a three-level response mechanism is established. When an abnormality is detected, the local sound and light alarm, dynamic adjustment of charging power, and V2H emergency power supply mode are activated in sequence; The protocol conversion module effectively converts data formats between different communication protocols by establishing a data conversion mechanism for input and output protocols, and automatically identifies and adapts to the communication protocol of the newly connected device through a dynamic protocol discovery mechanism using a reinforcement learning algorithm; The energy scheduling module is used to obtain real-time information on grid load, real-time electricity prices, and the operating status of household electrical appliances, and evaluate users' charging needs through data analysis. By applying a rolling horizon optimization algorithm, it dynamically adjusts the charging power to optimize users' charging costs, and evaluates the operating status of interruptible household appliances through a household load elasticity classification model. The privacy processing module is used to perform privacy protection processing on the data generated by charging and the user's voice commands; The integrated control module connects the multimodal interaction module, security protection module, protocol conversion module, energy scheduling module and privacy processing module through the home network to form a unified smart home management and scheduling platform.

2. The smart home collaborative control system based on a multimodal interactive charging pile according to claim 1, characterized in that: The conversion of the user's voice command into text data is performed by the following formula: In the formula, Represents the converted text data, Represents the collected voice signal, represents a deep neural network for speech recognition, Indicates recognition error.

3. The smart home collaborative control system based on a multimodal interactive charging pile according to claim 2, characterized in that: The formula for analyzing the video stream of the charging scene collected in real time by the smart camera is as follows: In the formula, represents the output of the joint visual analysis, Indicates that the video stream is based on and current waveform The output of the joint model is used to identify charging status, anomalies or environmental changes, Indicates the deviation that occurs during the multimodal fusion process.

4. The smart home collaborative control system based on a multimodal interactive charging pile according to claim 3, characterized in that: The safety protection module performs a safety risk assessment of the charging scenario by applying the following formula: In the formula, Indicates the probability of detecting an anomaly, between 0 and 1. express activation function, Represents the sensor state vector The corresponding weight vector The dot product of Indicates the abnormality score and corresponding weight obtained by video analysis, Represents the bias term, which is used to adjust the output threshold of the model.

5. The smart home collaborative control system based on a multimodal interactive charging pile according to claim 4, characterized in that: The safety protection module also establishes a safety protection system, applying a three-level fuse control logic: Level 1 response: local sound and light alarm (response time < 1s); Secondary response: dynamic power regulation (gradient load reduction speed: 10kw / s); Level 3 response: V2H emergency power supply switching (switching time < 200ms).

6. The smart home collaborative control system based on a multimodal interactive charging pile according to claim 5, characterized in that: The protocol conversion module performs dynamic data conversion using the following formula: In the formula, Indicates the converted protocol data. Indicates that based on input protocol data and model parameters Complete data format conversion, Indicates the error generated during the conversion process.

7. The smart home collaborative control system based on a multimodal interactive charging pile according to claim 6, characterized in that: The protocol conversion module automatically identifies new protocols using the following formula: In the formula, Indicates the updated new protocol identification and adaptation model parameters, Represents a historical protocol sample set As input, through the reinforcement learning model The learned strategy is used to identify and adapt to new protocol changes.

8. The smart home collaborative control system based on a multimodal interactive charging pile according to claim 7, characterized in that: The energy scheduling module evaluates user charging demand using the following formula: In the formula, represents the charging demand assessment value, Indicates the current grid load. Indicates the maximum load of the power grid, represents the current electricity price, represents the maximum electricity price, Indicates the operating status of the home elastic device, expressed as 0-1. 、 The corresponding weight coefficient is automatically assigned by the system.

9. The smart home collaborative control system based on a multimodal interactive charging pile according to claim 8, characterized in that: The dynamic adjustment of charging power is achieved by applying the following formula: In the formula, Indicates the adjusted charging power, Indicates the maximum allowed charging power, Indicates the maximum load of the power grid, Indicates the grid load safety threshold, Indicates the current actual grid load. Indicates the dynamic adjustment coefficient of charging power.

10. A smart home collaborative control method based on multimodal interactive charging piles, characterized in that: The following steps are involved: The user's voice commands are collected through voice interaction devices and converted into text data through voice recognition technology. At the same time, the smart camera collects real-time video streams of the charging scene and combines them with the current waveform data of the charging pile for analysis. Computer vision technology is used to identify charging status, user behavior, and environmental changes. Smart sensors monitor the status parameters of charging piles and their surroundings in real time, analyze videos of charging scenes captured by smart cameras, and apply image processing algorithms to determine whether there are any abnormalities such as electrical fires, illegal intrusions, or equipment leakage. A three-level response mechanism is established for different security risks. When an anomaly is detected, local sound and light alarms, dynamic charging power adjustment, and V2H emergency power supply mode are activated in sequence. By establishing a data conversion mechanism for input and output protocols, the data formats between different communication protocols are effectively converted. Through a dynamic protocol discovery mechanism, reinforcement learning algorithms are used to automatically identify and adapt to the communication protocols of newly connected devices. Obtain real-time information on grid load, real-time electricity prices, and the operating status of household electrical appliances, and evaluate users' charging needs through data analysis. By applying a rolling time domain optimization algorithm, the charging power is dynamically adjusted to optimize users' charging costs. The operating status of interruptible household appliances is evaluated through a household appliance load elasticity grading model. Privacy protection is performed on charging data and user voice commands.

Images