Dynamic energy-saving optimization method based on machine vision and behavior habit prediction

Abstract

The invention provides a dynamic energy-saving optimization method based on machine vision and behavior habit prediction, and relates to the technical field of industrial internet and intelligent manufacturing, and the method comprises the steps: collecting video data, equipment operation data, environment data and time-space data through a vision sensing unit, an equipment and environment sensing unit; based on the video data, the equipment operation data, the environment data and the time-space data, performing fusion analysis through a visual analysis channel and an equipment analysis channel, and constructing a behavior-equipment habit model; based on the behavior-equipment habit model, predicting an energy consumption demand, generating a preliminary control strategy through a multi-objective optimization algorithm, and calling an expert rule base to perform simulation evaluation on the preliminary control strategy; if the simulation evaluation has the potential influence, generating prompt information to request an artificial decision; if the potential influence does not exist, outputting a final control strategy; and executing the final control strategy, and collecting feedback data to optimize and update the behavior-equipment habit model.

Description

Technical Field

[0001] This application belongs to the field of industrial internet and intelligent manufacturing technology, specifically involving a dynamic energy-saving optimization method based on machine vision and behavioral habit prediction. Background Technology

[0002] In the field of industrial internet and smart energy management technology, existing energy-saving solutions for factory areas mostly rely on simple sensor data (such as infrared sensors) or timed control strategies, which have significant limitations. These solutions have a single sensing dimension, only achieving basic presence detection, and cannot obtain rich information such as the number of personnel, their behavior trajectories, and equipment operating status, resulting in insufficient decision-making basis. At the same time, each energy-consuming system (such as lighting, air conditioning, ventilation, etc.) often operates independently, lacking collaborative optimization, making it difficult to achieve overall energy-saving benefits. In addition, most existing technologies are based on passive responses to the current environmental conditions, unable to predict future energy demand based on historical data and behavioral patterns, thus failing to achieve proactive pre-adjustment. More seriously, control strategies are often disconnected from predictive analysis and lack safety assessment mechanisms. Aggressive energy saving may affect core production activities or personnel comfort, posing safety risks. Although some solutions attempt to improve by adding sensors or optimizing single systems, these are costly and fail to solve fundamental problems such as multi-source data fusion, behavioral habit modeling, and safe collaborative control. Summary of the Invention

[0003] This application provides a dynamic energy-saving optimization method based on machine vision and behavioral habit prediction to solve one of the above-mentioned technical problems.

[0004] The technical solution adopted in this application is as follows: This application provides a dynamic energy-saving optimization method based on machine vision and behavioral habit prediction, including: Video data, equipment operation data, environmental data, and spatiotemporal data are collected through visual sensing units, equipment and environmental sensing units. Based on the video data, equipment operation data, environmental data, and spatiotemporal data, a behavior-device habit model is constructed through fusion analysis via visual analysis channel and equipment analysis channel. Based on the behavior-device habit model, energy demand is predicted, a preliminary control strategy is generated through a multi-objective optimization algorithm, and the preliminary control strategy is simulated and evaluated by calling an expert rule base. If the simulated assessment has potential impacts, a prompt message is generated requesting manual decision-making; if there are no potential impacts, the final control strategy is output. The final control strategy is executed, and feedback data is collected to optimize and update the behavior-device habit model.

[0005] According to one embodiment of this application, the visual analysis channel includes: The video data is subjected to target detection, target tracking, and behavior recognition, and the personnel behavior status is output.

[0006] According to one embodiment of this application, the device analysis channel includes: The device operation data is preprocessed and time-series pattern learned to output the device operation rules; The temporal pattern learning employs a long short-term memory network model.

[0007] According to one embodiment of this application, the step of invoking an expert rule base to simulate and evaluate the preliminary control strategy includes: The potential impact of the initial control strategy on core production activities is assessed based on an expert rule base. The expert rule base includes security rules built based on historical data and domain knowledge.

[0008] According to one embodiment of this application, executing the final control strategy includes: The final control strategy is then sent to at least one energy consumption system for execution. The energy consumption system includes at least one of a lighting system, an air conditioning system, a ventilation system, or a water pump system.

[0009] According to one embodiment of this application, the collection of feedback data to optimize and update the behavior-device habit model includes: The parameters of the visual analysis channel and the device analysis channel are adjusted based on the feedback data to update the behavior-device habit model.

[0010] According to one embodiment of this application, it also includes: Before constructing the behavior-device habit model, the video data, device operation data, environmental data, and spatiotemporal data are cleaned and normalized.

[0011] A second aspect of this application provides a dynamic energy-saving optimization device based on machine vision and behavioral habit prediction, comprising: The data acquisition module is used to acquire video data, equipment operation data, environmental data, and spatiotemporal data through the visual sensing unit, equipment and environmental sensing unit; The data processing and analysis module is used to perform fusion analysis on the video data, equipment operation data, environmental data and spatiotemporal data through the visual analysis channel and the equipment analysis channel to construct a behavior-equipment habit model. The core decision-making module is used to predict energy demand based on the behavior-equipment habit model, generate a preliminary control strategy through a multi-objective optimization algorithm, and call an expert rule base to simulate and evaluate the preliminary control strategy; if the simulation evaluation has potential impacts, a prompt message is generated to request manual decision-making; if there are no potential impacts, the final control strategy is output. The control execution module is used to execute the final control strategy and collect feedback data to optimize and update the behavior-device habit model.

[0012] According to one embodiment of this application, the data processing and analysis module includes a visual analysis unit and a time series prediction unit; The visual analysis unit is used to perform target detection, target tracking, and behavior recognition on the video data; The timing prediction unit is used to preprocess the device operation data and learn timing patterns.

[0013] According to one embodiment of this application, the core decision-making module includes a strategy generation unit and an expert evaluation unit; The strategy generation unit is used to generate the preliminary control strategy through a multi-objective optimization algorithm; The expert evaluation unit is used to call the expert rule base to perform simulated evaluations.

[0014] Due to the adoption of the above technical solution, the beneficial effects achieved by this application are as follows: This application collects video data, equipment operation data, environmental data, and spatiotemporal data through a visual sensing unit, equipment and environmental sensing unit, which solves the problem of the single perception dimension in the existing technology, realizes comprehensive monitoring of personnel behavior, equipment status and environmental factors, and provides a rich data foundation for precise energy-saving decision-making.

[0015] By fusing and analyzing multi-source data through visual analysis and equipment analysis channels, a behavior-equipment habit model is constructed. This model effectively learns the correlation between personnel behavior patterns and equipment operating rules, overcoming the problem of existing technologies neglecting behavioral patterns and achieving accurate prediction of energy demand and self-learning of habits.

[0016] Based on the behavior-equipment habit model to predict energy demand, and through the multi-objective optimization algorithm to generate preliminary control strategies, it has achieved a leap from passive energy saving to active prediction and pre-adjustment, solved the problems of poor system coordination and lack of predictive ability, and significantly improved the overall energy efficiency (the energy saving rate is expected to increase by 15%-30%).

[0017] By calling on the expert rule base to simulate and evaluate the initial control strategy, and generating prompts to request manual decision-making when there are potential impacts, it ensures that the energy-saving strategy will not have a negative impact on core production activities, perfectly balancing the contradiction between energy saving and safety, and between energy saving and user experience, and solving the problem of the disconnect between control strategy and safety.

[0018] By executing the final control strategy and collecting feedback data to optimize and update the behavior-device habit model, a closed-loop control and self-learning mechanism is formed, which overcomes the defect of weak system adaptability and enables the energy-saving system to dynamically adapt to environmental changes and behavioral habit evolution, maintaining high efficiency and reliability in the long term. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart illustrating a dynamic energy-saving optimization method based on machine vision and behavioral habit prediction, provided in an embodiment of this application. Detailed Implementation

[0020] To more clearly illustrate the overall concept of this application, a detailed explanation is provided below with reference to the accompanying drawings.

[0021] Many specific details are set forth in the following description to provide a thorough understanding of this application. However, this application may also be implemented in other ways different from those described herein. Therefore, the scope of protection of this application is not limited to the specific embodiments disclosed below. It should be noted that, unless otherwise specified, the embodiments of this application and the features thereof can be combined with each other.

[0022] In this application, unless otherwise expressly specified and limited, the "above" or "below" of the second feature can mean that the first and second features are in direct contact, or that the first and second features are in indirect contact through an intermediate medium. In the description of this specification, references to terms such as "an embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples.

[0023] Example 1 like Figure 1As shown, a dynamic energy-saving optimization method based on machine vision and behavioral habit prediction includes: Video data, equipment operation data, environmental data, and spatiotemporal data are collected through visual sensing units, equipment and environmental sensing units.

[0024] As described above, this step involves comprehensively collecting environmental and device data through a multi-source sensing system, providing a foundation for subsequent behavior-device habit model construction. Visual sensing units (such as high-definition cameras and depth cameras) are responsible for capturing video data, including dynamic human behavior, object movement trajectories, and scene changes. Device and environmental sensing units, including smart meters, temperature sensors, humidity sensors, light sensors, and motion sensors, are used to collect device operating data (such as power consumption, operating status, start / stop time), environmental data (such as temperature, humidity, light intensity, and air quality), and spatiotemporal data (such as timestamps, geographical location, and regional division information). This data is transmitted in real-time to the central processing platform via IoT protocols (such as MQTT and CoAP) to ensure data synchronization and integrity. The acquisition process employs multimodal data fusion technology to align visual and non-visual data in the temporal dimension, forming a unified data stream to overcome the limitations of single-sensor perception and provide high-precision input for subsequent analysis.

[0025] For example, in a smart office area, a visual sensing unit is deployed in a wide-angle camera on the ceiling, continuously collecting video streams to identify behaviors such as people entering the office area, walking to their workstations, sitting and working, and gathering for discussions. Equipment and environmental sensing units include smart meters installed in the air conditioners (monitoring energy consumption curves), dimmers in the lighting system (recording brightness adjustments), and environmental sensors (monitoring real-time temperature and humidity). Spatiotemporal data is marked using a system clock and an electronic map of the area, for example, office area A between 9:00 AM and 10:00 AM on a weekday. The collected data undergoes preliminary filtering by edge computing devices (such as removing redundant video frames and sensor noise) to form a structured dataset. For example, video data is labeled as "person sitting, stable trajectory," equipment data is recorded as "air conditioner set to 26°C, power consumption 1.2kW," environmental data is "temperature 24°C, illumination 500 lux," and spatiotemporal data is "time: weekday 9:30 AM, location: workstation cluster in area A." This process ensures multi-dimensional data coverage, providing real-world support for habit models to learn the correlation between personnel behavior and equipment energy consumption.

[0026] It should be noted that, in specific implementation scenarios, the above solutions can be further expanded in terms of sensing units to include infrared thermal imaging cameras (for behavior detection in nighttime or low-light environments), radar sensors (for non-invasive trajectory tracking), or wearable devices (such as smart bracelets to supplement individual physiological data). Data dimensions can be increased by adding sound sensors (to collect environmental noise to assess comfort), barometric pressure sensors (for ventilation system optimization), or device logs (such as PLC controller outputs to refine operational status). Acquisition methods can leverage 5G networks for low-latency transmission or introduce edge computing nodes for local preprocessing (such as video compression and anomaly detection) to reduce the load on the central platform. Furthermore, spatiotemporal data can be expanded to include seasonal variables (such as summer / winter patterns) or dynamic geofencing (such as adjusting the collection area based on population density).

[0027] Based on the video data, equipment operation data, environmental data, and spatiotemporal data, a behavior-device habit model is constructed through fusion analysis via visual analysis channels and equipment analysis channels.

[0028] As described above, this step establishes parallel visual analysis and equipment analysis channels to perform deep fusion analysis on the collected multi-source data, constructing a behavior-equipment habit model that reflects the inherent correlation between personnel behavior patterns and equipment operation rules. The visual analysis channel employs computer vision technology, identifying people and objects in the image through target detection, establishing personnel movement trajectories through multi-target tracking algorithms, and then analyzing the specific activity states of personnel using behavior recognition algorithms. The equipment analysis channel uses temporal pattern recognition methods to extract features and mine patterns from equipment operation data and environmental parameters, identifying the periodicity and correlation patterns of equipment operation. The outputs of the two channels are integrated through a feature-level fusion strategy, employing a weighted fusion method based on an attention mechanism. The fusion weights are dynamically adjusted according to the importance of visual behavior features and equipment operation features in different scenarios, ultimately constructing a habit model that accurately represents the mapping relationship between "specific behavior - environmental state - equipment operation." This model is essentially a multi-input, multi-output nonlinear mapping function that can predict the optimal equipment control strategy based on real-time input visual features, equipment state, and environmental parameters.

[0029] For example, in a smart office application scenario, the visual analysis channel captures video of personnel activities through deployed cameras. Target detection identifies five people in the area. Tracking reveals that three people remain stationary at their workstations, while two move between meeting rooms. Behavior recognition determines that those at workstations are in a "sitting and standing" state, while the moving individuals are in a "walking and talking" state. Simultaneously, the equipment analysis channel collects air conditioning system operation data, finding that the current temperature is maintained at 24℃, humidity at 45%, and lighting brightness at 70%. Time-series analysis identifies typical equipment operation patterns in the area on weekday mornings. Fusion analysis of the data from both channels reveals a stable correlation between "sitting and standing" behavior and equipment settings of "air conditioning at 24℃ and lighting at 70% brightness," with this correlation being most significant between 9:00 and 11:00 AM on weekdays. Based on these findings, the system's behavior-equipment habit model accurately describes the correspondence between personnel work behavior and comfort equipment settings in the office area during specific time periods, providing a basis for subsequent energy-saving optimization decisions.

[0030] It should be noted that, in specific implementation scenarios, the above solutions can be further improved by adding a voice analysis channel to the analysis channel, enabling the detection and identification of personnel activity status through sound event detection. Regarding the fusion method, more advanced cross-modal fusion technologies can be employed, such as using graph neural networks for spatiotemporal feature fusion, or introducing transfer learning to leverage pre-trained models to improve fusion performance in small sample scenarios. In terms of model architecture, it can be expanded into a hierarchical habit model, including a short-term habit layer (capturing intraday patterns), a periodic habit layer (learning weekly / monthly patterns), and a scenario habit layer (identifying special event patterns). Furthermore, the habit model can be expanded to incorporate more influencing dimensions, such as external factors like weather data, holiday information, and energy price signals, forming an environmentally adaptive intelligent habit model.

[0031] Based on the predicted energy demand using the behavior-device habit model, a preliminary control strategy is generated using a multi-objective optimization algorithm, and the preliminary control strategy is simulated and evaluated by calling an expert rule base.

[0032] As described above, this step, based on the established behavior-equipment habit model, analyzes the correlation between historical behavior patterns and equipment operation data using time-series prediction methods (such as autoregressive models or neural network predictors) to predict energy demand in specific future periods, such as predicting air conditioning load or lighting energy consumption in office areas during lunch breaks. Subsequently, a multi-objective optimization algorithm (such as non-dominated sorting genetic algorithm or particle swarm optimization) is used to generate a preliminary control strategy. This algorithm considers multiple objectives such as energy efficiency, user comfort, equipment lifespan, and operating costs. By solving the Pareto optimal solution set, conflicting objectives are balanced, and a preliminary strategy such as "reducing air conditioning power by 15% and delaying lighting on-time" is output. Finally, an expert rule base is invoked for simulation evaluation. This rule base is built based on domain knowledge and includes safety thresholds, behavioral constraints, and risk rules (such as "the strategy must not cause temperature changes exceeding 2°C / min"). Fuzzy logic or decision trees are used to simulate the potential impact of the strategy after its execution, assessing whether it will negatively affect core activities (such as production safety or personnel comfort), thereby deciding whether to adjust the strategy or request human intervention.

[0033] For example, in a smart office environment, a behavior-device habit model predicts that energy demand will increase by 20% during the 10:00-11:00 AM weekday period due to meetings with people gathering. Based on the prediction, a multi-objective optimization algorithm generates an initial control strategy with the goal of minimizing energy consumption and maximizing comfort: "Raise the air conditioning temperature to 26°C 10 minutes in advance and reduce the lighting brightness by 10%." Subsequently, an expert rule base simulates and evaluates this strategy. The rule base includes rules such as "If the lighting adjustment exceeds 15% and the screen is detected to be in operation, it is marked as a potential risk of visual fatigue." The evaluation finds that the brightness adjustment is within a safe range, but because a projector is detected during the meeting, the rule "Prohibit significantly dimming the lighting in projection mode" is triggered, thus automatically correcting the strategy to "Adjust the air conditioning only, keep the lighting unchanged" to ensure that the meeting quality is not affected.

[0034] It should be noted that, in specific implementation scenarios, based on the above solutions, in terms of prediction, a multivariate regression model that integrates external data (such as weather forecasts or electricity price signals) can be used, or deep reinforcement learning can be introduced to achieve dynamic demand prediction; in terms of optimization algorithms, multi-objective hierarchical optimization can be used, prioritizing safety objectives (such as fire safety compliance), or a real-time feedback mechanism can be integrated to achieve adaptive weight adjustment; in terms of expert rule base, a self-learning rule system can be used, supplementing the rule base by training neural networks with historical decision data, or high-fidelity simulation evaluation can be performed by combining digital twin technology to predict the long-term impact of the strategy in complex environments (such as sudden equipment failures).

[0035] If the simulated assessment has potential impacts, a prompt message is generated requesting manual decision-making; if there are no potential impacts, the final control strategy is output.

[0036] As described above, this step is a key decision-making stage based on the simulation evaluation results of an expert rule base. A tiered response mechanism is established to ensure the safety and applicability of the control strategy. When the simulation evaluation identifies potential impacts, the system generates structured prompts containing risk types (e.g., equipment safety, personnel comfort, production efficiency), quantitative assessments of the impact, and recommended alternatives. These prompts are pushed to authorized personnel for manual decision-making via preset interfaces (e.g., management platform, mobile terminal). The manual decision-making interface displays strategy details, risk assessment data, and comparative solutions, allowing decision-makers to modify parameters or directly approve execution. When the simulation evaluation confirms no potential impact, the system automatically outputs the final control strategy. This strategy includes an execution priority label and validity verification conditions and is directly issued to the execution unit. The entire decision-making process employs a two-way confirmation mechanism. For manually generated decisions, the system records decision logs for subsequent model optimization, while for automatically generated strategies, the system retains a manual intervention channel to ensure safe and controllable strategy execution under any circumstances.

[0037] For example, in a smart office environment, the system generates an initial strategy based on a behavior-device habit model: "Turn off the air conditioning in the area 30 minutes after personnel are detected leaving." After expert rule base simulation and evaluation, it is found that there is sensitive experimental equipment in the area requiring a constant temperature environment, posing a risk of equipment damage. The system immediately generates a prompt message: "Temperature control requirements detected for experimental equipment in area A; it is recommended to modify the air conditioning shutdown strategy," and pushes this to the equipment administrator through the management platform. After seeing the detailed risk assessment data on the decision-making interface, the administrator selects the modification plan "Only shut off ventilation, maintain basic temperature control" and confirms execution. In another office area without special equipment, after the same strategy is assessed as risk-free, the system automatically outputs the final control strategy and executes it immediately. Simultaneously, the strategy includes a verification condition: "Automatically restore ventilation when the area temperature exceeds 28℃," achieving a balance between safety and energy saving.

[0038] It should be noted that, in specific implementation scenarios, the above solutions can be further enhanced by implementing a multi-level early warning mechanism for generating alerts, using different notification methods (such as email, SMS, and app push notifications) based on risk levels, and supporting collaborative decision-making by multiple administrators. For the human decision-making interface, a visual simulation function can be integrated to dynamically display the expected effects of strategy execution in three dimensions, assisting decision-makers in intuitive understanding. Regarding strategy output, a strategy version management system can be established to support strategy rollback, comparison, and batch deployment. Furthermore, this system can be integrated with the organization's existing operations and maintenance management platform to achieve standardized management of the decision-making process, or digital twin technology can be introduced for deeper prediction of strategy impact.

[0039] The final control strategy is executed, and feedback data is collected to optimize and update the behavior-device habit model.

[0040] As described above, this step involves a closed-loop process of final control strategy execution and system self-optimization. During the execution phase, the system distributes the evaluated and confirmed final control strategy to each execution unit via the equipment control interface, including but not limited to the inverters of the air conditioning system, the dimming controllers of the lighting system, and the damper actuators of the ventilation equipment, ensuring accurate strategy implementation. Simultaneously, the system continuously collects feedback data after strategy execution through a deployed sensor network. This data includes multi-dimensional information such as actual equipment operating parameters, environmental state changes, and personnel behavioral responses. In the model optimization phase, the system employs an incremental learning mechanism to compare and analyze the collected feedback data with the predicted data before strategy execution, calculating the strategy execution deviation and actual effect indicators. Based on these analysis results, the system adjusts the parameters and optimizes the structure of the behavior-equipment habit model. Specific optimization processes include, but are not limited to, updating the behavior recognition weights in the visual analysis channel, adjusting the temporal pattern learning parameters in the equipment analysis channel, and correcting the attention allocation mechanism of multi-source data fusion. This enables the model to adapt to environmental changes and habit evolution, forming a continuously improving intelligent closed loop.

[0041] For example, in the practical application of a smart office area, the system executes the final control strategy of "raising the air conditioner setting temperature by 1°C and reducing the lighting brightness by 20% during the weekday lunch break (12:00-13:00). The control command is sent to the building automation system via the BACnet protocol. During the execution of the strategy, the system monitors the actual temperature change curve through a temperature sensor, records the lighting brightness data through a light sensor, and collects the activity status of personnel through a visual sensing unit. It detects that some people actively turn on their desk lamps to supplement the lighting after the brightness adjustment. Based on this feedback data, the system analysis finds that the original strategy is over-optimized in terms of lighting adjustment. It then adjusts the weight allocation of "lighting comfort" and "energy saving effect" in the behavior-device habit model through an incremental learning algorithm, and adopts a more conservative brightness adjustment range in subsequent strategy generation, such as adjusting the reduction range from 20% to 15%. At the same time, it adds a supplementary strategy of "automatically fine-tuning the main lighting when the desk lamp is detected" to achieve synchronous optimization of the model and strategy.

[0042] It should be noted that, in specific implementation scenarios, based on the above solutions, a multi-strategy collaborative execution mechanism can be selected at the execution level to support coordinated control of cross-system devices, such as the coordinated adjustment of air conditioners and curtains; in terms of feedback data collection, subjective evaluation data can be incorporated, collecting user comfort scores through mobile terminals to achieve the fusion analysis of subjective and objective data; in terms of model optimization, transfer learning technology can be introduced to quickly apply the training results of other similar scenarios to new scenarios, significantly shortening the model optimization cycle. Furthermore, the system can establish a strategy effectiveness evaluation system, quantitatively evaluating the strategy execution effect by setting key performance indicators (such as energy saving rate, comfort index, and equipment lifespan impact coefficient), and feeding the evaluation results back to the model optimization process.

[0043] According to one embodiment of this application, the visual analysis channel includes: The video data is subjected to target detection, target tracking, and behavior recognition, and the personnel behavior status is output.

[0044] As described above, object detection is first performed on the video data. A deep learning neural network is used to identify specific target objects in the video frames, primarily people, but also including vehicles, equipment, and other related objects. The object detection outputs the position coordinates of each identified target in the image, the bounding box size, and the target category confidence score.

[0045] Next, target tracking is performed, using a multi-target tracking algorithm to establish the motion trajectory of the same target across consecutive video frames. This process assigns a unique identifier to each detected target and records its position sequence at different times to form a complete motion trajectory through feature matching and motion model prediction.

[0046] Finally, behavior recognition is performed. Based on the trajectory sequence generated by target tracking, combined with the target's posture changes and spatiotemporal context information, a behavior classification model is used to identify specific behavior patterns. Behavior recognition comprehensively considers multi-dimensional information such as target motion characteristics, duration, and interacting objects, and outputs a semantic description of the person's behavioral state.

[0047] The entire visual analysis channel transforms raw video data into structured human behavior status information through the sequential processing of these three stages, providing a visual analysis foundation for the subsequent construction of behavior-device habit models.

[0048] According to one embodiment of this application, the device analysis channel includes: The device operation data is preprocessed and time-series pattern learned to output the device operation rules; The temporal pattern learning employs a long short-term memory network model.

[0049] As mentioned above, the equipment operation data is first preprocessed, including data cleaning, outlier handling, and normalization. Data cleaning removes noise and invalid segments from the collected data; outlier handling uses statistical methods to identify and correct equipment operation parameters that exceed reasonable ranges; and normalization converts equipment data of different dimensions to a standard numerical range, establishing a standardized data foundation for subsequent analysis.

[0050] Subsequently, time-series pattern learning is performed, employing a Long Short-Term Memory (LSTM) network model to analyze the preprocessed equipment operation data. This network, through gating mechanisms at the input, forget, and output gates, selectively memorizes and forgets historical information, effectively capturing long-term dependencies in the equipment operation data. By learning the patterns of equipment parameter changes over time, the model identifies the cyclical characteristics, trend characteristics, and abnormal patterns of equipment operation.

[0051] The final output reveals the operational patterns of the equipment, including start-up and shutdown cycles, load variation patterns, energy consumption characteristics, and operational characteristics related to other equipment. These patterns are represented in the form of structured feature vectors, providing a data analysis foundation for subsequent construction of behavior-equipment habit models.

[0052] According to one embodiment of this application, the step of invoking an expert rule base to simulate and evaluate the preliminary control strategy includes: The potential impact of the initial control strategy on core production activities is assessed based on an expert rule base. The expert rule base includes security rules built based on historical data and domain knowledge.

[0053] As mentioned above, an expert rule base is first established. This rule base is built based on historical operating data, equipment characteristic knowledge, and domain expert experience, and includes rules for safe equipment operation, production process assurance, and energy efficiency management. Safety rules involve restrictions on equipment start-up and shutdown frequencies, parameter adjustment ranges, and constraints on linked equipment; production assurance rules ensure that control strategies do not affect core production processes and product quality standards; and energy efficiency rules define energy consumption indicators and operating efficiency thresholds.

[0054] During simulation and evaluation, the initial control strategy is matched and verified against rules in the expert rule base. The evaluation process analyzes the potential changes in equipment status, fluctuations in environmental parameters, and impacts on the production process after the strategy is implemented, identifying any risks of violating safety rules or affecting production stability. For strategies involving the linkage of multiple devices, it is also necessary to evaluate whether the coordinated operation between the devices meets the overall system stability requirements.

[0055] The assessment results include a determination of policy compliance, identification of potential risks, and improvement recommendations. For policies that may impact core production activities, a detailed risk analysis report is generated, specifying the violated rules and regulations, assessing the degree of impact, and suggesting corrective actions, providing a basis for subsequent policy optimization or manual decision-making. The entire assessment process is implemented through a rule-based reasoning engine to ensure the accuracy and consistency of the assessment results.

[0056] According to one embodiment of this application, executing the final control strategy includes: The final control strategy is then sent to at least one energy consumption system for execution. The energy consumption system includes at least one of a lighting system, an air conditioning system, a ventilation system, or a water pump system.

[0057] As described above, the final control strategy, after evaluation and confirmation, is parsed into executable instructions for the devices and distributed to the corresponding energy consumption systems via the communication network. The strategy distribution employs a hierarchical transmission mechanism: first, the overall control strategy is sent to the area controller, and then the area controller converts it into specific device instructions and distributes them to each execution terminal.

[0058] The instructions received by the lighting system include switch control, brightness adjustment, and color temperature adjustment; the instructions received by the air conditioning system include temperature setting, fan speed adjustment, and operating mode switching; the instructions received by the ventilation system include air volume control, start / stop sequence, and fresh air ratio adjustment; and the instructions received by the water pump system include flow control, pressure setting, and operating frequency adjustment.

[0059] A two-way confirmation mechanism is established during strategy execution. Each energy-consuming system provides feedback on its actual operating status after executing instructions, including instruction execution results, equipment operating parameters, and abnormal status information. The system verifies the effectiveness of the control strategy by monitoring this feedback data in real time and provides a data foundation for subsequent strategy optimization. In case of abnormal execution, the system activates backup strategies or issues early warning signals to ensure the safe and stable operation of the energy-consuming systems.

[0060] According to one embodiment of this application, the collection of feedback data to optimize and update the behavior-device habit model includes: The parameters of the visual analysis channel and the device analysis channel are adjusted based on the feedback data to update the behavior-device habit model.

[0061] As described above, a deployed sensor network continuously collects multi-dimensional feedback data after the strategy is executed, including changes in environmental parameters, actual equipment operating status, personnel behavioral responses, and system energy consumption data. This feedback data is used to evaluate the degree of deviation between the actual effect and the predicted effect of the control strategy.

[0062] Based on the feedback data analysis results, targeted adjustments were made to the parameters of the visual analysis channel and the device analysis channel. For the visual analysis channel, adjustments included optimizing the feature extraction weights of the target detection model, improving the classification threshold of the behavior recognition algorithm, and correcting the correlation parameters of target tracking. For the device analysis channel, adjustments included updating the weight matrix of the temporal pattern learning network, optimizing the sensitivity parameters of feature extraction, and recalibrating the recognition criteria for device operating patterns.

[0063] By adjusting parameters, the behavior-equipment habit model is iteratively updated, enabling it to better reflect the dynamic relationship between human behavior and equipment operation in the real environment. The updated model will be used for subsequent control strategy generation, forming a complete closed loop from strategy execution to effect evaluation and model optimization. This process ensures that the model can continuously adapt to environmental changes and habit evolution, constantly improving the system's control accuracy and intelligence level.

[0064] According to one embodiment of this application, it also includes: Before constructing the behavior-device habit model, the video data, device operation data, environmental data, and spatiotemporal data are cleaned and normalized.

[0065] As mentioned above, the first step is data cleaning, which involves processing outliers and missing values ​​in video data, equipment operation data, environmental data, and spatiotemporal data. For video data, image noise and detection loss caused by factors such as changes in lighting and occlusion are eliminated; for equipment operation data, abnormal readings caused by sensor malfunctions or communication interruptions are identified and corrected; for environmental data, transient abrupt changes caused by external interference are removed; and for spatiotemporal data, the continuity of timestamps and the rationality of spatial coordinates are verified.

[0066] Then, normalization processing is performed to convert various types of data with different dimensions and magnitudes to a unified numerical range. Scale normalization is applied to the location coordinates in the video data, min-max normalization is applied to parameters such as power and frequency in the equipment operation data, Z-score normalization is applied to measured values ​​such as temperature and humidity in the environmental data, and time information in the spatiotemporal data is converted into a periodic feature representation.

[0067] Data cleansing and normalization processes result in data with a consistent format and units, eliminating noise and dimensional discrepancies in the original data. This provides a standardized data foundation for building accurate and reliable behavior-device habit models. This preprocessing ensures the effective fusion of data from different sources and of different types, improving the convergence speed and generalization ability of model training.

[0068] A second aspect of this application provides a dynamic energy-saving optimization device based on machine vision and behavioral habit prediction, comprising: The data acquisition module is used to acquire video data, equipment operation data, environmental data, and spatiotemporal data through the visual sensing unit, equipment and environmental sensing unit; The data processing and analysis module is used to perform fusion analysis on the video data, equipment operation data, environmental data and spatiotemporal data through the visual analysis channel and the equipment analysis channel to construct a behavior-equipment habit model. The core decision-making module is used to predict energy demand based on the behavior-equipment habit model, generate a preliminary control strategy through a multi-objective optimization algorithm, and call an expert rule base to simulate and evaluate the preliminary control strategy; if the simulation evaluation has potential impacts, a prompt message is generated to request manual decision-making; if there are no potential impacts, the final control strategy is output. The control execution module is used to execute the final control strategy and collect feedback data to optimize and update the behavior-device habit model.

[0069] According to one embodiment of this application, the data processing and analysis module includes a visual analysis unit and a time series prediction unit; The visual analysis unit is used to perform target detection, target tracking, and behavior recognition on the video data; The timing prediction unit is used to preprocess the device operation data and learn timing patterns.

[0070] According to one embodiment of this application, the core decision-making module includes a strategy generation unit and an expert evaluation unit; The strategy generation unit is used to generate the preliminary control strategy through a multi-objective optimization algorithm; The expert evaluation unit is used to call the expert rule base to perform simulated evaluations.

[0071] Example 2 1. Data Collection Video data (1920×1080 resolution, 30fps) is acquired via a visual sensing unit, including personnel entry / exit, walking trajectories, and sitting / standing behaviors. Equipment and environmental sensing units collect data on equipment operation (e.g., air conditioning set temperature, lighting brightness, equipment power consumption), environmental data (temperature, humidity, light intensity), and spatiotemporal data (timestamps, location). Data is transmitted to the central processing platform in JSON format, with a sampling frequency of 1Hz. Multi-source data fusion overcomes the limitations of single sensors, providing comprehensive input for model construction.

[0072] 2. Construct a behavior-device habit model Model building is divided into a visual analysis channel and a device analysis channel: Visual analysis channel: The YOLOv5 object detection algorithm was used to identify bounding boxes of people in the video, and then the DeepSORT algorithm was used for object tracking to generate a sequence of people's trajectories. Behavior recognition, based on trajectory and pose estimation, employed a spatiotemporal graph convolutional network (ST-GCN) to classify behavioral states (such as "walking," "sitting," and "clustering"). The output is a sequence of people's behavioral states. ,in The action label representing time t.

[0073] Equipment analysis channel: Equipment operation data, after preprocessing (missing value imputation, Z-score normalization), is input into a Long Short-Term Memory (LSTM) network for temporal pattern learning. The LSTM model structure includes an input layer (equipment power consumption, environmental data), a hidden layer (128 units), and an output layer (equipment state probabilities). The loss function is mean squared error (MSE), and the optimizer uses Adam. The output is a sequence of equipment operation patterns. .

[0074] Model fusion: Will and It integrates with spatiotemporal data (such as weekdays / weekends) and calculates the association weights between behavior and device through an attention mechanism. The integration formula is:

[0075] in The output of the habit model is α, β, and γ, which are attention weights learned through training. This is a spatiotemporal context vector. The model outputs a behavior-device habit model, representing the mapping relationship between human behavior and device energy consumption (e.g., "sitting position corresponds to air conditioning at 26°C and lighting at 70% brightness"). Dual-channel fusion and attention mechanisms improve model accuracy, making habit learning more closely aligned with real-world scenarios.

[0076] 3. Predicting energy demand and generating preliminary control strategies Based on behavior-device habit model This method predicts energy demand for the next 30 minutes. The prediction uses an Autoregressive Integral Moving Average (ARIMA) model, with historical energy consumption data as input and predicted values ​​as output. .

[0077] Subsequently, a preliminary control strategy is generated using a multi-objective optimization algorithm. The optimization objectives are to minimize energy consumption and maximize comfort, and the objective function is defined as follows:

[0078] in Total energy consumption (kWh) For comfort deviation (calculated based on PMV index), the weights are... =0.6, =0.4 The constraints were determined using the analytic hierarchy process (AHP). These constraints included the equipment's operating range (e.g., air conditioning temperature 22-28°C) and safety thresholds. The optimization algorithm employed a non-dominated sorting genetic algorithm (NSGA-II), outputting a Pareto optimal solution set from which an initial control strategy (e.g., "reduce air conditioning power by 10% and dim lighting by 20%)" was selected. Multi-objective optimization balanced energy saving and comfort, avoiding extreme strategies resulting from a single objective.

[0079] 4. Expert rule base evaluation and decision-making The initial control strategy is simulated and evaluated using an expert rule base. The rule base is built based on historical events and domain knowledge and includes rules such as: Rule 1: If the forecaster is present for more than 2 hours and the strategy involves turning off the air conditioning, the "potential impact" flag is triggered.

[0080] Rule 2: If the rate of change in ambient temperature is >0.5°C / min and the strategy adjusts the lighting brightness by >30%, a risk assessment shall be conducted.

[0081] The evaluation process uses a fuzzy logic system to calculate the strategy risk score (RR) (0-1). If R > 0.7, a prompt message is generated (e.g., "The strategy may affect work efficiency, please confirm"), requesting manual decision-making from the administrator; otherwise, the final control strategy is output. The combination of expert rules and fuzzy logic enables quantitative risk assessment, ensuring strategy security.

[0082] 5. Execution control strategy and feedback optimization The final control strategy is sent to actuators (such as smart dimmers and variable frequency air conditioner controllers) via the MQTT protocol. The system collects feedback data in real time, including actual energy consumption, environmental changes, and human feedback (comfort scores are collected via a mobile app).

[0083] Feedback data is used to update the behavior-device habit model: an online learning mechanism is employed, using feedback data as the incremental training set to adjust the LSTM and attention weights. The update formula is:

[0084] in For model parameters, For learning rate, For loss function, These are actual observations. The model is automatically updated every 24 hours, forming a self-evolving closed loop. Online learning enables the model to continuously adapt to behavioral changes, improving long-term energy-saving performance.

[0085] For any parts not mentioned in this application, existing technologies may be used or referenced.

[0086] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0087] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A dynamic energy saving optimization method based on machine vision and behavior habit prediction, characterized in that, Comprising: Collecting video data, equipment operation data, environmental data and spatio-temporal data through visual sensing units, equipment and environmental sensing units; Based on the video data, equipment operation data, environmental data and spatio-temporal data, fusion analysis is carried out through visual analysis channel and equipment analysis channel to construct behavior-equipment habit model; Based on the behavior-equipment habit model, the energy demand is predicted, the preliminary control strategy is generated through multi-objective optimization algorithm, and the expert rule base is called to simulate and evaluate the preliminary control strategy; If the simulation evaluation has potential impact, prompt information is generated to request manual decision; if there is no potential impact, the final control strategy is output; The final control strategy is executed, and feedback data is collected to optimize and update the behavior-equipment habit model. 2.The method of claim 1, wherein, The visual analysis channel comprises: Target detection, target tracking and behavior recognition are performed on the video data, and the personnel behavior state is output. 3.The method of claim 1, wherein, The equipment analysis channel comprises: The equipment operation data is preprocessed and time series pattern learning is performed, and the equipment operation rule is output; Wherein, the time series pattern learning adopts long short-term memory network model. 4.The method of claim 1, wherein, The simulation evaluation of the preliminary control strategy by calling the expert rule base comprises: Based on the expert rule base, the potential impact of the preliminary control strategy on the core production activities is evaluated; Wherein, the expert rule base includes safety rules constructed based on historical data and domain knowledge. 5.The method of claim 1, wherein, The execution of the final control strategy comprises: The final control strategy is issued to at least one energy consumption system for execution; Wherein, the energy consumption system includes at least one of lighting system, air conditioning system, ventilation system or water pump system. 6.The method of claim 1, wherein, The collection of feedback data to optimize and update the behavior-equipment habit model comprises: Based on the feedback data, the parameters of the visual analysis channel and the equipment analysis channel are adjusted to update the behavior-equipment habit model.

7. The method of dynamic energy saving optimization based on machine vision and behavior habit prediction according to any one of claims 1 to 6, characterized in that, Further comprising: Before constructing the behavior-equipment habit model, the video data, equipment operation data, environmental data and spatio-temporal data are subjected to data cleaning and normalization processing.

8. A dynamic energy saving optimization device based on machine vision and behavior habit prediction, characterized in that, Comprising: A data collection module for collecting video data, equipment operation data, environmental data and spatio-temporal data through visual sensing units, equipment and environmental sensing units; A data processing and analysis module for fusion analysis through visual analysis channel and equipment analysis channel based on the video data, equipment operation data, environmental data and spatio-temporal data to construct behavior-equipment habit model; A core decision module for predicting energy demand based on the behavior-equipment habit model, generating a preliminary control strategy through a multi-objective optimization algorithm, and simulating and evaluating the preliminary control strategy by calling an expert rule base; if the simulation evaluation has potential impact, prompt information is generated to request manual decision; if there is no potential impact, the final control strategy is output; A control execution module for executing the final control strategy and collecting feedback data to optimize and update the behavior-equipment habit model.

9. The dynamic energy-saving optimization device based on machine vision and behavior habit prediction according to claim 8, characterized in that: The data processing and analysis module comprises a visual analysis unit and a time sequence prediction unit; The visual analysis unit is used for target detection, target tracking and behavior recognition on the video data; The time sequence prediction unit is used for preprocessing and time sequence mode learning on the device operation data.

10. The dynamic energy-saving optimization device based on machine vision and behavior habit prediction according to claim 8, characterized in that: The core decision module comprises a strategy generation unit and an expert evaluation unit; The strategy generation unit is used for generating the preliminary control strategy through a multi-objective optimization algorithm; The expert evaluation unit is used for calling the expert rule base for simulation evaluation.

Images