Intelligent Energy Control Method and System for Public Places Based on Video Detection

Abstract

The present invention relates to an intelligent energy control method and system for public places based on video detection, belonging to the technical field of intelligent lighting control. It includes a controller, the power supply end of the controller is connected to a power supply, the signal input end of the controller is connected to a computer through a WIFI module and an MTQQ server, the computer communicates with a video server through RTSP, the control output end of the controller is connected to a relay drive module, the relay drive module is connected to a relay, the relay is connected to an electrical wiring terminal, the electrical wiring terminal is connected to an electrical appliance, the output interface of the controller is connected to an indicator light, and the input interface of the controller is connected to a switch. The present invention can achieve accurate identification and monitoring, improve the reliability and real-time performance of remote control, and has a flexible control method, which can greatly improve the user experience and has strong practicability.

Description

Technical Field

[0001] The present invention relates to an intelligent energy control method and system for public places based on video detection, belonging to the technical field of intelligent lighting control. Background Art

[0002] At present, since the Internet of Things intelligent lighting system allows users to remotely monitor and control lighting devices after being connected to the network, greatly improving energy efficiency, convenience and user experience, it has been widely applied. For example, Philips Hue and LIFX, etc., allow users to remotely control and monitor home lighting devices through a dedicated application or network interface; some can achieve intelligent dimming, and users can adjust the light brightness according to needs and environmental conditions, which helps to save energy and provides a user-customized lighting experience. There are also some intelligent lighting systems that support voice control and integrate voice assistants such as Amazon Alexa and Google Assistant, and users can easily control the light status through voice commands. There are also technologies that can schedule the on / off time of lights and set automation rules, such as adjusting lights according to sunrise / sunset time or human body sensing. There are also technologies that can provide energy monitoring and reporting functions to help users understand the energy consumption of lighting devices and take measures to reduce energy waste. The patent named Interactive Classroom Lighting Intelligent Control System and Control Method (CN105934051B) developed a classroom lighting intelligent control system to solve the problem of low intelligent control level of classroom lighting in colleges and universities, resulting in waste of lighting resources. The system consists of an infrared sensor, an illuminance sensor and a camera, and controls the lights through the obtained information to achieve real-time interaction between the intelligent lighting control system and students, reducing the waste of lighting resources and improving the utilization rate of self-study classrooms. However, the above technologies have the following defects: First, due to the limitations of the video monitoring method, it is impossible to accurately identify the monitoring images in large public places, and the recognition rate of small targets in the distance is too low; second, the entire technology requires too many sensors and is too cumbersome; third, it is only limited to lighting control, and these technologies are mainly concentrated in the lighting field of small scenarios such as homes, and have not been widely applied to the electrical control systems of large public places. Summary of the Invention

[0003] According to the deficiencies in the above prior art, the problems to be solved by the present invention are: to provide an intelligent energy control method and system for public places based on video detection that can achieve accurate identification and monitoring, improve the reliability and real-time performance of remote control, and has a flexible control method, which can greatly improve the user experience.

[0004] The technical solution adopted by the present invention to solve its technical problems is:

[0005] The intelligent energy control method for public places based on video detection includes the following steps:

[0006] S1. Initialize parameters for easy subsequent query of the current status. Meanwhile, the system will load a pre-set tag list, which contains all the object types that the model is expected to recognize, such as "person". The purpose of doing this is to enable the system to not only recognize the objects in the video frame but also accurately know what each object is when analyzing the video frame;

[0007] S2. Load the YOLO model for object detection;

[0008] S3. Communicate with the video server within the local area network through RTSP to request and control the playback and transmission of the streaming media, and obtain the video stream of the surveillance camera; RTSP (Real Time Streaming Protocol) is a network protocol used for transmitting streaming media data over a computer network. It is an application layer protocol and is commonly used to achieve real-time transmission of audio and video over the Internet and local area networks. The running program reads the video stream of the target through the IP, username, password, and channel number provided by the video server for subsequent program use;

[0009] S4. Process each frame of the video stream with the loaded YOLO model;

[0010] S5. While the YOLO model processes each frame of the video, add the subtraction method to process the frame; improve the overall recognition accuracy;

[0011] S6. Make an execution action decision based on the results of the frame processing in steps S4 and S5;

[0012] S7. Based on the decision in step S6, perform different actions and send the information of the required actions to the MQTT server;

[0013] S8. The MQTT server connects to the controller through the WIFI module. After receiving the control command sent by the MQTT server, the controller controls the relay to work according to the control command to turn off or on the electrical appliance.

[0014] Further preferably, the initialized parameters include: whether the program is currently in a state with people; whether the actions to be performed in the "no - people state" have been run; record the start time of the "no - people state"; detect whether the program has just started; customize how long to perform the actions in the "no - people state" in the "no - people state".

[0015] Further preferably, in step S2, the method for loading the YOLO model is the asynchronous loading method. Asynchronous loading can improve the responsiveness of the program because the main thread will not be blocked while waiting for the resource to be loaded.

[0016] Further preferably, in step S2, the loaded YOLO model adopts the combination of the YOLO algorithm in deep learning and the advanced Transformer technology to achieve more efficient and accurate personnel monitoring. A specially designed and optimized personnel recognition model is adopted.

[0017] YOLO is a popular real-time object detection system. The core idea of YOLO is to regard the object detection task as a regression problem through a single neural network, directly predicting the bounding boxes and class probabilities from the image pixels. Different from traditional object detection systems, YOLO takes the entire image as the input of the network and predicts multiple bounding boxes and class probabilities at one time, thus greatly improving the processing speed.

[0018] Transformer is a model architecture mainly used to process sequence-to-sequence tasks. Its core is the self-attention mechanism, which allows the model to consider all positions in the sequence when processing data, helping to capture long-range dependencies in the data. The Transformer model consists of multiple layers of encoders and decoders, and each layer contains a self-attention layer and a feed-forward neural network.

[0019] The main reason for using Transformer to optimize the YOLO model is to utilize the powerful spatial relationship learning ability of Transformer to enhance the object detection performance of YOLO. The self-attention mechanism of Transformer can help the YOLO model better understand the relationships between different parts of the image, especially in complex scenes, which can help the model more accurately identify and locate objects.

[0020] Among them, the detailed steps of model simplification and optimization are as follows:

[0021] S21. Collect data: Obtain the surveillance footage of public places containing people;

[0022] S22. Label data: Use labeling tools to label the bounding boxes of people in the images in the dataset;

[0023] S23. Download pre-trained weights: Instead of training the model from scratch, before training, use the weights pre-trained on a large-scale dataset as the starting point. This method can significantly improve the model training efficiency, accelerate the convergence speed, and enhance the adaptability of the model to new scenarios;

[0024] S24. Integrated Transformer Architecture Design: Based on the traditional YOLO model, the Transformer architecture is integrated. The introduction of Transformer enables the model to better understand the context relationships and complex dynamics in videos, especially being more effective when dealing with occlusions and group behaviors.

[0025] S25. Adjust the Loss Function: Adjust the loss function of the YOLO model to make the model better adapt to the requirements of human recognition, especially optimizing the prediction accuracy for position, size, and confidence.

[0026] S26. Training: Use the labeled dataset to train the YOLO model combined with Transformer. In this process, the model learns how to accurately identify and track people from complex backgrounds.

[0027] S27. Validation and Evaluation: Evaluate the performance of the model by testing it on an independent validation set to ensure that the model can maintain high accuracy and robustness in different scenarios.

[0028] S28. Simplify the Model: Reduce the consumption of computing resources and streamline the model by optimizing the Transformer module and adjusting the input resolution to adapt to the needs of real-time video analysis, reducing the network complexity while ensuring the recognition efficiency.

[0029] Further preferably, in step S4, to solve the problem that the frame acquisition speed of the video stream is much greater than the frame processing speed of the model, and there will be problems such as video frame lag and computer memory overflow of the running program after a long time, the video stream processing adopts the following steps:

[0030] S41. Create a queue with a maximum capacity of 2 to transfer image data between the "image input method" and the "image processing method".

[0031] S42. Create two processes, one for image input and the other for image processing.

[0032] S43. Input the video stream obtained in process S3 into the queue frame by frame. Because the queue has the feature of first in first out, only need to control the queue size. If the number of image frames in the queue exceeds 1, take out one from the queue to keep the queue size and ensure that the image frame in the queue is the current latest picture.

[0033] Among them, when the image frame is the current latest picture, the processing steps for each image frame are as follows:

[0034] S431. First, obtain the names of all layers in the network, obtain the indices of the unconnected output layers, and use these layers to obtain the final output of the YOLO model.

[0035] S432. Obtain a frame of image from the image queue, obtain the height, width, and number of channels of the image, and convert the image into the input format required by the YOLO model, that is, convert the image into the blob format;

[0036] S433. Input the blob into the YOLO model, perform forward propagation, and obtain the output of the model;

[0037] S434. Parse the output of the YOLO model, extract the category, confidence, and bounding box information of the detected objects; loop through each piece of information in the model output, obtain the confidence of the detection result for the human category, filter out the detection results with low confidence by setting a threshold, and extract the bounding box information of the detected human objects, including the top-left coordinates (x, y), width w, and height h;

[0038] S435. Use non-maximum suppression to filter out the bounding boxes with high overlap and retain the most representative detection results;

[0039] S436. Traverse the valid bounding boxes, draw rectangular boxes and labels on the image frame, and annotate them through a pre-set label list.

[0040] Further preferably, in step S5, the steps of processing the picture by the subtraction method are as follows:

[0041] S51. Always save the video of the last five minutes in the background;

[0042] S52. Divide the video picture into 4*4 regions on average;

[0043] S53. Take the average of the number of frames per second in each region to obtain the average frame per second. Subtract the average frame of the next second from the average frame of one second. If there is no one, the picture is black. If there is someone, there is a human outline in the picture;

[0044] S54. Extract the data and draw it as a curve graph, draw the picture change curve of this time period. If the picture has no change, it is the "no one state", and the curve is relatively stable. If the picture has a change, it is the "someone state", and the curve fluctuates;

[0045] S55. Through the analysis of the video pictures of the last five minutes, judge whether there is a target in the distance, so as to assist the YOLO model in making a decision on the processing result.

[0046] Further preferably, the actions in step S7 include actions in the "someone state" and actions in the "no one state";

[0047] Among them, the action content in the "someone state" is:

[0048] S71-1. Obtain the location where the current image is taken;

[0049] S71-2. Detect whether the MQTT connection has been established. If not, establish the connection; if so, proceed to the next step;

[0050] S71-3. Construct the message to be sent {"ClassRoom": f"{num}","DevNo": "01","Value": "1"}, including the room name ClassRoom, device name DevNo, and message content Value (here '1' represents the turn-on operation);

[0051] S71-4. Send the message to the MQTT server in JSON format;

[0052] The action content in the "unoccupied state" is:

[0053] S72-1. Obtain the location where the current image is taken;

[0054] S72-2. Detect whether the MQTT connection has been established. If not, establish the connection; if so, proceed to the next step;

[0055] S72-3. Construct the message to be sent {"ClassRoom": f"{num}","DevNo": "01","Value": "0"}, including the room name ClassRoom, device name DevNo, and message content Value (here '0' represents the turn-off operation);

[0056] S72-4. Send the message to the MQTT server in JSON format.

[0057] The MQTT mentioned above is a lightweight communication protocol specifically designed for reliable communication between Internet of Things (IoT) devices. It has the following characteristics: The MQTT protocol uses the publish / subscribe model, where the message publisher publishes messages to a specific topic, and subscribers can subscribe to the topics they are interested in and receive the corresponding messages. This model makes message passing more flexible and decentralized. The MQTT protocol is very suitable for low-bandwidth and unstable network environments because its protocol header is very concise, occupying less network bandwidth and resources. MQTT supports multiple message transmission quality levels (QoS) to meet different application requirements. The QoS levels include at most once transmission (QoS 0), at least once transmission (QoS 1), and exactly once transmission (QoS 2). MQTT has a reconnection mechanism. When the device reconnects after a network interruption, it can restore the previous subscriptions and ensure reliable message transmission. The MQTT protocol supports data transmission using SSL / TLS encryption to provide security protection. Due to its simplicity, lightweight nature, and reliability, MQTT is widely used in the IoT field, such as sensor data collection, remote monitoring, smart homes, etc.

[0058] The intelligent energy control system for public places based on video detection includes a controller. The power supply terminal of the controller is connected to a power supply. The signal input terminal of the controller is connected to a computer through a WIFI module and an MTQQ server. The computer communicates with a video server through RTSP. The control output terminal of the controller is connected to a relay drive module. The relay drive module is connected to a relay. The relay is connected to an electrical wiring terminal. The electrical wiring terminal is connected to an electrical appliance. The output interface of the controller is connected to an indicator light. The input interface of the controller is connected to a switch. An emergency switch circuit is connected in parallel across the relay. The emergency switch circuit is connected to the power supply and the electrical wiring terminal.

[0059] The computer obtains the monitoring video stream through RTSP, processes it through the improved YOLO model combined with the difference matrix, makes decisions through logical code and sends signals to the controller. The controller ESP8266 is used to connect to the cloud server, establishing a stable communication bridge between the controller and the cloud server, and achieving precise control of electrical appliances in public places. In addition, we set up a switch with the convenient function of being able to turn off all electrical appliances in public places with one key, providing a more convenient and intelligent control experience for users.

[0060] The 5VIN power supply terminal of the described controller is connected to the 5V power output terminal of the AC-DC power converter P1. The 5VIN power supply terminal of the controller is also connected to a resistor R10 and a light-emitting diode D6 and then grounded;

[0061] The D1 output terminal of the described controller is connected to relay driver circuits U2 and U3. The relay driver circuit U2 is connected to relay RY1. Relay RY1 is connected to electrical wiring port P4. An emergency switch S1 is connected in parallel across relay RY1, and the emergency switch S1 is connected to electrical wiring port P4. The relay driver circuit U3 is connected to relay RY2. Relay RY2 is connected to electrical wiring port P4. An emergency switch S2 is connected in parallel across relay RY2, and the emergency switch S2 is connected to electrical wiring port P4. The D1 output terminal of the controller is also connected to resistor R4 and light-emitting diode D2 and then grounded;

[0062] The D2 port of the described controller is connected to relay driver circuits U4 and U5. The relay driver circuit U4 is connected to relay RY3. Relay RY3 is connected to electrical wiring port P5. An emergency switch S3 is connected in parallel across relay RY3, and the emergency switch S3 is connected to electrical wiring port P5. The relay driver circuit U5 is connected to relay RY4. Relay RY4 is connected to electrical wiring port P5. An emergency switch S4 is connected in parallel across relay RY4, and the emergency switch S4 is connected to electrical wiring port P5. The D2 output terminal of the controller is also connected to resistor R3 and light-emitting diode D1 and then grounded;

[0063] The D5 port of the controller is connected to an external switch interface P3 and then grounded. The external switch interface P3 is connected to a switch; The D6 port of the controller is connected to resistor R6 and light-emitting diode D3 and then grounded; The D7 port of the controller is connected to resistor R7 and light-emitting diode D4 and then grounded; The D8 port of the controller is connected to resistor R9 and light-emitting diode D5 and then grounded.

[0064] The beneficial effects of the present invention are:

[0065] The method and system for intelligent energy control in public places based on video detection according to the present invention comprehensively utilize Internet of Things technology and computer vision technology. Only by obtaining the video content of public places can it achieve comprehensive control of electrical appliances in public places. Through one system, different control instructions can be sent to several electrical appliances simultaneously, including functions such as remote monitoring, timing scheduling, and physical switch integration, providing a more intelligent and flexible control method for large public places.

[0066] The method and system for intelligent energy control in public places based on video detection according to the present invention adopt the YOLO model optimized by integrating the Transformer architecture to accurately collect personnel information in images. At the same time, to further make up for the instability of YOLO detection, a method combined with the subtraction method is also proposed, that is, a technique for judging whether there is a target in the picture by using the matrix generated by the difference between the average frames, overcoming the problem of inaccurate monitoring of small targets in the long-distance view, and accurately controlling electrical appliances through the above combination.

[0067] The energy intelligent control method and system for public places based on video detection according to the present invention avoid the picture delay caused by the model processing images through a queue; by optimizing the network communication mechanism, the problems that may be faced in terms of network stability in the prior art are solved, and the reliability and real-time performance of remote control are improved.

[0068] The energy intelligent control method and system for public places based on video detection according to the present invention innovatively and effectively integrate traditional physical switches with modern remote control technologies, ensuring that users can flexibly select and switch between the two control methods, thereby improving the user experience and the usability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 is the system schematic diagram of the present invention;

[0070] Figure 2 is the controller circuit diagram of the present invention;

[0071] Figure 3 is the relay drive circuit and emergency switch circuit of the present invention Figure 1 ;

[0072] Figure 4 is the relay drive circuit and emergency switch circuit of the present invention Figure 2 ;

[0073] Figure 5 is the indicator light circuit diagram of the present invention;

[0074] Figure 6 is the flowchart of the program operation of the embodiment of the present invention;

[0075] In the figure, 1. controller; 2. power supply; 3. relay drive module; 4. relay; 5. electrical connection terminal; 6. electrical appliance; 7. indicator light; 8. switch; 9. computer; 10. video server; 11. emergency switch circuit. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0076] The following further describes the embodiments of the present invention with reference to the drawings:

[0077] As Figure 6 shown, the energy intelligent control method for public places based on video detection according to the present invention includes the following steps:

[0078] S1. Initialize parameters. Meanwhile, the system loads a pre-set tag list, which contains the object types that the model can recognize. The initialized parameters include: whether the program is currently in the presence of people; whether the actions to be performed in the "absence of people state" have been run; record the start time of the "absence of people state"; detect whether the program has just been started; customize how often to perform the actions in the "absence of people state" in the "absence of people state".

[0079] S2. Load the YOLO model used for object detection. The method for loading the YOLO model is an asynchronous loading method. The loaded YOLO model combines the YOLO algorithm and Transformer technology. The Transformer technology is used to simplify the YOLO model. The simplification steps are as follows:

[0080] S21. Collect data: Obtain surveillance camera footage of public places containing people.

[0081] S22. Label data: Use a labeling tool to label the bounding boxes of people in the images in the dataset.

[0082] S23. Download pre-trained weights: Instead of training the model from scratch, use the weights pre-trained on a large-scale dataset as the starting point before training.

[0083] S24. Integrate the Transformer architecture design: Integrate the Transformer architecture on the basis of the traditional YOLO model.

[0084] S25. Adjust the loss function: Adjust the loss function of the YOLO model.

[0085] S26. Train: Use the labeled dataset to train the YOLO model combined with Transformer.

[0086] S27. Validate and evaluate: Evaluate its performance by testing the model on an independent validation set.

[0087] S28. Simplify the model: Reduce the consumption of computing resources and streamline the model by optimizing the Transformer module and adjusting the input resolution.

[0088] S3. Communicate with the video server within the local area network through RTSP, request and control the playback and transmission of the streaming media, and obtain the video stream of the surveillance camera.

[0089] S4. Process each frame of the video stream with the loaded YOLO model. The steps for processing the video stream are as follows:

[0090] S41. A queue with a maximum capacity of 2 is created to transfer image data between the "image input method" and the "image processing method".

[0091] S42. Two processes are created, one for image input and the other for image processing.

[0092] S43. The video stream obtained in process S3 is input into the queue frame by frame. Since the queue has the feature of first in first out, only the queue size needs to be controlled. If the number of image frames in the queue exceeds 1, one is taken out from the queue to keep the queue size and ensure that the image frame in the queue is the current latest picture.

[0093] Among them, when the image frame is the current latest picture, the processing steps for each image frame are as follows:

[0094] S431. First, obtain the names of all layers in the network, obtain the indexes of unconnected output layers, and use these layers to obtain the final output of the YOLO model.

[0095] S432. Obtain a frame of image from the image queue, obtain the height, width and number of channels of the image, and convert the image into the input format required by the YOLO model, that is, convert the image into blob format.

[0096] S433. Input the blob into the YOLO model for forward propagation to obtain the output of the model.

[0097] S434. Analyze the output of the YOLO model, extract the category, confidence and bounding box information of the detected objects; loop through each piece of information in the model output, obtain the confidence of the detection result for the human category, filter out the detection results with low confidence by setting a threshold, and extract the bounding box information of the detected human objects, including the upper left corner coordinates (x, y), width w and height h.

[0098] S435. Use non-maximum suppression to filter out the bounding boxes with high overlap and retain the most representative detection results.

[0099] S436. Traverse the valid bounding boxes, draw rectangular boxes and labels on the image frame, and annotate them through a pre-set label list.

[0100] S5. While the YOLO model processes each frame of the video, the subtraction method is added to process the picture; the steps for processing the picture by the subtraction method are as follows:

[0101] S51. Always save the video of the last five minutes in the background.

[0102] S52. Divide the video picture evenly into 4 * 4 regions.

[0103] S53. In each area, average the number of frames per second to obtain the average number of frames per second. Subtract the average number of frames in one second from the average number of frames in the next second. If there is no person, the screen is black; if there is a person, there is a person's edge line on the screen.

[0104] S54. Extract the data and plot it as a curve graph to draw the screen change curve for this period. If the screen does not change, it is the "no person state", and the curve is relatively stable; if the screen changes, it is the "person state", and the curve fluctuates.

[0105] S55. By analyzing the video screen in the past five minutes, determine whether there is a target in the distance, so as to assist the YOLO model in making a decision on the processing result.

[0106] S6. Make an execution action decision based on the screen processing results of steps S4 and S5.

[0107] S7. Based on the decision in step S6, perform different actions and send the information of the required actions to the MQTT server; the actions include actions in the "person state" and actions in the "no person state".

[0108] Among them, the action content in the "person state" is:

[0109] S71-1. Obtain the position where the current image is taken.

[0110] S71-2. Detect whether the MQTT connection has been established. If not, establish the connection; if so, proceed to the next step.

[0111] S71-3. Construct the message to be sent {"ClassRoom": f"{num}","DevNo": "01","Value": "1"}, including the room name ClassRoom, device name DevNo, and message content Value.

[0112] S71-4. Send the message to the MQTT server in json format.

[0113] The action content in the "no person state" is:

[0114] S72-1. Obtain the position where the current image is taken.

[0115] S72-2. Detect whether the MQTT connection has been established. If not, establish the connection; if so, proceed to the next step.

[0116] S72-3. Construct the message to be sent: {"ClassRoom": f"{num}", "DevNo": "01", "Value": "0"}, including the room name ClassRoom, device name DevNo, and message content Value;

[0117] S72-4. Send the message to the MQTT server in JSON format;

[0118] S8. The MQTT server connects to the controller through the WIFI module. After receiving the control command sent by the MQTT server, the controller controls the relay to work according to the control command to turn off or on the electrical appliance.

[0119] As Figure 1 shown, the intelligent energy control system in public places based on video detection includes a controller 1. The power supply end of the controller 1 is connected to the power supply 2. The signal input end of the controller 1 is connected to the computer 9 through the WIFI module and the MTQQ server. The computer 9 communicates with the video server 10 through RTSP. The control output end of the controller 1 is connected to the relay drive module 3. The relay drive module 3 is connected to the relay 4. The relay 4 is connected to the electrical appliance terminal 5. The electrical appliance terminal 5 is connected to the electrical appliance 6. The output interface of the controller 1 is connected to the indicator light 7. The input interface of the controller 1 is connected to the switch 8. An emergency switch circuit 11 is connected in parallel at both ends of the relay 4. The emergency switch circuit 11 is connected to the power supply 2 and the electrical appliance terminal 5.

[0120] As Figure 2 shown, the 5VIN power supply end of the controller 1 is connected to the 5V power output end of the AC-DC power converter P1. The 5VIN power supply end of the controller 1 is also connected to the resistor R10 and the light-emitting diode D6 and then grounded;

[0121] As Figure 3 shown, the D1 output end of the controller 1 is connected to the relay drive circuit U2 and the relay drive circuit U3. The relay drive circuit U2 is connected to the relay RY1. The relay RY1 is connected to the electrical appliance connection port P4. An emergency switch S1 is connected in parallel at both ends of the relay RY1. The emergency switch S1 is connected to the electrical appliance connection port P4. The relay drive circuit U3 is connected to the relay RY2. The relay RY2 is connected to the electrical appliance connection port P4. An emergency switch S2 is connected in parallel at both ends of the relay RY2. The emergency switch S2 is connected to the electrical appliance connection port P4. The D1 output end of the controller 1 is also connected to the resistor R4 and the light-emitting diode D2 and then grounded;

[0122] As Figure 4As shown, the D2 port of the controller 1 is connected to the relay driver circuits U4 and U5. The relay driver circuit U4 is connected to the relay RY3, the relay RY3 is connected to the electrical wiring port P5, and an emergency switch S3 is connected in parallel across the two ends of the relay RY3. The emergency switch S3 is connected to the electrical wiring port P5. The relay driver circuit U5 is connected to the relay RY4, the relay RY4 is connected to the electrical wiring port P5, and an emergency switch S4 is connected in parallel across the two ends of the relay RY4. The emergency switch S4 is connected to the electrical wiring port P5. The D2 output terminal of the controller 1 is also connected to a resistor R3 and a light-emitting diode D1 and then grounded;

[0123] As Figure 5 shown, the D5 port of the controller 1 is connected to the external switch interface P3 and then grounded. The external switch interface P3 is connected to the switch 8. The D6 port of the controller 1 is connected to a resistor R6 and a light-emitting diode D3 and then grounded. The D7 port of the controller 1 is connected to a resistor R7 and a light-emitting diode D4 and then grounded. The D8 port of the controller 1 is connected to a resistor R9 and a light-emitting diode D5 and then grounded.

[0124] The controller 1 uses a single-chip microcomputer of the model ESP8266MOD. The ESP8266MOD is connected to the MQTT server through a WIFI module, receives the control commands issued by the MQTT server, and controls the relay actions. The relay driver circuit is driven by a dedicated relay driver chip 3105. When a low-level signal is given in the input signal section, the relay conducts, the CH section and the L section conduct, and the electrical appliance works. The relay driver circuit is controlled by two control signals Relay1 and Relay2. The control signals are connected to the pins of the ESP8266, and the light-emitting diodes D1 and D2 display the states of the control signals.

[0125] The light-emitting diode D3 displays the WIFI connection status. When the WIFI is not connected, D3 blinks at a frequency of 2 Hz, and when it is normally connected, it is constantly on. The light-emitting diode D4 displays the connection status between the controller and the server. When the server is not normally connected, D4 blinks at a frequency of 2 Hz, and when it is normally connected, it blinks at a frequency of 0.5 Hz. The light-emitting diode D5 displays the state of the relay. When the relay is off, D5 is constantly off, and when the relay is on, D5 is constantly on. The switch 8 is connected to the controller 1. Through the cooperation of the switch 8 and the controller 1, all electrical appliances in public places can be turned off with one key. When the WIFI is abnormal and the server is abnormal and the relay cannot work, the emergency switches S1, S2, S3, and S4 are used to control the on and off of the circuit respectively.

[0126] The present invention is not limited to the above specific embodiments. Changes, modifications, additions, or substitutions made by those of ordinary skill in the art within the essence of the present invention shall also fall within the protection scope of the present invention.

Claims

1. An intelligent energy control method for public places based on video detection, characterized in that: It includes the following steps: S1. Initialize parameters. At the same time, the system loads a set list of tags, and the list contains the object types that the model can recognize; S2. Load the YOLO model used for object detection; S3. Communicate with the video server within the local area network through RTSP, request and control the playback and transmission of the streaming media, and obtain the video stream of the surveillance camera; S4. Process each frame of the video stream with the loaded YOLO model; S5. While the YOLO model processes each frame of the video, the frame subtraction method is added to process the picture; S6. Make an execution action decision according to the picture processing results of steps S4 and S5; S7. According to the decision in step S6, perform different actions and send the information of the required actions to the MQTT server; S8. The MQTT server connects to the controller through the WIFI module. After receiving the control command sent by the MQTT server, the controller controls the relay to work according to the control command to turn off or turn on the electrical appliance; In step S2, the loaded YOLO model combines the YOLO algorithm and Transformer technology, and the Transformer technology is used to simplify the YOLO model. The simplification steps are as follows: S21. Collect data: Obtain the surveillance pictures in public places containing people; S22. Label data: Use the labeling tool to label the bounding boxes of people for the images in the dataset; S23. Download pre-trained weights: Do not train the model from scratch. Before training, use the weights pre-trained on a large-scale dataset as the starting point; S24. Integrate the Transformer architecture design: On the basis of the traditional YOLO model, the Transformer architecture is integrated; S25. Adjust the loss function: Adjust the loss function of the YOLO model; S26. Train: Use the labeled dataset to train the YOLO model combined with Transformer; S27. Validate and evaluate: Evaluate its performance by testing the model on an independent validation set; S28. Simplify the model: Reduce the consumption of computing resources and streamline the model by optimizing the Transformer module and adjusting the input resolution; In step S5, the steps for the frame subtraction method to process the picture are as follows: S51. Always save the video of the last five minutes in the background; S52. Divide the video picture into 4*4 regions on average; S53. In each region, take the average number of frames per second to get the average frame per second. Subtract the average frame of the next second from the average frame of one second. If there is no one, the picture is black. If there is someone, there is a person's edge line in the picture; S54. Extract the data and draw it into a curve graph to draw the picture change curve of this time period. If the picture does not change, it is the "no person state", and the curve is relatively stable. If the picture changes, it is the "someone state", and the curve fluctuates; S55. Through the analysis of the video pictures in the recent five minutes, judge whether there is a target in the distance, so as to assist the YOLO model processing results to make a decision.

2. The method for intelligent energy control in public places based on video detection according to claim 1, wherein: The initialized parameters include: whether the program is currently in the presence of people; whether the actions to be performed in the "absence of people state" have been run; recording the start time of the "absence of people state"; detecting whether the program has just been started; customizing how often the actions performed in the "absence of people state" are executed in the "absence of people state".

3. The intelligent energy control method for public places based on video detection according to claim 1, characterized in that: In step S2, the method used to load the YOLO model is an asynchronous loading method.

4. The energy intelligent control method for public places based on video detection according to claim 1, wherein: In step S4, the steps for video stream processing are as follows: S41. Create a queue with a maximum capacity of 2 for transferring image data between the "image input method" and the "image processing method". S42. Create two processes, one for image input and the other for image processing. S43. Input the video stream obtained in process S3 into the queue frame by frame. Since the queue has the feature of first in first out, only the queue size needs to be controlled. If the number of image frames in the queue exceeds 1, take out one from the queue to keep the queue size and ensure that the image frame in the queue is the current latest picture. Among them, when the image frame is the current latest picture, the processing steps for each image frame are as follows: S431. First, obtain the names of all layers in the network, obtain the indices of the unconnected output layers, and use these layers to obtain the final output of the YOLO model. S432. Obtain a frame of image from the image queue, obtain the height, width, and number of channels of the image, and convert the image into the input format required by the YOLO model, that is, convert the image into blob format. S433. Input the blob into the YOLO model for forward propagation to obtain the output of the model. S434. Parse the output of the YOLO model to extract the category, confidence, and bounding box information of the detected objects; loop through each piece of information in the model output to obtain the confidence of the detection result for the human category, filter out the detection results with low confidence by setting a threshold, and extract the bounding box information of the detected human objects, including the upper left corner coordinates (x, y), width w, and height h. S435. Use non-maximum suppression to filter out the bounding boxes with high overlap and retain the most representative detection results. S436. Traverse the valid bounding boxes, draw rectangular boxes and labels on the image frame, and annotate them through a pre-set label list.

5. The method for intelligent energy control in public places based on video detection according to claim 1, wherein: The actions in step S7 include actions in the "presence of people state" and actions in the "absence of people state". Among them, the actions in the "presence of people state" are as follows: S71-1. Obtain the location where the current image is taken. S71-2. Detect whether an MQTT connection has been established. If not, establish the connection; if so, proceed to the next step. S71-3. Construct the message to be sent {"ClassRoom": f"{num}","DevNo": "01","Value": "1"}, including the room name ClassRoom, device name DevNo, and message content Value. S71-4. Send the message to the MQTT server in json format. The actions in the "absence of people state" are as follows: S72-1. Obtain the location where the current image is taken. S72-2. Detect whether the MQTT connection has been established. If not, establish the connection; if so, proceed to the next step. S72-3. Construct the message to be sent {"ClassRoom": f"{num}","DevNo": "01","Value": "0"}, including the room name ClassRoom, device name DevNo, and message content Value. S72-4. Send the message to the MQTT server in JSON format.

6. A control system for the intelligent control method of public place energy based on video detection as described in claim 1, characterized in that: It includes a controller (1). The power supply terminal of the controller (1) is connected to the power supply (2). The signal input terminal of the controller (1) is connected to the computer (9) through the WIFI module and the MTQQ server. The computer (9) communicates with the video server (10) through RTSP. The control output terminal of the controller (1) is connected to the relay drive module (3). The relay drive module (3) is connected to the relay (4). The relay (4) is connected to the electrical wiring terminal (5). The electrical wiring terminal (5) is connected to the electrical appliance (6). The output interface of the controller (1) is connected to the indicator light (7). The input interface of the controller (1) is connected to the switch (8).

7. The intelligent energy control system for public places based on video detection according to claim 6, characterized in that: An emergency switch circuit (11) is connected in parallel across both ends of the relay (4). The emergency switch circuit (11) is connected to the power supply (2) and the electrical wiring terminal (5).

8. The intelligent energy control system for public places based on video detection according to claim 6, wherein: The 5VIN power supply terminal of the controller (1) is connected to the 5V power output terminal of the AC-DC power converter P1. The 5VIN power supply terminal of the controller (1) is also connected to the resistor R10 and the light-emitting diode D6 and then grounded. The D1 output terminal of the controller (1) is connected to the relay drive circuits U2 and U3. The relay drive circuit U2 is connected to the relay RY1. The relay RY1 is connected to the electrical wiring port P4. An emergency switch S1 is connected in parallel across both ends of the relay RY1. The emergency switch S1 is connected to the electrical wiring port P4. The relay drive circuit U3 is connected to the relay RY2. The relay RY2 is connected to the electrical wiring port P4. An emergency switch S2 is connected in parallel across both ends of the relay RY2. The emergency switch S2 is connected to the electrical wiring port P4. The D1 output terminal of the controller (1) is also connected to the resistor R4 and the light-emitting diode D2 and then grounded. The D2 port of the controller (1) is connected to the relay drive circuits U4 and U5. The relay drive circuit U4 is connected to the relay RY3. The relay RY3 is connected to the electrical wiring port P5. An emergency switch S3 is connected in parallel across both ends of the relay RY3. The emergency switch S3 is connected to the electrical wiring port P5. The relay drive circuit U5 is connected to the relay RY4. The relay RY4 is connected to the electrical wiring port P5. An emergency switch S4 is connected in parallel across both ends of the relay RY4. The emergency switch S4 is connected to the electrical wiring port P5. The D2 output terminal of the controller (1) is also connected to the resistor R3 and the light-emitting diode D1 and then grounded. The D5 port of the described controller (1) is connected to the external switch interface P3 and then grounded, and the external switch interface P3 is connected to the switch (8); the D6 port of the controller (1) is connected to the resistor R6 and the light-emitting diode D3 and then grounded; the D7 port of the controller (1) is connected to the resistor R7 and the light-emitting diode D4 and then grounded; the D8 port of the controller (1) is connected to the resistor R9 and the light-emitting diode D5 and then grounded.

Images