A fan operating status recognition algorithm and device based on video analysis

Abstract

The present invention proposes a fan operation status identification algorithm and device based on video analysis, wherein the method includes the following steps: first training a fan detection model, and then training a fan state classification model. The two models set up can realize that the position of the fan is automatically detected and obtained by the model without manual prior calibration. Fan detection is only performed on key frames, which effectively reduces the amount of calculation. The device includes a video acquisition module, a fan positioning module and a state identification module. The video acquisition module is respectively connected to the fan positioning module and the state identification module, and the fan positioning module is connected to the state identification module. The present application adopts a deep learning algorithm to collect video data to analyze the operation status of fans in substations and converter stations in real time, and issues alarms for abnormal conditions, thereby improving the reliability of substation and converter station systems.

Description

Technical Field

[0001] The present invention relates to the field of transformer substations and battery swap stations, and in particular to a fan operating status recognition algorithm and device based on video analysis. Background Art

[0002] Many devices in substations and converter stations require fans for heat dissipation. Therefore, the ability to detect the operating status of fan equipment in real time is important for the normal operation of substation and converter station systems.

[0003] However, the fan detection device in the prior art has the following disadvantages:

[0004] 1. Sensor hardware increases hardware costs, and the cost of modifying existing fan equipment is huge;

[0005] 2. Fan status monitoring is located at the fan end, lacking system redundancy.

[0006] The patent, CN212657024U, titled "A Fan Status Detection Circuit," was filed by the Institute of Inspection and Quarantine. It proposes a hardware-based detection circuit. This solution uses a control chip to measure the voltage at the fan interface and, by comparing the actual voltage with relevant fan parameters, calculates the fan's speed, thereby determining the fan's operating status.

[0007] Patent publication number CN113606176B, titled "A Method and Apparatus for Detecting Fan Operating Status," proposes a sensor-based solution. The steps of this solution are:

[0008] 1. Send a speed control signal to the fan through the baseboard controller

[0009] 2. The fan controller obtains the signal from the fan speed sensor and converts the signal frequency into the fan speed. By comparing the difference between the two speeds, it is determined whether the fan is operating normally.

[0010] The disadvantage of the "A Fan Status Detection Circuit" is that it uses the voltage value at the fan input to monitor the fan's operating status, which cannot truly reduce system errors. The purpose of fan status monitoring is to promptly detect abnormal fan conditions. This solution directly uses the fan's voltage signal. If the fan device experiences an abnormality, it cannot be detected, resulting in a self-monitoring contradiction. Possible reasons for failure include: voltage sensor failure, normal voltage but fan shaft failure. The disadvantage of the "A Fan Operating Status Detection Method and Device" is that it requires the fan to have a speed sensor, which is costly for retrofitting existing fan devices. On the other hand, the monitoring system (speed sensor) is still coupled to the fan device, which cannot meet more stringent reliability requirements. For example, the system expects an alarm to be issued when the fan stops abnormally. However, if the fan is abnormal, it is very likely that the speed sensor or fan controller will also be abnormal, and the alarm may not be issued.

[0011] However, the above two technical solutions are both based on dedicated circuits and sensors, and still cannot solve the two technical problems of the above technologies well. For this reason, this application proposes a solution based on video analysis, which can use a camera to obtain video data of the fan, and realize real-time monitoring of the fan's operating status through analysis of the video data, effectively reducing enterprise costs. Moreover, the system is independently deployed and simple and convenient to operate. Summary of the Invention

[0012] To solve the above technical problems, the present invention proposes a fan operating status identification algorithm and device based on video analysis. By adopting a deep learning algorithm and collecting video data, the operating status of fans in substations and converter stations can be analyzed in real time, and alarms can be issued for abnormal conditions, thereby improving the reliability of substation and converter station systems.

[0013] To achieve the above object, the technical solution adopted by the present invention is:

[0014] A fan operating status recognition algorithm based on video analysis is characterized by comprising the following steps:

[0015] S1: Train the fan detection model. The training steps are:

[0016] S11: Use a camera to randomly collect pictures of fans in the substation and converter station, and manually mark the circumscribed rectangular box of each fan in the picture;

[0017] S12: Construct a yolov5 target detection network model, the number of categories detected by the network is 1;

[0018] S13: Input the data marked in step S11 into the yolov5 target detection network model in step S12, and use the stochastic gradient descent method to train the yolov5 target detection network model until the yolov5 target detection network model converges;

[0019] S2: Train the fan status classification model. The training steps are as follows:

[0020] S21: Use a camera to randomly collect videos of the fan running, mark each video as running, and create video set 1; then randomly collect videos of the fan stopping, mark each video as stopping, and create video set 2; merge the above two video sets into a total video set;

[0021] S22: For each video in step S21, convert it into a set of continuous image frame sequences, and label each set of image frame sequences with a label corresponding to the video;

[0022] S23: For each set of image frame sequences in step S22, take the first frame image and use the aforementioned fan detection model to detect the circumscribed rectangle of the fan;

[0023] S24: Using the circumscribed rectangle of the fan in step S23, crop a sub-image from each image in the image frame sequence in step S22, and arrange all sub-images in the group in the original order to form a new sub-image sequence. The new sub-image sequence is labeled with the label of the corresponding image frame sequence. Each image frame sequence in step S22 corresponds to a sub-image sequence.

[0024] S25: For all sub-image sequences in step S24, let i equal 1. Starting from the i-th image, take N images in succession, scale each image to 128*128, and then divide each RGB channel of the image by 255 to obtain a normalized image sequence of length N. This image sequence together with the corresponding label is used as a training sample;

[0025] S26: Let i be 2, 3, ..., and repeat step S25 until N consecutive images cannot be obtained, and merge all training samples into a training sample set;

[0026] S27: Construct an image binary classification model based on convolutional long short-term memory model;

[0027] S28: Input the training sample set in step S26 into the image binary classification model in step S27, and use the stochastic gradient descent method to train the image binary classification model until the image binary classification model converges.

[0028] As a preferred technical solution of the present invention: in step S1, the fan detection model can be deployed in all fan positioning modules after being trained once.

[0029] In the above structure: the method of the present invention can realize the training of the fan detection model and the fan status classification model. By setting the two models, the position of the fan can be automatically detected and obtained by the model without manual prior calibration. Fan detection is only performed on key frames, which effectively reduces the amount of calculation.

[0030] The system can also capture the target fan region based on the fan's position within a continuous sequence of video frames and stitch them together into a continuous sub-image sequence showing only the target fan. A neural network model (including a range of different architectures, such as 3D convolutional networks, long short-term memory networks, and transformer networks, with the only requirement being that the model's input be a sequence of images of length n, each of the same size) is used to classify these sub-image sequences and determine whether the fan was operating within that time interval, thereby identifying fan status information.

[0031] The present invention can detect the positions of multiple fans on video key frames. By reading video data and converting it into continuous image frames, the target fan area is intercepted on a continuous video frame sequence according to the detected fan positions and spliced ​​into a continuous sub-image sequence containing only the target fans. Then, a neural network model is used to input the sub-image sequence of each fan and directly output the operating status of the fan.

[0032] A device for a fan operating status recognition algorithm based on video analysis is characterized by comprising a video acquisition module, a fan positioning module, and a status recognition module, wherein the video acquisition module is connected to the fan positioning module and the status recognition module respectively, and the fan positioning module is connected to the status recognition module. The video acquisition module is used to collect video data of the fan and use it as input to the fan positioning module and the status recognition module. The fan positioning module identifies the position of each fan from the input image, and the status recognition module reads the position of the fan and identifies the status information of each fan from a continuous video frame sequence.

[0033] As a preferred technical solution of the present invention: the video capture module includes a camera installed in front of the fan, a video analysis server, and network cables and supporting equipment for connecting the camera and the server. The camera captures video stream data and transmits it to the video analysis server. The video analysis server converts the video stream into a continuous image sequence and inputs it into the fan positioning module and the state recognition module respectively. The fan positioning module reads the key frames in the image sequence, and the state recognition module reads the frame sequence in the image sequence.

[0034] As a preferred technical solution of the present invention: the camera is a fixed camera or a rotating dome camera. When the camera is a fixed camera, the key frame read by the fan positioning module is the first frame of the image sequence. When the camera is a rotating dome camera, the key frame read by the fan positioning module is the first frame of the image sequence captured after the rotating dome camera stopped rotating for the last time.

[0035] As a preferred technical solution of the present invention, the key frame is scaled to 640*640, and each RGB channel of the image is divided by 255 to obtain a normalized key frame. The normalized key frame is input into the fan detection model, and all results with confidence levels greater than a set threshold are retained. In this way, all fans on the key frame and their position coordinates are obtained.

[0036] As a preferred technical solution of the present invention: when the camera is set up, it can ensure that all fans account for no less than 25% of the end point of the video screen, and the deflection angle in each direction is no more than 30 degrees.

[0037] As a preferred technical solution of the present invention: the minimum resolution of the camera is 1280*720, and the minimum frame rate is 25 frames per second.

[0038] As a preferred technical solution of the present invention: the video analysis server is a single computer or a distributed computer system and a necessary computer software system based on the hardware of the computer system.

[0039] As a preferred technical solution of the present invention: the recognition process of the state recognition module is as follows:

[0040] a. Read the video frame sequence F=[F0, F1, F2, ..., F n ], the video frame sequence is uniformly sampled in the time dimension with a sampling interval r, and F s =[F0,F 1*r ,F 2*r ,...,F m*r ], as the array of frames to be detected;

[0041] b. For the above frame array F s Group them, each group contains F s The size of each group of consecutive video frames is N, the order remains unchanged, and they do not overlap, then the i-th group is

[0042] c. For each video group According to the fan position output by the fan positioning module, for each fan, each fan is intercepted from each video frame in the group and saved as a separate sub-image. Then the sub-image sequence of the j-th fan in the i-th video frame array is

[0043] For each fan sub-image sequence in each group Each sub-image is resized to a uniform size and pre-processed before being fed into a trained fan status classification model. The model then outputs whether the fan is running, i.e., the fan's running status within the time period corresponding to this set of video frames.

[0044] In the above structure, the device for fan operating status recognition based on video analysis includes three components: a video acquisition module, a fan positioning module, and a status recognition module. The video acquisition module captures video data of the fans, which serves as input to the fan positioning module and the status recognition module. The fan positioning module identifies the position of each fan from the input image. The status recognition module reads the fan position and identifies the operating status of each fan from a sequence of continuous video frames.

[0045] The core of the video acquisition module is a camera mounted in front of the fan, a video analysis server, and the network cables and equipment connecting the camera and server. The camera can reuse existing cameras in substations and converter stations, or use a dedicated camera, as long as it meets the minimum resolution of 1280*720 and a frame rate of 25 frames per second. The camera installation conditions must be such that all fans occupy no less than 25% of the video image end point, and the deflection angle in all directions is no more than 30 degrees. The video analysis server does not refer to a specific computer model; it can be a single computer or a distributed computer system, as well as the necessary computer software system based on the computer hardware. The camera captures the video stream data and transmits it to the video analysis server, which converts the video stream into a continuous image sequence.

[0046] The fan positioning module reads the key frame in the image sequence. For a fixed camera, the key frame is generally the first frame of the image sequence; for a rotatable camera such as a dome camera, the key frame is the first frame of the image sequence captured after the camera last stopped rotating.

[0047] Scale the keyframe to 640*640 and divide each RGB channel of the image by 255 to obtain a normalized keyframe. Input the normalized keyframe into the fan detection model and retain all results with a confidence score greater than a set threshold. This yields all fans and their location coordinates on the keyframe.

[0048] The process of the state recognition module:

[0049] a) Read the video frame sequence F=[F0,F1,F2,...,F n ], the video frame sequence is uniformly sampled in the time dimension with a sampling interval r, and F s=[F0,F 1*r ,F 2*r ,...,F m*r ], as the array of frames to be detected;

[0050] b) For the above frame array F s Group them, each group contains F s The size of each group of consecutive video frames is N, the order remains unchanged, and they do not overlap, then the i-th group is

[0051] c) For each video group According to the fan position output by the fan positioning module, for each fan, each fan is intercepted from each video frame in the group and saved as a separate sub-image. Then the sub-image sequence of the j-th fan in the i-th video frame array is

[0052] For each fan sub-image sequence in each group Each sub-image is resized to a uniform size and pre-processed before being fed into a trained fan status classification model. The model then outputs whether the fan is running, i.e., the fan's running status within the time period corresponding to this set of video frames.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] This application uses a camera to obtain video data of the fan, and realizes real-time monitoring of the fan operation status by analyzing the video data.

[0055] In this application, the fan position is automatically detected by the algorithm, without manual calibration. Fan detection is only performed on key frames to reduce the amount of calculation.

[0056] This application can capture the target fan area based on the fan's position in a continuous sequence of video frames and splice it into a continuous sub-image sequence that only shows the target fan. A neural network model (including a range of neural network models with different architectures, such as 3D convolutional networks, long short-term memory networks, and transformer networks, with the only requirement being that the model's input is a sequence of images of length n, each of the same size) is used to classify the sub-image sequence and determine whether the fan was operating during that time interval.

[0057] The present invention effectively realizes real-time monitoring of fan operation status based on video data analysis. The system is independently deployed and is not restricted by fan model, and there is no need to replace existing fans. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a schematic diagram of the principle framework of the present invention;

[0059] Figure 2 This is a schematic diagram of the installation of the video acquisition module in the present invention;

[0060] Figure 3 It is a flow chart of the state recognition module in the present invention. DETAILED DESCRIPTION

[0061] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0062] The present invention proposes a fan operating status recognition algorithm based on video analysis, which includes the following steps:

[0063] S1: Train the fan detection model. The training steps are:

[0064] S11: Use a camera to randomly collect pictures of fans in the substation and converter station, and manually mark the circumscribed rectangular box of each fan in the picture;

[0065] S12: Construct a yolov5 target detection network model, the number of categories detected by the network is 1;

[0066] S13: Input the data marked in step S11 into the yolov5 target detection network model in step S12, and use the stochastic gradient descent method to train the yolov5 target detection network model until the yolov5 target detection network model converges;

[0067] S2: Train the fan status classification model. The training steps are as follows:

[0068] S21: Use a camera to randomly collect videos of the fan running, mark each video as running, and create video set 1; then randomly collect videos of the fan stopping, mark each video as stopping, and create video set 2; merge the above two video sets into a total video set;

[0069] S22: For each video in step S21, convert it into a set of continuous image frame sequences, and label each set of image frame sequences with a label corresponding to the video;

[0070] S23: For each set of image frame sequences in step S22, take the first frame image and use the aforementioned fan detection model to detect the circumscribed rectangle of the fan;

[0071] S24: Using the circumscribed rectangle of the fan in step S23, crop a sub-image from each image in the image frame sequence in step S22, and arrange all sub-images in the group in the original order to form a new sub-image sequence. The new sub-image sequence is labeled with the label of the corresponding image frame sequence. Each image frame sequence in step S22 corresponds to a sub-image sequence.

[0072] S25: For all sub-image sequences in step S24, let i equal 1. Starting from the i-th image, take N images in succession, scale each image to 128*128, and then divide each RGB channel of the image by 255 to obtain a normalized image sequence of length N. This image sequence together with the corresponding label is used as a training sample;

[0073] S26: Let i be 2, 3, ..., and repeat step S25 until N consecutive images cannot be obtained, and merge all training samples into a training sample set;

[0074] S27: Construct an image binary classification model based on convolutional long short-term memory model;

[0075] S28: Input the training sample set in step S26 into the image binary classification model in step S27, and use the stochastic gradient descent method to train the image binary classification model until the image binary classification model converges.

[0076] In step S1, the fan detection model can be deployed in all fan positioning modules after being trained once.

[0077] The method of the present invention can realize the training of the fan detection model and the fan status classification model. By setting the two models, the fan position can be automatically detected and obtained by the model without manual prior calibration. Fan detection is only performed on key frames, which effectively reduces the amount of calculation.

[0078] The system can also capture the target fan region based on the fan's position within a continuous sequence of video frames and stitch them together into a continuous sub-image sequence showing only the target fan. A neural network model (including a range of different architectures, such as 3D convolutional networks, long short-term memory networks, and transformer networks, with the only requirement being that the model's input be a sequence of images of length n, each of the same size) is used to classify these sub-image sequences and determine whether the fan was operating within that time interval, thereby identifying fan status information.

[0079] The present invention can detect the positions of multiple fans on video key frames. By reading video data and converting it into continuous image frames, the target fan area is intercepted on a continuous video frame sequence according to the detected fan positions and spliced ​​into a continuous sub-image sequence containing only the target fans. Then, a neural network model is used to input the sub-image sequence of each fan and directly output the operating status of the fan.

[0080] like Figure 1-3As shown, the present invention also proposes a device for fan operation status recognition algorithm based on video analysis, including a video acquisition module, a fan positioning module and a status recognition module, wherein the video acquisition module is respectively connected to the fan positioning module and the status recognition module, and the fan positioning module is connected to the status recognition module. The video acquisition module is used to collect video data of the fan and use it as input to the fan positioning module and the status recognition module. The fan positioning module identifies the position of each fan from the input image, and the status recognition module reads the position of the fan and identifies the status information of each fan from a continuous video frame sequence.

[0081] The video capture module includes a camera installed in front of the fan, a video analysis server, and network cables and supporting equipment for connecting the camera and the server. The camera captures video stream data and transmits it to the video analysis server. The video analysis server converts the video stream into a continuous image sequence and inputs it into the fan positioning module and the state recognition module respectively. The fan positioning module reads the key frames in the image sequence, and the state recognition module reads the frame sequence in the image sequence.

[0082] The camera is a fixed camera or a rotating dome camera. When the camera is a fixed camera, the key frame read by the fan positioning module is the first frame of the image sequence. When the camera is a rotating dome camera, the key frame read by the fan positioning module is the first frame of the image sequence captured after the rotating dome camera stopped rotating for the last time.

[0083] The keyframe is scaled to 640*640, and each RGB channel of the image is divided by 255 to obtain a normalized keyframe. The normalized keyframe is input into the fan detection model, and all results with confidence levels greater than a set threshold are retained. This means that all fans on the keyframe and their location coordinates are obtained.

[0084] When the camera is set up, the proportion of all fans at the end of the video screen is not less than 25%, and the deflection angle in each direction is not greater than 30 degrees.

[0085] The minimum resolution of the camera is 1280*720, and the minimum frame rate is 25 frames per second.

[0086] The video analysis server is a single computer or a distributed computer system and a necessary computer software system based on the computer system hardware.

[0087] The recognition process of the state recognition module is as follows:

[0088] a. Read the video frame sequence F=[F0, F1, F2, ..., F n], the video frame sequence is uniformly sampled in the time dimension with a sampling interval r, and F s =[F0,F 1*r ,F 2*r ,...,F m*r ], as the array of frames to be detected;

[0089] b. For the above frame array F s Group them, each group contains F s The size of each group of consecutive video frames is N, the order remains unchanged, and they do not overlap, then the i-th group is

[0090] c. For each video group According to the fan position output by the fan positioning module, for each fan, each fan is intercepted from each video frame in the group and saved as a separate sub-image. Then the sub-image sequence of the j-th fan in the i-th video frame array is

[0091] For each fan sub-image sequence in each group Each sub-image is resized to a uniform size and pre-processed before being fed into a trained fan status classification model. The model then outputs whether the fan is running, i.e., the fan's running status within the time period corresponding to this set of video frames.

[0092] The device for fan operating status recognition based on video analysis consists of three components: a video acquisition module, a fan positioning module, and a status recognition module. The video acquisition module captures video data of the fans, which serves as input to the fan positioning and status recognition modules. The fan positioning module identifies the position of each fan from the input image. The status recognition module reads the fan position and identifies the operating status of each fan from a sequence of continuous video frames.

[0093] The core of the video acquisition module is a camera mounted in front of the fan, a video analysis server, and the network cables and equipment connecting the camera and server. The camera can reuse existing cameras in substations and converter stations, or use a dedicated camera, as long as it meets the minimum resolution of 1280*720 and a frame rate of 25 frames per second. The camera installation conditions must be such that all fans occupy no less than 25% of the video image end point, and the deflection angle in all directions is no more than 30 degrees. The video analysis server does not refer to a specific computer model; it can be a single computer or a distributed computer system, as well as the necessary computer software system based on the computer hardware. The camera captures the video stream data and transmits it to the video analysis server, which converts the video stream into a continuous image sequence.

[0094] The fan positioning module reads the key frame in the image sequence. For a fixed camera, the key frame is generally the first frame of the image sequence; for a rotatable camera such as a dome camera, the key frame is the first frame of the image sequence captured after the camera last stopped rotating.

[0095] Scale the keyframe to 640*640 and divide each RGB channel of the image by 255 to obtain a normalized keyframe. Input the normalized keyframe into the fan detection model and retain all results with a confidence score greater than a set threshold. This yields all fans and their location coordinates on the keyframe.

[0096] The process of the state recognition module:

[0097] a) Read the video frame sequence F=[F0,F1,F2,...,F n ], the video frame sequence is uniformly sampled in the time dimension with a sampling interval r, and F s =[F0,F 1*r ,F 2*r ,...,F m*r ], as the array of frames to be detected;

[0098] b) For the above frame array F s Group them, each group contains F s The size of each group of consecutive video frames is N, the order remains unchanged, and they do not overlap, then the i-th group is

[0099] c) For each video group According to the fan position output by the fan positioning module, for each fan, each fan is intercepted from each video frame in the group and saved as a separate sub-image. Then the sub-image sequence of the j-th fan in the i-th video frame array is

[0100] For each fan sub-image sequence in each group Each sub-image is resized to a uniform size and pre-processed before being fed into a trained fan status classification model. The model then outputs whether the fan is running, i.e., the fan's running status within the time period corresponding to this set of video frames.

[0101] This application uses a camera to obtain video data of the fan, and realizes real-time monitoring of the fan operation status by analyzing the video data.

[0102] In this application, the fan position is automatically detected by the algorithm, without manual calibration. Fan detection is only performed on key frames to reduce the amount of calculation.

[0103] This application can capture the target fan area based on the fan's position in a continuous sequence of video frames and splice it into a continuous sub-image sequence that only shows the target fan. A neural network model (including a range of neural network models with different architectures, such as 3D convolutional networks, long short-term memory networks, and transformer networks, with the only requirement being that the model's input is a sequence of images of length n, each of the same size) is used to classify the sub-image sequence and determine whether the fan was operating during that time interval.

[0104] The present invention effectively realizes real-time monitoring of fan operation status based on video data analysis. The system is independently deployed and is not restricted by fan model, and there is no need to replace existing fans.

[0105] The above description is merely a preferred embodiment of the present invention and does not constitute any other form of limitation to the present invention. Any modification or equivalent variation based on the technical essence of the present invention shall still fall within the scope of protection claimed by the present invention.

Claims

1. A fan operating status recognition algorithm based on video analysis, characterized by: The steps include: S1: Train the fan detection model. The training steps are: S11: Use a camera to randomly collect pictures of fans in the substation and converter station, and manually mark the circumscribed rectangular box of each fan in the picture; S12: Construct a yolov5 target detection network model, the number of categories detected by the network is 1; S13: Input the data marked in step S11 into the yolov5 target detection network model in step S12, and use the stochastic gradient descent method to train the yolov5 target detection network model until the yolov5 target detection network model converges; S2: Train the fan status classification model. The training steps are as follows: S21: Use a camera to randomly collect videos of the fan running, mark each video as running, and create video set 1; then randomly collect videos of the fan stopping, mark each video as stopping, and create video set 2; merge the above two video sets into a total video set; S22: For each video in step S21, convert it into a set of continuous image frame sequences, and label each set of image frame sequences with a label corresponding to the video; S23: For each set of image frame sequences in step S22, take the first frame image and use the aforementioned fan detection model to detect the circumscribed rectangle of the fan; S24: Using the circumscribed rectangle of the fan in step S23, crop a sub-image from each image in the image frame sequence in step S22, and arrange all sub-images in the group in the original order to form a new sub-image sequence. The new sub-image sequence is labeled with the label of the corresponding image frame sequence. Each image frame sequence in step S22 corresponds to a sub-image sequence. S25: For all sub-image sequences in step S24, let i equal 1. Starting from the i-th image, take N images in succession, scale each image to 128*128, and then divide each RGB channel of the image by 255 to obtain a normalized image sequence of length N. This image sequence together with the corresponding label is used as a training sample; S26: Let i be 2, 3, ..., and repeat step S25 until N consecutive images cannot be obtained, and merge all training samples into a training sample set; S27: Construct an image binary classification model based on convolutional long short-term memory model; S28: Input the training sample set in step S26 into the image binary classification model in step S27, and use the stochastic gradient descent method to train the image binary classification model until the image binary classification model converges.

2. The fan operating status recognition algorithm based on video analysis according to claim 1, characterized in that: In step S1, the fan detection model can be deployed in all fan positioning modules after being trained once.

3. The device for fan operating status recognition algorithm based on video analysis according to claim 1, characterized in that: The system includes a video acquisition module, a fan positioning module, and a state recognition module. The video acquisition module is connected to the fan positioning module and the state recognition module respectively, and the fan positioning module is connected to the state recognition module. The video acquisition module is used to collect video data of the fans and use it as input to the fan positioning module and the state recognition module. The fan positioning module identifies the position of each fan from the input image, and the state recognition module reads the position of the fan and identifies the state information of each fan from a continuous video frame sequence.

4. The device for fan operating status recognition algorithm based on video analysis according to claim 3, characterized in that: The video capture module includes a camera installed in front of the fan, a video analysis server, and network cables and supporting equipment for connecting the camera and the server. The camera captures video stream data and transmits it to the video analysis server. The video analysis server converts the video stream into a continuous image sequence and inputs it into the fan positioning module and the state recognition module respectively. The fan positioning module reads the key frames in the image sequence, and the state recognition module reads the frame sequence in the image sequence.

5. The device for fan operating status recognition algorithm based on video analysis according to claim 4, characterized in that: The camera is a fixed camera or a rotating dome camera. When the camera is a fixed camera, the key frame read by the fan positioning module is the first frame of the image sequence. When the camera is a rotating dome camera, the key frame read by the fan positioning module is the first frame of the image sequence captured after the rotating dome camera stopped rotating for the last time.

6. The device for fan operating status recognition algorithm based on video analysis according to claim 1 or 5, characterized in that: The keyframe is scaled to 640*640, and each RGB channel of the image is divided by 255 to obtain a normalized keyframe. The normalized keyframe is input into the fan detection model, and all results with confidence levels greater than a set threshold are retained. This means that all fans on the keyframe and their location coordinates are obtained.

7. The device for fan operating status recognition algorithm based on video analysis according to claim 5, characterized in that: When the camera is set up, the proportion of all fans at the end of the video screen is not less than 25%, and the deflection angle in each direction is not greater than 30 degrees.

8. The device for fan operating status recognition algorithm based on video analysis according to claim 7, characterized in that: The minimum resolution of the camera is 1280*720, and the minimum frame rate is 25 frames per second.

9. The device for fan operating status recognition algorithm based on video analysis according to claim 8, characterized in that: The video analysis server is a single computer or a distributed computer system and a necessary computer software system based on the computer system hardware.

10. The device for fan operation status recognition algorithm based on video analysis according to claim 4, characterized in that: The recognition process of the state recognition module is as follows: a. Read the video frame sequence F=[F0, F1, F2, ..., F n ], the video frame sequence is uniformly sampled in the time dimension with a sampling interval r, and F s =[F0,F 1*r ,F 2*r ,...,F m*r ], as the array of frames to be detected; b. For the above frame array F s Group them, each group contains F s The size of each group of consecutive video frames is N, the order remains unchanged, and they do not overlap, then the i-th group is c. For each video group According to the fan position output by the fan positioning module, for each fan, each fan is intercepted from each video frame in the group and saved as a separate sub-image. Then the sub-image sequence of the j-th fan in the i-th video frame array is For each fan sub-image sequence in each group Each sub-image is resized to a uniform size and pre-processed before being fed into a trained fan status classification model. The model then outputs whether the fan is running, i.e., the fan's running status within the time period corresponding to this set of video frames.

Images