Cabinet door state video detection model training method, detection method and related device

Abstract

The invention relates to the technical field of video detection, in particular to a cabinet door state video detection model training method, a cabinet door state video detection method and a related device, and the cabinet door state video detection model training method comprises the steps of obtaining a target detection main sample set and a target detection auxiliary sample set; training the improved YOLOV5 network model by using the training sample subset and the verification sample subset to obtain a target detection model; and training the MobileNetV2 classifier by using the target detection secondary sample set to obtain a target classification model. A cabinet door state video detection model takes a YOLOV5 algorithm as a main body, three parts of a backbone network, a feature pyramid network and a network header are improved, an Autogument data enhancement strategy is introduced, a historical cabinet door state monitoring video frame cutting training sample is used for training to obtain a target detection model, and the target detection model is used for detecting the cabinet door state. The generalization ability and the detection ability of the model are effectively improved through rich training samples and an improved YOLOV5 algorithm.

Description

Technical Field

[0001] The present invention relates to the technical field of video detection, in particular to a cabinet door state video detection model training method, a detection method and related devices. Background Art

[0002] With the construction of production capacity automation over the years, the number of oilfield automation equipment in use has increased significantly. Most of the control equipment is located in the control cabinet (box) at the oilfield site, with scattered installation and harsh operating environment. If the control cabinet (box) door is not closed in time in working conditions such as wind, sand, and water vapor, the harsh working conditions will cause short circuits in the cabinet circuits, component burnout, etc., causing various equipment to stop, seriously affecting production. In addition, the number of control cabinet (box) doors in the oilfield is huge and the distribution range is extremely wide. It is difficult to detect and deal with the phenomenon of cabinet doors not being closed or locked in time. At this time, with the help of cameras widely distributed at the oilfield site, video analysis methods can be used to realize automatic detection of unclosed control cabinet doors, realize all-weather and all-round supervision, and play an important role in ensuring the normal production of oilfield development.

[0003] By using the target detection method, the switch detection of the control cabinet (box) door is realized. The traditional target detection method can only judge the switch status of the distribution box door, and cannot identify more detailed situations, such as whether the box door is locked. In this regard, video analysis methods are often used for target detection to determine the status of the control cabinet (box) door. The general methods of existing video analysis are: Step 1: Use monitoring equipment to obtain monitoring video; Step 2: Extract relevant feature images from the surveillance video in step 1; Step 3: Manually annotate the images obtained in step 2 as sample data; 75% of the annotated sample data are used as training samples, and 25% of the annotated sample data are used as test samples; Step 4: Construct a neural network model, and use the training samples and test samples to train the neural network model to obtain a trained neural network model; Step 5: Use monitoring equipment to obtain real-time monitoring video, and obtain N consecutive frames of images to be tested at intervals as a group of images to be tested; Step 6: Input the test images belonging to the same test group into the trained neural network model to extract the coordinates of the target; Step 7: Calculate the intersection-and-union ratio of the target in the same frame of the image to be tested; if the intersection-and-union ratio of the target is less than the set threshold, it is considered that the corresponding target is not matched, and the target is counted as temporarily unmonitored, otherwise it is counted as successful; Step 8: Repeat the actions in step 5 to obtain the next group to be tested, and continue to repeat steps 6 and 7.

[0004] The above video analysis process can achieve accurate recognition of static targets under sufficient annotation, but in some specific scenarios, this method has the following defects: 1. The neural network used in the model has weak generalization and insufficient recognition of targets; 2. There are quite a lot of small target types in the control cabinet door pictures, which have little information. The above training process did not effectively process the samples, resulting in poor model training results. Summary of the invention

[0005] The present invention provides a cabinet door status video detection model training method, detection method and related devices, which overcome the shortcomings of the above-mentioned prior art and can effectively solve the problems of poor model training effect and inaccurate model detection in the control cabinet door status detection method.

[0006] One of the technical solutions of the present invention is achieved by the following measures: a cabinet door state video detection model training method, comprising: Obtain a target detection main sample set and a target detection sub-sample set, and divide the target detection main sample set into a training sample subset and a verification sample subset, wherein the cabinet door monitoring picture in the target detection sub-sample set is the same as the training sample subset, the training sample subset includes a number of training samples, each training sample includes a cabinet door monitoring picture marked with a cabinet door part and a label identifying the cabinet door state, the verification sample subset includes a number of verification samples, each verification sample includes a cabinet door monitoring picture and a label identifying the cabinet door state, and the target detection sub-sample set includes a number of detection sub-samples, each detection sub-sample includes a cabinet door monitoring picture and a label identifying the cabinet door state; The improved YOLOV5 network model is trained using the training sample subset and the validation sample subset to obtain a target detection model, wherein the improved YOLOV5 network model is based on the YOLOV5 network model, the backbone network, feature pyramid network, and network head in the model are improved, and the Autoaugment automatic data enhancement set is introduced to obtain the improved YOLOV5 network model; Use the model averaging algorithm to average the weights of the target detection model; Use the target detection sub-sample set to train the MobileNetV2 classifier to obtain the target classification model; Based on the trained target detection model and target classification model, a cabinet door status video detection model is established.

[0007] The following are further optimizations and / or improvements to the above technical solutions: The above-mentioned improved YOLOV5 network model includes ConvNeXt-Tiny backbone network, P6 layer feature pyramid network, decoupled network head and Autoaugment automatic data enhancement set.

[0008] The above method obtains the target detection main sample set and the target detection secondary sample set, and divides the target detection main sample set into a training sample subset and a verification sample subset, including: Obtain historical cabinet door status monitoring videos, intercept the historical cabinet door status monitoring videos by frame, obtain several cabinet door monitoring pictures, and form a main sample set for target detection; Divide the target detection main sample set into a training sample subset and a verification sample subset according to a set ratio; Duplicate the training sample subset to construct a secondary sample set for target detection; The door monitoring images in the training sample subset and the target detection sub-sample set are processed using cropping and scaling rules.

[0009] The above cropping and scaling rules include: The door parts of each door monitoring image in the training sample subset are labeled using the true value annotation frame to complete the processing of the training sample subset; The door part of each door monitoring image in the target detection sub-sample set is annotated using the true value annotation frame, the annotated part of each door monitoring image is enlarged to twice its original size and then cropped, and the cropped door monitoring image is reduced to a size of 384 points.

[0010] The second technical solution of the present invention is achieved by the following measures: a cabinet door status video detection method, comprising: Acquire real-time monitoring video of the work site, and acquire N consecutive frames of monitoring images of the work site to be tested every N frames; The monitoring image of the work site to be tested is input into the target detection model, and the position information of the control cabinet door is obtained through non-maximum suppression, wherein the target detection model is trained using the cabinet door status video detection model training method; Based on the position information of the control cabinet door, the target image of the control cabinet door is extracted and input into the target classification model to obtain the state information of the control cabinet door, wherein the target classification model is trained by using the door state video detection model training method; Repeat the above steps until all the on-site monitoring images to be tested are detected.

[0011] The following are further optimizations and / or improvements to the above technical solutions: The above also includes executing early warning operations according to the status information of the control cabinet door, including: According to the status information of the control cabinet door, determine whether the control cabinet door of the current frame is in an uncorresponding and unclosed state; In response to "yes", it is determined whether the state information of the cabinet doors of the control cabinet corresponding to the M frames after the current frame are all in an open state. If so, a warning operation is performed and a warning message is issued. If not, no warning operation is performed. If the response is no, the early warning operation is not performed, and the status information of the control cabinet door corresponding to the next frame is determined.

[0012] The third technical solution of the present invention is achieved by the following measures: a cabinet door state video detection model training device, comprising: A training data acquisition unit acquires a target detection main sample set and a target detection sub-sample set, and divides the target detection main sample set into a training sample subset and a verification sample subset, wherein the cabinet door monitoring picture in the target detection sub-sample set is the same as the training sample subset, the training sample subset includes a plurality of training samples, each training sample includes a cabinet door monitoring picture marked with a cabinet door part and a label identifying the cabinet door state, the verification sample subset includes a plurality of verification samples, each verification sample includes a cabinet door monitoring picture and a label identifying the cabinet door state, and the target detection sub-sample set includes a plurality of detection sub-samples, each detection sub-sample includes a cabinet door monitoring picture and a label identifying the cabinet door state; In the first training unit, the improved YOLOV5 network model is trained using the training sample subset and the verification sample subset to obtain a target detection model, wherein the improved YOLOV5 network model is based on the YOLOV5 network model, the backbone network, feature pyramid network, and network head in the model are improved, and the Autoaugment automatic data enhancement set is introduced to obtain the improved YOLOV5 network model; The reprocessing unit averages the weights of the target detection model using a model averaging algorithm; In the second training unit, the MobileNetV2 classifier is trained using the target detection sub-sample set to obtain a target classification model; The model composition unit establishes a cabinet door status video detection model based on the trained target detection model and target classification model.

[0013] The fourth technical solution of the present invention is achieved by the following measures: a cabinet door status video detection device, comprising: The test data acquisition unit acquires the real-time monitoring video of the work site and acquires N consecutive frames of monitoring images of the work site to be tested every N frames; A first detection unit inputs the monitoring image of the work site to be tested into a target detection model, and obtains the position information of the cabinet door of the control cabinet through non-maximum suppression, wherein the target detection model is trained by using the cabinet door status video detection model training method described in any one of claims 1 to 4; A second detection unit extracts a target image of the control cabinet door based on the position information of the control cabinet door, and inputs the target image into a target classification model to obtain status information of the control cabinet door, wherein the target classification model is obtained by training using the door status video detection model training method according to any one of claims 1 to 4; The early warning unit performs early warning operations according to the status information of the control cabinet door.

[0014] The cabinet door status video detection model of the present invention is based on the YOLOV5 algorithm, and is improved in the backbone network, feature pyramid network and the first three parts of the network, and the Autoaugment data enhancement strategy is introduced. The target detection model is trained using historical cabinet door status monitoring video frame cutting training samples. The rich training samples and the improved YOLOV5 algorithm effectively improve the generalization and detection capabilities of the model, and after the training, SWA is used to obtain a better model weight, further improving the generalization ability of the model. And by training the MobileNetV2 network to obtain a target classification model, the detection ability of the cabinet door status video detection model obtained by the present invention is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Attached Figure 1 This is a flow chart of the model training method of the present invention.

[0016] Attached Figure 2 The present invention is a flow chart of a method for detecting cabinet door status via video.

[0017] Attached Figure 3 This is a flow chart of another cabinet door status video detection method of the present invention.

[0018] Attached Figure 4 The figure is a schematic diagram of the structure of the model training device of the present invention.

[0019] Attached Figure 5 The figure is a schematic diagram of the structure of the cabinet door status video detection device of the present invention. DETAILED DESCRIPTION

[0020] The present invention is not limited by the following embodiments, and specific implementation methods can be determined based on the technical solution of the present invention and actual conditions.

[0021] The present invention will be further described below in conjunction with embodiments and drawings: Embodiment 1: As attached Figure 1 As shown, an embodiment of the present invention discloses a cabinet door state video detection model training method, comprising: Step S110, obtain the target detection main sample set and the target detection sub-sample set, and divide the target detection main sample set into a training sample subset and a verification sample subset, wherein the cabinet door monitoring picture in the target detection sub-sample set is the same as the training sample subset, the training sample subset includes a number of training samples, each training sample includes a cabinet door monitoring picture marked with a cabinet door part and a label identifying the cabinet door status, the verification sample subset includes a number of verification samples, each verification sample includes a cabinet door monitoring picture and a label identifying the cabinet door status, and the target detection sub-sample set includes a number of detection sub-samples, each detection sub-sample includes a cabinet door monitoring picture and a label identifying the cabinet door status.

[0022] The above step S110 includes: (1) Obtain historical cabinet door status monitoring videos, intercept the historical cabinet door status monitoring videos by frame, obtain several cabinet door monitoring pictures, and form a main sample set for target detection; here, the historical cabinet door status monitoring videos can be obtained by selecting cabinet door status monitoring videos at multiple angles, multiple distances, and multiple weather conditions, so as to increase the richness of samples and improve the accuracy of model detection; (2) Dividing the target detection main sample set into a training sample subset and a validation sample subset according to a set ratio; the set ratio here is set according to the actual situation, and can be but not limited to 8:2; (3) Copy the training sample subset to construct the target detection secondary sample set; (4) Use cropping and scaling rules to process the cabinet door monitoring images in the training sample subset and the target detection sub-sample set.

[0023] The above cropping and scaling rules include: The door parts of each door monitoring image in the training sample subset are labeled using the true value annotation frame to complete the processing of the training sample subset; The door part of each door monitoring image in the target detection sub-sample set is annotated using the true value annotation frame, the annotated part of each door monitoring image is enlarged to twice its original size and then cropped, and the cropped door monitoring image is reduced to a size of 384 points.

[0024] Here, each sub-sample in the target detection sub-sample set is cropped by zooming in and out, which can improve the training accuracy of the MobileNetV2 classifier and thus improve the accuracy of cabinet door status detection.

[0025] Step S120, using the training sample subset and the verification sample subset to train the improved YOLOV5 network model to obtain a target detection model, wherein the improved YOLOV5 network model is based on the YOLOV5 network model, the backbone network, feature pyramid network, and network head in the model are improved, and the Autoaugment automatic data enhancement set is introduced to obtain it.

[0026] The improved YOLOV5 network model in this embodiment is based on the YOLOV5 network model, improves the backbone network, feature pyramid network, and network head in the model, and introduces the Autoaugment automatic data enhancement set, as follows: (1) Autoaugment automatic data enhancement set Autoaugment describes the process of finding the best enhancement strategy as a search problem. The data enhancement strategy problem includes three parts: (1) search space (2) search algorithm (3) evaluation index. The search space may vary from task to task. The Autoaugment data enhancement strategy contains five sub-policies, each of which includes two operations. Each operation corresponds to the strength and execution probability of a data enhancement method. For the search algorithm, reinforcement learning and evolutionary algorithms are usually used to explore the search space in iterations. The evaluation index is the model trained and tested on the proxy task (training set subset), which is used as feedback for the search algorithm. The designed scale-aware search space includes image-level and bounding box-level enhancements. The image-level enhancement function includes the function of zooming in and out of the entire image. For the bounding box-level enhancement, the color and geometric operations of the target object are searched in the image. AutoaugmentV2 can be used in this patent.

[0027] The model in this embodiment uses the Autoaugment automatic data enhancement set to greatly enhance the generalization ability of the model and improve the model's detection effect on cabinet doors.

[0028] (2) The backbone network is improved to ConvNeXt-Tiny backbone network In this embodiment, the ConvNeXt-Tiny backbone network is used to replace the CSP-DarkNet backbone network in the YOLOV5 algorithm, which can greatly improve the detection capability of the model while maintaining a high reasoning speed.

[0029] ConvNeXt is a pure convolutional neural network, which is benchmarked against Swin Transformer. By using a training method similar to Swin Transformer, ConvNeXt has faster inference speed and higher accuracy than Swin Transformer under the same FLOPs. Specifically, ConvNeXt adjusts the ratio of blocks between different stages from (3, 4, 6, 3) to (3, 3, 9, 3), and changes the initial downsampling module, which includes a convolution kernel size of The convolutional layer with a stride of 2 and a max pooling downsampling with a stride of 2 are replaced by a convolutional kernel size of Convolutional layer with a stride of 4. In addition, the self-attention mechanism is replaced by depthwise separable convolution, and the MLP layer is replaced by flipped convolution blocks. The so-called flipped convolution block is actually a network structure with thin sides and thick middle. The kernel size of the depthwise separable convolution is increased, and the position of the depthwise separable convolution in the flipped convolution block is moved up to cope with the self-attention mechanism in front of the MLP layer. Finally, the GELU activation function is used instead of the RELU activation function, and layer normalization is used instead of batch normalization, and fewer activation functions and batch normalization are used.

[0030] (3) The feature pyramid network is improved to the P6-layer feature pyramid network The original feature pyramid network of the YOLOV5 algorithm includes a top-down feature pyramid and a top-down path aggregation network, and it has three feature maps, P3, P4, and P5. In order to improve the detection performance of the control cabinet door, this embodiment introduces the P6 feature map based on the original three feature maps with downsampling multiples of 8, 16, and 32. The downsampling multiple of the P6 feature map is 64, which is obtained by downsampling the P5 feature map. The downsampling method is exactly the same as the previous downsampling between P3, P4, and P5.

[0031] (4) The network head is improved to a decoupled network head Referring to the detection head of YOLOX, the detection head of YOLOV5 is also decoupled to separate the confidence branch and the category branch. The purpose is to improve the accuracy of the confidence branch, thereby promoting the subsequent NMS post-processing algorithm.

[0032] Step S130, averaging the weights of the target detection model using a model averaging algorithm.

[0033] In this step, SWA random weight averaging is used. SWA is the average of multiple checkpoints along the SGD optimization trajectory. It has a higher constant learning rate or periodic learning rate and can find a weight Wswa that is closer to the optimal solution and has better generalization. The specific formula of SWA random weight averaging is as follows: Step S140, using the target detection sub-sample set to train the MobileNetV2 classifier to obtain a target classification model.

[0034] The biggest highlight of the MobileNetV2 network is that it uses the inverted residual structure. In the residual structure proposed by ResNet, first use Convolution achieves dimensionality reduction, and then passes Convolution, finally through Convolution achieves dimensionality increase, that is, the two ends are large and the middle is small. In MobileNetV2, the order of dimensionality reduction and dimensionality increase is swapped, and Convolution is replaced by Depthwise separable convolution is a convolution with small ends and large middle.

[0035] The target classification model of this embodiment uses the MobileNetV2 classifier, and the model is fully trained using the target detection sub-sample set to improve the accuracy of the classification of the present invention and ultimately better identify the state of the cabinet door.

[0036] Step S150, establishing a cabinet door status video detection model based on the trained target detection model and target classification model.

[0037] The present invention discloses a door status video detection model training method. The door status video detection model is based on the YOLOV5 algorithm, and is improved in the backbone network, feature pyramid network and the first three parts of the network. The Autoaugment data enhancement strategy is introduced, and the target detection model is obtained by training with historical door status monitoring video frame cutting training samples. The rich training samples and the improved YOLOV5 algorithm effectively improve the generalization ability and detection ability of the model, and after the training, SWA is used to obtain a better model weight, further improving the generalization ability of the model. In addition, the target classification model is obtained by training the MobileNetV2 network, which improves the detection ability of the door status video detection model obtained by the present invention.

[0038] Embodiment 2: As attached Figure 2 As shown, an embodiment of the present invention discloses a method for detecting a cabinet door status by video, comprising: Step S210, obtaining a real-time monitoring video of the work site, and obtaining N consecutive frames of monitoring images of the work site to be tested every N frames; Step S220, inputting the monitoring image of the work site to be tested into the target detection model, and obtaining the position information of the cabinet door of the control cabinet through non-maximum suppression, wherein the target detection model is obtained by training using the cabinet door status video detection model training method disclosed in the above embodiment; Step S230, based on the position information of the control cabinet door, extracting the target image of the control cabinet door, and inputting it into the target classification model to obtain the state information of the control cabinet door, wherein the target classification model is trained by using the cabinet door state video detection model training method according to any one of claims 1 to 4; Step S240, repeating the above steps 220 to 230 until all the work site monitoring images to be tested are detected.

[0039] Since a large number of candidate boxes are generated at the position of the same target during the target detection process, and these candidate boxes may overlap with each other, in step S220 of this embodiment, the optimal target bounding box is found through non-maximum suppression and maximum suppression, and redundant bounding boxes are eliminated to ensure the accuracy and efficiency of detection.

[0040] This embodiment performs cabinet door status detection through the target detection model and target classification model obtained through training in the above embodiment. Based on the advantages of the above model training, it can effectively improve the accuracy of cabinet door status detection, facilitate timely and accurate on-site production disposal, and play a reliable role in ensuring the normal production of oil field on-site development.

[0041] Embodiment 3: As attached Figure 3 As shown, an embodiment of the present invention discloses a method for detecting a cabinet door status by video, comprising: Step S310, obtaining a real-time monitoring video of the work site, and obtaining N consecutive frames of monitoring images of the work site to be tested every N frames; Step S320, inputting the monitoring image of the work site to be tested into the target detection model, and obtaining the position information of the cabinet door of the control cabinet through non-maximum suppression, wherein the target detection model is trained by using the cabinet door status video detection model training method disclosed in the above embodiment; Step S330, based on the position information of the control cabinet door, extracting the target image of the control cabinet door, and inputting it into the target classification model to obtain the state information of the control cabinet door, wherein the target classification model is obtained by training using the cabinet door state video detection model training method according to any one of claims 1 to 4; Step S340, repeating the above steps 220 to 230 until all the monitoring images of the work site to be tested are detected; Step S350, performing an early warning operation according to the status information of the control cabinet door, including: (1) According to the status information of the control cabinet door, determine whether the control cabinet door of the current frame is in an uncorresponding and unclosed state; (2) In response to "yes", it is determined whether the state information of the control cabinet doors corresponding to the M frames after the current frame are all in the open state. If so, a warning operation is performed and a warning message is issued. If not, no warning operation is performed. (3) If the response is no, the warning operation is not performed, and the status information of the control cabinet door corresponding to the next frame is determined.

[0042] Embodiment 4: As attached Figure 4 As shown, the embodiment of the present invention discloses a cabinet door state video detection model training device, comprising: A training data acquisition unit acquires a target detection main sample set and a target detection sub-sample set, and divides the target detection main sample set into a training sample subset and a verification sample subset, wherein the cabinet door monitoring picture in the target detection sub-sample set is the same as the training sample subset, the training sample subset includes a plurality of training samples, each training sample includes a cabinet door monitoring picture marked with a cabinet door part and a label identifying the cabinet door state, the verification sample subset includes a plurality of verification samples, each verification sample includes a cabinet door monitoring picture and a label identifying the cabinet door state, and the target detection sub-sample set includes a plurality of detection sub-samples, each detection sub-sample includes a cabinet door monitoring picture and a label identifying the cabinet door state; In the first training unit, the improved YOLOV5 network model is trained using the training sample subset and the verification sample subset to obtain a target detection model, wherein the improved YOLOV5 network model is based on the YOLOV5 network model, the backbone network, feature pyramid network, and network head in the model are improved, and the Autoaugment automatic data enhancement set is introduced to obtain the improved YOLOV5 network model; The reprocessing unit averages the weights of the target detection model using a model averaging algorithm; In the second training unit, the MobileNetV2 classifier is trained using the target detection sub-sample set to obtain a target classification model; The model composition unit establishes a cabinet door status video detection model based on the trained target detection model and target classification model.

[0043] Embodiment 5: As attached Figure 5 As shown, the embodiment of the present invention discloses a cabinet door status video detection device, comprising: The test data acquisition unit acquires the real-time monitoring video of the work site and acquires N consecutive frames of monitoring images of the work site to be tested every N frames; A first detection unit inputs the monitoring image of the work site to be tested into a target detection model, and obtains the position information of the cabinet door of the control cabinet through non-maximum suppression, wherein the target detection model is trained by using the cabinet door status video detection model training method described in any one of claims 1 to 4; A second detection unit extracts a target image of the control cabinet door based on the position information of the control cabinet door, and inputs the target image into a target classification model to obtain status information of the control cabinet door, wherein the target classification model is obtained by training using the door status video detection model training method according to any one of claims 1 to 4; The early warning unit performs early warning operations according to the status information of the control cabinet door.

[0044] Embodiment 6: The embodiment of the present invention discloses a storage medium, on which is stored a computer program that can be read by a computer, and the computer program is configured to execute a cabinet door status video detection method when running.

[0045] The above storage medium may include, but is not limited to: a USB flash drive, a read-only memory, a mobile hard disk, a magnetic disk or an optical disk, and other media that can store computer programs.

[0046] Embodiment 7: The embodiment of the present invention discloses an electronic device, including a processor and a memory, wherein the memory stores a computer program, and the computer program is loaded and executed by the processor to implement a cabinet door status video detection method.

[0047] The processor may be a central processing unit (CPU), a general purpose processor, a digital signal processor (DSP), an ASIC, an FPGA or other programmable logic device, a transistor logic device, a hardware component or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. It may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like. The memory may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory, a mobile hard disk, a magnetic disk or an optical disk.

[0048] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The schemes in the embodiments of the present application may be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0049] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0050] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0051] The above technical features constitute the best embodiment of the present invention, which has strong adaptability and best implementation effect. Non-essential technical features can be added or reduced according to actual needs to meet the requirements of different situations.

Claims

1. A door status video detection model training method, characterized in that: include: Obtain a target detection main sample set and a target detection sub-sample set, and divide the target detection main sample set into a training sample subset and a verification sample subset, wherein the cabinet door monitoring picture in the target detection sub-sample set is the same as the training sample subset, the training sample subset includes a number of training samples, each training sample includes a cabinet door monitoring picture marked with a cabinet door part and a label identifying the cabinet door state, the verification sample subset includes a number of verification samples, each verification sample includes a cabinet door monitoring picture and a label identifying the cabinet door state, and the target detection sub-sample set includes a number of detection sub-samples, each detection sub-sample includes a cabinet door monitoring picture and a label identifying the cabinet door state; The improved YOLOV5 network model is trained using the training sample subset and the validation sample subset to obtain a target detection model, wherein the improved YOLOV5 network model is based on the YOLOV5 network model, the backbone network, feature pyramid network, and network head in the model are improved, and the Autoaugment automatic data enhancement set is introduced to obtain the improved YOLOV5 network model; Use the model averaging algorithm to average the weights of the target detection model; Use the target detection sub-sample set to train the MobileNetV2 classifier to obtain the target classification model; Based on the trained target detection model and target classification model, a cabinet door status video detection model is established.

2. The door status video detection model training method according to claim 1 is characterized in that: The improved YOLOV5 network model includes a ConvNeXt-Tiny backbone network, a P6-layer feature pyramid network, a decoupled network head, and an Autoaugment self-automatic data enhancement set.

3. The door status video detection model training method according to claim 1 or 2, characterized in that: The step of obtaining a target detection main sample set and a target detection secondary sample set, and dividing the target detection main sample set into a training sample subset and a verification sample subset, includes: Obtain historical cabinet door status monitoring videos, intercept the historical cabinet door status monitoring videos by frame, obtain several cabinet door monitoring pictures, and form a main sample set for target detection; Divide the target detection main sample set into a training sample subset and a verification sample subset according to a set ratio; Duplicate the training sample subset to construct a secondary sample set for target detection; The door monitoring images in the training sample subset and the target detection sub-sample set are processed using cropping and scaling rules.

4. The door status video detection model training method according to claim 3 is characterized in that: The cropping and scaling rules include: The door parts of each door monitoring image in the training sample subset are labeled using the true value annotation frame to complete the processing of the training sample subset; The door part of each door monitoring image in the target detection sub-sample set is annotated using the true value annotation frame, the annotated part of each door monitoring image is enlarged to twice its original size and then cropped, and the cropped door monitoring image is reduced to a size of 384 points.

5. A method for detecting cabinet door status via video, characterized in that: include: Acquire real-time monitoring video of the work site, and acquire N consecutive frames of monitoring images of the work site to be tested every N frames; Input the monitoring image of the work site to be tested into the target detection model, and obtain the position information of the control cabinet door through non-maximum suppression, wherein the target detection model is trained using the cabinet door status video detection model training method described in any one of claims 1 to 4; Based on the position information of the control cabinet door, a target image of the control cabinet door is extracted, and the target image is input into a target classification model to obtain the state information of the control cabinet door, wherein the target classification model is obtained by training using the door state video detection model training method according to any one of claims 1 to 4; Repeat the above steps until all the on-site monitoring images to be tested are detected.

6. The method for detecting cabinet door status by video according to claim 5, characterized in that: It also includes executing early warning operations according to the status information of the control cabinet door, including: According to the status information of the control cabinet door, determine whether the control cabinet door of the current frame is in an uncorresponding and unclosed state; In response to "yes", it is determined whether the state information of the cabinet doors of the control cabinet corresponding to the M frames after the current frame are all in an open state. If so, a warning operation is performed and a warning message is issued. If not, no warning operation is performed. If the response is no, the early warning operation is not performed, and the status information of the control cabinet door corresponding to the next frame is determined.

7. A cabinet door status video detection model training device, the cabinet door status video detection model training device using the cabinet door status video detection model training method according to any one of claims 1 to 4, characterized in that: include: A training data acquisition unit acquires a target detection main sample set and a target detection sub-sample set, and divides the target detection main sample set into a training sample subset and a verification sample subset, wherein the cabinet door monitoring picture in the target detection sub-sample set is the same as the training sample subset, the training sample subset includes a plurality of training samples, each training sample includes a cabinet door monitoring picture marked with a cabinet door part and a label identifying the cabinet door state, the verification sample subset includes a plurality of verification samples, each verification sample includes a cabinet door monitoring picture and a label identifying the cabinet door state, and the target detection sub-sample set includes a plurality of detection sub-samples, each detection sub-sample includes a cabinet door monitoring picture and a label identifying the cabinet door state; In the first training unit, the improved YOLOV5 network model is trained using the training sample subset and the verification sample subset to obtain a target detection model, wherein the improved YOLOV5 network model is based on the YOLOV5 network model, the backbone network, feature pyramid network, and network head in the model are improved, and the Autoaugment automatic data enhancement set is introduced to obtain the improved YOLOV5 network model; The reprocessing unit averages the weights of the target detection model using a model averaging algorithm; In the second training unit, the MobileNetV2 classifier is trained using the target detection sub-sample set to obtain a target classification model; The model composition unit establishes a cabinet door status video detection model based on the trained target detection model and target classification model.

8. A cabinet door status video detection device, wherein the remote sensing image semantic recognition device based on contrastive learning uses the cabinet door status video detection method according to any one of claims 5 to 6, characterized in that: include: The test data acquisition unit acquires the real-time monitoring video of the work site and acquires N consecutive frames of monitoring images of the work site to be tested every N frames; A first detection unit inputs the monitoring image of the work site to be tested into a target detection model, and obtains the position information of the cabinet door of the control cabinet through non-maximum suppression, wherein the target detection model is trained by using the cabinet door status video detection model training method described in any one of claims 1 to 4; A second detection unit extracts a target image of the control cabinet door based on the position information of the control cabinet door, and inputs the target image into a target classification model to obtain status information of the control cabinet door, wherein the target classification model is obtained by training using the door status video detection model training method according to any one of claims 1 to 4; The early warning unit performs early warning operations according to the status information of the control cabinet door.

9. A storage medium, characterized in that: The storage medium stores a computer program that can be read by a computer, and the computer program is configured to execute the cabinet door status video detection method according to any one of claims 5 to 6 when running.

10. An electronic device, characterized in that: It comprises a processor and a memory, wherein the memory stores a computer program, and the computer program is loaded and executed by the processor to implement the cabinet door status video detection method as described in any one of claims 5 to 6.

Images