Dynamic architecture target detection method and system based on live working scene

Abstract

The invention discloses a dynamic architecture target detection method and system based on a live working scene, and the method comprises the steps: obtaining a field image of a field detection region, and inputting the field image into a target detection network; extracting a first feature of the field image through a first feature extraction network of the target detection network; judging the type of the field image through a dynamic selector of the target detection network; when it is judged that the field image is the simple type image, inputting a first feature of the field image into a detection head of a target detection network, and obtaining a target detection result of the field image through the detection head; or when it is judged that the field image is the complex type image, a second feature of the field image is extracted through a second feature extraction network of the target detection network; fusing the first feature and the second feature through a fusion module to obtain a final feature; and inputting the final feature into a detection head, and obtaining a target detection result of the field image through the detection head.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision target detection, and more specifically, to a dynamic architecture target detection method and system based on live working scenarios. Background Art

[0002] In modern power systems, live working is a common working method, where workers need to perform maintenance, installation or inspection of power equipment while it is live. This working method is widely used in the maintenance and operation of power systems, including transmission lines, substations and other places. In live working scenarios, workers need to wear insulating clothing and use insulating tools to protect themselves from electric shock. At the same time, workers need to strictly abide by safety operating procedures to ensure that no safety accidents occur during the operation.

[0003] With the advancement of science and technology, live working in modern power systems increasingly relies on intelligent technology. Intelligent monitoring systems can be used to monitor the insulation shielding of workers during live working and the wearing of insulating equipment by personnel, so as to timely discover potential safety hazards.

[0004] In recent years, deep learning technology has made great progress in the field of image processing, especially in the field of target detection. By using deep learning models, features in images can be better extracted, and with strong generalization capabilities, accurate target detection can be achieved in different scenarios. Therefore, applying deep learning technology to live working tasks is expected to improve the accuracy and robustness of insulation shielding detection of equipment and whether personnel are wearing insulation equipment in live working scenarios.

[0005] In this context, combining target detection methods and intelligent monitoring systems for intelligent monitoring can not only improve safety, but also help workers complete their work faster. The target detection method can automatically identify whether the insulation shielding in the image or video is complete and whether the personnel insulation equipment is worn, effectively discover potential safety hazards and protect the lives of workers; in addition, during the live work training process, the training operations can be monitored to help workers understand the operating procedures more quickly and proficiently, thereby improving work efficiency. Therefore, when optimizing the intelligent monitoring system, in addition to considering safety, it is also necessary to consider how to cleverly combine factors that improve work efficiency to achieve more comprehensive intelligent monitoring.

[0006] However, existing target detection methods may not perform well when dealing with complex situations such as occlusion, complex background, and illumination changes, which may lead to missed detection or false detection. In addition, the complexity of the background at the live working site is also a challenge. Sometimes the background is complex, and sometimes it is simple, which may waste resources to detect some relatively simple background images during re-detection.

[0007] Therefore, when optimizing the intelligent monitoring system, it is necessary to consider how to better adapt to different background conditions to avoid resource waste and improve overall efficiency. Summary of the invention

[0008] The technical solution of the present invention provides a method and system for dynamic architecture target detection based on live working scenarios to solve the problem of how to perform target detection on the dynamic architecture of live working scenarios while considering different background conditions.

[0009] In order to solve the above problems, the present invention provides a dynamic structural target detection method based on a live working scenario, the method comprising:

[0010] Acquire a scene image of the scene detection area, and input the scene image into a target detection network;

[0011] Extracting a first feature of the scene image through a first feature extraction network of the target detection network;

[0012] Determining the type of the scene image by a dynamic selector of the target detection network;

[0013] When it is determined that the scene image is a simple type image, the first feature of the scene image is input into the detection head of the target detection network, and the target detection result of the scene image is obtained through the detection head; or

[0014] When it is determined that the scene image is a complex type image, the second feature of the scene image is extracted through the second feature extraction network of the target detection network; the first feature and the second feature are fused through a fusion module to obtain a final feature; the final feature is input into a detection head, and the target detection result of the scene image is obtained through the detection head.

[0015] Preferably, the method further comprises:

[0016] A live video of the live detection area is obtained, and the live video is input into the target detection network.

[0017] Preferably, the determining the type of the scene image by the dynamic selector of the target detection network comprises:

[0018] The scene image is subjected to information compression processing, including: a global pooling operation and a channel dimension splicing operation, wherein the global pooling operation is:

[0019]

[0020] in represents global average pooling, represents channel cascade, w1, w2, ...w L represents the learning parameters, b1, b2, ...b L is the bias term, Represent multi-scale features;

[0021] Through the global pooling operation, the scene images of different scales are converted into feature vectors of fixed dimensions; by using linear transformation, the weights and biases of the fixed-dimensional feature vectors are adjusted, and finally through channel cascading Connect the feature vectors of each scale to form a larger feature vector, and compress the multi-scale features into a vector of dimension d.

[0022] The vector of dimension d After the linear layer, feature dimension reduction is performed.

[0023]

[0024] The prediction score is obtained through the activation function:

[0025]

[0026] The type of the scene image is determined by the prediction score.

[0027] Preferably, fusing the first feature and the second feature through a fusion module to obtain a final feature includes:

[0028] The fusion module receives the first feature And the second feature And the first feature And the second feature Represent the features of the current step t and the next step t+1 respectively;

[0029] For the second feature The fusion module calculates the second attention heat map through the convolution layer and the Sigmoid operation, and measures the importance of each position in the fusion through the second attention heat map;

[0030] For the first feature The fusion module calculates the first attention heat map through convolution layer and Sigmoid operation;

[0031] Based on the first attention heat map and the second attention heat map, the first feature is calculated by the fusion module And the second feature The weighted sum of the features is taken as the final feature.

[0032] Preferably, the method further includes optimizing the dynamic selector, determining a prediction score of the scene image according to the number of targets in the scene image; and judging the type of the scene image based on the prediction score.

[0033] According to another aspect of the present invention, the present invention provides a dynamic structure target detection system based on a live working scenario, the system comprising:

[0034] An acquisition unit, used for acquiring a scene image of a scene detection area and inputting the scene image into a target detection network;

[0035] A first extraction unit, configured to extract a first feature of the scene image through a first feature extraction network of the target detection network;

[0036] A determination unit, configured to determine the type of the scene image through a dynamic selector of the target detection network;

[0037] A first result unit is used to input the first feature of the scene image into the detection head of the target detection network when it is determined that the scene image is a simple type image, and obtain the target detection result of the scene image through the detection head; or

[0038] The second extraction unit is used to extract the second feature of the scene image through the second feature extraction network of the target detection network when it is determined that the scene image is a complex type image; the fusion unit is used to fuse the first feature and the second feature through a fusion module to obtain the final feature; the second result unit is used to input the final feature into the detection head to obtain the target detection result of the scene image through the detection head.

[0039] Preferably, the acquisition unit is further used for:

[0040] A live video of the live detection area is obtained, and the live video is input into the target detection network.

[0041] Preferably, the judging unit is used to judge the type of the scene image through the dynamic selector of the target detection network, and is also used to:

[0042] The scene image is subjected to information compression processing, including: a global pooling operation and a channel dimension splicing operation, wherein the global pooling operation is:

[0043]

[0044] in represents global average pooling, represents channel cascade, w1, w2, ...w Lrepresents the learning parameters, b1, b2, ...b L is the bias term, Represent multi-scale features;

[0045] Through the global pooling operation, the scene images of different scales are converted into feature vectors of fixed dimensions; by using linear transformation, the weights and biases of the fixed-dimensional feature vectors are adjusted, and finally through channel cascading Connect the feature vectors of each scale to form a larger feature vector, and compress the multi-scale features into a vector of dimension d.

[0046] The vector of dimension d After the linear layer, feature dimension reduction is performed.

[0047]

[0048] The prediction score is obtained through the activation function:

[0049]

[0050] The type of the scene image is determined by the prediction score.

[0051] Preferably, the fusion unit is used to fuse the first feature and the second feature through a fusion module to obtain a final feature, and is also used to:

[0052] The fusion module receives the first feature And the second feature And the first feature And the second feature Represent the features of the current step t and the next step t+1 respectively;

[0053] For the second feature The fusion module calculates the second attention heat map through the convolution layer and the Sigmoid operation, and measures the importance of each position in the fusion through the second attention heat map;

[0054] For the first feature The fusion module calculates the first attention heat map through convolution layer and Sigmoid operation;

[0055] Based on the first attention heat map and the second attention heat map, the first feature is calculated by the fusion module And the second feature The weighted sum of the features is taken as the final feature.

[0056] Preferably, the judgment unit is further used to optimize the dynamic selector, determine the prediction score of the scene image according to the number of objects in the scene image, and judge the type of the scene image based on the prediction score.

[0057] The technical solution of the present invention provides a method and system for dynamic structural target detection based on live working scenes, wherein the method includes: obtaining a field image of the field detection area, and inputting the field image into a target detection network; extracting the first feature of the field image through the first feature extraction network of the target detection network; judging the type of the field image through the dynamic selector of the target detection network; when the field image is judged to be a simple type image, inputting the first feature of the field image into the detection head of the target detection network, and obtaining the target detection result of the field image through the detection head; or when the field image is judged to be a complex type image, extracting the second feature of the field image through the second feature extraction network of the target detection network; through a fusion module, fusing the first feature and the second feature to obtain the final feature; inputting the final feature into the detection head, and obtaining the target detection result of the field image through the detection head. The technical solution of the present invention provides a method and system for dynamic structural target detection based on live working scenes, which considers how to better adapt to different background conditions, thereby avoiding resource waste and improving overall detection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] A more complete understanding of exemplary embodiments of the present invention may be obtained by referring to the following drawings:

[0059] Figure 1 A flow chart of a dynamic architecture target detection method based on a live working scenario according to a preferred embodiment of the present invention;

[0060] Figure 2 It is a flow chart of a dynamic architecture target detection method based on a live working scenario according to a preferred embodiment of the present invention;

[0061] Figure 3 A convolutional neural network structure diagram generally selected as a backbone network in a feature extraction network according to a preferred embodiment of the present invention;

[0062] Figure 4 A schematic diagram of a fusion module receiving a first multi-scale feature and a second group of multi-scale features according to a preferred embodiment of the present invention;

[0063] Figure 5 A flow chart of a dynamic structural target detection method based on a live working scenario according to a preferred embodiment of the present invention; and

[0064] Figure 6The present invention is a structural diagram of a dynamic architecture target detection system based on a live working scenario according to a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0065] Now, exemplary embodiments of the present invention are described with reference to the accompanying drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. These embodiments are provided to disclose the present invention in detail and completely and to fully convey the scope of the present invention to those skilled in the art. The terms used in the exemplary embodiments shown in the accompanying drawings are not intended to limit the present invention. In the accompanying drawings, the same units / elements are marked with the same reference numerals.

[0066] Unless otherwise specified, the terms (including technical terms) used herein have the commonly understood meanings to those skilled in the art. In addition, it is understood that the terms defined in commonly used dictionaries should be understood to have the same meanings as those in the context of the relevant fields, and should not be understood as idealized or overly formal meanings.

[0067] Figure 1 The present invention is a flowchart of a dynamic architecture target detection method based on a live working scenario according to a preferred embodiment of the present invention.

[0068] The present invention uses a target detection method and an intelligent monitoring system to perform intelligent monitoring, which can not only improve safety, but also help workers complete their work faster. The target detection method can automatically identify whether the insulation shielding in the image or video is complete and whether the personnel insulation equipment is equipped, effectively discover potential safety hazards, and protect the lives of workers; in addition, during the live working training process, the training operation can be monitored to help workers understand the operating procedures more quickly and skillfully, thereby improving work efficiency. Therefore, when optimizing the intelligent monitoring system, in addition to considering safety, it is also necessary to consider how to cleverly combine factors that improve work efficiency to achieve more comprehensive intelligent monitoring. However, the target detection method may perform poorly when dealing with complex situations such as occlusion, complex backgrounds, and changes in illumination, which may lead to missed detection or false detection. In addition, the complexity of the background at the live working site is also a challenge. Sometimes the background is complex, and sometimes it is simple, which may waste resources to detect some relatively simple background images when re-detecting. Therefore, when optimizing the intelligent monitoring system, it is necessary to consider how to better adapt to different background conditions to avoid waste of resources and improve overall efficiency.

[0069] The goal of the present invention is to reduce the computational cost of the model while ensuring the accuracy and robustness of the model in complex backgrounds.

[0070] To achieve the above object, according to one aspect of the present invention, a dynamic structural target detection method based on live working scenarios is provided.

[0071] like Figure 1 As shown, the present invention provides a dynamic structure target detection method based on a live working scenario, the method comprising:

[0072] Step 101: Acquire a scene image of a scene detection area, and input the scene image into a target detection network;

[0073] Preferably, the method further comprises:

[0074] Obtain live video of the live detection area and input the live video into the target detection network.

[0075] The present invention uses a mounted camera to take images or videos of live-line work on site as input and send them to the network.

[0076] Step 102: extracting a first feature of the scene image through a first feature extraction network of the target detection network;

[0077] The present invention constructs a target detection network, and the network structure includes: a feature extraction network, a dynamic selector, and a detection head.

[0078] Step 103: Determine the type of the scene image through the dynamic selector of the target detection network;

[0079] Preferably, determining the type of the scene image by a dynamic selector of the target detection network includes:

[0080] The information compression processing of the scene image includes: global pooling operation and channel dimension splicing operation, where the global pooling operation is:

[0081]

[0082] in represents global average pooling, represents channel cascade, w1, w2, ...w L represents the learning parameters, b1, b2, ...b L is the bias term, Represent multi-scale features;

[0083] Through the global pooling operation, the scene images of different scales are converted into feature vectors of fixed dimension; by using linear transformation, the weights and biases of the fixed dimension feature vectors are adjusted, and finally the feature vectors of each scale are connected through the channel cascade c to form a larger feature vector, compressing the multi-scale features into a vector of dimension d.

[0084] The vector of dimension d After the linear layer, feature dimension reduction is performed.

[0085]

[0086] The prediction score is obtained through the activation function:

[0087]

[0088] The type of the scene image is determined by the prediction score.

[0089] Preferably, the method further includes optimizing the dynamic selector, determining the prediction score of the scene image according to the number of objects in the scene image, and judging the type of the scene image based on the prediction score.

[0090] Step 104: when it is determined that the scene image is a simple type image, the first feature of the scene image is input into the detection head of the target detection network, and the target detection result of the scene image is obtained through the detection head; or

[0091] Step 105: When it is determined that the scene image is a complex type image, the second feature of the scene image is extracted by the second feature extraction network of the target detection network; the first feature and the second feature are fused through the fusion module to obtain the final feature; the final feature is input into the detection head, and the target detection result of the scene image is obtained through the detection head.

[0092] Preferably, the first feature and the second feature are fused through a fusion module to obtain a final feature, including:

[0093] The fusion module receives the first feature And the second feature And the first feature And the second feature Represent the features of the current step t and the next step t+1 respectively;

[0094] For the second feature The fusion module calculates the second attention heat map through the convolution layer and the Sigmoid operation, and uses the second attention heat map to measure the importance of each position in the fusion;

[0095] For the first feature The fusion module calculates the first attention heat map through convolution layer and Sigmoid operation;

[0096] Based on the first attention heat map and the second attention heat map, the first feature is calculated through the fusion module And the second feature The weighted sum of the features is taken as the final feature.

[0097] The feature extraction network of the present invention is composed of two groups of backbone networks and feature pyramid networks. For simple images, the network only needs to go through the first group of backbone networks and pyramid networks to extract features and then input into the detection head to obtain the detection results. For difficult images, after going through the first group of networks, the second group of backbone networks and pyramid networks are entered to extract features to continue extracting features, and then input into the detection head for detection.

[0098] The dynamic selector of the present invention extracts features through the first group of backbone networks and pyramid networks and sends them to the dynamic selector for judgment. If it is a simple image, its features are input into the detection head to obtain the detection results; if it is judged to be a difficult image, its features are sent to the second group of backbone networks and pyramid networks to extract features to continue extracting features, and then input into the detection head for detection.

[0099] The dynamic selector of the present invention provides an optimization strategy for the dynamic selector in order to assign simple images to the detector more quickly and assign difficult images to the second detector more accurately, and defines the difficulty criterion according to the corresponding training loss difference between the two detectors of the image.

[0100] The present invention uses a camera to take the image or video of the live working on site as input and send it to the network;

[0101] The present invention constructs a target detection network, and the network structure includes: a feature extraction network, a dynamic selector, and a detection head.

[0102] The feature extraction network of the present invention is composed of two groups of backbone networks and feature pyramid networks. For simple images, the network only needs to go through the first group of backbone networks and pyramid networks to extract features and then input into the detection head to obtain the detection results. For difficult images, after going through the first group of networks, the second group of backbone networks and pyramid networks are entered to extract features to continue extracting features, and then input into the detection head for detection.

[0103] The image of the present invention first passes through a feature extraction network, which is composed of two groups of backbone networks and a feature pyramid network. First, an image x is input, and multi-scale features x1, x2, and x3 are obtained through the first group of backbone networks and the feature pyramid network. The output multi-scale features x1, x2, and x3 are passed through a dynamic selector to determine whether the input image is a simple image or a difficult image. If it is a simple image, the multi-scale features x1, x2, and x3 are input into the detection head. Otherwise, the image is subjected to a second feature extraction, the original image is input into the second group of backbone networks, and x1, x2, and x3 are fused with the second group of backbone networks. The features extracted by the second group of backbone networks are set as x 21 , x 22 , x 23 , x11 With x 21 Concatenate and then perform 1×1 convolution to reduce the dimension. 22 , x 23 Similarly, the second set of backbone networks finally extracts the feature x 21 , x 22 , x 23 , enter the second set of backbone networks and feature pyramid networks to obtain multi-scale features, fuse the first set of outputs with the second set of outputs to obtain the final multi-scale features, and input them into the detection head.

[0104] If the image of the present invention is a difficult image, in order to reduce the computational complexity of the dynamic selector, it is necessary to compress the information to obtain the compressed features: the global pooling operation and the channel dimension splicing operation. After that, the features are mapped to the difficulty score through the linear mapping layer.

[0105]

[0106] in represents global average pooling, c represents channel cascade, w1, w2, ...w L represents the learnable parameters, b1, b2, ..., b L First, the feature maps of different scales are converted into feature vectors of fixed dimensions through global average pooling; then the weights and biases of the fixed-dimensional feature vectors are adjusted by using linear transformation, and finally the channel cascade is used. Connect the feature vectors of each scale to form a larger feature vector. This operation compresses the multi-scale feature f1 into a vector of dimension d.

[0107] The present invention further converts the vector with dimension d into After a linear layer, feature dimensionality reduction is performed.

[0108]

[0109] This operation will Mapping to a lower-dimensional space can retain the most important features while reducing the computational burden and number of parameters of the model.

[0110] Then use an activation function to get its prediction score:

[0111]

[0112] The activation function can convert the input real number into a probability distribution, so that the model can classify different categories and output the probability of each category. This can make the model better adapt to different tasks and data, and improve the accuracy and robustness of the model.

[0113] The dynamic selector obtained according to the above steps can assign simple images to the detector more quickly and assign difficult images to the second detector more accurately. An optimization strategy for the dynamic selector is given, and the difficulty criterion is defined according to the corresponding training loss difference between the two detectors of the image.

[0114] The fusion module of the present invention receives a first set of multi-scale features and the second set of multi-scale features These two sets of features represent the features of the current step t and the next step t+1 respectively. For the second set of features The fusion module calculates an attention heat map through convolutional layers and sigmoid operations to measure the importance of each position in the fusion. Similarly, for the first set of features The same operation is performed to obtain another attention heat map. Using these two attention heat maps, the fusion module calculates and Finally, the calculated weighted sum is used as the updated This updated feature will be used in subsequent steps.

[0115] The present invention provides a dynamic structure target detection method based on live working scenarios, comprising:

[0116] Feature extraction network module, the module structure includes: two sets of backbone networks and feature pyramid network combination, which are used to extract multi-scale features in images.

[0117] The dynamic detector module is used to judge the difficulty of the image. Simple images only need to go through the first set of backbone networks and pyramid networks to extract features and then be input into the detection head to obtain the detection results. For difficult images, after going through the first set of networks, they enter the second set of backbone networks and pyramid networks to extract features and continue to extract features, and then they are input into the detection head for detection.

[0118] The fusion module is used to fuse the first set of multi-scale features with the second set of multi-scale features and to update the first set of multi-scale features to better capture the features of the target.

[0119] The detection head module is used to detect the position of the target in the image.

[0120] The present invention provides a dynamic architecture target detection method based on live working scenarios, and adds a dynamic selector to an effective model to select complex or simple backgrounds to help the model process faster.

[0121] The dynamic architecture target detection method based on the live working scenario of the embodiment of the present invention is as follows: Figure 2As shown, the following steps are included:

[0122] S1: By using the installed camera, the image or video of the live work on site is taken as input and sent to the network;

[0123] In the embodiment of the present invention, the COCO dataset and the captured live working scenes are preferably used as the training set, verification set and test set, which includes 50,000 images as the training dataset, 25,000 images as the verification dataset, and 5,000 images as the test dataset.

[0124] S2: Build a target detection network, the network structure includes: feature extraction network, dynamic selector, and detection head.

[0125] S3: According to the feature extraction network obtained in step S2, the feature extraction network is composed of two sets of backbone networks and feature pyramid networks. For simple images, the network only needs to go through the first set of backbone networks and pyramid networks to extract features and then input them into the detection head to obtain the detection results. For difficult images, after going through the first set of networks, they enter the second set of backbone networks and pyramid networks to extract features and continue to extract features, and then input them into the detection head for detection.

[0126] In the embodiment of the present invention, Figure 3 As shown in the figure, the backbone network in the feature extraction network usually selects classic convolutional neural network structures such as ResNet50 or ResNet101. These backbone networks can extract high-level semantic information and feature representation of images. The feature pyramid network uses upsampling and downsampling operations to construct a multi-scale feature pyramid from the different levels of features extracted by the backbone network. Through the upsampling operation, the feature map with low resolution and high semantic information can be converted into the corresponding high-resolution feature map; and through the downsampling operation, the feature map with high resolution and low semantic information can be converted into the corresponding low-resolution feature map. In addition, the high-level features are added to the low-level features after upsampling, and a fine feature map is generated through iteration. The feature pyramid constructed in this way can effectively capture information of different scales in the image.

[0127] S4: According to the dynamic selector obtained in step S2, features are extracted through the first set of backbone networks and pyramid networks and then sent to the dynamic selector for judgment. If it is a simple image, its features are input into the detection head to obtain the detection results; if it is judged to be a difficult image, its features are sent to the second set of backbone networks and pyramid networks to extract features to continue extracting features, and then input into the detection head for detection.

[0128] In the embodiment of the present invention, Figure 3, the image first passes through the feature extraction network, which is composed of two sets of backbone networks and feature pyramid networks. First input image x, and get multi-scale features x1, x2, x3 through the first set of backbone networks and feature pyramid networks. The output multi-scale features x1, x2, x3 are passed through a dynamic selector to determine whether the input image is a simple image or a difficult image. If it is a simple image, the multi-scale features x1, x2, x3 are input into the detection head. Otherwise, perform a second feature extraction on the image, input the original image into the second set of backbone networks, fuse x1, x2, x3 with the second set of backbone networks, and set the features extracted by the second set of backbone networks as x 21 , x 22 , x 23 , x 11 With x 21 Concatenate and then perform 1×1 convolution to reduce the dimension. 22 , x 23 Similarly, the second set of backbone networks finally extracts the feature x 21 , x 22 , x 23 , enter the second set of backbone networks and feature pyramid networks to obtain multi-scale features, fuse the first set of outputs with the second set of outputs to obtain the final multi-scale features, and input them into the detection head.

[0129] If the image is a difficult image, in order to reduce the computational complexity of the dynamic selector, it is necessary to compress its information and obtain the compressed features: the global pooling operation and the channel dimension concatenation operation. After that, the features are mapped to the difficulty score through the linear mapping layer.

[0130] The specific steps are:

[0131]

[0132] in represents global average pooling, represents channel cascade, w1, w2, ...w L represents learnable parameters, b1, b2, ...b L First, the feature maps of different scales are converted into feature vectors of fixed dimensions through global average pooling; then the weights and biases of the fixed-dimensional feature vectors are adjusted by using linear transformation, and finally the channel cascade is used. Connect the feature vectors of each scale to form a larger feature vector. This operation compresses the multi-scale feature f1 into a vector of dimension d.

[0133] Then the vector with dimension d is After a linear layer, feature dimensionality reduction is performed.

[0134]

[0135] This action will Mapping to a lower-dimensional space can retain the most important features while reducing the computational burden and number of parameters of the model.

[0136] Then an activation function is used to get the prediction score:

[0137]

[0138] The activation function can convert the input real number into a probability distribution, so that the model can classify different categories and output the probability of each category. This can make the model better adapt to different tasks and data, and improve the accuracy and robustness of the model.

[0139] S5: Based on the dynamic selector obtained in step S2, in order to assign simple images to the detector more quickly and assign difficult images to the second detector more accurately, an optimization strategy for the dynamic selector is given, and the difficulty criterion is defined according to the corresponding training loss difference between the two detectors of the image.

[0140] S6: Figure 4 , the fusion module receives the first set of multi-scale features and the second set of multi-scale features These two sets of features represent the features of the current step t and the next step t+1 respectively. For the second set of features The fusion module calculates an attention heat map through convolutional layers and sigmoid operations to measure the importance of each position in the fusion. Similarly, for the first set of features The same operation is performed to obtain another attention heat map. Using these two attention heat maps, the fusion module calculates and Finally, the calculated weighted sum is used as the updated This updated feature will be used in subsequent steps.

[0141] The present invention also provides a dynamic structure target detection method based on live working scenarios, such as Figure 5 As shown, including:

[0142] The feature extraction network module 101 includes a combination of two backbone networks and a feature pyramid network, which is used to extract multi-scale features in an image.

[0143] The dynamic detector module 102 is used to judge the difficulty of the image. Simple images only need to go through the first set of backbone networks and pyramid networks to extract features and then be input into the detection head to obtain the detection results. For difficult images, after going through the first set of networks, they enter the second set of backbone networks and pyramid networks to extract features and continue to extract features, and then are input into the detection head for detection.

[0144] The fusion module 103 is used to fuse the first set of multi-scale features with the second set of multi-scale features and to update the first set of multi-scale features to better capture the features of the target.

[0145] The detection head module 104 is used to detect the position of the object in the image.

[0146] The invention provides a high-strength insulating blanket for low-voltage non-stop power supply operation, comprising an upper insulating layer, a middle insulating layer, a lower insulating layer and a magnetic attraction device.

[0147] The upper insulating layer of the present invention is divided into three layers, namely, TPU film, nylon fiber and TPU film. The middle insulating layer is TPU film, the lower insulating layer and the upper insulating layer have the same structure, and the magnetic attraction device is a circular sheet embedded between the upper insulating layer and the lower insulating layer of the insulating blanket.

[0148] In one embodiment of the present invention, the COCO dataset and the photographed live working scenes are preferably used as the training set, verification set and test set, which includes 50,000 images as the training dataset, 25,000 images as the verification dataset, and 5,000 images as the test dataset.

[0149] In one embodiment of the present invention, Figure 3 As shown in the figure, the backbone network in the feature extraction network usually selects classic convolutional neural network structures such as ResNet50 or ResNet101. These backbone networks can extract high-level semantic information and feature representation of images. The feature pyramid network uses upsampling and downsampling operations to construct a multi-scale feature pyramid from the different levels of features extracted by the backbone network. Through the upsampling operation, the feature map with low resolution and high semantic information can be converted into the corresponding high-resolution feature map; and through the downsampling operation, the feature map with high resolution and low semantic information can be converted into the corresponding low-resolution feature map. In addition, the high-level features are added to the low-level features after upsampling, and a fine feature map is generated through iteration. The feature pyramid constructed in this way can effectively capture information of different scales in the image.

[0150] In one embodiment of the present invention, Figure 3, the image first passes through the feature extraction network, which is composed of two sets of backbone networks and feature pyramid networks. First input image x, and get multi-scale features x1, x2, x3 through the first set of backbone networks and feature pyramid networks. The output multi-scale features x1, x2, x3 are passed through a dynamic selector to determine whether the input image is a simple image or a difficult image. If it is a simple image, the multi-scale features x1, x2, x3 are input into the detection head. Otherwise, perform a second feature extraction on the image, input the original image into the second set of backbone networks, fuse x1, x2, x3 with the second set of backbone networks, and set the features extracted by the second set of backbone networks as x 21 , x 22 , x 23 , x 11 With x 21 Concatenate and then perform 1×1 convolution to reduce the dimension. 22 , x 23 Similarly, the second set of backbone networks finally extracts the feature x 21 , x 22 , x 23 , enter the second set of backbone networks and feature pyramid networks to obtain multi-scale features, fuse the first set of outputs with the second set of outputs to obtain the final multi-scale features, and input them into the detection head.

[0151] The present invention adds a dynamic selector, which can further perform feature extraction for complex images. On the one hand, it wastes resources to detect some relatively simple background images, and on the other hand, it helps the model adapt to complex scenes and improves classification accuracy by extracting effective features. In this regard, the prior art mainly realizes accurate distinction between sensitive targets to be tested and other targets in complex scenes by setting a feature pyramid network embedded with a contextual attention module, so that the model can adapt to complex scenes. Although this method can help the model better understand the association and semantic information between targets when processing complex scenes to a certain extent, the introduction of the attention module may increase the computational complexity of the model, especially when the module is embedded in the feature pyramid network, which may lead to higher time and space complexity, thereby increasing the reasoning time and resource requirements of the model. Therefore, a relatively reasonable model design is needed to help overcome these defects and further improve model performance.

[0152] The present invention provides a dynamic structural target detection method based on a live working scenario, and the dynamic structural target detection method based on a live working scenario comprises the following steps:

[0153] S1: By using the installed camera, the image or video of the live work on site is taken as input and sent to the network;

[0154] S2: Build a target detection network, the network structure includes: feature extraction network, dynamic selector, and detection head.

[0155] S3: According to the feature extraction network obtained in step S2, the feature extraction network is composed of two sets of backbone networks and feature pyramid networks. For simple images, the network only needs to go through the first set of backbone networks and pyramid networks to extract features and then input them into the detection head to obtain the detection results. For difficult images, after going through the first set of networks, they enter the second set of backbone networks and pyramid networks to extract features and continue to extract features, and then input them into the detection head for detection.

[0156] S4: According to the dynamic selector obtained in step S2, features are extracted through the first set of backbone networks and pyramid networks and then sent to the dynamic selector for judgment. If it is a simple image, its features are input into the detection head to obtain the detection results; if it is judged to be a difficult image, its features are sent to the second set of backbone networks and pyramid networks to extract features to continue extracting features, and then input into the detection head for detection.

[0157] S5: Based on the dynamic selector obtained in step S2, in order to assign simple images to the detector more quickly and assign difficult images to the second detector more accurately, an optimization strategy for the dynamic selector is given, and the difficulty criterion is defined according to the corresponding training loss difference between the two detectors of the image.

[0158] S6: Figure 4 , the fusion module receives the first set of multi-scale features and the second set of multi-scale features, which represent the features of the current step and the next step respectively. For the second set of features, the fusion module calculates an attention heat map through the convolution layer and the Sigmoid operation to measure the importance of each position in the fusion. Similarly, the same operation is performed on the first set of features to obtain another attention heat map. The present invention uses these two attention heat maps, and the fusion module calculates the weighted sum of and. Finally, the calculated weighted sum is used as the updated feature, which will be used in subsequent steps.

[0159] The present invention provides a dynamic structure target detection method based on live working scenarios, comprising the following structure:

[0160] The feature extraction network module 101 includes a combination of two backbone networks and a feature pyramid network, which is used to extract multi-scale features in an image.

[0161] The dynamic detector module 102 is used to judge the difficulty of the image. Simple images only need to go through the first set of backbone networks and pyramid networks to extract features and then be input into the detection head to obtain the detection results. For difficult images, after going through the first set of networks, they enter the second set of backbone networks and pyramid networks to extract features and continue to extract features, and then are input into the detection head for detection.

[0162] The fusion module 103 is used to fuse the first set of multi-scale features with the second set of multi-scale features and to update the first set of multi-scale features to better capture the features of the target.

[0163] The detection head module 104 is used to detect the position of the object in the image.

[0164] Figure 6 The present invention is a structural diagram of a dynamic architecture target detection system based on a live working scenario according to a preferred embodiment of the present invention.

[0165] like Figure 6 As shown, the present invention provides a dynamic architecture target detection system based on live working scenarios, the system comprising:

[0166] An acquisition unit 601 is used to acquire a scene image of a scene detection area and input the scene image into a target detection network;

[0167] Preferably, the acquiring unit 601 is further used for:

[0168] Obtain live video of the live detection area and input the live video into the target detection network.

[0169] A first extraction unit 602, configured to extract a first feature of the scene image through a first feature extraction network of the target detection network;

[0170] A determination unit 603, configured to determine the type of the scene image through a dynamic selector of the target detection network;

[0171] Preferably, the judging unit 603 is used to judge the type of the scene image through the dynamic selector of the target detection network, and is also used to:

[0172] The information compression processing of the scene image includes: global pooling operation and channel dimension splicing operation, where the global pooling operation is:

[0173]

[0174] in represents global average pooling, represents channel cascade, w1, w2, ...w L represents the learning parameters, b1, b2, ...b L is the bias term, Represent multi-scale features;

[0175] Through the global pooling operation, the scene images of different scales are converted into feature vectors of fixed dimensions; by using linear transformation, the weights and biases of the fixed-dimensional feature vectors are adjusted, and finally through channel cascading Connect the feature vectors of each scale to form a larger feature vector, and compress the multi-scale features into a vector of dimension d.

[0176] The vector of dimension d After the linear layer, feature dimension reduction is performed.

[0177]

[0178] The prediction score is obtained through the activation function:

[0179]

[0180] The type of the scene image is determined by the prediction score.

[0181] Preferably, the judgment unit 603 is further used to optimize the dynamic selector, determine the prediction score of the scene image according to the number of objects in the scene image, and judge the type of the scene image based on the prediction score.

[0182] The first result unit 604 is used to input the first feature of the scene image into the detection head of the target detection network when it is determined that the scene image is a simple type image, and obtain the target detection result of the scene image through the detection head; or

[0183] The second extraction unit 605 is used to extract the second feature of the scene image through the second feature extraction network of the target detection network when it is determined that the scene image is a complex type of image; the fusion unit 606 is used to fuse the first feature and the second feature through the fusion module to obtain the final feature; the second result unit 607 is used to input the final feature into the detection head to obtain the target detection result of the scene image through the detection head.

[0184] Preferably, the fusion unit 606 is used to fuse the first feature and the second feature through a fusion module to obtain a final feature, and is also used to:

[0185] The fusion module receives the first feature And the second feature And the first feature And the second feature Represent the features of the current step t and the next step t+1 respectively;

[0186] For the second feature The fusion module calculates the second attention heat map through the convolution layer and the Sigmoid operation, and uses the second attention heat map to measure the importance of each position in the fusion;

[0187] For the first feature The fusion module calculates the first attention heat map through convolution layer and Sigmoid operation;

[0188] Based on the first attention heat map and the second attention heat map, the first feature is calculated through the fusion module And the second feature The weighted sum of the features is taken as the final feature.

[0189] An embodiment of the present invention provides a dynamic structural target detection system based on a live working scenario, which corresponds to another embodiment of the present invention providing a dynamic structural target detection method based on a live working scenario, and will not be repeated here.

[0190] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The schemes in the embodiments of the present invention may be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0191] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0192] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0193] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0194] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0195] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

[0196] The invention has been described above with reference to a few embodiments. However, it is readily apparent to a person skilled in the art that other embodiments than the ones disclosed above are equally within the scope of the invention, as defined by the appended patent claims.

[0197] Generally, all terms used in the claims are to be interpreted according to their ordinary meaning in the technical field, unless explicitly defined otherwise herein. All references to "a / said / the [means, components, etc.]" are to be openly interpreted as at least one instance of the means, components, etc., unless explicitly stated otherwise. The steps of any method disclosed herein do not necessarily have to be performed in the exact order disclosed, unless explicitly stated otherwise.

Claims

1. A dynamic structural target detection method based on a live working scenario, the method comprising: Acquire a scene image of the scene detection area, and input the scene image into a target detection network; Extracting a first feature of the scene image through a first feature extraction network of the target detection network; Determining the type of the scene image by a dynamic selector of the target detection network; When it is determined that the scene image is a simple type image, the first feature of the scene image is input into the detection head of the target detection network, and the target detection result of the scene image is obtained through the detection head; or When it is determined that the scene image is a complex type image, the second feature of the scene image is extracted through the second feature extraction network of the target detection network; the first feature and the second feature are fused through a fusion module to obtain a final feature; the final feature is input into a detection head, and the target detection result of the scene image is obtained through the detection head.

2. The method according to claim 1, further comprising: A live video of the live detection area is obtained, and the live video is input into the target detection network.

3. The method according to claim 1, wherein determining the type of the scene image by the dynamic selector of the target detection network comprises: The scene image is subjected to information compression processing, including: a global pooling operation and a channel dimension splicing operation, wherein the global pooling operation is: in represents global average pooling, c represents channel cascade, w1, w2, ...w L represents the learning parameters, b1, b2, ...b L is the bias term, Represent multi-scale features; Through the global pooling operation, the scene images of different scales are converted into feature vectors of fixed dimension; by using linear transformation, the weights and biases of the fixed dimension feature vectors are adjusted, and finally the feature vectors of each scale are connected through the channel cascade c to form a larger feature vector, compressing the multi-scale features into a vector of dimension d. The vector of dimension d After the linear layer, feature dimension reduction is performed. The prediction score is obtained through the activation function: The type of the scene image is determined by the prediction score.

4. According to the method of claim 1, the fusing the first feature and the second feature by a fusion module to obtain a final feature comprises: The fusion module receives the first feature And the second feature And the first feature And the second feature Represent the features of the current step t and the next step t+1 respectively; For the second feature The fusion module calculates the second attention heat map through the convolution layer and the Sigmoid operation, and measures the importance of each position in the fusion through the second attention heat map; For the first feature The fusion module calculates the first attention heat map through convolution layer and Sigmoid operation; Based on the first attention heat map and the second attention heat map, the first feature is calculated by the fusion module And the second feature The weighted sum of the features is taken as the final feature.

5. The method according to claim 1 further includes optimizing the dynamic selector, determining a prediction score of the scene image according to the number of targets in the scene image; and judging the type of the scene image based on the prediction score.

6. A dynamic structure target detection system based on live working scenarios, the system comprising: An acquisition unit, used for acquiring a scene image of a scene detection area and inputting the scene image into a target detection network; A first extraction unit, configured to extract a first feature of the scene image through a first feature extraction network of the target detection network; A determination unit, configured to determine the type of the scene image through a dynamic selector of the target detection network; A first result unit is used to input the first feature of the scene image into the detection head of the target detection network when it is determined that the scene image is a simple type image, and obtain the target detection result of the scene image through the detection head; or A second extraction unit is used to extract a second feature of the scene image through a second feature extraction network of the target detection network when it is determined that the scene image is a complex type image; A fusion unit, configured to fuse the first feature and the second feature through a fusion module to obtain a final feature; The second result unit is used to input the final feature into the detection head, and obtain the target detection result of the scene image through the detection head.

7. The system according to claim 6, wherein the acquisition unit is further configured to: A live video of the live detection area is obtained, and the live video is input into the target detection network.

8. The system according to claim 6, wherein the judging unit is used to judge the type of the scene image through the dynamic selector of the target detection network, and is also used to: Performing information compression processing on the on-site image includes: The global pooling operation is a channel dimension splicing operation, where the global pooling operation is: in represents global average pooling, c represents channel cascade, w1, w2, ...w L represents the learning parameters, b1, b2, ...b L is the bias term, Represent multi-scale features; Through the global pooling operation, the scene images of different scales are converted into feature vectors of fixed dimension; by using linear transformation, the weights and biases of the fixed dimension feature vectors are adjusted, and finally the feature vectors of each scale are connected through the channel cascade c to form a larger feature vector, compressing the multi-scale features into a vector of dimension d. The vector of dimension d After the linear layer, feature dimension reduction is performed. The prediction score is obtained through the activation function: The type of the scene image is determined by the prediction score.

9. The system according to claim 6, wherein the fusion unit is used to fuse the first feature and the second feature through a fusion module to obtain a final feature, and is also used to: The fusion module receives the first feature And the second feature And the first feature And the second feature Represent the features of the current step t and the next step t+1 respectively; For the second feature The fusion module calculates the second attention heat map through the convolution layer and the Sigmoid operation, and measures the importance of each position in the fusion through the second attention heat map; For the first feature The fusion module calculates the first attention heat map through convolution layer and Sigmoid operation; Based on the first attention heat map and the second attention heat map, the first feature is calculated by the fusion module And the second feature The weighted sum of the features is taken as the final feature.

10. According to the system of claim 6, the judgment unit is also used to optimize the dynamic selector, determine the prediction score of the scene image according to the number of targets in the scene image; and judge the type of the scene image based on the prediction score.

Images