Target detection method and device for electrified access of calibration device, and medium

Abstract

The invention discloses a target detection method for live access of a calibration device, and discloses a device with the target detection method for live access of the calibration device and a storage medium. According to the target detection method for electrified access of the calibration device, by designing a network structure suitable for target detection of electrified access of the calibration device, the recognition efficiency of a model is improved, positioning is accurate, and a more efficient and safer solution is provided for electrified access of the device.

Description

Technical Field

[0001] The present invention relates to the field of target detection technology, and in particular to a target detection method, device and medium for live access of a calibration device. Background Art

[0002] With the continuous development of the power system, the demand for live access to devices is also increasing. The traditional method of live access to devices relies on manual operation, which is inefficient, prone to misoperation, and poses safety hazards. Although the existing target detection method based on deep learning has made some progress, it has low accuracy, poor robustness, and large computational complexity in the scenario of live access to devices, making it difficult to meet the needs of practical applications. Therefore, there is an urgent need to develop a new target detection method to improve the efficiency and safety of live access to devices.

[0003] The Chinese patent with publication number CN113642553A discloses "a non-constrained license plate precision positioning method combining overall and component target detection", and proposes a license plate precision positioning method based on the YOLOv3 network. This method realizes the detection of license plate vertex targets without increasing the amount of additional calculations, but it is not applicable to the live access scenario of the existing detection device. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the present invention proposes a target detection method for live access of a calibration device, which can improve the recognition efficiency and precise positioning of the model by designing a network structure suitable for target detection of live access of a calibration device, and provide a more efficient and safer solution for live access of the device.

[0005] The present invention also proposes a system and a storage medium having the above-mentioned target detection method for live access of a calibration device.

[0006] According to the first aspect of the present invention, the target detection method for live connection of a calibration device is characterized by comprising the following steps:

[0007] Acquire a video and an image of a calibration device in a live connection scenario, and create a first data set and a second data set based on the video and the image;

[0008] Based on the principles of target detection model recognition efficiency and precise positioning, determine the network structure suitable for target detection with live access to the calibration device;

[0009] According to the principle of the highest recognition accuracy of the target detection model, determine and optimize the loss function of the model;

[0010] Based on the network structure suitable for target detection with live access to the calibration device, the first data set is used for preliminary training to obtain an initial model, and the initial model is trained using a transfer learning technique based on the second data set. If the loss function converges and the recognition accuracy is greater than a preset value, a target detection model is obtained;

[0011] The target detection model is used to predict the image to be detected, and the prediction result is output for the live connection of the calibration device.

[0012] The target detection method for live access of a calibration device according to an embodiment of the present invention has at least the following beneficial effects: the target detection method for live access of a calibration device provided by the present invention can effectively improve the recognition efficiency and precise positioning capability of the model by designing a network structure suitable for target detection for live access of a calibration device and adopting transfer learning technology for training. The recognition efficiency and precise positioning of the model are improved, providing a more efficient and safer solution for live access of the device.

[0013] According to some embodiments of the present invention, the step of acquiring a video and an image of a calibration device in a live connection scenario, and creating a first data set and a second data set based on the video and the image, comprises:

[0014] Acquire videos and images of the calibration device when it is powered on;

[0015] The collected video is processed by saving images frame by frame; the collected images are preliminarily screened together with images obtained after processing the video to form a comprehensive data set;

[0016] A first data set and a second data set are created based on the integrated data set.

[0017] According to some embodiments of the present invention, the network structure based on the target detection suitable for live access of the calibration device, using the first data set for preliminary training to obtain an initial model, further training the initial model based on the second data set using a transfer learning technique, and if the loss function converges and the recognition accuracy is greater than a preset value, then obtaining the target detection model step, further includes:

[0018] Based on the determined network structure, training the first data set to obtain an initial model;

[0019] The second data set is trained using transfer learning technology to obtain a target detection model.

[0020] According to some embodiments of the present invention, the step of training the second data set using the transfer learning technology, and obtaining the target detection model if the loss function converges and the recognition accuracy is greater than a preset value, includes:

[0021] Divide the first data set into a first training set and a first validation set in a ratio of 8:2;

[0022] Divide the second data set into a second training set and a second validation set in a ratio of 8:2;

[0023] Obtaining a model weight parameter of the first data set;

[0024] Migrating the weight parameters to the second data set, training the second data set through the network structure, and adjusting the parameters and the optimizer to obtain a training result;

[0025] The training result is judged and verified through a second verification set. If the loss function converges and the recognition accuracy is greater than 90%, the training result is used as a target detection model.

[0026] According to some embodiments of the present invention, the network structure includes an attention mechanism module and a detection head module, and the construction process of the attention mechanism module includes:

[0027] The first feature of the input is passed through the convolution layer with a convolution kernel size of 1×1 three times to obtain the second feature, the third feature and the fourth feature;

[0028] The second feature is normalized, ReLU activated, passed through a convolution layer with a convolution kernel size of 3×3, normalized, and finally passed through a convolution with a convolution kernel size of 1×1 to obtain the processed second feature;

[0029] The third and fourth features are concatenated, and then convolved with a convolution kernel size of 3×3, and then convolved with a convolution kernel size of 1×1. Finally, normalization is performed, and the ReLU activation function is used to obtain the processed third and fourth features.

[0030] The processed second feature and the processed third and fourth features are spliced ​​to obtain a fifth feature;

[0031] The fifth feature is passed through a convolution layer with a convolution kernel size of 1×1, an unpooling operation is performed, and then concatenated with the first feature to obtain a concatenated fifth feature;

[0032] The concatenated five features are convolved with a convolution kernel size of 1×1 to obtain the sixth feature.

[0033] According to some embodiments of the present invention, the construction process of the detection head module includes:

[0034] The result obtained by the attention mechanism module is sent to a 1×1 convolutional layer and then to a 3×3 convolutional layer;

[0035] The result after the 3×3 convolution processing is divided into a first path and a second path, wherein the first path sequentially enters the 3×3 convolution and the 1×1 convolution to complete the classification;

[0036] The second path enters the 3×3 convolution and is divided into two parts. The first part enters the 1×1 convolution to complete the calculation of the center point coordinates, width and height of the target detection box; the second part enters the 1×1 convolution to complete the calculation of the intersection and union ratio of the target bounding box and the true bounding box, obtain the confidence score, and complete the positioning.

[0037] According to some embodiments of the present invention, the step of determining and optimizing the loss function includes:

[0038] Set L of the attention mechanism module att The loss function is as follows:

[0039]

[0040] Where N represents the number of channels of the feature map, H represents the height of the feature map, W represents the width of the feature map, and F i,h,w represents the pixel value of the i-th channel, h-th row, and w-th column of the feature map, A represents the average activation value of the feature map, ω is the learnable parameter in the attention mechanism module, and λ is the regularization hyperparameter;

[0041] Set the L loss function of the detection head module as follows:

[0042] L=λ cls L cls +λ loc L loc +λ iou L iou

[0043]

[0044] Where L cls represents the classification loss, L loc represents the positioning loss, L iou represents the IoU loss, λ cls represents the parameter of balanced classification loss, λ loc represents the parameter of balanced localization loss, λ iou Represents the parameter of balancing IoU loss; N pos Represents the number of positive samples, Pos represents the positive sample set, It indicates the probability of predicting that the i-th sample is a positive sample, Neg indicates the negative sample set, Indicates the probability that the jth sample is a negative sample; smooth L1 represents the L1 loss function, represents the bounding box parameters (such as center coordinates) of the i-th sample predicted by the model, t i Represents the true bounding box parameters; IoU represents the intersection over union ratio, which is used to measure the degree of overlap between the predicted bounding box and the true bounding box. represents the predicted bounding box, b i represents the true bounding box; x represents the input value.

[0045] According to some embodiments of the present invention, the step of using the target detection model to perform prediction and outputting a prediction result for hot-plugging the calibration device includes:

[0046] Based on the target detection model, input an image;

[0047] Performing reasoning using a target detection model to detect access points in the image using the target detection model;

[0048] Outputs prediction results for the energized access points of the calibration device.

[0049] According to the second aspect of the present invention, the target detection device for live access of a calibration device is characterized by comprising:

[0050] A preprocessing module capable of acquiring a video and an image of a scene in which the calibration device is powered on, and creating a first data set and a second data set based on the video and the image;

[0051] The network structure building module can determine the network structure suitable for target detection with live access to the calibration device based on the principles of target detection model recognition efficiency and precise positioning;

[0052] The loss function setting module can determine and optimize the loss function of the model based on the principle of the highest recognition accuracy of the target detection model;

[0053] A detection model training module, which can use the first data set to perform preliminary training to obtain an initial model based on the network structure suitable for target detection with live access to the calibration device, and further train the initial model using a transfer learning technique based on the second data set, and obtain a target detection model if the loss function converges and the recognition accuracy is greater than a preset value;

[0054] The prediction module can use the target detection model to predict the image to be detected and output the prediction result for the calibration device to be powered on.

[0055] Furthermore, the preprocessing module comprises:

[0056] Acquire videos and images of the calibration device when it is powered on;

[0057] The collected video is processed by saving images frame by frame; the collected images are preliminarily manually screened together with images obtained after processing the video to form a comprehensive data set;

[0058] A first data set and a second data set are created based on the comprehensive data set, and data enhancement processing is performed on the first data set and the second data set.

[0059] Furthermore, the detection model training module includes:

[0060] Based on the determined network structure, training the first data set to obtain an initial model;

[0061] The second data set is trained using transfer learning technology to obtain a target detection model.

[0062] Furthermore, the process of using the transfer learning technology to train the second data set to obtain the target detection model also includes:

[0063] Divide the first data set into a first training set and a first validation set in a ratio of 8:2;

[0064] Divide the second data set into a second training set and a second validation set in a ratio of 8:2;

[0065] Obtaining a model weight parameter of the first data set;

[0066] Migrating the weight parameters to the second data set, training the second data set through the network structure, and adjusting the parameters and the optimizer to obtain a training result;

[0067] The training result is judged and verified through a second verification set. If the loss function converges and the recognition accuracy is greater than 90%, the training result is used as a target detection model.

[0068] Furthermore, the network structure includes an attention mechanism module and a detection head module, and the construction process of the attention mechanism module includes:

[0069] The first feature of the input is passed through the convolution layer with a convolution kernel size of 1×1 three times to obtain the second feature, the third feature and the fourth feature;

[0070] The second feature is normalized, ReLU activated, passed through a convolution layer with a convolution kernel size of 3×3, normalized, and finally passed through a convolution with a convolution kernel size of 1×1 to obtain the processed second feature;

[0071] The third and fourth features are concatenated, and then convolved with a convolution kernel size of 3×3, and then convolved with a convolution kernel size of 1×1. Finally, normalization is performed, and the ReLU activation function is used to obtain the processed third and fourth features.

[0072] The processed second feature and the processed third and fourth features are spliced ​​to obtain a fifth feature;

[0073] The fifth feature is passed through a convolution layer with a convolution kernel size of 1×1, an unpooling operation is performed, and then concatenated with the first feature to obtain a concatenated fifth feature;

[0074] The concatenated five features are convolved with a convolution kernel size of 1×1 to obtain the sixth feature.

[0075] Furthermore, the construction process of the detection head module includes:

[0076] The result obtained by the attention mechanism module is sent to a 1×1 convolutional layer and then to a 3×3 convolutional layer;

[0077] The result after the 3×3 convolution processing is divided into a first path and a second path, wherein the first path sequentially enters the 3×3 convolution and the 1×1 convolution to complete the classification;

[0078] The second path enters the 3×3 convolution and is divided into two parts. The first part enters the 1×1 convolution to complete the calculation of the center point coordinates, width and height of the target detection box; the second part enters the 1×1 convolution to complete the calculation of the intersection and union ratio of the target bounding box and the true bounding box, obtain the confidence score, and complete the positioning.

[0079] Furthermore, the loss function setting module includes:

[0080] Set L of the attention mechanism module att The loss function is as follows:

[0081]

[0082] Where N represents the number of channels of the feature map, H represents the height of the feature map, W represents the width of the feature map, and F i,h,w represents the pixel value of the i-th channel, h-th row, and w-th column of the feature map, A represents the average activation value of the feature map, ω is the learnable parameter in the attention mechanism module, and λ is the regularization hyperparameter;

[0083] Set the L loss function of the detection head module as follows:

[0084] L=λ cls L cls +λ loc L loc +λiou L iou

[0085]

[0086] Where L cls represents the classification loss, L loc represents the positioning loss, L iou represents the IoU loss, λ cls represents the parameter of balanced classification loss, λ loc represents the parameter of balanced localization loss, λ iou Represents the parameter of balancing IoU loss; N pos Represents the number of positive samples, Pos represents the positive sample set, It indicates the probability of predicting that the i-th sample is a positive sample, Neg indicates the negative sample set, Indicates the probability that the jth sample is a negative sample; smooth L1 represents the L1 loss function, represents the bounding box parameters (such as center coordinates) of the i-th sample predicted by the model, t i Represents the true bounding box parameters; IoU represents the intersection over union ratio, which is used to measure the degree of overlap between the predicted bounding box and the true bounding box. represents the predicted bounding box, b i represents the true bounding box; x represents the input value.

[0087] Furthermore, the prediction module includes:

[0088] Based on the target detection model, input an image;

[0089] Performing reasoning using a target detection model to detect access points in the image using the target detection model;

[0090] Output the prediction results of the calibration device's energized access points.

[0091] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0092] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0093] Figure 1 A schematic diagram of a target detection method for live connection of a calibration device according to an embodiment of the present invention;

[0094] Figure 2It is a structural block diagram of a target detection device for live access of a calibration device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0095] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0096] In the description of the present invention, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., and orientations or positional relationships indicated are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0097] In the description of the present invention, "several" means one or more, "more" means more than two, "greater than", "less than", "exceed" etc. are understood as not including the number itself, and "above", "below", "within" etc. are understood as including the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0098] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention in combination with the specific content of the technical solution.

[0099] Reference Figure 1 , an embodiment of the present invention provides a target detection method for live connection of a calibration device, the method comprising at least the following steps:

[0100] Step S100: Acquire a video and an image of a calibration device in a live connection scenario, and create a first data set and a second data set based on the video and the image.

[0101] Step S200: Based on the principles of target detection model recognition efficiency and precise positioning, determine a network structure suitable for target detection with live access to the calibration device.

[0102] Step S300: Determine and optimize the loss function of the model based on the principle of the highest recognition accuracy of the target detection model.

[0103] Step S400: Based on the network structure suitable for target detection with power connected to the calibration device, use the first data set for preliminary training to obtain an initial model, and further train the initial model based on the second data set using transfer learning technology. If the loss function converges and the recognition accuracy is greater than a preset value, a target detection model is obtained.

[0104] Step S500: using the target detection model to predict the image to be detected, and outputting the prediction result for the live connection of the calibration device.

[0105] Furthermore, in order to describe the purpose of the present application in more detail, the above steps are described in more detail. The present invention includes the following steps:

[0106] Step S100: Obtain a video and an image of a calibration device in a live connection scenario, and create a first data set and a second data set based on the video and the image. Specifically, the process includes:

[0107] Step S101: Acquire video and images of a calibration device in a live connection scenario.

[0108] Step S102: the collected video is processed by saving images frame by frame; the collected images are preliminarily manually screened together with images obtained after processing the video to integrate them into a comprehensive data set.

[0109] Step S103: Create a first data set and a second data set based on the comprehensive data set, and perform data enhancement processing on the first data set and the second data set.

[0110] Dataset A and dataset B are created based on the comprehensive dataset, and data enhancement processing is performed on dataset A and dataset B.

[0111] Step S104, dividing the first data set into a first training set and a first validation set at a ratio of 8:2; dividing the second data set into a second training set and a second validation set at a ratio of 8:2.

[0112] Divide dataset A and dataset B into training set A, validation set A, training set B, and validation set B in a ratio of 8:2. These sets will play a role in subsequent steps.

[0113] Step S200: Based on the principle of target detection model recognition efficiency and accurate positioning, determine a network structure suitable for target detection with live access to the calibration device. The network structure includes:

[0114] Step S210: Setting the attention module.

[0115] Step S220: Setting the detection head module.

[0116] Wherein, step S210, setting the attention module includes:

[0117] Step S211, the input first feature is passed through a convolution layer with a convolution kernel size of 1×1 three times to obtain a second feature, a third feature, and a fourth feature.

[0118] Step S212: normalize the second feature, use the ReLU activation function, pass it through a convolution layer with a convolution kernel size of 3×3, perform a normalization operation, and finally pass it through a convolution with a convolution kernel size of 1×1 to obtain the processed second feature.

[0119] Step S213: concatenate the third feature and the fourth feature, and then perform a convolution with a convolution kernel size of 3×3, and then a convolution with a convolution kernel size of 1×1. Finally, perform a normalization operation, and use the ReLU activation function to obtain the processed third and fourth features.

[0120] Step S214: concatenate the processed second feature with the processed third and fourth features to obtain the fifth feature.

[0121] Step S215: Pass the fifth feature through a convolution layer with a convolution kernel size of 1×1, perform an unpooling operation, and then concatenate it with the first feature to obtain a concatenated fifth feature.

[0122] Step S216: Perform convolution on the concatenated five features with a convolution kernel size of 1×1 to obtain a sixth feature.

[0123] On the other hand, step S220, setting the detection head module includes:

[0124] Step S221, sending the result obtained by the attention mechanism module to the 1×1 convolution layer, and then to the 3×3 convolution layer;

[0125] Step S222, dividing the result after the 3×3 convolution processing into a first path and a second path, wherein the first path sequentially enters the 3×3 convolution and the 1×1 convolution to complete the classification;

[0126] Step S223, the second path enters the 3×3 convolution and is then divided into two parts. The first part enters the 1×1 convolution to complete the calculation of the center point coordinates, width and height of the target detection box; the second part enters the 1×1 convolution to complete the calculation of the intersection and union ratio of the target bounding box and the true bounding box, obtain the confidence score, and complete the positioning.

[0127] Step S300: Determine and optimize the loss function of the model based on the principle of the highest recognition accuracy of the target detection model. This includes:

[0128] Step S301: Set the attention mechanism module Latt The loss function is as follows:

[0129]

[0130] Where N represents the number of channels of the feature map, H represents the height of the feature map, W represents the width of the feature map, and F i,h,w represents the pixel value of the i-th channel, h-th row, and w-th column of the feature map, A represents the average activation value of the feature map, ω is the learnable parameter in the attention mechanism module, and λ is the regularization hyperparameter;

[0131] Step S302: Set the L loss function of the detection head module, as follows:

[0132] L=λ cls L cls +λ loc L loc +λ iou L iou

[0133]

[0134] Where L cls represents the classification loss, L loc represents the positioning loss, L iou represents the IoU loss, λ cls represents the parameter of balanced classification loss, λ loc represents the parameter of balanced localization loss, λ iou Represents the parameter of balancing IoU loss; N pos Represents the number of positive samples, Pos represents the positive sample set, It indicates the probability of predicting that the i-th sample is a positive sample, Neg indicates the negative sample set, Indicates the probability that the jth sample is a negative sample; smooth L1 represents the L1 loss function, represents the bounding box parameters (such as center coordinates) of the i-th sample predicted by the model, t i Represents the true bounding box parameters; IoU represents the intersection over union ratio, which is used to measure the degree of overlap between the predicted bounding box and the true bounding box. represents the predicted bounding box, b i represents the true bounding box; x represents the input value.

[0135] Step S400: Based on the network structure suitable for target detection with live access to the calibration device, the first data set is used for preliminary training to obtain an initial model, and the initial model is further trained using transfer learning technology based on the second data set. If the loss function converges and the recognition accuracy is greater than a preset value, a target detection model is obtained. Specifically, it includes:

[0136] Step S410: Based on the determined network structure, the first data set is trained to obtain an initial model;

[0137] Step S420: Use transfer learning technology to train the second data set to obtain a target detection model.

[0138] Preferably, the above step S420 specifically includes:

[0139] Step S421, obtaining model weight parameters of the first data set;

[0140] Save the model weight parameters trained during the initial training on dataset A.

[0141] Step S422: Migrate the weight parameters to the second data set, train the second data set through the network structure, and adjust the parameters and optimizer to obtain training results.

[0142] Transfer learning technology is introduced to migrate the model weight parameters in step S421 to data set B, and data set B is trained using a network structure suitable for live access point detection of the calibration device. The parameters and optimizer are adjusted to better adapt to the task. The initial weights obtained by training data set A will be used as pre-training weights when training data set B using transfer learning technology. The common features learned on data set A are retained. The annotation information in data set B provides key information about the live access points of the calibration device for the target detection model to be obtained, helping the network to converge faster, thereby deeply learning the situation of target detection of the live access points of the calibration device, and finally training a target detection model.

[0143] Step S423: judge the training result and verify it through the second verification set. If the loss function converges and the recognition accuracy is greater than 90%, the training result is used as the target detection model.

[0144] The training result of step S422 is judged and verified through the verification set B. Here, the loss function and evaluation index after training can be directly observed. If the loss function converges and the recognition accuracy is greater than 90%, it means that the training has achieved the expected effect and the target detection model is obtained.

[0145] Step S500: Use the target detection model to predict the image to be detected, and output the prediction result for the calibration device to be powered on. Specifically, it includes:

[0146] Step S501: input an image based on the target detection model;

[0147] Step S502: Perform inference using a target detection model to detect access points in the image using the target detection model;

[0148] Step S503: output the prediction result of the energized access point of the calibration device.

[0149] According to another aspect of the present invention, there is provided a target detection device for electrically connecting a calibration device, such as Figure 2 As shown, the device 20 comprises:

[0150] A preprocessing module 201, capable of acquiring a video and an image in a scenario where the calibration device is powered on, and creating a first data set and a second data set based on the video and the image;

[0151] The network structure building module 202 can determine the network structure suitable for target detection with live access to the calibration device based on the principles of target detection model recognition efficiency and precise positioning;

[0152] The loss function setting module 203 can determine and optimize the loss function of the model according to the principle of the highest recognition accuracy of the target detection model;

[0153] The detection model training module 204 can use the first data set to perform preliminary training to obtain an initial model based on the network structure suitable for target detection with live access to the calibration device, and further train the initial model using a transfer learning technique based on the second data set. If the loss function converges and the recognition accuracy is greater than a preset value, a target detection model is obtained;

[0154] The prediction module 205 can use the target detection model to predict the image to be detected, and output the prediction result for the calibration device to be powered on.

[0155] Furthermore, the preprocessing module 201 includes:

[0156] Acquire videos and images of the calibration device when it is powered on;

[0157] The collected video is processed by saving images frame by frame; the collected images are preliminarily manually screened together with images obtained after processing the video to form a comprehensive data set;

[0158] Creating a first data set and a second data set based on the comprehensive data set, and performing data enhancement processing on the first data set and the second data set;

[0159] Divide the first data set into a first training set and a first validation set in a ratio of 8:2;

[0160] The second data set is divided into a second training set and a second validation set in a ratio of 8:2.

[0161] Furthermore, the network structure includes an attention mechanism module and a detection head module.

[0162] Furthermore, the construction process of the attention mechanism module includes:

[0163] The first feature of the input is passed through the convolution layer with a convolution kernel size of 1×1 three times to obtain the second feature, the third feature and the fourth feature;

[0164] The second feature is normalized, ReLU activated, passed through a convolution layer with a convolution kernel size of 3×3, normalized, and finally passed through a convolution with a convolution kernel size of 1×1 to obtain the processed second feature;

[0165] The third and fourth features are concatenated, and then convolved with a convolution kernel size of 3×3, and then convolved with a convolution kernel size of 1×1. Finally, normalization is performed, and the ReLU activation function is used to obtain the processed third and fourth features.

[0166] The processed second feature and the processed third and fourth features are spliced ​​to obtain a fifth feature;

[0167] The fifth feature is passed through a convolution layer with a convolution kernel size of 1×1, an unpooling operation is performed, and then concatenated with the first feature to obtain a concatenated fifth feature;

[0168] The concatenated five features are convolved with a convolution kernel size of 1×1 to obtain the sixth feature.

[0169] Furthermore, the construction process of the detection head module includes:

[0170] The result obtained by the attention mechanism module is sent to a 1×1 convolutional layer and then to a 3×3 convolutional layer;

[0171] The result after the 3×3 convolution processing is divided into a first path and a second path, wherein the first path sequentially enters the 3×3 convolution and the 1×1 convolution to complete the classification;

[0172] The second path enters the 3×3 convolution and is divided into two parts. The first part enters the 1×1 convolution to complete the calculation of the center point coordinates, width and height of the target detection box; the second part enters the 1×1 convolution to complete the calculation of the intersection and union ratio of the target bounding box and the true bounding box, obtain the confidence score, and complete the positioning.

[0173] Furthermore, the loss function setting module 203 includes:

[0174] Set L of the attention mechanism moduleatt The loss function is as follows:

[0175]

[0176] Where N represents the number of channels of the feature map, H represents the height of the feature map, W represents the width of the feature map, and F i,h,w represents the pixel value of the i-th channel, h-th row, and w-th column of the feature map, A represents the average activation value of the feature map, ω is the learnable parameter in the attention mechanism module, and λ is the regularization hyperparameter;

[0177] Set the L loss function of the detection head module as follows:

[0178] L=λ cls L cls +λ loc L loc +λ iou L iou

[0179]

[0180]

[0181] Where L cls represents the classification loss, L loc represents the positioning loss, L iou represents the IoU loss, λ cls represents the parameter of balanced classification loss, λ loc represents the parameter of balanced localization loss, λ iou Represents the parameter of balancing IoU loss; N pos Represents the number of positive samples, Pos represents the positive sample set, It indicates the probability of predicting that the i-th sample is a positive sample, Neg indicates the negative sample set, Indicates the probability that the jth sample is a negative sample; smooth L1 represents the L1 loss function, represents the bounding box parameters (such as center coordinates) of the i-th sample predicted by the model, t i Represents the true bounding box parameters; IoU represents the intersection over union ratio, which is used to measure the degree of overlap between the predicted bounding box and the true bounding box. represents the predicted bounding box, b i represents the true bounding box; x represents the input value.

[0182] Furthermore, the detection model training module 204 includes:

[0183] Based on the determined network structure, training the first data set to obtain an initial model;

[0184] The second data set is trained using transfer learning technology to obtain a target detection model.

[0185] Furthermore, the process of using the transfer learning technology to train the second data set to obtain the target detection model also includes:

[0186] Obtaining a model weight parameter of the first data set;

[0187] Migrating the weight parameters to the second data set, training the second data set through the network structure, and adjusting the parameters and the optimizer to obtain a training result;

[0188] The training result is judged and verified through a second verification set. If the loss function converges and the recognition accuracy is greater than 90%, the training result is used as a target detection model.

[0189] Furthermore, the prediction module 205 includes:

[0190] Based on the target detection model, input an image;

[0191] Performing reasoning using a target detection model to detect access points in the image using the target detection model;

[0192] Output the prediction results of the calibration device's energized access points.

[0193] Another embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions for executing the above-mentioned Figure 1 A target detection method for live access to a calibration device is shown.

[0194] The embodiment of the present application designs a network structure suitable for target detection during live access of the calibration device, thereby improving the recognition efficiency and precise positioning of the model, and providing a more efficient and safer solution for live access of the device.

[0195] The device embodiments described above are only illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0196] It will be appreciated by those skilled in the art that all or some of the steps and systems in the disclosed method above may be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that may be used to store desired information and may be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0197] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the above-mentioned implementation mode. Technical personnel familiar with the field can also make various equivalent deformations or substitutions without violating the spirit of the present application. These equivalent deformations or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. A target detection method for live access of a calibration device, characterized in that: The following steps are involved: Acquire a video and an image of a calibration device in a live connection scenario, and create a first data set and a second data set based on the video and the image; Based on the principles of target detection model recognition efficiency and precise positioning, determine the network structure suitable for target detection with live access to the calibration device; According to the principle of the highest recognition accuracy of the target detection model, determine and optimize the loss function of the model; Based on the network structure suitable for target detection with live access to the calibration device, the first data set is used for preliminary training to obtain an initial model, and the initial model is trained using a transfer learning technique based on the second data set. If the loss function converges and the recognition accuracy is greater than a preset value, a target detection model is obtained; The target detection model is used to predict the image to be detected, and the prediction result is output for the live connection of the calibration device.

2. The method according to claim 1, characterized in that The step of acquiring a video and an image of a scene in which the calibration device is powered on, and creating a first data set and a second data set based on the video and the image, comprises: Acquire videos and images of the calibration device when it is powered on; The collected video is processed by saving images frame by frame; the collected images are preliminarily screened together with images obtained after processing the video to form a comprehensive data set; A first data set and a second data set are created based on the integrated data set.

3. The method according to claim 2, characterized in that The step of performing preliminary training using the first data set to obtain an initial model based on the network structure for target detection suitable for live access of the calibration device, further training the initial model using a transfer learning technique based on the second data set, and obtaining a target detection model if the loss function converges and the recognition accuracy is greater than a preset value, further includes: Based on the determined network structure, training the first data set to obtain an initial model; The second data set is trained using transfer learning technology to obtain a target detection model.

4. The method according to claim 3, characterized in that The second data set is trained by using the transfer learning technology, and if the loss function converges and the recognition accuracy is greater than a preset value, a target detection model is obtained. The step of obtaining the target detection model includes: Divide the first data set into a first training set and a first validation set in a ratio of 8:2; Divide the second data set into a second training set and a second validation set in a ratio of 8:2; Obtaining a model weight parameter of the first data set; Migrating the weight parameters to the second data set, training the second data set through the network structure, and adjusting the parameters and the optimizer to obtain a training result; The training result is judged and verified through a second verification set. If the loss function converges and the recognition accuracy is greater than 90%, the training result is used as a target detection model.

5. The method according to claim 1, characterized in that The network structure includes an attention mechanism module and a detection head module. The construction process of the attention mechanism module includes: The first feature of the input is passed through the convolution layer with a convolution kernel size of 1×1 three times to obtain the second feature, the third feature and the fourth feature; The second feature is normalized, ReLU activated, passed through a convolution layer with a convolution kernel size of 3×3, normalized, and finally passed through a convolution with a convolution kernel size of 1×1 to obtain the processed second feature; The third and fourth features are concatenated, and then convolved with a convolution kernel size of 3×3, and then convolved with a convolution kernel size of 1×1. Finally, normalization is performed, and the ReLU activation function is used to obtain the processed third and fourth features. The processed second feature and the processed third and fourth features are spliced ​​to obtain a fifth feature; The fifth feature is passed through a convolution layer with a convolution kernel size of 1×1, an unpooling operation is performed, and then concatenated with the first feature to obtain a concatenated fifth feature; The concatenated five features are convolved with a convolution kernel size of 1×1 to obtain the sixth feature.

6. The method according to claim 5, characterized in that The construction process of the detection head module includes: The result obtained by the attention mechanism module is sent to a 1×1 convolutional layer and then to a 3×3 convolutional layer; The result after the 3×3 convolution processing is divided into a first path and a second path, wherein the first path sequentially enters the 3×3 convolution and the 1×1 convolution to complete the classification; The second path enters the 3×3 convolution and is divided into two parts. The first part enters the 1×1 convolution to complete the calculation of the center point coordinates, width and height of the target detection box; the second part enters the 1×1 convolution to complete the calculation of the intersection and union ratio of the target bounding box and the true bounding box, obtain the confidence score, and complete the positioning.

7. The method according to claim 1, characterized in that The step of determining and optimizing the loss function of the model includes: Set L of the attention mechanism module att The loss function is as follows: Where N represents the number of channels of the feature map, H represents the height of the feature map, W represents the width of the feature map, and F i,h,w represents the pixel value of the i-th channel, h-th row, and w-th column of the feature map, A represents the average activation value of the feature map, ω is the learnable parameter in the attention mechanism module, and λ is the regularization hyperparameter; Set the L loss function of the detection head module as follows: L=λ cls L cls +λ loc L loc +λ iou L iou Where L cls represents the classification loss, L loc represents the positioning loss, L iou represents the IoU loss, λ cls represents the parameter of balanced classification loss, λ loc represents the parameter of balanced localization loss, λ iou N represents the parameter for balancing IoU loss; pos Represents the number of positive samples, Pos represents the positive sample set, It indicates the probability of predicting that the i-th sample is a positive sample, Neg indicates the negative sample set, Indicates the probability that the jth sample is a negative sample; smooth L1 represents the L1 loss function, represents the bounding box parameters (such as center coordinates) of the i-th sample predicted by the model, t i Represents the true bounding box parameters; IoU represents the intersection over union ratio, which is used to measure the degree of overlap between the predicted bounding box and the true bounding box. represents the predicted bounding box, b i represents the true bounding box; x represents the input value.

8. The method according to claim 1, characterized in that The step of using the target detection model to make predictions and outputting prediction results for the live connection of the calibration device includes: Based on the target detection model, input an image; Performing reasoning using a target detection model to detect access points in the image using the target detection model; Output the prediction results of the calibration device's energized access points.

9. A target detection device for live access to a calibration device, characterized in that: include: A preprocessing module capable of acquiring a video and an image of a scene in which the calibration device is powered on, and creating a first data set and a second data set based on the video and the image; The network structure building module can determine the network structure suitable for target detection with live access to the calibration device based on the principles of target detection model recognition efficiency and precise positioning; The loss function setting module can determine and optimize the loss function of the model based on the principle of the highest recognition accuracy of the target detection model; A detection model training module, which can use the first data set to perform preliminary training to obtain an initial model based on the network structure suitable for target detection with live access to the calibration device, and further train the initial model using a transfer learning technique based on the second data set, and obtain a target detection model if the loss function converges and the recognition accuracy is greater than a preset value; The prediction module can use the target detection model to predict the image to be detected and output the prediction result for the calibration device to be powered on.

10. The device according to claim 9, characterized in that The preprocessing module comprises: Acquire videos and images of the calibration device when it is powered on; The collected video is processed by saving images frame by frame; the collected images are preliminarily screened together with images obtained after processing the video to form a comprehensive data set; A first data set and a second data set are created based on the integrated data set.

11. The device according to claim 10, characterized in that The detection model training module also includes: Based on the determined network structure, training the first data set to obtain an initial model; The second data set is trained using transfer learning technology to obtain a target detection model.

12. The device according to claim 11, characterized in that The detection model training module includes: Divide the first data set into a first training set and a first validation set in a ratio of 8:2; Divide the second data set into a second training set and a second validation set in a ratio of 8:2; Obtaining a model weight parameter of the first data set; Migrating the weight parameters to the second data set, training the second data set through the network structure, and adjusting the parameters and the optimizer to obtain a training result; The training result is judged and verified through a second verification set. If the loss function converges and the recognition accuracy is greater than 90%, the training result is used as a target detection model.

13. The device according to claim 9, characterized in that The network structure includes an attention mechanism module and a detection head module. The construction process of the attention mechanism module includes: The first feature of the input is passed through the convolution layer with a convolution kernel size of 1×1 three times to obtain the second feature, the third feature and the fourth feature; The second feature is normalized, ReLU activated, passed through a convolution layer with a convolution kernel size of 3×3, normalized, and finally passed through a convolution with a convolution kernel size of 1×1 to obtain the processed second feature; The third and fourth features are concatenated, and then convolved with a convolution kernel size of 3×3, and then convolved with a convolution kernel size of 1×1. Finally, normalization is performed, and the ReLU activation function is used to obtain the processed third and fourth features. The processed second feature and the processed third and fourth features are spliced ​​to obtain a fifth feature; The fifth feature is passed through a convolution layer with a convolution kernel size of 1×1, an unpooling operation is performed, and then concatenated with the first feature to obtain a concatenated fifth feature; The concatenated five features are convolved with a convolution kernel size of 1×1 to obtain the sixth feature.

14. The device according to claim 13, characterized in that The construction process of the detection head module includes: The result obtained by the attention mechanism module is sent to a 1×1 convolutional layer and then to a 3×3 convolutional layer; The result after the 3×3 convolution processing is divided into a first path and a second path, wherein the first path sequentially enters the 3×3 convolution and the 1×1 convolution to complete the classification; The second path enters the 3×3 convolution and is divided into two parts. The first part enters the 1×1 convolution to complete the calculation of the center point coordinates, width and height of the target detection box; the second part enters the 1×1 convolution to complete the calculation of the intersection and union ratio of the target bounding box and the true bounding box, obtain the confidence score, and complete the positioning.

15. The device according to claim 9, characterized in that The loss function setting module comprises: Set L of the attention mechanism module att The loss function is as follows: Where N represents the number of channels of the feature map, H represents the height of the feature map, W represents the width of the feature map, and F i,h,w represents the pixel value of the i-th channel, h-th row, and w-th column of the feature map, A represents the average activation value of the feature map, ω is the learnable parameter in the attention mechanism module, and λ is the regularization hyperparameter; Set the L loss function of the detection head module as follows: L=λ cls L cls +λ loc L loc +λ iou L iou Where L cls represents the classification loss, L loc represents the positioning loss, L iou represents the IoU loss, λ cls represents the parameter of balanced classification loss, λ loc represents the parameter of balanced localization loss, λ iou N represents the parameter for balancing IoU loss; pos Represents the number of positive samples, Pos represents the positive sample set, It indicates the probability of predicting that the i-th sample is a positive sample, Neg indicates the negative sample set, Indicates the probability that the jth sample is a negative sample; smooth L1 represents the L1 loss function, represents the bounding box parameters (such as center coordinates) of the i-th sample predicted by the model, t i Represents the true bounding box parameters; IoU represents the intersection over union ratio, which is used to measure the degree of overlap between the predicted bounding box and the true bounding box. represents the predicted bounding box, b i represents the true bounding box; x represents the input value.

16. The device according to claim 9, characterized in that The prediction module comprises: Based on the target detection model, input an image; Performing reasoning using a target detection model to detect access points in the image using the target detection model; Output the prediction results of the calibration device's energized access points.

17. A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the method according to any one of claims 1 to 8.

Images