Target detection model, target detection method, equipment and storage medium

Abstract

The invention discloses a target detection model, a target detection method, equipment and a storage medium, and relates to the technical field of image processing. The model comprises a backbone network, a neck network and a detection head. And at least one attention module is arranged between any two adjacent modules between the first up-sampling module and the eighth decomposition module of the neck network. At least one attention module is arranged in the neck network of the target detection model, so that the target detection model not only notices global and local information of features, but also fully considers the spatial relationship and position information of pixel coordinates during detection, and thus high-precision detection of small objects can be better carried out. Therefore, the target detection module provided by the embodiment of the invention can improve the detection accuracy of the detection target in the related image of the electric power system, thereby avoiding the occurrence of events affecting the safety of the electric power system and the personal safety, and improving the operation efficiency and operation safety of the electric power system.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a target detection model, a target detection method, a device and a storage medium. Background Art

[0002] my country has the largest and most complex power system in the world, with many unattended power facilities. In order to ensure the safety of the power system and personnel, it is necessary to monitor the environment around the power system. However, at present, the huge power transmission and distribution lines are spread all over the country. For this situation, manual monitoring is almost impossible and extremely inefficient.

[0003] At present, cameras can be used to collect images around the power system, and then image recognition technology can be used to determine detection targets in the images that may affect the safety of the power system and personal safety.

[0004] However, in scenarios where the power system is relatively large, for example, in scenarios where high-voltage lines in the power system are monitored, small objects in the image are difficult to detect effectively because the detection targets such as people and objects are smaller than the high-voltage lines, and the camera is usually installed in a place where a section of the high-voltage line can be monitored, with a relatively high viewing angle. In other words, the target detection accuracy is low, which may lead to incidents that affect the safety of the power system and personal safety, and reduce the operating efficiency and safety of the power system. Summary of the invention

[0005] The present invention provides a target detection model, a target detection method, a device and a storage medium to solve the technical problem of low target detection accuracy in related technologies.

[0006] According to one aspect of the present invention, a target detection model is provided, the model comprising: a backbone network, a neck network and a detection head; the backbone network comprises: a first convolution module, a second convolution module, a first decomposition module, a third convolution module, a second decomposition module, a fourth convolution module, a third decomposition module, a fifth convolution module, a fourth decomposition module and a spatial pyramid pooling function SPPF module connected in sequence; the neck network comprises: a first upsampling module, a first aggregation module, a fifth decomposition module, a second upsampling module, a second aggregation module, a sixth decomposition module, a sixth convolution module, a third aggregation module, a seventh decomposition module, a seventh convolution module, a fourth aggregation module and an eighth decomposition module connected in sequence; the neck network further comprises at least one attention module, each attention module being arranged between any two adjacent modules between the first upsampling module and the eighth decomposition module; the detection head comprises: a first detection module, a second detection module and a third detection module; the detection head is used to determine the detection result of the detection target in the power system related image input to the backbone network;

[0007] The second decomposition module is also connected to the second aggregation module, the third decomposition module is also connected to the first aggregation module, the SPPF module is respectively connected to the first up-sampling module and the fourth aggregation module, the fifth decomposition module is also connected to the third aggregation module, the sixth decomposition module is also connected to the first detection module, the seventh decomposition module is connected to the second detection module, and the eighth decomposition module is connected to the third detection module;

[0008] The attention module is used to compress the input feature map in the width and height directions respectively to obtain a width compression map and a height compression map, perform convergence convolution operations, regularization activation operations and splitting operations on the width compression map and the height compression map, perform regularization activation operations on the height map and the width map after the split operation respectively to obtain height weight and width weight, and obtain an output feature map based on the input feature map, the height weight and the width weight.

[0009] According to another aspect of the present invention, there is provided a target detection method, comprising:

[0010] Acquire relevant images of the power system to be inspected;

[0011] The power system related image is input into a target detection model to obtain a detection result of the detection target output by the target detection model; wherein the target detection model is the target detection model provided by any of the above embodiments.

[0012] According to another aspect of the present invention, there is provided a target detection device, comprising:

[0013] The acquisition module is used to acquire images related to the power system to be detected.

[0014] The determination module is used to input the power system related image into the target detection model to obtain the detection result of the detection target output by the target detection model.

[0015] Among them, the target detection model is the target detection model provided by any of the above embodiments.

[0016] According to another aspect of the present invention, there is provided an electronic device, the electronic device comprising:

[0017] at least one processor; and

[0018] a memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the target detection method described in any embodiment of the present invention.

[0020] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and the computer program is used to enable a processor to implement the target detection method described in any embodiment of the present invention when executed.

[0021] According to another aspect of the present invention, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the target detection method according to any embodiment of the present invention.

[0022] The technical solution of the embodiment of the present invention, by setting at least one attention module in the neck network of the target detection model, enables the target detection model to not only pay attention to the global and local information of the features when performing detection, but also fully consider the spatial relationship and position information of the pixel coordinates, so as to better perform high-precision detection of small objects. Therefore, the target detection module provided by this embodiment can improve the detection accuracy of the detection targets in the images related to the power system, thereby avoiding the occurrence of events that affect the safety of the power system and personal safety, and improving the operating efficiency and safety of the power system.

[0023] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 is a schematic diagram of the structure of a target detection model provided by an embodiment of the present invention;

[0026] Figure 2 is a schematic diagram of an SPPF module in a target detection model provided by an embodiment of the present invention;

[0027] Figure 3 is a schematic diagram of the structure of another target detection model provided by an embodiment of the present invention;

[0028] Figure 4 is a schematic diagram of an attention module in another target detection model provided by an embodiment of the present invention;

[0029] Figure 5 is a flow chart of a target detection method provided by an embodiment of the present invention;

[0030] Figure 6 is a schematic structural diagram of a target detection device provided by an embodiment of the present invention;

[0031] Figure 7 It is a schematic diagram of the structure of an electronic device for implementing the target detection method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0032] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the term "comprising" and any variation thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices. The acquisition, storage, use, processing, etc. of data in the embodiments of the present invention all comply with the relevant provisions of national laws and regulations.

[0034] Figure 1 Schematic diagram of a target detection model provided by an embodiment of the present invention. Figure 1 As shown, the target detection model provided in this embodiment includes a backbone network, a neck network and a detection head.

[0035] The backbone network includes: a first convolution module, a second convolution module, a first decomposition module, a third convolution module, a second decomposition module, a fourth convolution module, a third decomposition module, a fifth convolution module, a fourth decomposition module and a spatial pyramid pooling feature (SPPF) module connected in sequence.

[0036] The neck network includes: a first upsampling module, a first convergence module, a fifth decomposition module, a second upsampling module, a second convergence module, a sixth decomposition module, a sixth convolution module, a third convergence module, a seventh decomposition module, a seventh convolution module, a fourth convergence module and an eighth decomposition module connected in sequence. The neck network also includes at least one attention module, each attention module is arranged between any two adjacent modules between the first upsampling module and the eighth decomposition module.

[0037] The detection head comprises: a first detection module, a second detection module and a third detection module; the detection head is used to determine the detection result of the detection target in the power system related image input into the backbone network.

[0038] The second decomposition module is also connected to the second aggregation module, the third decomposition module is also connected to the first aggregation module, the SPPF module is respectively connected to the first upsampling module and the fourth aggregation module, the fifth decomposition module is also connected to the third aggregation module, the sixth decomposition module is also connected to the first detection module, the seventh decomposition module is connected to the second detection module, and the eighth decomposition module is connected to the third detection module.

[0039] The attention module is used to compress the input feature map in the width and height directions respectively to obtain a width compression map and a height compression map, perform convergence convolution operations, regularization activation operations and split operations on the width compression map and the height compression map, perform regularization activation operations on the height map and the width map after the split operation respectively to obtain height weight and width weight, and obtain the output feature map based on the input feature map, height weight and width weight.

[0040] The target detection model provided in this embodiment is used to output the detection result of the detection target in the power system related image. The power system related image in this embodiment refers to the image of the surrounding environment of the power system that needs to be monitored. Exemplarily, the power system related image in this embodiment can be an image near a high-voltage line.

[0041] The detection target in this embodiment may include a person, an object, or a predefined dangerous behavior. The object in this embodiment may be, for example, an object that may endanger the power system, such as a crane or other engineering vehicle passing under a high-voltage line. The dangerous behavior in this embodiment may be, for example, fishing under a high-voltage line.

[0042] In this embodiment, the detection result of the detection target determined by the detection head includes at least one of the following: the size of the detection target, the position of the detection target, and the category of the detection target.

[0043] The object detection model provided in this embodiment is an improved model based on the You Only Look Once version 8 (YOLOv8) model. The YOLOv8 model was originally designed as a real-time object recognition system and is known for its fast processing speed and relatively accurate prediction results. The YOLOv8 model significantly improves the processing speed by predicting the location and category of the object at one time in a single network.

[0044] However, even with the powerful performance of the YOLOv8 model, there are some unavoidable limitations. For the recognition of small objects, especially in complex power system environments, the YOLOv8 model often has difficulty accurately finding the detection target in the image. This is because small objects may not only be blocked by other devices or structures, but also have poor visibility in the image due to their small size, which increases the difficulty of recognition.

[0045] In the target detection model provided in this embodiment, at least one attention module is added to the neck network of the YOLOv8 model. Each attention module is arranged between any two adjacent modules between the first upsampling module and the eighth decomposition module. For example, an attention module may be arranged between the first upsampling module and the first convergence module and / or an attention module may be arranged between the fifth decomposition module and the second upsampling module. Figure 1 In the example, setting an attention module between the first upsampling module and the first aggregation module is used for explanation.

[0046] In YOLOv8, there has always been a problem that the precise spatial relationship of the input is lost during the upsampling process. To solve this problem, the object detection module provided in this embodiment is equivalent to integrating the spatial coordinate enhanced attention mechanism in the neck network of the YOLOv8 model. The spatial coordinate enhanced attention mechanism in this embodiment enhances feature learning by combining the spatial relationship based on pixel coordinates.

[0047] Different from the traditional method of processing feature maps, the spatial coordinate enhanced attention mechanism strengthens feature learning by combining spatial relationships based on pixel coordinates. By introducing specific convolution transformations, accurate location information can be provided, thereby improving the effect of feature extraction and better identifying and detecting objects of small size or complex environmental backgrounds.

[0048] The attention module in this embodiment is used to perform the following operations. Step 1.1: Compress the input feature map in the width direction to obtain a width compression map, and compress the input feature map in the height direction to obtain a height compression map. Step 1.2: Perform a convergence convolution operation, a regularization activation operation, and a split operation on the width compression map and the height compression map to obtain a height map and a width map after the split operation. Step 1.3: Perform a regularization activation operation on the height map after the split operation to obtain a height weight, and perform a regularization activation operation on the width map after the split operation to obtain a width weight. Step 1.4: According to the input feature map, the height weight, and the width weight, the output feature map is obtained.

[0049] In step 1.1, it is equivalent to aggregating features along the two spatial directions of width and height to obtain a width compression map and a height compression map. This allows the attention module to capture long-term dependencies and precise location information along these two spatial directions, which helps the target detection model to more accurately locate the target of interest. However, the width compression map and the height compression map obtained in step 1.1 do not provide precise location information. Therefore, in step 1.2, the width compression map and the height compression map are subjected to convergence convolution operations and regularization activation operations, so that precise location information can be determined. Afterwards, the output feature map is obtained based on steps 1.3 and 1.4.

[0050] It should be noted that the input feature map of the attention module is different based on the position of the attention module. Figure 1 Taking the position of the attention module in the example, the input feature map of the attention module is the feature map output by the first upsampling module. After obtaining the output feature map, the attention module inputs the output feature map to the first convergence module.

[0051] The YOLOv8 model has effectively encoded features of different sizes from bottom to top, and the spatial coordinate enhanced attention mechanism in the neck network of the target detection model of this embodiment further integrates and enhances features. By re-weighting the positional relationship of pixels through the spatial coordinate enhanced attention mechanism module, the neck network will be able to pay more attention to the most useful information. Therefore, the addition of an attention module based on the spatial coordinate enhanced attention mechanism in this embodiment is of great benefit.

[0052] The main difference between the spatial coordinate enhanced attention mechanism and the traditional attention mechanism is that the traditional attention mechanism is usually based on the global or local attention of features, while the spatial coordinate enhanced attention mechanism strengthens feature learning based on the spatial relationship of pixel coordinates.

[0053] In the traditional attention mechanism, the processed input feature map fully considers the global or local information of the features, but the spatial relationship and position information between features are often not fully processed, and there is a problem that the precise spatial relationship of the input will be lost during the upsampling process.

[0054] The spatial coordinate enhanced attention mechanism compresses the input feature map in two directions and then introduces a convolution transformation in the spatial domain, thereby providing accurate location information, strengthening the reinforcement effect of the spatial relationship based on pixel coordinates on feature learning, ensuring that the spatial relationship of the input is preserved during the upsampling process, and strengthening the spatial utilization of features. In addition, the spatial coordinate enhanced attention mechanism provides processing for the relationship between pixels and their cross-neighborhoods (i.e., the implementation process of steps 1.2 to 1.4), and learns the distribution relationship of features in space.

[0055] In general, the spatial coordinate enhanced attention mechanism is more comprehensive than the traditional attention mechanism in terms of feature processing. It not only pays attention to the global and local information of the features, but also fully considers the spatial relationship and position information of the pixel coordinates, so that it can better perform high-precision detection of small objects. Therefore, the target detection module provided in this embodiment can improve the detection accuracy of the detection targets in the images related to the power system, thereby avoiding incidents that affect the safety of the power system and personal safety, improving the operating efficiency and safety of the power system, and protecting the safety and stability of the power system. At the same time, the implementation of the attention module is relatively simple, so that the target detection model achieves the best balance between model performance and complexity, so as to ensure that the target detection model provided in this embodiment is both efficient and can exert effective detection performance.

[0056] Optionally, the first detection module in this embodiment is used to determine the detection result of the detection target of the first size, the second detection module is used to determine the detection result of the detection target of the second size, and the third detection module is used to determine the detection result of the detection target of the third size, and the first size, the second size and the third size increase in sequence. Each detection result may include: the size of the detection target, the position of the detection target and the category of the detection target. Based on this implementation method, the target detection model can realize the detection of detection targets of different scales, further improving the detection accuracy. At the same time, the target detection model can not only effectively identify small objects in the surrounding environment of the power system, but also effectively mark their positions, so as to realize real-time monitoring and control of the power system to ensure safety. Thereby improving the operating efficiency of the power grid and ensuring the stable supply of electricity.

[0057] In this embodiment, the power system related image is input into the backbone network, and each feature is downsampled five times. The generated feature maps of five scales are marked as follows: Figure 1B1, B2, B3, B4, and B5 in . This downsampling step ensures that the object detection model can flexibly handle objects of different sizes and scales.

[0058] Each convolution module in this embodiment (including the first convolution module, the second convolution module, the third convolution module, ..., the seventh convolution module) can also be called a convolution, batch normalization, activation (Conv, BatchNorm, SILU, abbreviated as: CBS) module. In the process from inputting power system related images to generating detection results, each CBS module plays a key role. It first performs a convolution operation on the input feature map, and then performs batch normalization processing. After these preprocessing, the CBS module will activate the information flow through SiLU and finally output the feature map.

[0059] Each decomposition module in this embodiment (including the first decomposition module, the second decomposition module, the third decomposition module, ..., the eighth convolution module) can also be called a channel-to-space factorization (C2f) module. The C2f module introduces a gradient shunt connection, which is a strategy that can strike a balance between rich information flow and light model. This design helps to enhance the feature extraction capability of the model while maintaining the high efficiency of the model.

[0060] The function of each convergence module in this embodiment (including the first convergence module, the second convergence module, the third convergence module and the fourth convergence module) is to connect multiple input feature maps to obtain an output feature map.

[0061] In order to adapt to the fixed-size output feature map, the backbone network uses the SPPF module to perform pooling on the input feature map. The SPPF module in this embodiment is slightly different from the traditional spatial pyramid pooling (SPP) module: it greatly reduces the computational workload and reduces latency by sequentially connecting three maximum pooling layers, further improving the computational efficiency and real-time performance of the model.

[0062] In the target detection model provided by this embodiment, by setting at least one attention module in the neck network, the target detection model not only pays attention to the global and local information of the features during detection, but also fully considers the spatial relationship and position information of the pixel coordinates, so as to better perform high-precision detection of small objects. Therefore, the target detection module provided by this embodiment can improve the detection accuracy of the detection targets in the images related to the power system, thereby avoiding the occurrence of events that affect the safety of the power system and personal safety, and improving the operating efficiency and safety of the power system.

[0063] Figure 2Schematic diagram of the SPPF module in a target detection model provided by an embodiment of the present invention. Figure 1 Based on the embodiment shown, different SPPF modules are provided. Figure 2 As shown, the SPPF module provided in this embodiment includes: a first convolution layer, a first pooling layer, a second pooling layer, a third pooling layer, a self-adjusting convolution layer, a convergence layer and a second convolution layer.

[0064] The first convolution layer, the self-adjusting convolution layer, the pooling layer, and the second convolution layer are connected in sequence, and the first pooling layer, the second pooling layer, and the third pooling layer are all connected to the pooling layer.

[0065] The self-adjusting convolutional layer is used to connect the multi-channel feature maps output by the first convolutional layer into a single-channel feature map.

[0066] A self-adjusting convolution layer is newly added in the SPPF module in this embodiment. Different from the traditional SPPF, the newly added self-adjusting convolution layer in this embodiment does not simply process features of different resolutions, but guides the output of the target area of ​​interest. In addition, feature maps of different channel scales can be connected in the SPPF module, and on this basis, the self-adjusting convolution layer connects the feature maps of different channels. Therefore, by introducing a self-adjusting convolution layer in the SPPF module, the SPPF module can be helped to learn the feature relationship between different resolutions, and the detection ability of the target detection model for objects of different scales is enhanced, thereby further improving the detection accuracy of the target detection model provided in this embodiment.

[0067] Figure 3 is a schematic diagram of the structure of another target detection model provided by an embodiment of the present invention. The target detection model provided by this embodiment is Figure 1 , Figure 2 Based on the illustrated embodiments and various optional implementations, a detailed description is given of the implementation of the target detection model in scenarios where there are multiple attention modules.

[0068] In this embodiment, the number of attention modules is 4, and the 4 attention modules are respectively: a first attention module, a second attention module, a third attention module and a fourth attention module.

[0069] like Figure 3 As shown, in the target detection model provided by this embodiment, the first attention module is arranged between the first upsampling module and the first convergence module. The second attention module is arranged between the second convergence module and the sixth decomposition module. The third attention module is arranged between the third convergence module and the seventh decomposition module. The fourth attention module is arranged between the fourth convergence module and the eighth decomposition module.

[0070] Each attention module in this embodiment includes a coordinate information embedding submodule and a coordinate attention generating submodule connected in sequence. It can be understood that the coordinate information embedding submodule implements spatial coordinate information embedding. The coordinate attention generating submodule implements coordinate attention generating.

[0071] The coordinate information embedding submodule is used to compress the input feature map in the width direction to obtain a width compressed map. The coordinate information embedding submodule is also used to compress the input feature map in the height direction to obtain a height compressed map.

[0072] The coordinate attention generation submodule is used to perform convergent convolution operations, regularized activation operations and split operations on the width compression map and the height compression map, and perform regularized activation operations on the height map and the width map after the split operation to obtain the height weight and the width weight, and obtain the output feature map based on the input feature map, the height weight and the width weight.

[0073] Figure 4 FIG. 1 is a schematic diagram of an attention module in another target detection model provided by an embodiment of the present invention. Figure 4 As shown, the coordinate information embedding submodule in the attention module includes a width compression layer and a height compression layer. The coordinate attention generation submodule includes a converged convolution layer, a first regularized activation layer, a splitting layer, a second regularized activation layer, a third regularized activation layer and a weighted layer. Optionally, the attention module in this embodiment also includes a residual block.

[0074] The following combination Figure 4 , explain the processing steps in the attention module in detail. Assume that the input feature map X∈R &×(×) . Among them, C represents the number of channels of the input feature map, H represents the resolution of the input feature map of a channel in the height direction, and W represents the resolution of the input feature map of a channel in the width direction. Figure 4 In , the global pooling technique is used to embed spatial coordinate information.

[0075] The width compression layer is implemented as follows: To achieve compression in the width direction. It represents the pixel value of each pixel with height h in the feature map of channel c after compression along the width direction. W refers to the number of pixels in the width direction. . c (h,i) represents the pixel value of the pixel with coordinate position (h,i) in the feature map of channel c. The output of the width compression layer is the width compression map: + ∈1 2×3×1 .

[0076] The highly compressed layers are: To achieve compression in the height direction. Indicates the pixel value of each pixel with width w in the feature map of channel c after compression along the height direction. H refers to the number of pixels in the height direction. . c (6,5) represents the pixel value of the pixel with coordinate position (6,5) in the feature map of channel c. The output of the highly compressed layer is a highly compressed image: z 9 ∈R &×1×) .

[0077] Based on the width compression layer and the height compression layer, two results representing the relationship between pixels and their cross-neighborhoods are obtained: the width compression map and the height compression map. However, these two results do not provide accurate location information. After that, the convolutional layer is converged as follows: + ,* 4 ])) implements the convergence convolution operation. Among them, F1 represents the convergence operation, and; represents the convolution transformation operation using the convolution transformation function. The feature map output by the convergence convolution layer: 1∈1 2×1×(3@-) .

[0078] The first regularization activation layer performs batch normalization on f1 and then uses a nonlinear activation function to activate it. The feature map output by the first regularization activation layer: 2∈1 2×1×(3@-) .

[0079] The split layer splits the feature map f2 into a height map and a width map. Height map: + ∈1 2×3×1 Width map: 4 ∈1 2×1×- .

[0080] The second regularization activation layer treats the height map as follows g + =B(< + (: + )) performs batch normalization and activation operations. Among them, < + represents a batch normalization operation on the height map. B represents an activation operation. g + ∈1 2×3×1 Indicates height weight.

[0081] The third regularization activation layer adjusts the width map as follows g 4 =B(<4(: 4 )) performs batch normalization and activation operations. Where <4 indicates batch normalization of the width map. B indicates activation operation. g 4 ∈1 2×1×- Represents the width weight.

[0082] The weighted layer is used in the following way Multiply the input feature map, height weight and width weight to get the output feature map F∈1 of the attention mechanism 2×3×- . Represents the height weight of the pixel with coordinate value i in the height map of channel c. Represents the width weight of the pixel with coordinate value j in the width map of channel c.

[0083] In the target detection model of this embodiment, the number of attention modules is 4, and the 4 attention modules are respectively: a first attention module, a second attention module, a third attention module and a fourth attention module. The first attention module is arranged between the first upsampling module and the first convergence module, the second attention module is arranged between the second convergence module and the sixth decomposition module, the third attention module is arranged between the third convergence module and the seventh decomposition module, and the fourth attention module is arranged between the fourth convergence module and the eighth decomposition module. Through this specific target detection model, the position information is further utilized and the accuracy is improved, which solves the problem of losing the precise spatial relationship of the input during the upsampling process, strengthens the reinforcing effect of the spatial relationship based on the pixel coordinates on feature learning, and thus further improves the detection accuracy.

[0084] When the target detection model is actually applied, the target detection model can be deployed to the edge device for target detection in the following way.

[0085] Step 1: Optimize the object detection model: Based on the computing and storage capabilities of edge devices, the object detection model needs to be optimized to meet resource constraints. The optimization process includes: model pruning, quantization, low-rank decomposition and other techniques.

[0086] Step 2: Edge device selection: Select the appropriate edge device and ensure that it has sufficient storage and computing power to run the network and process the input data. In addition, the device needs to be integrated with the power grid system to obtain relevant images of the power system to be inspected.

[0087] The edge device meets the following requirements: a quad-core CPU or above, including at least one graphics processing unit (GPU); memory and storage: at least 2GB of random access memory, 16GB of built-in storage; network connectivity: wired or wireless network connection; power supply: a stable power supply, which may include an alternating current (AC) power adapter or a rechargeable battery; deep learning tool chain.

[0088] Step 3: Environment configuration: Configure the runtime environment on the edge device. Install any dependencies required to run the model, such as runtime libraries, interpreters, etc.

[0089] Step 4 Model deployment: Deploy the optimized model to the device. This may involve uploading the model files to the device and ensuring that the application on the device can access and use them.

[0090] Step 5: Application development: Depending on the needs, you may need to develop an application or service to receive and process power system-related images, run inferences, and process the results.

[0091] Step 6: Integration and testing: Finally, integration testing is performed to ensure that the system deployed on the edge device can properly capture power system-related images, run the object detection model, and correctly process and display the results.

[0092] Step 7: Monitor and update: After deployment, you need to continuously monitor the performance of the target detection model and update and optimize the model as needed.

[0093] Figure 5 is a flow chart of a target detection method provided by an embodiment of the present invention. This embodiment can be applied to the scene of detecting images related to the power system. The method can be executed by a target detection device, and the target detection device can be implemented in the form of hardware and / or software. The target detection device can be configured in an electronic device, for example, a computer device. Figure 5 As shown, the method includes the following steps 501 and 502.

[0094] Step 501: Acquire images related to the power system to be detected.

[0095] In this embodiment, the images related to the power system to be inspected can be collected by visual sensors such as cameras.

[0096] Step 502: Inputting the power system related image into the target detection model to obtain the detection result of the detection target output by the target detection model.

[0097] Among them, the target detection model is the target detection model provided in any of the above embodiments.

[0098] As described in the above embodiments, by setting at least one attention module in the neck network, the target detection model not only pays attention to the global and local information of the features during detection, but also fully considers the spatial relationship and position information of the pixel coordinates, so as to better perform high-precision detection of small objects. Therefore, the target detection method provided in this embodiment can improve the detection accuracy of the detection targets in the images related to the power system, thereby avoiding the occurrence of events that affect the safety of the power system and personal safety, and improving the operating efficiency and safety of the power system.

[0099] Optionally, a self-adjusting convolutional layer can also be introduced into the SPPF module of the target detection model, which can help the SPPF module learn the feature relationship between different resolutions and enhance the target detection model's detection ability for objects of different scales, thereby further improving the detection accuracy of the target detection method provided in this embodiment.

[0100] Furthermore, the detection result in this embodiment includes: the size of the detection target, the position of the detection target, and the category of the detection target.

[0101] The loss function used when training the target detection model is: L(A,B)=L p (a p ,b p )+L s (a s ,b s )+L c (a c ,b c ), where A represents the actual value of m training test images, B represents the predicted value of m training test images, and L p (a p ,b p ) represents the total position loss value of m training detection images, L s (a s ,b s ) represents the total loss value of the size of m training detection images, L c (a c ,b c ) represents the total category loss value of m training detection images. Where A=(a p ,a s ,a c ), B=(b p ,b s ,b c ).

[0102]

[0103] Among them, a p (i) represents the actual value of the position of the i-th training detection image, b p (i) represents the position prediction value of the target detection model for the i-th training detection image during the training process, H(a p (i),b p (i)) represents the position loss value of the i-th training detection image, a s (i) represents the actual size of the i-th training detection image, b s(i) represents the size prediction value of the object detection model for the i-th training detection image during the training process, and a c (i) represents the actual value of the category of the i-th training detection image, b c (i) represents the category prediction value of the target detection model for the i-th training detection image during the training process, H(a c (i),b c (i)) represents the category loss value of the i-th training detection image.

[0104] Among them, y represents the actual value of the i-th training detection image, represents the predicted value of the i-th training detection image, and δ is the preset model hyperparameter. It can be understood that when y is a p (i) for b p (i). When y is a s (i) for b s (i). When y is a c (i) for b c (i).

[0105] Although mean square error, as a common loss function of deep learning models, can estimate the solution of the optimal value function well, it may lead to overestimation of the value function. Overestimation means that the value function of a deep learning model exceeds its actual expected return, which may lead to wrong decisions based on excessively high expected returns.

[0106] The loss function provided in this embodiment has the following beneficial effects. More balanced error handling: The loss function is a combination of mean absolute error and mean square error. It behaves more like mean square error when handling small errors and behaves more like mean absolute error when handling large errors. By using this balance, the target detection model can find a more appropriate balance between maximizing accuracy and robustness. Better convergence: By using model hyperparameters, the degradation of model convergence properties caused by large gradient changes can be alleviated.

[0107] This embodiment can effectively reduce the impact of a large loss function through the loss function, better deal with wrong decisions based on excessively high expected returns, and prevent the phenomenon of overestimation of the value function. It further improves the detection accuracy of the target detection model for small objects, and improves the generalization ability and robustness of the target detection model.

[0108] Figure 6 Schematic diagram of a target detection device provided by an embodiment of the present invention. Figure 6As shown, the target detection device provided in this embodiment includes: an acquisition module 61 and a determination module 62.

[0109] The acquisition module 61 is used to acquire images related to the power system to be detected.

[0110] The determination module 62 is used to input the power system related image into the target detection model to obtain the detection result of the detection target output by the target detection model.

[0111] Among them, the target detection model is the target detection model provided by any of the above embodiments.

[0112] The target detection device provided in this embodiment can also be used to train the target detection model. The loss function used in the training process is Figure 5 The loss functions used in the illustrated embodiments are the same and will not be described again here.

[0113] The target detection device provided in the embodiment of the present invention can execute the target detection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0114] Figure 7 1 is a schematic diagram of the structure of an electronic device for implementing the target detection method of an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0115] like Figure 7 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0116] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0117] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as a target detection method.

[0118] In some embodiments, the target detection method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the target detection method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the target detection method in any other appropriate manner (e.g., by means of firmware).

[0119] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0120] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0121] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0122] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0123] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0124] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0125] An embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the target detection method provided by any embodiment of the present invention.

[0126] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0127] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0128] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A target detection model, characterized in that: The model includes: a backbone network, a neck network and a detection head; the backbone network includes: a first convolution module, a second convolution module, a first decomposition module, a third convolution module, a second decomposition module, a fourth convolution module, a third decomposition module, a fifth convolution module, a fourth decomposition module and a spatial pyramid pool function SPPF module connected in sequence; the neck network includes: a first upsampling module, a first aggregation module, a fifth decomposition module, a second upsampling module, a second aggregation module, a sixth decomposition module, a sixth convolution module, a third aggregation module, a seventh decomposition module, a seventh convolution module, a fourth aggregation module and an eighth decomposition module connected in sequence; the neck network also includes at least one attention module, each attention module is arranged between any two adjacent modules between the first upsampling module and the eighth decomposition module; the detection head includes: a first detection module, a second detection module and a third detection module; the detection head is used to determine the detection result of the detection target in the power system related image input to the backbone network; The second decomposition module is also connected to the second aggregation module, the third decomposition module is also connected to the first aggregation module, the SPPF module is respectively connected to the first up-sampling module and the fourth aggregation module, the fifth decomposition module is also connected to the third aggregation module, the sixth decomposition module is also connected to the first detection module, the seventh decomposition module is connected to the second detection module, and the eighth decomposition module is connected to the third detection module; The attention module is used to compress the input feature map in the width and height directions respectively to obtain a width compression map and a height compression map, perform convergence convolution operations, regularization activation operations and splitting operations on the width compression map and the height compression map, perform regularization activation operations on the height map and the width map after the split operation respectively to obtain height weight and width weight, and obtain an output feature map based on the input feature map, the height weight and the width weight.

2. The model according to claim 1, characterized in that The SPPF module includes: a first convolution layer, a first pooling layer, a second pooling layer, a third pooling layer, a self-adjusting convolution layer, a pooling layer and a second convolution layer; The first convolution layer, the self-adjusting convolution layer, the pooling layer and the second convolution layer are connected in sequence, and the first pooling layer, the second pooling layer and the third pooling layer are all connected to the pooling layer; The self-adjusting convolution layer is used to connect the multi-channel feature map output by the first convolution layer into a single-channel feature map.

3. The model according to claim 1, characterized in that The number of the attention modules is 4, and the 4 attention modules are respectively: a first attention module, a second attention module, a third attention module and a fourth attention module; The first attention module is arranged between the first upsampling module and the first convergence module, the second attention module is arranged between the second convergence module and the sixth decomposition module, the third attention module is arranged between the third convergence module and the seventh decomposition module, and the fourth attention module is arranged between the fourth convergence module and the eighth decomposition module.

4. The model according to claim 1, characterized in that The attention module includes a coordinate information embedding submodule and a coordinate attention generation submodule connected in sequence; The coordinate information embedding submodule is used to compress the input feature map in the width direction to obtain a width compressed map, and the coordinate information embedding submodule is also used to compress the input feature map in the height direction to obtain a height compressed map; The coordinate attention generation submodule is used to perform convergence convolution operations, regularization activation operations and splitting operations on the width compression map and the height compression map, and perform regularization activation operations on the height map and the width map after the split operation to obtain height weight and width weight, and obtain the output feature map according to the input feature map, the height weight and the width weight.

5. The model according to any one of claims 1 to 4, characterized in that The first detection module is used to determine the detection result of the detection target of a first size, the second detection module is used to determine the detection result of the detection target of a second size, and the third detection module is used to determine the detection result of the detection target of a third size, and the first size, the second size and the third size increase in sequence.

6. The model according to claim 5, characterized in that The detection result includes: the size of the detection target, the position of the detection target and the category of the detection target.

7. A target detection method, characterized in that: The method comprises: Acquire relevant images of the power system to be inspected; The power system related image is input into a target detection model to obtain a detection result of the detection target output by the target detection model; wherein the target detection model is a target detection model as described in any one of claims 1 to 6.

8. The method according to claim 7, characterized in that The detection result includes: the size of the detection target, the location of the detection target and the category of the detection target; The loss function used when training the target detection model is: L(A,B)=L p (a p ,b p )+L s (a s ,b s )+L c (a c ,b c ), where A represents the actual value of m training test images, B represents the predicted value of m training test images, and L p (a p ,b p ) represents the total position loss value of m training detection images, L s (a s ,b s ) represents the total loss value of the size of m training detection images, L c (a c ,b c ) represents the total category loss value of m training detection images; Among them, a p (i) represents the actual value of the position of the i-th training detection image, b p (i) represents the position prediction value of the target detection model for the i-th training detection image during the training process, H(a p (i),b p (i)) represents the position loss value of the i-th training detection image, a s (i) represents the actual size of the i-th training detection image, b s (i) represents the size prediction value of the object detection model for the i-th training detection image during the training process, and a c (i) represents the actual value of the category of the i-th training detection image, b c (i) represents the category prediction value of the target detection model for the i-th training detection image during the training process, H(a c (i),b c (i)) represents the category loss value of the i-th training detection image; Among them, y represents the actual value of the i-th training detection image, represents the predicted value of the i-th training detection image, and δ is the preset model hyperparameter.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the target detection method described in claim 7 or 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is used to enable a processor to implement the target detection method described in claim 7 or 8 when executed.

Images