Method and device for detecting snake target in image

By using RepNCSPELAN4_CAA and GSConv technologies in snake detection, the problems of insufficient computing redundancy and feature extraction in the prior art are solved, the detection accuracy and efficiency are improved, and more efficient snake target recognition is achieved.

CN119091118APending Publication Date: 2024-12-06WUHAN POLYTECHNIC UNIVERSITY

Patent Information

Application Number
CN202411141264.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-20
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The prior art has problems of computational redundancy and insufficient feature extraction in snake detection, resulting in unsatisfactory detection speed and accuracy.

Method used

The RepNCSPELAN4_CAA multi-scale feature extraction module and GSConv detail capture convolutional layer are used to aggregate key layer information and enhance context-dependent feature extraction, optimize the hybrid encoder, and improve the accuracy and efficiency of snake object detection.

Benefits of technology

It significantly improves the accuracy and efficiency of snake object detection, maintains the lightweight model network structure, and can more efficiently capture the details and context information of the snake body in the image.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119091118A_ABST
    Figure CN119091118A_ABST
Patent Text Reader

Abstract

The invention provides a method and device for detecting a snake target in an image, and the method comprises the steps: obtaining a target image; wherein the target image is an image which needs to be subjected to snake target detection; inputting the target image into a preset snake target detection model, and outputting a snake target detection result about the target image through the preset snake target detection model; wherein the preset snake target detection model is obtained by training based on at least one group of training data, each group of training data in the at least one group of training data comprises a sample image and corresponding annotation data, and the annotation data comprises category information and position information of the snake target annotated in the sample image. By using the method of the invention, the precision and efficiency of snake target detection can be effectively improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of snake detection technology, and in particular to a method, device, electronic device, non-transitory computer-readable storage medium, and computer program product for detecting snake targets in an image. Background Art

[0002] The current drastic changes in the global environment have led to a decline in biodiversity. According to the International Union for Conservation of Nature, more than 28% of species worldwide are facing the threat of extinction, among which snakes are of particular concern. Snakes have more than 3,900 different species. They are not only diverse, but also play a key role in the ecosystem and are essential for maintaining the balance of biodiversity. Although most snake species are endangered, people often try to catch or kill them because of their fear of the potential danger of snakes, which inadvertently puts them in danger. A common misconception is that all snakes are poisonous and dangerous. This concept is particularly prevalent in areas where snakes are more common, especially among people who do not know enough about snakes. This misunderstanding not only poses a serious obstacle to scientists' protection of endangered rare snakes, but also is not conducive to maintaining the diversity of species in nature. Therefore, the development of a fast and accurate snake identification technology solution is crucial for scientific monitoring and ecological protection.

[0003] At present, in the existing technology, technicians have used machine learning and deep learning technologies to carry out snake detection work, but the existing technology solutions still have at least the following problems that need to be solved urgently; for example, 1) computational redundancy may occur during the recognition process, resulting in detection speed that is lower than expected; 2) there are deficiencies in extracting original image features, which directly affects the recognition accuracy. Summary of the invention

[0004] The present invention provides a method, an apparatus, an electronic device, a non-transitory computer-readable storage medium, and a computer program product for detecting snake targets in an image, so as to solve the defects in the prior art.

[0005] The present invention provides a method for detecting snake targets in an image, comprising: Acquire a target image; wherein the target image is an image that needs to be subjected to snake target detection; Inputting the target image into a preset snake target detection model, and outputting a snake target detection result about the target image through the preset snake target detection model; Among them, the preset snake target detection model is trained based on at least one set of training data, each set of training data in the at least one set of training data includes a sample image and corresponding annotation data, and the annotation data includes category information and location information of the snake target annotated in the sample image.

[0006] The present invention also provides a device for detecting snake targets in an image, comprising: The image acquisition module is configured to: acquire a target image; wherein the target image is an image that needs to be subjected to snake target detection; The target detection module is configured to: input the target image into a preset snake target detection model, and output a snake target detection result about the target image through the preset snake target detection model; The preset snake target detection model is trained based on at least one set of training data, each set of training data in the at least one set of training data includes a sample image and corresponding annotation data, and the annotation data includes the category information and location information of the snake target annotated in the sample image. The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the above methods for detecting snake targets in an image is implemented.

[0007] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for detecting snake targets in an image as described in any one of the above is implemented.

[0008] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above methods for detecting snake targets in an image.

[0009] As described above, the method for detecting snake targets in an image provided by the embodiment of the present invention can, on the one hand, effectively improve the accuracy and efficiency of snake target detection by aggregating key layer information and strengthening context-dependent feature extraction based on the RepNCSPELAN4_CAA. On the other hand, by optimizing the hybrid encoder part using GSConv, not only can the lightweight of the model network structure be maintained, but also the network can more efficiently capture the details of the snake body in the image and its context information. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0011] Figure 1 It is one of the flow charts of the method for detecting snake targets in an image provided by the present invention; Figure 2 is a schematic structural diagram of the preset snake target detection model provided by the present invention; Figure 3 It is a structural schematic diagram of the multi-scale feature extraction module RepNCSPELAN4_CAA provided by the present invention; Figure 4 GSConv is a schematic diagram of the structure of the detail capture convolutional layer provided by the present invention; Figure 5 This is the second flow chart of the method for detecting snake targets in an image provided by the present invention; Figure 6 is a schematic structural diagram of a device for detecting snake targets in an image provided by the present invention; Figure 7 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0012] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0013] To solve the above problems of the prior art, the inventors of the present invention have proposed, after research, a solution for detecting snake targets in an image as described in the following embodiment.

[0014] Example: The following describes the implementation details of the solution for detecting snake targets in images according to the present invention in conjunction with the accompanying drawings.

[0015] Figure 1 FIG. 1 is a flow chart of a method for detecting snake targets in an image provided by an exemplary embodiment of the present invention. This embodiment can be applied to electronic devices (eg, servers or cloud computing platforms), such as Figure 1 As shown, the method for detecting snake targets in an image includes the following steps: S110, acquiring a target image.

[0016] The target image is an image that needs to be used for snake target detection; that is, the image includes snake targets that need to be identified.

[0017] The present disclosure does not limit the acquisition method. For example, an electronic device (eg, a server or a cloud computing platform) as an execution subject may communicate with a server storing the “target image” to acquire the target image.

[0018] S120, inputting the target image into a preset snake target detection model, and outputting a snake target detection result about the target image through the preset snake target detection model.

[0019] Among them, the preset snake target detection model is trained based on at least one set of training data, each set of training data in the at least one set of training data includes a sample image and corresponding annotation data, and the annotation data includes category information and location information of the snake target annotated in the sample image.

[0020] Optionally, refer to Figure 2 The preset snake target detection model includes a cascaded backbone network, a hybrid encoder and a decoder. The backbone network and the hybrid encoder may include a detail capture convolution layer GSConv and a multi-scale feature extraction module RepNCSPELAN4_CAA. The specific details will be described below and will not be repeated here.

[0021] As described above, the method for detecting snake targets in an image provided by the embodiment of the present invention can, on the one hand, effectively improve the accuracy and efficiency of snake target detection by aggregating key layer information and strengthening context-dependent feature extraction based on the RepNCSPELAN4_CAA. On the other hand, by optimizing the hybrid encoder part using GSConv, not only can the lightweight of the model network structure be maintained, but also the network can more efficiently capture the details of the snake body in the image and its context information.

[0022] exist Figure 1 Based on the embodiment, as an optional implementation method, refer to Figure 2 , the backbone network includes a cascaded first convolution block, a multi-scale feature extraction module ( Figure 2 RepNCSPELAN4_CAA), a multi-scale feature extraction unit, and the first detail capture convolutional layer (i.e., Figure 2 GSConv as shown).

[0023] As an optional example, the first convolution block includes two cascaded first convolution layers (ie, Figure 2 ConvN shown); the multi-scale feature extraction unit includes a cascaded first multi-scale feature extraction sub-unit (including Figure 2 In the backbone network shown in the figure, the first group of cascaded ADown and RepNCSPELAN4_CAA in top-down order), the second multi-scale feature extraction subunit (including Figure 2 In the backbone network shown in the figure, the second group of cascaded ADown and RepNCSPELAN4_CAA) along the top-down order and the third multi-scale feature extraction subunit (including Figure 2In the backbone network shown in FIG. 1 , the third group of cascaded ADown and RepNCSPELAN4_CAA are arranged in a top-down order. Optionally, the first multi-scale feature extraction subunit, the second multi-scale feature extraction subunit and the third multi-scale feature extraction subunit each include a cascaded downsampling layer (ie, Figure 2 ADown) and 1 multi-scale feature extraction module ( Figure 2 RepNCSPELAN4_CAA as shown).

[0024] exist Figure 1 Based on the embodiment, as an optional implementation method, refer to Figure 2 , the hybrid encoder includes a second detail-capturing convolutional layer (based on Figure 2 The upper and lower position relationship shown is GSConv located in the upper left corner of the hybrid encoder), the third detail capture convolution layer (based on Figure 2 The upper and lower position relationship shown in the figure is GSConv, which is located above the self-attention module AIFI and is closest to it), the self-attention module AIFI, and the cross-scale feature fusion unit CCFM (i.e. Figure 2 The network is shown with dark areas covering the hybrid encoder).

[0025] As an optional example, the cross-scale feature fusion unit CCFM includes a cascaded (data flows in the direction indicated by the arrow) first fusion block, a second fusion block, a third fusion block, a fourth fusion block and one of the multi-scale feature extraction modules. The first fusion block includes a cascaded fourth detail capture convolution layer (i.e. Figure 2 As shown, GSConv connected to the self-attention module AIFI), upsampling layer Upsample and connection operation module cat; the second fusion block includes a cascade of the multi-scale feature extraction module, the fifth detail capture convolution layer, the upsampling layer and the connection operation module; the third fusion block includes a cascade of the multi-scale feature extraction module, the sixth detail capture convolution layer and the connection operation module; the fourth fusion block includes a cascade of the multi-scale feature extraction module, the seventh detail capture convolution layer and the connection operation module.

[0026] exist Figure 1 Based on the embodiment, as an optional implementation method, refer to Figure 2 , the decoder comprises a cascade of perceptual query selection units ( Figure 2 IoU_aware query selection) and prediction head unit ( Figure 2 Decoder&&Head shown); wherein the prediction head unit includes a classification prediction head and a bounding box regression prediction head.

[0027] exist Figure 1Based on the embodiment, as an optional implementation method, refer to Figure 2 , the output end of the multi-scale feature extraction module in the first multi-scale feature extraction subunit is connected to the input end of the second detail capture convolution layer; the output end of the multi-scale feature extraction module in the second multi-scale feature extraction subunit is connected to the input end of the third detail capture convolution layer; the output end of the first detail capture convolution layer GSConv is connected to the input end of the self-attention module AIFI; the output end of the second detail capture convolution layer is connected to the connection operation module in the second fusion block; the output end of the third detail capture convolution layer is connected to the connection operation module in the first fusion block; the output end of the self-attention module AIFI is connected to the input end of the fourth detail capture convolution layer; the output end of the fourth detail capture convolution layer is connected to the connection operation module in the fourth fusion block; the output end of the fifth detail capture convolution layer is connected to the connection operation module in the third fusion block; the output end of the multi-scale feature extraction module in the third fusion block, the output end of the multi-scale feature extraction module in the fourth fusion block, and the output end of the multi-scale feature extraction module cascaded with the fourth fusion block are respectively connected to the perceptual query selection unit.

[0028] exist Figure 1 Based on the embodiment, as an optional implementation method, refer to Figure 2 The convolution kernel parameters of the first convolution layer are 3×3, and the step size is 2; the convolution kernel parameters of the first detail capture convolution layer, the second detail capture convolution layer, the third detail capture convolution layer, the fourth detail capture convolution layer, and the fifth detail capture convolution layer are all 1×1; the convolution kernel parameters of the sixth detail capture convolution layer and the seventh detail capture convolution layer are 3×3, and the step size is 2.

[0029] exist Figure 1 Based on the embodiment, as an optional implementation method, refer to Figure 3 , the multi-scale feature extraction module includes cascaded channel-halved convolutional layers (i.e., Figure 3 The red Conv shown), the tensor segmentation processing layer (i.e., Figure 3 Chunk (2,1) as shown), feature extraction fusion sub-network, contextual anchor attention layer (i.e., Figure 3 The blue CAA shown), the second convolutional layer ( Figure 3 Conv shown in green).

[0030] As an optional example, the feature extraction fusion sub-network includes a first branch, a second branch and a first connection layer ( Figure 3 The first branch is connected in parallel with the second branch and is respectively connected to the input end of the first connection layer.

[0031] As an optional example, the first branch includes a cascade of first feature extraction fusion submodules ( Figure 3 As shown, according to the arrows, the first green RepNCSP), channel halving convolution layer (i.e., Figure 3 The red Conv shown), the second feature extraction fusion submodule ( Figure 3 As shown, according to the arrows, the second green RepNCSP) and the channel halving convolution layer (i.e., Figure 3 The first output end of the tensor segmentation processing layer is connected to the input end of the first connection layer, and the output end of the channel halving convolution layer in the first branch is connected to the input end of the first connection layer.

[0032] As an optional example, in the second branch, an output end of the tensor segmentation processing layer is connected to an input end of the first connection layer.

[0033] It should be noted that, refer to Figure 3 The channel parameters of the channel-halved convolution layer are [B, C / 2, H, W], where B represents the batch size, which indicates the amount of feature map data processed simultaneously; C / 2 indicates that the number of channels in the current layer is half of the number of input data channels (but usually refers to a specific, rounded number of channels); H and W represent the height and width of the feature map, respectively. Figure 3 The blue rectangles shown represent feature maps, e.g., the input channel is halved in the convolutional layer (i.e., Figure 3 The parameters corresponding to the feature map of the red Conv) are [B, C, H, W]. After being processed by the channel halving convolution layer and the tensor segmentation processing layer, the output feature map parameters become [B, C / 4, H, W].

[0034] exist Figure 1 Based on the embodiment, as an optional implementation, the structures of the first detail capture convolution layer, the second detail capture convolution layer, the third detail capture convolution layer, the fourth detail capture convolution layer, the fifth detail capture convolution layer, the sixth detail capture convolution layer, and the seventh detail capture convolution layer are the same.

[0035] Reference Figure 4 As an optional implementation, the first detail capture convolution layer includes a third convolution layer (ie, Figure 4 The orange Conv shown), the depth separation convolution layer (i.e., Figure 4 The dark blue DWConv shown), the second connection layer (i.e., Figure 4The input end of the third convolutional layer is used to receive the input channel feature tensor, and the output end of the third convolutional layer is connected to the input end of the depth separation convolutional layer and the input end of the second connection layer respectively; the output end of the depth separation convolutional layer is connected to the input end of the second connection layer; the output end of the second connection layer is connected to the input end of the tensor shape transformation layer.

[0036] The following is combined with the above Figure 2 , Figure 3 as well as Figure 4 , the working process or principle of each part in the preset snake target detection model is summarized as follows: 1) ResNet18 is used as the backbone network, combined with the RepNCSPELAN4_CAA Block module, to extract multi-scale feature maps of the image. The backbone network outputs feature maps of three different scales, corresponding to the outputs of the last three stages of the network.

[0037] Specifically, the steps for constructing the RepNCSPELAN4_CAA module include: first, halving the number of channels through a convolutional layer, and then evenly dividing the output into two parts with dimensions [B, C / 4, H, W] through Chunk. This division strategy lays the foundation for multi-path processing of features. For one of the parts after Chunk processing, the features are finely processed by the RepNCSP module and the convolutional layer respectively. This model further enhances the depth and complexity of the features by stacking a combination of two RepNCSP modules and convolutional layers. The processing results of each path are fused through the Concat operation, and finally refined by another convolutional layer to obtain a rich model output.

[0038] 2) The encoder with hybrid structure includes GSConv module, which is used to more efficiently capture the details of the snake body in the image and its contextual information; AIFI module, which is used to perform feature interaction based on self-attention mechanism on the high-level semantic feature map output by the backbone network; CCFM module, which uses convolutional network to perform cross-scale sampling and fusion on the low-level semantic feature maps of the middle two layers of the backbone network and the high-level semantic feature maps encoded by AIFI module, and outputs the fused feature sequence. Based on the encoder, IoU-aware query selection module is further used to select features with high classification scores and IoU scores based on the feature sequence output by the encoder to initialize the object query of the decoder.

[0039] Specifically, the GSConv module construction steps include: the GSConv module processes the input image (e.g. Figure 4Input C1channels) performs a 1×1 convolution to divide the number of input channels into two (e.g. Figure 4 C2 / 2channels as shown), and then performs a 5×1 depthwise separable convolution on the feature image, so that the number of output channels remains half of the total number of final outputs. Finally, it obtains the output feature image by integrating and rearranging the feature image (e.g. Figure 4 Output C2 channels as shown).

[0040] 3) Using the Transformer decoder structure, the final prediction box and classification result are generated through iterative optimization based on the image feature sequence from the encoder and the initialization query. The prediction head includes the classification prediction head and the bounding box regression prediction head, which are used for target classification prediction and bounding box coordinate prediction respectively. Among them, the initialization query refers to generating anchor boxes according to pre-defined rules.

[0041] Based on the above embodiment, as an optional implementation mode, refer to Figure 5 The training steps of the preset snake target detection model include: S510: Obtain the at least one set of training data.

[0042] Each set of training data in the at least one set of training data includes a sample image and corresponding annotation data, and the annotation data includes category information and position information of the snake target annotated in the sample image.

[0043] It should be noted that, when constructing the training data, in order to ensure the diversity of sample data, the original sample images may be firstly enhanced, and then the enhanced sample images may be annotated. The data enhancement may include but is not limited to flipping the image, random cropping, etc.

[0044] S520, respectively inputting the sample images in each group of training data into the snake target detection model to be trained, and the snake target detection model to be trained outputs the corresponding prediction result of snake target detection.

[0045] The prediction result may include location information of a predicted bounding box of a snake target and predicted classification or category information of the snake target.

[0046] S530: Using a preset comprehensive loss function, based on the labeled data in each set of training data and the corresponding prediction results, obtain a function value of the comprehensive loss function.

[0047] As an optional example, S530 may include the following steps: I1) Based on the labeled data in each set of training data and the corresponding prediction results, the function value of the strong intersection-over-union loss function and the function value of the focal loss function are calculated respectively.

[0048] Optionally, calculating a function value of a strong intersection-over-union loss function based on the prediction results in each set of training data and the corresponding labeled data includes: The function value of the strong intersection-union loss function is calculated by the following calculation formulas (1) to (4); (1) (2) (3) (4) Wherein, e represents a natural base; A represents the predicted bounding box area determined based on the boundary position information of the snake target in the prediction result; B represents the real bounding box area determined based on the boundary position information of the snake target in the annotated data; represents the absolute value of the width distance between the predicted bounding box and the first edge corresponding to the true bounding box; represents the absolute value of the width distance between the predicted bounding box and the corresponding second edge of the true bounding box; represents the absolute value of the height distance between the predicted bounding box and the first edge corresponding to the real bounding box; represents the absolute value of the height distance between the predicted bounding box and the corresponding second edge of the true bounding box; represents the width of the real bounding box determined based on the boundary position information of the snake target in the annotated data; Represents the height of the real bounding box determined based on the boundary position information of the snake target in the labeled data.

[0049] It should be noted that in the loss function In, use As a penalty factor, it does not cause the anchor box to be enlarged, because The denominator of is only related to the size of the target box, but has nothing to do with the size of the smallest outer box between the anchor box and the target box. Different from the penalty factors in other loss functions, The design of ensures that its value does not change when the anchor box is enlarged. In addition, unless the anchor box completely overlaps with the target box, It will not degenerate to 0, showing good adaptability to the target size. By combining the penalty factor of size adaptation And the penalty function of the anchor box quality, the loss function It not only optimizes the accuracy of bounding box regression, but also improves the reliability and efficiency of the model in practical applications, so that the model can achieve higher performance and better adaptability when dealing with real-time object detection tasks.

[0050] Optionally, calculating a function value of a focal loss function based on the prediction results in each set of training data and the corresponding labeled data includes: The function value of the focal loss function is calculated by the following calculation formulas (5) to (6): (5) (6) in, Indicates t The class prediction probability in the prediction results corresponding to the group training data; Indicates t The weight factor corresponding to the group training data; represents the modulation factor; Indicates t The predicted probability of the positive category in the prediction results corresponding to the group training data; Indicates t The category information in the labeled data corresponding to the group training data.

[0051] I2) Using preset weights, weighted fusion is performed on the function value of the strong intersection-over-union loss function and the function value of the focal loss function to obtain the function value of the comprehensive loss function.

[0052] Optionally, using preset weights, weighted fusion is performed on the function value of the strong intersection-over-union loss function and the function value of the focal loss function to obtain the function value of the comprehensive loss function, including: The function value of the comprehensive loss function is obtained by the following calculation formula (7); (7) in, represents the weight coefficient of the strong intersection-over-union loss function; Represents the weight coefficient of the focal loss function.

[0053] S540. Based on the function value of the comprehensive loss function, the snake target detection model to be trained is trained until a preset training completion condition is met, and the preset snake target detection model is obtained from the snake target detection model to be trained.

[0054] As an optional example, S540 may include the following steps: ①, transmitting the function value of the comprehensive loss function back to the snake target detection model to be trained. ②, the snake target detection model to be trained adjusts the network parameters of each network layer according to the function value of the comprehensive loss function. ③, iteratively executing the step of obtaining the function value of the comprehensive loss function to the step of adjusting the network parameters of each network layer until the preset training completion condition is met.

[0055] Among them, the preset training completion condition may be that the number of iterative training times meets a preset number threshold, or the function value of the comprehensive loss function is less than a preset loss threshold, that is, the comprehensive loss function converges.

[0056] As mentioned above, the loss function The design of fully considers the size and quality of the anchor box. By introducing a size-adaptive penalty factor and a gradient adjustment mechanism based on the quality of the anchor box, it effectively guides the anchor box to regress along a more direct and efficient path, thereby significantly improving the convergence speed of the model. Not only is the IoU value between the anchor box and the target box considered, but the penalty term is also dynamically adjusted to adapt to anchor boxes of different qualities. This design has given more attention to anchor boxes of medium quality, accelerating the adaptability and accuracy of the model to complex scenes. In addition, the method for detecting snake targets in images provided in an embodiment of the present invention, on the one hand, can effectively improve the accuracy and efficiency of snake target detection based on the RepNCSPELAN4_CAA by aggregating key layer information and strengthening context-dependent feature extraction. On the other hand, using GSConv to optimize the hybrid encoder part can not only keep the model network structure lightweight, but also enable the network to more efficiently capture the details of the snake body in the image and its context information.

[0057] The following is a description of the device for detecting snake targets in an image provided by the present invention. The device for detecting snake targets in an image described below and the method for detecting snake targets in an image described above can be referenced to each other.

[0058] Figure 6 FIG. 1 is a schematic diagram of a device for detecting snake targets in an image provided by an exemplary embodiment of the present invention. Figure 6As shown, the device for detecting snake targets in an image includes: an image acquisition module 610, configured to: acquire a target image; wherein the target image is an image that needs to be detected as a snake target; a target detection module 620, configured to: input the target image into a preset snake target detection model, and output a snake target detection result about the target image through the preset snake target detection model; wherein the preset snake target detection model is trained based on at least one set of training data, each set of training data in the at least one set of training data includes a sample image and corresponding annotation data, and the annotation data includes category information and position information of the snake target annotated in the sample image.

[0059] Figure 7 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 7 As shown, the electronic device may include: a processor 710, a communication interface 720, a memory 730 and a communication bus 740, wherein the processor 710, the communication interface 720 and the memory 730 communicate with each other through the communication bus 740. The processor 710 may call the logic instructions in the memory 730 to execute a method for detecting snake targets in an image, the method comprising: obtaining a target image; wherein the target image is an image that needs to be detected as a snake target; inputting the target image into a preset snake target detection model, and outputting a snake target detection result about the target image through the preset snake target detection model; wherein the preset snake target detection model is obtained by training based on at least one set of training data, each set of training data in the at least one set of training data includes a sample image and corresponding annotation data, and the annotation data includes category information and position information of the snake target annotated in the sample image.

[0060] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0061] On the other hand, the present invention also provides a computer program product, which includes a computer program, and the computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the method for detecting snake targets in an image provided by the above-mentioned methods, and the method includes: obtaining a target image; wherein the target image is an image that needs to be detected as a snake target; inputting the target image into a preset snake target detection model, and outputting a snake target detection result about the target image through the preset snake target detection model; wherein the preset snake target detection model is trained based on at least one set of training data, and each set of training data in the at least one set of training data includes a sample image and corresponding annotation data, and the annotation data includes category information and location information of the snake target annotated in the sample image.

[0062] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the methods for detecting snake targets in images provided by the above-mentioned methods, the methods comprising: obtaining a target image; wherein the target image is an image that requires snake target detection; inputting the target image into a preset snake target detection model, and outputting a snake target detection result about the target image through the preset snake target detection model; wherein the preset snake target detection model is trained based on at least one set of training data, each set of training data in the at least one set of training data includes a sample image and corresponding annotation data, and the annotation data includes category information and location information of the snake target annotated in the sample image.

[0063] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0064] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0065] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting snake targets in an image, characterized in that: include: Acquire a target image; wherein the target image is an image that needs to be subjected to snake target detection; Inputting the target image into a preset snake target detection model, and outputting a snake target detection result about the target image through the preset snake target detection model; Among them, the preset snake target detection model is trained based on at least one set of training data, each set of training data in the at least one set of training data includes a sample image and corresponding annotation data, and the annotation data includes category information and location information of the snake target annotated in the sample image.

2. The method according to claim 1, characterized in that: The preset snake target detection model includes a cascaded backbone network, a hybrid encoder and a decoder.

3. The method according to claim 2, characterized in that The backbone network includes a cascaded first convolution block, a multi-scale feature extraction module, a multi-scale feature extraction unit, and a first detail capture convolution layer; The first convolution block includes two cascaded first convolution layers; The multi-scale feature extraction unit includes a cascaded first multi-scale feature extraction sub-unit, a second multi-scale feature extraction sub-unit and a third multi-scale feature extraction sub-unit; The first multi-scale feature extraction subunit, the second multi-scale feature extraction subunit and the third multi-scale feature extraction subunit respectively include a cascaded downsampling layer and one of the multi-scale feature extraction modules; The hybrid encoder includes a second detail capturing convolutional layer, a third detail capturing convolutional layer, a self-attention module, and a cross-scale feature fusion unit; The cross-scale feature fusion unit includes a cascaded first fusion block, a second fusion block, a third fusion block, a fourth fusion block and one of the multi-scale feature extraction modules; The first fusion block includes a cascaded fourth detail capture convolutional layer, an upsampling layer, and a connection operation module; The second fusion block includes a cascade of the multi-scale feature extraction module, the fifth detail capture convolution layer, the upsampling layer and the connection operation module; The third fusion block includes a cascaded multi-scale feature extraction module, a sixth detail capture convolutional layer, and a connection operation module; The fourth fusion block includes a cascaded multi-scale feature extraction module, a seventh detail capture convolutional layer, and a connection operation module; The decoder comprises a cascaded perceptual query selection unit and a prediction head unit; wherein the prediction head unit comprises a classification prediction head and a bounding box regression prediction head; An output end of the multi-scale feature extraction module in the first multi-scale feature extraction subunit is connected to an input end of the second detail capture convolutional layer; An output end of the multi-scale feature extraction module in the second multi-scale feature extraction subunit is connected to an input end of the third detail capture convolutional layer; An output end of the first detail capture convolutional layer is connected to an input end of the self-attention module; An output end of the second detail capturing convolutional layer is connected to a connection operation module in the second fusion block; An output end of the third detail capturing convolutional layer is connected to a connection operation module in the first fusion block; An output end of the self-attention module is connected to an input end of the fourth detail capturing convolutional layer; An output end of the fourth detail capturing convolutional layer is connected to a connection operation module in the fourth fusion block; An output end of the fifth detail capture convolutional layer is connected to a connection operation module in the third fusion block; An output end of the multi-scale feature extraction module in the third fusion block, an output end of the multi-scale feature extraction module in the fourth fusion block, and an output end of the multi-scale feature extraction module cascaded with the fourth fusion block are respectively connected to the perceptual query selection unit.

4. The method according to claim 3, characterized in that The multi-scale feature extraction module includes a cascaded channel halving convolution layer, a tensor segmentation processing layer, a feature extraction fusion subnetwork, a context anchor attention layer, and a second convolution layer; The feature extraction fusion sub-network includes a first branch, a second branch and a first connection layer, the first branch and the second branch are connected in parallel and are respectively connected to the input end of the first connection layer; The first branch includes a cascade of a first feature extraction fusion submodule, a channel halving convolutional layer, a second feature extraction fusion submodule, and a channel halving convolutional layer; The first output end of the tensor segmentation processing layer is connected to the input end of the first connection layer, and the output end of the channel halving convolution layer in the first branch is connected to the input end of the first connection layer. In the second branch, an output end of the tensor segmentation processing layer is connected to an input end of the first connection layer.

5. The method according to claim 3, characterized in that: The first detail capture convolution layer, the second detail capture convolution layer, the third detail capture convolution layer, the fourth detail capture convolution layer, the fifth detail capture convolution layer, the sixth detail capture convolution layer, and the seventh detail capture convolution layer have the same structure; The first detail capture convolution layer includes a third convolution layer, a depth separation convolution layer, a second connection layer and a tensor shape transformation layer; The input end of the third convolutional layer is used to receive the input channel feature tensor, and the output end of the third convolutional layer is connected to the input end of the depth separation convolutional layer and the input end of the second connection layer respectively; An output end of the depth separation convolutional layer is connected to an input end of the second connection layer; An output terminal of the second connection layer is connected to an input terminal of the tensor shape transformation layer.

6. The method according to any one of claims 1 to 5, characterized in that The training steps of the preset snake target detection model include: Acquiring the at least one set of training data; Inputting the sample images in each group of training data into the snake target detection model to be trained respectively, and the snake target detection model to be trained outputs the corresponding prediction result of snake target detection; Using a preset comprehensive loss function, based on the labeled data in each set of training data and the corresponding prediction results, obtaining a function value of the comprehensive loss function; Based on the function value of the comprehensive loss function, the snake target detection model to be trained is trained until a preset training completion condition is met, and the preset snake target detection model is obtained from the snake target detection model to be trained.

7. The method according to claim 6, characterized in that Using a preset comprehensive loss function, based on the labeled data in each set of training data and the corresponding prediction results, obtaining a function value of the comprehensive loss function includes: Based on the labeled data in each set of training data and the corresponding prediction results, respectively calculating the function value of the strong intersection-over-union loss function and the function value of the focal loss function; Using preset weights, the function value of the strong intersection-over-union loss function and the function value of the focal loss function are weightedly fused to obtain the function value of the comprehensive loss function.

8. The method according to claim 7, characterized in that Based on the prediction results in each set of training data and the corresponding labeled data, a function value of a strong intersection-over-union loss function is calculated, including: The function value of the strong intersection-union loss function is calculated by the following calculation formulas (1) to (4); (1) (2) (3) (4) Wherein, e represents a natural base; A represents the predicted bounding box area determined based on the boundary position information of the snake target in the prediction result; B represents the real bounding box area determined based on the boundary position information of the snake target in the annotated data; represents the absolute value of the width distance between the predicted bounding box and the first edge corresponding to the true bounding box; represents the absolute value of the width distance between the predicted bounding box and the corresponding second edge of the true bounding box; represents the absolute value of the height distance between the predicted bounding box and the first edge corresponding to the real bounding box; represents the absolute value of the height distance between the predicted bounding box and the corresponding second edge of the true bounding box; represents the width of the real bounding box determined based on the boundary position information of the snake target in the annotated data; represents the height of the real bounding box determined based on the boundary position information of the snake target in the annotated data; Calculating the function value of the focal loss function based on the prediction results in each set of training data and the corresponding labeled data, including: The function value of the focal loss function is calculated by the following calculation formulas (5) to (6): (5) (6) in, Indicates t The class prediction probability in the prediction results corresponding to the group training data; Indicates t The weight factor corresponding to the group training data; represents the modulation factor; Indicates t The predicted probability of the positive category in the prediction results corresponding to the group training data; Indicates t The category information in the labeled data corresponding to the group training data; Using preset weights, the function values ​​of the strong intersection-over-union loss function and the function values ​​of the focal loss function are weightedly fused to obtain the function value of the comprehensive loss function, including: The function value of the comprehensive loss function is obtained by the following calculation formula (7); (7) in, represents the weight coefficient of the strong intersection-over-union loss function; Represents the weight coefficient of the focal loss function.

9. The method according to claim 6, characterized in that Based on the function value of the comprehensive loss function, the to-be-trained snake target detection model is trained until a preset training completion condition is met, and the preset snake target detection model is obtained from the to-be-trained snake target detection model, including: The function value of the comprehensive loss function is transmitted back to the snake target detection model to be trained; The snake target detection model to be trained adjusts the network parameters of each network layer according to the function value of the comprehensive loss function; Iterate the steps of obtaining the function value of the comprehensive loss function to adjusting the network parameters of each network layer until a preset training completion condition is met.

10. A device for detecting snake targets in an image, characterized in that: include: The image acquisition module is configured to: acquire a target image; wherein the target image is an image that needs to be subjected to snake target detection; The target detection module is configured to: input the target image into a preset snake target detection model, and output a snake target detection result about the target image through the preset snake target detection model; Among them, the preset snake target detection model is trained based on at least one set of training data, each set of training data in the at least one set of training data includes a sample image and corresponding annotation data, and the annotation data includes category information and location information of the snake target annotated in the sample image.

Citation Information

Patent Citations

  • Lightweight road defect detection method based on dynamic deformable attention mechanism

    CN118279562A

  • General target detection method for adaptive attention guidance mechanism

    WO2021139069A1

Cited By

  • Cold-rolled strip steel edge defect detection method and system

    CN122048942A