Target detection method, device and storage medium

Abstract

The present application discloses a target detection method, device and storage medium, belonging to the field of image detection technology. In an embodiment of the present application, a target detection method includes: obtaining an original image for target detection; inputting the obtained original image into a pre-built image segmentation model to obtain a segmentation mask of the original image as a true label of a feature encoding network; inputting the original image into a feature encoding network of a target detector to obtain a relevant template that highlights the key area and semantic information of the target; inputting the relevant template into a feature decoding network of the target detector to obtain an initial template; optimizing the parameters of the feature decoding network of the target detector by comparing the original image with the initial template; and using the optimized target detector for target detection. The present application can actively learn and optimize the relevant templates, and emphasize and highlight the key information in the image, thereby significantly improving the performance of target detection.

Description

Technical Field

[0001] The present application belongs to the field of image detection technology, and specifically relates to a target detection method, device and storage medium. Background Art

[0002] In the field of two-dimensional object detection, existing technologies generally adopt a direct and passive method to process input images. The core of these methods is to conduct a detailed analysis of the input image in order to extract features closely related to object detection, and then determine whether there is a specific object in the image based on these features. However, this direct image processing method often fails to meet actual detection requirements when faced with complex and changing image scenes and carefully disguised images. Summary of the invention

[0003] Purpose of the invention: An embodiment of the present application provides a target detection method, aiming to overcome the technical problem that current detection methods have poor detection performance when faced with complex and changeable image scenes and carefully disguised images; another purpose of an embodiment of the present application is to provide a target detection device.

[0004] Technical solution: A target detection method disclosed in an embodiment of the present application includes:

[0005] Get the original image of the detection target;

[0006] Inputting the acquired original image into a pre-built image segmentation model to obtain a segmentation mask of the original image as a true label of a feature encoding network;

[0007] Inputting the original image into a feature encoding network of an object detector to obtain a template related to highlighting key target areas and semantic information;

[0008] Inputting the relevant template into a feature decoding network of a target detector to obtain an initial template;

[0009] By comparing the original image with the initial template, optimizing the parameters of the feature decoding network of the target detector;

[0010] The optimized target detector is used for target detection.

[0011] In some embodiments, the object detector is fine-tuned using the encrypted original image.

[0012] In some embodiments, the segmentation mask includes the shape, texture, position and size information of the object in the image.

[0013] In some embodiments, the step of inputting the original image into a feature encoding network of a target detector to obtain a template related to highlighting key target areas and semantic information includes:

[0014] Input the original image into the convolution layer to extract local features of the image;

[0015] The extracted local features of the image are input into the pooling layer for feature sampling.

[0016] In some embodiments, optimizing the parameters of the feature decoding network of the object detector by comparing the original image with the initial template includes:

[0017] Determining a gradient of a loss function based on a difference or similarity between the original image and the initial template;

[0018] Based on the gradient of the loss function, a back propagation algorithm and a gradient descent method are used to update the parameters of the feature encoding network of the target detector.

[0019] In some embodiments, the feature encoding network of the target detector adopts a convolutional neural network architecture, and the loss function formula of the feature encoding network is as follows:

[0020] ;

[0021] Among them, J E is the feature encoding network loss, ε is the parameter of the feature encoding network, G is the segmentation mask, and S is the related template;

[0022] The encrypted image formula is as follows:

[0023] ;

[0024] Among them, I is the input original image and S is the related template.

[0025] In some embodiments, the feature decoding network of the target detector adopts a convolutional neural network architecture, and the loss function formula of the feature decoding network is as follows:

[0026] ;

[0027] Among them, J D is the feature decoding network loss, S is the relevant template, is the initial template for the relevant template, I is the original image, Parameters of the feature decoding network.

[0028] The present application also discloses a target detection device, comprising:

[0029] An acquisition module, wherein the acquisition module is configured to acquire an original image of the detection target;

[0030] A first processing module, wherein the first processing module is configured to input the acquired original image into a pre-built image segmentation model to obtain a segmentation mask of the original image;

[0031] A second processing module, wherein the second processing module is configured to input the original image into a feature encoding network of a target detector to obtain a template related to highlighting key target areas and semantic information;

[0032] A third processing module, wherein the third processing module is configured to input the relevant template into a feature decoding network of a target detector to obtain an initial template;

[0033] An optimization module, wherein the optimization module is configured to optimize parameters of a feature encoding network and a feature decoding network of the target detector by comparing the original image with the initial template;

[0034] A detection module is configured to perform target detection using the optimized target detector.

[0035] The present application also discloses an electronic device, comprising:

[0036] at least one processor; and

[0037] a memory communicatively coupled to the at least one processor;

[0038] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the target detection method described in the above embodiment.

[0039] The present application also discloses a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the target detection method described in the above embodiment.

[0040] Beneficial effect: A target detection method of an embodiment of the present application includes: obtaining an original image for target detection; inputting the obtained original image into a pre-built image segmentation model to obtain a segmentation mask of the original image; inputting the original image into a feature encoding network of a target detector to obtain a relevant template that highlights the key area and semantic information of the target; inputting the encrypted image into a feature decoding network of a target detector to obtain an initial template; optimizing the parameters of the feature decoding network of the target detector by comparing the original image with the initial template; and using the optimized target detector for target detection. The original image is segmented by an image segmentation model, and the boundaries and regions of each target in the original image are accurately extracted, and a high-quality segmentation mask is generated as a true label of the feature encoding network. The feature encoding network actively learns and generates relevant templates closely related to the content of the original image to encrypt the original image, and emphasizes and highlights the key image information in the original image. The relevant template is converted into an initial template by a feature decoding network, and the effect of the encryption process is evaluated by comparing the original image with the initial template, and the parameters of the feature encoding network are optimized accordingly. The active strategy enables the present invention to better adapt to different image scenes and conditions, thereby improving the accuracy and robustness of target detection.

[0041] The target detection device of the embodiment of the present application includes the target detection method described above, and therefore can have all the technical features and technical effects of the above target detection method, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0043] Figure 1 Schematic diagram of the working process of the target detector of this application, in which SAM is an image segmentation model, ε is a feature encoding network, is the parameter of the feature decoding network, o is the target detector, J D is the loss function of the feature decoding network, J E is the loss function of the feature encoding network;

[0044] Figure 2 A flowchart of a target detection method for this application;

[0045] Figure 3 This is a schematic diagram of the structure of a target detection device of the present application. DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0047] It should be understood that when used in this specification and the appended claims, the terms "comprise" and "include" indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their collections. It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, the singular forms "a", "an" and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0048] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0049] As a preface to the embodiments of the present application, target detectors are mostly based on deep learning models, especially convolutional neural networks (CNNs). These models can learn rich features in images through hierarchical structures and a large amount of training data, thereby achieving effective detection of targets. However, these models often face the limitation of convergence of the global minimum of neural networks during training. The global minimum is the optimal solution that the model can achieve during training, but due to various factors such as the complexity of the model, the diversity of training data, and the choice of optimization algorithm, the global minimum cannot always be found accurately, or the solution found may not be the optimal solution in the true sense. This results in the weights obtained by existing target detectors during training may not be optimal, thereby limiting their detection performance to a certain extent.

[0050] In addition, the existing technologies generally lack the ability to actively emphasize and highlight key information in the input image. In complex image scenes, there is often a large amount of redundant information and noise, which can seriously interfere with the judgment of the target detector. However, the existing technologies usually only process these images passively, lacking an effective method to actively highlight and emphasize key information in the image, such as the boundary, shape, texture, etc. of the target. This passive processing method makes the target detector susceptible to interference from redundant information and noise when identifying the target, thereby reducing its detection performance.

[0051] In view of this, an embodiment of the present application provides a target detection method, aiming to solve at least one of the above technical problems.

[0052] See also Figures 1 to 3 As shown, a target detection method of an embodiment of the present application includes: obtaining an original image of a detection target; inputting the obtained original image into a pre-built image segmentation model (SegmentAnythingModel, SAM) to obtain a segmentation mask of the original image as a true label of a feature encoding network; inputting the original image into the feature encoding network of a target detector to obtain a relevant template that highlights the key area and semantic information of the target; inputting the relevant template into the feature decoding network of the target detector to obtain an initial template; optimizing the parameters of the feature decoding network of the target detector by comparing the original image with the initial template; and using the optimized target detector for target detection. The original image is segmented by an image segmentation model, and the boundaries and regions of each target in the original image are accurately extracted to generate a high-quality segmentation mask, and the relevant template closely related to the content of the initial template is actively learned and generated by the feature encoding network to encrypt the initial template, and the key image information in the initial template is emphasized and highlighted, and the relevant template is converted into the initial template by the feature decoding network, and the effect of the encryption process is evaluated by comparing the original image with the initial template, and the parameters of the feature encoding network are optimized accordingly. The active strategy enables the present invention to better adapt to different image scenes and conditions, thereby improving the accuracy and robustness of target detection.

[0053] It is important to understand that the image segmentation model can deeply explore every pixel in the original image, perform detailed and precise segmentation based on the similarities and differences between pixels, extract the boundaries and regions of each target in the image, and generate high-quality segmentation masks. These segmentation masks not only contain key feature information such as the shape and texture of the target, but also accurately reflect the position and size of the target in the image. Through the segmentation mask, it is easy to identify the target pixels and background pixels in the original image, and further extract the shape, texture and other features of the target, thereby providing a rich basis for the target detection algorithm. For background pixels, you can choose to ignore or perform other processing to improve the accuracy and efficiency of target detection.

[0054] In some embodiments, the encrypted images are used to fine-tune the object detector. Since the encryption process has emphasized the key information in the image, fine-tuning the object detector using these encrypted images can make the object detector pay more attention to the key information in the encrypted image, prompting the object detector to capture and identify this information more keenly, thereby improving its detection performance in open-world images. The object detector has been exposed to a variety of image environments and target types during training. This allows it to adapt to a wider range of scenarios and show greater robustness and stability in practical applications.

[0055] In some embodiments, the segmentation mask is used as the true label of the feature encoding network, and the original image is input into the feature encoding network of the target detector to obtain a relevant template that highlights the key area of ​​the target and the semantic information, including: inputting the original image into the convolution layer to extract the local features of the image; inputting the extracted local features of the image into the pooling layer for feature sampling. By adopting a combination of multiple convolutional layers and pooling layers, hierarchical features from local to global are gradually extracted from the original image. Specifically, the convolution layer is responsible for capturing and extracting features such as edges, corners, etc. in the local area of ​​the image. As the number of network layers increases, these local features will be gradually combined into higher-level features, such as shape, texture, etc. The pooling layer is responsible for downsampling these features to reduce the number of parameters and computational complexity in the network, while also helping to improve the robustness of the target detector and prevent overfitting.

[0056] It should be understood that in the process of training the feature encoding network, a large number of images containing rich target information and complex background environments are used as input to the feature encoding network. After being processed by the convolution layer and the pooling layer, they will be converted into relevant templates associated with the image content. Through continuous training and optimization, the feature encoding network can gradually learn how to extract key information related to target detection from the input image and generate high-quality relevant templates, which can help the target detector to more accurately and efficiently identify and locate targets in the image.

[0057] In some embodiments, the parameters of the feature decoding network of the target detector are optimized by comparing the original image with the initial template, including: determining the gradient of the loss function based on the difference or similarity between the original image and the initial template; based on the gradient of the loss function, using the back propagation algorithm and the gradient descent method to update the parameters of the feature encoding network of the target detector. The above process is iterative, and through multiple iterations, the value of the loss function can be gradually reduced, thereby optimizing the performance of the feature encoding network, so that the encryption image capability of the feature encoding network is optimal. In some embodiments, the feature encoding network of the target detector adopts a convolutional neural network architecture, and the loss function formula of the feature encoding network is as follows:

[0058] ;

[0059] Among them, J E is the feature encoding network loss, ε is the parameter of the feature encoding network, G is the segmentation mask, and S is the related template;

[0060] The encrypted image formula is as follows:

[0061] ;

[0062] Among them, I is the input original image and S is the related template.

[0063] In some embodiments, the feature encoding network of the target detector adopts a convolutional neural network architecture, and the loss function formula of the feature decoding network is as follows:

[0064] ;

[0065] Among them, J D is the feature decoding network loss, S is the relevant template, is the initial template, I is the original image, is the parameter of the feature decoding network. It should be understood that during the training of the feature decoding network, the feature decoding network is responsible for accurately restoring the initial template from the relevant template; when the feature decoding network receives the encrypted image (relevant template) as input, it begins to gradually restore the information in the relevant template, especially those closely related to the original image. This process usually involves multiple deconvolution layers, upsampling layers, and possible convolution layers. Through the collaborative work of deconvolution layers, upsampling layers, and possible convolution layers, the feature decoding network can gradually restore the image resolution and details in the relevant template while retaining key information related to the original image.

[0066] An embodiment of the present application also provides a target detection device, including: an acquisition module, the acquisition module is configured to acquire an original image for target detection; a first processing module, the first processing module is configured to input the acquired original image into a pre-built image segmentation model to obtain a segmentation mask of the original image as a true label of a feature encoding network; a second processing module, the second processing module is configured to input the original image into a feature encoding network of a target detector to obtain a relevant template that highlights key target areas and semantic information; a third processing module, the third processing module is configured to input the encrypted image into a feature decoding network of a target detector to obtain an initial template; an optimization module, the optimization module is configured to optimize the parameters of a feature encoding network and a feature decoding network of a target detector by comparing the original image with the initial template; and a detection module, the detection module is configured to use an optimized target detector for target detection.

[0067] An embodiment of the present application also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the target detection method described in the above embodiment.

[0068] An embodiment of the present application also provides a non-transitory computer-readable storage medium storing computer instructions, where the computer instructions are used to enable a computer to execute the target detection method described in the above embodiment.

[0069] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0070] The above is a detailed introduction to a target detection method, device and storage medium provided in the embodiments of the present application, and the principles and implementation methods of the present application are explained by using specific examples. The description of the above embodiments is only used to help understand the technical solution and core idea of ​​the present application; ordinary technicians in this field should understand that they can still modify the technical solutions recorded in the aforementioned embodiments, or replace some of the technical features therein with equivalents; and these modifications or replacements do not make the essence of the corresponding technical solution deviate from the scope of the technical solution of the embodiments of the present application.

Claims

1. A target detection method, characterized in that: include: Get the original image of the detection target; Inputting the acquired original image into a pre-built image segmentation model to obtain a segmentation mask of the original image as a true label of a feature encoding network; Inputting the original image into a feature encoding network of an object detector to obtain a template related to highlighting key target areas and semantic information; The feature encoding network of the target detector adopts a convolutional neural network architecture, and the loss function formula of the feature encoding network is as follows: ; Among them, J E is the feature encoding network loss, ε is the parameter of the feature encoding network, G is the segmentation mask, and S is the related template; The encrypted image formula is as follows: ; Among them, I is the input original image, S is the relevant template; Input the encrypted image into the feature decoding network of the target detector to obtain the initial template; The feature decoding network of the target detector adopts a convolutional neural network architecture, and the loss function formula of the feature decoding network is as follows: ; Among them, J D is the feature decoding network loss, S is the relevant template, is the initial template, I is the original image, Parameters of the feature decoding network; Optimizing a feature encoding network and a feature decoding network of the target detector; The optimized target detector is used for target detection.

2. The target detection method according to claim 1, characterized in that: The segmentation mask includes the shape, texture, position and size information of the target in the image.

3. The target detection method according to claim 1, characterized in that: The step of inputting the original image into a feature encoding network of a target detector to obtain a template related to highlighting key target areas and semantic information includes: Input the original image into the convolution layer to extract local features of the image; The extracted local features of the image are input into the pooling layer for feature sampling.

4. A target detection device, characterized in that: The method for executing the target detection method according to any one of claims 1 to 3 comprises: An acquisition module, wherein the acquisition module is configured to acquire an original image of the detection target; A first processing module, wherein the first processing module is configured to input the acquired original image into a pre-built image segmentation model to obtain a segmentation mask of the original image; A second processing module, wherein the second processing module is configured to input the original image into a feature encoding network of a target detector to obtain a template related to highlighting key target areas and semantic information; A third processing module, wherein the third processing module is configured to input the encrypted image into a feature decoding network of a target detector to obtain an initial template; An optimization module, wherein the optimization module is configured to optimize a feature decoding network and an encoding network of the target detector; A detection module is configured to perform target detection using the optimized target detector.

5. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the target detection method according to any one of claims 1 to 3.

6. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to enable a computer to execute the target detection method according to any one of claims 1-3.

Images