Target detection network training method, target detection method and related device

Abstract

The present application discloses a training method for a target detection network, a target detection method and related devices, and relates to the technical field of weakly supervised target detection in computer vision. A training data set including an original image is obtained and input into a target detection network including a classifier and a locator, and a self-attention map is generated to generate an enhanced mask image. The original image is input into the locator to obtain a predicted activation image, and a first loss value with the mask image is calculated. Then the bounding box is obtained and the mask image is sampled to obtain an enhanced image and input into the classifier to obtain a predicted classification result, and a second loss value is generated by calculating the classification label with the original image. The model weights of the target detection network are adjusted according to the first loss value and the second loss value until the iteration termination condition is reached to obtain the target detection network. In this way, a unified network architecture of the classifier and the locator is designed, and the accuracy and performance of positioning and classification are improved through the self-attention map, the mask image and the enhanced image.

Description

Technical Field

[0001] The present application relates to the technical field of weakly supervised target detection in computer vision, and in particular to a target detection network training method, a target detection method, and related devices. Background Art

[0002] Weakly supervised object detection aims to achieve effective localization and classification simultaneously using only image-level supervision without bounding box annotations. Due to its low annotation cost, weakly supervised object detection has attracted increasing attention in the research community. However, related techniques have primarily prioritized localization performance over classification, resulting in an imbalance in the development of these two tasks and negatively impacting the performance of weakly supervised localization and classification, particularly the top-1 localization accuracy.

[0003] Top-1 localization is considered accurate only when both the top-1 classification and the localization are accurate. Therefore, reducing classification accuracy indirectly reduces top-1 localization accuracy. While related technologies can improve both classification and localization accuracy by designing two independent classification and localization models, this increases computational overhead. Summary of the Invention

[0004] The present application aims to solve at least one of the technical problems existing in the prior art. To this end, the embodiments of the present application provide a training method for a target detection network, a target detection method, and related devices, which can enable a classifier and a locator to be in a unified network architecture without affecting each other, and effectively improve the accuracy and performance of positioning and classification at the same time.

[0005] In a first aspect, an embodiment of the present application provides a method for training an object detection network, comprising:

[0006] Obtain a training data set; the training data set includes a plurality of original images, and the original images include classification labels;

[0007] Inputting the original image into a target detection network; the target detection network includes a classifier and a locator;

[0008] Generate a self-attention map of the original image using a preset supervision model, and generate a mask image based on the self-attention map and the original image;

[0009] Inputting the original image into the locator to perform target positioning to obtain a predicted activation image, and generating a first loss value according to the predicted activation image and the mask image;

[0010] Obtaining a bounding box, and sampling the mask image according to the bounding box to obtain an enhanced image;

[0011] Inputting the enhanced image into the classifier for target classification to obtain a predicted classification result, and generating a second loss value based on the predicted classification result and the classification label;

[0012] The model weights of the target detection network are adjusted according to the first loss value and the second loss value until an iteration termination condition is reached, thereby obtaining the trained target detection network.

[0013] In some embodiments of the present application, the supervised model includes a Transformer encoder having a preset number of attention heads; and generating a self-attention map of the original image using the preset supervised model includes:

[0014] Inputting the original image into the supervised model to obtain an attention tensor; the attention tensor has a first preset dimension, and the first preset dimension corresponds to the attention head;

[0015] Selecting a second preset dimension based on the first preset dimension;

[0016] An attention value of the second preset dimension is selected from the attention tensor, and the self-attention map is calculated based on the attention value.

[0017] In some embodiments of the present application, the self-attention map includes multiple activation values; and generating a mask image according to the self-attention map and the original image includes:

[0018] Obtaining foreground pixels and background pixels of the original image according to the self-attention map;

[0019] Creating a Gaussian mixture model based on the foreground pixels and the background pixels;

[0020] Obtaining a response threshold range, and segmenting the self-attention map into a foreground region, a background region, and a candidate region according to the response threshold range and the activation value;

[0021] Based on the low-level features of the original image, using the Gaussian mixture model to calculate the segmentation probability of each pixel in the candidate area, and dividing the pixel into the foreground area or the background area according to the segmentation probability;

[0022] The mask image is obtained according to the foreground area and the background area.

[0023] In some embodiments of the present application, the target detection network further includes a backbone network, the locator includes a UNet network, a three-layer deconvolutional neural network, and a normalization network; and inputting the original image into the locator for target positioning to obtain a predicted activation image includes:

[0024] Inputting the original image into the backbone network to obtain a first feature map;

[0025] Inputting the first feature map into the UNet network to obtain a second feature map;

[0026] Inputting the second feature map into the deconvolution neural network to obtain a third feature map;

[0027] The third feature map is input into the normalized network to obtain the predicted activation map.

[0028] In some embodiments of the present application, obtaining a bounding box and sampling the mask image according to the bounding box to obtain an enhanced image includes:

[0029] uniformly sampling within a plurality of candidate bounding boxes to obtain the bounding box;

[0030] Randomly sampling the mask image using the bounding box to obtain a sampling area;

[0031] Masking the sampling area using a preset filling value to obtain an enhanced mask image;

[0032] Performing a dot product between the original image and the enhanced mask image to obtain the enhanced image.

[0033] In some embodiments of the present application, the classifier includes a Transformer encoder, a two-layer convolutional neural network, and a global pooling layer; inputting the enhanced image into the classifier for target classification to obtain a predicted classification result includes:

[0034] Inputting the enhanced image into the classifier to obtain a training image sequence;

[0035] The training image sequence is input into the convolutional neural network and the global pooling layer to obtain the predicted classification result.

[0036] In a second aspect, an embodiment of the present application further provides a target detection method, applying the target detection network training method described in the embodiment of the first aspect of the present application, including:

[0037] Acquire an image to be detected and input it into the target detection network; wherein the target detection network includes a pre-trained backbone network, a locator and a classifier, and the backbone network is connected to the locator and the classifier respectively;

[0038] Generating a feature map corresponding to the image to be detected using the backbone network;

[0039] The feature map is input into the locator to obtain a positioning result, and the feature map is input into the classifier to obtain a classification result.

[0040] In a third aspect, an embodiment of the present application further provides a training device for a target detection network, applying the training method for a target detection network as described in the embodiment of the first aspect of the present application, including:

[0041] An acquisition module, configured to acquire a training data set; wherein the training data set includes a plurality of original images, and the original images include classification labels;

[0042] An input module, configured to input the original image into a target detection network; the target detection network includes a classifier and a locator;

[0043] A generation module, configured to generate a self-attention map of the original image using a preset supervision model, and generate a mask image based on the self-attention map and the original image;

[0044] a first training module, configured to input the original image into the locator for target positioning to obtain a predicted activation image, and generate a first loss value according to the predicted activation image and the mask image;

[0045] an enhancement module, configured to obtain a bounding box and sample the mask image according to the bounding box to obtain an enhanced image;

[0046] A second training module is configured to input the enhanced image into the classifier for target classification to obtain a predicted classification result, and generate a second loss value based on the predicted classification result and the classification label;

[0047] An adjustment module is used to adjust the model weights of the target detection network according to the first loss value and the second loss value until the iteration termination condition is reached to obtain the trained target detection network.

[0048] In a fourth aspect, an embodiment of the present application further provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the target detection network training method as described in the embodiment of the first aspect of the present application or the target detection method as described in the embodiment of the second aspect of the present application.

[0049] In a fifth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores a program, and the program is executed by a processor to implement the target detection network training method as described in the embodiment of the first aspect of the present application or the target detection method as described in the embodiment of the second aspect of the present application.

[0050] The embodiments of the present application include at least the following beneficial effects:

[0051] Embodiments of the present application provide a method for training a target detection network, a target detection method, and related devices. The target detection network training method involves obtaining a training dataset comprising multiple original images, which is then input into a target detection network comprising a classifier and a locator. A self-attention map corresponding to the original image is generated using a preset supervised model, and a mask image is further generated based on the self-attention map and the original image. The original image is input into the locator for target localization, generating a predicted activation image. A first loss value is then generated based on the predicted activation image and the mask image. A bounding box is then obtained and the mask image is sampled to obtain an enhanced image. The enhanced image is input into a classifier for target classification, obtaining a predicted classification result. A second loss value is then generated based on the predicted classification result and the classification label of the original image. Finally, the model weights of the target detection network are adjusted based on the first and second loss values ​​until an iteration termination condition is met, thereby obtaining a trained target detection network. This design utilizes a unified network architecture for the classifier and locator. By utilizing the self-attention map, mask image, and enhanced image, the accuracy and performance of target localization and classification can be effectively improved, resulting in highly accurate localization and classification results.

[0052] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become obvious from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0054] Figure 1 This is a flowchart of a method for training an object detection network provided by one embodiment of the present application;

[0055] Figure 2 yes Figure 1 Flow diagram of step S103;

[0056] Figure 3 yes Figure 1 Another flow chart of step S103;

[0057] Figure 4 yes Figure 1 Flow chart of step S104;

[0058] Figure 5 yes Figure 1 Flow diagram of step S105;

[0059] Figure 6 This is a schematic diagram of random sampling provided by an embodiment of the present application;

[0060] Figure 7 yes Figure 1 Flow chart of step S106;

[0061] Figure 8 This is a diagram of the training process architecture of the target detection network provided by one embodiment of the present application;

[0062] Figure 9 This is a schematic diagram of low-level feature enhancement provided by an embodiment of the present application;

[0063] Figure 10 This is a comparison chart of visualization results provided by an embodiment of the present application;

[0064] Figure 11 This is a comparison chart of positioning effects on the CUB-200-2011 dataset provided by an embodiment of the present application;

[0065] Figure 12 This is a comparison chart of classification results on the CUB-200-2011 dataset provided by an embodiment of the present application;

[0066] Figure 13 This is a comparison chart of positioning effects on the ILSVRC 2012 dataset provided by an embodiment of the present application;

[0067] Figure 14 This is a comparison chart of classification results on the ILSVRC 2012 dataset provided by an embodiment of the present application;

[0068] Figure 15 This is a flowchart of a target detection method provided by an embodiment of the present application;

[0069] Figure 16 This is a flow chart of the target detection method provided by one embodiment of the present application;

[0070] Figure 17 This is a schematic diagram of a training device module for an object detection network provided by one embodiment of the present application;

[0071] Figure 18 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application.

[0072] Reference numerals: acquisition module 100 , input module 200 , generation module 300 , first training module 400 , enhancement module 500 , second training module 600 , adjustment module 700 , electronic device 1000 , processor 1001 , memory 1002 . DETAILED DESCRIPTION

[0073] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0074] The following describes in detail embodiments of the present application. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application and are not to be construed as limiting the present application.

[0075] In the description of this application, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, they cannot be understood as limitations on this application.

[0076] In the description of this application, "several" means one or more, "many" means more than two, "greater than," "less than," and "exceed" are understood to exclude the number itself, while "above," "below," and "within" are understood to include the number itself. The terms "first" and "second" are used solely to distinguish technical features and are not to be construed as indicating or implying relative importance, or as implicitly specifying the number or order of the technical features indicated.

[0077] In the description of this application, unless otherwise clearly defined, terms such as setting, installing, and connecting should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in this application based on the specific content of the technical solution.

[0078] Weakly supervised object detection aims to achieve effective localization and classification simultaneously using only image-level supervision without bounding box annotations. Due to its low annotation cost, weakly supervised object detection has attracted increasing attention in the research community. Related technologies primarily prioritize localization performance over classification, resulting in an imbalance in the development of these two tasks. This negatively impacts the performance of weakly supervised localization and classification, particularly top-1 localization accuracy. Top-1 localization is considered accurate only when both the top-1 classification and pure localization are accurate. Therefore, reducing classification accuracy indirectly reduces top-1 localization accuracy.

[0079] For example, in "Weakly Supervised Object Localization via Transformer with Implicit Spatial Calibration", Bai et al. introduced a spatial calibration module for precise positioning, sacrificing classification accuracy and resulting in a decrease in Top-1 positioning. Meanwhile, in "Contrastive learning of Class-agnostic Activation Map for Weakly Supervised Object Localization and Semantic Segmentation", Xie et al. emphasized improving classification and localization accuracy, achieving competitive accuracy in Top-1 positioning. However, this approach relies on two independent classification and localization models, increasing computational overhead. Therefore, the challenge facing weakly supervised object detection is how to unify classification and localization into a single framework and improve both tasks simultaneously.

[0080] Class Activation Maps (CAMs) are the most representative approach for weakly supervised object detection. Zhou et al. first proposed a classification-based indirect object localization method in "Learning Deep Features for Discriminative Localization." This method trains a convolutional neural network (CNN), multiplies the weights of the last fully connected layer of the network model by the corresponding feature map from the classification network, and normalizes the result to generate an object localization map, also known as a Class Activation Map (CAM). The CAM reflects the object location that the network focuses on based on the given classification information, but it only highlights the most discriminative regions and ignores the full context of the object. To address this issue, several CAM-based techniques have been proposed, including adversarial erasure, spatial relation activation, and low-level semantic feature activation (SPOL). Compared to the first two methods, SPOL demonstrates stronger performance. SPOL emphasizes the importance of low-level features, which contain richer global information and can help achieve more global localization. However, it also activates shallow background noise, hindering global localization. Therefore, it is necessary to find lower-level image information with less noise to improve localization and classification performance.

[0081] In recent years, the Transformer encoder structure has been applied to extract features for image recognition in computer vision. It applies the Transformer's self-attention mechanism to images by dividing the image into small blocks and treating them as "words" that can be processed by the Transformer encoder. The advantage of the self-attention mechanism is that it can take into account the interrelationships between all elements in the input sequence, not just local relationships. This enables the Transformer to learn a global image representation and use this global representation to generate a self-attention activation map similar to a CAM. In "LCTR: On Awakening the Local Continuity of Transformer for Weakly Supervised Object Localization", Chen et al. multiplied the self-attention weights of the Transformer classification with the corresponding feature map to obtain the target localization map. In the paper "Emerging Properties in Self-Supervised Vision Transformers", Caron et al. proposed a self-supervised Transformer model called DINO, which uses the self-attention mechanism to generate an attention map that covers more object context. In "Proxy Probing Decoder for WeaklySupervised Object Localization: A Baseline Investigation", Xu et al. further proposed a proxy probing decoder that uses DINO self-attention maps as pseudo-supervision to promote localization, significantly alleviating the problem of partial target activation in localization. However, it ignores the improvement of classification performance, resulting in relatively low Top-1 localization accuracy even with relatively good localization performance.

[0082] Based on this, the embodiments of the present application provide a target detection network training method, a target detection method and related devices, which can enable the classifier and the locator to be in a unified network architecture without affecting each other, and through self-attention maps, mask images and enhanced images, the accuracy and performance of target positioning and classification can be effectively improved, and high-precision positioning and classification results can be obtained.

[0083] The embodiments of the present application provide a training method for a target detection network, a target detection method, and related devices, which are specifically illustrated by the following embodiments. First, the training method for the target detection network in the embodiments of the present application is described.

[0084] The training method of the target detection network provided in the embodiment of the present application relates to the field of computer vision technology, and in particular to the technical field of weakly supervised target detection in computer vision. The training method of the target detection network provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be a computer program running in a terminal or a server side. For example, a computer program can be a native program or a software module in an operating system; it can be a local (Native) application (APP, Application), that is, a program that needs to be installed in the operating system to run, such as a client that supports the training of the target detection network, that is, a program that can be run only by downloading it to a browser environment. In short, the above-mentioned computer program can be an application, module or plug-in in any form. Wherein, the terminal communicates with the server via a network. The training method of the target detection network can be executed by a terminal or a server, or performed in collaboration with a terminal and a server.

[0085] In some embodiments, the terminal can be a smartphone, tablet computer, laptop computer, desktop computer or smartwatch, etc. The server can be an independent server, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms; it can also be a service node in a blockchain system, where each service node in the blockchain system forms a peer-to-peer (P2P) network. The P2P protocol is an application layer protocol running on the Transmission Control Protocol (TCP). The server can be installed with a server for the training system of the target detection network, through which the terminal can interact, for example, the server can be installed with corresponding software, and the software can be an application for implementing the training method of the target detection network, etc., but is not limited to the above forms. The terminal and the server can be connected via Bluetooth, USB (Universal Serial Bus) or network communication connection methods, which are not limited in this embodiment.

[0086] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention can also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0087] The following describes a method for training an object detection network according to an embodiment of the present invention.

[0088] Reference Figure 1 As shown, an embodiment of the present application provides a method for training a target detection network, including but not limited to the following steps S101 to S107.

[0089] Step S101: Obtain a training data set.

[0090] In some embodiments, the training dataset includes multiple original images, each of which includes a classification label. The classification label is an identifier or category name used to indicate the category to which the original image belongs. For example, for an original image of an animal category, the classification label may be "cat," "dog," "bird," etc. For another example, for an original image of a fruit category, the classification label may be "apple," "banana," "strawberry," etc., although this embodiment does not impose any limitation on this.

[0091] Step S102: input the original image into the target detection network.

[0092] In some embodiments, the object detection network includes a classifier and a localizer, and each original image in the training dataset is used to train the classifier and localizer in the object detection network. Specifically, weakly supervised object detection includes classification and localization tasks. The classification task is used to determine whether a specific class of object exists in an image, while the localization task is to generate candidate regions that may contain the object.

[0093] For example, in a weakly supervised object detection problem involving dogs and cats, the classification task is to determine whether a dog or cat is present in an image. Through this classification task, the object detection network can learn to recognize objects of different categories and distinguish them from other objects. The localization task, on the other hand, involves determining the approximate location or bounding box of the target object in an image. Since only image-level label information is available, it is impossible to obtain a precise bounding box annotation for the target. Therefore, the goal of the localization task is to generate a candidate region containing the target.

[0094] Step S103: Generate a self-attention map of the original image using a preset supervision model, and generate a mask image based on the self-attention map and the original image.

[0095] In some embodiments, a self-attention map corresponding to the original image is generated using a preset supervised model. It is understood that a self-attention map is an image representation generated by calculating the degree of correlation between each pixel in the image and other pixels. A self-attention map can reflect the interdependencies between different regions in the image. In weakly supervised object detection, a self-attention map is used to indicate regions of the image that may contain objects or regions of interest.

[0096] It should be understood that the preset supervision model in this embodiment can be the DINO model of the Transformer architecture, or the image classification VIT model of the Transformer architecture, or the ResNet model of the deep convolutional neural network structure, etc. Those skilled in the art can set it according to actual task requirements and computing resources and other factors, and this embodiment does not limit this.

[0097] In some embodiments, a mask image is generated based on the self-attention map and the original image. The mask image is an auxiliary image used for image processing and computer vision tasks. The mask image is a binary image of the same size as the original image, in which the pixel value is used to indicate whether the pixel should be considered or excluded. The pixels in the mask image are encoded in different ways, such as black and white pixels or grayscale pixels, so that the region of interest can be selected, the foreground and background can be distinguished, specific areas can be masked or hidden, etc. Therefore, the mask image can be used to perform local operations on the original image or extract information from a specific area. For example, there is a mask image representing the position of a bird, in which the pixel value of the bird part is 1 and the other parts are 0. In this way, the bird part in the original image is extracted, and the other areas are masked.

[0098] In some embodiments, the mask image is jointly generated by the self-attention map and the original image. Since the high-attention area contains more detailed information, while the low-attention area is the opposite, the embodiment of the present application can highlight or emphasize the high-attention area in the self-attention map and suppress the low-attention area. The mask image obtained in this way is clearer and more comprehensive than the mask image generated directly based on the original image or the self-attention image.

[0099] Step S104: input the original image into a locator to perform target positioning to obtain a predicted activation image, and generate a first loss value according to the predicted activation image and the mask image.

[0100] In some embodiments, the original image is input into the locator in the target detection network for target positioning, and a predicted activation image is obtained. The activation image represents the area where the locator locates the target on the original image, specifically by analyzing the feature responses of the locator at different positions in the image, thereby inferring the area that may contain the target. Among them, the pixel value of the activation image represents the importance or confidence of the corresponding position in the target positioning task. A higher pixel value generally indicates that the area is more likely to contain the target. Therefore, the activation image plays a role in positioning and visualization in weakly supervised target detection. By observing the activation image, it is possible to infer the degree of attention of the target detection network to different areas and find areas that may contain targets.

[0101] In some embodiments, a first loss value is calculated based on the predicted activation image and the mask image. For example, the first loss value can be a mean square error loss, which is a loss function commonly used in regression tasks. It is used to measure the average difference between the predicted value and the true value. Since there are usually only image-level labels in weakly supervised target detection tasks, even if the target is known to exist in the original image, the specific location of the target is unknown. Therefore, the mask image can be used as a supervisory signal to guide the locator to generate the corresponding predicted activation image, and the locator is trained by calculating the mean square error loss feedback between the predicted activation image and the mask image.

[0102] Step S105 : obtaining a bounding box, and sampling the mask image according to the bounding box to obtain an enhanced image.

[0103] In some embodiments, a bounding box is obtained and a mask image is sampled based on the bounding box to obtain a corresponding enhanced image. Specifically, a sampling region is obtained by sampling the mask image using the bounding box. This sampling region is then masked and combined with the original image to obtain a data-enhanced training image, i.e., the enhanced image. Because the sampling region includes a small portion of background, the foreground and background can be fused, allowing the object detection network to learn to recognize and classify objects in the surrounding environment, thereby improving the generalization performance of classification.

[0104] Step S106: Input the enhanced image into the classifier for target classification to obtain a predicted classification result, and generate a second loss value based on the predicted classification result and the classification label.

[0105] In some embodiments, the enhanced image is input as a training image into a classifier in an object detection network for object classification, resulting in a corresponding predicted classification result. It is understood that the predicted classification result is a predicted classification label, and the predicted classification result and the classification label corresponding to the original image are further calculated to generate a second loss value to be fed back to the training classifier.

[0106] In some embodiments, the second loss value is cross-entropy loss, which is a commonly used loss function, usually used in classification tasks. It is used to measure the difference between the predicted result and the true label, and is used as an optimization target to train the model. In the classification task, the classifier predicts each category and uses the true label represented by one-hot encoding. Specifically, for a given sample, assuming that there are K categories of labels, the prediction result can be expressed as a probability distribution containing K elements, where each element represents the probability of the category. The true label is correspondingly represented as a K-dimensional vector, in which only the elements at the corresponding category positions are 1, and the other positions are 0. The core idea of ​​cross-entropy loss is to minimize the difference between the model prediction and the true label, so that the model's prediction result is closer to the actual situation. By minimizing the cross-entropy loss, the model can learn a more accurate classification decision boundary and improve classification accuracy. During the training process, optimization algorithms such as gradient descent are usually used to minimize the cross-entropy loss.

[0107] Step S107: Adjust the model weights of the target detection network according to the first loss value and the second loss value until the iteration termination condition is reached to obtain a trained target detection network.

[0108] In some embodiments, the model weights of the target detection network are adjusted according to the first loss value and the second loss value. Specifically, the weights in the locator can be adjusted and retrained according to the mean square error loss, and the weights in the classifier can be adjusted and retrained according to the cross entropy loss until the iteration termination condition is reached. It is understandable that the iteration termination condition can be a preset number of training times, or the first loss value can be less than the first preset value and / or the second loss value can be less than the second preset value, etc., thereby obtaining a trained target detection network, and this embodiment does not limit this.

[0109] Reference Figure 2 As shown, in some embodiments of the present application, the above-mentioned step S103 may include but is not limited to the following steps S201 to S203.

[0110] Step S201: Input the original image into the supervision model to obtain the attention tensor.

[0111] In some embodiments, the supervised model includes a Transformer encoder having a preset number of attention heads. Specifically, Transformer is a neural network architecture for sequence modeling, which was originally proposed for machine translation tasks. It introduces a self-attention mechanism that can effectively capture the relationship between different positions in the input sequence. In the Transformer encoder, the self-attention mechanism is used to model the dependencies within the input sequence, and the attention head is a variant of the self-attention mechanism that is used to learn multiple different attention weights in parallel. Each attention head has its own query, key, and value, and the final representation of the encoder is obtained by weighted summing them.

[0112] In some embodiments, an original image is input into a supervised model including a Transformer encoder to obtain an attention tensor, wherein the attention tensor has a first preset dimension, and the first preset dimension corresponds to an attention head. For example, when the Transformer encoder in the supervised model is provided with 12 attention heads, a 12-dimensional attention tensor is obtained. For another example, when the Transformer encoder in the supervised model is provided with 9 attention heads, a 9-dimensional attention tensor is obtained, i.e., each attention head corresponds to a one-dimensional attention tensor.

[0113] Step S202: Select a second preset dimension according to the first preset dimension.

[0114] In some embodiments, a second preset dimension is selected based on the first preset dimension, and the first preset dimension includes the second preset dimension. For example, when the first preset dimension is 12, the attention tensors of the 9th, 10th, and 11th dimensions can be selected, that is, the selected second preset dimensions are continuous 9 to 11; for another example, the attention tensors of the 0th, 5th, and 10th dimensions are selected, that is, the selected second preset dimensions are discontinuous 0, 5, and 10. It can be understood that the dimension count starts from 0, and the dimensions corresponding to the 12-dimensional attention tensor are 0 to 11, and this embodiment does not limit this.

[0115] Step S203: Select the attention value of the second preset dimension from the attention tensor, and calculate the self-attention map based on the attention value.

[0116] In some embodiments, the attention value of the second preset dimension is selected from the attention tensor, so as to calculate the self-attention map according to the attention value. For example, the supervised model is a DINO model with 12 attention heads, and the original image I is input to the DINO model to obtain an attention tensor F containing 12 dimensions. attention , that is, F attention =DINO(I), the second preset dimension selected is the last three dimensions, that is, the attention values ​​corresponding to the last three attention tensors are selected from the 12-dimensional attention tensor to calculate the self-attention map. Specifically, the last three attention values ​​are selected to calculate the average value as the attention map F dino , that is, F dino =AVG(F attention [9:]), where F dino The size of the image is w×h, where w=W / P, h=H / P, P is the size of an image block in the Transformer encoder, WxH is the dimension of the original image, W represents the width of the original image, and H represents the height of the original image.

[0117] Reference Figure 3 As shown, in some embodiments of the present application, the above-mentioned step S103 may also include but is not limited to the following steps S301 to S305.

[0118] Step S301: Obtain foreground pixels and background pixels of the original image according to the self-attention map.

[0119] In some embodiments, the self-attention map includes multiple activation values. Specifically, each pixel or each image block in the self-attention map corresponds to an activation value, and the range of the activation value is [0, 1]. The foreground pixel and background pixel corresponding to the original background are obtained according to the self-attention map. Since the original background and the self-attention map are Figure 1 One-to-one correspondence: the pixel position in the self-attention map can be mapped to the pixel position in the original image, and the pixel value at that position is used as the foreground pixel or background pixel. Specifically, the foreground pixel and background pixel in the original image are initialized using the pixel activation value in the self-attention map.

[0120] As you can understand, foreground and background refer to different objects or regions within an image. The foreground is the object or region of interest, typically the primary content you wish to extract or highlight from the image. Accurately separating the foreground from the background is essential for further analysis or processing. The background, on the other hand, is the area outside the foreground, including other objects corresponding to the foreground, the surrounding environment, or background noise. Therefore, by identifying and classifying foreground and background pixels, we can achieve accurate image segmentation, enabling a better understanding of the image content and subsequent analysis or processing.

[0121] Step S302: Create a Gaussian mixture model based on the foreground pixels and the background pixels.

[0122] In some embodiments, a Gaussian mixture model is created using the foreground and background pixels of the original image as seed pixels. The seed pixels are used as prior information for the Gaussian mixture model. Using the initialized foreground and background seed pixels, a Gaussian mixture model is constructed to represent the color distribution of the image. In each iteration, a graph cut algorithm is used to update the pixel labels based on the current Gaussian mixture model, re-estimate the probability of each pixel belonging to the foreground or background, and determine the optimal segmentation result by minimizing the energy function. Finally, iterative optimization is performed until a termination condition is met, such as reaching the maximum number of iterations or the change in the energy function is below a certain threshold.

[0123] In some embodiments, the iterative training of the Gaussian mixture model incorporates low-level features of the original image for guidance. For example, information such as the original image's color and texture is incorporated to iteratively determine the probability of a pixel belonging to the foreground or background. With this rough feature information of the foreground and background, the Gaussian mixture model can be used to generate a mask image with a clearer foreground.

[0124] Step S303: Obtain a response threshold range, and segment the self-attention map into a foreground area, a background area, and a candidate area according to the response threshold range and the activation value.

[0125] In some embodiments, a response threshold range is obtained, and the self-attention map is segmented into a foreground region, a background region, and a candidate region based on the response threshold range and the activation value. For example, the response threshold range is [0.02, 0.1]. When the activation value corresponding to a pixel in the self-attention map is less than 0.02, the pixel is segmented into the background region; when the activation value corresponding to the pixel in the self-attention map is greater than 0.1, the pixel is segmented into the foreground region; when the activation value corresponding to the pixel in the self-attention map is within the response threshold range, the pixel is segmented into the candidate region.

[0126] In step S304 , based on the low-level features of the original image, a Gaussian mixture model is used to calculate the segmentation probability of each pixel in the candidate area, and the pixel is divided into the foreground area or the background area according to the segmentation probability.

[0127] In some embodiments, based on low-level features such as color and texture of the original image, a Gaussian mixture model is used to learn the image features of the foreground and background, and then the segmentation probability of each pixel in the candidate area is calculated to discriminate the pixels in the candidate area. It is understandable that the segmentation probability of a pixel calculated by the Gaussian mixture model may be a set of data, including multiple probabilities, and the highest probability is selected as the final segmentation probability to segment the pixel into a foreground area or a background area. For example, the pixel activation value in the candidate area is 0.08. After the Gaussian mixture model iteratively judges based on the low-level features of the original image, the highest segmentation probability is 0.12, which is greater than the response threshold range, and therefore the pixel is divided into the foreground area.

[0128] Step S305 , obtaining a mask image according to the foreground area and the background area.

[0129] In some embodiments, each pixel in the candidate area is divided into a foreground area or a background area through a Gaussian mixture model, thereby setting the mask of the foreground area in the original image to 1 and the mask of the background area in the original image to 0 to obtain a corresponding binary mask image.

[0130] In some embodiments, a GrabCut algorithm is used to generate a mask image based on the self-attention map and the corresponding original image. GrabCut is an interactive algorithm that uses a Gaussian mixture model to model the color distribution in the image. The image features of the foreground and background are learned to discriminate the pixels of the candidate area to obtain the highest segmentation probability for each pixel in the candidate area, and the candidate area is segmented into the foreground area or the background area, thereby generating a mask image M that combines the rich low-level features of the original image. G , that is, M G =GrabCut(I, F dino ). Among them, the mask image M G The size is 1xWxH, WxH is the original image dimension, W represents the width of the original image, H represents the height of the original image, that is, the mask image M G The size of is the same as the original image. Therefore, if we have a rough feature map of the foreground and background, we can use the GrabCut algorithm to generate an enhanced binary mask image with a clearer foreground.

[0131] Reference Figure 4 As shown, in some embodiments of the present application, the above-mentioned step S104 may include but is not limited to the following steps S401 to S404.

[0132] Step S401: Input the original image into the backbone network to obtain a first feature map.

[0133] In some embodiments, the target detection network also includes a pre-trained backbone network, which is composed of a Transformer encoder with fixed parameters. At the same time, the locator includes a UNet network, a three-layer deconvolutional neural network and a normalization network. Specifically, the backbone network is connected to the UNet network of the locator, and the original image is input into the backbone network to obtain the first feature map F i , F i =f(I;W), where i represents the i-th feature block of Transformer, and F i The size is D×w×h, where D is the dimension of the first feature map, W is the parameter of the backbone network, I is the original image, and F i is the output of the backbone network.

[0134] Step S402: Input the first feature map into the UNet network to obtain a second feature map.

[0135] In some embodiments, the first feature map F i Input into the UNet network to obtain the second feature map F u , Among them, F u The size of is 256×w×h, 256 is the dimension of the second feature map, is the parameter of the UNet network model, F u is the output of the UNet network.

[0136] Step S403: Input the second feature map into a deconvolutional neural network to obtain a third feature map.

[0137] In some embodiments, the second feature map is input into the deconvolution neural network to obtain a third feature map F d , in, is the parameter of the deconvolution neural network model, F d is the output of the deconvolutional neural network.

[0138] Step S404: Input the third feature map into the normalization network to obtain a predicted activation map.

[0139] In some embodiments, the third feature map is input into the normalized network to obtain the predicted activation map F a , in, is the weight of the normalized network, F a It is the predicted activation map of the final output, whose size is 1×(2 3 ×w)×(2 3 ×h), where 3 corresponds to three deconvolution layers.

[0140] Reference Figure 5 As shown, in some embodiments of the present application, the above-mentioned step S105 may include but is not limited to the following steps S501 to S504.

[0141] Step S501 : uniformly sampling is performed within a plurality of candidate bounding boxes to obtain a bounding box.

[0142] In some embodiments, uniform sampling is performed within multiple candidate bounding boxes to obtain a bounding box. Specifically, the size and position of the candidate bounding box can be randomly generated. The bounding box coordinates are sampled uniformly according to the height and width of the mask image, which can be obtained by r w and r h and parameter λ is scaled to control the size of the bounding box, r x ~Unif(0,W), r y ~Unif(0,H), Among them, r x ~Unif(0,W) and are the uniformly sampled x coordinate and the width of the bounding box. y ~Unif(0,H) and They represent the coordinates of uniformly sampled y and the height of the bounding box, respectively. λ is obtained by sampling from a uniform distribution (0-1). The final bounding box coordinates are A=(r x , r y , r w , r h ).

[0143] Step S502 : randomly sampling the mask image using the bounding box to obtain a sampling area.

[0144] In some embodiments, reference Figure 6 As shown in the random sampling diagram, the bounding box is used to randomly sample the mask image to obtain the sampling area as shown in the upper right corner. The sampling area includes the foreground and background parts. The foreground of the original mask image is the white bird part and the background is the black part.

[0145] Step S503: Mask the sampling area using a preset filling value to obtain an enhanced mask image.

[0146] In some embodiments, the sampling area is masked using a preset filling value, specifically filling the sampling area with a value of "1" to obtain an enhanced mask image, such as Figure 6 The image in the upper right corner is the enhanced mask image after sampling.

[0147] Step S504: performing a dot product between the original image and the enhanced mask image to obtain an enhanced image.

[0148] In some embodiments, the original image is multiplied with the enhanced mask image to obtain an enhanced image. Figure 6 As shown, the original mask image M consists of the bounding box and the upper left corner G Random sampling to obtain the enhanced mask image in the upper right corner The white area is 1 and the black area is 0. Then enhance the mask image By performing a dot multiplication with the original image I in the lower right corner, we can get the enhanced image in the lower left corner. It can be seen that the enhanced image retains the foreground of the original image and adds some background information, which can effectively improve the generalization performance of classification.

[0149] Reference Figure 7 As shown, in some embodiments of the present application, the above-mentioned step S106 may include but is not limited to the following steps S601 to S602.

[0150] Step S601: input the enhanced image into a classifier to obtain a training image sequence.

[0151] In some embodiments, the classifier includes a Transformer encoder, a two-layer convolutional neural network, and a global pooling layer. Specifically, the enhanced image is input as a training image into the classifier to obtain a training image sequence F t , Then convert F t The size of Where D represents the dimension of each image block, represents the enhanced image, W t Represents the weight of the Transformer encoder, F t Represents the output of the Transformer encoder.

[0152] Step S602: Input the training image sequence into the convolutional neural network and the global pooling layer to obtain the predicted classification result.

[0153] In some embodiments, the training image sequence F t Input to the convolutional neural network and global average pooling layer (GAP) to obtain the predicted classification results Among them, W c represents the weight of the convolutional neural network, Represents the predicted classification results.

[0154] It is understandable that the application of Transformer encoder can mine global dependencies, and on this basis, the application of convolutional neural network CNN can explore local image features that are conducive to classification, so that the classifier can achieve more accurate classification under the guidance of category information.

[0155] The present application is described below through a complete embodiment:

[0156] Reference Figure 8 The training process architecture diagram of the object detection network shown in the figure shows that the original image is input into the backbone network to obtain the corresponding feature map, which is then input into the localizer for target localization. The mask image obtained by GrabCut enhancement is used as a supervisory signal to train the localizer, resulting in a predicted activation image. Simultaneously, a self-attention map is generated using the supervised model, and the mask image is enhanced using the GrabCut algorithm. The enhanced image is then randomly and uniformly sampled using a bounding box based on the low-level features of the original image. This enhanced image is then input into the classifier as a training image for target classification, and the predicted classification results are obtained to train the classifier.

[0157] Further, refer to Figure 9 As shown in the schematic diagram of low-level feature enhancement, the first picture in the upper left corner of Figure (a) is the self-attention map output by the supervision model, and the original mask image is obtained therefrom, but the boundary of the original mask image is not clear and complete. This embodiment combines the low-level features of the original image and inputs them into Low-level Cues, thereby obtaining the enhanced mask image in the upper right corner of Figure (a), which has a clear and complete boundary. Specifically, in Figure (b), the enhanced mask image and the original image are combined to further mine Global information and Local information, wherein Global information is the global information for positioning, and Local information is the category local information required by the classifier. Positioning based on global information is more accurate and complete, and classification based on category local information can improve accuracy, thereby improving the accuracy and performance of classification and positioning, respectively.

[0158] In some embodiments of the present application, reference is made to Figure 10 The visualization result comparison chart shown is a visualization result comparison chart of this embodiment on the CUB-200-2011 and ILSVC 2012 datasets, and is combined with reference Figures 11 to 14 The effect comparison diagram shown in the figure, Figure 11 This is a comparison chart of the positioning effect on the CUB-200-2011 dataset. Figure 12 This is a comparison chart of the classification effect on the CUB-200-2011 dataset. Figure 13 This is a comparison chart of the positioning effect on the ILSVRC 2012 dataset. Figure 14 The following figure compares the classification results on the ILSVRC 2012 dataset. As can be seen, for the same self-attention map, the object detection training method of this embodiment achieves clearer boundary localization and classification, and compared with other localization or classification methods, the localization and classification results of this embodiment are significantly improved.

[0159] By designing a unified network architecture for the classifier and localizer, the low-level features of the original image are fully integrated to produce a mask image that more comprehensively covers the target. The localizer uses pseudo-label training that incorporates low-level signals to generate more localized activation images of the global target region, greatly improving positioning accuracy. Simultaneously, the classifier combines the respective strengths of the Transformer and CNN, using CNN to explore local features that are beneficial for classification based on the Transformer's global dependencies, thereby improving classification performance. Biqikong introduces a new data augmentation method for classification, utilizing mask information from low-level features to preserve the foreground and integrate a small portion of the background. This encourages learning to recognize and classify objects in the surrounding environment, thereby improving the generalization performance of classification.

[0160] The embodiment of the present invention also provides a target detection method that can classify and locate images, and applies the training method of the target detection network as described above, referring to Figure 15 As shown, in some embodiments of the present application, the target detection method may include but is not limited to the following steps S701 to S703.

[0161] Step S701: Obtain an image to be detected and input it into the target detection network.

[0162] In some embodiments, reference Figure 16 The target detection method process architecture diagram shows that the target detection network includes a pre-trained backbone network, a locator and a classifier. Specifically, the backbone network is connected to the locator and the classifier respectively, obtains the image to be detected and inputs it into the target detection network for weakly supervised target detection to perform positioning and classification.

[0163] Step S702: Generate a feature map corresponding to the image to be detected using the backbone network.

[0164] In some embodiments, a backbone network is first used to generate a feature map corresponding to the image to be detected. In object detection, the image can be segmented into multiple regions, and then the image blocks of each region are passed as input to the Transformer encoder. The encoder maps the image blocks into high-dimensional feature vectors, representing the semantic information of the region. These feature vectors are used as feature maps, where each position corresponds to a region of the input image. The feature vector at each position captures the semantic and contextual information of the region. The backbone network achieves effective modeling of local and global features through the multi-layer self-attention mechanism of the Transformer encoder.

[0165] Step S703: input the feature map into the locator to obtain a positioning result, and input the feature map into the classifier to obtain a classification result.

[0166] In some embodiments, the feature map is input into a localizer to obtain a localization result, and the feature map is input into a classifier to obtain a classification result. It is understood that the localization result is the localization activation image, and the classification result is the predicted class label, thereby completing the localization and classification of weakly supervised object detection of the image to be detected.

[0167] The specific implementation of the target detection method of this embodiment is basically the same as the specific implementation of the training method of the above-mentioned target detection network, and will not be repeated here.

[0168] The embodiment of the present invention also provides a training device for a target detection network, which can implement the training method of the target detection network. Figure 17 As shown, in some embodiments of the present application, the training device of the target detection network includes:

[0169] An acquisition module 100 is configured to acquire a training data set, wherein the training data set includes a plurality of original images, and the original images include classification labels;

[0170] Input module 200, used to input the original image into the target detection network; the target detection network includes a classifier and a locator;

[0171] A generation module 300 is used to generate a self-attention map of the original image using a preset supervision model, and generate a mask image based on the self-attention map and the original image;

[0172] A first training module 400 is configured to input the original image into a locator for target positioning to obtain a predicted activation image, and generate a first loss value based on the predicted activation image and the mask image;

[0173] The enhancement module 500 is used to obtain a bounding box and sample the mask image according to the bounding box to obtain an enhanced image;

[0174] The second training module 600 is used to input the enhanced image into the classifier for target classification to obtain a predicted classification result, and generate a second loss value according to the predicted classification result and the classification label;

[0175] The adjustment module 700 is used to adjust the model weights of the target detection network according to the first loss value and the second loss value until the iteration termination condition is reached to obtain a trained target detection network.

[0176] The specific implementation of the target detection network training device of this embodiment is basically the same as the specific implementation of the target detection network training method described above, and will not be repeated here.

[0177] Figure 18The electronic device 1000 provided in an embodiment of the present application is shown. The electronic device 1000 includes: a processor 1001, a memory 1002, and a computer program stored in the memory 1002 and executable on the processor 1001. When the computer program is executed, it is used to execute the above-mentioned object detection network training method or object detection method.

[0178] The processor 1001 and the memory 1002 may be connected via a bus or other means.

[0179] Memory 1002, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs, such as the object detection network training method or object detection method described in the embodiments of this application. Processor 1001 implements the above-mentioned object detection network training method or object detection method by running the non-transitory software program and instructions stored in memory 1002.

[0180] The memory 1002 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store a training method or a target detection method for executing the above-mentioned target detection network. In addition, the memory 1002 may include a high-speed random access memory 1002, and may also include a non-transitory memory 1002, such as at least one storage device storage device, a flash memory device or other non-transitory solid-state memory device. In some embodiments, the memory 1002 may optionally include a memory 1002 remotely arranged relative to the processor 1001, and these remote memories 1002 may be connected to the electronic device 1000 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0181] The non-transient software program and instructions required to implement the above-mentioned target detection network training method or target detection method are stored in the memory 1002. When executed by one or more processors 1001, the above-mentioned target detection network training method or target detection method is executed, for example, Figure 1 Steps S101 to S107 of the method, Figure 2 Steps S201 to S203 of the method, Figure 3 Steps S301 to S305 of the method, Figure 4 Steps S401 to S404 of the method, Figure 5 Steps S501 to S504 of the method, Figure 7 Steps S601 to S602 of the method, Figure 15 Method steps S701 to S703 in .

[0182] The embodiment of the present application also provides a storage medium, which is a computer-readable storage medium, and the storage medium stores a computer program, which implements the training method or target detection method of the above-mentioned target detection network when the computer program is executed by the processor. The memory is a non-transient computer-readable storage medium that can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory optionally includes a memory remotely arranged relative to the processor, and these remote memories can be connected to the processor via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0183] The object detection network training method, object detection method, and related devices provided in the embodiments of the present application have the following advantages and effects:

[0184] 1. Use the low-level features of the original image to enhance the Transformer self-attention and generate a mask image of the locator as a pseudo-label. Traditional methods usually only focus on local high-level semantic features, such as CNN network features, and directly generate pseudo-labels based on this. However, the pseudo-labels generated by this method cannot cover more target areas, resulting in low-quality pseudo-labels, which in turn causes the positioning results to be inaccurate. This application proposes to make full use of the low-level signals of the image, such as the texture and color of the target in the image, to generate a more accurate mask, and use it to train a more accurate locator. In this way, the rich detail information in the image can be better utilized to improve the accuracy of the locator in the target position.

[0185] 2. Unify the locator and classifier into one network architecture. Previous methods usually focus more on the generation of the locator, and directly reference a trained classification network, such as ResNet50 or EfficientNet-B7, for classification. This means that two tests are required, which increases the computational overhead. To this end, this application proposes to treat the classifier and locator equally, concentrating them in one network architecture, while each performs its own function without affecting each other. When detecting an image, it only needs to be input into a backbone network, and then input into the classifier and locator respectively.

[0186] 3. The classification method that combines Transformer and CNN mines local category signals based on global information. Traditional CNN focuses on extracting local features and spatial patterns of images, and performs classification predictions through convolution, pooling, and fully connected layers. By introducing the Transformer encoder, the ability to model global dependencies can be introduced into the CNN model. The Transformer's attention mechanism allows the model to better capture the associations and dependencies between different locations when processing images, rather than just being limited to local areas. By combining Transformer with CNN, the model can comprehensively utilize the local feature extraction capabilities of CNN and the global dependency modeling capabilities of Transformer to obtain a more comprehensive and accurate image representation. This combination helps the model better understand the image content and improve classification accuracy and generalization capabilities.

[0187] The embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0188] Those skilled in the art will appreciate that all or some of the steps and systems in the method disclosed above can be implemented as software, firmware, hardware, and appropriate combinations thereof. Some physical components or all physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, and the computer-readable medium can include computer storage media (or non-transitory media) and communication media (or temporary media). As known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory, or other memory technology, CD-ROM, digital versatile disks (DVD), or other optical disk storage, magnetic cassettes, magnetic tapes, storage device storage, or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, as is well known to those skilled in the art, communication media typically includes computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0189] It should also be understood that the various implementations provided in the embodiments of this application can be arbitrarily combined to achieve different technical effects. The above is a specific description of the preferred implementation of this application, but this application is not limited to the above implementation. Those skilled in the art can also make various equivalent modifications or substitutions under the conditions that do not violate the spirit of this application.

Claims

1. A training method for a target detection network, characterized in that: include: Obtain a training data set; the training data set includes a plurality of original images, and the original images include classification labels; Inputting the original image into the target detection network; The target detection network includes a classifier, a backbone network and a locator, and the locator includes a UNet network, a three-layer deconvolutional neural network and a normalization network; Inputting the original image into the supervised model to obtain an attention tensor; The attention tensor has a first preset dimension, which corresponds to an attention head. A second preset dimension is selected based on the first preset dimension, an attention value of the second preset dimension is selected from the attention tensor, and a self-attention map is calculated based on the attention value. The foreground pixels and background pixels of the original image are obtained based on the self-attention map. A Gaussian mixture model is created based on the foreground pixels and the background pixels, a response threshold range is obtained, and the self-attention map is segmented into a foreground area, a background area, and a candidate area based on the response threshold range and the activation value. Based on the low-level features of the original image, the Gaussian mixture model is used to calculate the segmentation probability of each pixel in the candidate area, and the pixel is divided into the foreground area or the background area based on the segmentation probability. A mask image is obtained based on the foreground area and the background area. The supervised model includes a Transformer encoder, the Transformer encoder has a preset number of attention heads, and the self-attention map includes multiple activation values. Inputting the original image into the backbone network to obtain a first feature map, inputting the first feature map into the UNet network to obtain a second feature map, inputting the second feature map into the deconvolutional neural network to obtain a third feature map, inputting the third feature map into the normalization network to obtain a predicted activation image, and generating a first loss value according to the predicted activation image and the mask image; Obtaining a bounding box, and sampling the mask image according to the bounding box to obtain an enhanced image; Inputting the enhanced image into the classifier for target classification to obtain a predicted classification result, and generating a second loss value based on the predicted classification result and the classification label; The model weights of the target detection network are adjusted according to the first loss value and the second loss value until an iteration termination condition is reached, thereby obtaining the trained target detection network.

2. The method for training a target detection network according to claim 1, wherein: The obtaining of the bounding box and sampling the mask image according to the bounding box to obtain an enhanced image includes: uniformly sampling within a plurality of candidate bounding boxes to obtain the bounding box; Randomly sampling the mask image using the bounding box to obtain a sampling area; Masking the sampling area using a preset filling value to obtain an enhanced mask image; Performing a dot product between the original image and the enhanced mask image to obtain the enhanced image.

3. The object detection network training method according to claim 1 or 2, characterized in that: The classifier includes a Transformer encoder, a two-layer convolutional neural network and a global pooling layer; Inputting the enhanced image into the classifier for target classification to obtain a predicted classification result includes: Inputting the enhanced image into the classifier to obtain a training image sequence; The training image sequence is input into the convolutional neural network and the global pooling layer to obtain the predicted classification result.

4. A target detection method, characterized in that: The method for training an object detection network according to any one of claims 1 to 3 comprises: Acquire an image to be detected and input it into the target detection network; wherein the target detection network includes a pre-trained backbone network, a locator and a classifier, and the backbone network is connected to the locator and the classifier respectively; Generating a feature map corresponding to the image to be detected using the backbone network; The feature map is input into the locator to obtain a positioning result, and the feature map is input into the classifier to obtain a classification result.

5. A training device for a target detection network, characterized in that: Applying the training method of the target detection network according to any one of claims 1 to 4, comprising: An acquisition module, configured to acquire a training data set; wherein the training data set includes a plurality of original images, and the original images include classification labels; An input module, configured to input the original image into a target detection network; the target detection network includes a classifier, a backbone network, and a locator, wherein the locator includes a UNet network, a three-layer deconvolutional neural network, and a normalization network; A generation module is configured to input the original image into a supervised model to obtain an attention tensor; the attention tensor has a first preset dimension, the first preset dimension corresponds to the attention head, a second preset dimension is selected based on the first preset dimension, an attention value of the second preset dimension is selected from the attention tensor, and a self-attention map is calculated based on the attention value, foreground pixels and background pixels of the original image are obtained based on the self-attention map, a Gaussian mixture model is created based on the foreground pixels and the background pixels, a response threshold range is obtained, and the self-attention map is segmented into a foreground area, a background area, and a candidate area based on the response threshold range and the activation value, based on the low-level features of the original image, the segmentation probability of each pixel in the candidate area is calculated using the Gaussian mixture model, and the pixel is divided into the foreground area or the background area based on the segmentation probability, and a mask image is obtained based on the foreground area and the background area, the supervised model includes a Transformer encoder, the Transformer encoder has a preset number of attention heads, and the self-attention map includes a plurality of activation values; a first training module, configured to input the original image into the backbone network to obtain a first feature map, input the first feature map into the UNet network to obtain a second feature map, input the second feature map into the deconvolutional neural network to obtain a third feature map, input the third feature map into the normalization network to obtain a predicted activation image, and generate a first loss value according to the predicted activation image and the mask image; an enhancement module, configured to obtain a bounding box and sample the mask image according to the bounding box to obtain an enhanced image; A second training module is configured to input the enhanced image into the classifier for target classification to obtain a predicted classification result, and generate a second loss value based on the predicted classification result and the classification label; An adjustment module is used to adjust the model weights of the target detection network according to the first loss value and the second loss value until the iteration termination condition is reached to obtain the trained target detection network.

6. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method for training a target detection network according to any one of claims 1 to 3 or the target detection method according to claim 4 is implemented.

7. A computer-readable storage medium, characterized in that The storage medium stores a program, and the program is executed by a processor to implement the training method of the target detection network according to any one of claims 1 to 3 or the target detection method according to claim 4.

Images