Target detection model training method and target detection method

Abstract

The invention discloses a target detection model training method and a target detection method. The method comprises the following steps: acquiring a target image for training; performing feature extraction on the target image by adopting the initial target detection model to obtain a plurality of query vectors; a first detection head and a second detection head of the initial target detection model are adopted to predict the multiple query vectors to obtain multiple sets of prediction results, the first detection head generates a corresponding prediction result for each query vector, and the second detection head generates multiple corresponding prediction results for each query vector; loss corresponding to the multiple sets of prediction results is determined, parameters of the initial target detection model are iteratively updated according to the loss, iteration is stopped under the condition that a stopping condition is met, and the target detection model is obtained. According to the method and the device, the technical problem that the model training efficiency is low due to the fact that more training parameters are introduced into an auxiliary training method adopted in the related technology and large burden is brought to model training is solved.

Description

Technical Field

[0001] The present application relates to the field of computer vision, and more specifically, to a method for training a target detection model and a method for target detection. Background Art

[0002] In the field of object detection, deep learning-based detection models are commonly used. However, when these technologies employ auxiliary training methods to improve the performance of object detection models, they introduce a large number of training parameters, which increases the time required for each model cycle. Therefore, these technologies increase detection performance by increasing model complexity, sacrificing training efficiency and effective resource utilization.

[0003] To address the above-mentioned problems, no effective solutions have been proposed so far. Summary of the Invention

[0004] The embodiments of the present application provide a method for training a target detection model and a method for target detection, so as to at least solve the technical problem that the auxiliary training method adopted by the related technology introduces more training parameters, brings a greater burden to the training of the model, and leads to low model training efficiency.

[0005] According to one aspect of an embodiment of the present application, a training method for a target detection model is provided, comprising: obtaining a target image for training, wherein the target image includes a detection target; using an initial target detection model to perform feature extraction on the target image to obtain multiple query vectors, wherein the query vectors are used to locate the detection target; using a first detection head and a second detection head of the initial target detection model to respectively predict the multiple query vectors to obtain multiple groups of prediction results, wherein the first detection head generates a corresponding prediction result for each query vector, and the second detection head generates multiple corresponding prediction results for each query vector; determining the losses corresponding to the multiple groups of prediction results, and iteratively updating the parameters of the initial target detection model based on the losses, and stopping the iteration when the stopping condition is met to obtain the target detection model.

[0006] In some embodiments of the present application, the first detection head and the second detection head of the initial target detection model are used to predict multiple query vectors respectively to obtain multiple groups of prediction results, including: inputting multiple query vectors into the first detection head respectively to obtain multiple first prediction results, wherein the first prediction results include a first prediction box and a first prediction category, and each query vector corresponds to a first prediction result; inputting multiple query vectors into the second detection head respectively to obtain multiple second prediction results corresponding to each query vector, wherein the second prediction results include a second prediction box and a second prediction category; and determining multiple groups of prediction results based on the first prediction results and the second prediction results.

[0007] In some embodiments of the present application, multiple groups of prediction results are determined based on the first prediction result and the second prediction result, including: constructing a cost matrix with the real annotation box corresponding to the target image as a row and all the second prediction boxes as columns, wherein each element of the cost matrix represents the matching cost between the corresponding real annotation box and the second prediction box, and the real annotation box is the real annotation box of the detection target; performing a one-to-one matching between the real annotation box and the second prediction box based on the cost matrix to obtain an optimal matching result, wherein the matching cost corresponding to the optimal matching result is the smallest; updating the second prediction box that matches the real annotation box in the optimal matching result to the second prediction result corresponding to the second detection head; and determining multiple groups of prediction results based on the first prediction result and the updated second prediction result.

[0008] In some embodiments of the present application, the loss includes a first loss and a second loss corresponding to the first detection head and the second detection head respectively; determining the losses corresponding to multiple groups of prediction results includes: determining the first loss between the first prediction result of the first detection head and the true annotation box of the target image, and determining the second loss between the second prediction result of the second detection head and the true annotation box; obtaining the first weight of the first detection head and the second weight of the second detection head; determining the losses corresponding to multiple groups of prediction results based on the first loss, the first weight, the second loss and the second weight.

[0009] In some embodiments of the present application, after determining the losses corresponding to multiple groups of prediction results based on the first loss, the first weight, the second loss and the second weight, the method also includes: obtaining the attenuation factor corresponding to the current training iteration round, and updating the first weight and the second weight respectively based on the attenuation factor, the first loss and the second loss, wherein the attenuation factor is used to quantify the degree of influence of the first weight of the previous iteration round and the second weight of the previous iteration round on the first weight of the current iteration round and the second weight of the current iteration round, respectively, and the current iteration round is the next iteration round of the previous iteration round.

[0010] In some embodiments of the present application, an initial target detection model is used to extract features of a target image to obtain multiple query vectors, including: using an encoder of the initial target detection model to generate multiple feature vectors corresponding to the target image, wherein the multiple feature vectors are used to cover features of different positions and detail levels of the target image; using a decoder of the initial target detection model to determine query scores corresponding to the multiple feature vectors, and determining multiple query vectors from the multiple feature vectors based on the query scores, wherein the query scores are used to quantitatively represent the possibility of the feature vector matching the detection target.

[0011] In some embodiments of the present application, the loss includes a third loss corresponding to the query score; the third loss is determined by: using the decoder of the initial target detection model to generate a query score prediction value for each feature vector, and determining the true query score value of each feature vector based on the true annotation box information of the target image; and determining the third loss based on the query score prediction value and the true query score value.

[0012] In some embodiments of the present application, the true value of the query score of each feature vector is determined based on the real annotation box information of the target image, including: dividing the target image into multiple grids, where each grid corresponds to a feature vector; obtaining the coordinate information of all annotation boxes from the real annotation box information; and determining the true value of the query score of each feature vector based on the positional relationship between the coordinate information and each grid.

[0013] In some embodiments of the present application, the true value of the query score of each feature vector is determined based on the positional relationship between the coordinate information and each grid, including: when the positional relationship indicates that there is no intersection between the target grid and the annotation box, determining the true value of the query score corresponding to the target grid is zero, wherein the target grid is any one of the multiple grids; when the positional relationship indicates that there is an intersection between the target grid and the annotation box, determining the true value of the query score corresponding to the target grid based on the offset between the center point of the target grid and the center point of the annotation box.

[0014] In some embodiments of the present application, after determining the third loss, the method further includes: obtaining the total number of feature vectors and scaling the third loss using the total number; or scaling the query score prediction value to within a preset range.

[0015] In some embodiments of the present application, after obtaining the target image for training, it also includes: determining the first size information of the minimum enclosing rectangle corresponding to all real annotation boxes in the target image, wherein the first size information includes length information and width information; when the ratio of the first size information to the second size information of the target image meets a preset threshold, cropping the target image according to the minimum enclosing rectangle to obtain a sub-target image; splicing the sub-target image with a random image to obtain an updated target image.

[0016] In some embodiments of the present application, splicing the sub-target image with the random image includes: splicing the sub-target image with the target image before cropping and a first number of random images according to a first probability; and splicing the sub-target image with a second number of random images according to a second probability, wherein the cumulative sum of the first probability and the second probability is one, and the second number is greater than the first number.

[0017] In some embodiments of the present application, during the inference stage of the target detection model, the first weight and the second weight are compared, the detection head corresponding to the minimum weight indicated by the comparison result is used as the main detection head, and the detection heads other than the main detection head are removed.

[0018] In some embodiments of the present application, the decoder includes a first feedforward neural network and a second feedforward neural network, wherein the first feedforward neural network is connected to the first detection head, and the second feedforward neural network is connected to the second detection head.

[0019] According to another aspect of an embodiment of the present application, a method for target detection is also provided, including: obtaining an image to be detected, wherein the image to be detected includes a target to be detected; using a target detection model to predict the image to be detected to obtain a prediction result corresponding to the target to be detected, wherein the target detection model is obtained by joint training based on a first detection head and a second detection head, and the prediction result is output by a target detection head of the target detection model, and the target detection head is the detection head with the largest weight between the first detection head and the second detection head, and the first detection head generates a corresponding prediction result for each feature vector, and the second detection head generates multiple corresponding prediction results for each feature vector.

[0020] According to another aspect of the embodiments of the present application, a training device for a target detection model is also provided, including: an acquisition module for acquiring a target image for training, wherein the target image includes a detection target; a processing module for using an initial target detection model to perform feature extraction on the target image to obtain multiple query vectors, wherein the query vectors are used to locate the detection target; a prediction module for using the first detection head and the second detection head of the initial target detection model to predict the multiple query vectors respectively to obtain multiple groups of prediction results, wherein the first detection head generates a corresponding prediction result for each query vector, and the second detection head generates multiple corresponding prediction results for each query vector; a determination module for determining the losses corresponding to the multiple groups of prediction results, and iteratively updating the parameters of the initial target detection model based on the losses, and stopping the iteration when the stopping condition is met to obtain the target detection model.

[0021] According to another aspect of the embodiments of the present application, an electronic device is also provided, including: a memory and a processor, the memory being used to store program instructions; the processor being connected to the memory and being used to execute a training method for implementing the above-mentioned target detection model or a target detection method.

[0022] According to another aspect of an embodiment of the present application, a non-volatile storage medium is also provided, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the above-mentioned target detection model training method or target detection method by running the computer program.

[0023] According to another aspect of the embodiments of the present application, a computer program product is also provided, including computer instructions, which, when executed by a processor, implement the above-mentioned target detection model training method or target detection method.

[0024] In an embodiment of the present application, a multi-detection head mechanism is introduced to generate multiple sets of prediction results based on multiple query vectors, thereby achieving the purpose of enhancing the detection performance of the model, thereby effectively improving the convergence speed of the model without introducing additional training parameters. This solves the technical problem that the auxiliary training method adopted by the relevant technology introduces more training parameters, brings a greater burden to the training of the model, and leads to low model training efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0026] Figure 1 This is a hardware structure block diagram of a computer terminal for a method for training a target detection model according to an embodiment of the present application;

[0027] Figure 2 is a flowchart of a method for training a target detection model according to an embodiment of the present application;

[0028] Figure 3 is a flow chart of a method for target detection according to an embodiment of the present application;

[0029] Figure 4 2 is a schematic diagram of a decoding layer structure of a method for training a target detection model according to an embodiment of the present application;

[0030] Figure 5 This is a flowchart of a query score generation method for training a target detection model according to an embodiment of the present application;

[0031] Figure 6a 2 is a schematic diagram of a target image before data enhancement according to a training method of a target detection model according to an embodiment of the present application;

[0032] Figure 6b 3 is a schematic diagram of a target image after data enhancement according to a training method of a target detection model in an embodiment of the present application;

[0033] Figure 7 2 is a schematic diagram of a training acceleration effect of a target detection model training method according to an embodiment of the present application;

[0034] Figure 8It is a structural diagram of a training device for a target detection model according to an embodiment of the present application. DETAILED DESCRIPTION

[0035] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0036] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0037] In order to better understand the embodiments of the present application, the technical terms involved in the embodiments of the present application are explained as follows:

[0038] Real-time Detection Model (RT-DETR): A target detection model based on the Transformer architecture. In the embodiments of this application, RT-DETR is used as the basic model to improve its detection performance and training efficiency in specific scenarios by introducing auxiliary training strategies and acceleration schemes.

[0039] Hungarian Algorithm: An algorithm used in object detection to match predicted boxes with true labeled boxes. It aims to determine the best pairing of predicted boxes and true value boxes by optimizing the matching cost. In this embodiment of the application, the Hungarian Algorithm is used to ensure that each predicted box generated by multiple detection heads can find the most appropriate labeled box for a one-to-one match, thereby optimizing the model training process.

[0040] Decoder: A network structure used to convert the feature vector output by the encoder into a final prediction result, such as the category and location of the target. In the embodiment of this application, the decoder is responsible for decoding the target prediction information from the feature vector output by the encoder and improving the efficiency of information processing between different detection heads through a dynamic adjustment mechanism.

[0041] Detection head: The part of the RT-DETR model that generates the final category prediction and bounding box regression from the decoder output. In this embodiment, the detection head is expanded to multiple to assist in the training process, where each detection head can focus on different types of matching, such as one-to-one detection or one-to-many detection, thereby improving overall detection accuracy and efficiency.

[0042] Query Score: The evaluation score of each query vector (feature vector) in the decoder, which is used to determine which queries will be further processed to generate the final detection results. In this embodiment of the application, the query score loss function is improved to ensure the stability and convergence speed of the model during training.

[0043] Exponential Moving Average (EMA): A statistical method used to smooth data sequences, giving more weight to recent observations. In this embodiment, the EMA-based dynamic detection head adjustment module adaptively updates the weight of each detection head to reflect the changes in their contribution to the overall model performance, thereby automatically selecting the most effective detection heads during training or inference.

[0044] Mosaic data augmentation: An image preprocessing technique that generates new training samples by randomly selecting multiple images and stitching them together, aiming to improve the generalization ability of the model. In this embodiment, an adaptive probabilistic mosaic data augmentation method based on target distribution is proposed. This method adjusts the stitching strategy based on whether the detection targets in the image are concentrated, allowing the model to effectively learn target features even in complex monitoring scenarios.

[0045] The multi-detection head-assisted training methods used in related technologies, such as introducing additional decoders into the model, can enhance the model's detection performance, but they also significantly increase the number of model parameters, increasing the computational burden during training and slowing down the model training process. Furthermore, the weight of the detection heads in related technologies is typically fixed, meaning that regardless of the task or dataset, the model distributes the force of each detection head in the same manner, limiting the model's adaptability and flexibility in different environments and data distributions.

[0046] In addition, the Mosaic data augmentation method used in related technologies lacks consideration of target distribution when stitching images. When the camera is far away from the target, this method may result in a large amount of target information loss in the stitched image, especially after cropping. This weakens the model's ability to recognize small targets at a distance, affecting its performance in specific application environments such as gas station scenarios. In the RT-DETR model, of the 8,400 feature vectors output by the encoder, the decoder only selects 300 feature vectors with the highest query scores for processing. However, since the query scores constantly change during training, different feature vectors are selected at different training stages, which affects the consistency and stability of the anchor box offset information learned by the decoder, prolongs the time required for the model to achieve ideal detection performance, and slows down the model's convergence.

[0047] In order to solve the above technical problems, the embodiments of the present application provide corresponding solutions, which are described in detail below.

[0048] The target detection model training method or target detection method embodiment provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 The hardware structure block diagram of a computer terminal for implementing a method for training a target detection model or a method for detecting a target is shown. Figure 1 As shown, the computer terminal 10 may include one or more (illustrated by 102a, 102b, ..., 102n in the figure) processors (the processor may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions connected via a wired and / or wireless network. In addition, it may also include: a display, a keyboard, a cursor control device, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, and a BUS bus. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0049] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuitry." The data processing circuitry may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuitry may be a single, independent processing module, or may be incorporated in whole or in part into any of the other components of the computer terminal 10. As described in the embodiments of the present application, the data processing circuitry serves as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0050] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the target detection model training method or the target detection method in the embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, realizing the above-mentioned target detection model training method or target detection method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor, and these remote memories can be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include but are not limited to the Internet, corporate intranet, local area network, mobile communication network and combinations thereof.

[0051] The transmission module 106 is configured to receive or transmit data via a network. A specific example of the aforementioned network may include a wireless network provided by the communications provider of the computer terminal 10. In one embodiment, the transmission module 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission module 106 may be a radio frequency (RF) module, which is configured to communicate with the Internet wirelessly.

[0052] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 .

[0053] It should be noted that, in some optional embodiments, the above Figure 1 The computer terminal shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of hardware elements and software elements. Figure 1 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the computer terminal described above.

[0054] In the above-mentioned operating environment, an embodiment of the present application provides a method for training a target detection model or an embodiment of a target detection method. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0055] Figure 2 is a flow chart of a method for training a target detection model according to an embodiment of the present application, such as Figure 2 As shown, the method includes the following steps:

[0056] Step S202: Acquire a target image for training, wherein the target image includes a detection target.

[0057] In the above step S202, the target image refers to an image containing an object to be detected (i.e., a detection target), and the image may come from a surveillance camera. For example, an image in a gas station scene may contain targets such as people, cars, and gas pumps.

[0058] Since the distance between the surveillance camera and the target varies, especially when shooting at a long distance, the detection target may be very small, which may make it difficult for the model to accurately detect these small targets. To solve this problem, after obtaining the target image for training, the following steps can also be performed: determine the first size information of the minimum bounding rectangle corresponding to all true annotation boxes in the target image, where the first size information includes length information and width information; when the ratio of the first size information to the second size information of the target image meets the preset threshold, crop the target image according to the minimum bounding rectangle to obtain a sub-target image; and splice the sub-target image with the random image to obtain an updated target image.

[0059] The first size information refers to the size of the minimum bounding rectangle corresponding to all ground-truth annotation boxes in the target image, including length and width. This information is used to measure the proportion of the detected target in the image, providing a basis for determining whether the detected target is centered and whether cropping is necessary. The second size information refers to the size of the target image itself. By comparing it with the first size information, it is used to assess the size ratio of the target relative to the entire image.

[0060] The preset threshold is used to determine whether the target size ratio meets the standard value of the cropping condition. If the ratio of the target's minimum bounding rectangle size to the image size is lower than this threshold, it means that the target occupies a small proportion in the image. It is determined that the detected target is concentrated in the target image, and measures need to be taken to enhance its information content in the image.

[0061] In some embodiments of the present application, the following stitching strategy can be used to stitch the sub-target image with the random image: stitch the sub-target image with the target image before cropping and a first number of random images according to a first probability; stitch the sub-target image with a second number of random images according to a second probability, wherein the cumulative sum of the first probability and the second probability is one, and the second number is greater than the first number.

[0062] The first probability is the probability value used to decide whether to splice the sub-target image with the target image before cropping and a certain number of random images during the data augmentation process. By introducing the information of the original target image, it ensures that the model can simultaneously learn the correlation between the target and the background during training, thereby enhancing the model's adaptability to complex backgrounds.

[0063] The second probability corresponds to the first probability and is used to determine whether to splice the sub-target image with a larger number of random images. By using more random images, the diversity of model training samples is increased, especially the learning of the target's own features is strengthened, thereby improving the generalization performance of the model in target detection tasks.

[0064] The first and second numbers of random images refer to the number of random images used to combine with the sub-target image during the Mosaic data augmentation process. A smaller number of random images aims to maintain the connection with the original target image, while a larger number of random images aims to improve the model's recognition ability by combining more images.

[0065] It should be noted that the first and second probabilities can be dynamically adjusted based on the concentration of the target in the image. If the target is relatively concentrated, the first probability can be increased to ensure that the model learns the relationship between the target and the background. Conversely, the second probability can be increased to strengthen the model's recognition of the target.

[0066] For example, in scenarios like gas station monitoring, the target may be concentrated in a certain part of the image, while the background may be very complex. The above-mentioned stitching strategy ensures that the model can understand the relationship between the target and the background while learning the target features, improving the model's detection performance in complex environments. In addition, by stitching with more random images, the diversity and richness of the target features are increased, helping the model to more accurately identify and locate targets, especially in long-distance monitoring situations where the targets are small and difficult to identify.

[0067] Specifically, for each input image, the length and width of the minimum bounding rectangle of all the annotation boxes in the image are first calculated; if the length and width of the bounding rectangle are both less than 2 / 3 of the corresponding side length of the image, it is considered that the target distribution in the image is relatively concentrated. At this time, the bounding rectangle is used to crop the original image to obtain a sub-image containing the main target area (i.e., sub-target image); in order to allow the model to understand the multi-scale information of the target in the scene and the background information outside the cropping range, the cropped sub-image can be compared with the original image with a probability of 50%. Figure 1 The image is used as input and mosaicked with two other random images to generate new training samples. To improve target visibility and enhance the model's ability to perceive target information, the cropped sub-image with a 50% probability is mosaicked with three other random images to generate new training samples. These two different mosaicking strategies allow the model to learn the relationship between the target and the background while highlighting the characteristics of the target itself, making the model more accurate in detecting targets in complex scenes.

[0068] Figure 6a and Figure 6b is a comparison diagram of target images before and after data enhancement according to a training method of a target detection model in an embodiment of the present application, such as Figure 6a As shown in the figure, in the context of a gas station image, the original mosaic data augmentation method simply randomly selects four images and stitches them together to generate a composite image. The advantage of this method is that it can quickly increase the diversity of training samples, exposing the model to a wider range of visual features during training. However, the original mosaic method ignores the actual distribution of objects in the image. Especially in scenes like gas stations, objects may be concentrated in certain areas of the image, and the original mosaic method may split or crop these areas during stitching, resulting in missing or distorted object information in the training samples, affecting the model's detection performance.

[0069] In contrast, Figure 6b The improved Mosaic generation result for this application first crops the image based on the adaptive probability of the target distribution to ensure the integrity and prominence of the target area. By cropping, unnecessary background information is removed, making the target more prominent in the composite image, which helps the model focus on learning target features. Although the target is cropped, a certain proportion of the splicing with the original image is retained. This allows the model to understand the background environment of the target while learning the target features, enhancing its adaptability in complex scenes. In addition, splicing with different numbers of random images can create training samples containing multi-scale targets and backgrounds, which helps the model learn to detect targets at different scales.

[0070] In step S204 , the initial target detection model is used to perform feature extraction on the target image to obtain a plurality of query vectors, wherein the query vectors are used to locate the detection target.

[0071] In the above step S204, the initial target detection model refers to a target detection model whose architecture has been set but has not yet been optimized and trained before the training starts, such as the RT-DETR model. It is based on the Transformer architecture and uses the encoder and decoder to extract features and predict targets on the input image. In the Transformer architecture, the query vector is a vector that the decoder further processes the features extracted by the encoder to predict the target position and category.

[0072] In some embodiments of the present application, features of the target image can be extracted in the following manner to obtain multiple query vectors: an encoder of the initial target detection model is used to generate multiple feature vectors corresponding to the target image, wherein the multiple feature vectors are used to cover features of different positions and detail levels of the target image; a decoder of the initial target detection model is used to determine query scores corresponding to the multiple feature vectors, and multiple query vectors are determined from the multiple feature vectors based on the query scores, wherein the query scores are used to quantify the possibility of the feature vector matching the detected target.

[0073] Specifically, in the Transformer architecture, the encoder is responsible for converting the input target image into a series of feature vectors. These vectors contain features at various locations and levels of detail in the image. For example, the encoder processes the image through a multi-layer self-attention mechanism and a feedforward neural network, and the generated feature vectors are used by the decoder for subsequent object detection. The decoder is responsible for decoding the feature vectors generated by the encoder. For example, through self-attention and cross-attention mechanisms, the decoder can further refine and combine the feature vectors to generate higher-level representations, thereby determining the query scores corresponding to these vectors.

[0074] The query score is a score calculated by the decoder for each feature vector, which is used to quantify the possibility that the feature vector matches the detection target. For example, in the RT-DETR model, the decoder can select the top K feature vectors as query vectors based on the query score, where K can be 300, for example.

[0075] Figure 5 This is a query score generation flow chart of a training method for a target detection model according to an embodiment of the present application, such as Figure 5 As shown, in some embodiments of the present application, the process includes the following steps:

[0076] (1) Flatten operation, that is, to organize the feature vectors output by the encoder and convert the feature vectors from a three-dimensional structure into a two-dimensional matrix. In (B, 8400, C), B represents the batch size, that is, the number of images processed simultaneously; 8400 represents the number of feature vectors generated by the encoder; C represents the dimension of each feature vector, that is, the complexity of the feature information.

[0077] (2) The feature vectors generated by the decoder are fed into the bounding box regression head and the classification head, respectively. The bounding box regression head is used to predict the coordinates of the target box, while the classification head is used to predict the category to which the target belongs. The classification head outputs a matrix of the likelihood of each of the 8400 feature vectors corresponding to different categories. Its dimension is (B, 8400, number of categories), where B and 8400 remain unchanged. The number of categories is the total number of categories that the model can recognize.

[0078] (3) Taking the maximum value operation, that is, for the output of the classification head, that is, for each feature vector, the model calculates the probability that it belongs to each possible category and selects the maximum probability value as the query score of the feature vector. This means that the query score is actually the model's highest confidence quantification of whether the feature vector represents a target, thereby being able to more directly reflect the correlation between the feature vector and the detected target.

[0079] (4) Obtain the query score, that is, obtain a two-dimensional matrix with a size of (B, 8400), which represents the query score of the 8400 feature vectors of each image.

[0080] (5) After obtaining the query scores of all feature vectors, a top filter is performed, that is, the query vectors with the highest scores (nmu) are selected from the 8400 query scores. Here, nmu is a pre-set parameter, for example, 300, which means that the decoder will focus on processing the top 300 feature vectors that are most relevant to the target.

[0081] (6) Decoding layer input, that is, the filtered query vector (B, num, C) is input into the decoder for further processing, where num is the number of filtered query vectors (such as 300) and C represents the dimension of each feature vector.

[0082] The purpose of the above process is to solve the problems of large computational complexity and slow processing speed when the RT-DETR model processes images in long-distance monitoring scenarios. By screening out high-scoring query vectors, the decoder can perform target detection more efficiently while maintaining high detection accuracy.

[0083] In step S206, the first detection head and the second detection head of the initial target detection model are used to predict multiple query vectors respectively to obtain multiple groups of prediction results, wherein the first detection head generates a corresponding prediction result for each query vector, and the second detection head generates multiple corresponding prediction results for each query vector.

[0084] In step S206 above, the detection head is located after the decoder of the model, receives the feature vector (i.e., query vector) output by the decoder as input, and converts it into a specific detection output. Its specific functions include:

[0085] (1) Classification prediction: Each detection head contains a classification branch, which is used to predict the class of the target corresponding to each query vector (or feature vector). For example, it can be a fully connected layer that outputs a vector representing the probability distribution of each class.

[0086] (2) Bounding box regression: Another branch is used to predict the bounding box position of the target, that is, the four coordinates of the bounding box or its offset relative to the anchor point. It can also be achieved through a series of fully connected layers. The final output is the four regression coefficients corresponding to each query vector, which are used to define the actual position of the target.

[0087] In the RT-DETR model, the detection head is designed to consist of a series of linear layers connected in series. These linear layers can convert the high-dimensional feature vector output by the decoder into low-dimensional detection output, including category prediction and bounding box regression.

[0088] In the decoder stage of the model, each query vector generates a prediction result through the first detection head. This result includes the prediction of the target category and bounding box. This approach follows the basic logic of DETR and RT-DETR, performing target detection in a one-to-one manner, maintaining the simplicity and efficiency of the model.

[0089] By introducing a second detection head, which is a one-to-many prediction head, it also receives each query vector as input but generates multiple prediction results. This can capture more target possibilities during the training phase and enhance the model's adaptability and detection performance.

[0090] During the training phase, the multiple sets of predictions generated by the second detection head can complicate model training, increasing computational cost and time. Furthermore, due to the unique distribution of objects in scenarios such as gas stations, directly using multiple sets of predictions for matching and loss calculation can affect the model's convergence speed and ultimate performance. To effectively utilize the predictions from the one-to-many detection heads, the Hungarian matching algorithm can be used to ensure that each ground-truth box is matched by the most appropriate prediction box, even if the predictions come from different prediction groups.

[0091] In some embodiments of the present application, the decoder in the initial target detection model includes a first feedforward neural network and a second feedforward neural network, the first feedforward neural network is connected to the first detection head, and the second feedforward neural network is connected to the second detection head.

[0092] Specifically, an additional feed-forward neural network (FFN) is introduced into the decoder. At this point, there are two FFNs in the decoder: the original FFN is connected to the one-to-one detection head, and the new FFN is connected to the one-to-many prediction head. Both detection heads participate in model training together. It should be noted that during the inference phase, the one-to-many detection head and the additional FFN are removed, restoring the network structure to the original DETR configuration, thereby ensuring that no additional inference burden is added.

[0093] A one-to-one detection head (i.e., the first detection head) means that each query vector matches only one real target. During model training, the decoder generates a certain number of query vectors, each of which is connected to a detection head, which generates a prediction box and a category prediction. A one-to-many detection head (i.e., the second detection head) means that each query vector may generate multiple prediction results. In RT-DETR, in addition to the original detection head, an additional FFN is introduced to connect to the one-to-many prediction head. This means that each query vector input to the decoder can not only generate a prediction box, but also generate multiple candidate prediction boxes through the additional prediction head.

[0094] Figure 4 is a schematic diagram of the decoding layer structure of a training method for a target detection model according to an embodiment of the present application, such as Figure 4 As shown, Input refers to the input of the model, that is, the feature vector output by the encoder; Self-Attention is an important component in the decoder structure, called the "self-attention" mechanism. Self-attention allows each position vector in the decoder to interact with other position vectors in its sequence, thereby enhancing the understanding of global information; Cross-Attention is the "cross attention" mechanism. Unlike self-attention, cross-attention allows the output of the decoder to interact with the output of the encoder, which means that the decoder can further extract information related to target detection from the features provided by the encoder; FFN refers to feedforward neural network; o2o refers to detection head and Hungarian matching. It should be noted that the Hungarian matching algorithm is used to perform one-to-one matching on the results of multiple detection heads, which can form the following Figure 4 The dual-to-one detection head network architecture is shown.

[0095] Specifically, after the input features enter the decoder, the self-attention mechanism is first used to interact between features to strengthen the intrinsic connection of feature representation. Through cross-attention, the decoder further integrates the features output by the encoder to improve the quality and pertinence of the features. The two feedforward neural networks perform nonlinear transformations on the processed features respectively to prepare data for two different detection heads. These two detection heads (one-to-one detection head and one-to-many detection head) respectively generate prediction results. The former ensures the accurate detection of the target, while the latter increases the scope and possibility of detection.

[0096] In some embodiments of the present application, the first detection head and the second detection head of the initial target detection model can be used to predict multiple query vectors respectively to obtain multiple groups of prediction results. Specifically: the multiple query vectors are respectively input into the first detection head to obtain multiple first prediction results, wherein the first prediction results include a first prediction box and a first prediction category, and each query vector corresponds to a first prediction result; the multiple query vectors are respectively input into the second detection head to obtain multiple second prediction results corresponding to each query vector, wherein the second prediction results include a second prediction box and a second prediction category; and multiple groups of prediction results are determined based on the first prediction results and the second prediction results.

[0097] The first prediction result is a single prediction generated by the first detection head, which includes the first prediction box and the first prediction category. It is used to accurately locate and identify the primary object corresponding to each query vector. The second prediction result is a set of multiple predictions generated by the second detection head. Each prediction includes a second prediction box and a second prediction category. It is used to provide additional object location and category information, enhancing the detection capability and adaptability of the model.

[0098] Multiple groups of prediction results can be determined in the following manner: construct a cost matrix with the true annotation box corresponding to the target image as a row and all second prediction boxes as columns, wherein each element of the cost matrix represents the matching cost between the corresponding true annotation box and the second prediction box, and the true annotation box is the true annotation box of the detection target; perform one-to-one matching between the true annotation box and the second prediction box according to the cost matrix to obtain the optimal matching result, wherein the matching cost corresponding to the optimal matching result is the smallest; update the second prediction box that matches the true annotation box in the optimal matching result to the second prediction result corresponding to the second detection head; determine multiple groups of prediction results based on the first prediction result and the updated second prediction result.

[0099] Multiple prediction sets are generated by the model's different detection heads (the first and second heads), including multiple predicted boxes and predicted categories. The cost matrix is ​​constructed during model training to quantify the cost of matching the model's predicted boxes with the ground-truth labeled boxes. Lower costs indicate more accurate matches.

[0100] Specifically, for each second prediction box generated by the second detection head, the matching cost between it and each real labeled box in the target image is calculated, for example, by calculating the distance or IOU (Intersection over Union) between the predicted box and the real box. These cost values ​​form a matrix, in which the rows represent the real labeled boxes, the columns represent the second prediction boxes, and each element represents the corresponding matching cost; the Hungarian algorithm is used to find the best matching scheme based on the cost matrix to ensure that each real labeled box matches a prediction box and the total cost is minimized; after determining the optimal matching result, the successfully matched second prediction box is replaced with its corresponding complete prediction result, including the box's location information and category prediction; the prediction result of the first detection head is merged with the updated prediction result of the second detection head to form multiple groups of prediction results.

[0101] Step S208: determine the losses corresponding to the multiple sets of prediction results, and iteratively update the parameters of the initial target detection model based on the losses. When the stopping condition is met, stop the iteration to obtain the target detection model.

[0102] In step S208, the distance loss between each predicted box and the true box can be calculated, for example, using IoU loss or bounding box regression loss, as well as classification loss for category prediction. It should be noted that for a one-to-many detection head, the losses between multiple predicted boxes and the true box can be calculated, and a preset strategy (such as selecting the minimum loss value or average loss) can be used to determine their final loss contribution.

[0103] In some embodiments of the present application, the loss includes a first loss and a second loss corresponding to the first detection head and the second detection head, respectively; the losses corresponding to multiple groups of prediction results can be determined by the following steps: determining the first loss between the first prediction result of the first detection head and the true annotation box of the target image, and determining the second loss between the second prediction result of the second detection head and the true annotation box; obtaining the first weight of the first detection head and the second weight of the second detection head; determining the losses corresponding to multiple groups of prediction results based on the first loss, the first weight, the second loss and the second weight.

[0104] After determining the losses corresponding to multiple groups of prediction results based on the first loss, the first weight, the second loss, and the second weight, the following steps can also be performed: obtaining the attenuation factor corresponding to the current training iteration round, and updating the first weight and the second weight respectively based on the attenuation factor, the first loss, and the second loss, wherein the attenuation factor is used to quantify the degree of influence of the first weight of the previous iteration round and the second weight of the previous iteration round on the first weight of the current iteration round and the second weight of the current iteration round, respectively, and the current iteration round is the next iteration round of the previous iteration round.

[0105] During the inference phase of the target detection model, the first weight and the second weight are compared, and the detection head corresponding to the minimum weight indicated by the comparison result is used as the main detection head, and the detection heads other than the main detection head are removed.

[0106] In order to enhance the generalization of the multi-detection head assisted training strategy, in some embodiments of the present application, a dynamic adjustment module of the detection head based on the exponential moving average can be used to enable the model to adaptively learn the importance of multiple detection heads in different data sets. Taking the model with two detection heads as an example, the two detection heads are respectively denoted as detection head A and detection head B. During the training process, each detection head will generate a corresponding loss, denoted as L a and L b Initially, the loss ratios of the two detection heads are set to 1, i.e., α a =α b = 1. Total loss L total It is obtained by the weighted sum of the losses of the two detection heads, and the formula is as follows:

[0107] L total =L a ×α a +L b ×α b (Formula 1)

[0108] In order to dynamically adjust the weight of the detection head, the EMA strategy is adopted, that is, in each training iteration, the weight of the detection head is updated according to the current loss, where the EMA update formula is:

[0109]

[0110] In the above formulas 1 to 3, α a is the first weight, α b is the second weight, L a is the first loss, L b is the second loss, and λ is the attenuation factor.

[0111] In the inference phase, the module first compares the weights α of the two detection heads a and α b ,The detection head with a smaller weight has a smaller loss on the output of the entire dataset, indicating that the detection head can better fit the current dataset.,Therefore, the detection head with a smaller weight can be selected as the,primary detection head and the other detection heads can be removed.

[0112] It should be noted that the loss can also include a third loss corresponding to the query score; the third loss is determined in the following way: the decoder of the initial target detection model is used to generate a query score prediction value for each feature vector, and the true query score value of each feature vector is determined based on the true annotation box information of the target image; the third loss is determined based on the query score prediction value and the true query score value.

[0113] In order to ensure that the matching relationship between each feature vector and the target query (i.e., the query vector) is consistent, the query score can be added to the loss calculation. For the loss function of the query score, considering that the query score in the RT-DETR model is only responsible for selecting the largest K feature vectors, the cross entropy loss can be selected as the loss function. It is expected that the feature vector selected by the query score will remain as unchanged as possible through training. The loss function needs to calculate the difference between the predicted value and the actual value of the query score. Its calculation formula is as follows:

[0114]

[0115] In the above formula 4, N represents the total number of feature vectors (such as 8400), y i represents the true query score of the i-th feature vector, Represents the query score predicted by the model.

[0116] In some embodiments of the present application, the true value of the query score of each feature vector can be determined by the following steps: dividing the target image into multiple grids, where each grid corresponds to a feature vector; obtaining the coordinate information of all annotation boxes from the true annotation box information; and determining the true value of the query score of each feature vector based on the positional relationship between the coordinate information and each grid.

[0117] The true value of the query score of each feature vector is determined based on the positional relationship between the coordinate information and each grid: when the positional relationship indicates that the target grid and the annotation box do not have an intersection, the true value of the query score corresponding to the target grid is determined to be zero, where the target grid is any one of the multiple grids; when the positional relationship indicates that the target grid and the annotation box have an intersection, the true value of the query score corresponding to the target grid is determined based on the offset between the center point of the target grid and the center point of the annotation box.

[0118] Specifically, for the true value of the query score, the grid outside each annotation box can be set to 0, and the value of the center point inside the annotation box is 1. The closer the grid is to the center of the annotation box, the higher the query score. The calculation formula is as follows:

[0119]

[0120] In the above formula 5, c is the coordinate of the current grid, box is the center point of the annotation box, Δx and Δy are the offsets between the center point of the annotation box and the center of the grid, and w and h are the width and height of the annotation box.

[0121] After determining the third loss, the following steps may be performed: obtaining the total number of feature vectors and scaling the third loss using the total number; or scaling the query score prediction value to within a preset range.

[0122] Visualization of the predicted query score reveals that it is not between [0, 1]. Since the model's primary task is object detection, and query score training is an auxiliary to this task, the greater the deviation between the predicted query score and [0, 1], the greater the calculated loss, causing the model to focus more on the query score and less on the detection task, affecting the model's detection performance. To address this issue, the query score loss needs to be kept within a small range. The following strategies can be used:

[0123] (1) When calculating the loss, the predicted value of the query score is scaled to between [0, 1];

[0124] (2) The query score loss is scaled by the total number of feature vectors (e.g., 8400).

[0125] Through the above steps S202 to S208, a multi-detection head mechanism is introduced to generate multiple sets of prediction results based on multiple query vectors, thereby achieving the purpose of enhancing the detection performance of the model, thereby effectively improving the convergence speed of the model without introducing additional training parameters. The technical effect solves the technical problem that the auxiliary training method adopted by the relevant technology introduces more training parameters, brings a greater burden to the training of the model, and leads to low model training efficiency.

[0126] Figure 3 is a flow chart of another target detection method according to an embodiment of the present application. Figure 3 As shown, the method includes:

[0127] Step S302: Acquire an image to be detected, wherein the image to be detected includes an object to be detected.

[0128] In step S304, a target detection model is used to predict the image to be detected to obtain a prediction result corresponding to the target to be detected, wherein the target detection model is obtained based on the joint training of the first detection head and the second detection head, and the prediction result is output by the target detection head of the target detection model. The target detection head is the detection head with the largest weight among the first detection head and the second detection head. The first detection head generates a corresponding prediction result for each feature vector, and the second detection head generates multiple corresponding prediction results for each feature vector.

[0129] In the above step S304, the feature vector is a vector obtained by the target detection model through feature extraction of the image to be detected.

[0130] It should be noted that Figure 3 The preferred implementation of the embodiment shown can be found in Figure 2 The corresponding solutions in the illustrated embodiment will not be described in detail here.

[0131] To facilitate understanding of the technical solution of this application, the following is an explanation with reference to some specific embodiments. Specifically, the following steps are included:

[0132] Step 1: The user inputs a sample image or dataset, and the dataset is enhanced using the adaptive probabilistic mosaic data augmentation method based on target distribution to improve the quality of images captured by surveillance cameras.

[0133] Step 2: The backbone network and neck network of the RT-DETR model generate 8,400 feature vectors based on the image. The decoder then generates corresponding scores (query scores) based on these feature vectors and selects the largest 300 as the input of the decoding layer.

[0134] Step 3: The decoding layer decodes these 300 feature vectors and passes the results to multiple detection heads to generate multiple sets of prediction boxes and categories.

[0135] Step 4: The Hungarian matching algorithm performs a one-to-one match between each set of detection results and the annotation box, and outputs the final prediction results for each set.

[0136] Step 5: In the loss calculation module, the true query scores are generated based on the labeled data. The predicted query scores are normalized using softmax, and the loss of the query scores is calculated. For each set of prediction loss values, the dynamic adjustment module calculates the loss for multiple sets of prediction results. Finally, the backpropagation algorithm is used to update the model parameters.

[0137] Step 6: Count the weight of each detection head by exponential moving average for training in the next cycle.

[0138] Step 7: In the inference phase, select the detection head with the smallest weight as the main detection head, remove other detection heads, and perform inference on the input data.

[0139] Based on the multi-detection head solution and combined with the idea of ​​dense one-to-one matching, this application designs a multi-detection head auxiliary training method with a small number of parameters to effectively improve the detection effect of the RT-DETR model.

[0140] Most related technologies fix a single detection head as the primary detection head, and the weights of multiple detection heads are fixed, which may not be applicable to other tasks. To address this, this application designs a dynamic detection head adjustment module based on exponential moving average. This module can adaptively learn the importance of each detection head, allowing for flexible and effective training in different tasks and selecting the most effective detection head for model inference.

[0141] The Mosaic data enhancement method randomly selects four images for stitching, but this method does not take into account the distribution of targets in the image. When the camera is far away, the mosaic stitching and cropping operation may ignore most of the targets in the image. For example, in a gas station scene, some cameras are located far away from the target, making the targets in the captured images smaller and concentrated in a certain area, which has a negative impact on the training of the detection model. To solve this problem, this application proposes an adaptive probabilistic Mosaic data enhancement method based on target distribution. By cropping out irrelevant background areas in the image, the stitched image is more focused on the target itself, reducing the negative impact of background interference on model training, and can more effectively utilize the effective information in the image, improve the effect of data enhancement, and make the model more adaptable and practical in monitoring scenarios.

[0142] In RT-DETR, the encoder outputs 8400 feature vectors, and the encoder calculates the category prediction score of each feature vector based on these 8400 feature vectors. The decoder selects some of the target queries (300) for detection based on the prediction scores. In the decoder and subsequent Hungarian matching and loss calculation, only these queries are processed. This method effectively reduces the amount of calculation and improves the training speed of the model. However, the above method will lead to a problem: the selected queries may change in different training stages, which causes the queries processed by the decoder to change, that is, the decoder may learn the offset information of the predefined anchor box based on different queries in different training stages. In this regard, the present application improves the loss function of the RT-DETR model, hoping to make the feature vectors selected by the query score as unchanged as possible through training, so as to speed up the convergence of the RT-DETR model.

[0143] Figure 7FIG. 1 is a schematic diagram of the training acceleration effect of a target detection model training method according to an embodiment of the present application. Figure 7 The training acceleration effect diagram shows the comparison of the training acceleration scheme used in this application with the baseline model in terms of training performance. The horizontal axis of the diagram represents the training cycles (epochs), and the vertical axis represents the average precision (AP) at a specific IoU threshold mAP@.5, expressed as a percentage. mAP@.5 refers to the average precision of the model when the IoU threshold is set to 0.5. Figure 7 It is shown in the figure that the training acceleration scheme of this application only takes 44 cycles to achieve the training effect of the baseline in 60 cycles. This means that the use of this scheme can achieve the same detection accuracy in a shorter time, saving about 30% of the training time and significantly improving the training speed.

[0144] Figure 8 is a structural diagram of a training device for a target detection model according to an embodiment of the present application, such as Figure 8 As shown, the device includes:

[0145] An acquisition module 802 is configured to acquire a target image for training, wherein the target image includes a detection target;

[0146] A processing module 804 is configured to extract features from the target image using the initial target detection model to obtain a plurality of query vectors, wherein the query vectors are used to locate the detected target;

[0147] A prediction module 806 is configured to use the first detection head and the second detection head of the initial object detection model to predict the multiple query vectors, respectively, to obtain multiple sets of prediction results, wherein the first detection head generates one corresponding prediction result for each query vector, and the second detection head generates multiple corresponding prediction results for each query vector;

[0148] The determination module 808 is used to determine the losses corresponding to the multiple sets of prediction results, and iteratively update the parameters of the initial target detection model based on the losses. When the stopping condition is met, the iteration is stopped to obtain the target detection model.

[0149] It should be noted that Figure 8 The training setup for the target detection model shown is used to perform Figure 2 The training method of the target detection model shown in Figure 2 The explanations for the training methods of the target detection model in

[15] also apply to Figure 8 The training device of the target detection model shown is not described here in detail.

[0150] An embodiment of the present application also provides an electronic device, which includes a memory and a processor, wherein the memory is used to store program instructions; the processor is connected to the memory and is used to execute the steps of the target detection model training method or target detection method in each embodiment of the present application.

[0151] An embodiment of the present application also provides a non-volatile storage medium, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the steps of the target detection model training method or the target detection method in each embodiment of the present application by running the computer program.

[0152] An embodiment of the present application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the target detection model training method or target detection method in each embodiment of the present application.

[0153] An embodiment of the present application also provides a computer program, which, when executed by a processor, implements the steps of the target detection model training method or the target detection method in each embodiment of the present application.

[0154] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0155] In the above embodiments of the present application, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0156] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of the units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0157] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0158] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0159] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0160] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for training a target detection model, characterized in that: include: Acquire a target image for training, wherein the target image includes a detection target; Performing feature extraction on the target image using an initial target detection model to obtain a plurality of query vectors, wherein the query vectors are used to locate the detection target; Using the first detection head and the second detection head of the initial object detection model to predict the multiple query vectors respectively, to obtain multiple groups of prediction results, wherein the first detection head generates a corresponding prediction result for each query vector, and the second detection head generates multiple corresponding prediction results for each query vector; Determine the losses corresponding to the multiple sets of prediction results, and iteratively update the parameters of the initial target detection model based on the losses. When a stopping condition is met, stop the iteration to obtain the target detection model.

2. The method according to claim 1, characterized in that The first detection head and the second detection head of the initial object detection model are used to predict the multiple query vectors respectively to obtain multiple groups of prediction results, including: Inputting the multiple query vectors into the first detection head respectively to obtain multiple first prediction results, wherein the first prediction results include a first prediction box and a first prediction category, and each query vector corresponds to a first prediction result; Inputting the multiple query vectors into the second detection head respectively to obtain multiple second prediction results corresponding to each query vector, wherein the second prediction results include a second prediction box and a second prediction category; The multiple groups of prediction results are determined according to the first prediction result and the second prediction result.

3. The method according to claim 2, characterized in that Determining the multiple groups of prediction results based on the first prediction result and the second prediction result includes: Constructing a cost matrix using the true labeled box corresponding to the target image as a row and all the second predicted boxes as columns, wherein each element of the cost matrix represents the matching cost between the corresponding true labeled box and the second predicted box, and the true labeled box is the true labeled box of the detected object; Performing a one-to-one matching between the true labeled box and the second predicted box according to the cost matrix to obtain an optimal matching result, wherein the matching cost corresponding to the optimal matching result is the smallest; Updating the second predicted frame in the optimal matching result that matches the true labeled frame to the second predicted result corresponding to the second detection head; The multiple groups of prediction results are determined based on the first prediction result and the updated second prediction result.

4. The method according to claim 1, wherein The loss includes a first loss and a second loss corresponding to the first detection head and the second detection head respectively; Determining the losses corresponding to the multiple sets of prediction results includes: Determining a first loss between a first prediction result of the first detection head and a true labeled box of the target image, and determining a second loss between a second prediction result of the second detection head and the true labeled box; Obtaining a first weight of the first detection head and a second weight of the second detection head; Determine the losses corresponding to the multiple groups of prediction results based on the first loss, the first weight, the second loss, and the second weight.

5. The method according to claim 4, characterized in that After determining the losses corresponding to the multiple groups of prediction results based on the first loss, the first weight, the second loss and the second weight, the method further includes: obtaining an attenuation factor corresponding to the current training iteration round, and updating the first weight and the second weight respectively based on the attenuation factor, the first loss and the second loss, wherein the attenuation factor is used to quantify the degree of influence of the first weight of the previous iteration round and the second weight of the previous iteration round on the first weight of the current iteration round and the second weight of the current iteration round, respectively, and the current iteration round is the next iteration round of the previous iteration round.

6. The method according to claim 1, characterized in that The initial target detection model is used to extract features from the target image to obtain multiple query vectors, including: Generate a plurality of feature vectors corresponding to the target image using an encoder of the initial target detection model, wherein the plurality of feature vectors are used to cover features of different positions and detail levels of the target image; The decoder of the initial target detection model is used to determine query scores corresponding to the multiple feature vectors respectively, and the multiple query vectors are determined from the multiple feature vectors based on the query scores, wherein the query scores are used to quantify the possibility of the feature vector matching the detection target.

7. The method according to claim 6, characterized in that The loss includes a third loss corresponding to the query score; the third loss is determined by: Generate a query score prediction value for each of the feature vectors using a decoder of the initial target detection model, and determine a true query score value for each of the feature vectors based on true annotation box information of the target image; The third loss is determined according to the query score prediction value and the query score true value.

8. The method according to claim 7, characterized in that Determining a true value of the query score of each of the feature vectors based on the true annotation box information of the target image includes: Dividing the target image into a plurality of grids, wherein each grid corresponds to a feature vector; Acquire coordinate information of all annotation boxes from the real annotation box information; The true value of the query score of each of the feature vectors is determined according to the positional relationship between the coordinate information and each grid.

9. The method according to claim 8, characterized in that Determining a true value of the query score of each of the feature vectors based on a positional relationship between the coordinate information and each grid, including: When the position relationship indicates that the target grid and the marked box do not have an intersection, determining that a true value of the query score corresponding to the target grid is zero, wherein the target grid is any one of the multiple grids; When the positional relationship indicates that the target grid and the annotation box have an intersection, a true value of the query score corresponding to the target grid is determined according to an offset between a center point of the target grid and a center point of the annotation box.

10. The method according to claim 7, characterized in that After determining the third loss, the method further includes: Obtaining the total number of the feature vectors and using the total number to scale the third loss; or scaling the query score prediction value to within a preset interval.

11. The method according to claim 1, wherein After acquiring the target image for training, the method further includes: Determining first size information of a minimum bounding rectangle corresponding to all real annotation boxes in the target image, wherein the first size information includes length information and width information; When a ratio of the first size information to the second size information of the target image meets a preset threshold, cropping the target image according to the minimum bounding rectangle to obtain a sub-target image; The sub-target image is spliced ​​with the random image to obtain an updated target image.

12. The method according to claim 11, characterized in that The sub-target image is concatenated with a random image, comprising: splicing the sub-target image with the target image before clipping and a first number of random images according to a first probability; The sub-target image is spliced ​​with a second number of random images according to a second probability, wherein a cumulative sum of the first probability and the second probability is one, and the second number is greater than the first number.

13. The method according to claim 5, characterized in that During the inference phase of the target detection model, the first weight and the second weight are compared, a detection head corresponding to the minimum weight indicated by the comparison result is used as a main detection head, and detection heads other than the main detection head are removed.

14. The method according to claim 6, characterized in that The decoder includes a first feedforward neural network and a second feedforward neural network, wherein the first feedforward neural network is connected to the first detection head, and the second feedforward neural network is connected to the second detection head.

15. A method for target detection, characterized in that: include: Acquire an image to be detected, wherein the image to be detected includes a target to be detected; A target detection model is used to predict the image to be detected to obtain a prediction result corresponding to the target to be detected, wherein the target detection model is obtained based on the joint training of the first detection head and the second detection head, and the prediction result is output by the target detection head of the target detection model. The target detection head is the detection head with the largest weight between the first detection head and the second detection head. The first detection head generates a corresponding prediction result for each feature vector, and the second detection head generates multiple corresponding prediction results for each feature vector.

16. A training device for a target detection model, characterized in that: include: An acquisition module, configured to acquire a target image for training, wherein the target image includes a detection target; a processing module, configured to extract features from the target image using an initial target detection model to obtain a plurality of query vectors, wherein the query vectors are used to locate the detection target; a prediction module, configured to use the first detection head and the second detection head of the initial object detection model to respectively predict the multiple query vectors to obtain multiple sets of prediction results, wherein the first detection head generates one corresponding prediction result for each query vector, and the second detection head generates multiple corresponding prediction results for each query vector; A determination module is used to determine the losses corresponding to the multiple groups of prediction results, and iteratively update the parameters of the initial target detection model based on the losses. When a stopping condition is met, the iteration is stopped to obtain the target detection model.

17. An electronic device, characterized in that: include: A memory and a processor, the memory being used to store program instructions; the processor being connected to the memory and being used to execute the training method for implementing the target detection model described in any one of claims 1 to 14, or the target detection method of claim 15.

18. A non-volatile storage medium, characterized in that: The non-volatile storage medium includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the target detection model training method described in any one of claims 1 to 14, or executes the target detection method in claim 15 by running the computer program.

19. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by a processor, they implement the target detection model training method described in any one of claims 1 to 14, or execute the target detection method in claim 15.

Images