Training of bounding box distribution model, target detection method and device

Abstract

Embodiments of the present specification disclose a training method of a bounding box distribution model to solve the uncertainty problem of network prediction of an existing target detection model and improve the accuracy of target detection. The method comprises: obtaining a sample anchor box image set, each sample anchor box image comprising an initial anchor box of a target object region in a sample image and a corresponding real bounding box; inputting the sample anchor box image set into an initial bounding box distribution model for model iterative training until a convergence condition of the bounding box distribution model is met, to obtain a trained bounding box distribution model; the initial bounding box distribution model comprises a variational autoencoder, and each model iterative training of the bounding box distribution model comprises: obtaining a sample prediction image comprising a predicted bounding box corresponding to the initial anchor box by using the variational autoencoder; and adjusting model parameters of the bounding box distribution model according to the initial anchor box, the predicted bounding box, the real bounding box and a preset loss function of the bounding box distribution model.

Description

Technical Field

[0001] This application relates to the field of computer machine learning technology, and in particular to a training method for a bounding box distribution model, a training method for a target classification model, a target detection method, and an apparatus. Background Technology

[0002] Deep learning (DL) is a new research direction in the field of machine learning (ML). It has been introduced into machine learning to bring it closer to its original goal: artificial intelligence (AI). Deep learning learns the inherent patterns and hierarchical representations of sample data. The information gained during this learning process greatly aids in the interpretation of data such as text, images, and sound. Its ultimate goal is to enable machines to possess analytical and learning capabilities like humans, capable of recognizing data such as text, images, and sound. Deep learning is a complex machine learning algorithm that has achieved results in speech and image recognition far exceeding previous related technologies. Common deep learning models include fully connected (FC) networks, convolutional neural networks (CNNs), and recurrent neural networks (RNNs).

[0003] Uncertainty in deep learning model predictions includes arbitrary uncertainty and knowledge uncertainty, with modeling and discrimination being particularly critical. These two types of uncertainty are especially pronounced in object detection because object detection models face knowledge uncertainty between training and prediction data, and arbitrary uncertainty in the labeling of training data. Current technologies often rely on multi-model approaches to address these uncertainties, requiring significant computational resources and numerous network parameters, inevitably impacting the performance of object detection models. Since object detection tasks typically demand fast prediction times, existing models cannot adequately address these uncertainties, resulting in low accuracy. Summary of the Invention

[0004] The purpose of the embodiments in this specification is to provide a training method for a bounding box distribution model, a training method for a target classification model, a target detection method and apparatus, so as to solve the uncertainty problem of network prediction in existing target detection models and improve the accuracy of target detection.

[0005] To achieve the above objectives, the embodiments in this specification adopt the following technical solutions:

[0006] Firstly, a training method for a bounding box distribution model is provided, including:

[0007] Obtain a set of sample anchor box images, where each sample anchor box image includes the initial anchor box of the target object region in the sample image and the corresponding ground truth bounding box;

[0008] The sample anchor box image set is input into the initial bounding box distribution model for iterative training until the convergence condition of the bounding box distribution model is met, thus obtaining the trained bounding box distribution model.

[0009] The initial bounding box distribution model includes a variational autoencoder, and the specific steps for training the bounding box distribution model in each iteration include:

[0010] The variational autoencoder is used to fit the distribution of the initial anchor boxes in the sample anchor box image. Based on the probability distribution obtained after the distribution fitting, sampling and reconstruction processing is performed to obtain a sample prediction image including the predicted bounding boxes corresponding to the initial anchor boxes. The model parameters of the bounding box distribution model are adjusted according to the initial anchor boxes, the predicted bounding boxes corresponding to the initial anchor boxes, the ground truth bounding boxes corresponding to the initial anchor boxes, and the loss function of the preset bounding box distribution model. The loss function includes a first loss function for characterizing the difference between the predicted bounding boxes and the initial anchor boxes, and a second loss function for characterizing the difference between the predicted bounding boxes and the ground truth bounding boxes.

[0011] Secondly, a training method for a target classification model is provided, including:

[0012] A sample prediction image set is obtained, each sample prediction image including the predicted bounding box of the target object region in the sample image and the corresponding true classification information of the target object. The sample prediction image set is obtained by inputting the sample anchor box image set into a pre-trained bounding box distribution model, which is trained according to the training method of the bounding box distribution model as described in the first aspect.

[0013] The sample prediction image set is input into the initial target classification model for iterative training until the convergence condition of the target classification model is met, thus obtaining the trained target classification model.

[0014] Thirdly, a target detection method is provided, including:

[0015] The image to be detected is input into the target detection head model to obtain the initial image to be detected, which includes the initial anchor box of the target object region in the image to be detected.

[0016] The initial image to be detected is input into a pre-trained bounding box distribution model to obtain a predicted image to be detected including the predicted bounding boxes corresponding to the initial anchor boxes, wherein the bounding box distribution model is trained according to the training method of the bounding box distribution model as described in the first aspect.

[0017] Fourthly, a target detection device is provided, comprising:

[0018] The extraction module is used to input the image to be detected into the target detection head model to obtain an initial image to be detected, wherein the initial image to be detected includes the initial anchor box of the target object region in the image to be detected;

[0019] The distribution detection module is used to input the initial image to be detected into a pre-trained bounding box distribution model to obtain a predicted image to be detected including the predicted bounding boxes corresponding to the initial anchor boxes, wherein the bounding box distribution model is trained according to the training method of the bounding box distribution model as described in the first aspect.

[0020] Fifthly, an electronic device is provided, comprising:

[0021] processor;

[0022] Memory used to store the processor's executable instructions;

[0023] The processor is configured to execute the instructions to implement the method as described in the first aspect, or the second aspect, or the third aspect.

[0024] A sixth aspect provides a computer-readable storage medium that, when instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to perform the method as described in the first aspect, or the second aspect, or the third aspect.

[0025] The bounding box distribution model training scheme provided in the embodiments of this specification proposes a bounding box distribution model based on a variational autoencoder. During the training process, the initial bounding box distribution model used includes a variational autoencoder. The variational autoencoder is used to fit the distribution of the initial anchor boxes in the sample anchor box image. Based on the probability distribution obtained after distribution fitting, sampling and reconstruction processing is performed to obtain a sample prediction image including the predicted bounding box. The variational autoencoder no longer learns the individual samples, but rather the distribution of the samples. Through iterative training, the predicted bounding boxes of the sample prediction images output by the bounding box distribution model are fitted to the corresponding real bounding boxes, effectively solving the uncertainty problem of network prediction. This enables the trained bounding box distribution model to accurately detect the position and size of targets in the image during target detection, thereby improving the accuracy of target detection.

[0026] The target classification model training method provided in this specification is based on a pre-trained bounding box distribution model to obtain a sample prediction image set. Each sample prediction image includes the predicted bounding box of the target object region in the sample image and the corresponding true classification information of the target object. The sample prediction image set is used to train a target classification model for target detection. Since the bounding box distribution model based on variational autoencoder can accurately detect the position and size of the target in the image, the target classification model trained using the sample prediction image set can accurately identify and classify the detected target.

[0027] The target detection scheme provided in the embodiments of this specification first uses a target detection head model to obtain an initial target image containing initial anchor boxes of the target object region based on the input target image. Then, the initial target image is input into a pre-trained bounding box distribution model. The bounding box distribution model based on variational autoencoder is used to obtain a predicted target image containing the predicted bounding boxes corresponding to the initial anchor boxes. The bounding box distribution model based on variational autoencoder learns the distribution of samples and iteratively trains the model to fit the predicted bounding boxes of the output predicted target image to the corresponding real bounding boxes, effectively solving the uncertainty problem of network prediction. Therefore, it can accurately detect the position and size of the target in the image, thereby improving the accuracy of target detection. Attached Figure Description

[0028] The accompanying drawings, which are included to provide a further understanding of this specification and form part of this specification, illustrate exemplary embodiments and are used to explain this specification, but do not constitute an undue limitation thereof. In the drawings:

[0029] Figure 1 A flowchart illustrating a training method for a bounding box distribution model provided as an embodiment of this specification;

[0030] Figure 2 A schematic diagram illustrating the operation of a variational automatic encoder provided for one embodiment of this specification;

[0031] Figure 3 A schematic diagram of a network structure for implementing a training method for a bounding box distribution model, provided as an embodiment of this specification;

[0032] Figure 4 A schematic flowchart of a target detection method provided for one embodiment of this specification;

[0033] Figure 5 A schematic diagram of a training device for a bounding box distribution model provided as an embodiment of this specification;

[0034] Figure 6A schematic diagram of a target detection device provided for one embodiment of this specification;

[0035] Figure 7 This is a schematic diagram of an electronic device provided as an embodiment of the present specification. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0037] The terms "first," "second," etc., used in this specification and claims are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this specification can be implemented in orders other than those illustrated or described herein. Furthermore, in this specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0038] In object detection scenarios, object detection models face uncertainty issues in network prediction. This includes knowledge uncertainty between training and prediction data, and arbitrary uncertainty in the labeling of training data. Current technologies address these uncertainties by reducing the performance of object detection models. However, object detection tasks typically require fast prediction times, making it difficult for existing models to effectively resolve these uncertainties, resulting in low accuracy.

[0039] In view of the above problems, this application proposes the following technical concept:

[0040] The object detection task has three main objectives: (1) to detect the location of objects in an image, where multiple objects may exist in the same image; (2) to detect the size of the objects, which is usually a rectangular box that just surrounds the object; and (3) to identify and classify the detected objects. Object detection models typically include a bounding box distribution model for achieving the main objectives (1) and (2), and an object classification model for achieving the main objective (3).

[0041] This application proposes an improved approach to bounding box distribution models, providing a variational autoencoder-based bounding box distribution model for object detection. By utilizing the variational autoencoder to learn the distribution of samples, and through iterative training, the predicted bounding boxes of the sample prediction images output by the bounding box distribution model are fitted to the corresponding ground truth bounding boxes. This effectively solves the uncertainty problem of network prediction, enabling the trained bounding box distribution model to accurately detect the position and size of objects in images. Furthermore, using the variational autoencoder-based bounding box distribution model for training object classification models, and for object detection, can improve the accuracy of object detection.

[0042] In view of this, the embodiments of this specification aim to provide a method for training a bounding box distribution model, thereby obtaining a bounding box distribution model based on a variational autoencoder for object detection; a method for training an object classification model, thereby obtaining an object classification model for object detection; and an object detection method based on the trained bounding box distribution model, in order to solve the uncertainty problem of network prediction in existing object detection models and improve the accuracy of object detection.

[0043] It should be understood that the training methods for the bounding box distribution model, the object classification model, and the object detection method provided in the embodiments of this specification can all be executed by an electronic device or by software installed in an electronic device, specifically by a terminal device or a server device. The training methods for the bounding box distribution model, the object classification model, and the object detection method can be executed by the same electronic device, or they can be executed by different electronic devices.

[0044] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0045] In object detection, the bounding box B(x,y,w,h,p) represents the positional information in the image, where (x,y) represents the center coordinates of bounding box B, (w,h) represents the length and width of bounding box B, and p represents the classification information corresponding to bounding box B. In the embodiments of this specification, the initial anchor box, predicted bounding box, and ground truth bounding box are the existence forms of the bounding box at different stages, all of which include the above five dimensions of information.

[0046] Please refer to Figure 1 The above is a flowchart illustrating a training method for a bounding box distribution model, provided as an embodiment of this specification. The method may include:

[0047] S101. Obtain a set of sample anchor box images. Each sample anchor box image includes the initial anchor box of the target object region in the sample image and the corresponding ground truth bounding box.

[0048] In one alternative implementation, a set of sample images can be input into the object detection head model to obtain an initial set of sample anchor boxes. The initial set of sample anchor boxes includes multiple initial sample anchor boxes, and each initial sample anchor box includes an initial anchor box of the target object region in the sample image.

[0049] Specifically, the object detection head model can employ a two-stage object detection algorithm. Simultaneously, for iterative model training, the sample image set needs to be labeled to obtain the ground truth bounding box corresponding to each initial anchor box. Labeling can be done manually, combining the initial anchor box and its corresponding ground truth bounding box from each initial sample anchor box image to form a sample anchor box image set. Specifically, an open-source image set can be used as the sample image set; this open-source dataset can be any image set used for object detection training.

[0050] Commonly used object detection algorithms can be divided into two categories: one is the R-CNN family of algorithms based on Region Proposal, including R-CNN, Fast R-CNN, Faster R-CNN, etc.; the other is one-stage object detection algorithms, including YOLO, SSD, etc. Two-stage object detection algorithms require the algorithm to first generate candidate boxes, and then classify and regress the candidate boxes.

[0051] Taking Fast R-CNN as an example, the structure and processing flow of the two-stage object detection algorithm head are explained: Selective Search is performed on the input image to obtain 1K to 2K candidate regions; the image is input into the DeepConvNet network to obtain a feature map, and the candidate regions are projected onto the feature map to obtain the corresponding feature matrix; each feature matrix is ​​scaled to a fixed size (7*7) feature map through the RoI pooling layer; and the structural features (feature vector) of the RoI (region of interest) are obtained through two fully connected layers FCs (FC6, FC7).

[0052] The structural features of RoIs generated using a two-stage object detection algorithm can be represented as {{p i},{b i}}, i = 1, ..., N, where b i p represents the initial anchor box representing the target object region in the image. i This represents the classification information of the target object corresponding to the initial anchor box. Initial anchor box b i This can be understood as multiple prior boxes with different aspect ratios predefined by the object detection model, centered on the anchor point, with the initial anchor box b. i Specifically, it refers to the initial anchor frame b. iLocation information, including center coordinates and length and width.

[0053] Each anchor point is an element on the feature map of the last convolutional layer of the backbone network. The initial anchor boxes are a series of boxes generated with each anchor point as the center. The specific number of boxes corresponding to one anchor point is controlled by two parameters: `scales` and `aspect ratios`. `scales` refers to the pixel size of the longest side of the box for each shape, while `aspect ratios` refers to the specific shapes and describes the aspect ratio. For example, `scales[8,16,32]` and `aspect ratios[0.5,1,2]` mean that one anchor will generate 9 initial anchor boxes. Note that the coordinates of the initial anchor boxes correspond to the original image size, while the feature map is scaled down many times compared to the original image.

[0054] In the sample anchor frame image set, each sample anchor frame image may include the initial anchor frame of the target object region in the sample image and the corresponding ground truth bounding box. The initial anchor frames of multiple sample anchor frames included in the sample anchor frame image set can be represented as {b i}(i=1,…,N).

[0055] S102. Input the sample anchor box image set into the initial bounding box distribution model for iterative training until the convergence condition of the bounding box distribution model is met, and obtain the trained bounding box distribution model.

[0056] When training a bounding box distribution model using a set of sample anchor box images, each time the initial anchor boxes {b} can be obtained from multiple sample anchor box images. i The initial anchor boxes {b1,…,b2} are randomly sampled from m mini-batch anchor box images in} (i=1,…,N). m}, correspondingly, the initial anchor frame {b1,…,b m The corresponding true bounding box can be represented as b ground ~p ground (b ground The true bounding box refers to the location information in a manually labeled sample image, and the probability distribution p of the true bounding box is... ground (b ground It can follow a Gaussian distribution.

[0057] The initial bounding box distribution model may include a variational autoencoder, and the training of the bounding box distribution model in each iteration may specifically include the following steps:

[0058] S1021. Use a variational autoencoder to fit the distribution of the initial anchor boxes in the sample anchor box image. Based on the probability distribution obtained after the distribution fitting, perform sampling reconstruction processing to obtain a sample prediction image including the prediction bounding box corresponding to the initial anchor box.

[0059] In the embodiments described in this specification, the variational autoencoder (VAE) is structurally composed of an encoding network (also called a variational inference model) and a decoding network (also called a generative model). The encoding network q φ (z|b) generates the corresponding latent variable z from the initial anchor box b of the input sample anchor box image, and decodes the network p. θ (b estimation |z) Reconstruct the input from the latent variable z to generate the predicted bounding box b corresponding to the initial anchor box b. estimation .

[0060] The theoretical foundation of Variational Autoencoders (VAEs) is the Gaussian Mixture Model, which states that the distribution of any data can be viewed as a superposition of several Gaussian distributions. In essence, VAEs learn not just individual samples, but the patterns within them. The trained model not only reconstructs samples but also mimics their patterns. VAEs essentially change the distribution of samples during the network encoding process; "variational" can be understood as altering the distribution. "Learning the patterns of samples" specifically refers to learning the distribution of samples. Assuming the distribution function of the samples is known, a sample can be drawn from this distribution function and then propagated forward through the network's decoding layer to generate a new sample.

[0061] Specifically, such as Figure 2 As shown, the encoding network computes the initial anchor boxes for each input data (sample anchor box image) b = {b1, ..., b}. m The corresponding mean μ and variance σ 2 Then, sampling is performed from the latent variable space to obtain z = {z1, ..., ... m The new data (predicted bounding boxes corresponding to the initial anchor boxes) is generated through the decoding network. If the sampled data z from the latent variable space follows the probability distribution of the input data b, then the new data b generated based on the sampled data z... estimation It also follows the probability distribution of the input data b.

[0062] In the encoding network, two neural networks μ are constructed. i =f1(b i )and Calculate the mean and logarithmic variance, and select the fit. Instead of direct fitting Because Since it is always non-negative, it needs to be processed with an activation function, while fitting... No activation function is needed because it can be positive or negative. The encoding network can be a two-layer MLP (Multilayer Perceptron), with the second layer generating the mean and log-variance. The third output of the encoding network is latent variable sampling using reparameter resampling techniques. The decoding network can also be a two-layer MLP, using the sampled latent variables to reconstruct the input. Connecting the encoding and decoding networks together forms a variational autoencoder.

[0063] Specifically, S1021 may include the following steps:

[0064] S1021-a. Calculate the Gaussian mixture model parameters of the latent variables for the initial anchor boxes of each input sample anchor box image using the encoding network of the variational autoencoder. Fit the variational posterior distribution of the latent variables according to the Gaussian mixture model parameters. Sample the variational posterior distribution of the latent variables. The latent variables are used to describe the attributes of the initial anchor boxes of the sample anchor box images.

[0065] Gaussian mixture models (GMMs) can fit data distributions of arbitrary shapes using a mixture representation of the probability distributions of multidimensional Gaussian models. A GMM uses K (3–5) Gaussian models to characterize the features of each pixel in an image. The main parameters of a GMM are variance and mean. Latent variables encode certain attributes observable from the input. Considering all possible attributes, a distribution describing the input is ultimately obtained. For example, in facial data, these attributes might be facial expression, hairstyle, hair color, gender, etc. Using features including facial expression, hairstyle, hair color, and gender, the distribution describing the facial data can be recovered. This is achieved using an encoding network q... φ (z|b) can be used to estimate the Gaussian mixture model parameters of the latent variable z corresponding to the initial anchor box b of the sample anchor box image, namely the mean μ and variance σ. 2 Based on the Gaussian mixture model parameters (mean μ and variance σ) 2 The variational posterior distribution q of the latent variable z can be determined. φ (z|b), this distribution follows a Gaussian distribution and approximates the posterior distribution p of the latent variable z. θ (z|b estimation Furthermore, the reparameter recalculation technique can be used to refine the variational posterior distribution q of the latent variable z. φ (z|b) is used for sampling.

[0066] S1021-b: Using the decoding network of a variational autoencoder, a predicted bounding box corresponding to the initial anchor box is generated based on the sampled latent variables, resulting in a sample prediction image including the predicted bounding box corresponding to the initial anchor box.

[0067] Suppose there exists an initial anchor box b with a posterior distribution p of the latent variable z. θ (z|b estimation Furthermore, assuming that the distribution is an independent, multivariate Gaussian distribution, a generative model, i.e., a decoding network p, is trained. θ (b estimation |z), then it is possible to extract from distribution p θ (z|b estimation The sampled z generates b estimation Because the variational posterior distribution q of the latent variable z corresponding to the initial anchor frame b. φ (z|b) approximate posterior distribution p θ (z|b estimation Therefore, there is reason to believe that, based on the variational posterior distribution q of the latent variables... φ The latent variable z sampled from (z|b) is then used in the decoding network p of the variational autoencoder. θ (b estimation |z) can then generate the predicted bounding box b corresponding to the initial anchor box b. estimation .

[0068] S1022. Update the model parameters of the bounding box distribution model based on the initial anchor box, the predicted bounding box corresponding to the initial anchor box, the true bounding box corresponding to the initial anchor box, and the loss function of the preset bounding box distribution model. The loss function includes a first loss function for characterizing the difference between the predicted bounding box and the initial anchor box, and a second loss function for characterizing the difference between the predicted bounding box and the true bounding box.

[0069] In the embodiments of this specification, please refer to the network structure of a training method for implementing a bounding box distribution model. Figure 3 The system includes a variational autoencoder and a discriminant network. The variational autoencoder includes an encoding network and a decoding network. The discriminant network is used to distinguish the differences between the predicted bounding box and the ground truth bounding box. Specifically, S1022 may include the following steps:

[0070] S1022-a. Using the variational autoencoder, determine the loss value of the first loss function based on the reconstruction loss between the initial anchor box of the sample anchor box image and the predicted bounding box corresponding to the initial anchor box, and the first divergence between the variational posterior distribution of the latent variable and the prior distribution of the latent variable.

[0071] The first loss function can be considered as the loss function of a variational autoencoder. The reconstruction loss between the initial anchor box of the sample anchor box image and the predicted bounding box corresponding to the initial anchor box, and the first divergence between the variational posterior distribution of the latent variable and the prior distribution of the latent variable are determined. The loss value of the first loss function is then determined based on the reconstruction loss and the first divergence.

[0072] The reconstruction loss is used to calculate the similarity between the generated predicted bounding box and the initial anchor box. The first divergence can specifically be the KL (Kullback-Leibler) divergence, which measures the degree of difference between two distributions; the smaller the difference between the two distributions, the smaller the KL divergence, and vice versa; when the two distributions are identical, the KL divergence is 0.

[0073] S1022-b: Using a discriminant network, determine the loss value of the second loss function based on the second divergence between the conditional distribution of the predicted bounding box corresponding to the initial anchor box of the sample anchor box image and the probability distribution of the true bounding box, and the cross-union ratio loss between the predicted bounding box corresponding to the initial anchor box of the sample anchor box image and the true bounding box.

[0074] The second loss function can be considered as the loss function of the discriminant network. The discriminant network is used to distinguish between the predicted bounding boxes and the ground truth bounding boxes corresponding to the initial anchor boxes of the sample anchor box image. This determines the second divergence between the conditional distribution of the predicted bounding box and the probability distribution of the ground truth bounding box, as well as the intersection-union ratio (IUR) loss between the predicted and ground truth bounding boxes. The loss value of the second loss function is then determined based on the second divergence and the IUR loss.

[0075] Specifically, the discriminative network can distinguish between predicted and ground truth bounding boxes using two aspects: Intersection over Union (IoU) and KL divergence. IoU represents the ratio of the intersection to the union of two boxes, and it is a measure of the degree of overlap between them. Ideally, the boxes should completely overlap, i.e., IoU is 1. The IoU loss, IoULoss, can be calculated as follows: IoULoss = 1 - IoU.

[0076] S1022-c: Determine the loss value of the bounding box distribution model based on the loss value of the first loss function and the loss value of the second loss function, and adjust the model parameters of the bounding box distribution model based on the loss value of the bounding box distribution model.

[0077] The model parameters of the bounding box distribution model refer to the parameters used to characterize the structure of the bounding box distribution model, mainly including the parameters φ of the encoding network and the parameters θ of the decoding network. Specifically, the model parameters of the bounding box distribution model can include the network parameters of each layer in the variational autoencoder and the network parameters of each layer in the decision network. For each network layer, the network parameters can include, but are not limited to, the number of neurons in each network layer, the connection relationships between neurons in other network layers, and the connection weights.

[0078] In one alternative implementation, backpropagation can be used to adjust the model parameters of the bounding box distribution model. Specifically, based on the loss value of the bounding box distribution model, and with the goal of reducing the loss value, the network parameters of each layer in the discriminator network, decoder network, and encoder network are adjusted layer by layer through backpropagation, starting from the last layer of the discriminator network. It should be noted that since the discriminator network utilizes the conditional distribution of the predicted bounding boxes, it includes the parameters θ of the decoder network.

[0079] For example, starting from the last layer of the discriminator network, gradient descent is used to calculate the partial derivative of the loss value of the bounding box distribution model based on the structure of each layer in the discriminator, decoder, and encoder networks, as well as the connection relationships and connection weights between different layers. This yields the loss value of each layer, which is used to characterize the prediction differences caused by each layer. Then, with the goal of reducing the loss value of the bounding box distribution model, the network parameters of each layer are updated sequentially based on the loss values ​​of each layer.

[0080] In the embodiments of this specification, the loss function of the bounding box distribution model can be any appropriate form of loss function, which can be set according to actual needs. This specification does not impose specific limitations on this. Specifically, the loss value of the bounding box distribution model can be determined by weighted summation of the loss values ​​of the first loss function and the second loss function. The loss value of the first loss function can be determined by difference between the negative reconstruction loss between the predicted bounding box corresponding to the initial anchor box of the sample anchor box image and the initial anchor box, and the first divergence between the variational posterior distribution and the prior distribution of the latent variable. The loss value of the second loss function can be determined by weighted summation of the second divergence between the conditional distribution and the probability distribution of the predicted bounding box corresponding to the initial anchor box of the sample anchor box image, and the intersection-over-union (IoU) loss between the predicted bounding box and the true bounding box.

[0081] Optionally, the loss function of the bounding box distribution model can be defined by the following formula [1]:

[0082] L reg (b,b estimation ,b ground ) = Loss1 + Loss2

[0083] Where b represents the initial anchor frame of the sample anchor frame image; b ground b represents the ground truth bounding box corresponding to the initial anchor box of the sample anchor box image; estimation The initial anchor box of the sample anchor box image is represented by the predicted bounding box; Loss1 represents the first loss function; Loss2 represents the second loss function.

[0084] The first loss function can be defined by the following formula [2]:

[0085]

[0086] Where φ represents the parameters of the encoding network; θ represents the parameters of the decoding network; z represents the latent variable corresponding to the initial anchor box of the sample anchor box image; p θ (z) represents the prior distribution of the latent variable, which follows a Gaussian distribution; q φ (z|b) represents the variational posterior distribution of the latent variables corresponding to the initial anchor boxes of the sample anchor box image estimated using the coding network, the distribution following a Gaussian distribution and approximately p. θ (z|b estimation ), p θ (z|b estimation ) represents the posterior distribution of the latent variables corresponding to the initial anchor frame; p θ (b estimation |z) represents the conditional distribution of the predicted bounding box generated by the decoding network based on the sampled latent variables; Represents the mathematical expectation. D represents the negative reconstruction loss between the predicted bounding box corresponding to the initial anchor box of the sample anchor box image and the initial anchor box; KL [(q φ (z|b)||p θ (z))] represents the first KL divergence between the variational posterior distribution of the latent variable and the prior distribution of the latent variable.

[0087] The second loss function is defined by the following formula [3]:

[0088] Loss2=β1D KL [(p θ (b estimation |z)||pground (b ground ))]+β2IoULoss(b estimation ,b ground )

[0089] Where, p ground (b ground ) represents the probability distribution of the ground truth bounding boxes corresponding to the initial anchor boxes of the sample anchor box image, and the distribution follows a Gaussian distribution; p θ (b estimation |z) represents the conditional distribution of the predicted bounding box generated by the decoding network based on the sampled latent variables; D KL [(p θ (b estimation |z)||p ground (b ground ))] represents the second KL divergence between the conditional distribution of the predicted bounding box corresponding to the initial anchor box of the sample anchor box image and the probability distribution of the true bounding box; IoULoss(b estimation ,b ground ) represents the cross-union ratio loss between the predicted bounding box and the ground truth bounding box corresponding to the initial anchor box of the sample anchor box image; β1 represents the weight of the second KL divergence; β2 represents the weight of the cross-union ratio loss.

[0090] It is worth noting that the process described in S1021-S1022 above is only one iteration training process for the bounding box distribution model. In practical applications, it may be necessary to repeat the model iteration training multiple times to achieve a better training effect for the bounding box distribution model. Therefore, the above-described model iteration training process for the bounding box distribution model can be executed multiple times until the convergence condition of the bounding box distribution model is met. Specifically, meeting the convergence condition can refer to reaching a preset number of training iterations, or it can refer to the loss value of the loss function reaching a preset value.

[0091] At this point, the bounding box distribution model based on variational autoencoders has acquired the capability for object detection. By training the model to fit the predicted bounding boxes of the output sample images to the corresponding ground truth bounding boxes, it can accurately detect the position and size of objects in the image. Therefore, the bounding box distribution model can be used for object detection. It is understood that the inference process of the bounding box distribution model based on variational autoencoders differs from the training process; it may not include a discriminator network. Instead, the corresponding predicted bounding boxes can be generated from the initial anchor boxes of the input image using only an encoding and decoding network.

[0092] It should be noted that, in order to prevent model complexity, this embodiment assumes that each dimension of the bounding box distribution model based on the variational autoencoder is independent of the others. For example, the bounding box distribution model follows a Gaussian mixture model distribution, as shown in Equation [4]:

[0093]

[0094] Where σ represents the standard deviation, b estimation This represents the predicted bounding box, where π is a constant and p θ This represents the Gaussian mixture model distribution of the decoding network in the variational autoencoder.

[0095] In one optional implementation, a set of sample anchor box images can be input into a pre-trained bounding box distribution model to obtain a set of sample prediction images. Each sample prediction image includes a predicted bounding box of the target object region in the sample image and the corresponding true classification information of the target object. The sample prediction image set is used to train a target classification model for object detection. It can be understood that the predicted bounding box of the target object region in the sample image refers to the predicted bounding box corresponding to the initial anchor box of the target object region in the sample image obtained through the bounding box distribution model. The classification information of the target object corresponding to the predicted bounding box refers to the true classification information of the true bounding box corresponding to the initial anchor box of the target object region in the sample image. The true classification information can be obtained by labeling the sample image set. The trained target classification model and the bounding box distribution model based on a variational autoencoder can be combined to form a target detection model for object detection. Similar to the training method of the bounding box distribution model described above, embodiments of this specification also provide a training method for a target classification model, including:

[0096] Step 1: Obtain a sample prediction image set. Each sample prediction image includes the predicted bounding box of the target object region in the sample image and the corresponding true classification information of the target object. The sample prediction image set is obtained by inputting the sample anchor box image set into a pre-trained bounding box distribution model. The bounding box distribution model is based on the above... Figure 1 The bounding box distribution model shown was trained using the training method described above.

[0097] Step 2: Input the sample prediction image set into the initial target classification model for iterative training until the convergence condition of the target classification model is met, and obtain the trained target classification model.

[0098] Specifically, target classification models can employ softmax classifiers, SVM classifiers, etc. A softmax classifier outputs the probabilities of N+1 categories, where N is the total number of target categories and 1 represents the background, for a total of N+1 nodes. The loss function of the target classification model can also be set based on the selected classifier.

[0099] The bounding box distribution model training method provided in this specification includes an initial bounding box distribution model comprising a variational autoencoder. The variational autoencoder is used to fit the distribution of the initial anchor boxes in the sample anchor box image. Based on the probability distribution obtained after distribution fitting, sampling and reconstruction processing is performed to obtain a sample prediction image including the predicted bounding box. The variational autoencoder no longer learns the individual samples, but rather the distribution of the samples. Through iterative model training, the predicted bounding boxes of the sample prediction images output by the bounding box distribution model are fitted to the corresponding real bounding boxes, effectively solving the uncertainty problem of network prediction. This enables the trained bounding box distribution model to accurately detect the position and size of targets in the image during target detection, thereby improving the accuracy of target detection.

[0100] Furthermore, the trained bounding box distribution model adopts a single model approach and single forward propagation, which reduces the computational cost of the model. The trained bounding box distribution model is implemented based on a hybrid density network, which learns a Gaussian mixture model to realize the network output, calculates arbitrary uncertainty and knowledge uncertainty, and can improve the performance of the model.

[0101] The target classification model training method provided in this specification is based on a pre-trained bounding box distribution model to obtain a sample prediction image set. Each sample prediction image includes the predicted bounding box of the target object region in the sample image and the corresponding true classification information of the target object. The sample prediction image set is used to train a target classification model for target detection. Since the bounding box distribution model based on variational autoencoder can accurately detect the position and size of the target in the image, the target classification model trained using the sample prediction image set can accurately identify and classify the detected target.

[0102] Corresponding to the methods described above, embodiments of this specification also provide a target detection method capable of... Figure 1 The bounding box distribution model trained by the method shown can quickly and accurately generate corresponding predicted bounding boxes from the initial anchor boxes of the input image. Then, it is combined with the target classification model to obtain the classification information of the predicted bounding boxes, thereby completing the target detection task and improving the target detection efficiency.

[0103] Please refer to Figure 4 The following is a flowchart illustrating a target detection method according to an embodiment of this specification. The method may include the following steps:

[0104] S401. Input the image to be detected into the target detection head model to obtain the initial image to be detected, wherein the initial image to be detected includes the initial anchor box of the target object region in the image to be detected.

[0105] In specific implementation, the target detection head model can employ a two-stage target detection algorithm head, which can be used to obtain the initial anchor boxes of the target object region in the image to be detected. In this embodiment of the specification, for ease of distinction, the output image including the initial anchor boxes obtained by the target detection head model is referred to as the initial image to be detected.

[0106] For example, the structure and processing flow of the two-stage object detection algorithm head include: performing Selective Search on the input image to obtain 1K to 2K candidate regions; inputting the image into the DeepConvNet network to obtain a feature map, and simultaneously projecting the candidate regions onto the feature map to obtain the corresponding feature matrix; scaling each feature matrix to a fixed size (7*7) feature map through the RoI pooling layer; and then obtaining the structural features (feature vector) of the RoI (region of interest) through two fully connected layers FCs (FC6, FC7).

[0107] The structural features of the RoI of the image to be detected, generated using a two-stage object detection algorithm, can be represented as {{p i},{b i}}, i = 1, ..., N, where b i p represents the initial anchor box representing the target object region in the image. i This represents the classification information of the target object corresponding to the initial anchor box. Initial anchor box b i This can be understood as multiple prior boxes with different aspect ratios predefined by the object detection model, centered on the anchor point, with the initial anchor box b. i Specifically, it refers to the initial anchor frame b. i Location information, including center coordinates and length and width.

[0108] S402. Input the initial image to be detected into a pre-trained bounding box distribution model to obtain a predicted image to be detected including the predicted bounding boxes corresponding to the initial anchor boxes, wherein the bounding box distribution model is based on the above... Figure 1 The bounding box distribution model shown was trained using the training method described.

[0109] Specifically, the pre-trained bounding box distribution model fits the distribution of the initial anchor boxes of the initial image to be detected, and performs sampling and reconstruction processing based on the probability distribution obtained after the distribution fitting to obtain the predicted image to be detected, which includes the predicted bounding boxes corresponding to the initial anchor boxes. In the embodiments of this specification, for ease of distinction, the output image of the bounding box distribution model that includes the predicted bounding boxes corresponding to the initial anchor boxes is referred to as the predicted image to be detected.

[0110] In one optional implementation, the target detection method may further include the following steps:

[0111] S403. Input the predicted image to be detected into the pre-trained target classification model, and output the classification information of the target object corresponding to the predicted bounding box in the predicted image to be detected.

[0112] The target classification model can employ a Softmax classifier, an SVM classifier, etc. The Softmax classifier outputs the probabilities of N+1 categories, where N is the total number of target categories and 1 represents the background, for a total of N+1 nodes. The target classification model can be trained using the methods described above, or other training methods; however, this specification does not impose specific limitations on the embodiments described.

[0113] The bounding box distribution model may include a trained encoding network and a decoding network. Specifically, S402 may include the following steps:

[0114] S4021. Calculate the Gaussian mixture model parameters of the latent variables for the initial anchor boxes of each input initial image to be detected using the trained encoding network. Fit the variational posterior distribution of the latent variables according to the Gaussian mixture model parameters. Sample the variational posterior distribution of the latent variables. The latent variables are used to describe the attributes of the initial anchor boxes of the initial images to be detected.

[0115] S4022. Using the trained decoding network, generate the predicted bounding box corresponding to the initial anchor box based on the sampled latent variables, to obtain the predicted image to be detected including the predicted bounding box corresponding to the initial anchor box.

[0116] The target detection method provided in this specification first uses a target detection head model to obtain an initial target image containing initial anchor boxes of the target object region based on the input target image. Then, the initial target image is input into a pre-trained bounding box distribution model. A bounding box distribution model based on variational autoencoder is used to obtain a predicted target image containing the predicted bounding boxes corresponding to the initial anchor boxes. The bounding box distribution model based on variational autoencoder learns the distribution of samples and iteratively trains the model to fit the predicted bounding boxes of the output predicted target image to the corresponding real bounding boxes, effectively solving the uncertainty problem of network prediction. Therefore, it can accurately detect the position and size of the target in the image, thereby improving the accuracy of target detection.

[0117] Furthermore, the predicted image to be detected is input into a pre-trained target classification model, which can accurately identify and classify the detected targets. Since the bounding box distribution model based on variational autoencoders effectively addresses the problem of existing target detection models relying on multiple models, requiring significant computational resources and numerous network parameters, using the bounding box distribution model based on variational autoencoders for target detection can improve the efficiency of target detection.

[0118] It should be noted that the training methods for the bounding box distribution model, the training methods for the target classification model, and the target detection methods provided in the embodiments of this specification can be applied to various business scenarios that require target detection, such as OCR (Optical Character Recognition), ID card detection, and general target detection tasks. The embodiments of this specification do not specifically limit these applications.

[0119] In addition, with the above Figure 1 Corresponding to the training method of the bounding box distribution model shown, this specification also provides a training device for the bounding box distribution model. Figure 5 This is a schematic diagram of the structure of a training device 500 for a bounding box distribution model provided in an embodiment of this specification, including:

[0120] The acquisition module 501 is used to acquire a set of sample anchor box images, each sample anchor box image including the initial anchor box of the target object region in the sample image and the corresponding ground bounding box;

[0121] The training control module 502 is used to input the sample anchor box image set into the initial bounding box distribution model for iterative training of the model until the convergence condition of the bounding box distribution model is met, so as to obtain the trained bounding box distribution model.

[0122] The initial bounding box distribution model includes a variational autoencoder 503, and the bounding box distribution model is trained in each iteration using the variational autoencoder 503 and the parameter tuning module 504.

[0123] The variational autoencoder 503 is used to perform distribution fitting on the initial anchor boxes of the sample anchor box image, and to perform sampling reconstruction processing based on the probability distribution obtained after distribution fitting to obtain a sample prediction image including the prediction bounding box corresponding to the initial anchor box.

[0124] The parameter adjustment module 504 is used to adjust the model parameters of the bounding box distribution model according to the initial anchor box, the predicted bounding box corresponding to the initial anchor box, the true bounding box corresponding to the initial anchor box, and the loss function of the preset bounding box distribution model; the loss function includes a first loss function for characterizing the difference between the predicted bounding box and the initial anchor box, and a second loss function for characterizing the difference between the predicted bounding box and the true bounding box.

[0125] Specifically, one possible structure of the variational autoencoder 503 includes an encoding network 531 and a decoding network 532, wherein:

[0126] The encoding network 531 is used to calculate the Gaussian mixture model parameters of the corresponding latent variables for the initial anchor boxes of each input sample anchor box image, fit the variational posterior distribution of the latent variables according to the Gaussian mixture model parameters, and sample the variational posterior distribution of the latent variables. The latent variables are used to describe the attributes of the initial anchor boxes of the sample anchor box images.

[0127] The decoding network 532 is used to generate the predicted bounding box corresponding to the initial anchor box based on the sampled latent variables, so as to obtain a sample prediction image including the predicted bounding box corresponding to the initial anchor box.

[0128] The initial bounding box distribution model may also include a discriminant network. Specifically, one possible structure of the parameter adjustment module 504 may include:

[0129] The first loss determination submodule 541 is used to determine the loss value of the first loss function by using the variational autoencoder based on the reconstruction loss between the initial anchor box of the sample anchor box image and the predicted bounding box corresponding to the initial anchor box, and the first divergence between the variational posterior distribution of the latent variable and the prior distribution of the latent variable.

[0130] The second loss determination submodule 542 is used to determine the loss value of the second loss function by using the discriminant network based on the second divergence between the conditional distribution of the predicted bounding box corresponding to the initial anchor box of the sample anchor box image and the probability distribution of the true bounding box, and the cross-union ratio loss between the predicted bounding box corresponding to the initial anchor box of the sample anchor box image and the true bounding box.

[0131] The adjustment submodule 543 determines the loss value of the bounding box distribution model based on the loss value of the first loss function and the loss value of the second loss function, and adjusts the model parameters of the bounding box distribution model based on the loss value of the bounding box distribution model.

[0132] The bounding box distribution model training device provided in the embodiments of this specification proposes a bounding box distribution model based on a variational autoencoder. During the training process of the bounding box distribution model, the initial bounding box distribution model used includes a variational autoencoder. The variational autoencoder is used to fit the distribution of the initial anchor boxes of the sample anchor box image. Based on the probability distribution obtained after distribution fitting, sampling and reconstruction processing is performed to obtain a sample prediction image including the predicted bounding box. The variational autoencoder no longer learns the individual samples, but rather the distribution of the samples. Through iterative training of the model, the predicted bounding boxes of the sample prediction images output by the bounding box distribution model are fitted to the corresponding real bounding boxes, effectively solving the uncertainty problem of network prediction. This enables the trained bounding box distribution model to accurately detect the position and size of targets in the image during target detection, thereby improving the accuracy of target detection.

[0133] The trained bounding box distribution model adopts a single model approach and single forward propagation, which reduces the computational cost of the model. The trained bounding box distribution model is based on a hybrid density network, which learns a Gaussian mixture model to realize the network output, calculates arbitrary uncertainty and knowledge uncertainty, and can improve the performance of the model.

[0134] Obviously, the training device for the bounding box distribution model in the embodiments of this specification can be used as described above. Figure 1 The entity executing the training method for the bounding box distribution model shown is thus able to implement the training method for the bounding box distribution model in... Figure 1 The functions implemented are the same, so they will not be described in detail here.

[0135] Furthermore, corresponding to the training method of the target classification model described above, embodiments of this specification also provide a training apparatus for a target classification model. A possible structure of a training apparatus for a target classification model may include:

[0136] The image set generation module is used to acquire a sample prediction image set. Each sample prediction image includes a predicted bounding box of the target object region in the sample image and the corresponding true classification information of the target object. The sample prediction image set is obtained by inputting the sample anchor box image set into a pre-trained bounding box distribution model, which is based on the above... Figure 1 The bounding box distribution model shown was trained using the training method described above.

[0137] The training control module is used to input the sample prediction image set into the initial target classification model for iterative training until the convergence condition of the target classification model is met, thus obtaining the trained target classification model.

[0138] The target classification model training device provided in this specification acquires a sample prediction image set based on a pre-trained bounding box distribution model. Each sample prediction image includes a predicted bounding box of the target object region in the sample image and the corresponding true classification information of the target object. The sample prediction image set is used to train a target classification model for target detection. Since the bounding box distribution model based on the variational autoencoder can accurately detect the position and size of the target in the image, the target classification model trained using the sample prediction image set can accurately identify and classify the detected target.

[0139] Obviously, the training device for the target classification model in the embodiments of this specification can serve as the execution body for the above-described target classification model training method, and thus can realize the function of the target classification model training method. Since the principle is the same, it will not be described again here.

[0140] In addition, with the above Figure 4 Corresponding to the target detection method shown, this specification also provides a target detection device in its embodiments. Figure 6 This is a schematic diagram of the structure of a target detection device 600 provided in the embodiments of this specification, including:

[0141] The extraction module 601 is used to input the image to be detected into the target detection head model to obtain the initial image to be detected, including the initial anchor box;

[0142] Distribution detection module 602 is used to input the initial image to be detected into a pre-trained bounding box distribution model to obtain a predicted image to be detected including the predicted bounding boxes corresponding to the initial anchor boxes, wherein the bounding box distribution model is based on the above... Figure 1 The bounding box distribution model shown was trained using the training method described.

[0143] In one alternative implementation, the target detection device may further include:

[0144] The classification and detection module 603 is used to input the predicted image to be detected into a pre-trained target classification model and output the classification information of the target object corresponding to the predicted bounding box in the predicted image to be detected.

[0145] Specifically, the bounding box distribution model includes a trained encoding network and a decoding network. Correspondingly, one possible structure of the distribution detection module 602 includes:

[0146] The variational inference submodule 621 is used to calculate the Gaussian mixture model parameters of the corresponding latent variables for the initial anchor boxes of each input initial image to be detected, fit the variational posterior distribution of the latent variables according to the Gaussian mixture model parameters, and sample the variational posterior distribution of the latent variables. The latent variables are used to describe the attributes of the initial anchor boxes of the initial images to be detected.

[0147] The generation submodule 622 is used to generate the predicted bounding box corresponding to the initial anchor box based on the sampled latent variables, so as to obtain the predicted image to be detected including the predicted bounding box corresponding to the initial anchor box.

[0148] The target detection device provided in the embodiments of this specification first uses a target detection head model to obtain an initial target image containing initial anchor boxes of the target object region based on the input target image. Then, the initial target image is input into a pre-trained bounding box distribution model. The bounding box distribution model based on variational autoencoder is used to obtain a predicted target image containing the predicted bounding boxes corresponding to the initial anchor boxes. The bounding box distribution model based on variational autoencoder learns the distribution of samples and iteratively trains the model to fit the predicted bounding boxes of the output predicted target image to the corresponding real bounding boxes, effectively solving the uncertainty problem of network prediction. Therefore, it can accurately detect the position and size of the target in the image, thereby improving the accuracy of target detection.

[0149] Furthermore, the predicted image to be detected is input into a pre-trained target classification model, which can accurately identify and classify the detected targets. Since the bounding box distribution model based on variational autoencoders effectively addresses the problem of existing target detection models relying on multiple models, requiring significant computational resources and numerous network parameters, using the bounding box distribution model based on variational autoencoders for target detection can improve the efficiency of target detection.

[0150] Obviously, the target detection device in the embodiments of this specification can be used as described above. Figure 4 The entity executing the target detection method shown is therefore capable of implementing the target detection method in... Figure 4 The functions implemented are the same, so they will not be described in detail here.

[0151] Figure 7 This is a schematic diagram of the structure of an electronic device according to one embodiment of this specification. Please refer to it. Figure 7At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations.

[0152] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 7 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0153] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0154] The processor reads the corresponding computer program from non-volatile memory into memory and runs it, forming a training device for the bounding box distribution model at the logical level. The processor executes the program stored in memory and specifically performs the following operations:

[0155] Obtain a set of sample anchor box images, where each sample anchor box image includes the initial anchor box of the target object region in the sample image and the corresponding ground truth bounding box;

[0156] The sample anchor box image set is input into the initial bounding box distribution model for iterative training until the convergence condition of the bounding box distribution model is met, thus obtaining the trained bounding box distribution model.

[0157] The initial bounding box distribution model includes a variational autoencoder, and the specific steps for training the bounding box distribution model in each iteration include:

[0158] The variational autoencoder is used to fit the distribution of the initial anchor boxes in the sample anchor box image. Based on the probability distribution obtained after the distribution fitting, sampling and reconstruction processing is performed to obtain a sample prediction image including the predicted bounding boxes corresponding to the initial anchor boxes. The model parameters of the bounding box distribution model are adjusted according to the initial anchor boxes, the predicted bounding boxes corresponding to the initial anchor boxes, the ground truth bounding boxes corresponding to the initial anchor boxes, and the loss function of the preset bounding box distribution model. The loss function includes a first loss function for characterizing the difference between the predicted bounding boxes and the initial anchor boxes, and a second loss function for characterizing the difference between the predicted bounding boxes and the ground truth bounding boxes.

[0159] Alternatively, the processor reads the corresponding computer program from non-volatile memory into memory and runs it, forming a target classification device at the logical level. The processor executes the program stored in memory and specifically performs the following operations:

[0160] A sample prediction image set is obtained, where each sample prediction image includes a predicted bounding box of the target object region in the sample image and the corresponding true classification information of the target object. The sample prediction image set is obtained by inputting the sample anchor box image set into a pre-trained bounding box distribution model, which is based on the above... Figure 1 The bounding box distribution model shown was trained using the training method described above.

[0161] The sample prediction image set is input into the initial target classification model for iterative training until the convergence condition of the target classification model is met, thus obtaining the trained target classification model.

[0162] Alternatively, the processor reads the corresponding computer program from non-volatile memory into memory and runs it, forming a target detection device at the logical level. The processor executes the program stored in memory and specifically performs the following operations:

[0163] The image to be detected is input into the target detection head model to obtain the initial image to be detected, which includes the initial anchor box of the target object region in the image to be detected.

[0164] The initial image to be detected is input into a pre-trained bounding box distribution model to obtain a predicted image to be detected that includes the predicted bounding boxes corresponding to the initial anchor boxes. The bounding box distribution model is based on the above... Figure 1 The bounding box distribution model shown was trained using the training method described.

[0165] The above is as described in this instruction manual. Figure 1 The method executed by the training apparatus for the bounding box distribution model disclosed in the illustrated embodiments, or the method executed by the training apparatus for the target classification model disclosed in the embodiments of this specification, or the method executed by the training apparatus for the target classification model disclosed in this specification. Figure 4 The method executed by the target detection device disclosed in the illustrated embodiments can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this specification. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this specification can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0166] It should be understood that the electronic device in the embodiments of this specification can realize a training device for a bounding box distribution model. Figure 1 The function or target detection device of the illustrated embodiment is in Figure 4 The embodiments shown have the same function. Since the principle is the same, the embodiments in this specification will not be described again here.

[0167] Of course, in addition to software implementation, the electronic device described in this specification does not exclude other implementation methods, such as logic devices or a combination of hardware and software. In other words, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0168] This specification also provides an embodiment of a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by a portable electronic device including multiple applications, enable the portable electronic device to perform... Figure 1 The method of the illustrated embodiment is specifically used to perform the following operations:

[0169] Obtain a set of sample anchor box images, where each sample anchor box image includes the initial anchor box of the target object region in the sample image and the corresponding ground truth bounding box;

[0170] The sample anchor box image set is input into the initial bounding box distribution model for iterative training until the convergence condition of the bounding box distribution model is met, thus obtaining the trained bounding box distribution model.

[0171] The initial bounding box distribution model includes a variational autoencoder, and the specific steps for training the bounding box distribution model in each iteration include:

[0172] The variational autoencoder is used to fit the distribution of the initial anchor boxes in the sample anchor box image. Based on the probability distribution obtained after the distribution fitting, sampling and reconstruction processing is performed to obtain a sample prediction image including the predicted bounding boxes corresponding to the initial anchor boxes. The model parameters of the bounding box distribution model are adjusted according to the initial anchor boxes, the predicted bounding boxes corresponding to the initial anchor boxes, the ground truth bounding boxes corresponding to the initial anchor boxes, and the loss function of the preset bounding box distribution model. The loss function includes a first loss function for characterizing the difference between the predicted bounding boxes and the initial anchor boxes, and a second loss function for characterizing the difference between the predicted bounding boxes and the ground truth bounding boxes.

[0173] This specification also provides an embodiment of a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by a portable electronic device including multiple applications, enable the portable electronic device to specifically perform the following operations:

[0174] A sample prediction image set is obtained, where each sample prediction image includes a predicted bounding box of the target object region in the sample image and the corresponding true classification information of the target object. The sample prediction image set is obtained by inputting the sample anchor box image set into a pre-trained bounding box distribution model, which is based on the above... Figure 1 The bounding box distribution model shown was trained using the training method described above.

[0175] The sample prediction image set is input into the initial target classification model for iterative training until the convergence condition of the target classification model is met, thus obtaining the trained target classification model.

[0176] This specification also provides an embodiment of a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by a portable electronic device including multiple applications, enable the portable electronic device to perform... Figure 4The method of the illustrated embodiment is specifically used to perform the following operations:

[0177] The image to be detected is input into the target detection head model to obtain the initial image to be detected, which includes the initial anchor box of the target object region in the image to be detected.

[0178] The initial image to be detected is input into a pre-trained bounding box distribution model to obtain a predicted image to be detected that includes the predicted bounding boxes corresponding to the initial anchor boxes. The bounding box distribution model is based on the above... Figure 1 The bounding box distribution model shown was trained using the training method described.

[0179] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0180] In summary, the above description is merely a preferred embodiment of this specification and is not intended to limit the scope of protection of this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of protection of this specification.

[0181] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0182] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0183] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0184] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

Claims

1. A training method for a bounding box distribution model, characterized in that, include: Obtain a set of sample anchor box images, where each sample anchor box image includes the initial anchor box of the target object region in the sample image and the corresponding ground truth bounding box; The sample anchor box image set is input into the initial bounding box distribution model for iterative training until the convergence condition of the bounding box distribution model is met, thus obtaining the trained bounding box distribution model. The initial bounding box distribution model includes a variational autoencoder, and the specific steps for training the bounding box distribution model in each iteration include: The initial anchor boxes of the sample anchor box image are fitted with the distribution using the variational autoencoder, and sampling reconstruction is performed based on the probability distribution obtained after the distribution fitting to obtain a sample prediction image including the prediction bounding box corresponding to the initial anchor box. The model parameters of the bounding box distribution model are adjusted based on the initial anchor box, the predicted bounding box corresponding to the initial anchor box, the ground truth bounding box corresponding to the initial anchor box, and the loss function of the preset bounding box distribution model. The loss function includes a first loss function for characterizing the difference between the predicted bounding box and the initial anchor box, and a second loss function for characterizing the difference between the predicted bounding box and the ground truth bounding box.

2. The method according to claim 1, characterized in that, The variational autoencoder includes an encoding network and a decoding network; The step of using the variational autoencoder to fit the distribution of the initial anchor boxes in the sample anchor box image, and then performing sampling reconstruction based on the probability distribution obtained after the distribution fitting, to obtain a sample prediction image including the predicted bounding boxes corresponding to the initial anchor boxes, specifically includes: The encoding network is used to calculate the Gaussian mixture model parameters of the corresponding latent variables for the initial anchor boxes of each input sample anchor box image. The variational posterior distribution of the latent variables is fitted according to the Gaussian mixture model parameters. The variational posterior distribution of the latent variables is sampled. The latent variables are used to describe the attributes of the initial anchor boxes of the sample anchor box images. The decoding network is used to generate the predicted bounding box corresponding to the initial anchor box based on the sampled latent variables, resulting in a sample prediction image including the predicted bounding box corresponding to the initial anchor box.

3. The method according to claim 2, characterized in that, The initial bounding box distribution model also includes a discriminant network; The step of adjusting the model parameters of the bounding box distribution model based on the initial anchor box, the predicted bounding box corresponding to the initial anchor box, the ground truth bounding box corresponding to the initial anchor box, and the loss function of the preset bounding box distribution model specifically includes: The variational autoencoder is used to determine the loss value of the first loss function based on the reconstruction loss between the initial anchor box of the sample anchor box image and the predicted bounding box corresponding to the initial anchor box, and the first divergence between the variational posterior distribution of the latent variable and the prior distribution of the latent variable. The discriminant network is used to determine the loss value of the second loss function based on the second divergence between the conditional distribution of the predicted bounding box corresponding to the initial anchor box of the sample anchor box image and the probability distribution of the true bounding box, as well as the cross-union ratio loss between the predicted bounding box and the true bounding box corresponding to the initial anchor box of the sample anchor box image. The loss value of the bounding box distribution model is determined based on the loss value of the first loss function and the loss value of the second loss function, and the model parameters of the bounding box distribution model are adjusted based on the loss value of the bounding box distribution model.

4. The method according to claim 3, characterized in that, Also includes: The loss value of the bounding box distribution model is determined by weighted summation of the loss values ​​of the first loss function and the second loss function.

5. The method according to any one of claims 1 to 4, characterized in that, The acquisition of the sample anchor box image set specifically includes: Input the sample image set into the target detection head model to obtain the initial sample anchor box image set. Each initial sample anchor box image in the initial sample anchor box image set includes the initial anchor box of the target object region in the sample image. The initial anchor frame in each initial sample anchor frame image and the ground truth bounding box corresponding to the initial anchor frame are combined to form the sample anchor frame image set. The ground truth bounding box corresponding to each initial anchor frame in the sample anchor frame image set is obtained by annotating the sample image set.

6. A training method for a target classification model, characterized in that, include: A sample prediction image set is obtained, wherein each sample prediction image includes a predicted bounding box of the target object region in the sample image and the true classification information of the corresponding target object. The sample prediction image set is obtained by inputting the sample anchor box image set into a pre-trained bounding box distribution model, which is trained according to the training method of the bounding box distribution model as described in any one of claims 1 to 5. The sample prediction image set is input into the initial target classification model for iterative training until the convergence condition of the target classification model is met, thus obtaining the trained target classification model.

7. A target detection method, characterized in that, include: The image to be detected is input into the target detection head model to obtain the initial image to be detected, which includes the initial anchor box of the target object region in the image to be detected. The initial image to be detected is input into a pre-trained bounding box distribution model to obtain a predicted image to be detected including the predicted bounding boxes corresponding to the initial anchor boxes, wherein the bounding box distribution model is trained according to the training method of the bounding box distribution model as described in any one of claims 1 to 5.

8. The method according to claim 7, characterized in that, Also includes: The predicted image to be detected is input into a pre-trained target classification model, which outputs the classification information of the target object corresponding to the predicted bounding box in the predicted image to be detected.

9. A target detection device, characterized in that, include: The extraction module is used to input the image to be detected into the target detection head model to obtain an initial image to be detected, wherein the initial image to be detected includes the initial anchor box of the target object region in the image to be detected; The distribution detection module is used to input the initial image to be detected into a pre-trained bounding box distribution model to obtain a predicted image to be detected including the predicted bounding boxes corresponding to the initial anchor boxes, wherein the bounding box distribution model is trained according to the training method of the bounding box distribution model as described in any one of claims 1 to 5.

10. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device is able to perform the method as described in any one of claims 1 to 8.

Images