Target detection method and device

Abstract

An embodiment of the present invention discloses a target detection method and device, which includes: obtaining an image to be detected; using a feature extraction layer of a pre-established target detection model to extract features of the image to be detected, and determining the features of the image to be detected corresponding to the image to be detected; using a feature regression layer of the pre-established target detection model and the features of the image to be detected, determining a probability value corresponding to each initial frame boundary position information corresponding to each detection target detected in the image to be detected; and for each detection target, based on the probability value corresponding to each initial frame boundary position information corresponding to the detection target, determining the target detection frame position information corresponding to the detection target, wherein the pre-established target detection model is: a model obtained by training based on a sample image and its corresponding calibration information, and the calibration information includes: calibration frame position information corresponding to the sample target in the corresponding sample image, so as to achieve more accurate detection of the boundaries of the target in the image.

Description

Technical Field

[0001] The present invention relates to the field of target detection technology, and in particular to a target detection method and device. Background Art

[0002] At present, the training data for training target detection models generally defines the true boundaries of the target in the image as clearly as possible. However, there are still some sample images in the training data where the boundaries of the target are blurred and unclear, and other uncertainties. For example, the target in the image is partially obscured or the outline of some areas of the target in the image is not clear due to lighting reasons. As a result, the bounding box position information in the calibration information corresponding to such sample images in the training data cannot clearly represent the true boundaries of the target in the image.

[0003] The current target detection model's feature regression layer's box regression method generally uses a single-value regression method, which assumes that the bounding box of the target in the image satisfies the Dirac distribution. This means that the bounding box position information in the calibration information corresponding to the sample images in the training data is the true boundary of the target in the sample image. Accordingly, when training the target detection model using the aforementioned training data, the target detection model fails to learn the uncertainty of the target's boundary. Subsequently, using this target detection model, the detection results for targets in the image that are partially occluded or have unclear outlines due to lighting conditions will have large errors. This means that the position of the detection box for targets in the image that appear in these situations is not accurate enough. Summary of the Invention

[0004] The present invention provides a target detection method and device to achieve more accurate detection of the boundaries of targets in images. The specific technical solution is as follows:

[0005] In a first aspect, an embodiment of the present invention provides a target detection method, the method comprising:

[0006] Obtaining an image to be detected;

[0007] Performing feature extraction on the image to be detected using a feature extraction layer of a pre-established target detection model to determine the image features to be detected corresponding to the image to be detected;

[0008] Utilizing the feature regression layer of the pre-established target detection model and the features of the image to be detected, the probability value corresponding to each initial frame boundary position information corresponding to each detection target detected in the image to be detected is determined; and for each detection target, based on the probability value corresponding to each initial frame boundary position information corresponding to the detection target, the target detection frame position information corresponding to the detection target is determined, wherein the pre-established target detection model is: a model trained based on a sample image and its corresponding calibration information, and the calibration information includes: calibration frame position information corresponding to the sample target in the corresponding sample image.

[0009] Optionally, the step of determining, for each detection target, the target detection frame position information corresponding to the detection target based on the probability value corresponding to each initial frame boundary position information corresponding to the detection target, includes:

[0010] For each detection target, based on the probability value corresponding to each initial frame boundary position information corresponding to the detection target in each direction, the target detection frame boundary position information corresponding to the detection target in each direction is determined to determine the target detection frame position information corresponding to the detection target, wherein the direction includes the top, bottom, left and right directions of the image to be detected.

[0011] Optionally, the step of determining, for each detected target, target detection frame boundary position information corresponding to the detected target in each orientation based on the probability value corresponding to each initial frame boundary position information corresponding to the detected target in each orientation includes:

[0012] For each detection target, the target detection frame boundary position information corresponding to the detection target in each direction is determined based on the probability value corresponding to each initial frame boundary position information corresponding to the detection target in each direction and the preset integral formula.

[0013] Optionally, the preset integral formula is expressed as:

[0014]

[0015] Wherein, P represents the target translation value between the position represented by the target detection frame boundary position information corresponding to the detection target in the target orientation and the position of the activation point corresponding to the detection target predicted by the pre-established target detection model, a i Indicates the preset translation value corresponding to the i-th initial frame boundary position information of the detected target in the target orientation, p(a i) represents the probability value corresponding to the preset translation value corresponding to the i-th initial frame boundary position information of the detection target at the target orientation, n+1 represents the total number of preset translation values ​​corresponding to the initial frame boundary position information of the detection target at the target orientation, and the target orientation is any orientation among the orientations.

[0016] Optionally, before the step of determining, using the feature regression layer of the pre-established target detection model and the features of the image to be detected, a probability value corresponding to each initial frame boundary position information corresponding to each detection target detected in the image to be detected; and determining, for each detection target, the target detection frame position information corresponding to the detection target based on the probability value corresponding to each initial frame boundary position information corresponding to the detection target, the method further includes:

[0017] The process of training a pre-established object detection model includes:

[0018] Obtaining multiple sample images and calibration information corresponding to each sample image;

[0019] Obtain an initial target detection model;

[0020] For each sample image, input the sample image into the feature extraction layer of the initial target detection model to extract the sample image features corresponding to the sample image;

[0021] For each sample image, input the sample image features corresponding to the sample image into the feature regression layer of the initial target detection model to determine the probability value corresponding to each prediction box boundary position information corresponding to the sample target in each direction in the sample image; and for each sample target in the sample image, determine the current box position information corresponding to the sample target based on the probability value corresponding to each prediction box boundary position information corresponding to each direction corresponding to the sample target and a preset integral formula;

[0022] For each sample image, determine the current loss value using a preset loss function, the current position information corresponding to each sample target in the sample image, and the calibration frame position information corresponding to each sample target in the calibration information corresponding to the sample image;

[0023] Determining whether the current loss value exceeds a preset loss threshold;

[0024] If it is determined that the current loss value exceeds the preset loss threshold, the model parameters of the feature extraction layer and the feature regression layer of the initial object detection model are adjusted, and the step of inputting the sample image into the feature extraction layer of the initial object detection model for each sample image to extract the sample image features corresponding to the sample image is returned to execution;

[0025] If it is determined that the current loss value does not exceed the preset loss threshold, it is determined that the initial target detection model has reached a convergence state, and a pre-established target detection model including a feature extraction layer and a feature regression layer is determined.

[0026] Optionally, after the steps of determining, using the feature regression layer of the pre-established target detection model and the features of the image to be detected, a probability value corresponding to each initial frame boundary position information corresponding to each detected target in the image to be detected; and determining, for each detected target, target detection frame position information corresponding to the detected target based on the probability value corresponding to each initial frame boundary position information corresponding to the detected target, the method further includes:

[0027] Output the probability value corresponding to each initial frame boundary position information corresponding to each detected detection target, and / or the target detection frame position information corresponding to each detection target.

[0028] In a second aspect, an embodiment of the present invention provides a target detection device, the device comprising:

[0029] An acquisition module is configured to acquire an image to be detected;

[0030] A first determination module is configured to perform feature extraction on the image to be detected using a feature extraction layer of a pre-established target detection model to determine image features to be detected corresponding to the image to be detected;

[0031] The second determination module is configured to use the feature regression layer of the pre-established target detection model and the features of the image to be detected to determine the probability value corresponding to each initial frame boundary position information corresponding to each detection target detected in the image to be detected; and for each detection target, based on the probability value corresponding to each initial frame boundary position information corresponding to the detection target, determine the target detection frame position information corresponding to the detection target, wherein the pre-established target detection model is: a model trained based on a sample image and its corresponding calibration information, and the calibration information includes: calibration frame position information corresponding to the sample target in the corresponding sample image.

[0032] Optionally, the second determining module is specifically configured to

[0033] For each detection target, based on the probability value corresponding to each initial frame boundary position information corresponding to the detection target in each direction, the target detection frame boundary position information corresponding to the detection target in each direction is determined to determine the target detection frame position information corresponding to the detection target, wherein the direction includes the top, bottom, left and right directions of the image to be detected.

[0034] Optionally, the second determining module is specifically configured to

[0035] For each detection target, the target detection frame boundary position information corresponding to the detection target in each direction is determined based on the probability value corresponding to each initial frame boundary position information corresponding to the detection target in each direction and the preset integral formula.

[0036] Optionally, the preset integral formula is expressed as:

[0037]

[0038] Wherein, P represents the target translation value between the position represented by the target detection frame boundary position information corresponding to the detection target in the target orientation and the position of the activation point corresponding to the detection target predicted by the pre-established target detection model, a i Indicates the preset translation value corresponding to the i-th initial frame boundary position information of the detected target in the target orientation, p(a i ) represents the probability value corresponding to the preset translation value corresponding to the i-th initial frame boundary position information of the detection target at the target orientation, n+1 represents the total number of preset translation values ​​corresponding to the initial frame boundary position information of the detection target at the target orientation, and the target orientation is any orientation among the orientations.

[0039] Optionally, the device further includes:

[0040] The model training module is configured to determine, by using the feature regression layer of the pre-established target detection model and the features of the image to be detected, a probability value corresponding to each initial frame boundary position information corresponding to each detection target detected in the image to be detected; and before determining the target detection frame position information corresponding to each detection target based on the probability value corresponding to each initial frame boundary position information corresponding to the detection target, train the pre-established target detection model, wherein the model training module is specifically configured to obtain a plurality of sample images and calibration information corresponding to each sample image;

[0041] Obtain an initial target detection model;

[0042] For each sample image, input the sample image into the feature extraction layer of the initial target detection model to extract the sample image features corresponding to the sample image;

[0043] For each sample image, input the sample image features corresponding to the sample image into the feature regression layer of the initial target detection model to determine the probability value corresponding to each prediction box boundary position information corresponding to the sample target in each direction in the sample image; and for each sample target in the sample image, determine the current box position information corresponding to the sample target based on the probability value corresponding to each prediction box boundary position information corresponding to each direction corresponding to the sample target and a preset integral formula;

[0044] For each sample image, determine the current loss value using a preset loss function, the current position information corresponding to each sample target in the sample image, and the calibration frame position information corresponding to each sample target in the calibration information corresponding to the sample image;

[0045] Determining whether the current loss value exceeds a preset loss threshold;

[0046] If it is determined that the current loss value exceeds the preset loss threshold, the model parameters of the feature extraction layer and the feature regression layer of the initial object detection model are adjusted, and the process of inputting the sample image into the feature extraction layer of the initial object detection model for each sample image to extract the sample image features corresponding to the sample image is returned to execution;

[0047] If it is determined that the current loss value does not exceed the preset loss threshold, it is determined that the initial target detection model has reached a convergence state, and a pre-established target detection model including a feature extraction layer and a feature regression layer is determined.

[0048] Optionally, the device further includes:

[0049] The output module is configured to determine the probability value corresponding to each initial frame boundary position information corresponding to each detection target detected in the image to be detected by using the feature regression layer of the pre-established target detection model and the features of the image to be detected; and for each detection target, based on the probability value corresponding to each initial frame boundary position information corresponding to the detection target, determine the target detection frame position information corresponding to the detection target, and then output the probability value corresponding to each initial frame boundary position information corresponding to each detection target, and / or the target detection frame position information corresponding to each detection target.

[0050] From the above content, it can be seen that an embodiment of the present invention provides a target detection method and device, which obtains an image to be detected; uses a feature extraction layer of a pre-established target detection model to extract features of the image to be detected, and determines the image features to be detected corresponding to the image to be detected; uses a feature regression layer of a pre-established target detection model and the image features to be detected to determine the probability value corresponding to each initial frame boundary position information corresponding to each detection target detected in the image to be detected; and for each detection target, based on the probability value corresponding to each initial frame boundary position information corresponding to the detection target, determines the target detection frame position information corresponding to the detection target, wherein the pre-established target detection model is: a model trained based on a sample image and its corresponding calibration information, and the calibration information includes: calibration frame position information corresponding to the sample target in the corresponding sample image.

[0051] By applying the embodiments of the present invention, the regression layer based on the pre-established target detection model can regress the probability value corresponding to each initial frame boundary position information corresponding to each detected target, that is, regress the probability distribution of the frame boundaries in various directions corresponding to each detected target, that is, the uncertainty of the frame boundaries, and then based on the probability value corresponding to each initial frame boundary position information corresponding to each detected target, determine the position information of the frame boundaries in various directions to determine the target detection frame position information corresponding to the detected target, so as to achieve more accurate detection of the frame boundaries of the target in the image. Of course, implementing any product or method of the present invention does not necessarily require achieving all of the advantages described above at the same time.

[0052] The innovative features of the embodiments of the present invention include:

[0053] 1. The regression layer based on the pre-established target detection model can regress the probability value corresponding to each initial frame boundary position information corresponding to each detection target, that is, regress the probability distribution of each frame boundary corresponding to each detection target, that is, the uncertainty of the frame boundary, and then determine the position information of each frame boundary based on the probability value corresponding to each initial frame boundary position information corresponding to each detection target, so as to determine the target detection frame position information corresponding to the detection target, so as to achieve more accurate detection of the frame boundary of the target in the image.

[0054] 2. For each detection target, based on the probability value corresponding to each initial frame boundary position information corresponding to the detection target in the four directions of up, down, left and right of the detection image, that is, the probability distribution of the frame boundary in each direction of the detection target, that is, the uncertainty, determine the target detection frame boundary position information corresponding to the detection target in each direction, so as to determine the target detection frame position information corresponding to the detection target, and obtain more accurate frame boundaries in each direction to determine the position information of the detection target with more accurate position.

[0055] 3. For each detection target, based on the probability value corresponding to each initial frame boundary position information corresponding to the detection target in each direction and the preset integral formula, the target detection frame boundary position information corresponding to the detection target in each direction is obtained by integration to determine the position information of the detection target with more accurate position.

[0056] 4. In the process of training the pre-established target detection model, the probability value corresponding to each prediction box boundary position information of the sample target in each direction in the sample image is regressed through the feature regression layer of the initial target detection model, that is, the probability value corresponding to the different prediction box boundary position information in each direction is regressed, that is, the probability distribution of the sample target in each direction, that is, the uncertainty, and then the current box position information corresponding to the sample target is determined by combining the preset integral formula and the probability value corresponding to each prediction box boundary position information corresponding to each direction of the sample target; for each sample image, the preset loss function and the probability value of each sample in the sample image are used. The current position information corresponding to the target and the calibration frame position information corresponding to each sample target in the calibration information corresponding to the sample image are used to determine the current loss value to determine whether to adjust the model parameters of the feature extraction layer and the feature regression layer of the initial target detection model, that is, whether to train the initial target detection model to obtain a pre-established target detection model that has the ability to learn the probability distribution, that is, the uncertainty, of the frame boundaries of the target in various directions in the predicted image, and to enable the pre-established target detection model to learn a more robust and robust representation of image features, thereby providing a basis for the accurate determination of the frame boundary position information of subsequent targets and providing a reference for the subsequent tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely some embodiments of the present invention. Those skilled in the art can derive other drawings based on these drawings without inventive effort.

[0058] Figure 1 A schematic diagram of a flow chart of a target detection method provided by an embodiment of the present invention;

[0059] Figure 2 A flowchart of the training process of a pre-established object detection model;

[0060] Figure 3 A visual diagram of the probability value corresponding to each initial box boundary position information corresponding to the detection target in each direction;

[0061] Figure 4A schematic diagram of the structure of a target detection device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0062] The following will provide a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0063] It should be noted that the terms "including" and "having" and any variations thereof in the embodiments of the present invention and the accompanying drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to the process, method, product, or apparatus.

[0064] The present invention provides a method and apparatus for object detection, so as to achieve more accurate detection of the boundary of an object in an image.

[0065] Figure 1 A flow chart of a target detection method provided by an embodiment of the present invention. The method may include the following steps:

[0066] S101: Obtain an image to be detected.

[0067] The target detection method provided by the embodiment of the present invention can be applied to any electronic device with computing capabilities, and the electronic device can be a terminal or a server. In one implementation, the electronic device can be a vehicle-mounted device, which is arranged on a vehicle. The vehicle can also be provided with an image acquisition device, which can acquire images of the environment in which the vehicle is located. The electronic device is connected to the image acquisition device, and the image acquired by the image acquisition device can be obtained as the image to be detected. In another implementation, the electronic device can be a non-vehicle-mounted device, and the electronic device can be connected to an image acquisition device that shoots a target scene, and the image acquired by the image acquisition device for the target scene can be obtained as the image to be detected. In one case, the target scene can be a road scene, a square scene, or an indoor scene, which is all possible.

[0068] The image to be detected may be an RGB (Red Green Blue) image or an infrared image, which is acceptable. The embodiment of the present invention does not limit the type of the image to be detected.

[0069] S102: Using a feature extraction layer of a pre-established target detection model to perform feature extraction on the image to be detected, and determining the image features to be detected corresponding to the image to be detected.

[0070] In this step, the electronic device uses the feature extraction layer of the pre-established object detection model to perform convolution and pooling operations on the image to be detected to extract image features corresponding to the image to be detected as the image features to be detected. The feature extraction layer can use any method known in the relevant art for extracting image features, and the present invention is not limited thereto.

[0071] S103: Using the feature regression layer of the pre-established target detection model and the features of the image to be detected, determine the probability value corresponding to each initial frame boundary position information corresponding to each detection target detected in the image to be detected; and for each detection target, based on the probability value corresponding to each initial frame boundary position information corresponding to the detection target, determine the target detection frame position information corresponding to the detection target.

[0072] The pre-established object detection model is a model trained based on sample images and their corresponding calibration information. The calibration information includes the position of a calibration box corresponding to the sample object in the sample image. The feature regression layer of the pre-established object detection model has a predetermined number of regression outputs for each orientation, and each regression output for each orientation corresponds to a preset translation value.

[0073] After the electronic device obtains the features of the image to be detected, it uses the feature regression layer and the features of the image to be detected to regress and obtain the probability value corresponding to each initial frame boundary position information corresponding to each detection target detected in the image to be detected, wherein the probability value corresponding to each initial frame boundary position information corresponding to each detection target includes: the probability value corresponding to each initial frame boundary position information corresponding to the detection target in each direction, that is, the probability value corresponding to each initial frame boundary position information corresponding to the four directions of up, down, left and right of the image to be detected. The probability value corresponding to each initial frame boundary position information corresponding to the detection target in each direction can represent the discretized generalized probability density function of the frame boundary corresponding to the detection target in each direction, that is, the probability distribution of the frame boundary corresponding to the detection target in each direction, that is, it can represent the uncertainty of the position of the frame boundary corresponding to the detection target in each direction.

[0074] Subsequently, the electronic device determines the target detection frame boundary position information corresponding to each detection target in each direction based on the probability value corresponding to each initial frame boundary position information corresponding to the detection target in each direction, that is, the probability distribution of the frame boundary corresponding to the detection target in each direction, so as to determine the target detection frame position information corresponding to the detection target.

[0075] Among them, the process of using the feature regression layer and the features of the image to be detected to regress and obtain the probability value corresponding to each initial frame boundary position information corresponding to each detection target detected in the image to be detected can be: inputting the features of the image to be detected into the feature regression layer of a pre-established target detection model, the feature regression layer performs regression processing on the features of the image to be detected, and regresses to obtain a preset number of regression values ​​corresponding to each detection target in each orientation; for the regression value of each detection target in each orientation, using a preset excitation function, i.e., a softmax function, and a preset number of regression values ​​corresponding to the detection target in the orientation, a preset number of probability values ​​are determined, i.e., the probability value corresponding to each initial frame boundary position information corresponding to each detection target is obtained. Each regression value in the preset number of regression values ​​corresponding to the detection target in each orientation corresponds to a preset translation value mentioned later. After the detection target is translated from its corresponding activation point by the preset translation value, the initial frame boundary position information corresponding to the detection target can be obtained. The position of the activation point corresponding to the detection target is the position predicted and regressed by the pre-established target detection model. The way in which the pre-established target detection model predicts the position of the activation point corresponding to the regressed detection target can refer to the way in which the target detection model in the related technology predicts the position of the activation point corresponding to the target in the regression image, which will not be repeated here.

[0076] Wherein, the preset translation value is determined based on the calibration information corresponding to the sample image required for the pre-established target detection model obtained through training, and the calibration distance between each frame boundary of the calibration frame represented by the calibration frame position information and the center point of the calibration frame, that is, the calibration distance between the frame boundaries of the four directions of the calibration frame represented by the calibration frame position information and the center point of the calibration frame. Wherein, the minimum value in the preset translation value can be 0, and the maximum value in the preset translation value is the calibration distance with the largest value in the above-mentioned calibration distance. The values ​​between each translation value in the preset translation value can increase uniformly from the minimum value to the maximum value, or can increase unevenly, which is all possible.

[0077] By applying the embodiments of the present invention, the regression layer based on the pre-established target detection model can regress the probability value corresponding to each initial frame boundary position information corresponding to each detection target, that is, regress the probability distribution of each frame boundary corresponding to each detection target, that is, the uncertainty of the frame boundary, and then based on each detection target, based on the probability value corresponding to each initial frame boundary position information corresponding to the detection target, determine the position information of each frame boundary to determine the target detection frame position information corresponding to the detection target, so as to achieve more accurate detection of the frame boundary of the target in the image.

[0078] In another embodiment of the present invention, the step of determining, for each detected target, the target detection frame boundary position information corresponding to the detected target in each orientation based on the probability value corresponding to each initial frame boundary position information corresponding to the detected target in each orientation, may include:

[0079] For each detection target, the target detection frame boundary position information corresponding to the detection target in each direction is determined based on the probability value corresponding to each initial frame boundary position information corresponding to the detection target in each direction and the preset integral formula.

[0080] In this implementation, the electronic device uses the feature regression layer of the pre-established target detection model to determine the probability value corresponding to each initial frame boundary position information corresponding to each detection target in each orientation, that is, to determine the probability distribution corresponding to the frame boundary of each detection target in each orientation, that is, the uncertainty. Then, by integration, based on the probability value corresponding to each initial frame boundary position information corresponding to the detection target in each orientation and the preset integration formula, the target detection frame boundary position information corresponding to the detection target in each orientation is determined.

[0081] In another embodiment of the present invention, in the process of training the pre-established target detection model, the pre-established target detection model can learn the probability density function p(x) corresponding to the position of the box boundary corresponding to each direction of the target in the image, and in order to simplify the regression complexity of the feature regression layer of the target detection model, the regression range of the feature regression layer is correspondingly limited, and the regression range of the feature regression layer is limited to a0 and a n Between, a0 is the minimum value among the preset translation values ​​mentioned above, and a n is the maximum value among the preset translation values ​​mentioned above; and is discretized. Accordingly, the expression of the preset integral formula can be:

[0082]

[0083] Among them, P represents the target translation value between the position represented by the target detection box boundary position information corresponding to the detection target in the target orientation and the position of the activation point corresponding to the detection target predicted by the pre-established target detection model, and a i Indicates the preset translation value corresponding to the i-th initial frame boundary position information of the detected target in the target orientation, p(a i ) represents the probability value corresponding to the preset translation value corresponding to the i-th initial frame boundary position information of the detection target at the target orientation, that is, the probability value corresponding to the i-th initial frame boundary position information of the detection target at the target orientation, n+1 represents the total number of preset translation values ​​corresponding to the initial frame boundary position information of the detection target at the target orientation, and the target orientation is any orientation in the orientation.

[0084] Here, n is a positive integer, and n+1 is the preset number mentioned above.

[0085] In another embodiment of the present invention, before the step of determining, using the feature regression layer of the pre-established target detection model and the features of the image to be detected, a probability value corresponding to each initial frame boundary position information corresponding to each detected target in the image to be detected; and determining, for each detected target, the target detection frame position information corresponding to the detected target based on the probability value corresponding to each initial frame boundary position information corresponding to the detected target, the method may further include:

[0086] The process of training a pre-established target detection model, where, for example, Figure 2 As shown, the process may include the following steps:

[0087] S201: Obtain multiple sample images and calibration information corresponding to each sample image;

[0088] S202: Obtaining an initial target detection model;

[0089] S203: For each sample image, input the sample image into the feature extraction layer of the initial object detection model to extract the sample image features corresponding to the sample image;

[0090] S204: For each sample image, input the sample image features corresponding to the sample image into the feature regression layer of the initial object detection model to determine the probability value corresponding to each prediction box boundary position information corresponding to each orientation of the sample object in the sample image; and for each sample object in the sample image, determine the current box position information corresponding to the sample object based on the probability value corresponding to each prediction box boundary position information corresponding to each orientation of the sample object and a preset integral formula;

[0091] S205: For each sample image, determine a current loss value using a preset loss function, current position information corresponding to each sample target in the sample image, and calibration frame position information corresponding to each sample target in the calibration information corresponding to the sample image;

[0092] S206: Determine whether the current loss value exceeds a preset loss threshold;

[0093] S207: If it is determined that the current loss value exceeds the preset loss threshold, the model parameters of the feature extraction layer and the feature regression layer of the initial object detection model are adjusted, and the process returns to S203;

[0094] S208: If it is determined that the current loss value does not exceed the preset loss threshold, it is determined that the initial target detection model has reached a convergence state, and a pre-established target detection model including a feature extraction layer and a feature regression layer is determined.

[0095] In this implementation, the electronic device may further include a process of training a pre-established object detection model before determining the object detection result corresponding to the image to be detected. Accordingly, the electronic device obtains multiple sample images and calibration information corresponding to each sample image, including position information of a calibration frame corresponding to a sample object in the corresponding sample image. An initial target detection model is obtained, and the feature regression layer of the initial target detection model corresponds to a predetermined number of regression outputs for the four directions of the image, namely, the up, down, left, and right. For each sample image, the sample image is input into the feature extraction layer of the initial target detection model to extract the sample image features corresponding to the sample image. The sample image features corresponding to the sample image are input into the feature regression layer of the initial target detection model, and the feature regression layer performs regression processing on the sample image features to regress a preset number of regression values ​​corresponding to each sample target in each direction. For the regression value of each sample target in each direction, a preset excitation function, i.e., a softmax function, and a preset number of regression values ​​corresponding to the sample target in the direction are used to determine a preset number of probability values, i.e., determine the probability value corresponding to each prediction box boundary position information corresponding to the sample target in each direction in the sample image. For each sample target in the sample image, based on the probability value corresponding to each prediction box boundary position information corresponding to each direction corresponding to the sample target and a preset integral formula, the current box position information corresponding to the sample target is obtained by integration.

[0096] For each sample image, a current loss value is determined using a preset loss function, the current position information corresponding to each sample target in the sample image, and the calibration frame position information corresponding to each sample target in the calibration information corresponding to the sample image; a determination is made as to whether the current loss value exceeds a preset loss threshold; if the current loss value exceeds the preset loss threshold, it is determined that the initial target detection model has not reached a convergence state, and the model parameters of the feature extraction layer and feature regression layer of the initial target detection model are adjusted using a preset optimization algorithm, and the process returns to S203; if the current loss value does not exceed the preset loss threshold, it is determined that the initial target detection model has reached a convergence state, and a pre-established target detection model including a feature extraction layer and a feature regression layer is determined. The pre-established target detection model learns a probability density function of the position of the frame boundary of the target in the image through the above-mentioned training process, and can predict the probability distribution of the position of the frame boundary of the target in the image. It can explicitly model the uncertainty of the frame boundary of the target in the image, and the pre-established target detection model learns a more robust and robust representation of image features. This provides a basis for the subsequent accurate determination of the position information of the target in the image.

[0097] The preset loss function may be any type of loss function that can be used in a neural network model in the related art, and the present invention does not limit this. The preset optimization algorithm may include but is not limited to a gradient descent method.

[0098] In another embodiment of the present invention, after S103, the method may further include the following steps:

[0099] Output the probability value corresponding to each initial frame boundary position information corresponding to each detected detection target, and / or the target detection frame position information corresponding to each detection target.

[0100] In this implementation, in one case, the electronic device may directly output the probability value corresponding to each piece of initial frame boundary position information corresponding to each detected detection target, and / or the target detection frame position information corresponding to each detection target in the form of a numerical value. In another case, the electronic device may output the probability value corresponding to each piece of initial frame boundary position information corresponding to each detected detection target, and / or the target detection frame position information corresponding to each detection target in the form of a graphic.

[0101] Among them, when the electronic device outputs the target detection frame position information corresponding to each detection target, it can draw a detection frame represented by the target detection frame position information corresponding to each detection target at the corresponding position of the image to be detected, so as to output the target detection frame position information corresponding to each detection target, so that the user can intuitively observe the position of the detection target.

[0102] Among them, when outputting the probability value corresponding to each initial frame boundary position information corresponding to each detected detection target, the probability value corresponding to each initial frame boundary position information corresponding to each detection target can be visualized, that is, for the probability value corresponding to the initial frame boundary position information corresponding to each detection target in each orientation, the probability value corresponding to each initial frame boundary position information corresponding to the detection target in the orientation is used to draw a histogram, wherein the ordinate of the histogram can represent the probability value, and the abscissa of the histogram can represent the preset translation value. Figure 3 As shown, Figure 3 The left image in the figure is the image to be detected, and the four histograms on the right represent Figure 3 The probability values ​​corresponding to the preset translation values ​​at the left, top, right, and bottom positions of target A in the left image, that is, the probability values ​​corresponding to the box boundary position information at the left, top, right, and bottom positions of target A in the left image, that is, the probability distribution.

[0103] The probability distribution and uncertainty of the frame boundaries of the detected target in various directions can be observed intuitively in the form of graphics. If the probability value corresponding to the initial frame boundary position information represented by the histogram corresponding to a certain direction of the target presents a relatively flat probability distribution, it can be determined that the position of the frame boundary of the target in this direction has a high uncertainty; if the probability value corresponding to the initial frame boundary position information represented by the histogram corresponding to a certain direction of the target presents a relatively sharp distribution, it can be determined that the position of the frame boundary of the target in this direction has a high certainty, which can indicate that the pre-established target detection model has a clearer prediction and more accurate judgment of the position of the frame boundary. Figure 3 As shown, the positions of the frame boundaries on the upper side, the left side, and the right side of target A in the left image are relatively clear. Accordingly, the histogram corresponding to the upper side of target A determined by the target detection process provided by the embodiment of the present invention represents that the probability value corresponding to its corresponding initial frame boundary position information presents a relatively sharp distribution, that is, the probability distribution corresponding to the frame boundary corresponding to the upper side of target A presents a relatively sharp distribution; and the histogram corresponding to the left side of target A represents that the probability value corresponding to its corresponding initial frame boundary position information presents a relatively sharp distribution, that is, the probability distribution corresponding to the frame boundary corresponding to the left side of target A presents a relatively sharp distribution; and the histogram corresponding to the right side of target A represents that the probability value corresponding to its corresponding initial frame boundary position information presents a relatively sharp distribution, that is, the probability distribution corresponding to the frame boundary corresponding to the right side of target A presents a relatively sharp distribution. However, due to light factors, Figure 3The position of the box boundary on the lower side of target A in the middle left image is relatively blurred. Accordingly, the histogram corresponding to the lower side of target A determined by the target detection process provided by the embodiment of the present invention represents that the probability value corresponding to the initial box boundary position information presents a relatively flat distribution, that is, the probability distribution corresponding to the box boundary corresponding to the upper side of target A presents a relatively flat distribution.

[0104] Corresponding to the above method embodiment, the embodiment of the present invention provides a target detection device, such as Figure 4 As shown, the device may include:

[0105] An acquisition module 410 is configured to obtain an image to be detected;

[0106] The first determination module 420 is configured to perform feature extraction on the image to be detected using a feature extraction layer of a pre-established target detection model to determine the image features to be detected corresponding to the image to be detected;

[0107] The second determination module 430 is configured to determine the probability value corresponding to each initial frame boundary position information corresponding to each detection target detected in the image to be detected by using the feature regression layer of the pre-established target detection model and the features of the image to be detected; and for each detection target, based on the probability value corresponding to each initial frame boundary position information corresponding to the detection target, determine the target detection frame position information corresponding to the detection target, wherein the pre-established target detection model is: a model trained based on a sample image and its corresponding calibration information, and the calibration information includes: calibration frame position information corresponding to the sample target in the corresponding sample image.

[0108] By applying the embodiments of the present invention, the regression layer based on the pre-established target detection model can regress the probability value corresponding to each initial frame boundary position information corresponding to each detection target, that is, regress the probability distribution of each frame boundary corresponding to each detection target, that is, the uncertainty of the frame boundary, and then based on each detection target, based on the probability value corresponding to each initial frame boundary position information corresponding to the detection target, determine the position information of each frame boundary to determine the target detection frame position information corresponding to the detection target, so as to achieve more accurate detection of the frame boundary of the target in the image.

[0109] In another embodiment of the present invention, the second determining module 430 is specifically configured to

[0110] For each detection target, based on the probability value corresponding to each initial frame boundary position information corresponding to the detection target in each direction, the target detection frame boundary position information corresponding to the detection target in each direction is determined to determine the target detection frame position information corresponding to the detection target, wherein the direction includes the top, bottom, left and right directions of the image to be detected.

[0111] In another embodiment of the present invention, the second determining module 430 is specifically configured to

[0112] For each detection target, the target detection frame boundary position information corresponding to the detection target in each direction is determined based on the probability value corresponding to each initial frame boundary position information corresponding to the detection target in each direction and the preset integral formula.

[0113] In another embodiment of the present invention, the expression of the preset integral formula is:

[0114]

[0115] Wherein, P represents the target translation value between the position represented by the target detection frame boundary position information corresponding to the detection target in the target orientation and the position of the activation point corresponding to the detection target predicted by the pre-established target detection model, a i Indicates the preset translation value corresponding to the i-th initial frame boundary position information of the detected target in the target orientation, p(a i ) represents the probability value corresponding to the preset translation value corresponding to the i-th initial frame boundary position information of the detection target at the target orientation, n+1 represents the total number of preset translation values ​​corresponding to the initial frame boundary position information of the detection target at the target orientation, and the target orientation is any orientation among the orientations.

[0116] In another embodiment of the present invention, the apparatus further comprises:

[0117] A model training module (not shown in the figure) is configured to determine a probability value corresponding to each initial frame boundary position information corresponding to each detection target detected in the image to be detected using the feature regression layer of the pre-established target detection model and the features of the image to be detected; and for each detection target, before determining the target detection frame position information corresponding to the detection target based on the probability value corresponding to each initial frame boundary position information corresponding to the detection target, train a pre-established target detection model, wherein the model training module is specifically configured to obtain multiple sample images and calibration information corresponding to each sample image;

[0118] Obtain an initial target detection model;

[0119] For each sample image, input the sample image into the feature extraction layer of the initial target detection model to extract the sample image features corresponding to the sample image;

[0120] For each sample image, input the sample image features corresponding to the sample image into the feature regression layer of the initial target detection model to determine the probability value corresponding to each prediction box boundary position information corresponding to the sample target in each direction in the sample image; and for each sample target in the sample image, determine the current box position information corresponding to the sample target based on the probability value corresponding to each prediction box boundary position information corresponding to each direction corresponding to the sample target and a preset integral formula;

[0121] For each sample image, determine the current loss value using a preset loss function, the current position information corresponding to each sample target in the sample image, and the calibration frame position information corresponding to each sample target in the calibration information corresponding to the sample image;

[0122] Determining whether the current loss value exceeds a preset loss threshold;

[0123] If it is determined that the current loss value exceeds the preset loss threshold, the model parameters of the feature extraction layer and the feature regression layer of the initial object detection model are adjusted, and the process of inputting the sample image into the feature extraction layer of the initial object detection model for each sample image to extract the sample image features corresponding to the sample image is returned to execution;

[0124] If it is determined that the current loss value does not exceed the preset loss threshold, it is determined that the initial target detection model has reached a convergence state, and a pre-established target detection model including a feature extraction layer and a feature regression layer is determined.

[0125] In another embodiment of the present invention, the apparatus further comprises:

[0126] An output module (not shown in the figure) is configured to determine the probability value corresponding to each initial frame boundary position information corresponding to each detection target detected in the image to be detected using the feature regression layer of the pre-established target detection model and the features of the image to be detected; and for each detection target, based on the probability value corresponding to each initial frame boundary position information corresponding to the detection target, after determining the target detection frame position information corresponding to the detection target, output the probability value corresponding to each initial frame boundary position information corresponding to each detection target detected, and / or the target detection frame position information corresponding to each detection target.

[0127] The above-mentioned system and device embodiments correspond to the system embodiment and have the same technical effects as the method embodiment. For detailed descriptions, please refer to the method embodiment. The device embodiment is obtained based on the method embodiment. For detailed descriptions, please refer to the method embodiment section and will not be repeated here. It should be understood by those skilled in the art that the accompanying drawings are only schematic diagrams of one embodiment, and the modules or processes in the accompanying drawings are not necessarily required to implement the present invention.

[0128] Those skilled in the art will appreciate that the modules in the apparatuses of the embodiments may be distributed in the apparatuses of the embodiments as described in the embodiments, or may be located in one or more apparatuses different from the embodiments with corresponding changes. The modules in the above embodiments may be combined into one module or further divided into multiple sub-modules.

[0129] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A target detection method, characterized in that: The method comprises: Obtaining an image to be detected; Performing feature extraction on the image to be detected using a feature extraction layer of a pre-established target detection model to determine the image features to be detected corresponding to the image to be detected; Determine, by using the feature regression layer of the pre-established target detection model and the features of the image to be detected, a probability value corresponding to each initial frame boundary position information corresponding to each detection target detected in the image to be detected; and determine, for each detection target, the target detection frame position information corresponding to the detection target based on the probability value corresponding to each initial frame boundary position information corresponding to the detection target, wherein the pre-established target detection model is a model trained based on a sample image and its corresponding calibration information, the calibration information including: calibration frame position information corresponding to the sample target in the corresponding sample image; The step of determining, for each detection target, the target detection frame position information corresponding to the detection target based on the probability value corresponding to each initial frame boundary position information corresponding to the detection target, includes: For each detection target, based on the probability value corresponding to each initial frame boundary position information corresponding to the detection target in each direction and the preset integral formula, the target detection frame boundary position information corresponding to the detection target in each direction is determined to determine the target detection frame position information corresponding to the detection target, wherein the direction includes the top, bottom, left and right directions of the image to be detected.

2. The method according to claim 1, wherein The expression of the preset integral formula is: Wherein, P represents the target translation value between the position represented by the target detection frame boundary position information corresponding to the detection target in the target orientation and the position of the activation point corresponding to the detection target predicted by the pre-established target detection model, a i Indicates the preset translation value corresponding to the i-th initial frame boundary position information of the detected target in the target orientation, p(a i ) represents the probability value corresponding to the preset translation value corresponding to the i-th initial frame boundary position information of the detection target at the target orientation, n+1 represents the total number of preset translation values ​​corresponding to the initial frame boundary position information of the detection target at the target orientation, and the target orientation is any orientation among the orientations.

3. The method according to claim 1, wherein Determining, by using the feature regression layer of the pre-established target detection model and the features of the image to be detected, a probability value corresponding to each initial frame boundary position information corresponding to each detection target detected in the image to be detected; Before the step of determining, for each detected target, target detection frame position information corresponding to the detected target based on the probability value corresponding to each initial frame boundary position information corresponding to the detected target, the method further includes: The process of training a pre-established object detection model includes: Obtaining multiple sample images and calibration information corresponding to each sample image; Obtain an initial target detection model; For each sample image, input the sample image into the feature extraction layer of the initial target detection model to extract the sample image features corresponding to the sample image; For each sample image, input the sample image features corresponding to the sample image into the feature regression layer of the initial target detection model to determine the probability value corresponding to each prediction box boundary position information corresponding to the sample target in each direction in the sample image; and for each sample target in the sample image, determine the current box position information corresponding to the sample target based on the probability value corresponding to each prediction box boundary position information corresponding to each direction corresponding to the sample target and a preset integral formula; For each sample image, determine the current loss value using a preset loss function, the current position information corresponding to each sample target in the sample image, and the calibration frame position information corresponding to each sample target in the calibration information corresponding to the sample image; Determining whether the current loss value exceeds a preset loss threshold; If it is determined that the current loss value exceeds the preset loss threshold, the model parameters of the feature extraction layer and the feature regression layer of the initial object detection model are adjusted, and the step of inputting the sample image into the feature extraction layer of the initial object detection model for each sample image to extract the sample image features corresponding to the sample image is returned to execution; If it is determined that the current loss value does not exceed the preset loss threshold, it is determined that the initial target detection model has reached a convergence state, and a pre-established target detection model including a feature extraction layer and a feature regression layer is determined.

4. The method according to any one of claims 1 to 3, wherein Determining, by using the feature regression layer of the pre-established target detection model and the features of the image to be detected, a probability value corresponding to each initial frame boundary position information corresponding to each detection target detected in the image to be detected; After determining, for each detected target, the target detection frame position information corresponding to the detected target based on the probability value corresponding to each initial frame boundary position information corresponding to the detected target, the method further includes: Output the probability value corresponding to each initial frame boundary position information corresponding to each detected detection target, and / or the target detection frame position information corresponding to each detection target.

5. A target detection device, characterized in that: The device comprises: An acquisition module is configured to acquire an image to be detected; A first determination module is configured to perform feature extraction on the image to be detected using a feature extraction layer of a pre-established target detection model to determine image features to be detected corresponding to the image to be detected; The second determination module is configured to use the feature regression layer of the pre-established target detection model and the features of the image to be detected to determine the probability value corresponding to each initial frame boundary position information corresponding to each detection target detected in the image to be detected; and for each detection target, based on the probability value corresponding to each initial frame boundary position information corresponding to the detection target, determine the target detection frame position information corresponding to the detection target, wherein the pre-established target detection model is: a model trained based on a sample image and its corresponding calibration information, and the calibration information includes: calibration frame position information corresponding to the sample target in the corresponding sample image; The second determination module is specifically configured to determine, for each detection target, the target detection frame boundary position information corresponding to the detection target in each direction based on the probability value corresponding to each initial frame boundary position information corresponding to the detection target in each direction and a preset integral formula, so as to determine the target detection frame position information corresponding to the detection target, wherein the direction includes the top, bottom, left and right directions of the image to be detected.

6. The device according to claim 5, characterized in that The expression of the preset integral formula is: Wherein, P represents the target translation value between the position represented by the target detection frame boundary position information corresponding to the detection target in the target orientation and the position of the activation point corresponding to the detection target predicted by the pre-established target detection model, a i Indicates the preset translation value corresponding to the i-th initial frame boundary position information of the detected target in the target orientation, p(a i ) represents the probability value corresponding to the preset translation value corresponding to the i-th initial frame boundary position information of the detection target at the target orientation, n+1 represents the total number of preset translation values ​​corresponding to the initial frame boundary position information of the detection target at the target orientation, and the target orientation is any orientation among the orientations.

Images