Image detection method and device in dense scene, terminal and medium

Abstract

The application is suitable for the field of image detection technology, and provides an image detection method, device, terminal and medium in a dense scene, wherein the method comprises: identifying a sample image based on an image detection model to obtain an output prediction box, the prediction box corresponding to a prediction region of a target object in the sample image; determining a target proportion of a target region area in the prediction box in the prediction box based on an overlapping relationship between the prediction box and a real box; determining a target loss function based on the target proportion and an area intersection ratio between the prediction box and the real box; performing model iterative training on the image detection model based on the target loss function, and performing image detection based on the image detection model after the iterative training is completed. The scheme can improve the target detection accuracy in a dense image.

Description

Technical Field

[0001] This application belongs to the field of computer vision technology, and in particular relates to an image detection method, device, terminal and medium in dense scenes. Background Technology

[0002] Dense object detection is a computer vision task with applications in multiple fields such as smart services, smart agriculture, smart transportation, and smart tourism. It can be applied to scenarios such as intelligent supermarket shelf management, crop yield prediction, urban traffic flow estimation, and dense crowd detection in scenic spots.

[0003] Unlike natural scenes where targets of interest are sparsely distributed in images, dense target detection tasks often face three challenges:

[0004] 1) Numerous and crowded small-sized targets. Taking the typical dense object detection dataset SKU-110K as an example, the average number of targets of interest in an image is 147.4, and the maximum number of targets in a single image reaches 718. In comparison, the average number of targets of interest per image in the Pascal VOC 2012 dataset and the MS COCO 2018 dataset is only 2.3 and 7.7, respectively.

[0005] 2) The predicted bounding boxes are highly sensitive to boundaries. All targets of interest are clustered together, with almost no pixel spacing between adjacent targets. If the size of the predicted bounding boxes becomes too large, pixels belonging to other targets on the boundaries will be mixed in.

[0006] 3) Target features are not significant. Because a single image contains hundreds of targets of interest, convolutional neural networks are not sensitive to these numerous small targets when downsampling.

[0007] For the reasons mentioned above, in image detection results in dense scenes, adjacent predicted bounding boxes are not compact enough, and a large number of candidate boxes often overlap with each other, easily including confusing regions.

[0008] Furthermore, when determining the confidence level of candidate boxes, some optimal candidate boxes that are affected by surrounding bounding boxes are directly discarded, while some suboptimal candidate boxes are retained by the network as the final result because of their relatively high prediction confidence, which seriously affects the accuracy of target detection in dense images. Summary of the Invention

[0009] This application provides an image detection method, apparatus, terminal, and medium for dense scenes, to solve the problem that in image detection results in dense scenes, adjacent predicted bounding boxes are not compact enough, a large number of candidate boxes often overlap with each other in a large proportion, easily include confusing regions, and seriously affect the target detection accuracy in dense images.

[0010] The first aspect of this application provides an image detection method for dense scenes, including:

[0011] The sample image is identified based on the image detection model to obtain the output prediction box, which corresponds to the prediction region of the target object in the sample image;

[0012] Based on the overlap between the predicted bounding box and the ground truth bounding box, a target percentage of the area of ​​the target region in the predicted bounding box is determined; wherein, the target percentage includes a first percentage of the area of ​​the predicted bounding box falling into the ground truth bounding box in the predicted bounding box, or, the target percentage includes a first percentage of the area of ​​the predicted bounding box falling into the ground truth bounding box in the predicted bounding box and a second percentage of the area of ​​the predicted bounding box not falling into the ground truth bounding box in the predicted bounding box, wherein the ground truth bounding box corresponds to the labeled area of ​​the target object in the sample image;

[0013] Based on the target proportion and the area intersection-union ratio between the predicted box and the ground truth box, the target loss function is determined;

[0014] Based on the target loss function, the image detection model is iteratively trained, and image detection is performed based on the image detection model after iterative training.

[0015] A second aspect of this application provides an image detection apparatus for dense scenes, comprising:

[0016] The recognition module is used to recognize the sample image based on the image detection model and obtain the output prediction box, wherein the prediction box corresponds to the prediction region of the target object in the sample image;

[0017] The data determination module is used to determine the target proportion of the target region area in the prediction box within the prediction box based on the overlap relationship between the prediction box and the ground truth box; wherein, the target proportion includes a first proportion of the first area of ​​the prediction box falling into the ground truth box within the prediction box, or, the target proportion includes a first proportion of the first area of ​​the prediction box falling into the ground truth box within the prediction box and a second proportion of the second area of ​​the prediction box not falling into the ground truth box within the prediction box, wherein the ground truth box corresponds to the labeled area of ​​the target object in the sample image;

[0018] The loss function determination module is used to determine the target loss function based on the target proportion and the area intersection-union ratio between the predicted box and the ground truth box;

[0019] The model training module is used to perform iterative training of the image detection model based on the target loss function, and to perform image detection based on the image detection model after iterative training.

[0020] A third aspect of this application provides a terminal including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described in the first aspect.

[0021] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0022] The fifth aspect of this application provides a computer program product that, when run on a terminal, causes the terminal to perform the steps of the method described in the first aspect.

[0023] As can be seen from the above, in this embodiment, the overlap relationship between the predicted bounding box obtained by the image detection model and the pre-labeled ground truth bounding box in the sample image is determined. The target proportion of the area of ​​the predicted bounding box that falls into or does not fall into the ground truth bounding box is determined. Based on this, the target loss function is determined by combining the area intersection-union ratio between the predicted bounding box and the ground truth bounding box. Finally, the image detection model is iteratively trained based on the target loss function, and image detection is performed based on the iteratively trained image detection model. This process introduces the target proportion to reward the area of ​​the predicted bounding box that falls into the labeled bounding box based on the area intersection-union ratio, and to penalize and suppress the area that exceeds the labeled bounding box. This reduces the mutual overlap between adjacent candidate boxes, improves the model training effect, and enhances the target detection accuracy in dense images. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a flowchart of an image detection method in a dense scene provided in an embodiment of this application;

[0026] Figure 2 This is a data flow diagram corresponding to the dense intersection-union ratio loss provided in the embodiments of this application;

[0027] Figure 3This is a schematic diagram of the process for obtaining a target loss function containing at least one target loss function term, provided in an embodiment of this application.

[0028] Figure 4 This is a structural diagram of an image detection device for dense scenes provided in an embodiment of this application;

[0029] Figure 5 This is a structural diagram of a terminal provided in an embodiment of this application. Detailed Implementation

[0030] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0031] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0032] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0033] It should also be further understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0034] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrases "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0035] In specific implementations, the terminals described in the embodiments of this application include, but are not limited to, other portable devices such as mobile phones, laptop computers, or tablet computers with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads). It should also be understood that in some embodiments, the device is not a portable communication device, but a desktop computer with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads).

[0036] The following discussion describes terminals that include displays and touch-sensitive surfaces. However, it should be understood that terminals may include one or more other physical user interface devices such as physical keyboards, mice, and / or joysticks.

[0037] The terminal supports a variety of applications, such as one or more of the following: drawing applications, presentation applications, word processing applications, website creation applications, disc burning applications, spreadsheet applications, game applications, telephone applications, video conferencing applications, email applications, instant messaging applications, exercise support applications, photo management applications, digital camera applications, digital camcorder applications, web browsing applications, digital music player applications, and / or digital video player applications.

[0038] Various applications that can run on a terminal can use at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface and the corresponding information displayed on the terminal can be adjusted and / or changed between and / or within applications. In this way, the terminal's common physical architecture (e.g., the touch-sensitive surface) can support various applications with user interfaces that are intuitive and transparent to the user.

[0039] It should be understood that the sequence number of each step in this embodiment does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of this application embodiment.

[0040] To illustrate the technical solution described in this application, specific embodiments are provided below.

[0041] See Figure 1 , Figure 1 This is a flowchart of an image detection method for dense scenes provided in an embodiment of this application. Figure 1 .like Figure 1 As shown, an image detection method for dense scenes includes the following steps:

[0042] Step 101: Based on the image detection model, identify the sample image to obtain the output prediction box.

[0043] Image detection models include YOLO (You Only Look Once), CenterNET, and CNN.

[0044] When an image detection model identifies a target object from a sample image, a prediction box corresponding to the identification result will appear. The prediction box corresponds to the predicted region of the target object in the sample image.

[0045] The target objects in the sample image are closely arranged content to be identified; that is, the sample image is an image containing densely arranged content.

[0046] Step 102: Based on the overlap relationship between the predicted bounding box and the ground truth bounding box, determine the target proportion of the target area in the predicted bounding box within the predicted bounding box.

[0047] The target area includes the area where the predicted bounding box falls into the actual bounding box, or the area where the predicted bounding box falls into the actual bounding box and the area where the predicted bounding box does not fall into the actual bounding box.

[0048] Correspondingly, the target proportion includes the first proportion of the area of ​​the predicted box falling into the real box within the predicted box.

[0049] Alternatively, the target proportion includes a first proportion of the area of ​​the predicted box that falls into the real box within the predicted box and a second proportion of the area of ​​the predicted box that does not fall into the real box within the predicted box.

[0050] The sample image is a content-annotated image, in which the target objects in the image are annotated. The annotation information includes the ground truth bounding box corresponding to the target object, which corresponds to the annotated area of ​​the target object in the sample image.

[0051] Step 103: Determine the target loss function based on the target proportion and the area intersection-union ratio between the predicted box and the ground truth box.

[0052] The area intersection-over-union ratio (IoU) between the predicted bounding box and the ground truth bounding box is an important metric for evaluating the quality of the predicted bounding box. It measures the similarity between the predicted bounding box D and the ground truth bounding box G, and can be expressed in the following form:

[0053]

[0054] Among them, S D∩G S represents the area of ​​intersection between the regions contained in the predicted bounding box and the ground truth bounding box. D∪G This represents the area of ​​the union of the regions contained in the predicted bounding box and the ground truth bounding box.

[0055] When the predicted bounding box and the ground truth bounding box do not overlap, the IoU value is 0; the larger the overlap area between the predicted bounding box and the ground truth bounding box, the higher the IoU value; when the predicted bounding box and the ground truth bounding box completely overlap, the IoU value reaches a maximum of 1.

[0056] Given the above characteristics, the area intersection-union ratio (IoU) between the predicted bounding box and the ground truth bounding box has two main functions:

[0057] 1) Identifying positive and negative samples. IoU is considered an indicator of how well the correct detection box is separated from a large number of negative samples.

[0058] 2) Remove redundant positive bounding boxes. To ensure that appropriate bounding boxes are predicted, the model network is allowed to output a large number of predicted boxes. Therefore, a target of interest often contains many positive predicted boxes. IoU can find the predicted box with the highest similarity to the ground truth bounding box and remove the suboptimal bounding boxes.

[0059] In the first function, the identification formula can be expressed in the following form:

[0060]

[0061] A value of D = 0 indicates that the current predicted bounding box is a negative sample, and a value of D = 1 indicates that the current predicted bounding box is a positive sample. ξ represents the threshold for distinguishing between positive and negative samples, which is a hyperparameter set empirically, ξ∈(0,1). During network training and testing, ξ is generally set to a lower threshold to output more potential predicted bounding boxes. However, when evaluating the detection accuracy of the network, ξ is set to fixed values ​​such as 0.5, 0.75, and 0.95 to obtain a quantitative indicator under the same standard.

[0062] In the latter function, IoU is widely used in the network deduplication process based on the NMS (Non-Maximum Suppression) algorithm. In the network detection head, a large number of candidate boxes overlap with each other. The NMS algorithm can remove redundant bounding boxes and retain the detection boxes that best contain the target.

[0063] In this step, a target percentage is introduced to penalize and suppress areas outside the labeled box in the predicted frame, based on the area intersection ratio.

[0064] In implementation, based on the target proportion and the area intersection-union ratio between the predicted bounding box and the ground truth bounding box, the target loss function is determined, including:

[0065] Based on the target proportion and the area intersection-union ratio between the predicted box and the ground truth box, at least one target loss function term is determined; a target loss function containing at least one target loss function term is obtained.

[0066] By using the target proportion and the area intersection-union ratio between the predicted box and the ground truth box, and through different combinations of the two, different target loss function terms are determined. Based on different area ratio relationships, appropriate loss function terms are determined, thus obtaining a suitable target loss function.

[0067] Specifically, in one optional implementation, based on the target proportion and the area intersection-union ratio between the predicted bounding box and the ground truth bounding box, at least one target loss function term is determined, including:

[0068] Based on the first proportion and the area intersection-union ratio between the predicted box and the ground truth box, the first loss function term is determined.

[0069] This process corresponds to the case where the target percentage includes the first percentage.

[0070] Alternatively, a first loss function term may be determined based on the first proportion and the area intersection-union ratio between the predicted box and the ground truth box, and a second loss function term may be determined based on the second proportion.

[0071] This process corresponds to the case where the target percentage includes both the first percentage and the second percentage.

[0072] In the specific implementation process, based on the first proportion and the area intersection-union ratio (IUU) between the predicted bounding box and the ground truth bounding box, the first loss function term is determined. Specifically, this can be obtained by performing a function operation on the first proportion and the IUU. Optionally, this process specifically includes:

[0073] Based on the first proportion and the area intersection-union ratio between the predicted box and the ground truth box, the first loss function term is determined according to the following formula:

[0074]

[0075] in, Let Ii be the first loss function term, and IIoU be the inner loop intersection-union ratio; where, IoU is the area intersection-union ratio between the predicted bounding box and the ground truth bounding box, and IAR is the first percentage. This is the first loss function term.

[0076] In existing application scenarios, targets of interest are typically sparsely distributed in natural scenes. When two bounding boxes have some overlap, the one with lower confidence is naturally discarded. However, in dense scenes, because the bounding boxes predicted by the network are not absolutely accurate, two adjacent positive bounding boxes often have a large proportion of overlap. In such cases, directly using IoU to remove candidate boxes with a large overlap ratio as suboptimal boxes will cause some optimal candidate boxes influenced by surrounding bounding boxes to be directly discarded by the NMS algorithm, while some suboptimal candidate boxes will be retained by the network as the final result due to their relatively high prediction confidence.

[0077] In this embodiment, to overcome the aforementioned problems in dense scenes, an internal region reward term is designed for the detector in the image detection model to better identify candidate boxes based on IoU. We define the reward term as IAR, which is the first proportion mentioned above.

[0078] Specifically, the formula for calculating IAR is as follows:

[0079]

[0080] Among them, S D S represents the area of ​​the prediction box. G The area represents the ground truth bounding box, while IAR represents the area bonus where the predicted bounding box falls inside the ground truth bounding box.

[0081] Unlike IoU, which simply aims for a larger overlap with the true bounding boxes, IAR tends to predict more introverted bounding boxes.

[0082] Based on this, in the embodiments of this application, IAR and IoU are further combined to propose a new evaluation method called Inner Loop Crossover Ratio (IIoU), which can be expressed as:

[0083]

[0084] For any predicted bounding box D, find its corresponding ground truth bounding box G, where D, We can first calculate the area S of D and G. D and S G Then calculate the union area and intersection area S of D and G respectively. D∩G S D∪G Next, we can obtain the IoU and IAR of the predicted box D and the ground truth box G. The IoU is obtained by calculating their product and taking the square root.

[0085] As an evaluation criterion for dense object detection, IIoU inherits the advantages of IoU. Moreover, the larger the area of the predicted bounding box that exceeds the ground truth bounding box, the smaller the IIoU, and IIoU is always less than or equal to IoU. When (D∩G) = D, that is, when the predicted bounding box coincides with the ground truth bounding box, IIoU = IoU; when (D∩G) < D, that is, when the union area of the predicted bounding box and the ground truth bounding box is less than the predicted bounding box, IIoU < IoU.

[0086] In the above process, compared with IoU, IIoU adds a new penalty term and introduces an internal region reward factor to penalize and suppress the region of the predicted bounding box that is outside the ground truth annotation box, ensuring that the predicted bounding box can fall inside the ground truth bounding box as much as possible while maintaining a larger intersection over union, so as to ensure that the quality of the predicted bounding box during model training and testing can be comprehensively evaluated, and sub-optimal candidate bounding boxes are more likely to be identified and removed. Therefore, the image detection model has greater potential to improve the detection performance.

[0087] Furthermore, the process of determining the second loss function term based on the second ratio can specifically be taking the second ratio directly as the second loss function term. This second ratio forms the background loss The calculation formula is:

[0088]

[0089] where S D -S D∩G represents the area of the background region outside the target object in the predicted bounding box. That is the second loss function term.

[0090] This background loss can penalize the region of the predicted bounding box that is outside the ground truth bounding box (i.e., the background region of the target object), constraining the predicted bounding box to always converge inward rather than spread outward.

[0091] This loss function term penalizes the background region included in the model output result. If the predicted bounding box is inside the ground truth bounding box, the loss of this term is 0.

[0092] Correspondingly, when determining the first loss function term based on the first ratio and the intersection over union between the predicted bounding box and the ground truth bounding box, the obtained objective loss function is:

[0093]

[0094] Differently, when determining the first loss function term based on the first ratio and the intersection over union between the predicted bounding box and the ground truth bounding box, and determining the second loss function term based on the second ratio, the obtained objective loss function is:

[0095]

[0096] Furthermore, in an alternative implementation, a distance loss may be further introduced.

[0097] Because the gradient vanishing problem easily occurs during model training when the loss function term is determined based on the overlap between the predicted and ground truth boxes, this problem specifically refers to two scenarios. The first is when there is absolutely no overlap between the predicted and ground truth boxes. In this case, it's impossible to further determine the degree of positional deviation between the region selected by the predicted box and the ground truth box, leading to gradient vanishing during model training. The second is when the detection box completely falls inside the ground truth box, regardless of whether the detection box is located at the exact center of the ground truth box or has a positional offset relative to the center; in this case, gradient vanishing also occurs.

[0098] Therefore, distance loss is introduced. The gradient is updated by directly constraining the distance between the center points of the predicted bounding box and the ground truth bounding box. Distance loss. As shown below:

[0099]

[0100] Where E is the smallest bounding rectangle of the predicted box D and the ground truth box G. 2 (D,G) is the squared Euclidean distance between the center points of the prediction boxes D and G. It is the squared Euclidean distance of the diagonal of E. The distance loss is proportional to the distance between D and G, therefore the distance loss is... It can comprehensively measure the positional differences between the ground truth bounding boxes and the predicted bounding boxes. Distance loss. This is the third loss function term.

[0101] In dense scenes, adjacent targets of interest have almost no spacing (less than one pixel), causing candidate bounding boxes to inevitably include areas of other surrounding targets (i.e., background areas) when selecting a target. This negatively impacts both detection accuracy and recall.

[0102] In this embodiment of the application, in order to limit the size of the bounding box and reduce mutual interference between the neighborhood bounding boxes, the first loss function term, the second loss function term and the third loss function term are integrated to obtain the dense intersection-union ratio loss as the target loss function.

[0103] Combination Figure 2 As shown, the objective loss function consists of three branches:

[0104] First, there's the dense crossover ratio loss based on IIoU. The second term constrains the overlap between the predicted bounding box and the ground truth bounding box in dense scenes, and is the distance loss. The distance between the center points of the predicted bounding box and the ground truth bounding box is constrained and penalized; the third term is the background loss. Constraints are applied to background regions where the predicted bounding boxes do not fall within the ground truth bounding boxes in dense scenes. The resulting target loss function is as follows:

[0105]

[0106] That is, such as Figure 3 As shown, the process of obtaining a target loss function that includes at least one target loss function term includes:

[0107] Step 301: Determine the minimum bounding rectangle between the predicted bounding box and the ground truth bounding box;

[0108] Step 302: Obtain a first distance and a second distance. The first distance is the squared Euclidean distance between the center point of the predicted box and the center point of the true box, and the second distance is the squared Euclidean distance between the diagonal of the minimum bounding rectangle.

[0109] Step 303: Use the ratio of the first distance to the second distance as the third loss function term;

[0110] Step 304: Obtain a target loss function that includes at least one target loss function term and the third loss function term.

[0111] Introducing a distance loss function term to constrain and penalize the distance between the center points of the predicted bounding box and the ground truth bounding box, combined with constraining and penalizing the overlapping regions or non-overlapping background regions between the predicted bounding box and the ground truth bounding box in dense scenes, further ensures the effectiveness of the loss function and avoids gradient vanishing during model training.

[0112] Furthermore, in an optional implementation, the set loss function term in the target loss function is equipped with a weighting factor.

[0113] The target loss function expression corresponding to the aforementioned dense intersection-union ratio loss then becomes:

[0114]

[0115] The distance loss and background loss are adjusted by a weighting factor α within the range of (0,1). This becomes the dominant loss branch during training. Here, α is a hyperparameter that can be adjusted as needed.

[0116] Step 104: Based on the target loss function, perform iterative training on the image detection model, and perform image detection based on the image detection model after iterative training.

[0117] When the target loss function has a weighting factor, this step, based on the target loss function, iteratively trains the image detection model, including:

[0118] In each iteration of training the image detection model, the calculated value of the set loss function term is obtained;

[0119] Based on the calculated value, the weighting factor of the set loss function term is adjusted; wherein the value of the weighting factor is positively correlated with the calculated value of the set loss function term;

[0120] The target loss function is updated according to the values ​​of the coefficients, and the image detection model is iteratively trained based on the updated target loss function.

[0121] That is, each time the model is trained, the weighting factor of the set loss function term in the target loss function will be adjusted once based on the calculated value of the set loss function term, so that the image detection model can be trained iteratively based on the target loss function after the weighting factor is adjusted.

[0122] Furthermore, in order to ensure the balanced adjustment of the weighting factors, this application embodiment introduces a loss inheritance strategy called an adaptive loss equalizer.

[0123] The adaptive loss equalizer introduces the concept of softmax regularization, mapping the loss weights to a probability value within the range [0,1]. Assume... The k loss function terms that need to be integrated represent the total loss. It can be calculated using the following formula:

[0124]

[0125] Assuming in the Nth network iteration Get bigger and If the value decreases, the adaptive loss equalizer will dynamically increase in the next iteration. Weight and reduce The weights. This training strategy allows the network to focus more on the error of each iteration sample.

[0126] In one specific implementation, the aforementioned defined loss function term is the i-th loss function term, and there can be multiple defined loss function terms. The coefficients of the i-th loss function term are then calculated based on the following formula:

[0127]

[0128] in, Let α be the i-th loss function term. i The coefficient of the i-th loss function term is set to a value;

[0129] Where α1+α2+…+α n =1, n is the total number of loss function terms in the target loss function, i≤n, and i and n are both positive integers.

[0130] In one calculation method, taking the target loss function corresponding to the aforementioned dense intersection-union ratio loss as an example, the following assumptions can be made:

[0131]

[0132]

[0133] The objective loss function, under the action of the adaptive dynamic equalizer, can be expressed as:

[0134]

[0135] In this process, α is transformed from a manually set hyperparameter into a dynamically changing adjustment factor that varies with the training samples, enabling model regression iterative training with better crossover ratio (CUP) representation capabilities.

[0136] After the model is trained, image detection using this model can accurately detect densely packed target objects in an image.

[0137] In this embodiment, the overlap relationship between the predicted bounding boxes obtained by the image detection model from the sample image and the pre-labeled ground truth bounding boxes in the sample image is determined. The target proportion of the area of ​​the predicted bounding box that falls into or does not fall into the ground truth bounding box is determined. Based on this, the target loss function is determined by combining the area intersection-union ratio between the predicted bounding box and the ground truth bounding box. Finally, the image detection model is iteratively trained based on the target loss function, and image detection is performed based on the iteratively trained image detection model. This process introduces the target proportion to reward the area of ​​the predicted bounding box that falls into the labeled bounding box based on the area intersection-union ratio, and to penalize and suppress the area that exceeds the labeled bounding box. This reduces the mutual overlap between adjacent candidate boxes, improves the model training effect, and enhances the target detection accuracy in dense images.

[0138] See Figure 4 , Figure 4 This is a structural diagram of an image detection device for dense scenes provided in an embodiment of this application. For ease of explanation, only the parts related to the embodiment of this application are shown.

[0139] The image detection device 400 for dense scenes includes:

[0140] The recognition module 401 is used to recognize the sample image based on the image detection model and obtain the output prediction box, wherein the prediction box corresponds to the prediction region of the target object in the sample image;

[0141] The data determination module 402 is used to determine the target proportion of the target region area in the prediction box within the prediction box based on the overlap relationship between the prediction box and the ground truth box; wherein, the target proportion includes a first proportion of the first area of ​​the prediction box falling into the ground truth box within the prediction box, or, the target proportion includes a first proportion of the first area of ​​the prediction box falling into the ground truth box within the prediction box and a second proportion of the second area of ​​the prediction box not falling into the ground truth box within the prediction box, wherein the ground truth box corresponds to the labeled area of ​​the target object in the sample image;

[0142] The loss function determination module 403 is used to determine the target loss function based on the target proportion and the area intersection-union ratio between the predicted box and the ground truth box;

[0143] The model training module 404 is used to perform iterative training of the image detection model based on the target loss function, and to perform image detection based on the image detection model after iterative training.

[0144] The loss function determination module 403 is specifically used for:

[0145] Based on the target proportion and the area intersection-union ratio between the predicted box and the ground truth box, at least one target loss function term is determined;

[0146] A target loss function containing at least one of the target loss function terms is obtained.

[0147] The loss function determination module 403 is more specifically used for:

[0148] Based on the first proportion and the area intersection-union ratio between the predicted box and the ground truth box, a first loss function term is determined; or,

[0149] A first loss function term is determined based on the first proportion and the area intersection-union ratio between the predicted box and the ground truth box, and a second loss function term is determined based on the second proportion.

[0150] The loss function determination module 403 is further specifically used for:

[0151] Based on the first proportion and the area intersection-union ratio between the predicted box and the ground truth box, the first loss function term is determined according to the following formula:

[0152]

[0153] in, Let Ii be the first loss function term, and IIoU be the inner loop intersection-union ratio; where, IoU is the area intersection-union ratio between the predicted bounding box and the ground truth bounding box, and IAR is the first percentage.

[0154] The loss function determination module 403 is more specifically used for:

[0155] Determine the minimum bounding rectangle between the predicted bounding box and the ground truth bounding box;

[0156] Obtain a first distance and a second distance, wherein the first distance is the squared Euclidean distance between the center point of the predicted box and the center point of the ground truth box, and the second distance is the squared Euclidean distance between the center point and the center point of the ground truth box in the minimum bounding rectangle;

[0157] The ratio of the first distance to the second distance is used as the third loss function term;

[0158] A target loss function is obtained that includes at least one target loss function term and the third loss function term.

[0159] The target loss function includes a weighting factor in its defined loss function term; the model training module 404 is specifically used for:

[0160] In each iteration of training the image detection model, the calculated value of the set loss function term is obtained;

[0161] Based on the calculated value, the weighting factor of the set loss function term is adjusted; wherein the value of the weighting factor is positively correlated with the calculated value of the set loss function term;

[0162] The target loss function is updated according to the values ​​of the coefficients, and the image detection model is iteratively trained based on the updated target loss function.

[0163] Wherein, the set loss function term is the i-th loss function term, and the coefficient of the i-th loss function term is calculated based on the following formula:

[0164]

[0165] in, Let α be the i-th loss function term. iThe coefficient of the i-th loss function term is set to a value;

[0166] Where α1+α2+…+α n =1, n is the total number of loss function terms in the target loss function, i≤n, and i and n are both positive integers.

[0167] The image detection device for dense scenes provided in this application embodiment can implement all the processes of the above-described image detection method for dense scenes embodiment and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0168] Figure 5 This is a structural diagram of a terminal provided in an embodiment of this application. As shown in the figure, the terminal 5 of this embodiment includes: at least one processor 50 ( Figure 5 (Only one is shown in the diagram), memory 51, and computer program 52 stored in said memory 51 and executable on said at least one processor 50, wherein said processor 50 executes said computer program 52 to implement the steps in any of the above method embodiments.

[0169] The terminal 5 can be a computing device such as a desktop computer, laptop, handheld computer, or cloud server. The terminal 5 may include, but is not limited to, a processor 50 and a memory 51. Those skilled in the art will understand that... Figure 5 This is merely an example of terminal 5 and does not constitute a limitation on terminal 5. It may include more or fewer components than shown, or combine certain components, or different components. For example, the terminal may also include input / output devices, network access devices, buses, etc.

[0170] The processor 50 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0171] The memory 51 can be an internal storage unit of the terminal 5, such as a hard disk or memory of the terminal 5. The memory 51 can also be an external storage device of the terminal 5, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the terminal 5. Furthermore, the memory 51 can include both internal storage units and external storage devices of the terminal 5. The memory 51 is used to store the computer program and other programs and data required by the terminal. The memory 51 can also be used to temporarily store data that has been output or will be output.

[0172] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0173] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0174] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0175] In the embodiments provided in this application, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0176] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0177] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0178] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0179] The methods described in this application can be implemented in whole or in part by a computer program product. When the computer program product is run on a terminal, the terminal executes the steps in the various method embodiments described above.

[0180] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. An image detection method in a dense scene, characterized by, The method comprises: identifying a sample image based on an image detection model to obtain an output prediction box, the prediction box corresponding to a predicted region of a target object in the sample image; determining a target area ratio of a target region in the prediction box based on an overlapping relationship between the prediction box and a real box, wherein the target area ratio comprises a first area ratio of a first area of the prediction box falling within the real box in the prediction box, or the target area ratio comprises the first area ratio of the first area of the prediction box falling within the real box in the prediction box and a second area ratio of a second area of the prediction box not falling within the real box in the prediction box, the real box corresponding to a labeled region of the target object in the sample image; determining a first loss function term based on the first area ratio in the target area ratio and an area intersection ratio between the prediction box and the real box according to the following formula: ； wherein, is the first loss function term, is the inner loop IoU; wherein, ; is the area IoU between the predicted box and the real box, is the first proportion; obtaining a target loss function comprising at least one target loss function term, wherein the target loss function term comprises the first loss function term; performing model iterative training on the image detection model based on the target loss function, and performing image detection based on the image detection model after the iterative training is completed.

2. The method of claim 1, wherein, The target loss function term further comprises a second loss function term, and the second loss function term is the second area ratio.

3. The method of claim 1, wherein, The method of obtaining the target loss function comprising at least one target loss function term comprises: determining a minimum circumscribed rectangle of the prediction box and the real box; obtaining a first distance and a second distance, wherein the first distance is a squared Euclidean distance between the centers of the prediction box and the real box, and the second distance is a squared Euclidean distance corresponding to a diagonal line in the minimum circumscribed rectangle; using a distance ratio of the first distance and the second distance as a third loss function term; obtaining a target loss function comprising at least one target loss function term and the third loss function term.

4. The method of claim 1, wherein, The target loss function comprises a set loss function term provided with a weighting factor; and the method of performing model iterative training on the image detection model based on the target loss function comprises: obtaining a calculation value of the set loss function term in each iterative training process of the image detection model; adjusting the weighting factor of the set loss function term based on the calculation value, wherein the value of the weighting factor is positively correlated with the calculation value of the set loss function term; updating the target loss function according to the weighting factor, and performing model iterative training on the image detection model based on the updated target loss function.

5. The method of claim 4, wherein, The set loss function term is the first The set loss function term is the first The weighting factor of the first loss function term is calculated based on the following formula: ； wherein, is the first loss function term, is the first loss function term; wherein, , is the total amount of loss function terms in the target loss function, , are all positive integers.

6. An image detection device in a dense scene, characterized by The method comprises: an identification module configured to identify a sample image based on an image detection model to obtain an output prediction box, the prediction box corresponding to a predicted region of a target object in the sample image; The data determination module is configured to determine a target area proportion of the target region in the prediction box based on an overlapping relationship between the prediction box and the real box, wherein the target area proportion comprises a first proportion of a first area of the prediction box falling into the real box in the prediction box, or the target area proportion comprises the first proportion of the first area of the prediction box falling into the real box in the prediction box and a second proportion of a second area of the prediction box not falling into the real box in the prediction box, and the real box corresponds to a labeled region of the target object in the sample image. The loss function determination module is configured to determine a first loss function term based on the first proportion in the target area proportion and an area intersection over union between the prediction box and the real box according to the following formula: ； wherein, is the first loss function term, is the inner loop IoU; wherein, ; is the area IoU between the predicted box and the real box, is the first proportion; A target loss function comprising at least one target loss function term is obtained, and the target loss function term comprises the first loss function term. The model training module is configured to perform model iterative training on the image detection model based on the target loss function, and perform image detection based on the image detection model after the iterative training is completed.

7. A terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, the computer-readable storage medium comprising: The computer program is executed by the processor to implement the steps of the method according to any one of claims 1 to 5.

Images