Target detection candidate box screening method, system and device and medium

Abstract

The invention is suitable for the technical field of computer vision, and provides a target detection candidate box screening method, which comprises the following steps: detecting a target image through a pre-trained target detection model to obtain a plurality of detection candidate boxes; based on the color distribution, the contrast ratio and the texture features of each detection candidate frame, obtaining a first evaluation score of each detection candidate frame, the first evaluation score being used for evaluating the mirror image in the detection candidate frame; obtaining a final evaluation score of each detection candidate box based on the color distribution, the position feature and the first evaluation score of each detection candidate box; and sorting the final evaluation scores corresponding to the plurality of detection candidate frames, and selecting the detection candidate frame corresponding to the maximum final evaluation score as a target candidate frame. According to the method, the influence of the mirror image and the local image on the screening of the target candidate box is comprehensively considered, so that the possibility of occurrence of mirror image false detection and local false detection is reduced, and the screening accuracy is improved.

Description

Technical Field

[0001] The present application belongs to the field of computer vision technology, and in particular, relates to a method, system, device and medium for screening candidate frames for target detection. Background Art

[0002] As a core technology in the field of computer vision, object detection has made significant progress in academic research and industrial applications in recent years, and its importance is self-evident. This technology aims to accurately identify and locate specific objects or objects in images or videos, bringing revolutionary changes to many fields such as autonomous driving, security monitoring, medical health, retail logistics and agriculture.

[0003] In the existing target detection process, when the detection scene contains reflective objects such as mirrors and glass, a mirror image of the target to be detected may be generated. Such a mirror image is not the detection target itself, but its existence constitutes a significant interference to the target detection task. Especially in the field of autonomous driving, the mirror image may cause the autonomous driving system to misjudge the position of obstacles, thereby causing potential safety risks. In addition, during the detection process, there may be a situation where only part of the target object is captured instead of the whole. This local misdetection not only reduces the accuracy of the detection, but also may mislead subsequent analysis and decision-making, such as lesion identification in medical health, commodity inventory monitoring in retail logistics, and pest and disease detection in the agricultural field. Local misdetection may have an adverse effect on the final result.

[0004] In the prior art, in order to solve the above problems, the Soft-NMS method is usually used. During the detection process, multiple candidate boxes are sorted from high to low according to the confidence of the detection box, and then the detection box with the highest confidence is selected as the reference box, and the IoU between other detection boxes and the reference box is calculated; if the IoU exceeds the preset threshold, the confidence of the detection box is weighted according to the size of the IoU, thereby reducing the confidence of the detection box to reduce redundancy. In the above method, the determination of the optimal threshold has a great influence on the screening result. At the same time, the confidence of each detection box is weighted, which inevitably increases the calculation complexity, and its detection speed and accuracy are low. Summary of the invention

[0005] The embodiments of the present application provide a method, system, device and medium for screening candidate boxes for target detection, which can solve one of the above-mentioned problems in the prior art.

[0006] In a first aspect, an embodiment of the present application provides a method for screening candidate boxes for target detection, comprising:

[0007] Detect the target image through the pre-trained target detection model to obtain multiple detection candidate boxes;

[0008] Obtaining a first evaluation score for each of the detection candidate frames based on color distribution, contrast, and texture features of each of the detection candidate frames, wherein the first evaluation score is used to evaluate the mirror image in the detection candidate frame;

[0009] Obtaining a final evaluation score for each of the detection candidate boxes based on the color distribution, position features, and first evaluation score of each of the detection candidate boxes;

[0010] The final evaluation scores corresponding to the multiple detection candidate boxes are sorted, and the detection candidate box corresponding to the maximum final evaluation score is selected as the target candidate box.

[0011] Furthermore, obtaining a first evaluation score for each of the detection candidate frames based on the color distribution, contrast, and texture features of each of the detection candidate frames includes:

[0012] Calculating a pixel average value of the image in each of the detection candidate frames, wherein the pixel average value is used to represent the color distribution;

[0013] Calculating the pixel variance of the image in each of the detection candidate frames, wherein the pixel variance is used to represent the contrast;

[0014] Performing a convolution operation on each image in the detection candidate frame to obtain a convolution average value of the image in the detection candidate frame, wherein the convolution average value is used to represent the texture feature;

[0015] The calculation formula of the first evaluation score is as follows:

[0016]

[0017] Among them, μ x represents the pixel average value, represents the pixel variance, μ y The convolution average.

[0018] Furthermore, the calculation formula for calculating the pixel variance of each image in the detection candidate frame is as follows:

[0019]

[0020] Among them, x i represents the average value of all color channel pixel values ​​of each pixel in the detection candidate frame, μ x represents the pixel average value, and N represents the number of pixels in the detection candidate frame.

[0021] Furthermore, performing a convolution operation on each of the images in the detection candidate frame to obtain a convolution average of the images in the detection candidate frame includes:

[0022] A 3×3 convolution kernel is used to perform a convolution operation on each color channel of the image in the detection candidate frame to obtain a first average value, where the first average value represents the average value of all color channel pixel values ​​of each pixel point in the detection candidate frame after the convolution operation.

[0023] The calculation formula of the first average value is as follows:

[0024] y i =s*x i

[0025] Among them, s represents the convolution kernel, x i is a second average value, which represents the average value of all color channel pixel values ​​of each pixel point in the detection candidate frame before the convolution operation;

[0026] Based on the first average value, the convolution average value is calculated, wherein the calculation formula of the convolution average value is as follows:

[0027]

[0028] Among them, y i represents the first average value, and M represents the number of pixels in the detection candidate box after the convolution operation.

[0029] Furthermore, obtaining a final evaluation score of each detection candidate box based on the color distribution, position feature and first evaluation score of each detection candidate box includes:

[0030] For the color distribution, calculating the pixel average of the image in each detection candidate frame;

[0031] According to the position feature, the position coordinates of all vertices of the detection candidate box are obtained, and the coordinates of the center point of the detection candidate box are calculated;

[0032] For the first evaluation score, a second evaluation score is calculated by using an intersection-and-union method;

[0033] The calculation formula for the final evaluation score is as follows:

[0034]

[0035] Among them, μ x represents the pixel average value, x′ and y′ respectively represent the abscissa and ordinate of the center point coordinates, and s′ represents the second evaluation score.

[0036] Furthermore, the method of using intersection-over-union ratio to calculate the second evaluation score for the first evaluation score includes:

[0037] sorting the first evaluation scores corresponding to the plurality of detection candidate frames, and selecting the detection candidate frame corresponding to the maximum first evaluation score as the initial candidate frame;

[0038] Perform intersection and union operations on the remaining detection candidate frames and the initial candidate frame one by one, and calculate the intersection and union ratio between each detection candidate frame and the initial candidate frame. The calculation formula of the intersection and union ratio is as follows:

[0039]

[0040] Wherein, I represents the intersection-over-union ratio between the detection candidate box and the initial candidate box, A represents the initial candidate box, B represents the detection candidate box, |A∩B| is the intersection area of ​​the intersection area of ​​the detection candidate box and the initial candidate box, and |A∪B| is the union area of ​​the union area of ​​the detection candidate box and the initial candidate box;

[0041] The calculation formula of the second evaluation score is as follows:

[0042]

[0043] Among them, s1 represents the first evaluation score, exp represents the power function with the natural constant e as the base, and δ 2 is a hyperparameter.

[0044] Furthermore, the calculating the intersection-over-union ratio between each of the detected candidate boxes and the initial candidate box includes:

[0045] Based on the upper left corner coordinates, width and height of the detection candidate box and the initial candidate box, obtain the upper left corner coordinates and the lower right corner coordinates of the intersection area;

[0046] Based on the upper left corner coordinates and the lower right corner coordinates, calculate the intersection width and the intersection height of the intersection area;

[0047] Calculating the intersection area based on the intersection width and the intersection height;

[0048] The area of ​​the detection candidate box and the initial candidate box are calculated respectively, and the union area is obtained based on the two area areas and the intersection area.

[0049] In a second aspect, an embodiment of the present application provides a target detection candidate box screening system, including:

[0050] The first processing module is used to detect the target image through the pre-trained target detection model to obtain multiple detection candidate boxes;

[0051] A second processing module: used for obtaining a first evaluation score for each of the detection candidate frames based on the color distribution, contrast and texture features of each of the detection candidate frames, wherein the first evaluation score is used for evaluating the mirror image in the detection candidate frame;

[0052] A third processing module: used for obtaining a final evaluation score of each of the detection candidate boxes based on the color distribution, position features and the first evaluation score of each of the detection candidate boxes;

[0053] The fourth processing module is used to sort the final evaluation scores corresponding to the multiple detection candidate frames, and select the detection candidate frame corresponding to the maximum final evaluation score as the target candidate frame.

[0054] In a third aspect, an embodiment of the present application provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned target detection candidate box screening method when executing the computer program.

[0055] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, including a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the above-mentioned target detection candidate box screening method is implemented.

[0056] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0057] The present invention discloses a method for screening target detection candidate frames, which comprehensively considers the influence of mirror images and local images on the screening of target candidate frames, thereby reducing the possibility of mirror misdetection and local misdetection, and on this basis calculates the final evaluation score of each detection candidate frame, evaluates each detection candidate frame according to the size of the final evaluation score, improves the accuracy of screening, and at the same time, there is no need to determine the optimal threshold in the process of determining the final evaluation score, thereby reducing the number of screening times, thereby effectively reducing the amount of calculation in the target candidate frame screening process, and improving the screening accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0059] Figure 1 It is a flowchart of a method for screening candidate frames for target detection provided by an embodiment of the present invention;

[0060] Figure 2 It is a structural diagram of a target detection candidate box screening system provided by an embodiment of the present invention;

[0061] Figure 3 It is a schematic diagram of the structure of the terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0062] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0063] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0064] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0065] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0066] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0067] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0068] See also Figure 1 As shown, the present invention is a method for screening candidate frames for target detection, comprising the following steps:

[0069] S100, detecting the target image through a pre-trained target detection model to obtain multiple detection candidate boxes;

[0070] In this embodiment, the target detection model is pre-trained and can detect the target image when the target image is input, and output multiple detection candidate boxes. The multiple candidate boxes are closely surrounded by the target object, and each detection candidate box includes position information, wherein the position information includes vertex coordinates, width and height, and the vertex coordinates include the position coordinates of all vertices of the detection candidate box.

[0071] In this embodiment, the multiple detection candidate frames output by the target detection model include candidate frames of the mirror image of the target object and candidate frames of the local image of the target object. The target candidate frame is obtained by filtering out the candidate frames of the mirror image and the candidate frames of the local image.

[0072] In this embodiment, the type of target detection model is not unique, and can be any one of a series of models such as SSD, YOLO, and Faster R-CNN. The specific target detection model type is determined by the actual usage scenario and is not limited here.

[0073] S200, obtaining a first evaluation score for each of the detection candidate frames based on color distribution, contrast, and texture features of each of the detection candidate frames, where the first evaluation score is used to evaluate the mirror image in the detection candidate frame;

[0074] In this embodiment, since the real object and the reflective object may be different in texture and color, specifically, the reflective object may have blurred edges, color distortion or brightness changes, therefore, by extracting the image content in each detection candidate frame and analyzing the features related to blurred edges, color distortion or brightness changes in the image content, it is possible to distinguish the image of the real object and the image of the reflective object, thereby screening out the detection candidate frames related to the mirror image.

[0075] Specifically, in this embodiment, by comprehensively evaluating the color distribution, contrast and texture features of each detection candidate frame, the evaluation process is made more comprehensive and accurate. Among them, the color distribution reflects the distribution and richness of the colors in the image, which is helpful for identifying the color distortion of the image in the detection candidate frame; the contrast describes the difference between the light and dark areas in the image, which is helpful for identifying the blurred edges of the detection candidate frame image; the texture features describe the texture of the image surface, which helps to distinguish the changes in the brightness of the image in the detection candidate frame.

[0076] In some other embodiments, after obtaining the first evaluation score of each detection candidate frame through the color distribution, contrast and texture features of each detection candidate frame, by comparing the numerical values ​​of the first evaluation scores corresponding to multiple detection candidate frames, the detection candidate frame corresponding to the first evaluation score with the largest numerical value is retained as the final candidate frame. At this time, the final candidate frame does not contain the mirror image, thereby screening out the detection candidate frame containing the mirror image.

[0077] In some embodiments, the step S200 includes:

[0078] Calculating a pixel average value of the image in each of the detection candidate frames, wherein the pixel average value is used to represent the color distribution;

[0079] Calculating the pixel variance of the image in each of the detection candidate frames, wherein the pixel variance is used to represent the contrast;

[0080] Performing a convolution operation on each image in the detection candidate frame to obtain a convolution average value of the image in the detection candidate frame, wherein the convolution average value is used to represent the texture feature;

[0081] The calculation formula of the first evaluation score is as follows:

[0082]

[0083] Among them, μ x represents the pixel average value, represents the pixel variance, μ y The convolution average.

[0084] In this embodiment, by calculating the pixel average value of the image in the detection candidate frame, it is used to reflect the tendency of the overall color in the image, that is, to obtain the color distribution of the image in the detection candidate frame. In addition, by calculating the pixel variance of the image in each detection candidate frame, the discrete degree of the pixel brightness or color distribution in the detection candidate frame is quantified. A higher pixel variance indicates that the pixel value difference in the detection candidate frame is large, that is, the contrast of the image area is high, and it contains rich edge information; on the contrary, when the pixel variance is low, it indicates that the image area is relatively flat, the contrast is low, and the edge of the detection candidate frame is blurred. The convolution average value not only reflects the complexity and regularity of the image texture in the detection candidate frame, but also can reveal whether there is a specific texture pattern or structure in the image to a certain extent. Therefore, by performing a convolution operation on the image in the detection candidate frame, it can be used to evaluate the texture features in the detection candidate frame.

[0085] In some embodiments, the calculation formula for calculating the pixel variance of each image in the detection candidate frame is as follows:

[0086]

[0087] Among them, x i represents the average value of all color channel pixel values ​​of each pixel in the detection candidate frame, μ x represents the pixel average value, and N represents the number of pixels in the detection candidate frame.

[0088] The calculation formula for the pixel average is as follows:

[0089] In some embodiments, performing a convolution operation on each image in the detection candidate frame to obtain a convolution average of the images in the detection candidate frame includes:

[0090] A 3×3 convolution kernel is used to perform a convolution operation on each color channel of the image in the detection candidate frame to obtain a first average value, where the first average value represents the average value of all color channel pixel values ​​of each pixel point in the detection candidate frame after the convolution operation.

[0091] The calculation formula of the first average value is as follows:

[0092] y i =s*x i

[0093] Among them, s represents the convolution kernel, x i is a second average value, which represents the average value of all color channel pixel values ​​of each pixel point in the detection candidate frame before the convolution operation;

[0094] Based on the first average value, the convolution average value is calculated, wherein the calculation formula of the convolution average value is as follows:

[0095]

[0096] Among them, y i represents the first average value, and M represents the number of pixels in the detection candidate box after the convolution operation.

[0097] In this embodiment, a 3×3 convolution kernel is used for convolution operation. Specifically, the 3×3 convolution kernel is applied to the red, green and blue channels of the image one by one. The features on each channel are extracted by sliding the convolution kernel and calculating the weighted sum of the kernel and the local area of ​​the image. In this process, each color channel will generate a convolved feature map, which reflects the result of the original image after convolution transformation in different color dimensions. In addition, in order to obtain a comprehensive texture feature representation, the average pixel value at each pixel position of all color channels after the convolution operation is calculated, that is, the first average value. The above first average value not only integrates the information of the image on multiple color channels, but also reflects the effect of the convolution operation on the extraction of image texture features. In a preferred embodiment, the form of the convolution kernel is: s= The size of the above convolution kernel is moderate, which can capture the local details in the image without causing a sharp increase in the amount of calculation due to being too large, and each channel carries the information of the image in a specific color dimension.

[0098] It can be understood that the x in the calculation formula of pixel mean and pixel variance is i The first average value is calculated by the formula i have the same meaning, both are second average values, specifically expressed as the average value of all color channel pixel values ​​of each pixel in the detection candidate frame before the convolution operation, that is, the first evaluation score is obtained by comprehensively evaluating the initial image in the detection candidate frame, thereby reflecting the situation of the mirror image in the detection candidate frame.

[0099] S300, obtaining a final evaluation score of each of the detection candidate boxes based on the color distribution, position features and first evaluation score of each of the detection candidate boxes;

[0100] In some embodiments, the step S300 includes:

[0101] For the color distribution, calculating the pixel average of the image in each detection candidate frame;

[0102] According to the position feature, the position coordinates of all vertices of the detection candidate box are obtained, and the coordinates of the center point of the detection candidate box are calculated;

[0103] For the first evaluation score, a second evaluation score is calculated by using an intersection-and-union method;

[0104] The calculation formula for the final evaluation score is as follows:

[0105]

[0106] Among them, μ x represents the pixel average value, x′ and y′ respectively represent the abscissa and ordinate of the center point coordinates, and s′ represents the second evaluation score.

[0107] In this embodiment, the second evaluation score is calculated by using the intersection-and-union method, and then the second evaluation score related to the mirror image and the color distribution and position characteristics related to the local image are comprehensively considered in the screening process of the target candidate frame to calculate the final evaluation score, and multiple detection candidate frames are screened based on the final evaluation score to obtain the target candidate frame. By determining the target candidate frame in the above manner, the influence of the mirror image and the local image on the screening of the target candidate frame is comprehensively considered, thereby reducing the possibility of mirror misdetection and local misdetection. At the same time, the image in the detection candidate frame can be more comprehensively evaluated to improve the accuracy of target detection. In the process of determining the final evaluation score, the second evaluation score obtained by the intersection-and-union calculation is used as one of the considerations of the final evaluation score for calculation, without the need to determine the optimal threshold, thereby reducing the number of screening times and improving the screening accuracy.

[0108] In this embodiment, the position coordinates of the four vertices of the output detection candidate box are obtained by the target detection model, and are marked as the upper left corner (x1, y1), the lower left corner (x1, y2), the upper right corner (x2, y1), and the lower right corner (x2, y2). The center point coordinates (x', y') of the detection candidate box are obtained by calculation, where

[0109] In some embodiments, the first evaluation score is calculated by using an intersection-over-union method to obtain a second evaluation score, including:

[0110] sorting the first evaluation scores corresponding to the plurality of detection candidate frames, and selecting the detection candidate frame corresponding to the maximum first evaluation score as the initial candidate frame;

[0111] Perform intersection and union operations on the remaining detection candidate frames and the initial candidate frame one by one, and calculate the intersection and union ratio between each detection candidate frame and the initial candidate frame. The calculation formula of the intersection and union ratio is as follows:

[0112]

[0113] Wherein, I represents the intersection-over-union ratio between the detection candidate box and the initial candidate box, A represents the initial candidate box, B represents the detection candidate box, |A∩B| is the intersection area of ​​the intersection area of ​​the detection candidate box and the initial candidate box, and |A∪B| is the union area of ​​the union area of ​​the detection candidate box and the initial candidate box;

[0114] The calculation formula of the second evaluation score is as follows:

[0115]

[0116] Among them, s1 represents the first evaluation score, exp represents the power function with the natural constant e as the base, and δ 2 is a hyperparameter.

[0117] In this embodiment, an initial candidate box is selected from multiple detection candidate boxes obtained by the target detection model using a first evaluation score, and an intersection-over-union ratio between the initial candidate box and the remaining detection candidate boxes is calculated. A second evaluation score is calculated based on the intersection-over-union calculation result and the first evaluation score to improve the detection accuracy of the mirror image of the detection candidate box, thereby improving the accuracy of target detection.

[0118] In this embodiment, the second evaluation score δ 2 It is a hyperparameter, which is set by the user and dynamically adjusted by the user according to the output results of the target detection model, thereby improving the adaptability and flexibility of the target detection model and enhancing the user experience and satisfaction.

[0119] In some embodiments, calculating the intersection-over-union ratio between each of the detected candidate boxes and the initial candidate box includes:

[0120] Based on the upper left corner coordinates, width and height of the detection candidate box and the initial candidate box, obtain the upper left corner coordinates and the lower right corner coordinates of the intersection area;

[0121] Based on the upper left corner coordinates and the lower right corner coordinates, calculate the intersection width and the intersection height of the intersection area;

[0122] Calculating the intersection area based on the intersection width and the intersection height;

[0123] The area of ​​the detection candidate box and the initial candidate box are calculated respectively, and the union area is obtained based on the two area areas and the intersection area.

[0124] In this embodiment, the upper left corner coordinates, width and height of the initial candidate box and the remaining detection candidate boxes are obtained through the target detection model, and the upper left corner coordinates of the initial candidate box are marked as (x A ,yA ), whose width and height are marked as w A and h A , mark the upper left corner coordinates of the remaining detection candidate boxes as (x B ,y B ), whose width and height are marked as w B and h B , at this time, the coordinates of the upper left corner of the intersection area are The coordinates of the lower right corner of the intersection area are min(y A +h A ,y B +h B )), based on the coordinates of the upper left corner and the upper right corner of the intersection area, the calculation formula for the intersection width of the intersection area is: The calculation formula for the intersection height is: In summary, the intersection area of ​​the intersection area is: |A∩B|=w inter ×h inter .

[0125] In this embodiment, the union area is the sum of the area of ​​the detection candidate box and the initial candidate box minus the intersection area. Specifically, the calculation formula of the union area is as follows: |A∪B|=w A ×h A +w B ×h B -|A∩B|.

[0126] S400 , sorting the final evaluation scores corresponding to the multiple detection candidate boxes, and selecting the detection candidate box corresponding to the maximum final evaluation score as the target candidate box.

[0127] In this embodiment, by performing the above steps S100-S300, the image in each detection candidate frame is evaluated to obtain a final evaluation score, and multiple detection candidate frames are sorted from large to small according to the final evaluation scores, so as to select the detection candidate frame with the largest final evaluation score as the target candidate frame. The selection mechanism based on score sorting effectively avoids false detection and missed detection, thereby improving the accuracy of target detection.

[0128] In the process of screening the target candidate frames in this embodiment, the influence of the mirror image and the local image on the screening of the target candidate frames is comprehensively considered, and on this basis, the final evaluation score of each detection candidate frame is calculated, and each detection candidate frame is evaluated according to the size of the final evaluation score, thereby improving the accuracy of the screening. At the same time, there is no need to determine the optimal threshold in the process of determining the final evaluation score, thereby reducing the number of screening times, thereby effectively reducing the amount of calculation in the process of screening the target candidate frames, and at the same time improving the screening accuracy.

[0129] See also Figure 2 As shown, the present invention also provides a target detection candidate box screening system, the system comprising:

[0130] The first processing module 201 is used to detect the target image through a pre-trained target detection model to obtain multiple detection candidate frames;

[0131] The second processing module 202 is used to obtain a first evaluation score for each of the detection candidate frames based on the color distribution, contrast and texture features of each of the detection candidate frames, wherein the first evaluation score is used to evaluate the mirror image in the detection candidate frame;

[0132] The third processing module 203 is used to obtain a final evaluation score of each detection candidate box based on the color distribution, position feature and first evaluation score of each detection candidate box;

[0133] The fourth processing module 204 is used to sort the final evaluation scores corresponding to the multiple detection candidate frames, and select the detection candidate frame corresponding to the maximum final evaluation score as the target candidate frame.

[0134] It is understandable that if Figure 1 The contents of the target detection candidate frame screening method embodiment shown in the figure are all applicable to the target detection candidate frame screening system embodiment. The functions specifically implemented by the target detection candidate frame screening system embodiment are similar to those in the above embodiment. Figure 1 The target detection candidate box screening method embodiment shown in FIG. 1 is the same as that shown in FIG. 1 , and the beneficial effects achieved are the same as those of FIG. Figure 1 The beneficial effects achieved by the target detection candidate box screening method embodiment shown are also the same.

[0135] It should be noted that the information interaction, execution process and other contents between the above-mentioned systems are based on the same concept as the embodiment of the method of the present invention. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0136] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0137] See also Figure 3 As shown, an embodiment of the present invention further provides a computer device 3, comprising: a memory 302 and a processor 301 and a computer program 303 stored on the memory 302. When the computer program 303 is executed on the processor 301, a target detection candidate box screening method as described in any one of the above methods is implemented.

[0138] The computer device 3 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The computer device 3 may include, but is not limited to, a processor 301 and a memory 302. Those skilled in the art will appreciate that Figure 3 It is only an example of computer device 3 and does not constitute a limitation on computer device 3. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components, for example, it may also include input and output devices, network access devices, etc.

[0139] The processor 301 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0140] In some embodiments, the memory 302 may be an internal storage unit of the computer device 3, such as a hard disk or memory of the computer device 3. In other embodiments, the memory 302 may also be an external storage device of the computer device 3, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device 3. Further, the memory 302 may also include both an internal storage unit of the computer device 3 and an external storage device. The memory 302 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program, etc. The memory 302 may also be used to temporarily store data that has been output or is to be output.

[0141] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for screening candidate boxes for object detection as described in any one of the above methods is implemented.

[0142] In this embodiment, if the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0143] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for screening candidate boxes for target detection, characterized in that: include: Detect the target image through the pre-trained target detection model to obtain multiple detection candidate boxes; Obtaining a first evaluation score for each of the detection candidate frames based on color distribution, contrast, and texture features of each of the detection candidate frames, wherein the first evaluation score is used to evaluate the mirror image in the detection candidate frame; Obtaining a final evaluation score for each of the detection candidate boxes based on the color distribution, position features, and first evaluation score of each of the detection candidate boxes; The final evaluation scores corresponding to the multiple detection candidate boxes are sorted, and the detection candidate box corresponding to the maximum final evaluation score is selected as the target candidate box.

2. The method according to claim 1, characterized in that The obtaining, based on the color distribution, contrast and texture features of each of the detection candidate frames, a first evaluation score of each of the detection candidate frames comprises: Calculating a pixel average value of the image in each of the detection candidate frames, wherein the pixel average value is used to represent the color distribution; Calculating the pixel variance of the image in each of the detection candidate frames, wherein the pixel variance is used to represent the contrast; Performing a convolution operation on each image in the detection candidate frame to obtain a convolution average value of the image in the detection candidate frame, wherein the convolution average value is used to represent the texture feature; The calculation formula of the first evaluation score is as follows: Among them, μ x represents the pixel average value, represents the pixel variance, μ y The convolution average.

3. The method according to claim 2, characterized in that The calculation formula for calculating the pixel variance of each image in the detection candidate frame is as follows: Among them, x i represents the average value of all color channel pixel values ​​of each pixel in the detection candidate frame, μ x represents the pixel average value, and N represents the number of pixels in the detection candidate frame.

4. The method according to claim 2, characterized in that The performing a convolution operation on each image in the detection candidate frame to obtain a convolution average value of the image in the detection candidate frame includes: A 3×3 convolution kernel is used to perform a convolution operation on each color channel of the image in the detection candidate frame to obtain a first average value, where the first average value represents the average value of all color channel pixel values ​​of each pixel point in the detection candidate frame after the convolution operation. The calculation formula of the first average value is as follows: y i =s*x i Among them, s represents the convolution kernel, x i is a second average value, which represents the average value of all color channel pixel values ​​of each pixel point in the detection candidate frame before the convolution operation; Based on the first average value, the convolution average value is calculated, wherein the calculation formula of the convolution average value is as follows: Among them, y i represents the first average value, and M represents the number of pixels in the detection candidate box after the convolution operation.

5. The method according to claim 2, characterized in that The obtaining, based on the color distribution, position feature and first evaluation score of each detection candidate frame, a final evaluation score of each detection candidate frame comprises: For the color distribution, calculating the pixel average of the image in each detection candidate frame; According to the position feature, the position coordinates of all vertices of the detection candidate box are obtained, and the coordinates of the center point of the detection candidate box are calculated; For the first evaluation score, a second evaluation score is calculated by using an intersection-and-union method; The calculation formula for the final evaluation score is as follows: Among them, μ x represents the pixel average value, x′ and y′ respectively represent the abscissa and ordinate of the center point coordinates, and s′ represents the second evaluation score.

6. The method according to claim 5, characterized in that The step of calculating the second evaluation score by using an intersection-and-union method for the first evaluation score includes: sorting the first evaluation scores corresponding to the plurality of detection candidate frames, and selecting the detection candidate frame corresponding to the maximum first evaluation score as the initial candidate frame; Perform intersection and union operations on the remaining detection candidate frames and the initial candidate frame one by one, and calculate the intersection and union ratio between each detection candidate frame and the initial candidate frame. The calculation formula of the intersection and union ratio is as follows: Wherein, I represents the intersection-over-union ratio between the detection candidate box and the initial candidate box, A represents the initial candidate box, B represents the detection candidate box, |A∩B| is the intersection area of ​​the intersection area of ​​the detection candidate box and the initial candidate box, and |A∪B| is the union area of ​​the union area of ​​the detection candidate box and the initial candidate box; The calculation formula of the second evaluation score is as follows: Among them, s1 represents the first evaluation score, exp represents the power function with the natural constant e as the base, and δ 2 is a hyperparameter.

7. The method according to claim 6, characterized in that The calculating the intersection-over-union ratio between each of the detection candidate boxes and the initial candidate box includes: Based on the upper left corner coordinates, width and height of the detection candidate box and the initial candidate box, obtain the upper left corner coordinates and the lower right corner coordinates of the intersection area; Based on the upper left corner coordinates and the lower right corner coordinates, calculate the intersection width and the intersection height of the intersection area; Calculating the intersection area based on the intersection width and the intersection height; The area of ​​the detection candidate box and the initial candidate box are calculated respectively, and the union area is obtained based on the two area areas and the intersection area.

8. A target detection candidate box screening system, characterized in that: include: The first processing module is used to detect the target image through the pre-trained target detection model to obtain multiple detection candidate boxes; A second processing module: used for obtaining a first evaluation score for each of the detection candidate frames based on the color distribution, contrast and texture features of each of the detection candidate frames, wherein the first evaluation score is used for evaluating the mirror image in the detection candidate frame; A third processing module: used for obtaining a final evaluation score of each detection candidate frame based on the color distribution, position feature and first evaluation score of each detection candidate frame; The fourth processing module is used to sort the final evaluation scores corresponding to the multiple detection candidate frames, and select the detection candidate frame corresponding to the maximum final evaluation score as the target candidate frame.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images