Mask-based image target candidate point screening method and system

Abstract

The application provides a mask-based image target candidate point screening method, a target candidate region graph is obtained according to the probability of a probability graph, a convolution operation is performed on the target candidate region graph, a local probability feature graph is obtained, a maximum probability point is found in the local probability feature graph, a rectangular mask is constructed with the maximum probability point as the center, part of the local probability feature graph is shielded by using the rectangular mask, and the position information of the maximum probability point is stored, whether the points in the local probability feature graph have all been shielded is judged, if yes, the screening is stopped, and the current maximum probability point is taken as a target candidate point; if no, the maximum probability point is found again in the remaining local probability feature graph, and the process is repeated until the points in the local probability feature graph are all shielded, all the maximum probability points are obtained, and the maximum probability points are taken as target candidate points; the application screens the positions of all the target candidate points meeting the conditions in the probability graph in real time, and greatly improves the target detection efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a mask-based image target candidate point screening method and system. BACKGROUND

[0002] The statements in this section merely provide background technology related to the present application and do not necessarily constitute prior art.

[0003] With the vigorous development of image processing technology and target detection algorithm, algorithms for image specific target detection are successively published, including template matching and Hough transform voting algorithm based on image processing, target prediction based on convolutional neural network, etc.; the target prediction algorithm based on neural network usually combines the Non-Maximum Suppression (NMS) algorithm to obtain all target positions in the image because the prediction result obtained is a series of target rectangular frames.

[0004] The inventor finds that the existing image processing-based target detection method usually obtains a target position probability map as the result, and the traditional non-maximum suppression algorithm is difficult to apply in the probability map or voting map, and some methods of converting the probability map into a rectangular frame and then performing NMS will consume a large amount of computing resources, which is not conducive to the real-time operation of image processing-based target detection. SUMMARY

[0005] In order to solve the problems of the prior art, the present application provides a mask-based image target candidate point screening method and system, which can screen out all target candidate points in the probability map in real time, greatly improving the target detection efficiency.

[0006] In order to achieve the above purpose, the present application adopts the following technical scheme:

[0007] The first aspect of the present application provides a mask-based image target candidate point screening method.

[0008] A mask-based image target candidate point screening method, comprising the following processes:

[0009] Obtaining a probability map of a to-be-processed image;

[0010] Obtaining a target candidate region map according to the probability of the probability map;

[0011] Performing convolution operation on the target candidate region map to obtain a local probability feature map;

[0012] Finding the maximum probability point in the local probability feature map;

[0013] A rectangular mask is constructed with the maximum point as the center, the rectangular mask is used to shield part of the local probability feature map, and the position information of the maximum point is stored;

[0014] It is determined whether the points in the local probability feature map have been all shielded, if yes, the screening is stopped, and the current maximum point is taken as the target candidate point; if no, the maximum point is searched again in the remaining local probability feature map until the points in the local probability feature map are all shielded, and all the maximum points are obtained, and all the maximum points are taken as the target candidate points.

[0015] As an optional implementation manner of the first aspect of the present application, if the obtained is a voting map of the image to be processed, the voting map is normalized to a probability map in the range of 0 to 1.

[0016] As an optional implementation manner of the first aspect of the present application, the target candidate region map is obtained according to the probability of the probability map, comprising:

[0017] According to the probability of the probability map, the part with a probability lower than a set threshold is filtered out to obtain the target candidate region map.

[0018] As an optional implementation manner of the first aspect of the present application, the target candidate region map is subjected to a convolution operation to obtain the local probability feature map, comprising:

[0019] The target candidate region map is subjected to a convolution operation to suppress noise and extract the local probability feature of the region, and the local probability feature map is obtained.

[0020] As a further limitation of the first aspect of the present application, the width and height of the convolution kernel of the convolution operation are both set to Wherein, k is a constant coefficient, and A is the average value of the areas of all candidate regions.

[0021] As an optional implementation manner of the first aspect of the present application, the maximum probability point is searched in the local probability feature map, comprising:

[0022] The maximum point in the local probability feature map is searched according to the minMaxLoc() function of OpenCV.

[0023] As an optional implementation manner of the first aspect of the present application, the width of the rectangular mask is aW, and the height is aH, wherein W is the width of the target to be detected, H is the height of the target to be detected, and a is a constant coefficient.

[0024] The second aspect of the present application provides a mask-based image target candidate point screening system.

[0025] A mask-based image target candidate point screening system, characterized in that, comprising:

[0026] An image acquisition module is configured to acquire a probability map of an image to be processed.

[0027] A target candidate region map generation module is configured to obtain a target candidate region map according to probabilities of the probability map.

[0028] A local probability feature map generation module is configured to perform a convolution operation on the target candidate region map to obtain a local probability feature map.

[0029] A probability maximum point searching module is configured to search for a probability maximum point in the local probability feature map.

[0030] A mask shielding module is configured to construct a rectangular mask centered on the maximum point, shield part of the local probability feature map using the rectangular mask, and store position information of the maximum point.

[0031] A target candidate point generation module is configured to determine whether all points in the local probability feature map have been shielded, if yes, stop screening, and take the current maximum point as a target candidate point, if no, search for a maximum point in the remaining local probability feature map again until all points in the local probability feature map are shielded, and take all maximum points as target candidate points.

[0032] The third aspect of the present application provides a computer readable storage medium having a program stored thereon, the program being executed by a processor to implement the steps of the mask-based image target candidate point screening method according to the first aspect of the present application.

[0033] The fourth aspect of the present application provides an electronic device comprising a memory, a processor, and a program stored on the memory and executable on the processor, wherein the processor implements the steps of the mask-based image target candidate point screening method according to the first aspect of the present application when executing the program.

[0034] Compared with the prior art, the present application has the following advantages:

[0035] 1. The present application innovatively proposes a mask-based image target candidate point screening method, which filters out small probability parts in the probability map using a preset threshold, greatly reduces the subsequent calculation amount, and effectively reduces the operation time.

[0036] 2. The present application innovatively proposes a mask-based image target candidate point screening method, which uses a convolution kernel of adaptive size to perform convolution on the candidate region map to obtain a local probability feature map of the candidate region, that is, to statistically sum the probabilities of the convolution region, which has a certain correction effect compared with directly taking the maximum value in the probability map.

[0037] 3. The application innovatively provides a mask-based image target candidate point screening method, which removes the confirmed candidate points through a mask operation in a loop, so that repeated detection can be effectively avoided.

[0038] 4. The application innovatively provides a mask-based image target candidate point screening method, which can avoid the huge calculation amount of calculating the intersection-over-union ratio and also achieve better intersection-over-union ratio limitation compared with the non-maximum suppression algorithm.

[0039] 5. The target detection algorithm based on image processing provides a general target candidate point screening method, which is especially suitable for a detection algorithm for generating a target prediction probability graph, and can quickly and accurately determine the pixel coordinates of all candidate targets in the probability graph.

[0040] The advantages of the additional aspects of the application will be partially given in the following description, partially become obvious from the following description, or be learned through the practice of the application. BRIEF DESCRIPTION OF DRAWINGS

[0041] The drawings constituting a part of the specification of the application are used to provide further understanding of the application, the illustrative embodiments of the application and the description thereof are used to explain the application, and do not constitute improper limitation on the application.

[0042] Figure 1 A mask-based image target candidate point screening method flowchart provided for the embodiment 1 of the application;

[0043] Figure 2 A probability graph obtained when a certain image processing algorithm detects a target provided for the embodiment 1 of the application;

[0044] Figure 3 A target candidate region graph after threshold filtering provided for the embodiment 1 of the application;

[0045] Figure 4 A local probability feature graph after convolution operation provided for the embodiment 1 of the application;

[0046] Figure 5 A mask position graph when a mask operation is performed provided for the embodiment 1 of the application;

[0047] Figure 6 A mark of the screening result in the image provided for the embodiment 1 of the application. DETAILED DESCRIPTION

[0048] The application will be further described below in combination with the drawings and embodiments.

[0049] It should be noted that the following detailed description is exemplary in nature and is intended to provide further description of the application. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0050] It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments in accordance with the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, steps, operations, elements, components, and / or groups thereof, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof.

[0051] In the case of no conflict, the embodiments in the application and the features in the embodiments can be combined with each other.

[0052] Embodiment 1

[0053] As Figure 1 , Figure 2 , Figure 3 , Figure 4 , Figure 5 and Figure 6 indicate, the embodiment 1 of the present application provides a mask-based image target candidate point screening method based on ubuntu18.04 system as a running carrier, adopts a self-defined convolution operation to extract local probability statistical features of a probability graph, applies a self-defined threshold to filter points with low probability, and then applies a self-defined mask to shield the candidate points that have been detected in the probability graph to avoid repeated detection. The method can replace the application of non-maximum suppression in the target detection method based on image processing, has the same detection effect as NMS, but the principle is simpler, the code implementation is more concise and flexible, and therefore has a faster running speed. In the actual application process, all the positions of the target candidate points in the probability graph that meet the conditions can be screened out in real time. Specifically, the method includes the following processes:

[0054] S1: The input image is a probability graph generated by an image processing algorithm. If it is a voting graph, it is normalized to a probability graph of 0-1.

[0055] S2: The threshold thr is adaptively set according to the probability of the probability graph, and the parts with low probability are filtered out to obtain a target candidate region graph.

[0056] In this embodiment, the size of the adaptive threshold thr can be adjusted. Generally, it is set to 0.25. When thr is large, the area of the obtained candidate region is small, which can reduce the calculation amount of the subsequent convolution operation and speed up the running speed, but the real target with low prediction probability may be missed.

[0057] S3: performing convolution operation on the target candidate region map, suppressing noise and extracting local probability features of the region to obtain a local probability feature map;

[0058] First, the contour features of all candidate regions are extracted to calculate the areas of all candidate regions, the average A of the areas of candidate regions is calculated, and the size of the convolution kernel is adaptively set according to A, the width and height of the convolution kernel are both set to k is generally 1.2;

[0059] In this embodiment, the parameters of the convolution kernel can be adjusted, and the default convolution kernel parameters are all 1; when summing the convolution region, the summation range can be adjusted, for example, only the circular region with a radius of is summed

[0060] S4: quickly finding the maximum value and its position in the entire local probability feature map according to the minMaxLoc() function of OpenCV.

[0061] S5: creating a rectangular mask with the maximum value point as the center, the default width being aW and the default height being aH, wherein W and H are the width and height of the target to be detected, and using the mask to shield part of the local probability feature map;

[0062] In this embodiment, a = 1.0, which is equivalent to limiting the intersection over union (IOU) to 0.334, and the scale size a of the mask can be selected, wherein different scales of the mask can achieve different size limitations of the intersection over union (IOU) when processing.

[0063] S6: storing the coordinates of the maximum value point.

[0064] S7: judging whether the points in the current local probability feature map have been completely shielded, if yes, stopping the screening and entering S8, if not, entering S4 to continue finding the maximum value and its coordinates in the remaining local probability feature map, and repeating the steps S4, S5 and S6 n times until all the points in the local probability feature map are shielded.

[0065] In this embodiment, the number of candidate points that can be found in the image can be limited by adjusting the number of cycles n, and the calculation scale is also limited.

[0066] S8: taking the coordinates of all maximum value points as the coordinates of the target candidate points.

[0067] Embodiment 2

[0068] Embodiment 2 of the present application provides an image target candidate point screening system based on a mask, comprising:

[0069] An image acquisition module configured to acquire a probability map of an image to be processed;

[0070] The target candidate region graph generation module is configured to obtain a target candidate region graph according to the probabilities of the probability graph.

[0071] The local probability feature map generation module is configured to perform convolution operation on the target candidate region graph to obtain a local probability feature map.

[0072] The probability maximum point searching module is configured to search for a probability maximum point in the local probability feature map.

[0073] The mask shielding module is configured to construct a rectangular mask centered on the maximum point, shield part of the local probability feature map using the rectangular mask, and store position information of the maximum point.

[0074] The target candidate point generation module is configured to determine whether all points in the local probability feature map have been shielded, if yes, stop screening, and take the current maximum point as a target candidate point; if no, search for a maximum point in the remaining local probability feature map again until all points in the local probability feature map are shielded, and obtain all maximum points as target candidate points.

[0075] Specifically,

[0076] In the image acquisition module, if a voting graph of the image to be processed is acquired, the voting stickers are normalized into a probability graph in the range of 0-1.

[0077] In the target candidate region graph generation module, the target candidate region graph is obtained according to the probabilities of the probability graph, including:

[0078] According to the probabilities of the probability graph, part with a probability lower than a set threshold is filtered out to obtain the target candidate region graph.

[0079] In the local probability feature map generation module, the target candidate region graph is subjected to convolution operation to obtain a local probability feature map, including:

[0080] The target candidate region graph is subjected to convolution operation to suppress noise and extract local probability features of the region, thereby obtaining the local probability feature map, and the width and height of the convolution kernel of the convolution operation are both set to wherein k is a constant coefficient, and A is an average value of areas of all candidate regions.

[0081] In the probability maximum point searching module, the probability maximum point is searched in the local probability feature map, including:

[0082] The maximum point in the local probability feature map is searched according to the minMaxLoc() function of OpenCV.

[0083] In the mask shielding module, the width of the rectangular mask is aW and the height is aH, wherein W is the width of the target to be detected, H is the height of the target to be detected, and a is a constant coefficient.

[0084] Embodiment 3

[0085] Embodiment 3 of the present application provides a computer readable storage medium having a program stored thereon, the program being executed by a processor to implement the following steps:

[0086] A1: the input image is a probability map generated by an image processing algorithm, and if it is a voting map, it is normalized to a probability map of 0-1.

[0087] A2: adaptively set a threshold thr according to the probability of the probability map, and filter out the part with low probability to obtain a target candidate region map;

[0088] In this embodiment, the size of the adaptive threshold thr can be adjusted, which is generally set to 0.25. When thr is large, the area of the obtained candidate region is small, which can reduce the calculation amount of the subsequent convolution operation and speed up the running speed, but the real target with low prediction probability may be missed.

[0089] A3: perform convolution operation on the target candidate region map to suppress noise and extract local probability features of the region to obtain a local probability feature map;

[0090] First, the contour features of all candidate regions are extracted to calculate the areas of all candidate regions, and the average value A of the candidate region area is calculated, and the size of the convolution kernel is adaptively set according to A, and the width and height of the convolution kernel are both set to k is generally 1.2.

[0091] A4: quickly find the maximum value and its position in the whole local probability feature map according to the minMaxLoc() function of OpenCV.

[0092] A5: create a rectangular mask with the maximum value point as the center, the default width as aW and the height as aH, wherein W and H are the width and height of the target to be detected, and use the mask to shield part of the local probability feature map;

[0093] In this embodiment, a=1.0, which is equivalent to limiting the intersection over union (IOU) to 0.334.

[0094] A6: store the coordinates of the maximum value point.

[0095] A7: determine whether the points in the current local probability feature map have been completely shielded, if yes, stop screening and enter A8, if not, enter A4 to continue finding the maximum value and its coordinates in the remaining local probability feature map, and repeat steps A4, A5 and A6 n times until all points in the local probability feature map are shielded.

[0096] A8: the coordinates of the maximum point are taken as the coordinates of the target candidate point.

[0097] Embodiment 4

[0098] Embodiment 4 of the present application provides an electronic device, comprising a memory, a processor, and a program stored in the memory and executable on the processor, and the processor performs the following steps:

[0099] B1: the input image is a probability map generated by an image processing algorithm, and if it is a voting map, it is normalized to a probability map of 0-1.

[0100] B2: adaptively set a threshold thr according to the probability of the probability map, and filter out the part with low probability to obtain a target candidate region map;

[0101] In this embodiment, the size of the adaptive threshold thr can be adjusted, and is generally set to 0.25. When thr is large, the area of the obtained candidate region is small, which can reduce the calculation amount of the subsequent convolution operation and speed up the running speed, but the real target with low prediction probability may be missed.

[0102] B3: perform convolution operation on the target candidate region map to suppress noise and extract local probability features of the region to obtain a local probability feature map;

[0103] First, the contour features of all candidate regions are extracted to calculate the areas of all candidate regions, and the average value A of the candidate region area is calculated, and the size of the convolution kernel is adaptively set according to A, and the width and height of the convolution kernel are both set to k is generally 1.2.

[0104] B4: quickly find the maximum value and its position in the whole local probability feature map according to the minMaxLoc() function of OpenCV.

[0105] B5: create a rectangular mask centered on the maximum point with a default width of αW and a default height of αH, wherein W and H are the width and height of the target to be detected, and use the mask to shield part of the local probability feature map;

[0106] In this embodiment, α = 1.0, which is equivalent to limiting the intersection over union (IOU) to 0.334.

[0107] B6: store the coordinates of the maximum point.

[0108] B7: judging whether the points in the current local probability feature map have all been shielded, if yes, stopping screening, entering B8, if no, entering B4, continuing to find the maximum value and its coordinates in the remaining local probability feature map, and repeating the steps of B4, B5 and B6 for n times until the points in the local probability feature map are all shielded;

[0109] B8: taking the coordinates of all the maximum value points as the coordinates of the target candidate points.

[0110] Those skilled in the art will understand that embodiments of the present application can be provided as methods, systems, or computer program products. Accordingly, the present application can take the form of an implementation to hardware only, software only, or an implementation combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk memory and optical memory, etc.) containing computer-usable program code.

[0111] The present application is described in reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and combinations of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, a special purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions, which are executed via the processor of the computer or other programmable data processing apparatus, generate an apparatus that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 means for performing the function specified by one or more blocks

[0112] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including instruction means, which implement the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 means for performing the function specified by one or more blocks

[0113] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are performed on the computer or other programmable data processing apparatus to produce a computer implemented process, so that the instructions executed on the computer or other programmable data processing apparatus provide a process for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 means for performing the function specified by one or more blocks

[0114] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing relevant hardware, and the program can be stored in a computer readable storage medium. When the program is executed, the program can include the processes of the above-mentioned embodiment methods. The storage medium can be a magnetic disc, an optical disc, a read-only memory (ROM), a random access memory (RAM) or the like.

[0115] The above only describes the preferred embodiments of the present application and is not used to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement and the like within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A mask-based method for selecting candidate points for image targets, characterized in that, Includes the following processes: Obtain the probability map of the image to be processed; The target candidate region map is obtained based on the probability of the probability map; Perform a convolution operation on the target candidate region map to obtain a local probability feature map; Find the point with the maximum probability in the local probability feature map; Construct a rectangular mask centered on the maximum point, use the rectangular mask to shield a part of the local probability feature map, and store the location information of the maximum point; Determine whether all points in the local probability feature map have been masked. If so, stop filtering and use the current maximum point as the target candidate point. If not, search for the maximum point again in the remaining local probability feature map until all points in the local probability feature map are masked, obtain all the maximum points, and use all the maximum points as the target candidate points.

2. The mask-based image target candidate point selection method as described in claim 1, characterized in that, If the obtained image is a voting map of the image to be processed, the voting map is normalized to a probability map in the range of 0 to 1.

3. The mask-based image target candidate point selection method as described in claim 1, characterized in that, The target candidate region map is obtained based on the probability of the probability map, including: Based on the probabilities of the probability map, the portion with a probability lower than a set threshold is filtered out to obtain the target candidate region map.

4. The mask-based image target candidate point selection method as described in claim 1, characterized in that, Perform a convolution operation on the target candidate region map to obtain a local probability feature map, including: A convolution operation is performed on the target candidate region map to suppress noise and extract local probability features of the region, resulting in a local probability feature map.

5. The mask-based image target candidate point selection method as described in claim 4, characterized in that, The width and height of the convolution kernel in the convolution operation are both set to... Where k is a constant coefficient, and A is the average area of ​​all candidate regions.

6. The mask-based image target candidate point selection method as described in claim 1, characterized in that, Find the point with the maximum probability in the local probability feature map, including: Find the maximum value in the local probability feature map using OpenCV's minMaxLoc() function.

7. The mask-based image target candidate point selection method as described in claim 1, characterized in that, The rectangular mask has a width of αW and a height of αH, where W is the width of the target to be detected, H is the height of the target to be detected, and α is a constant coefficient.

8. A mask-based image target candidate point selection system, characterized in that, include: The image acquisition module is configured to acquire a probability map of the image to be processed. The target candidate region map generation module is configured to: obtain the target candidate region map based on the probability of the probability map; The local probability feature map generation module is configured to perform a convolution operation on the target candidate region map to obtain a local probability feature map. The probability maximum point search module is configured to: search for the probability maximum point in the local probability feature map; The masking module is configured to: construct a rectangular mask centered on the maximum point, use the rectangular mask to shield a portion of the local probability feature map, and store the location information of the maximum point; The target candidate point generation module is configured to: determine whether all points in the local probability feature map have been masked; if so, stop filtering and use the current maximum point as the target candidate point; if not, search for the maximum point again in the remaining local probability feature map until all points in the local probability feature map are masked, obtain all the maximum points, and use all the maximum points as the target candidate points.

9. A computer-readable storage medium having a program stored thereon, characterized in that, When executed by a processor, the program implements the steps in the mask-based image target candidate point selection method as described in any one of claims 1-7.

10. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the mask-based image target candidate point screening method as described in any one of claims 1-7.

Images