Real-time target detection method suitable for unconventional target detection

Abstract

The invention belongs to the technical field of automatic target detection, and relates to a real-time target detection method suitable for unconventional target detection, which comprises the following steps of: acquiring a frame of scene image from a detector, and selecting an interested target for tracking; training a target feature model; establishing a target feature library, and storing data of the target feature library into a memory; acquiring a frame of scene image from a detector, and executing target detection; establishing a binary image; performing region growth, expansion and clustering calculation on the image to determine a suspected target; and comparing the suspected target with a target feature library to determine a target detection result. According to the method, when the target detection task is executed, detection of any interested target category in a scene can be increased in time, real-time operation in embedded hardware can be achieved, rapid response and real-time processing of the detection task are achieved, and an accurate target detection result is provided.

Description

Technical Field

[0001] The invention belongs to the technical field of automatic target detection and relates to a real-time target detection method suitable for detecting unconventional targets. Background Art

[0002] Automatic target detection technology can automatically prompt observers to detect the location and category of potential targets in the scene. In the field of automatic target detection technology, there are two main directions: target detection based on image pixel feature extraction and target detection based on deep learning.

[0003] Detection technologies based on pixel feature extraction include target detection technologies based on Harris corner detection and edge extraction. The detection results of this type of technology are usually some pixel value mutation positions in the image, such as a bright or dark spot on the ground, the edge of an object in the scene, etc. Such detection results lack practical significance in scene detection tasks. The target detection method based on deep learning can have good adaptability to changes in the target's morphology, scale, and illumination, but this technology requires the collection and annotation of a large number of training samples and a long period of model training. It requires high-performance large-scale computing hardware support, and the computational complexity is large. The real-time performance running on embedded hardware is not ideal. When performing detection tasks, it is difficult to increase the detection target type in time according to on-site needs, especially for some unconventional targets such as special-shaped buildings and bunkers. It is difficult to respond quickly to the detection needs.

[0004] In summary, the problems existing in the existing technology are: the construction of data sets and model training are time-consuming, labor-intensive and costly; the detection target types cannot be increased in time according to the detection site conditions, and only specific target category detection based on the training model can be completed; real-time performance is difficult to guarantee. Summary of the invention

[0005] (I) Purpose of the invention

[0006] The purpose of the present invention is to provide a real-time target detection method suitable for unconventional target detection, which can timely increase the detection of any target category of interest in the scene when performing target detection tasks, and can run in real time in embedded hardware to achieve rapid response and real-time processing of detection tasks, and provide accurate target detection results.

[0007] (II) Technical solution

[0008] In order to solve the above technical problems, the present invention provides a real-time target detection method suitable for unconventional target detection, which comprises the following steps:

[0009] Step 1: Get a frame of scene image from the detector and select an object of interest for tracking.

[0010] Step 2: Train the target feature model: Keep tracking the target continuously, iteratively train the target feature model according to the image information of the tracked target area, and obtain the model posterior probability p and the target area image set image{}.

[0011] Step 3: Establish a target feature library and store the target feature library data in the hardware platform memory. Repeat steps 1 to 2 to save the target feature model posterior probability and target area image information to complete the establishment of the target feature library, which is recorded as:

[0012]

[0013] where A1, A2, …, A n represents n targets of interest, Indicates target A n A collection of images of all tracking regions during the tracking process.

[0014] Step 4: After completing the training and storage of the target feature library, start target detection. Obtain a frame of scene image from the detector and perform downsampling to obtain the downsampled image I. Starting from the coordinate point (0, 0) of image I, slide the window pixel by pixel from left to right and from top to bottom to traverse I, calculate the feature value of each window image, and calculate the cumulative value of the posterior probability of each window image feature value under each type of target based on the model posterior probability of the target feature library. Filter out the windows whose posterior probability is greater than the set threshold (the threshold is preferably 0.6 based on experience), record the upper left corner coordinates P(x, y) of the window and the target type id, id = 1, 2, ... n, corresponding to target A1, target A2, ... target A n .

[0015] Step 5: Create a binary image I Bin : Set the image pixel corresponding to the coordinate point position recorded in step 4 to (255-id), and set the values ​​of the remaining pixels to 0.

[0016] Step 6: Image I Bin Perform regional growth, expansion and clustering calculations to obtain multiple regional clusters, extract the outlines of the clusters, delete the clusters with small areas and irregular shapes, mark the remaining clusters as suspected targets, and record the center position coordinates and type ID of the suspected targets.

[0017] Step 7: Use the suspected target position recorded in step 6 as the upper left corner coordinate of the window, select a region image on the current scene image, and compare the selected region image with the region image set of the corresponding type target in the feature library according to the suspected target type id. Perform cross-correlation calculation. The image and position of the selected area whose cross-correlation coefficient is greater than the set threshold are the target detection results. To improve the detection accuracy, the threshold is preferably 0.9.

[0018] The specific steps of training the feature model in step 2 above are:

[0019] Step 2-1: According to the size of the selected target window, establish a set M{} of window areas with different position coordinates and sizes through proportional scaling, affine transformation, etc., calculate the overlap between each window in M{} and the tracking target window, and divide the set M{} into two subsets of positive and negative samples according to the overlap value.

[0020] Step 2-2: Calculate the eigenvalues ​​and posterior probabilities of each window image in the positive and negative samples to form a target feature model.

[0021] Step 2-3: Get the next frame of image from the detector, keep tracking the target, and update the target window coordinates and image. Recalculate the overlap between each window in M{} and the current tracking window, re-divide the positive and negative sample subsets according to the overlap value, calculate the feature value of each window image, and update the posterior probability.

[0022] Repeat steps 2-3 to train the feature model. When the model posterior probability update rate ε is less than the set threshold, the training ends, target tracking is stopped, and the model posterior probability is saved. Save the target window image collection during tracking

[0023] The target feature in step 2-2 adopts random forest feature. According to the random forest theory, a random forest feature value can be calculated for each image block, and each feature value corresponds to a posterior probability p=P + / (P + +N - ),P + Represents the number of positive samples, N - Represents the number of negative samples.

[0024] The clustering criterion in the above step 6 is: calculate the Euclidean distance between each growth area, cluster the areas with a distance value less than 5 pixels and the same pixel value in each area into one category, and mark the type id according to the pixel value; the criterion for eliminating irregular clusters is: calculate the outline of the clustered cluster and the aspect ratio R (rectangle width / height) of the circumscribed rectangle, eliminate the clusters with outline area less than the threshold (to obtain better detection results, set the area threshold to 20), and eliminate the clusters with R<0.2 and R>5.

[0025] (III) Beneficial effects

[0026] The real-time target detection method suitable for detecting unconventional targets provided by the above technical solution has the following beneficial effects:

[0027] (1) The present invention can be run in the existing tracker board of the optoelectronic system without the need for additional hardware platform configuration. The detection model training can be performed while executing target tracking. The training iteration cycle is short and there is no need to collect and label the training data set in advance. The present invention has the advantages of low cost, fast response and good real-time performance.

[0028] (2) The present invention can timely increase the detection target category at the task site according to the detection task requirements, and realize the detection of any target of interest in the scene;

[0029] (4) The present invention makes up for the shortcomings of long training and modeling cycle, long detection and calculation time, and low small target detection rate based on big data deep learning, and can complement deep learning.

[0030] (5) The present invention has a small amount of calculation and can implement real-time processing on embedded hardware. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 The present invention is a flowchart of a real-time target detection method suitable for detecting unconventional targets according to an embodiment of the present invention.

[0032] Figure 2 The figure is a schematic diagram of the process of clustering and removing irregular clumps according to an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to make the purpose, content and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below in conjunction with the drawings and examples.

[0034] The hardware platform of this embodiment adopts a self-developed video tracker circuit board based on TI's TMS320C6455 fixed-point digital signal processor. The real-time target detection method of this embodiment applicable to unconventional target detection is implemented in the image processing software package loaded on the hardware platform. Figure 1 As shown, the specific steps are as follows:

[0035] Step 1: Obtain the scene image of frame t0 from the detector and select an object of interest A1 for tracking.

[0036] Step 2: Continuously track target A1, iteratively train the target feature model based on the target area image information, and obtain the model posterior probability and target area image collection

[0037] Step 2-1: According to the window size of target A1, obtain a set of multiple windows M{} of different positions and sizes through proportional scaling, affine transformation, etc., calculate the overlap between each window and the tracking target window, and divide M{} into two subsets of positive samples and negative samples according to the overlap, and save the window coordinates and sizes in the positive and negative samples respectively.

[0038] Step 2-2: Calculate the eigenvalues ​​and posterior probabilities of each window image in the positive and negative samples to form a feature model of the A1 target.

[0039] Step 2-3: Get the t0+1 frame image from the detector, keep tracking the target, and update the window coordinates and image of the tracking target A1. Recalculate the overlap between each window in M{} and the current tracking window, re-divide the positive and negative sample subsets according to the overlap value, calculate the feature value of each window image, and update the posterior probability

[0040] Step 2-4: Repeat steps 2-3 to train the feature model. When the model posterior probability update rate ε is less than the set threshold, the training ends, target tracking is stopped, and the model posterior probability is saved. Save the target window image collection during tracking

[0041] Step 3: Select a new target of interest, repeat steps 1 and 2, and save each tracking target A1, A2, ...A n The feature model posterior probability and target window image form a multi-type target feature library. Note:

[0042]

[0043] where A1, A2, …, A n represents n targets of interest, Indicates target A n The image collection of all tracking areas during the tracking process stores the target feature library data into the falsh memory of the video tracker board.

[0044] Step 4: Start target detection. Get a scene image from the detector, downsample the image to get the downsampled image I, start from the coordinate point (0, 0) of image I, slide the window pixel by pixel from left to right and from top to bottom, calculate the feature value of each window image, and calculate the cumulative value of the posterior probability of each window image feature value under each type of target based on the model posterior probability in the target feature library. Filter out the windows with posterior probability greater than 0.6, record the upper left corner coordinates P(x, y) of the window and the target type id, id = 1, 2, ... n, corresponding to target A1, target A2, ... target An .

[0045] Step 5: Create a binary image I Bin . Set the image pixel value corresponding to the coordinate point position recorded in step 4 to (255-id), and set the remaining pixel values ​​to 0.

[0046] Step 6: Image I Bin Perform region growth and expansion, calculate the Euclidean distance between each growth region, cluster the regions with distance values ​​less than 5 pixels and the same pixel values ​​in each region into one category, and label the type id according to the pixel value. After clustering, multiple regional clusters will be obtained, such as Figure 2 Calculate the outline of the cluster and the aspect ratio R (i.e., rectangle width / height) of its circumscribed rectangle, remove clusters with outline areas less than 20 pixels and R<0.2, R>5, record the remaining clusters as suspected targets, and record the center position coordinates and type id of the suspected targets.

[0047] Step 7: Use the position coordinates recorded in step 6 as the upper left corner coordinates of the window, select the area image on the current scene image, and select the target image set of the corresponding type according to the type id and the feature library. Do cross-correlation calculation, and the window image with a cross-correlation coefficient greater than 0.9 is the target detection result.

[0048] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A real-time target detection method suitable for detecting unconventional targets, characterized in that: The following steps are involved: Step 1: Get a frame of scene image from the detector and select an object of interest for tracking; Step 2: Train the target feature model: Keep tracking the target continuously, iteratively train the target feature model according to the image information of the tracking target area, and obtain the model posterior probability p and the target area image set image{}; Step 3: Create a target feature library and store the target feature library data in the hardware platform memory; Step 4: After completing the training and storage of the target feature library, start target detection; Step 5: Create a binary image I Bin ; Step 6: Image I Bin Perform regional growth, expansion and clustering calculations to identify suspected targets; Step 7: Determine the target detection result based on the suspected target determined in step 6.

2. The real-time target detection method suitable for detecting unconventional targets according to claim 1, characterized in that: The specific steps of step 2 to train the feature model are: Step 2-1: According to the size of the selected target window, a set M{} of window regions with different position coordinates and sizes is established by means of proportional scaling and affine transformation. The overlap between each window in M{} and the target window is calculated, and the set M{} is divided into two subsets of positive and negative samples according to the overlap value. Step 2-2: Calculate the feature values ​​and posterior probabilities of each window image in the positive and negative samples to form a target feature model; Step 2-3: Get the next frame image from the detector, keep tracking the target, update the target window coordinates and image; recalculate the overlap between each window in M{} and the current tracking window, re-divide the positive and negative sample subsets according to the overlap value, calculate the feature value of each window image, and update the posterior probability; Step 2-4: Repeat steps 2-3 to train the feature model. When the model posterior probability update rate ε is less than the set threshold, the training ends, target tracking is stopped, and the model posterior probability is saved. Save the target window image collection during tracking 3. The real-time target detection method suitable for detecting unconventional targets as claimed in claim 2, characterized in that: In step 3, the process of establishing the target feature library is as follows: Repeat steps 1 to 2, save the posterior probability of the target feature model and the target area image information, and complete the establishment of the target feature library, which is recorded as: where A1, A2, …, A n represents n targets of interest, A collection of images representing all tracking areas during the tracking of target A1.

4. The real-time target detection method for detecting unconventional targets as claimed in claim 3, characterized in that: In step 4, the process of storing the target feature library data into the memory is as follows: A frame of scene image is acquired from the sensor and downsampled to obtain the downsampled image I. Starting from the image coordinate point (0, 0), the sliding window traverses I pixel by pixel from left to right and from top to bottom. Calculate the eigenvalue of each window image, and calculate the cumulative value of the posterior probability of the eigenvalue under each type of target based on the target feature library Filter out windows whose posterior probability is greater than the set threshold, and record the upper left corner coordinate P(x, y) of the window and the target type id.

5. The real-time target detection method suitable for detecting unconventional targets according to claim 4, characterized in that: In step 4, the threshold is set to 0.

6.

6. The real-time target detection method suitable for detecting unconventional targets according to claim 4, characterized in that: In step 5, in the binary image I Bin In the example, the image pixel corresponding to the coordinate point position recorded in step 4 is set to (255-id), and the values ​​of the remaining pixels are set to 0.

7. The real-time target detection method for detecting unconventional targets according to claim 6, characterized in that: In step 6, for image I Bin Perform regional growth, expansion and clustering calculations to obtain multiple regional clusters, extract the outlines of the clusters, delete the clusters with small areas and irregular shapes, record the remaining clusters as suspected targets, and record the center position coordinates and type id.

8. The real-time target detection method suitable for detecting unconventional targets according to claim 7, characterized in that: In step 6, the clustering criterion is: calculate the Euclidean distance between each growth area, cluster the areas with a distance value less than 5 pixels and the same pixel value in each area into one category, and mark the type id according to the pixel value; the criterion for eliminating irregular clusters is: calculate the outline of the clustered cluster and the aspect ratio R of the circumscribed rectangle, eliminate the clusters with an outline area less than the threshold, and eliminate the clusters with R<0.2 and R>5.

9. The real-time target detection method for detecting unconventional targets according to claim 8, characterized in that: In step 7, the center position coordinates recorded in step 6 are used as the upper left corner coordinates of the window, and a region image is selected on the current scene image. The region image set corresponding to the type id and the feature library is selected. Perform cross-correlation calculation, and the area image with a cross-correlation coefficient greater than the set threshold is the target detection result.

10. The real-time target detection method suitable for detecting unconventional targets according to claim 7, characterized in that: In step 9, the threshold is selected as 0.9.

Images