A method for automatically extracting targets from initial frame images in target tracking

Abstract

The present invention discloses a method for automatically extracting targets from initial frame images in target tracking. After manually selecting coordinates, performing image enhancement, saliency detection, and morphological processing on the input image, the corrected coordinates are calculated. Compared with other deep learning-based methods, the fast automatic extraction method of the present invention has a small amount of computation, low computational latency, and an adaptive and controllable process, making it very suitable for deployment and application on embedded platforms with limited computing power.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular relates to a method for quickly and automatically extracting an initial frame image target in target tracking. Background Art

[0002] Object tracking is a very common and important application in vision tasks. Given an input image sequence or video, quickly identifying the target to be tracked is a crucial step in achieving continuous tracking.

[0003] In complex scenarios or when the target to be tracked is unclear, manual target selection is the most reliable method, but manual target selection often leads to errors. Rapid real-time tracking and localization of specific targets of interest has become a critical function widely used in many visual scenarios.

[0004] In many cases, it is difficult to ideally select the target to be tracked in the initial frame image, which causes the subsequent tracker to process it based on the reference template containing background or other interference factors, which can easily lead to the failure of the target tracking task.

[0005] At present, manual framing or target detection algorithms such as convolutional neural networks are usually used to select targets in the initial frame image.

[0006] When manually selecting a target using a frame, you need to draw quickly. Otherwise, the target will move, causing the pixel position of the target to deviate in subsequent frames. If you select a target by clicking the center of the target, it is often difficult to completely select the center of the target.

[0007] Detection methods based on neural network models offer high accuracy and automation, requiring only pre-trained network models for unattended detection. However, these algorithms require platforms with high computing power, such as GPUs or AI chips, making it difficult for hardware platforms with limited computing power to meet real-time requirements.

[0008] For specific application scenarios, the target to be tracked is clear, and collecting scene data can help train effective network model algorithms. However, in some tracking applications, the target to be tracked is uncertain. In these cases, due to a lack of neural network training data, neural network detection models often struggle to achieve good detection results. Summary of the Invention

[0009] This patent proposes a method for fast automatic extraction of initial frame image targets in fast target tracking with small computational delay, so as to improve the problem of inaccurate manual selection without increasing excessive computational delay, thus meeting more usage scenarios.

[0010] The technical solution adopted by the present invention to solve the technical problem is: a method for automatically extracting the target in the initial frame image during target tracking, comprising the following steps:

[0011] S1: For the input initial frame image, manually select the target center of the image. There will often be a certain deviation at this time, so it is necessary to proceed to the next step;

[0012] S2, enhance the initial frame image preprocessing: In the case of infrared small targets or dim targets, the distinction between the background and the real target in the histogram is not large, and it is often difficult to achieve good results. Therefore, the Gamma transform is used to correct the image with too high or too low gray to enhance the contrast. This method is simple to calculate and has less delay: through the transformation formula I′=cI γ Perform a product operation on each pixel in the image, where I is the original pixel value in the image, I' is the new pixel value of the transformed image, c is a constant factor, and γ is a scale factor. When γ>1, the higher grayscale areas in the image will be stretched and the lower grayscale areas will be compressed. When γ<1, the lower grayscale areas will be stretched and the higher grayscale areas will be compressed.

[0013] S3, salient object extraction: saliency detection method is used to further extract the real object of the image;

[0014] S31, perform 5*5 Gaussian smoothing on the image to remove noise interference;

[0015] S32, determine the number of image channels: if it is a three-channel image, convert the initial frame image from the RGB color space to the CIELAB color space (L, a, b); if it is a single-channel image, splice the initial frame image into a three-channel image, which is equivalent to a converted Lab color space image;

[0016] S33, calculating the average values of L, a, and b of the entire image, where L represents brightness, and a and b represent color opposition dimensions;

[0017] S34, by the formula S=||I u -I Lab ||Calculate the L, a, and b values of each pixel in the Lab color space image and the mean of L, a, and b of the entire image, and calculate the Euclidean distance;

[0018] S35, dividing the saliency value of each pixel in the image by the maximum saliency value to obtain a final saliency image;

[0019] S4, morphological processing: edge detection and segmentation of the saliency image to outline the clear outer contour of the target;

[0020] S41, performs contour opening on smooth objects, disconnecting narrow necks and eliminating thin protrusions;

[0021] S42, through the formula f close =(f dilate ) erode Perform a closing operation on the expanded image to obtain the eroded image, where f dilate The expansion of the image is similar to the expansion of the domain, which expands the highlight area or white part of the image so that the result image is larger than the highlight area of the original image. erode Image erosion is similar to the erosion of a region, which reduces and refines the highlight area or white part of the image, making the result image smaller than the highlight area of the original image; after the closing operation, the outline of the image area will be more complete;

[0022] S5, calculate the corrected coordinates: calculate the coordinates of the center of mass in the image to obtain the corrected coordinates of the target, and complete the coordinate repositioning correction of the manually selected target: M 00 =∑ x ∑ y f(x,y) is the sum of the pixels of the image, where C x Indicates the x-axis value of the new coordinate position, C y Indicates the y-axis value of the new coordinate position.

[0023] Furthermore, in step S2, if the initial frame image is in a dim light condition of visible light, or an infrared image with a temperature difference that is not much different from the background, the initial frame image needs to be preprocessed first, and a nonlinear image enhancement method is used here for processing.

[0024] Furthermore, in step S2, the contrast of the input initial frame image is calculated by the formula C=∑ δ δ(i,j) 2 P δ (i, j) is used to calculate the adaptive control parameters. The contrast reflects the gradual change of the image level, where δ(i, j) = |ij|, P δ (i, j) is the pixel distribution probability when the grayscale difference between adjacent pixels is δ, and |ij| is the grayscale value difference between adjacent pixels.

[0025] The present invention utilizes an adaptive image enhancement algorithm to improve the difficulty in extracting features from dim targets. It then performs saliency detection on images with prominent targets, performs edge detection and segmentation on the saliency map based on morphological closing operations, and ultimately calculates and locates accurate target coordinates. Compared to other deep learning-based methods, the present invention's rapid, automated extraction method offers low computational complexity, low computational latency, and an adaptive and controllable process, making it ideal for deployment on embedded platforms with limited computing power. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0027] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0028] Reference Figure 1 As shown, the present invention designs a method to effectively improve the deviation of the manually selected initial frame target, the steps are:

[0029] S1: For the input initial frame image, the target center of the image is manually selected. There will often be a certain deviation at this time, so it is necessary to proceed to the next step.

[0030] S2, enhance the initial frame image preprocessing: In the case of infrared small targets or dim targets, the distinction between the background and the real target in the histogram is not large, and it is often difficult to achieve good results. Therefore, the Gamma transform is used to correct the image with too high or too low gray to enhance the contrast. This method is simple to calculate and has less delay: through the transformation formula I′=cI γ Perform a product operation on each pixel in the image, where I is the original pixel value in the image, I′ is the new pixel value of the transformed image, c is a constant factor, and γ is a scale scaling factor. When γ>1, the higher grayscale areas in the image will be stretched and the lower grayscale areas will be compressed. When γ<1, the lower grayscale areas will be stretched and the higher grayscale areas will be compressed.

[0031] It is not difficult to see that the two parameters c and γ are important factors in controlling the gamma transform. Generally, two values are selected, but fixed values are difficult to adapt to different input images and achieve good enhancement effects in various scenarios.

[0032] Here, according to the contrast of the input image, the formula C = ∑δδ(i, j) 2 P δ (i, j) is used to calculate the adaptive control parameters. The contrast reflects the gradual change of the image level, where δ(i, j) = |ij|, P δ(i, j) is the pixel distribution probability when the grayscale difference between adjacent pixels is δ, and |ij| is the grayscale value difference between adjacent pixels. Based on the contrast, the adaptive Gamma transform can be obtained as:

[0033] parameter Histogram distribution Parameter value C Number of peaks ≥ 3 C=255 / highest peak C Number of peaks ≤ 2 C＝1 parameter Image contrast normalization Parameter value γ Contrast normalization value ≥ 0.5 2 γ Contrast normalization value < 0.5 0.4 .

[0034] In step S2, if the input initial frame image is in a dim light condition of visible light, or an infrared image with little difference in temperature from the background, the input image needs to be preprocessed first. Here, nonlinear image enhancement methods are used for processing. Common image enhancement methods include histogram equalization, etc.

[0035] S3, salient target extraction: After step S2, the input initial frame image has been enhanced and preprocessed. At this time, the target has been highlighted compared to the background. The saliency detection method can be used to further extract the real target of the image.

[0036] S31, performing 5*5 Gaussian smoothing processing on the image to remove noise interference.

[0037] S32, determine the number of image channels: if it is a three-channel image, convert the initial frame image from the RGB color space to the CIELAB color space (L, a, b); if it is a single-channel image, splice the initial frame image into a three-channel image, which is considered equivalent to the converted Lab color space image in subsequent processing.

[0038] S33, calculating the average values of L, a, and b of the entire image, where L represents brightness, and a and b represent color opposition dimensions.

[0039] S34, by the formula S=||I u -I Lab ||Calculate the L, a, and b values of each pixel in the Lab color space image and the mean of L, a, and b of the entire image, and calculate the Euclidean distance.

[0040] S35: Divide the saliency value of each pixel in the image by the maximum saliency value to obtain a final saliency image.

[0041] S4, Morphological Processing: After the salient target extraction in the previous step, the real target can be clearly highlighted. However, sometimes the target's outline is not clear and the edges are incomplete, which will lead to deviations when the final center coordinates are located. Therefore, it is necessary to further perform edge detection and segmentation on the salient image to outline a clear outer contour of the target.

[0042] S41, opening and closing are two basic morphological operations, which are mainly used to find the maximum and minimum areas in the image; opening operation is performed on the contour of general smooth objects to disconnect narrow necks and eliminate thin protrusions.

[0043] S42, the closing operation will also smooth part of the contour, but unlike the opening operation, it usually bridges narrow discontinuities and slender grooves, eliminates small holes, and fills the breaks in the contour line. Therefore, choosing the closing operation can effectively avoid the holes caused by steps S2 and S3, making the target a completed area.

[0044] By formula f close =(f dilate ) erode Perform a closing operation on the expanded image to obtain the eroded image. First, f dilate Output the result, and then do f on this output result erode , where f dilate The expansion of the image is similar to the expansion of the domain, which expands the highlight area or white part of the image so that the result image is larger than the highlight area of the original image. erode Image erosion is similar to the erosion of a region, which reduces and refines the highlight area or white part of the image, making the result image smaller than the highlight area of the original image; after the closing operation, the outline of the image area will be more complete.

[0045] Based on the image in step S4, the coordinates of the center of mass in the image are calculated, which are the coordinates of the corrected target. This completes the coordinate repositioning correction of the manually selected target:

[0046]

[0047]

[0048] Among them, M 00 =∑ x ∑ y f(x,y) is the sum of the pixels of the image, where C x Indicates the x-axis value of the new coordinate position, C y Indicates the y-axis value of the new coordinate position.

[0049] The present invention designs a method to effectively improve the deviation of manually selected initial frame targets. It uses an adaptive image enhancement algorithm to improve the problem of difficult feature extraction of dim targets. Then, the image with prominent targets is subjected to saliency target detection. The saliency map is segmented by edge detection based on morphological closing operations, and the target coordinates are finally calculated and accurately positioned.

[0050] Compared with other deep learning-based methods, the automatic extraction method of the present invention has small computational complexity, low computational delay, and an adaptive and controllable process, making it very suitable for deployment and application on embedded platforms with limited computing power.

[0051] The above description is only for the purpose of illustrating the technical solution and is not intended to limit the same. Any technician familiar with the profession may use the technical content disclosed above to change or modify it into an equivalent embodiment. Those skilled in the art should understand that any modification or equivalent substitution that does not depart from the spirit and scope of the present invention is included in the scope of the claims of the present invention.

Claims

1. A method for automatically extracting an initial frame image target in target tracking, characterized by: Steps S1, for the input initial frame image, manually select the target center of the image; S2, use Gamma transformation to correct the image with too high or too low gray to enhance the contrast: through the transformation formula I′=cI γ Perform a product operation on each pixel in the image, where I is the original pixel value in the image, I′ is the new pixel value of the transformed image, c is a constant factor, and γ is a scale factor. When γ>1, the higher grayscale areas in the image are stretched and the lower grayscale areas are compressed. When γ<1, the lower grayscale areas are stretched and the higher grayscale areas are compressed. S3, uses saliency detection method to further extract the real target of the image: S31, perform 5*5 Gaussian smoothing on the image to remove noise interference; S32, determine the number of image channels: if it is a three-channel image, convert the initial frame image from the RGB color space to the CIELAB color space; if it is a single-channel image, splice the initial frame image into a three-channel image, which is equivalent to a converted Lab color space image; S33, calculating the average values of L, a, and b of the entire image, where L represents brightness, and a and b represent color opposition dimensions; S34, by the formula S=||I u -I Lab ||Calculate the L, a, and b values of each pixel in the Lab color space image and the mean of L, a, and b of the entire image, and calculate the Euclidean distance; S35, dividing the saliency value of each pixel in the image by the maximum saliency value to obtain a final saliency image; S4, perform edge detection and segmentation on the salient image to outline the clear outer contour of the target: S41, performs contour opening on smooth objects, disconnecting narrow necks and eliminating thin protrusions; S42, through the formula f close =(f dilate ) erode Perform a closing operation on the expanded image to obtain the eroded image, where f dilate For image expansion, the highlighted area or white part of the image is expanded so that the result image is larger than the highlighted area of the original image. erode For image erosion, the highlight area or white part in the image is reduced and refined, making the result image smaller than the highlight area of the original image; S5, calculate the coordinates of the center of mass in the image to obtain the coordinates of the corrected target, and complete the coordinate repositioning correction of the manually selected target: Where M 00 =∑ x ∑ y f(x, y) is the total number of pixels in the image, C x Indicates the x-axis value of the new coordinate position, C y Indicates the y-axis value of the new coordinate position.

2. The method for automatically extracting an initial frame image target in target tracking according to claim 1, characterized in that: In step S2, if the initial frame image is in a dim light condition of visible light, or is an infrared image with a temperature difference that is not much different from the background, a nonlinear image enhancement method is used to preprocess the initial frame image.

3. The method for automatically extracting an initial frame image target in target tracking according to claim 1, characterized in that: In the step S2, the contrast of the input initial frame image is calculated by the formula C=∑ δ δ(i, j) 2 P δ (i, j) calculates the adaptive control parameters, where δ(i, j) = |ij|, P δ (i, j) is the pixel distribution probability when the grayscale difference between adjacent pixels is δ, and |ij| is the grayscale value difference between adjacent pixels.

Images