An object imaging tracking method based on image field segmentation and fusion

Abstract

The present invention relates to a target imaging tracking method based on image field segmentation and fusion. By means of regional segmentation and fusion reconstruction of the image field, information acquisition of the region of interest is achieved. On this basis, recognition and tracking of regional targets can be realized. Through this method, high-precision image information of a large range of regions of interest can be obtained, and recognition and tracking of the image information can be completed. The present invention can acquire wide-area image information, has a stronger target area coverage ability, can realize fast recognition and tracking of multiple targets, and has significant advantages for multi-target recognition and tracking and large-range area monitoring.

Description

Technical Field

[0001] The present invention belongs to the field of optoelectronic engineering, and particularly relates to a target imaging and tracking method based on image field segmentation and fusion. Background Art

[0002] In recent decades, due to the in-depth development of disciplines such as optics, electronics, and automatic control, information imaging and target tracking technical means have also been continuously improved. Various imaging and tracking systems have been proposed and applied in engineering practice. However, there are contradictory relationships among the instantaneous coverage range, detection and tracking accuracy, and detection parallax in such systems, that is, it is impossible to achieve high-precision detection and tracking under wide-area instantaneous regional coverage, or it is impossible to perform wide-area instantaneous field-of-view coverage under high-precision detection and tracking, or it is impossible to avoid field-of-view overlap and cause target loss under wide-area instantaneous regional field-of-view coverage and high-precision tracking. This makes the current single set of systems unable to perform fast and high-precision detection and tracking of multiple targets in a large range and unable to meet the application requirements in more complex scenarios. The current high-precision tracking systems mainly adopt a gimbal structure based on mechanical means or a one-dimensional or two-dimensional swing-scanning mechanical structure. Among them, the gimbal structure has the advantages of high tracking accuracy and large motion range, but the instantaneous coverage range is small. When tracking multiple targets, mechanical motion switching is required. Therefore, the speed is slow and the target is easily lost. And the tracking systems based on one-dimensional or two-dimensional swing-scanning have the same disadvantages as the former, and the tracking range is smaller. For the tracking systems implemented based on wide-angle imaging optical structures, there are problems of low detection and tracking accuracy or parallax, and high-precision and stable tracking cannot be achieved. Summary of the Invention

[0003] The main problems solved by the present invention are: to overcome the deficiencies and defects of existing methods and technologies, and provide a target imaging and tracking method based on image field segmentation and fusion, which can achieve high-precision detection and tracking of large-range area coverage.

[0004] The technical solution adopted by the present invention to solve the above technical problems is: a target imaging and tracking method based on image field segmentation and fusion, including the following steps:

[0005] Step 1: Collect the optical information of the physical information field and establish a curvature image field;

[0006] Step 2: Segment and correct the curvature image field;

[0007] Step 3: Perform fusion processing on the segmented and corrected curvature image field information to obtain the entire image;

[0008] Step 4: Perform target recognition and tracking on the entire image.

[0009] Further, when collecting the optical information in Step 1, the optical path adopts a fully transmissive optical structure, and the curvature image field is specifically a spherical structure.

[0010] Further, the optical information of the physical information field collected in Step 1 enters the system through the same physical aperture, so that there is no observation parallax for targets at different distances.

[0011] Further, during the curvature image field segmentation process in Step 2, uniform segmentation is performed.

[0012] Further, the segmentation and correction of the curvature image field in Step 2 specifically include:

[0013] Based on the constructed curvature image field, the curvature image field is segmented and corrected by a secondary array system. The secondary array system is composed of multiple modular units, and the spatial arrays of the modular units are arranged. Each modular unit consists of an image field segmentation lens, an aperture stop, and an image field correction lens. The image field segmentation lens is used to deflect the light rays in the corresponding image field area so that they enter the lens barrel and are transmitted through the aperture stop, thereby realizing the segmentation of the curvature image field; the light rays passing through the aperture stop enter the image field correction lens, and the image field correction lens performs flat field correction on the corresponding image field area; when the secondary array systems work simultaneously, the segmentation and correction of the area curvature image field are completed.

[0014] Further, the fusion processing of the segmented and corrected curvature image field information in Step 3 to obtain the entire image specifically includes:

[0015] a) Image preprocessing, including basic operations of digital image information processing, establishing a matching template for the image, and performing transformation operations on the image;

[0016] b) Image registration, adopting a matching strategy to find the corresponding positions of the template or feature points in the image to be fused in the reference image, and then determining the transformation relationship between the two images;

[0017] c) Establishing a transformation model, calculating the parameter values in the mathematical model according to the corresponding relationship between the templates or image features, thereby establishing a mathematical transformation model for the two images;

[0018] d) Unified coordinate transformation, according to the established mathematical transformation model, transforming the image to be fused into the coordinate system of the reference image to complete the unified coordinate transformation;

[0019] e) Fusion reconstruction, performing fusion processing on the overlapping areas of the images to be processed to obtain a smooth and seamless panoramic image.

[0020] Further, the basic operations in Step a) include denoising, edge extraction, and / or histogram processing; performing Fourier transform and / or wavelet transform operations on the image.

[0021] The principle of the present invention lies in:

[0022] The acquisition, transmission, reconstruction, and tracking of physical information are realized by adopting the idea of object field segmentation and fusion. First, the physical information is acquired to construct a wide-field curvature image field. The curvature image field is segmented to form an array image field, and the array image field units are corrected. The corrected array image field is converted into optoelectronic information, which is fused and reconstructed after transmission and storage to form an integrated image field. Information extraction algorithms and tracking algorithms are used to track the target of interest in the information domain, thereby completing the staring imaging tracking within the region. This method does not require any mechanical movement, has a large coverage range, can complete the imaging tracking of the target within the region, can simultaneously realize the continuous tracking of multiple targets, does not require switching between targets, and has the characteristics of high tracking efficiency and not being easy to lose the target.

[0023] The advantages of the present invention compared with the prior art are as follows:

[0024] (1) The present invention combines computational imaging with traditional optoelectronic imaging tracking technology to realize the collection and processing of wide-field physical information, and has the characteristics of wide coverage range and high resolution accuracy.

[0025] (2) The present invention works in a staring imaging mode, does not require a scanning mechanism, and has low power consumption, high speed, and small weight.

[0026] (3) The present invention adopts a modular secondary array structure to segment and correct the curvature image field, which is beneficial to controlling the economic cost and improving the reliability of the system. Description of the Drawings

[0027] Figure 1 It is a schematic flow chart of a target imaging tracking method based on image field segmentation and fusion of the present invention;

[0028] Figure 2 It is a schematic diagram of physical information acquisition and image field establishment;

[0029] Figure 3 It is a schematic diagram of segmentation representation;

[0030] Figure 4 It is a schematic diagram of image field array segmentation and correction;

[0031] Figure 5 It is a video screenshot of tracking a moving target in a complex scene by the target imaging tracking method using image field segmentation and fusion. Detailed Embodiments

[0032] The present invention will be further described below in conjunction with the drawings and specific embodiments.

[0033] A target imaging tracking method based on image field segmentation and fusion according to the present invention realizes gaze imaging tracking of a region by using a technical method of image field segmentation and fusion. Its specific implementation mainly includes three parts, namely, the establishment of a curvature image field, image field segmentation correction, fusion reconstruction, and target tracking. The principle is as Figure 1 shown.

[0034] Step 1: Establishment of a curvature image field

[0035] Collect the optical information of the physical information field to establish a curvature image field. The purpose of establishing the curvature image field is to obtain the optical field information of the target physical scene, and correct the aberration of the optical field information to form an image field with better imaging characteristics. Constructing the image field into a spherical structure with a certain curvature is beneficial to obtaining more physical information and can increase the coverage of the target area. The optical path for obtaining optical information adopts a fully transmissive optical structure, which is easier to realize the acquisition of wide-area physical information and has more advantages in constructing a large-area curvature image field. As Figure 2 , where

[0036] Image field angle range: FOV = IFOV * Nv

[0037] where, IFOV is the instantaneous field of view, and Nv is the number of pixels;

[0038] System focal length: EFL = -d / IFOV

[0039] where, d is the pixel size;

[0040] Focal length of the information field collector: EFL0 = d / (IFOV * M)

[0041] where, M is the magnification of the secondary system;

[0042] Collection aperture size: D = d / (IFOV * (f / #))

[0043] Step 2: Image field segmentation correction

[0044] Segment and correct the curvature image field, and perform fusion processing on the information of the segmented curvature image field; the segmentation is shown as Figure 3 shown. It is extremely difficult to correct the field curvature of a large-area curvature image field. Segment it to obtain small-scale optical field units. According to the non-optical axis characteristics of the sphere, each sub-unit has an independent symmetric central optical axis and curvature. Thus, the field curvature correction of the large-curvature image field is degraded to the field curvature correction of small-scale units. Each correction unit adopts the same modular design. After integration, an array secondary structure is formed. Each correction sub-unit has good interchangeability, good optical consistency, and a compact structure. After the curvature image field is segmented and corrected by the array sub-units, a planar rectangular image field is obtained, and information is collected by a photoelectric sensor to convert the optical information into electronic image information, laying the foundation for information fusion. AsFigure 4 。

[0045] The array unit information after segmentation and correction is transmitted by optical fiber. Since the number of secondary modular array units is large, it is necessary to merge multiple optical fiber data transmissions into one optical fiber output to simplify the system circuit. The data output from the optical fiber is fused and reconstructed so that the information of each subunit is fused into an entire image. The information fusion process mainly uses methods such as feature point recognition or camera calibration. The method of feature point recognition requires a large degree of overlap between images for algorithm recognition and calculation, while camera calibration requires less image overlap. However, after the hardware changes, it often needs to be recalibrated. The image plane brightness consistency of the fused information is corrected to obtain an entire image with consistent brightness.

[0046] Step 3: Fusion reconstruction and target tracking

[0047] For target detection and recognition in large-scale complex scenes, it is improved and optimized based on the target detection and recognition algorithm of multi-scale, multi-view, and multi-level fusion, so that the algorithm can better adapt to the problems of small targets, few features, and low recognition rates, such as Figure 4 。When the target moves, its posture changes, the lighting conditions change, and there is interference from clutter background, making it very difficult to segment the image based on grayscale. The centroid or center of gravity of the target is inaccurate. Therefore, a tracking method based on image matching is adopted, that is, correlation tracking. Correlation tracking is based on image similarity measurement and finds the tracking method that is closest to the target template image in the real-time image obtained on-site. It does not require image segmentation and feature extraction processing, but only operates on the original image data, thus retaining all the information of the image.

[0048] Common correlation tracking algorithms include product correlation, mean absolute difference, sequential similarity detection, etc. In complex scenes, this is a practical target tracking method.

[0049] Using the SAD correlation algorithm, good tracking results can be obtained for cases where the average grayscale changes little. In the actual tracking process, since both the target and the background are constantly changing, there are inevitably deformations, occlusions, and sudden movements. Simply using the image at the best matching position of the current image as the template for the next frame image matching, the tracking result is easily deviated from the correct position due to a sudden change in a certain frame. As the error accumulates, a larger drift gradually occurs in the matching process, ultimately resulting in the loss of the target during the tracking process. Therefore, template update must be carried out. In practical applications, the most common method is timed update, that is, according to the application situation of different projects, the template is updated at fixed intervals, usually using intervals such as 0.1 second / 0.5 second / 1 second / 2 seconds for update.

[0050] The following is the sum of absolute differences SAD:

[0051]

[0052] Wherein, u and v are the positions of the template; x and y are the positions of the real-time image; M(u, v) is the target template, and S(x, y) is the search area; U and V are the sizes of the template, and X and Y are the sizes of the real-time image.

[0053] The information for completing image tracking can be displayed on the monitor or stored for easy viewing.

[0054] Figure 5 It is a video screenshot of tracking a moving target in a complex scene by the target imaging tracking method using image field segmentation and fusion. In this practical case, the image fusion resolution is 4416x3696, and 6 moving targets are continuously tracked. Stable tracking can be achieved when the moving targets cross and pass through the fusion boundary.

[0055] The content not described in detail in the specification of the present invention belongs to the well-known technology of those skilled in the art.

Claims

1. A target imaging tracking method based on image field segmentation and fusion, characterized in that, It includes the following steps: Step 1: Collect the optical information of the physical information field to establish a curvature image field; Step 2: Segment and correct the curvature image field; Step 3: Perform fusion processing on the segmented and corrected curvature image field information to obtain the entire image; Step 4: Perform target recognition and tracking on the entire image; In Step 1, when collecting the optical information, the optical path adopts a full-transmission optical structure, and the curvature image field is specifically a spherical structure; The optical information of the physical information field collected in Step 1 enters the system through the same physical aperture, so that there is no observation parallax for targets at different distances; In Step 2, uniform segmentation is performed during the segmentation process of the curvature image field; Step 2 for segmenting and correcting the curvature image field specifically includes: Based on the constructed curvature image field, use a secondary array system to segment and correct the curvature image field. The secondary array system consists of multiple groups of modular units, and the spatial arrays of each modular unit are arranged. Each modular unit is composed of an image field segmentation lens, an aperture stop, an image field correction lens, and a sensor. The image field segmentation lens is used to deflect the light rays in the corresponding image field area so that they enter the lens barrel and are transmitted through the aperture stop, thereby realizing the segmentation of the curvature image field; The light rays passing through the aperture stop enter the image field correction lens, and the image field correction lens performs flat-field correction on the corresponding image field area; when the secondary array system works simultaneously, the segmentation and correction of the curvature image field are completed.

2. The object imaging tracking method based on image field segmentation and fusion according to claim 1, characterized in that: Step 3 for performing fusion processing on the segmented and corrected curvature image field information to obtain the entire image specifically includes: a) Image preprocessing, including basic operations of digital image information processing, establishing a matching template for the image, and performing transformation operations on the image; b) Image registration, adopting a matching strategy to find the corresponding positions of the template or feature points in the image to be fused in the reference image, and then determining the transformation relationship between the two images; c) Establish a transformation model, calculate the parameter values in the mathematical model according to the corresponding relationship between the templates or image features, and thus establish a mathematical transformation model for the two images; d) Unified coordinate transformation, according to the established mathematical transformation model, transform the image to be fused into the coordinate system of the reference image to complete the unified coordinate transformation; e) Fusion reconstruction, perform fusion processing on the overlapping areas of the image to be processed to obtain a smooth and seamless panoramic image.

3. The object imaging tracking method based on image field segmentation and fusion according to claim 2, wherein: The basic operations in Step a) include denoising, edge extraction, and / or histogram processing; perform Fourier transform and / or wavelet transform operations on the image.

Images