An environment perception based target tracking method

Abstract

The present application relates to a kind of target tracking methods based on environmental perception, belong to artificial intelligence and target tracking field, the target tracking method by combining target tracking and semantic segmentation, continuously monitor the motion state of target object, provide support for decision-making, including image acquisition and pre-processing, environmental perception semantic segmentation, boundary fitting post-processing, target tracking and tracking parameter automatic update;Target tracking method based on environmental perception is shown in semantic segmentation can provide in-depth understanding of scene, including the position, state and context information of target, provide support for decision-making, and by combining target tracking and semantic segmentation can complement each other, improve the adaptability and robustness of system to complex environment, provide a solid foundation for the development of intelligent system.Target tracking parameter automatic update, ensure the stable tracking across scenes.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence technology and target tracking technology, and in particular to a target tracking method based on environmental perception. Background Technology

[0002] Target tracking technology identifies a specific target from a video frame and then continuously tracks its position and movement in subsequent frames. It plays a crucial role in various applications such as intelligent surveillance and autonomous driving. Traditional target tracking methods include feature-based and correlation-based methods. Feature-based methods extract and match feature points of the target, but are susceptible to changes in lighting and occlusion. Correlation-based methods locate the target by calculating the correlation between target templates in the search region, but their generalization ability is limited, and they are easily affected by interference when switching between different scenes, leading to target loss. Summary of the Invention

[0003] To address the aforementioned technical problems, this invention provides a target tracking method based on environmental perception. This method combines target tracking and semantic segmentation to continuously monitor the motion state of the target object, providing support for decision-making. The method includes the following steps:

[0004] Step S1: Using the images acquired by the optical lens and camera, evaluate the average brightness and variance information of the images to determine whether the current image lighting is appropriate, and complete the image preprocessing;

[0005] Step S2: Perform pixel-by-pixel segmentation on the image using a semantic segmentation algorithm to determine the environmental region and obtain a binary image;

[0006] Step S3: Use morphology-based dilation and erosion algorithms to find the boundary regions of different environmental areas in the binary image, and fit the boundary lines between different regions;

[0007] Step S4: Use a target tracking algorithm to track objects in the image and obtain the coordinate information of the objects in each frame;

[0008] Step S5: Determine the intersection relationship between the target coordinates and the binary map of the environment to obtain the current target's environmental region;

[0009] Step S6: Calculate the distance relationship between the target coordinates and the boundary line, and determine whether the current target is in a cross-region state within the boundary line area;

[0010] Step S7: Based on the target's location and status obtained in steps S5 and S6, the KCF tracking algorithm automatically updates its parameters to adapt to stable tracking in different scenarios.

[0011] Step S2 involves pixel-by-pixel segmentation of the captured image using the Enet semantic segmentation algorithm. Enet (Efficient Neural Network) is a high-efficiency neural network for semantic segmentation, designed to achieve high-precision image semantic segmentation. Input image resolutions range from 416*416*3 to 1024*1024*3 to accommodate different computing platforms, including HiSilicon HI3559A, Rockchip RK3588, and server-side environments. The model output is a 416*416*4 binary image containing environmental regions such as sky, water, buildings, and background. The Enet model structure has 20 layers, primarily consisting of convolutional bottleneck structures, downsampling and upsampling modules. The number of output channels increases from 16 to 64, then to 128, and finally decreases to 4. The dilated convolutional parts use dilation coefficients of 2, 4, 8, and 16.

[0012] In one embodiment of the present invention, step S3 further includes the following sub-steps:

[0013] S301: For environmental areas such as the sky, water surface, and land buildings, three binary images are subjected to morphological dilation operations using 3*3 convolution kernels to expand the boundary regions and fill the internal holes.

[0014] S302: Find the intersection regions of the binary graphs between the sky and the water surface, the sky and the land, and the water surface and the land;

[0015] S303: Find the contour points of the intersection region and use the least squares method to fit the boundary lines between different regions, including the sky-water boundary line, the sky-land boundary line, and the land-water boundary line.

[0016] In one embodiment of the present invention, the intersection relationship in step S5 is specifically as follows:

[0017] If the target coordinates are within the range of the sky binary map, flag1 = 1; if the target coordinates are within the range of the water surface binary map, flag1 = 2; if the target coordinates are within the range of the ground building binary map, flag1 = 3.

[0018] In one embodiment of the present invention, determining the relationship between the tracking target coordinates and multiple environmental boundary lines in step S6 includes the following sub-steps:

[0019] S601: Calculate the center coordinates and width and height of the tracking target bounding box;

[0020] S602: Calculate the Euclidean distance between the center coordinates of the target bounding box and every two points on the environment boundary line to obtain the minimum distance d between the center coordinates and the boundary line;

[0021] S603: If the minimum distance d is greater than the target's width and height, the target is considered to be in a single environment, flag2 = 0; if the minimum distance d is less than or equal to the target's width and height, the target is considered to be intersecting the environmental boundary line and undergoing cross-regional movement; if the target intersects the sky-water boundary line, flag2 = 1; if the target intersects the land-water boundary line, flag2 = 2; if the target intersects the sky-land boundary line, flag2 = 3.

[0022] In one embodiment of the present invention, step S7 of the KCF tracking algorithm determines adaptive parameter updates based on the environmental boundary line, including the following sub-steps:

[0023] S701: Based on the judgment in step S6 that the target has reached the boundary line, enable local contour enhancement of the search area to prevent the target from being similar to the background and thus submerging the target;

[0024] S702: Reduce the tracking search area and decrease the update rate of the target template;

[0025] S703: Increase the APCE value to stop the target template from updating, achieve memory tracking, and prevent the target template from being contaminated in the boundary area.

[0026] The technical solution of the present invention has the following advantages compared with the prior art:

[0027] 1. Compared with traditional target tracking solutions, this system combines semantic segmentation algorithms and target tracking algorithms to perceive the surrounding environment in real time, enabling stable tracking in various environments.

[0028] 2. The target tracking process is initialized using category and boundary information from semantic segmentation, and auxiliary information is provided during the tracking process to cope with complex situations such as occlusion, lighting and environmental changes;

[0029] Semantic segmentation technology enhances a machine's ability to perceive its surroundings by identifying the category of each pixel in an image, such as distinguishing between the sky, water, and buildings.

[0030] Combining target tracking and semantic segmentation can complement each other, improving the system's adaptability and robustness to complex environments and providing a solid foundation for the development of intelligent systems.

[0031] 3. In practical applications, the automatic parameter update module can automatically adjust the parameters of the tracking algorithm according to changes in the target's environment, avoiding the complexity of manual adjustments and taking into account the intelligence and flexibility of the entire system. Attached Figure Description

[0032] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings.

[0033] Figure 1a , 1b 1c represents the detection effect of different environmental perception boundary lines provided by the present invention;

[0034] Figure 2a , 2b 2c represents the stable tracking performance of the target tracking algorithm provided by this invention across different scenarios;

[0035] Figure 3 This is an algorithm flowchart of the target tracking method based on environmental perception provided by the present invention. Detailed Implementation

[0036] like Figure 3 As shown, this embodiment provides a target tracking method based on environmental perception. This method combines target tracking and semantic segmentation to continuously monitor the motion state of the target object, providing support for decision-making. The method includes the following steps:

[0037] Step S1: Using the images acquired by the optical lens and camera, evaluate the average brightness and variance information of the images to determine whether the current image lighting is appropriate, and complete the image preprocessing;

[0038] Step S2: Perform pixel-by-pixel segmentation on the image using a semantic segmentation algorithm to determine the environmental region and obtain a binary image;

[0039] Step S3: Use morphology-based dilation and erosion algorithms to find the boundary regions of different environmental areas in the binary image, and fit the boundary lines between different regions;

[0040] Step S4: Use a target tracking algorithm to track objects in the image and obtain the coordinate information of the objects in each frame;

[0041] Step S5: Determine the intersection relationship between the target coordinates and the binary map of the environment to obtain the current target's environmental region;

[0042] Step S6: Calculate the distance relationship between the target coordinates and the boundary line, and determine whether the current target is in a cross-region state within the boundary line area;

[0043] Step S7: Based on the target's location and status obtained in steps S5 and S6, the KCF tracking algorithm automatically updates its parameters to adapt to stable tracking in different scenarios.

[0044] At the same time, such as Figure 1a , 1bAs shown in Figure 1c, by combining target tracking and semantic segmentation algorithms, a more powerful and intelligent system is formed. Semantic segmentation provides a deep understanding of the scene, including the target's location, category, and contextual information. Target tracking, on the other hand, continuously monitors the motion state of these objects, supporting decision-making.

[0045] Step S2 involves segmenting the captured image pixel-by-pixel using the semantic segmentation algorithm Enet. The Enet network input is 416*416*3, and after forward inference, it outputs a 416*416*4 result, which are binary image masks for regions such as sky, water surface, buildings, and background.

[0046] Specifically, this includes semantic segmentation, a highly challenging yet widely applied technique in computer vision. Its core task is to identify semantic information in various regions of an image through pixel-level classification. This technique can not only identify objects in an image but also accurately depict their outlines, thus achieving a deep understanding of the image structure. Enet (Efficient Neural Network), a leader in this field, is a highly efficient neural network model specifically designed for semantic segmentation tasks. Enet was designed to achieve high-precision image semantic segmentation while maintaining a balance between computational efficiency and resource consumption.

[0047] Enet employs a typical encoder-decoder architecture, which allows the model to extract image features during the encoding phase and then map these features back to the semantic segmentation results at the original resolution during the decoding phase. Enet's network structure is meticulously designed, comprising four main parts: the initial block, the encoder, the decoder, and the final block. The initial block is the starting point of the model, responsible for preprocessing the input image and initial feature extraction. This part typically includes a series of convolutional layers and normalization layers, such as batch normalization, which helps extract basic low-level features such as edges and textures from the original image. Next is the encoder, which gradually reduces the size and number of channels of the feature map while increasing the feature depth through stacked convolutional layers and pooling operations to extract high-level features of the image. The design of this part is crucial because it directly affects the model's ability to understand image content. To further improve model efficiency, ENet employs depthwise separable convolution in its Encoder. This convolutional approach significantly reduces the number of parameters and computational complexity without sacrificing feature extraction capabilities. The Decoder is another core part of ENet. It restores the resolution of the segmentation result through upsampling and transposed convolution operations, mapping low-dimensional features back to the original image size. Upsampling increases the size of the feature map through interpolation techniques, while transposed convolution further refines the features based on the upsampling, achieving fine-grained feature mapping. This process not only needs to accurately restore the spatial resolution of the image but also needs to preserve the semantic information of the features, which is crucial for the final semantic segmentation result. The final block is the key step in ENet generating the final semantic segmentation map. This part may contain multiple convolutional layers to further refine the features and generate the final segmentation map. Furthermore, the final block also contains activation functions, such as Softmax or Sigmoid, to convert the output of the convolutional layers into a probability distribution, thereby achieving pixel-level class prediction. These operations ensure that the final output semantic segmentation map has high accuracy and readability.

[0048] To further improve Enet's performance, the model employs a series of optimization strategies. For example, dilated convolution, by introducing holes in the convolution kernel, increases the receptive field without increasing the number of parameters, helping to extract broader contextual information. 1x1 convolution is used to recombine information between feature channels, aiding in the extraction of more effective feature representations. Parallel branches, by processing feature maps of different resolutions in parallel, can simultaneously capture local and global information of the image, enhancing the model's generalization ability. Feature fusion, by fusing high-level features from the encoder with low-level features from the decoder, improves the model's ability to capture details. These optimization strategies enable Enet to achieve fast and accurate semantic segmentation on resource-constrained devices, making it suitable for various real-time image analysis tasks.

[0049] Step S3 uses a morphology-based dilation and erosion algorithm to find the boundary regions of different environmental areas in the binary image. This process can be achieved through the following steps:

[0050] (1) Apply morphological dilation operations to the three binary images of the sky, water surface, and land structures using a 3*3 convolution kernel to expand the boundary regions and fill the internal holes. (2) Find the intersection regions between the binary images of the sky and water surface, the sky and land, and the water surface and land. (3) Find the contour points of the intersection regions and use the least squares method to fit the boundary lines between the different regions.

[0051] Step S4 uses the KCF (Kernelized Correlation Filters) algorithm to track objects in the captured image. KCF is a classic algorithm for object tracking, based on kernel correlation filters. It achieves object tracking by mapping the target template to a high-dimensional feature space and evaluating the similarity between the target template and candidate regions using filters. Specifically, the KCF algorithm includes four main steps: feature extraction, kernel function mapping, filter training, and object tracking. In the feature extraction stage, features are first extracted from the target template and candidate regions, typically using techniques such as Fast Fourier Transform (FFT) to accelerate computation. Then, kernel functions are used to map these features to a high-dimensional feature space to better distinguish the target from the background. Next, in the filter training stage, a set of kernel correlation filters is obtained by learning the linear relationship between the target template and candidate regions. Finally, in the object tracking stage, the trained filters are used to track new image frames, selecting the region most similar to the target template from the candidate regions as the target's location. The KCF algorithm is fast and accurate, making it suitable for real-time object tracking tasks.

[0052] Step S5 determines the intersection between the tracking target coordinates and the environment binary map to obtain the current environmental region where the target is located. Specifically: if the tracking target coordinates are within the sky binary map range, flag1 = 1; if the tracking target coordinates are within the water surface binary map range, flag1 = 2; if the tracking target coordinates are within the ground building binary map range, flag1 = 3.

[0053] Step S6 determines the relationship between the tracking target coordinates and multiple environmental boundaries. Specifically, this includes: (1) obtaining the center coordinates and width and height of the tracking target bounding box; (2) calculating the Euclidean distance between the center coordinates of the target bounding box and every two points of the environmental boundaries, obtaining the minimum distance d between the center coordinates and the boundaries; (3) if the minimum distance d is greater than the target width and height, the target is considered to be in a single environment, flag2 = 0; if the minimum distance d is less than or equal to the target width and height, the target is considered to be intersecting with the environmental boundaries and undergoing cross-regional movement. If the target intersects with the sky-water boundary, flag2 = 1; if the target intersects with the land-water boundary, flag2 = 2; if the target intersects with the sky-land boundary, flag2 = 3. Figure 2a , 2b As shown in 2c.

[0054] Step S7 is based on the environmental boundary line judgment in S6, and the parameters are updated adaptively. Specifically, it includes: (1) based on the judgment in S6 that the target has reached the boundary line, local contour enhancement of the search area is enabled to prevent the target from being similar to the background and thus submerging the target; (2) the tracking search area is reduced to reduce the update rate of the target template;

[0055] (3) Increase the value of APCE to stop the target template from updating, realize memory tracking, and prevent the target template from being contaminated in the boundary area.

[0056] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A target tracking method based on environmental perception, wherein the target tracking method continuously monitors the motion state of a target object by combining target tracking and semantic segmentation to provide support for decision-making, characterized in that, Includes the following steps: Step S1: Using the images acquired by the optical lens and camera, evaluate the average brightness and variance information of the images to determine whether the current image lighting is appropriate, and complete the image preprocessing; Step S2: Perform pixel-by-pixel segmentation on the image using a semantic segmentation algorithm to determine the environmental region and obtain a binary image; Step S3: Use morphology-based dilation and erosion algorithms to find the boundary regions of different environmental areas in the binary image, and fit the boundary lines between different regions; Step S4: Use a target tracking algorithm to track objects in the image and obtain the coordinate information of the objects in each frame; Step S5: Determine the intersection relationship between the target coordinates and the binary map of the environment to obtain the current target's environmental region; Step S6: Calculate the distance relationship between the target coordinates and the boundary line, and determine whether the current target is in a cross-region state within the boundary line area; Step S7: Based on the target's location and status obtained in steps S5 and S6, the KCF tracking algorithm automatically updates its parameters to adapt to stable tracking in different scenarios. Step S2 involves segmenting the captured image pixel-by-pixel using the semantic segmentation algorithm Enet. The input image resolution ranges from 416*416*3 to 1024*1024*3 to accommodate different computing platforms. The model output is a 416*416*4 binary image of the environmental regions, including the sky, water, buildings, and background. The Enet model structure has 20 layers, mainly including convolutional bottleneck structures, downsampling and upsampling modules. The number of output channels increases from 16 to 64, then to 128, and finally decreases to 4. The dilated convolution part uses dilation coefficients of 2, 4, 8, and 16, respectively.

2. The target tracking method according to claim 1, characterized in that: Step S3 also includes the following sub-steps: S301: For environmental areas such as the sky, water surface, and land buildings, three binary images are subjected to morphological dilation operations using 3*3 convolution kernels to expand the boundary regions and fill the internal holes. S302: Find the intersection regions of the binary graphs between the sky and the water surface, the sky and the land, and the water surface and the land; S303: Find the contour points of the intersection region and use the least squares method to fit the boundary lines between different regions, including the sky-water boundary line, the sky-land boundary line, and the land-water boundary line.

3. The target tracking method according to claim 1, characterized in that: The intersection relationship in step S5 is as follows: If the target coordinates are within the range of the sky binary map, flag1 = 1; if the target coordinates are within the range of the water surface binary map, flag1 = 2; if the target coordinates are within the range of the ground building binary map, flag1 = 3.

4. The target tracking method according to claim 1, characterized in that: Step S6, determining the relationship between the tracking target coordinates and multiple environmental boundary lines, includes the following sub-steps: S601: Calculate the center coordinates and width and height of the tracking target bounding box; S602: Calculate the Euclidean distance between the center coordinates of the target bounding box and every two points on the environment boundary line to obtain the minimum distance d between the center coordinates and the boundary line; S603: If the minimum distance d is greater than the target's width and height, the target is considered to be in a single environment, flag2 = 0; if the minimum distance d is less than or equal to the target's width and height, the target is considered to be intersecting the environmental boundary line and undergoing cross-regional movement; if the target intersects the sky-water boundary line, flag2 = 1; if the target intersects the land-water boundary line, flag2 = 2; if the target intersects the sky-land boundary line, flag2 = 3.

5. The target tracking method according to claim 1, characterized in that: In step S7, the KCF tracking algorithm determines adaptive parameter updates based on the environmental boundary, including the following sub-steps: S701: Based on the judgment in step S6 that the target has reached the boundary line, enable local contour enhancement of the search area to prevent the target from being similar to the background and thus submerging the target; S702: Reduce the tracking search area and decrease the update rate of the target template; S703: Increase the APCE value to stop the target template from updating, achieve memory tracking, and prevent the target template from being contaminated in the boundary area.

Images