Method and system for seamless tracking of a target across cameras
By adaptively adjusting the CLAHE algorithm parameters and using multi-scale fusion technology, the problem of blurred vehicle details caused by uneven lighting in cross-camera seamless tracking was solved, thereby improving image quality and tracking accuracy.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- 东莞市杰瑞智能科技有限公司
- Filing Date
- 2026-03-05
- Publication Date
- 2026-05-19
AI Technical Summary
In highway scenarios, during seamless cross-camera tracking, uneven lighting leads to blurred vehicle details and significant differences in reflective areas. Existing algorithms, such as CLAHE, struggle to adapt to the lighting characteristics of different road sections, thus reducing the accuracy of vehicle target tracking.
By extracting reflective areas from an image, calculating their intensity and difference indices, adaptively adjusting the parameters of the CLAHE algorithm, and combining it with a pixel-level weight map for multi-scale fusion, a final enhanced image is generated, eliminating the algorithm failure problem caused by excessive differences in reflective areas.
It improves image quality in scenes with uneven lighting, ensures the reliability of vehicle feature extraction and tracking, and increases the accuracy of seamless target tracking across cameras.
Smart Images
Figure CN121788573B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing technology, and specifically to a method and system for seamless target tracking across cameras. Background Technology
[0002] Seamless vehicle target tracking aims to locate vehicles and reconstruct their motion trajectory in real time within a continuous sequence of images. Due to various problems that arise in seamless target tracking from a single camera's perspective (such as frequent occlusion between moving targets), its application has expanded from single-camera to multi-camera systems.
[0003] In highway scenarios, seamless cross-camera tracking can be categorized into overlapping and non-overlapping fields of view based on camera deployment. For non-overlapping areas, when a vehicle travels on different sections of the highway, variations in local lighting conditions can cause uneven image brightness, weakening feature representation and impacting stable tracking and recognition of the vehicle target. Vehicle images captured by cameras from different perspectives contain multiple reflective areas (e.g., windshields at different locations, wet road surfaces, metal guardrails, etc.), making existing brightness adjustment algorithms, such as CLAHE (Adaptive Histogram Equalization), difficult to adapt to the lighting characteristics of different road sections. Furthermore, when the differences in reflective areas are significant, the fixed parameter configuration of CLAHE is ill-suited to the lighting characteristics of different road sections, weakening the contrast of normal areas and blurring vehicle details, thus reducing the accuracy of subsequent seamless cross-camera target tracking. Summary of the Invention
[0004] To address the technical problem of reduced tracking accuracy caused by blurred vehicle details, this application provides a seamless target tracking method and system applicable across multiple cameras. The specific technical solution adopted is as follows:
[0005] In a first aspect, this application proposes a seamless target tracking method applicable across cameras, the method comprising the following steps:
[0006] Vehicle images are captured via camera;
[0007] After converting the vehicle image to HSV format, feature vectors are constructed based on the pixel features to divide the image into clusters. The clusters are then divided into reflective and non-reflective regions by analyzing the trend of pixel gray levels within the clusters.
[0008] The size difference of the vehicle image is calculated by the number of pixels in the reflective region; the initial reflective features are calculated based on the brightness and saturation of all clusters, and the reflective feature values of the reflective regions are calculated based on the differences in the initial reflective features; a reflective map is constructed by the distance between reflective regions and the differences in reflective feature values, and the spatial distribution contribution is calculated by the attention mechanism; the intensity difference of the vehicle image is calculated based on the reflective feature values and the spatial distribution contribution; the vehicle image is segmented based on the size difference and intensity difference.
[0009] A weight map is constructed by calculating weight values based on pixel intensity features and brightness values; different adjustment parameters are obtained by adjusting preset parameters based on the features of different types of reflective areas; and then, for vehicle images of different classification types, the weight map is combined with different adjustment parameters to obtain the final processed image.
[0010] The final processed image is used to train a skin to obtain the correlation between vehicles and different cameras;
[0011] By combining correlation settings, a tracking framework can be set up to enable seamless vehicle tracking.
[0012] In the aforementioned solution, this application considers that when a vehicle is driving on a highway, multiple reflective areas will appear on the vehicle body and in the surrounding environment. The reflective characteristics and reflective areas of these areas differ. When the differences are too large, the CLAHE algorithm will fail to process the image and will be unable to handle uneven illumination. This application extracts all reflective areas from the image and calculates the intensity characteristics and difference index of each reflective area. Based on the relationship between the difference index and a preset threshold, the image is divided into two types. For images with large differences in reflective areas, the initial parameters of the CLAHE algorithm are adaptively adjusted based on the intensity and scale characteristics of large and small reflective areas, respectively, to obtain two sets of differentiated enhancement results. Furthermore, the two are fused at multiple scales using a pixel-level weight map to generate the final enhanced image. This eliminates the problem of the CLAHE algorithm failing due to excessive differences in reflective areas, improves the image quality in uneven illumination scenarios, and provides a reliable guarantee for subsequent vehicle feature extraction and tracking tasks.
[0013] In one embodiment, the method for dividing clusters into reflective and non-reflective regions by analyzing the trend of pixel grayscale values within the clusters is as follows:
[0014] For each cluster, extract the gray value sequence outward from the center point in different directions within a preset range; calculate the first difference sequence of each gray value sequence using a first-order difference algorithm;
[0015] A sequence in the first-order difference sequence where all element values are negative is denoted as a trend-decreasing sequence. The ratio of the number of trend-decreasing sequences to the number of gray value sequences is calculated. The mean of the ratio in all clusters is used as a judgment threshold. Each ratio is compared with the judgment threshold. Clusters that are greater than the judgment threshold are denoted as reflective regions, and those that are less than the judgment threshold are denoted as non-reflective regions.
[0016] In one embodiment, the size difference is the ratio of the difference between the number of pixels in the largest reflective area and the number of pixels in the smallest reflective area to the average number of pixels in all reflective areas.
[0017] In one embodiment, the initial reflective characteristics are positively correlated with brightness and negatively correlated with saturation. The normalized value of the average difference between the initial reflective characteristics of the reflective area and all non-reflective areas is used as the reflective characteristic value of the reflective area.
[0018] In one embodiment, the method of constructing a reflectivity map based on the distance between reflective regions and the difference in reflectivity feature values, and calculating the spatial distribution contribution through an attention mechanism, is as follows:
[0019] For all reflective areas, calculate the Euclidean distance between the center points of the reflective areas, take the line connecting the center points as an edge, and take the product of the absolute value of the difference in reflective feature values of the reflective areas and the Euclidean distance between the center points as the edge weight, thus forming a graph of all reflective areas.
[0020] The graph attention network is used to aggregate features of the graph, calculate the attention weight of each edge, normalize the attention weights, and sum the attention weights of the nodes with those of their neighbors to obtain the spatial distribution contribution.
[0021] In one embodiment, the intensity difference is the coefficient of variation of the intensity characteristics of all reflective regions, and the intensity characteristics are positively correlated with the reflective characteristic value and the contribution of spatial distribution.
[0022] In one embodiment, the method for constructing a weight map by calculating weight values using pixel intensity features and brightness values is as follows:
[0023] For each reflective area, the brightness values of all pixels are normalized, and the normalized brightness value is multiplied by the intensity feature of the reflective area to obtain the intensity value of the pixel; the intensity value of pixels in non-reflective areas is 0; the intensity value and brightness value of each pixel are weighted by preset weights to obtain the weight value of each pixel; the weight values of all pixels are used to construct a weight map.
[0024] In one embodiment, the method for segmenting images based on size difference and intensity difference is as follows:
[0025] Calculate the mean of the difference index for all images; if either of the two difference indices exceeds its corresponding mean, the image is classified as a first-class image; otherwise, the image is classified as a second-class image.
[0026] In one embodiment, the method of adjusting preset parameters by using the features of different types of reflective regions to obtain different adjustment parameters, and then weighting the different adjustment parameters using a weighted image to obtain the final processed image for vehicle images of different classification types, is as follows:
[0027] For the first type of image, calculate the average number of pixels in all reflective areas, and mark reflective areas with a number greater than the average as large reflective areas and reflective areas with a number less than the average as small reflective areas.
[0028] The initial contrast limit and grid size are preset, and the contrast limit and grid size are adjusted based on the intensity features and the number of pixels.
[0029] For large reflective areas, the higher the average intensity characteristic of all large reflective areas, the smaller the contrast adjustment limit; the higher the average area ratio of large reflective areas, the larger the adjustment grid size. For small reflective areas, the higher the average intensity characteristic of all small reflective areas, the greater the contrast adjustment limit; the higher the average area ratio of small reflective areas, the smaller the adjustment grid size. The area ratio is the ratio of the number of pixels in the reflective area to the total number of pixels in the vehicle image.
[0030] For both large and small reflective areas, obtain a set of adjustment parameters; the adjustment parameters include adjusting the grid size and adjusting the contrast.
[0031] Based on the adjusted parameters, the vehicle image is processed using CLAHE to obtain two processed images.
[0032] The final processed image is obtained by weighting the two processed images using a weighted graph.
[0033] For the second type of image, the vehicle image is processed using CLAHE with adjustment parameters for large reflective areas to obtain the final processed image.
[0034] On the other hand, embodiments of this application also provide a seamless target tracking system applied across cameras, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of any of the above-described seamless target tracking methods applied across cameras.
[0035] The beneficial effects of this application are as follows:
[0036] This application considers that when a vehicle is traveling on a highway, multiple reflective areas appear on the vehicle body and in the surrounding environment. These reflective areas exhibit differences in reflective characteristics and area. When these differences are too large, the CLAHE algorithm will fail to process the image, failing to address uneven illumination. This application extracts all reflective areas from the image and calculates the intensity characteristics and difference index of each reflective area. Based on the relationship between the difference index and a preset threshold, the image is classified into two types. For images with significant differences in reflective areas, the initial parameters of the CLAHE algorithm are adaptively adjusted based on the intensity and scale characteristics of large and small reflective areas, respectively, resulting in two sets of differentiated enhancement results. These results are further fused at multiple scales using a pixel-level weight map to generate the final enhanced image. This eliminates the problem of the CLAHE algorithm failing due to excessive differences in reflective areas, improves image quality in uneven illumination scenarios, and provides a reliable guarantee for subsequent vehicle feature extraction and tracking tasks. Attached Figure Description
[0037] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0038] Figure 1 This is a flowchart of a method for seamless target tracking across cameras provided in one embodiment of this application. Detailed Implementation
[0039] To further illustrate the technical means and effects adopted by this application to achieve the intended inventive purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of the seamless target tracking method and system applied across cameras proposed in this application. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.
[0040] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.
[0041] Examples of seamless target tracking methods and systems applied across multiple cameras:
[0042] The following description, in conjunction with the accompanying drawings, details the specific scheme of the seamless target tracking method and system applied across cameras provided in this application.
[0043] Please see Figure 1 The diagram illustrates a flowchart of a seamless target tracking method applied across cameras according to an embodiment of this application, the method comprising the following steps:
[0044] Step S001: Acquire vehicle images using a camera.
[0045] Images of vehicles are collected by cameras deployed on highways. Specifically, multiple cameras are set up on the highway to obtain continuous frame images of vehicles traveling under different cameras on different road sections. When selecting cameras, cameras with the same basic configuration (e.g., frame rate, resolution) are chosen to avoid inaccurate results in subsequent image feature extraction due to differences in specifications.
[0046] During image acquisition, the images may be affected by vibrations from the camera sensor or vehicle movement, resulting in noise in the acquired images. Therefore, a filtering algorithm is used to filter all acquired images. In this embodiment, the filtering algorithm used is Gaussian filtering to eliminate noise interference caused by external factors.
[0047] At this point, the preprocessed vehicle image has been obtained.
[0048] Step S002: Divide the reflective and non-reflective regions by clustering the HSV format features of the image.
[0049] When light shines on highly reflective surfaces such as car paint, windows, and wet roads, multiple reflective areas are formed. The saturation component of the reflective areas is small, while the brightness component is large, and the brightness is highest at the center of the reflective area. Due to the different smoothness and reflective area of different object surfaces, the light spots formed have certain differences in reflective characteristics and spatial scale.
[0050] In image representation, reflective areas typically exhibit high grayscale values, forming light spots of varying sizes and brightness. The shape and distribution of these light spots are influenced by multiple factors such as the illumination angle, vehicle position, and camera viewing angle, exhibiting significant randomness and thus displaying dynamically changing distribution characteristics in different images.
[0051] When reflective areas with large scale differences exist in the same image, the parameter settings of the CLAHE algorithm face inherent limitations: it is difficult to simultaneously optimize light spots of different scales by fixing the initial parameters. For example, while increasing the contrast limit parameter can enhance the detailed features around the small-scale reflective area, it will weaken the structural information of the large-scale reflective area. Conversely, it will lead to insufficient contrast improvement in the small-scale area.
[0052] At this point, based on the size difference of the reflective areas in the image, it is identified whether there are light spots with large scale differences; then, based on the reflective characteristics of light spots of different scales and their spatial distribution contribution in the image, appropriate CLAHE parameters are used for processing respectively; finally, through a multi-scale image fusion strategy, an enhanced image that maintains good visual consistency in both the global and local aspects is synthesized.
[0053] Based on the above analysis, since the saturation and brightness of the reflective area differ significantly from those of the normal area, and the brightness is highest at the center of the reflective area, the vehicle image was converted to HSV format using OpenCV to obtain the brightness and saturation components of each pixel in the image.
[0054] Furthermore, a clustering algorithm is used to cluster the vehicle images. In this embodiment, the Mini-Batch K-Means clustering algorithm is used. The vehicle images are taken as input, and an initial number of center points is set (an empirical value of 20). The position, brightness value, and saturation value of each pixel in the image are used as the feature vector of that pixel. The metric distance is set as the Euclidean distance between the feature vectors of the pixel and the center pixel. This is used to divide the image into clusters, and a reflection check is performed on each cluster. In this embodiment, the initial number of center points is set to 20.
[0055] Specifically, taking the u-th cluster as an example, multiple radial directions are preset with the center point of the u-th cluster as the center, and the range of the radial directions is 4-8. The gray value sequences from the center point outward are extracted within a 5×5 interval. Furthermore, a trend analysis algorithm is used to perform trend analysis on the gray value sequences in each direction. In this embodiment, the trend analysis algorithm is the first-order difference algorithm, and 8 radial directions are selected for analysis. The 8 radial directions are extensions from the center point to its eight neighboring directions.
[0056] The grayscale value sequence in each direction is used as input, and the first-order difference algorithm is used to process it to obtain the monotonicity judgment result. In the monotonicity judgment result, if all elements in the first-order difference sequence in that direction are less than 0, it indicates that the grayscale value sequence has a stable decreasing trend as the distance from the center point increases.
[0057] The number of radial directions satisfying the downward trend is counted, and the average proportion of directions satisfying the downward trend in all clusters of the image is calculated as a judgment threshold. When the proportion of directions satisfying the downward trend in the u-th cluster exceeds the judgment threshold, the u-th cluster is determined to be composed of reflective region pixels and is recorded as a reflective region; the remaining clusters are composed of non-reflective region pixels and are recorded as non-reflective regions. If no radial direction with a downward trend exists in any cluster, then the vehicle image has no reflective regions.
[0058] At this point, the reflective areas of the vehicle image have been identified.
[0059] Step S003: Calculate the size difference and intensity difference of the vehicle image based on the number of pixels in the reflective area and the pixel features of the reflective area, and then segment the image.
[0060] In images of moving vehicles, there are multiple reflective areas. These reflective areas can severely affect subsequent target tracking. The differences in size and intensity among these reflective areas are random. When the differences in size or intensity of the reflective areas reach a certain level, the CLAHE algorithm with a fixed initial value will not accurately handle uneven lighting in the image. Therefore, different processing is required when the differences in reflective areas are around a preset threshold. In this embodiment, the threshold value is 0.8.
[0061] Therefore, we first calculate the size difference of the reflective areas in the image. The expression for the size difference is:
[0062] , This represents the average number of pixels in all reflective areas of the image. and These represent the number of pixels in the largest and smallest reflective areas of the image, respectively.
[0063] When a vehicle is traveling on a highway, multiple reflective areas will appear on the vehicle body. The area and intensity of different reflective areas will vary. There will be a significant contrast between the reflective features of reflective and non-reflective areas. Similarly, there will be differences in contrast between the same reflective area and different non-reflective areas. When evaluating the reflective features of a single reflective area, the contribution of the light environment distribution in the entire image should be fully considered.
[0064] For reflective areas, calculate the average saturation and brightness of all pixels in the reflective area, and calculate the initial reflective feature based on the average of the two. The larger the average brightness, the larger the initial reflective feature; the larger the average saturation, the smaller the initial reflective feature.
[0065] The initial reflective characteristics are positively correlated with brightness and negatively correlated with saturation.
[0066] It should be noted that positive correlation means that when one variable increases, the other variable also increases, and the two variables change in the same direction. When one variable changes from large to small or from small to large, the other variable also changes from large to small or from small to large. The specific relationship is determined by the actual application, and this application does not impose any special restrictions.
[0067] It should be noted that negative correlation means that when one variable increases, the other variable decreases accordingly, and the two variables change in opposite directions. When one variable changes from large to small or from small to large, the other variable also changes from small to large or from large to small. The specific relationship is determined by practical application, and this application does not impose any special restrictions.
[0068] Preferably, the expression for the initial reflective characteristics is:
[0069] , and Let represent the mean values of brightness and saturation of the i-th reflective region, respectively. Let e represent the natural constant with base e. This represents the initial reflective characteristics of the i-th reflective region.
[0070] A larger initial reflectivity indicates a greater reflective impact on all pixels within the i-th reflective region. The initial reflectivity of non-reflective regions is calculated using the same method. The normalized value of the average difference between the initial reflectivity of the i-th reflective region and the initial reflectivity of all non-reflective regions is then used as the reflectivity value of the i-th reflective region. The normalization value used in this embodiment is softmax normalization.
[0071] Furthermore, for all reflective areas, the Euclidean distance between the center points of the reflective areas is calculated, the line connecting the center points is taken as an edge, and the product of the absolute value of the difference in reflective characteristic values of the reflective areas and the Euclidean distance between the center points is taken as the edge weight. In this way, all reflective areas are formed into a graph.
[0072] Using a graph as input, each node represents a reflective region. Node attributes include the reflective feature value and center point coordinates of the reflective region, while edge attributes represent the edge weights of the graph. A Graph Attention Network (GAT) is used for feature aggregation. The attention weights calculated for each edge in the last graph attention layer are recorded and normalized using softmax. For each node, the attention weights between it and all its neighboring nodes are summed. This sum is taken as the contribution of the reflective region to the spatial distribution features, denoted as the spatial distribution contribution. It should be noted that in the imaging environment of vehicle images on different sections of highways, the distribution of reflective areas has spatial correlation. In this application, the node attributes and edge attributes of the GAT algorithm simultaneously consider the spatial distance and reflective feature differences of reflective areas, and combine the features of a single reflective area to characterize the spatial distribution relationship between different reflective areas.
[0073] The intensity characteristics of each reflective region are calculated based on its reflective characteristic value and spatial distribution contribution.
[0074] The intensity characteristics are positively correlated with the reflectivity characteristic value and the contribution of spatial distribution.
[0075] Preferably, in this embodiment, the expression for the intensity feature is:
[0076] , This represents the reflective characteristic value of the i-th reflective region. This represents the spatial distribution contribution of the i-th reflective region. and This represents the weighting coefficients, and the sum of the weighting coefficients is 1. This represents the intensity characteristic of the i-th reflective region. In this embodiment, the weighting coefficient is 0.5.
[0077] The intensity characteristics of each reflective region were obtained using the above method, and the coefficient of variation of the intensity characteristics of all reflective regions was calculated as the intensity difference of the vehicle image.
[0078] Thus, the intensity difference and size difference of the vehicle images were obtained, and the mean of the difference index of all images was calculated. If one of the two difference indices exceeds the mean of its respective, the image is classified as a first-class image (there is a difference), otherwise the image is classified as a second-class image (there is no difference).
[0079] At this point, all vehicle images have been differentiated.
[0080] Step S004: Construct a weight map using pixel intensity features and brightness values, and use different adjustment parameters to process the segmented image to obtain a processed map.
[0081] For the first type of image, there will be significant differences in the reflective areas of the image. These differences will cause the CLAHE algorithm to fail. In this case, it is necessary to construct a weight map based on the brightness of each pixel in the image.
[0082] For each reflective area, the brightness values of all pixels within it are normalized, and the intensity value of the pixel is obtained by multiplying the normalized brightness value by the intensity feature of the reflective area it belongs to. In this embodiment, the normalization value used is softmax normalization.
[0083] Taking the a-th pixel in the i-th reflective region as an example, the normalized brightness value of the a-th pixel in the i-th reflective region is used as a weight and multiplied by the intensity feature of the i-th reflective region to obtain the intensity value of the a-th pixel, denoted as . If a pixel is not located within the reflective area, its intensity value is 0.
[0084] Therefore, the intensity features and normalized brightness values of the pixels are weighted to obtain the weight value of each pixel.
[0085] Preferably, in this embodiment, the expression for the weight value is:
[0086] , This represents the intensity value of the r-th pixel. This represents the brightness value of the r-th pixel. This represents the weight value of the r-th pixel. These are the weighted values for intensity and brightness, respectively, and in this embodiment, they are each set to 0.5.
[0087] This yields the weight value of each pixel in the vehicle image, which is then combined with the pixel location to construct a weight map.
[0088] Furthermore, the mean number of pixels in all reflective areas is calculated. Reflective areas with a number greater than the mean are categorized as large reflective areas, while those with a number less than or equal to the mean are categorized as small reflective areas. After dividing the reflective areas into different sizes, different methods are used to calculate the parameters of the reflective areas after adjustment by the CLAHE algorithm, depending on the size of the division.
[0089] First, preset parameters for the CLAHE algorithm are set, namely, preset contrast limit and grid size. In this embodiment, the preset contrast limit is 2 and the grid side length is 8.
[0090] For large reflective areas, the higher the mean intensity characteristic of all large reflective areas, the smaller the contrast limitation; the higher the mean area ratio of large reflective areas, the larger the grid size. For small reflective areas, the higher the mean intensity characteristic of all small reflective areas, the greater the contrast limitation; the higher the mean area ratio of small reflective areas, the smaller the grid size. The area ratio is the ratio of the number of pixels in the reflective area to the total number of pixels in the vehicle image.
[0091] Preferably, the expression for adjusting the parameter is:
[0092] , This represents the number of pixels in the i-th reflective region. This represents the average number of pixels across all reflective areas. Indicates the preset contrast limit. Indicates the size of the preset grid. This represents the mean intensity characteristic of the highly reflective area. This represents the mean intensity characteristic of small reflective areas. This represents the average area ratio of highly reflective regions. This represents the average area ratio of small reflective regions. Let e represent the natural constant with base e. This indicates the adjusted contrast limit. This indicates the adjusted grid size.
[0093] Therefore, through the above steps, different adjustment parameters were calculated using different methods for reflective areas of different sizes. A set of adjustment parameters was used for large reflective areas, and a set of adjustment parameters was used for small reflective areas. For the vehicle image, two processed images were obtained by using CLAHE processing with two sets of adjustment parameters.
[0094] The two processed images are then weighted using a weighted graph to obtain the final processed image.
[0095] Preferably, the expression for the final processed graph is:
[0096] , This represents the image after parameter adjustment based on the small reflective area. This represents the image after parameter adjustments based on the large reflective area. Represents a weighted graph. The weight map represents the difference between the number 1 and the weight value of each pixel in the weight map. This represents the final processed image.
[0097] A higher W indicates that the pixel should retain more original features and requires more detail enhancement; a lower 1-W indicates that the pixel is more affected by reflections and requires more contrast suppression. Small reflective areas have weaker CLAHE processing parameters, preserving more details and textures, suitable for pixels with higher weights, so W is used to weight the processing results for small reflective areas; large reflective areas have stronger CLAHE processing parameters, suppressing reflections and overexposure, suitable for reflective pixels with lower weights, so 1-W is used to weight the processing results for large reflective areas.
[0098] For the second type of image, the difference in reflective areas is small. We can directly use the parameters of the large reflective areas in the first type of image to perform CLAHE processing and obtain its processed image.
[0099] At this point, CLAHE processing has been completed for all vehicle images, thus eliminating the effects of lighting.
[0100] Step S005: Train the skin on the final processed image to obtain the correlation between vehicles and different cameras.
[0101] Using labeling tools such as Labelme, all images that have had their illumination effects eliminated, i.e., the final processed images, are labeled with different types of vehicles (e.g., SUVs, sedans, trucks, vans, etc.). The bounding box shape is a matrix, resulting in a label file for all images. The label file and images are used as input, the loss function is set to CIOU, the optimizer is set to Adam, the number of iterations is set to 500, and the YOLOv8 model is used for training to obtain the object detection model.
[0102] In the tracking system, a trained target detection model is used to detect the vehicle to be detected, obtain the detection box of the target vehicle, obtain the vehicle model, calculate the color histogram of the vehicle within the detection box area, and construct the feature vector of the target vehicle by combining the vehicle model, color histogram, aspect ratio of the detection box, and detection time.
[0103] For target vehicles detected by two cameras, a preliminary screening is first performed based on the detection time and vehicle model. Only vehicles with the same model and a time interval within a reasonable range are retained as candidate matching pairs. In this embodiment, the reasonable range is the maximum allowable time difference estimated based on the distance between the cameras and the average speed of the vehicles.
[0104] For each candidate pair, the similarity of the color histogram and the similarity of the bounding box aspect ratio are calculated. These two similarities are then weighted proportionally to obtain a comprehensive similarity score. In this embodiment, the color histogram similarity uses cosine similarity, and the bounding box aspect ratio similarity is normalized using an exponential function of the absolute difference, with each weighted at 50%.
[0105] A preset similarity threshold is used; when the overall similarity score exceeds this threshold, the vehicles are identified as the same vehicle. To handle complex matching relationships between multiple vehicles, the Hungarian algorithm is used to solve the optimal matching problem for multiple vehicles. A similarity matrix is constructed, and the sum of the overall similarity scores of all candidate pairs is maximized to determine the correlation between vehicles from different cameras. In this embodiment, the similarity threshold is 0.75.
[0106] Step S006: Seamlessly track vehicles by setting up a tracking framework based on correlation.
[0107] In the tracking system, a Long Short-Term Memory (LSTM) network is used to model the historical motion patterns of the vehicle. A motion model is established by analyzing the changes in position, speed and direction in consecutive frames. The Kalman filter algorithm is combined to predict the trajectory and estimate the state of the vehicle during the occlusion period. At the same time, a spatial transformation network is used to spatially align and complete the appearance features of the vehicle under different degrees of occlusion to maintain the consistency of feature representation.
[0108] For the target re-identification task, a triplet loss function is used to ensure that the feature distance of the same vehicle under different cameras and different perspectives is less than the feature distance of different vehicles. Combined with the identity classification loss function, the discriminative ability of the features is improved. A difficult sample mining strategy is used to select challenging training samples to enhance the robustness of the model to changes in lighting, perspective, and partial occlusion.
[0109] By combining image data from all cameras, a distributed multi-camera collaborative tracking framework is established to transmit the detection results, feature representations, and trajectory information of each camera in real time. A unified spatiotemporal coordinate system is constructed, and the local coordinates of each camera are transformed to the global coordinate system through camera calibration parameters. The detection results and trajectory information of each camera are shared in real time through a message passing mechanism.
[0110] A multi-level data structure is used to maintain the complete motion history of the vehicle, including short-term trajectory segments and long-term trajectory sequences. The vehicle's position coordinates, velocity vector, acceleration, direction of motion, appearance features and timestamps are managed in a unified manner. Trajectory matching and fusion algorithms are designed to solve the problem of trajectory overlap and conflict between different cameras.
[0111] The multi-hypothesis tracking (MHT) algorithm is used to handle uncertain correlation situations. The correlation between vehicles and different cameras is used as its input. Through trajectory segment connection and trajectory lifecycle management, the consistency of vehicle identity during camera switching is ensured, and finally seamless target tracking across cameras is achieved.
[0112] Based on the same inventive concept as the above method, this embodiment of the invention also provides a seamless target tracking system applied across cameras, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of any one of the above-described seamless target tracking methods applied across cameras.
[0113] It should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.
[0114] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.
Claims
1. A method for seamless target tracking across multiple cameras, characterized in that, The method includes the following steps: Vehicle images are captured via camera; After converting the vehicle image to HSV format, feature vectors are constructed based on the pixel features to divide the image into clusters. The clusters are then divided into reflective and non-reflective regions by analyzing the trend of pixel gray levels within the clusters. The size difference of the vehicle image is calculated by the number of pixels in the reflective region; the initial reflective features are calculated based on the brightness and saturation of all clusters, and the reflective feature values of the reflective regions are calculated based on the differences in the initial reflective features; a reflective map is constructed by the distance between reflective regions and the differences in reflective feature values, and the spatial distribution contribution is calculated by the attention mechanism; the intensity difference of the vehicle image is calculated based on the reflective feature values and the spatial distribution contribution; the vehicle image is segmented based on the size difference and intensity difference. A weight map is constructed by calculating weight values based on pixel intensity features and brightness values; different adjustment parameters are obtained by adjusting preset parameters based on the features of different types of reflective areas; and then, for vehicle images of different classification types, the weight map is combined with different adjustment parameters to obtain the final processed image. The final processed image is used to train a skin to obtain the correlation between vehicles and different cameras; By combining correlation settings, a tracking framework can be established to seamlessly track vehicles; The method for segmenting images based on size difference and intensity difference is as follows: Calculate the mean of the difference index for all images; if either of the two difference indices exceeds its corresponding mean, the image is classified as a first-class image; otherwise, the image is classified as a second-class image. The method of adjusting preset parameters by utilizing the features of different types of reflective regions to obtain different adjustment parameters, and then weighting the different adjustment parameters using a weighted image for vehicle images of different classification types to obtain the final processed image, is as follows: For the first type of image, calculate the average number of pixels in all reflective areas, and mark reflective areas with a number greater than the average as large reflective areas, and reflective areas with a number less than or equal to the average as small reflective areas. The initial contrast limit and grid size are preset, and the contrast limit and grid size are adjusted based on the intensity features and the number of pixels. For large reflective areas, the larger the average intensity characteristic of all large reflective areas, the smaller the contrast adjustment limit; the larger the average area ratio of large reflective areas, the larger the adjustment grid size. For small reflective areas, the larger the average intensity characteristic of all small reflective areas, the larger the contrast adjustment limit; the larger the average area ratio of small reflective areas, the smaller the adjustment grid size. The area ratio is the ratio of the number of pixels in the reflective area to the total number of pixels in the vehicle image. For both large and small reflective areas, obtain a set of adjustment parameters; the adjustment parameters include adjusting the grid size and adjusting the contrast. Based on the adjusted parameters, the vehicle image is processed using CLAHE to obtain two processed images. The final processed image is obtained by weighting the two processed images using a weighted graph; where the larger the weight of the weighted graph, the larger the proportion of small reflective areas; conversely, the smaller the weight, the larger the proportion of large reflective areas. For the second type of image, the vehicle image is processed using CLAHE with adjustment parameters for large reflective areas to obtain the final processed image.
2. The seamless target tracking method applied across cameras as described in claim 1, characterized in that, The method for dividing clusters into reflective and non-reflective regions by analyzing the trend of pixel gray levels within the clusters is as follows: For each cluster, extract the gray value sequence outward from the center point in different directions within a preset range; calculate the first difference sequence of each gray value sequence using a first-order difference algorithm; A sequence in which all elements of a first-order difference sequence are negative is called a trend-decreasing sequence. Calculate the ratio of the number of trend-decreasing sequences to the number of grayscale sequences, calculate the mean of the ratio in all clusters as a judgment threshold, compare each ratio with the judgment threshold, and record clusters that are greater than the judgment threshold as reflective regions, and those that are less than the judgment threshold as non-reflective regions.
3. The seamless target tracking method applied across cameras as described in claim 1, characterized in that, The size difference is the ratio of the difference between the number of pixels in the largest reflective area and the number of pixels in the smallest reflective area to the average number of pixels in all reflective areas.
4. The seamless target tracking method applied across cameras as described in claim 1, characterized in that, The initial reflective characteristics are positively correlated with brightness and negatively correlated with saturation. The normalized value of the average difference between the initial reflective characteristics of the reflective area and all non-reflective areas is used as the reflective characteristic value of the reflective area.
5. The seamless target tracking method applied across cameras as described in claim 1, characterized in that, The method for constructing a reflectivity map based on the distance between reflective regions and the difference in reflectivity feature values, and calculating the spatial distribution contribution through an attention mechanism, is as follows: For all reflective areas, calculate the Euclidean distance between the center points of the reflective areas, take the line connecting the center points as an edge, and take the product of the absolute value of the difference in reflective feature values of the reflective areas and the Euclidean distance between the center points as the edge weight, thus forming a graph of all reflective areas. The graph attention network is used to aggregate features of the graph, calculate the attention weight of each edge, normalize the attention weights, and sum the attention weights of the nodes with those of their neighbors to obtain the spatial distribution contribution.
6. The seamless target tracking method applied across cameras as described in claim 1, characterized in that, The intensity difference is the coefficient of variation of the intensity characteristics of all reflective areas, and the intensity characteristics are positively correlated with the reflective characteristic value and the contribution of spatial distribution.
7. The seamless target tracking method applied across cameras as described in claim 6, characterized in that, The method for constructing a weight map by calculating weight values using pixel intensity features and brightness values is as follows: For each reflective area, the brightness values of all pixels are normalized, and the normalized brightness value is multiplied by the intensity feature of the reflective area to obtain the pixel intensity value; the intensity value of pixels in non-reflective areas is 0. The weight value of each pixel is obtained by weighting the intensity value and the brightness value by preset weights; A weight map is constructed using the weight values of all pixels.
8. A seamless target tracking system applicable across cameras, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of a seamless target tracking method applied across cameras as described in any one of claims 1-7.