Wheelchair following target identification method based on deep learning

Abstract

The invention relates to the technical field of image recognition, in particular to a wheelchair following target recognition method based on deep learning, which comprises the following steps of: calculating a gradient statistical value and a color contrast value of each pixel neighborhood based on an original video frame of a wheelchair visual angle, and obtaining a contrast active value of a local area; and adjusting a space coefficient of Gaussian filtering and a scaling factor of a reflection component based on the contrast active value of the local region, and combining with an illumination component to obtain an image frame after illumination compensation. According to the method, the neighborhood gradient statistical value and the color contrast value are calculated pixel by pixel for the image frame, the activity degree of local area details in the image is improved, meanwhile, the space parameters and the reflection component scaling factor of Gaussian filtering are dynamically adjusted according to the activity degree of the local area details, and the illumination compensation effect of the image is finer and more accurate.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and particularly to a wheelchair following target recognition method based on deep learning. Background Art

[0002] The wheelchair following target recognition method is a technical method that uses image recognition technology to collect video images through a camera mounted on a wheelchair, identify and real-time track the position of a target person. Its purpose is to help the mobility-impaired achieve independent travel, especially in the scenario without accompanying care, to realize the automatic and intelligent real-time following and movement of the wheelchair to the target person.

[0003] In the prior art, the accuracy of target recognition and tracking is relatively low in scenarios with drastic light changes or complex backgrounds. For example, the target position may be misjudged under strong light or shadow conditions, and in severe cases, it may lead to misfollowing or even interruption of tracking of the wheelchair. At the same time, the deviation of the target position and re-identification events in continuous multi-frame tracking are not effectively processed, making it difficult to cope with sudden movement or occlusion of the target, resulting in instability of target tracking and reducing the practicality of wheelchair autonomous following. Therefore, improvements are needed. Summary of the Invention

[0004] The purpose of the present invention is to solve the deficiencies existing in the prior art, and a wheelchair following target recognition method based on deep learning is proposed.

[0005] To achieve the above purpose, the present invention adopts the following technical solutions. The wheelchair following target recognition method based on deep learning includes the following steps:

[0006] Based on the original video frame from the wheelchair perspective, calculate the gradient statistical value and color contrast value of each pixel neighborhood to obtain the contrast activity value of the local area. Based on the local area contrast activity value, adjust the spatial coefficient of Gaussian filtering and the scaling factor of the reflection component, and combine with the illumination component to obtain the illumination-compensated image frame.

[0007] Based on the illumination-compensated image frame, segment the potential following target area, and calculate the color texture descriptor value of the area. According to the color texture descriptor value, compare with the predefined following target features to determine the target position, estimate the main color and spatial scale, and form a feature and attribute set of the target.

[0008] Based on the target feature and attribute set, compare the descriptors of consecutive frames to update the tracking trajectory, record the target continuous tracking duration and re-identification events, and generate single-frame tracking efficiency data points. Based on the single-frame tracking efficiency data points of consecutive multi-frames, associate the attribute values in the target feature and attribute set, and perform attribute value classification to form an attribute association performance metric group.

[0009] Based on the performance metric group associated with the attributes, calculate the difference metric of the performance metrics between different groups. Update the feature fusion weights or sample weighting factors according to the comparison result between the difference metric of the group performance and a preset threshold, and construct an adjusted processing parameter configuration.

[0010] Preferably, the steps for obtaining the contrast activity value are as follows:

[0011] Based on the original video frame from the wheelchair perspective, extract the grayscale values within the pixel neighborhood range centered on the current pixel pixel by pixel. Calculate the gradient differences in the horizontal and vertical directions of the pixels within the neighborhood using the Sobel operator and perform a square operation. Sum the squared gradient values to obtain the gradient statistical value of the pixel neighborhood.

[0012] Based on the gradient statistical value of the pixel neighborhood, calculate the Euclidean distance between the color value of the pixel in the RGB color space and the color values of the 8 adjacent pixels around it pixel by pixel. Take the sum of all Euclidean distances to obtain the color contrast value of the current pixel neighborhood.

[0013] Based on the color contrast value of the current pixel neighborhood, calculate the contrast activity value of the local area pixel by pixel.

[0014] Preferably, the steps for obtaining the image frame after light compensation are as follows:

[0015] Based on the contrast activity value of the local area, statistically analyze the distribution of the contrast activity values of the local areas at all pixel positions in the current image frame. Extract the pixel positions where the contrast activity value is higher than the average contrast activity value of all pixel positions in the image frame, and statistically analyze the proportion of the pixel positions to obtain the high-activity area proportion value.

[0016] Based on the high-activity area proportion value, calculate the spatial coefficient of Gaussian filtering.

[0017] According to the spatial coefficient of Gaussian filtering, adjust the scaling factor of the reflection component at the corresponding position in the image frame pixel by pixel, and fuse the adjusted reflection component with the illumination component at the corresponding position pixel by pixel to obtain the image frame after light compensation.

[0018] Preferably, the steps for obtaining the color texture descriptor value are as follows:

[0019] Based on the image frame after light compensation, perform gray-level threshold detection on the image frame pixel by pixel using the region growing method, extract the boundary contours of the gray-level continuous regions, and generate a set of potential following target region contours.

[0020] According to the set of potential following target region contours, extract the RGB color histogram and local binary texture features inside each contour region, and merge them to form the color texture feature vector of the region.

[0021] Calculate the color texture descriptor value of the region based on the color texture feature vector of the region.

[0022] Preferably, the steps for obtaining the feature and attribute set of the target are as follows:

[0023] According to the color texture descriptor value of the region, compare the color texture descriptor values of each predefined following target feature in the predefined following target feature set one by one, calculate the Euclidean distance between the color texture descriptor values, sort the Euclidean distances in ascending order, and select the region with the smallest Euclidean distance to form a target candidate region;

[0024] Based on the target candidate region, count the frequency of occurrence of each pixel RGB color value in the target candidate region pixel by pixel. After sorting in descending order according to the frequency of occurrence of the color value, select the RGB color value with the highest frequency as the main color of the target candidate region to obtain the main color of the target;

[0025] Based on the target candidate region, perform edge contour detection pixel by pixel and extract the top, bottom, left, and right four boundary positions of the edge contour of the target candidate region. Calculate the height and width of the edge contour according to the boundary positions, use the height and width of the edge contour as the spatial scale of the target candidate region, and combine the main color of the target to form the feature and attribute set of the target.

[0026] Preferably, the steps for obtaining the single-frame tracking performance data point are as follows:

[0027] Based on the feature and attribute set of the target, extract the color texture descriptor value, main color, and spatial scale included in the feature and attribute set of the target one by one, compare them item by item with the color texture descriptor value, main color, and spatial scale of the corresponding position region in each frame image of the continuous video frames, and determine the matching situation of the target position in the continuous video frames according to the similarity of the three features to generate the continuous frame target matching result;

[0028] According to the continuous frame target matching result, judge frame by frame whether the target position in the current frame image is successfully matched with the target position in the previous frame image. If it is successfully matched, accumulate the number of frames for which the target is continuously tracked. If the matching fails, reset the number of frames for which the target is continuously tracked, and at the same time record the number of target matching failures to form the target continuous tracking duration and re-identification event record;

[0029] Based on the target continuous tracking duration and re-identification event record, count the number of frames for which the target is continuously tracked in a single-frame image and the number of target matching failures in a single-frame image, and use the number of continuously tracked frames and the number of target matching failures as the evaluation indicators of the single-frame tracking performance to form a single-frame tracking performance data point.

[0030] Preferably, the step of obtaining the attribute correlation performance metric group is as follows:

[0031] Based on the single-frame tracking efficiency data points of consecutive multiple frames, extract the number of frames of continuous target tracking and the number of target matching failures in each single-frame tracking efficiency data point frame by frame. Perform threshold judgments on the number of continuous tracking frames and the number of matching failures respectively. When the number of continuous tracking frames is higher than the preset tracking frame threshold and the number of matching failures is lower than the preset failure number threshold, mark the frame tracking status as effective tracking; otherwise, mark it as ineffective tracking, and generate a target tracking validity status flag.

[0032] According to the target tracking validity status flag, associate the target features of the corresponding single-frame image with the spatial scale and main color included in the attribute set respectively with the corresponding frame tracking validity status, and form attribute correlation data of the target spatial scale and main color in the effective tracking state.

[0033] Based on the attribute correlation data of the target spatial scale and main color in the effective tracking state, perform clustering analysis according to the target spatial scale range and the main color numerical type respectively, classify and count the target spatial scale attributes and the target main color attributes, and form an attribute correlation performance metric group.

[0034] Preferably, the step of obtaining the adjusted processing parameter configuration is as follows:

[0035] Based on the attribute correlation performance metric group, respectively count the occurrence frequencies of all target spatial scale classifications and the occurrence frequencies of the main color classifications within each attribute correlation performance metric group, and form an attribute category frequency set of each attribute correlation performance metric group.

[0036] According to the attribute category frequency set, calculate the group performance difference metric value between different attribute correlation performance metric groups.

[0037] Based on the group performance difference metric value, compare the group performance difference metric value with a preset threshold. When the group performance difference metric value exceeds the preset threshold, update the feature fusion weight; otherwise, update the sample weighting factor to obtain the adjusted processing parameter configuration.

[0038] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0039] The present invention calculates the neighborhood gradient statistic value and color contrast value for each pixel of the image frame, enhancing the activity of local area details in the image. Meanwhile, it dynamically adjusts the spatial parameter of Gaussian filtering and the reflection component scaling factor according to the activity degree of local area details, making the light compensation effect of the image more refined and accurate. Further, by extracting the color texture descriptor value based on the compensated image frame and comparing it with the predefined target features, the main color and spatial scale of the target can be determined more efficiently and stably, improving the accuracy of following target recognition. In addition, through the dynamic comparison and tracking of the target features and attribute sets in consecutive frames, the deviation of the target position and re-identification events in the recognition process can be effectively captured and corrected, enhancing the stability of target recognition and continuous tracking. At the same time, taking the population difference performance of the attribute association performance metric as the basis for parameter configuration adjustment, the image recognition process can optimize the processing parameters according to the actual tracking effect, improving the reliability and robustness of wheelchair real-time following the target under complex environmental conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a schematic diagram of the steps of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0041] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0042] Please refer to Figure 1 , the present invention provides a technical solution, a wheelchair following target recognition method based on deep learning, including the following steps:

[0043] Based on the original video frame from the wheelchair perspective, calculate the gradient statistic value and color contrast value of each pixel neighborhood to obtain the contrast activity value of the local area; based on the contrast activity value of the local area, adjust the spatial coefficient of Gaussian filtering and the scaling factor of the reflection component, and combine it with the light component to obtain the image frame after light compensation;

[0044] Based on the image frame after light compensation, segment the potential following target area and calculate the color texture descriptor value of the area; according to the color texture descriptor value, compare with the predefined following target features to determine the target position, estimate the main color and spatial scale, and form the feature and attribute set of the target;

[0045] Based on the target feature and attribute set, compare the descriptors of consecutive frames to update the tracking trajectory, record the target continuous tracking duration and re-identification events, and generate single-frame tracking efficiency data points; based on the single-frame tracking efficiency data points of consecutive frames, associate the attribute values in the target feature and attribute set and classify the attribute values to form an attribute association performance metric group;

[0046] Based on the performance metric groups associated with attributes, calculate the difference metric of the group performance of the performance metrics between different groups. Based on the comparison result between the difference metric of the group performance and a preset threshold, update the feature fusion weight or the sample weighting factor, and construct an adjusted processing parameter configuration.

[0047] The steps for obtaining the contrast activity value are as follows:

[0048] Based on the original video frame from the wheelchair perspective, extract the grayscale values within the 3×3 pixel neighborhood centered on the current pixel pixel by pixel. Calculate the gradient differences in the horizontal and vertical directions of the pixels within the neighborhood through the Sobel operator and perform a square operation. Sum the squared gradient values to obtain the gradient statistic value of the pixel neighborhood.

[0049] Based on the gradient statistic value of the pixel neighborhood, calculate the Euclidean distance between the color value of the pixel and the color values of the 8 adjacent pixels around it in the RGB color space pixel by pixel. Take the sum of all the Euclidean distances to obtain the color contrast value of the current pixel neighborhood.

[0050] Based on the color contrast value of the current pixel neighborhood, calculate the contrast activity value of the local area pixel by pixel. The calculation formula is:

[0051]

[0052] where H is the contrast activity value of the local area, GS k is the gradient statistic value of the k-th adjacent pixel within the current pixel neighborhood, D k is the Euclidean distance in color between the current pixel and the k-th adjacent pixel, CCV is the color contrast value of the current pixel neighborhood, and the upper limit of the summation symbol being 8 represents a total of 8 adjacent pixels excluding the central pixel in the current pixel neighborhood.

[0053] Specifically, for the original video frames from the wheelchair perspective, the video stream is read frame by frame and converted into a three-channel color data matrix. In the specific implementation process, a certain number of image frames can be extracted per second according to the frame rate for processing. Then, pixel data is extracted from each image frame and converted into grayscale form. Next, when performing a local traversal of each pixel, a 3×3 neighborhood centered on the pixel is selected. This neighborhood includes the central pixel and the eight surrounding adjacent pixels. Then, gradient operations in the horizontal and vertical directions are performed on each pixel in the neighborhood. Here, the fixed convolution kernel method of the Sobel operator can be used for gradient extraction, which includes kernel matrices in the horizontal and vertical directions. When calculating, the grayscale values of the neighborhood are multiplied numerically with these two kernels respectively and then accumulated to obtain the differences of the horizontal and vertical components. Subsequently, these two components are squared and added together to obtain the local gradient sum of squares. Then, the same operation process is performed on each pixel until all pixels in the entire image frame are traversed. Then, the values of the gradient sum of squares in the neighborhoods of all pixel positions are statistically calculated. Each pixel corresponds to a calculation result and is stored in a two-dimensional array of the same size, thereby obtaining the corresponding gradient statistical distribution result. In subsequent calculation or analysis steps, the subsequent correlation operations can be carried out by retrieving the values in this two-dimensional array pixel by pixel and the gradient distribution relationship with the surrounding pixels, and finally the gradient statistical value of the pixel neighborhood is obtained.

[0054] Based on the gradient statistical value of the pixel neighborhood, continue to calculate the distance relationship in the RGB color space for each pixel. First, within the same 3×3 neighborhood range, obtain the R-channel, G-channel, and B-channel values of the central pixel and the eight surrounding adjacent pixels. Then, according to the Euclidean distance formula, perform difference and square operations on the corresponding color channels respectively. Add the squared differences of the R, G, and B channels and then take the square root. Thus, the color distance value between the central pixel and each adjacent pixel is obtained. Then, accumulate and add all the distance values corresponding to the number of adjacent pixels (8), and record the total color distance sum within the neighborhood of this pixel. After traversing all pixels in the image frame, the color difference sum distribution corresponding to each pixel neighborhood can be formed. If it is necessary to compare the local performance of some pixels, the color difference value of this pixel neighborhood can be compared with the color difference values of other regions to judge the degree of color mutation in the local block. Finally, after storing and summarizing these calculated numerical information, the color contrast value of the current pixel neighborhood is obtained.

[0055] Formula: The benefit of the formula is that by combining the gradient statistical value with the color distance within the same pixel neighborhood and then synthesizing the color contrast value of the overall neighborhood, it can more accurately reflect the prominence degree of this pixel position relative to the neighborhood. In subsequent image analysis, this can be used to more finely depict the local area changes and assist in subsequent following target detection.

[0056] GS k The steps for obtaining the parameter are as follows: calculate the sum of the squared gradient values of the k-th adjacent pixel in the same frame of image. The gradient value is generated by the difference between the horizontal and vertical directions of the aforementioned Sobel operator. The specific calculation can be expressed as: GS k =(Δx k ) 2 +(Δy k ) 2 , where Δx k and Δy k respectively represent the gray-scale differences of adjacent pixels in the horizontal and vertical directions. In order to obtain a more representative gradient value, all video frames within a certain time period will be included in the calculation according to the frame rate, and the gradient distribution sequence that appears at each adjacent pixel position will be recorded. For example, in a single-frame image with a resolution of 640×480, at an arbitrarily selected k-th adjacent position of a pixel, the gray-scale value at this position can be recorded as a gradient sequence after more than ten frames of measurement. After subtracting from the adjacent pixels respectively, Δx k and Δy k can be obtained. Squaring and adding these differences one by one can obtain GS k .

[0057] D k The steps for obtaining the parameter are as follows: obtain the RGB channel differences between the central pixel and the k-th adjacent pixel within the same neighborhood, and then obtain the color difference between the two pixels according to the three-dimensional Euclidean distance calculation formula . For example, if the channel values of the central pixel are set as R center = 100, G center = 120, B center = 130, and the channel values of the k-th adjacent pixel are R k = 110, G k = 100, B k = 140, then D k can be specifically calculated as

[0058]

[0059] The steps for obtaining the CCV parameter are as follows: accumulate the color distance values between the current pixel and the surrounding eight adjacent pixels. For example, if the color distances among the eight adjacent pixels in the same neighborhood are 12.08, 18.54, 5.37, 25.62, 9.90, 22.15, 14.47, 16.02 respectively, then

[0060] CCV = 12.08 + 18.54 + 5.37 + 25.62 + 9.90 + 22.15 + 14.47 + 16.02 = 124.15.

[0061] Calculation process:

[0062] Now, specific numerical values are taken for calculation. Let the gradient statistical values of eight adjacent pixels be GS1 = 35, GS2 = 40, GS3 = 25, GS4 = 55, GS5 = 70, GS6 = 45, GS7 = 30, GS8 = 60, and the corresponding color Euclidean distances be D1 = 20.01, D2 = 24.49, D3 = 18.06, D4 = 30.00, D5 = 10.20, D6 = 35.10, D7 = 26.35, D8 = 11.90. And CCV = 20.01 + 24.49 + 18.06 + 30.00 + 10.20 + 35.10 + 26.35 + 11.90 = 176.11.

[0063] First, calculate (GS k ×D k ) 2 Taking k = 1 as an example, (35 × 20.01) 2 =(700.35) 2 = 490490.1225. Calculate successively for k = 2 to 8 in the same way to get:

[0064] (40 × 24.49) 2 =(979.6) 2 = 959208.16;

[0065] (25 × 18.06) 2 =(451.5) 2 = 203857.25;

[0066] (55 × 30.00) 2 =(1650) 2 = 2722500;

[0067] (70 × 10.20) 2 =(714) 2 = 509796;

[0068] (45 × 35.10) 2 =(1579.5) 2 = 2493580.25;

[0069] (30 × 26.35) 2 =(790.5) 2 = 625890.25;

[0070] (60 × 11.90) 2 =(714) 2 = 509796;

[0071] Adding up all the above results, we can get

[0072]

[0073] Then, CCV 2 is obtained, 176.11 2 ≈31036.3321.

[0074] Therefore

[0075] This result indicates that, under the given neighborhood gradient and color difference values, the H value corresponding to the current pixel position reaches approximately 2922.12, reflecting that there may be a large superposition of color difference and gradient change amounts in the neighborhood of this pixel. Subsequently, this H value can be horizontally compared with the H values of other pixels in the same frame during target recognition or post-processing, so as to further determine which pixels have obvious contrast activity levels in the local area and provide data support for the subsequent wheelchair following target recognition.

[0076] The steps for obtaining the image frame after light compensation are as follows:

[0077] Based on the contrast activity values of the local areas, statistically analyze the distribution of the contrast activity values of the local areas at all pixel positions in the current image frame, extract the pixel positions in the image frame where the contrast activity values are higher than the average value of the contrast activity values at all pixel positions in the image frame, and statistically analyze the proportion of the pixel positions to obtain the high-activity area ratio value;

[0078] Based on the high-activity area ratio value, calculate the spatial coefficient of Gaussian filtering, and the calculation formula is:

[0079]

[0080] where, GSC is the spatial coefficient of Gaussian filtering, P m is the contrast activity value at the m-th pixel position in the image frame, MCA is the average value of the contrast activity values at all pixel positions in the current image frame, R m and B m are the R-channel and B-channel color values at the m-th pixel position respectively, G m is the G-channel color value at the m-th pixel position, Y g is the average value of the G-channel color values at all pixel positions in the current image frame, M is the number of all pixel positions in the current image frame, and Q is the high-activity area ratio value;

[0081] According to the spatial coefficient of Gaussian filtering, adjust the scaling factor of the reflection component at the corresponding position in the image frame pixel by pixel, and fuse the adjusted reflection component with the illumination component at the corresponding position pixel by pixel to obtain the image frame after light compensation.

[0082] Specifically, based on the distribution of the contrast activity value H in the local area, the corresponding H values of all pixels in the current image frame are statistically recorded. Each pixel corresponds to an H value, and a distribution array of H is formed by traversing the positions of all pixels in the image frame. Subsequently, the array is retrieved to obtain the mean value MCA of H at all pixel positions. Then, the relationship between the H value and MCA at each pixel position is compared one by one. Once a pixel with H greater than MCA is detected, it is classified as a highly active pixel. The accumulation of pixel positions is continuously carried out, and the coordinate information of all highly active pixels is retained. Then, the proportion of such highly active pixels in all pixels of the entire image frame is statistically calculated. When referring to an image frame with a resolution of 640×480, the total number of pixels is 307,200. If it is found that the H values of 61,440 pixels exceed MCA, then after comparison, the proportion value Q of the highly active area is calculated as 61,440 divided by 307,200, which is 0.2. If the on-site image frame has a resolution of 1920×1080, the corresponding number of pixels is 2,073,600. At this time, the total number of highly active pixels is obtained through the same traversal and comparison operations, and the corresponding proportion is calculated. In the whole process, a direct comparison between the H value and MCA is performed for each pixel, and all are determined whether they belong to the highly active area by the unified threshold MCA, without using any custom range segment division or skipping statistics, so as to avoid ignoring local peaks in the contrast activity value distribution. Finally, after summarization, the proportion value Q of the highly active area can be obtained and stored in the data result.

[0083] Formula: The benefit of the formula is that by combining the contrast activity value distribution, pixel color channel information, and the global average G-channel benchmark, the local area saliency and the red and blue channel intensities of the pixels are jointly incorporated into the calculation, so as to more fully reflect the degree of difference between the global and local in the spatial coefficient of Gaussian filtering.

[0084] P m The steps for obtaining the parameter are as follows: This parameter records the contrast activity value H of the m-th pixel position and needs to be calculated based on the previously obtained pixel neighborhood gradient and color contrast calculation formula. In the acquisition process, each frame of the image contains several pixel points, and each pixel corresponds to a contrast activity value H in a local neighborhood. By traversing in sequence, P m sequence can be formed to cover the pixel information of the entire frame. In specific implementation, the corresponding frame image can be split into row and column coordinates, and the gray gradient and color distance distribution are read row by row and column by column, and the local contrast activity value H is obtained by substituting into the aforementioned formula, denoted as P m .

[0085] The steps for obtaining the MCA parameter are as follows: This parameter represents the average value of the contrast activity values at all pixel positions in the current image frame and is used to measure the difference between the contrast activity value of each pixel and the global average level in the formula. The specific calculation formula can be written as Before acquisition, all Ps need to be m added together and divided by the total number of pixels M. To ensure the accuracy of the values, a complete statistics can be made for all pixels during the acquisition process. Example: For a frame image with 307,200 pixels, if the sum of its Ps m is 146,500,000, then

[0086] R m and B m The steps for obtaining the parameters are as follows: These two parameters represent the red channel value and the blue channel value at the m-th pixel position respectively, and need to be obtained by parsing the RGB data of each frame of the image. They can be extracted in the way of 24-bit color channels when reading the image. Each pixel will have a numerical range for the R, G, and B channels during storage, usually between 0 and 255 in 8-bit image data. To ensure accuracy, the data needs to be read completely during the acquisition process and the original color information should be retained. Example: In a certain frame of the image, if the color value at the 1000th pixel position is detected as R 1000 = 120 and B 1000 = 80.

[0087] G m The steps for obtaining the parameter are as follows: This parameter records the green channel value at the m-th pixel position. The way to obtain it is the same as that of the red and blue channels, which is directly extracted from the RGB original data to provide a reference for subsequent comparison with Y g Each pixel corresponds to an independent coordinate in the image. When reading, it can be traversed in the row-column order. A complete scan can obtain 307,200 G m values at a resolution of 640×480. Example: In the same frame, if the G 1000 value detected at the 1000th pixel position is G m = 200, then this value will directly replace the G in the formula

[0088] Y g The steps for obtaining the parameter are as follows: This parameter represents the average value of the G channel color values at all pixel positions in the current image frame. The specific calculation formula can be written as Similar to the way of obtaining MCA, first accumulate the G channel values at all pixel positions and then divide by the total number of pixels M. To obtain more realistic scene data, a complete statistics of the G channel of the entire frame image is required. For example: Among 307,200 pixels, if the total sum of all G m is approximately 52,960,000, then

[0089] The steps to obtain the M parameter are as follows: This parameter represents the total number of pixel positions in the current image frame. At a common resolution of 640×480, its value is 307,200. For image frames with a resolution of 1080P or other resolutions, the calculation is carried out in the same way, and the accurate value is obtained by multiplying the number of rows by the number of columns. To ensure consistency, it is necessary to match the RGB reading process mentioned earlier. Example: For a 640×480 frame image, M = 640×480 = 307,200.

[0090] The steps to obtain the Q parameter are as follows: This parameter represents the proportion value of the mentioned highly active area, which is the proportion of pixels with a contrast activity value higher than the average value. It is necessary to first compare the mean value of each pixel's P m and count the number of highly active pixels, and then divide by M to obtain Q.

[0091] Calculation process:

[0092] Select a single-frame 640×480 resolution image as an example. M = 307,200, and the following parameters are obtained through the previous acquisition:

[0093] MCA = 476.19, Y g = 172.38, Q = 0.2;

[0094] When scanning this frame of image, the red and blue channel values and contrast activity value information of some pixels are counted. In the example, first, for example, after taking out the total number M of pixels, partial results of the summation term are as follows:

[0095] For each pixel position m, execute (P m - MCA) 4 × (R m + B m ) ÷ [1 + |G m - Y g |] to obtain a batch of values. After accumulation, the following can be obtained:

[0096]

[0097] Take the cube root of this result:

[0098]

[0099] So the numerator part is 75,764.2, and the denominator M·Q = 307,200×0.2 = 61,440. Therefore:

[0100]

[0101] The result shows that the Gaussian filtering spatial coefficient obtained after comprehensively considering the contrast activity value, the intensity of the red and blue channels, and the deviation of the green channel from its average value for the current frame image is approximately 1.234. When the value is greater than 1, it means that the proportion of pixel regions with relatively high local contrast within this frame image is moderate and there is a certain increase in the red and blue channel values. The coefficient can be brought into the subsequent processing to perform per-pixel reflection component scaling, providing a quantitative basis for the illumination compensation link.

[0102] When adjusting the reflection component scaling factor at the corresponding position in the image frame per-pixel according to the Gaussian filtering spatial coefficient GSC, it is necessary to first record the existing reflection component of each pixel and combine it with its original brightness value, compare it with GSC to obtain the specific scaling multiple, and then complete the update operation by multiplying the original reflection component by the corresponding multiple. Combine the obtained brightness and color information to ensure that the position of each pixel can be matched during per-pixel fusion. Then, execute this operation process uniformly for all pixels within the entire frame range, traverse the pixel coordinates and adjust the reflection component values in sequence. Then, add the adjusted reflection component to the illumination component at the corresponding position. Specifically, simple accumulation can be used to form the final pixel brightness or color channel value, or it can be calculated by adding channels according to the data stored during the acquisition process. For example, perform fusion on the R, G, and B channels separately to avoid problems of missing calculation or duplicate calculation. During the statistical process, the per-pixel results will be temporarily stored, and then the new values in the entire frame image will be summarized and updated after all pixels are processed, so as to obtain the image frame after illumination compensation.

[0103] The steps to obtain the color texture descriptor value are as follows:

[0104] Based on the image frame after illumination compensation, perform gray-scale threshold detection on the image frame per-pixel using the region growing method, extract the boundary contours of the gray-scale continuous regions, and generate a set of potential following target region contours;

[0105] According to the set of potential following target region contours, extract the RGB color histogram and local binary texture features inside each contour region, and merge them to form the color texture feature vector of the region;

[0106] Based on the color texture feature vector of the region, calculate the color texture descriptor value of the region. The calculation formula is:

[0107]

[0108] Among them, W is the color texture descriptor value of the region, X t is the peak value of the RGB color histogram of the t-th region, BGS t is the gradient statistical value of the binary texture of the t-th region, RSD tis the standard deviation of the RGB color histogram of the t-th region, and RC is the total number of regions in the set of potential following target region contours.

[0109] Specifically, based on the image frame after illumination compensation, when performing the region growing operation, first determine a gray-scale range detection condition. For example, compare the gray-scale value g of each pixel with the preset range of 0 to 255. When g is within the set threshold interval, it is determined as a target pixel that can continue to grow. Then, select the starting coordinates in the image frame as the growth seed and try to connect in the up, down, left, and right directions in turn. For adjacent pixels, also check whether their gray-scale values fall within the same threshold interval. If the difference between the gray-scale value of the adjacent pixel and the seed gray-scale value is less than 20, it is regarded as meeting the connection condition. Whenever an adjacent pixel meets the condition, mark it as the current growth region and add the pixel position to the subsequent growth list. Then repeat this process to continue expanding outwards. It should be noted that the growth is directly interrupted for pixels exceeding the threshold interval, and region merging is not performed if there are large gray-scale differences at certain pixel positions. By this connected way, traverse the entire image frame in turn, count each piece of pixel block with continuous gray-scale and record their edge ranges. Finally, generate the boundary contour according to the coordinates of the accumulated pixel blocks. If several gray-scale connected regions are formed on an image frame of 640×480, summarize the edge positions of these connected regions according to row and column coordinates. If the area of a certain block is greater than 100 pixels, it is determined as a potential target contour. Through this process, multiple possible gray-scale continuous regions can be retained in the final contour set.

[0110] According to the set of potential following target region contours, scan the pixel range of each contour region separately, count the RGB values of all pixels inside it to obtain the corresponding color histogram. Subsequently, divide the histogram and accumulate the frequencies of each pixel's RGB channels. In practice, the RGB space can be divided into several discrete sections (for example, each channel is divided into 16 bins, so there are a total of 16×16×16 = 4096 combinations). Count the number of pixels falling into each combination, and then determine the section where the peak in the color histogram is located. Denote this peak as X. Then, when retrieving the texture features of this region, first perform binarization processing on this region, then calculate the gradient values for the binary result and record the gradient distribution of all pixels. Finally, sum up these distributions to obtain BGS. At the same time, it is also necessary to measure the standard deviation RSD of the RGB color histogram of this region. Calculate the sum of the squared differences between the values of each channel and the mean through pixel traversal and then take the square root to obtain the standard deviation value and organize it into the statistical results of the corresponding region. When there are multiple regions in the contour set, perform these scanning and statistical steps one by one to form a sequence of color texture feature vectors, and summarize the data such as X, BGS, and RSD of each region in the same grouping information. In this way, the complete feature vectors of each contour region can be obtained.

[0111] Formula: The advantage of the formula is that it synthesizes the color peak, texture gradient, and histogram statistical difference in the same expression. By squaring, cubing, and summing and comparing these parameters respectively, it can take into account both the main peak intensity of the regional color distribution and the amplitude of the texture gradient, and ingeniously introduces the standard deviation, a measure of dispersion, into the numerator and denominator, enabling a more comprehensive measurement of both the color and texture of the target area.

[0112] X t The steps to obtain the parameter are as follows: This parameter represents the peak of the RGB color histogram of the t-th region. Its source is to perform RGB statistics on the pixels within each region to form a bin count, find the RGB combination section with the highest frequency of occurrence, and use the ratio of the number of pixels in this section to the total number of pixels to judge the peak intensity. To obtain the specific peak value, each channel can be divided into N segments first, and the distribution of pixels in these segments can be counted. If a certain segment has the most cumulative pixels, the frequency or its corresponding intensity of this segment can be defined as the peak of this histogram. In practice, a target area (for example, containing 2000 pixels) in a 640×480 resolution frame image can be traversed to count the bin with the most occurrences of RGB, calculate the peak, and use it as X t .

[0113] BGS t The steps to obtain the parameter are as follows: This parameter records the gradient statistical value of the binary texture of the t-th region. When obtaining it, the pixels in the region need to be binarized first. A threshold (such as 120) can be set, so that pixels with a gray level higher than 120 are marked as 1, and those lower than 120 are marked as 0. Then, the gradient differences are calculated separately in the horizontal and vertical directions, squared and summed, and finally accumulated at all pixel positions within the region range and divided by the total number of pixels to obtain an average gradient value, which is the BGS t . For example, in the contour of a region with an area of 1500 pixels, after binarization, the sum of the horizontal gradients is calculated as 7500, and the sum of the vertical gradients is 6800, then BGS t =(7500 + 6800) / 1500 = 9.2.

[0114] RSD t The steps to obtain the parameter are as follows: This parameter represents the standard deviation of the RGB color histogram of the t-th region. First, calculate the mean μ for each pixel within the region and record the squared deviations of the R, G, and B channels. After summing the squared deviations of all pixels within the region range and dividing by the total number of pixels, finally take the square root to obtain the standard deviation. If the number of pixels in a region is K, and the pixel channel value distribution statistics are {(R k ,G k ,Bk )}, the average value μ of each channel can be calculated first R , μ G , μ B , and then according to the RSD is obtained t .

[0115] The steps to obtain the RC parameter are as follows: This parameter is equal to the total number of regions in the set of potential following target region contours. After region growth and contour extraction are completed, each connected potential target region needs to be marked and numbered, and the maximum value of the final region number is the RC. Example: If 5 effective connected components are identified through the previous gray threshold detection and region growth operations, then RC = 5.

[0116] Calculation process:

[0117] Take an example scenario. After region growth in the image frame, RC = 3 contour regions are obtained, and the following parameters are extracted for these 3 regions:

[0118] X1 = 25, BGS1 = 8.5, RSD1 = 40;

[0119] X2 = 30, BGS2 = 10.0, RSD2 = 36;

[0120] X3 = 42, BGS3 = 6.2, RSD3 = 50;

[0121] First, calculate the numerator:

[0122]

[0123] Among them:

[0124]

[0125] Add the three together:

[0126] 69312.5 + 55656 + 135962.4 = 260930.9;

[0127] Then, calculate the denominator:

[0128]

[0129] Among them:

[0130]

[0131] Add the three together:

[0132] 1625 + 1326 + 2542 = 5493;

[0133] So the overall formula is:

[0134]

[0135] First calculate the internal fraction:

[0136]

[0137] Take the fourth root of 47.52:

[0138]

[0139] The result shows that after comprehensively calculating the main color peak, texture gradient, standard deviation and other indicators of the colors in the 3 detected regions, the obtained W value is about 2.65. The larger the value, the more obvious the change range of the color distribution and texture characteristics within the region. When W exceeds 3, it can be determined that the color gradient and texture dispersion of the region are strong. When it is less than 2, it often indicates that the interior of the region is more uniform. In the subsequent target recognition or region comparison stage, more representative following target regions can be selected according to the high or low W value.

[0140] The steps for obtaining the feature and attribute set of the target are as follows:

[0141] According to the color texture descriptor values of the regions, compare the color texture descriptor values of each predefined following target feature in the predefined following target feature set one by one, calculate the Euclidean distance between the color texture descriptor values, sort the Euclidean distances in ascending order, and select the region with the smallest Euclidean distance to form the target candidate region;

[0142] Based on the target candidate region, count the frequency of occurrence of each pixel's RGB color value in the target candidate region pixel by pixel. After sorting in descending order according to the frequency of occurrence of the color value, select the RGB color value with the highest frequency as the main color of the target candidate region to obtain the main color of the target;

[0143] Based on the target candidate region, perform edge contour detection pixel by pixel and extract the top, bottom, left and right four boundary positions of the edge contour of the target candidate region. Calculate the height and width of the edge contour according to the boundary positions, use the height and width of the edge contour as the spatial scale of the target candidate region, and combine the main color of the target to form the feature and attribute set of the target.

[0144] Specifically, when comparing in sequence according to the acquired regional color texture descriptor values, it is necessary to first read the color texture descriptor value of each target item from the pre-compiled follow-up target feature set, and then construct a corresponding numerical array for the descriptor of each target item as a comparison benchmark, and then calculate the difference between the descriptor value of the current area and each benchmark value one by one and add the squares respectively, and finally take the square root of the sum to obtain the Euclidean distance. In this process, a distance result will be generated for each object to be compared and recorded in the array. The smaller the distance, the closer it is to the color texture representation of the current area. Therefore, it is necessary to sort all distance results from small to large, check the first place of the sorted result and extract the corresponding target item number and distance value, and mark the area represented by the item as a candidate object with higher similarity. In order to ensure that there is no interference due to extreme difference values in the similarity calculation, a difference threshold can be set in the system and distinguished. The threshold is determined by testing in the actual environment. For example, after collecting one hundred frames of indoor lighting scene images, this threshold is determined to be 10. When the comparison result If the distance is greater than 10, it is determined that the region is significantly different from the target feature. If the distance is between 0 and 10, it is within the acceptable range. At this time, the closest feature entry can be determined by arranging them in ascending order. If the minimum distance is less than 3, it is determined that the color and texture features of this region are highly consistent with an entry in the predefined set, so the region number is added to the subsequent candidate list. In addition, if there are multiple entries with similar distances that need to be recorded in parallel, they can all be temporarily stored and screened in subsequent steps. In order to ensure that there are enough test data for accurately measuring the distance size, it is necessary to cover images taken in different scenes, different time periods and under various lighting conditions in the pre-compiled follow-up target feature set, and try to include diverse color and texture distributions. In this way, the comparison results can have a certain degree of universality. After completing the distance calculation and sorting of the feature set and the current region, the region identified by the first item in the corresponding sorting list is regarded as the candidate region number in this round of processing. After recording the candidate region number, the pixel range of the corresponding candidate region is formed and enters the next link.

[0145] When statistically analyzing the RGB color values of each pixel within a target candidate region pixel by pixel, it is necessary to first read all pixel information within the row and column coordinate range of the candidate region, split and store the red channel, green channel, and blue channel data of each pixel in a statistical structure respectively. Subsequently, when performing frequency analysis on this statistical structure, each recorded pixel color value will be scanned and entered into the corresponding frequency counter. For example, when a pixel color combination of R = 128, G = 100, B = 90 is read, the occurrence count of this combination will be incremented by 1 in the record. After all pixels have been processed, these color combinations will be sorted in descending order according to the occurrence count. In actual operation, a reference value can be set to distinguish and eliminate irrelevant combinations with extremely low occurrence counts. For example, multiple frames of images can be tested first and combinations with an occurrence count less than 5 times can be excluded. This threshold of 5 times is obtained through multiple statistics in indoor and outdoor environments. When the counting and sorting of all color combinations are completed, the combination with the highest color occurrence frequency will be selected as the main color. For an image with a resolution of 640×480, if the area of the target candidate region reaches hundreds or thousands of pixels, an obvious main color combination usually appears in the record. At this time, the main color forms a statistical peak in the R, G, and B values. Finally, this main color can be written into a general color identification structure. If a more refined presentation is required, the average values of the three color channels can also be output through different channels while retaining their dispersion, so as to maintain the traceability of the subsequent detection process. After completing these statistical steps, the main color value of the candidate region can be obtained.

[0146] When performing edge contour detection pixel by pixel based on the target candidate region, it is necessary to scan the external coordinates of the candidate region and check the connection between the pixels in this region and the surrounding non-candidate pixels. If it is found that there are neighborhoods outside the candidate range around some pixels, these pixels are marked as boundary pixels. During this process, traversal can be carried out in the four-neighborhood or eight-neighborhood manner, or a method of size comparison can be added to identify those outer edge positions with incomplete connectivity relationships. If a pixel is within the region and immediately goes out of bounds or encounters non-candidate pixels after moving to any adjacent coordinate, this location is marked as an edge point. After statistically analyzing the row and column values of these edge points, the uppermost and lowermost vertical boundaries, as well as the leftmost and rightmost horizontal boundaries, can be obtained. By taking the difference between the uppermost and lowermost vertical coordinates, the overall height of the region can be obtained, and by taking the difference between the leftmost and rightmost horizontal coordinates, the overall width of the region can be obtained. If it is found during reading that the coordinate values of some boundaries have exceeded the set image edge threshold, it means that the candidate region is close to the image edge position, and such situations can be noted in the record. Based on this height and width, the planar scale of this target candidate region can be determined. Combining the main color data obtained from the previous paragraph, these attribute information can be uniformly stored to form a feature and attribute set. In the example, if the row coordinate of the uppermost boundary of a region is 50 and the row coordinate of the lowermost boundary is 200, the height is 200 minus 50, which is 150 pixels. Similarly, if the leftmost coordinate is 30 and the rightmost coordinate is 90 horizontally, the width is 60 pixels. When the area of the region reaches thousands of pixels, a visual statistical chart can be generated internally for verification. If there is a conflict between the feature boundary coordinates and the established shooting frame range, it means that this region may extend beyond the frame. At this time, the main color obtained previously can be used for further comparison.

[0147] The steps for obtaining single-frame tracking performance data points are as follows:

[0148] Based on the feature and attribute set of the target, the color texture descriptor values, main color, and spatial scale contained in the feature and attribute set of the target are extracted one by one, and compared item by item with the color texture descriptor values, main color, and spatial scale of the corresponding position regions in each frame image of the continuous video frames. According to the similarity of the three features, the matching situation of the target position in the continuous video frames is determined, and the continuous-frame target matching result is generated;

[0149] According to the continuous-frame target matching result, it is judged frame by frame whether the target position in the current frame image is successfully matched with the target position in the previous frame image. If it is successfully matched, the number of frames of continuous target tracking is accumulated. If the matching fails, the number of frames of continuous target tracking is reset, and at the same time, the number of target matching failures is recorded to form a record of the continuous target tracking duration and re-identification events;

[0150] Based on the target continuous tracking duration and re-identification event records, count the number of frames for which the target is continuously tracked in a single-frame image and the number of times the target matching fails in a single-frame image. Use the number of continuous tracking frames and the number of target matching failures as evaluation metrics for the single-frame tracking efficiency, and form single-frame tracking efficiency data points.

[0151] Specifically, based on the set of features and attributes of the target, when preparing to perform item-by-item comparison, first read the values of the color texture descriptor, the main color, and the spatial scale from this set in sequence. Then, extract the same three items of data from the corresponding regions in each frame of the consecutive video frames and record them in a list. For operability in comparison, the value range of the color texture descriptor will be set within an interval obtained through actual measurement in the software. For example, an interval of 0 to 50 is obtained through the color texture distribution recorded during multiple shootings. For the main color, it is statistically analyzed by RGB channels. Only the regions with a hue deviation of no more than 20 from the main color of the target are considered similar in the statistical results. When comparing the spatial scale, it is necessary to first check whether the difference between the height and width of this region is within the pre-set threshold range. For example, the threshold range is set with an upper and lower deviation of no more than 5% and a left and right deviation of no more than 5%. This setting can be determined with reference to the statistical results of multiple targets in the previously shot video. After completing the comparison of the three features of each frame with the target, a similarity value is generated frame by frame. Among them, the similarity of the color texture descriptor value can be quantified by dividing the absolute difference by the maximum possible difference. The similarity of the main color can be quantified by statistically analyzing the color offset. The spatial scale similarity can be calculated by the ratio difference of height and width. Finally, the three similarities are weighted and superimposed and compared with another empirical threshold. If the result after comparison is greater than this threshold, it is determined that the matching is successful; otherwise, it is determined that the matching fails. When the similarity remains higher than the threshold for a period of time, it indicates that the target position remains stable in each frame image. The system will centrally store this inter-frame comparison result and form the target matching result of consecutive frames.

[0152] Based on the continuous frame target matching results, each frame of the image is checked to determine whether the target position in the current frame image is consistent with the target position recorded in the previous frame image. In the specific implementation, the markers of successful or failed matching of the two frames of images are analyzed. If the matching is successful, the number of frames for continuous target tracking is incremented by 1. If the matching fails, the number of frames for continuous target tracking is immediately reset to 0. At the same time, the count of the failed matching event is incremented in a special record. The cumulative value of the number of failed matches can be used subsequently to identify situations of discontinuous tracking or tracking loss. For a video with a frame rate of 30 frames per second, when multiple consecutive matching failures occur, the specific time period during which the target information was not correctly compared can be viewed from the timeline. During the setting process, a mismatch threshold is usually configured as 10 frames. If a target cannot be successfully compared within 10 frames of the video, this situation can be marked as a re-identification event and its time point can be recorded. The threshold value is also determined through experimental data in the actual environment. For example, the tracking effect is tested under different indoor and outdoor lighting scenarios and the maximum allowable number of mismatched frames is statistically analyzed. After all frame pairings are counted, the number of frames for continuous target tracking and the number of matching failures can be obtained at each frame position. These data are simultaneously input into a structured form, and finally, the duration of continuous target tracking and the record of re-identification events are summarized.

[0153] Based on the duration of continuous target tracking and the record of re-identification events, the number of frames for continuous target tracking and the number of target matching failures are extracted at the single-frame image dimension. These two pieces of data are regarded as evaluation parameters and uniformly placed into the evaluation logic of single-frame tracking efficiency. Among them, the number of frames for continuous target tracking can be obtained by querying the cumulative value of the current frame. If a target is continuously tracked for 10 frames at a frame rate of 30, it means that the target remains in an identified state within 0.33 seconds. The number of target matching failures can be counted through the records within the same time period. For example, if 6 matching failures occur in the retrieval results of 200 consecutive frames, it means that there are 6 nodes where successful matching cannot be achieved during this period. In software implementation, the number of frames for continuous target tracking and the number of matching failures of a single-frame image are first compared with a range measured experimentally. For example, if the tracking frame number is between 0 and 50, it conforms to the general tracking statistical distribution. If the number of matching failures is between 0 and 10, it is not considered a large anomaly. Conversely, if the number of failures exceeds 10, it is marked as unstable tracking. Then, the filtered single-frame tracking frame number and the number of matching failures are combined into a complete efficiency data point. Subsequently, the evaluation indicators of each frame can be visualized in a chart, and the efficiency data points of different frames are comprehensively sorted to obtain the overall tracking situation of this segment of the video, thus forming single-frame tracking efficiency data points at the single-frame level.

[0154] The steps for obtaining the property association performance metric group are as follows:

[0155] Based on the single-frame tracking performance data points of consecutive multiple frames, extract the number of frames with continuous target tracking and the number of target matching failures in each single-frame tracking performance data point frame by frame. Perform threshold judgments on the number of continuous tracking frames and the number of matching failures respectively. When the number of continuous tracking frames is higher than the preset tracking frame threshold and the number of matching failures is lower than the preset failure number threshold, mark the frame tracking status as effective tracking; otherwise, mark it as ineffective tracking, and generate a target tracking effectiveness status mark.

[0156] According to the target tracking effectiveness status mark, associate the spatial scale and main color included in the target feature and attribute set of the corresponding single-frame image with the corresponding frame tracking effectiveness status respectively, and form attribute association data of the target spatial scale and main color in the effective tracking state.

[0157] Based on the attribute association data of the target spatial scale and main color in the effective tracking state, perform clustering analysis respectively according to the target spatial scale range and the main color numerical type, classify and count the target spatial scale attribute and the target main color attribute, and form an attribute association performance metric group.

[0158] Specifically, based on the single-frame tracking performance data points of consecutive multiple frames, first extract the number of frames for which the target is continuously tracked and the number of times the target matching fails contained in each data point and record them in a sequence. Then, compare the number of frames for which the target is continuously tracked for each data point in this sequence to see if it is greater than or equal to a pre-set frame number threshold. This frame number threshold is obtained from the statistics of multiple test videos. For example, 3000 frames of videos are collected in indoor and outdoor environments respectively, and the minimum frame number requirement for the target to maintain continuous tracking in different scenarios is examined. Finally, an average value plus a certain safety margin is selected as the pre-set frame number threshold. If it is found that the number of frames for which the target is continuously tracked in the current data point is not lower than this value, the first item is judged to be qualified. Then, compare whether the number of times the target matching fails is lower than a pre-set failure number threshold. This failure number threshold can be a reasonable value selected after counting the frequencies of matching anomalies in multiple types of scenarios. For example, if there are more than 15 matching failures in a row for 10 seconds at 30 frames per second under the test conditions, it is considered unstable. Thus, the obtained 15 can be incorporated into the final failure number threshold. When more matching anomaly data are continuously collected and summarized under different lighting and motion conditions, a final failure number threshold under the current system configuration and lighting conditions will be obtained. If the number of failures in the current data point is lower than this threshold, the second item is judged to be qualified. Only when both items are judged to be qualified is the tracking status corresponding to this frame marked as valid. If any one item is not satisfied, it is marked as invalid. Finally, the valid or invalid marks of each frame are added to the corresponding record in chronological order to form the target tracking effectiveness status mark. This enables quick knowledge of which frames are within the effective tracking range and which frames have a high mismatch or too low continuous frame number when looking back, helps to analyze in detail which period of the images has a matching problem in the follow-up, and provides a time series data with status labels for the subsequent steps.

[0159] According to the target tracking validity status flag, it is necessary to read the spatial scale and the main color recorded in the target feature and attribute set from the corresponding single-frame image, and then pair these two pieces of information with the valid or invalid flag of the current frame respectively. During the specific execution process, all the frames on the time axis will be listed sequentially, for example, arranged in numbers 1, 2, 3, etc. Take the height and width dimensions as the spatial scale from the feature and attribute set of each frame, and take the previously recorded RGB values of the main color. These values are already completely stored within a single frame and do not need to be calculated again. Then, add an identifier to the spatial scale and the main color of each frame. For example, mark frame 3 as valid and frame 4 as invalid, form a pair of them and record. In practice, for an image with a resolution of 640×480, the pixel values of the height and width can be directly recorded, or the difference between the top and bottom rows and the difference between the left and right columns can be described in pixels. If it is found that the height or width exceeds a reasonable range when analyzing some frames, the original record can be checked during the analysis. If the RGB values of the main color deviate from some known intervals, it also needs to be verified and compared later. By combining the spatial scale and the main color with the frame marking data in this way of associating with the validity status one by one, the spatial scale and the main color summary of all valid frames and invalid frames will be listed separately in the final output, forming the attribute association data of the target spatial scale and the main color under the tracking valid state.

[0160] Based on the attribute correlation data of the target space scale and the main color in the tracking valid state, cluster the space scales and color values corresponding to all frames determined to be valid. The space scale is first divided according to a certain interval. For example, according to the indoor wheelchair perspective, the height of most targets may be between 50 and 300 pixels, and the width is between 30 and 200 pixels. It can be segmented into several segments to count the number of occurrences in different height and width ranges. If the number of occurrences in a certain segment is particularly concentrated, record its category name and retain it in subsequent statistics. At the same time, similar processing is also carried out on the main color. They can be mapped to a color type table according to the RGB channel values. For example, define those with R>180 and G<100, B<100 as the red color system, or map other channel features to the blue color system, green color system, etc. Store the identified color types together with their occurrence frequencies in the valid frames. After all the data of the valid frames are completed with this differentiation, classification and summarization can be carried out for each category combination. For example, count the number of frames with a height in the range of 50 to 100 and a red color system and form a set, and then count the number of frames with a height in the range of 100 to 150 and a blue color system into another set. After such item-by-item induction, the classification results of the space scale and the main color can be crossed together to form an attribute correlation performance metric group. Finally, horizontal comparison can be carried out between different clustering categories, or further check whether the category combinations with more occurrences indicate common target forms, so as to call the summary formed after classification statistics the attribute correlation performance metric group and output it to the analysis of the next stage.

[0161] The steps for obtaining the adjusted processing parameter configuration are as follows:

[0162] Based on the attribute correlation performance metric group, respectively count the occurrence frequencies of all target space scale classifications and the occurrence frequencies of the main color classifications within each attribute correlation performance metric group, and form an attribute category frequency set for each attribute correlation performance metric group;

[0163] According to the attribute category frequency set, calculate the group performance difference metric value between different attribute correlation performance metric groups. The calculation formula is:

[0164]

[0165] Among them, F is the group performance difference metric value, FPA a is the occurrence frequency of the a-th attribute category in the first attribute correlation performance metric group, V a is the occurrence frequency of the a-th attribute category in the second attribute correlation performance metric group, and A is the total number of attribute categories in the attribute category frequency set;

[0166] Based on the group performance difference metric value, compare the group performance difference metric value with a preset threshold. When the group performance difference metric value exceeds the preset threshold, update the feature fusion weight; otherwise, update the sample weighting factor to obtain an adjusted processing parameter configuration.

[0167] Specifically, when counting the occurrences of each classification based on the target space scale classification and the main color classification contained in the attribute association performance metric group, it is necessary to first establish several classification labels according to a predetermined classification scheme. For the space scale part, the height and width values can be divided into intervals in ascending order, such as dividing the height into intervals of 50 to 100, 101 to 150, 151 to 200, etc., and screening each frame in each interval, accumulating the number of targets that meet the range into the corresponding classification count. Similarly, for the main color aspect, it is necessary to set several color type ranges or color system identifiers for the R, G, and B channels. For example, define R>180 and G<100, B<100 as the red color system, etc. For the target records in all valid frames, compare whether the main color of each target falls into the defined interval of a certain color system and count. When the statistics of both the space scale and the main color are completed respectively, a list of occurrence frequencies aggregated according to each classification label can be obtained. To ensure data integrity, all possible classification labels need to be processed. If some frame data cannot be matched to the established classification during the statistics process, it should also be recorded and confirmed during subsequent processing. Finally, after completing the scanning of all frames and the calculation of classification frequencies, an attribute category frequency set for different attribute association performance metric groups can be formed, and then these results can be summarized and the absolute number and relative frequency of each classification occurrence can be marked.

[0168] Formula: The benefit of the formula is that by comprehensively comparing the occurrence frequencies of the a-th attribute category in two different attribute association performance metric groups and introducing (FPA a +V a ) and the denominator (1 + |FPA a -V a |) modulation, it is possible to take into account the overall level of category distribution while measuring the difference, and obtain an index for measuring the "group performance" difference among multiple target attribute classification results, which is used to guide the subsequent update of feature fusion or weighting factors.

[0169] FPA aThe steps to obtain the parameter are as follows: This parameter refers to the occurrence frequency of the a-th attribute category within the first attribute correlation performance metric group, where the attribute categories can be composed of different combinations of spatial scale and main color, such as "small - range red", "medium - range blue", etc. In specific operations, it is necessary to first extract the spatial scale interval and main color interval according to each frame of valid target record in the previous steps, and then accumulate the number of occurrences of each combination category and organize it into a frequency table. For example, after observing 1000 frames of video, it is statistically obtained that the "small - range red" category appears 125 times, the "small - range blue" category appears 88 times, etc., so that each category has an occurrence frequency.

[0170] V a The steps to obtain the parameter are as follows: This parameter represents the occurrence frequency of the a-th attribute category within the second attribute correlation performance metric group, and its acquisition method is the same as that of FPA a Same. By statistically analyzing according to the aforementioned category division method, another set of V a can be obtained. This V a sequence needs to correspond one - to - one with FPA a to ensure that the category order remains consistent. Thus, item - by - item comparison can be made in the formula. For example, for the category of "small - range red", it is statistically 125 times in metric group A and 100 times in metric group B, then FPA a

[0171] = 125, and V a = 100.

[0172] The steps to obtain parameter A are as follows: This parameter is the total number of attribute categories in the attribute category frequency set. During use, it is necessary to first confirm the total number of categories divided by spatial scale and main color. For example, there are three levels of height: small - range (50 to 100), medium - range (101 to 150), and large - range (151 to 200), and combined with multiple colors such as red, blue, and green, there are about a dozen categories. If other colors are further expanded, it may reach dozens. After collecting all frame data and completing classification, the final total number of categories A can be obtained. When calculating the formula, it is necessary to ensure that A matches the definitions of FPA a and V a one - to - one. If 12 combinations are generated under the system configuration, then A = 12.

[0173] Calculation process:

[0174] Now select a scenario example, let A = 5, indicating that there are 5 attribute categories in total after completing the combination of spatial scale and main color. First, statistically analyze the occurrence frequencies of each category in metric group 1 (denoted as G1) as follows:

[0175]

[0176] Then count the frequency of occurrence V of each category in the metric group 2 (denoted as G2). a :

[0177] V1 = 32, V2 = 30, V3 = 40, V4 = 16, V5 = 20;

[0178] Substitute into the numerator part of the formula and calculate term by term

[0179] For a = 1:

[0180]

[0181] (-12) 2 ×52 = 144×52 = 7488;

[0182] Denominator = 1 + 12 = 13

[0183] This term = 7488÷13≈576.0

[0184] For a = 2:

[0185]

[0186] 5 2 ×65 = 25×65 = 1625;

[0187] Denominator = 1 + 5 = 6

[0188] This term = 1625÷6≈270.83

[0189] For a = 3:

[0190]

[0191] 0 2 ×80 = 0;

[0192] Denominator = 1 + 0 = 1

[0193] This term = 0÷1 = 0

[0194] For a = 4:

[0195]

[0196] (-6) 2 ×26 = 36×26 = 936;

[0197] Denominator = 1 + 6 = 7

[0198] This term = 936÷7≈133.71

[0199] For a = 5:

[0200]

[0201] 5 2 ×45 = 25×45 = 1125;

[0202] Denominator = 1 + 5 = 6

[0203] This term = 1125÷6 = 187.5

[0204] Adding the above 5 terms gives ∑ = 576.0 + 270.83 + 0 + 133.71 + 187.5 = 1168.04.

[0205] Then find the square root:

[0206]

[0207] Divide it by A = 5:

[0208]

[0209] The result shows that when comparing these two metric groups G1 and G2, the F value obtained is approximately 6.836. The larger the value, the greater the difference in the frequency distribution of attribute categories between the two groups. If there is a corresponding difference threshold set in the system, once F exceeds this threshold, it can trigger the update of the subsequent feature fusion weights or the change of the sample weighting factor, so that the subsequent processing parameter configuration can be readjusted.

[0210] When comparing based on the group performance difference metric value F, a record table can be established for each metric group in the above statistical process to collect the frequency data of attribute categories in different scenarios. For example, when collecting 2000 frames during indoor sports, the spatial scale data in the X - Y interval appears 300 times, and a certain color combination appears 120 times. When collecting another 3000 frames during outdoor sports, the occurrence times of the same category are corresponded one by one. After completion, an integrated category comparison table can be obtained. Then, calculate the occurrence frequency of the same category and substitute it into the formula F item by item to obtain the group performance difference metric value. When the comparison result is at the control threshold, an empirical threshold determined by the actual test records will be retrieved. This threshold comes from the statistical values of the difference metric in multiple scenarios, multiple resolutions, and multiple illuminations in the early stage, and selects the value that can best distinguish the difference in a certain interval. If the measured F result is higher than this threshold, immediately prepare to execute the update of the feature fusion weights. If F does not exceed it, consider modifying the sample weighting factor. During the operation, the existing weights and weighting factors will be read first and numerically adjusted. For example, the coefficient range of each channel is linearly distributed from 0 to 1. When an update is required, select the next - level increment or decrement to adjust the specified parameter, and make the new weighting factor effective through multiple iterations. Finally, form the adjusted processing parameter configuration and store this configuration in the system parameter set.

[0211] The above are only the preferred embodiments of the present invention, and do not limit the present invention in other forms. Any person skilled in the art may use the technical content disclosed above to make changes or modifications into equivalent embodiments with equivalent changes and apply them to other fields. However, as long as it does not depart from the technical solution content of the present invention, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention still belong to the protection scope of the technical solution of the present invention.

Claims

1. A wheelchair following target recognition method based on deep learning, characterized in that It includes the following steps: Based on the original video frames from the wheelchair perspective, calculate the gradient statistics and color contrast values of each pixel neighborhood to obtain the contrast activity value of the local area; based on the contrast activity value of the local area, adjust the spatial coefficient of Gaussian filtering and the scaling factor of the reflection component, and combine them with the illumination component to obtain the illumination-compensated image frame; Based on the illumination-compensated image frame, segment the potential following target area and calculate the color texture descriptor value of the area; According to the color texture descriptor value, compare with the predefined following target features to determine the target position, estimate the main color and spatial scale, and form the feature and attribute set of the target; Based on the feature and attribute set of the target, compare the descriptors of consecutive frames to update the tracking trajectory, record the target continuous tracking duration and re-identification events, and generate single-frame tracking efficiency data points; Based on the single-frame tracking efficiency data points of consecutive multiple frames, associate the attribute values in the feature and attribute set of the target and perform attribute value classification to form an attribute association performance metric group; Based on the attribute association performance metric group, calculate the group performance difference metric value of the performance metrics between different groups, and update the feature fusion weight or sample weighting factor through the comparison result between the group performance difference metric value and the preset threshold, and construct an adjusted processing parameter configuration.

2. The method for wheelchair following target recognition based on deep learning according to claim 1, wherein The steps for obtaining the contrast activity value are as follows: Based on the original video frames from the wheelchair perspective, extract the grayscale values within the pixel neighborhood centered on the current pixel pixel by pixel, calculate the gradient differences in the horizontal and vertical directions of the pixels in the neighborhood through the Sobel operator and perform a square operation, and sum the squared gradient values to obtain the gradient statistics value of the pixel neighborhood; Based on the gradient statistics value of the pixel neighborhood, calculate the Euclidean distance between the pixel and the color values of the surrounding 8 adjacent pixels in the RGB color space pixel by pixel, and take the sum of all Euclidean distances to obtain the color contrast value of the current pixel neighborhood; Based on the color contrast value of the current pixel neighborhood, calculate the contrast activity value of the local area pixel by pixel.

3. The method for wheelchair following target recognition based on deep learning according to claim 1, wherein The steps for obtaining the illumination-compensated image frame are as follows: Based on the contrast activity value of the local area, statistically analyze the distribution of the contrast activity values of the local areas at all pixel positions in the current image frame, extract the pixel positions where the contrast activity value is higher than the average contrast activity value of all pixel positions in the image frame, and statistically analyze the proportion of the pixel positions to obtain the high-activity area proportion value; Based on the high-activity area proportion value, calculate the spatial coefficient of Gaussian filtering; According to the spatial coefficient of Gaussian filtering, adjust the scaling factor of the reflection component at the corresponding position in the image frame pixel by pixel, and perform pixel-by-pixel fusion of the adjusted reflection component and the illumination component at the corresponding position to obtain the illumination-compensated image frame.

4. The method for wheelchair following target recognition based on deep learning according to claim 1, characterized in that, The steps for obtaining the color texture descriptor value are as follows: Based on the illumination-compensated image frame, perform grayscale threshold detection on the image frame pixel by pixel using the region growing method, extract the boundary contours of the grayscale continuous regions, and generate a set of potential following target area contours; According to the set of potential contour regions of the following target, extract the RGB color histogram and local binary texture features inside each contour region, and merge them to form the color texture feature vector of the region; Based on the color texture feature vector of the region, calculate the color texture descriptor value of the region.

5. The method for wheelchair following target recognition based on deep learning according to claim 1, wherein, The steps for obtaining the set of features and attributes of the target are as follows: According to the color texture descriptor value of the region, compare one by one the color texture descriptor values of each predefined following target feature in the predefined following target feature set, calculate the Euclidean distance between the color texture descriptor values, sort the Euclidean distances in ascending order, and select the region with the smallest Euclidean distance to form the target candidate region; Based on the target candidate region, count the frequency of occurrence of each pixel RGB color value in the target candidate region pixel by pixel. After sorting in descending order according to the frequency of occurrence of the color value, select the RGB color value with the highest frequency as the main color of the target candidate region to obtain the main color of the target; Based on the target candidate region, perform edge contour detection pixel by pixel and extract the top, bottom, left, and right four boundary positions of the edge contour of the target candidate region. Calculate the height and width of the edge contour according to the boundary positions, use the height and width of the edge contour as the spatial scale of the target candidate region, and combine the main color of the target to form the set of features and attributes of the target.

6. The method for wheelchair following target recognition based on deep learning according to claim 1, wherein, The steps for obtaining the single-frame tracking efficiency data points are as follows: Based on the set of features and attributes of the target, extract one by one the color texture descriptor value, main color, and spatial scale included in the set of features and attributes of the target, and compare them item by item with the color texture descriptor value, main color, and spatial scale of the corresponding position region in each frame image of the continuous video frames. Determine the matching situation of the target position in the continuous video frames according to the similarity of the three features, and generate the continuous-frame target matching result; According to the continuous-frame target matching result, judge frame by frame whether the target position in the current frame image is successfully matched with the target position in the previous frame image. If it is successfully matched, accumulate the number of frames of continuous target tracking. If the matching fails, reset the number of frames of continuous target tracking, and at the same time record the number of target matching failures to form the record of target continuous tracking duration and re-identification events; Based on the record of target continuous tracking duration and re-identification events, count the number of frames of continuous target tracking in a single-frame image and the number of target matching failures in a single-frame image. Use the number of continuous tracking frames and the number of target matching failures as the evaluation indicators of single-frame tracking efficiency to form the single-frame tracking efficiency data points.

7. The method for wheelchair following target recognition based on deep learning according to claim 1, wherein The steps for obtaining the set of property association performance metrics are as follows: Based on the single-frame tracking efficiency data points of multiple continuous frames, extract the number of frames of continuous target tracking and the number of target matching failures in each single-frame tracking efficiency data point frame by frame. Perform threshold judgment on the number of continuous tracking frames and the number of matching failures respectively. When the number of continuous tracking frames is higher than the preset tracking frame threshold and the number of matching failures is lower than the preset failure threshold, mark the frame tracking status as effective tracking, otherwise mark it as ineffective tracking, and generate the target tracking effectiveness status mark; According to the target tracking effectiveness status flag, associate the target features of the corresponding single-frame image with the spatial scale and main color included in the attribute set respectively with the tracking effectiveness status of the corresponding frame, and form the attribute association data of the target spatial scale and main color under the tracking effective state; Based on the attribute association data of the target spatial scale and main color under the tracking effective state, perform clustering analysis respectively according to the target spatial scale range and the main color numerical type, classify and count the target spatial scale attribute and the target main color attribute, and form an attribute association performance metric group.

8. The method for wheelchair following target recognition based on deep learning according to claim 1, characterized in that, The obtaining step of the adjusted processing parameter configuration is as follows: Based on the attribute association performance metric group, count the occurrence frequencies of all target spatial scale classifications and the occurrence frequencies of main color classifications within each attribute association performance metric group respectively, and form an attribute category frequency set of each attribute association performance metric group; According to the attribute category frequency set, calculate the group performance difference metric value between different attribute association performance metric groups; Based on the group performance difference metric value, compare the group performance difference metric value with a preset threshold. When the group performance difference metric value exceeds the preset threshold, update the feature fusion weight, otherwise update the sample weighting factor, and obtain the adjusted processing parameter configuration.

Images