A method and system for real-time identification and data processing of camera surveillance video.

Abstract

This invention provides a real-time recognition data processing method and system for camera surveillance videos, relating to the fields of computer vision and intelligent monitoring technology. The method includes: extracting the current video frame from a real-time video frame sequence based on a multi-dimensional environmental state descriptor and performing adaptive scene partitioning to obtain the confidence weights of each region; using the confidence weights of each region as input, performing joint nonlinear calibration on the core parameters of the target detection network through a lightweight neural network model to obtain an optimized detection parameter set; and using the optimized detection parameter set, performing multi-target synchronous recognition and deep feature abstraction on the current video frame to obtain a set of virtual feature contours including personnel and vehicles. This invention achieves a complete intelligent processing link from complex scene perception to global situational understanding and closed-loop control through environmental adaptive detection and deep feature abstraction, accurate identity verification, abnormal behavior recognition and early warning, and cross-domain trajectory association and collaborative response.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and intelligent monitoring technology, and in particular to a method and system for real-time identification and data processing of camera surveillance video. Background Technology

[0002] With the advancement of smart city construction, the intelligentization of video surveillance systems has become a development trend. In complex scenarios such as entrances and exits of smart parks, the system needs to perform real-time and accurate identification, identity verification, and behavior analysis of multiple targets such as personnel and vehicles to achieve proactive early warning and coordinated response. However, existing real-time identification data processing methods have shortcomings at the data processing level in practical applications, especially in dynamic and high-concurrency environments, which restricts system efficiency.

[0003] The shortcomings of existing technologies are specifically reflected in the entire data processing chain: First, insufficient environmental adaptability. The target detection model using fixed parameters cannot be dynamically adjusted according to the real-time environment. As a result, under conditions such as morning rush hour backlight, rainy or foggy weather, or nighttime, the accuracy of face and license plate recognition may plummet from over 95% to around 70%, and errors accumulate in subsequent processes. Second, resource scheduling and real-time performance deficiencies. During high-concurrency periods, the system performs equal calculations on all areas and targets within the video frame without prioritization scheduling. This may cause the overall processing latency to soar from the required 300ms to over 800ms, resulting in delayed gate response and warnings. Third, lack of data correlation. The algorithm only processes single video streams and cannot correlate and stitch the trajectory of the same target from different cameras based on spatiotemporal information. This makes cross-regional behavior analysis impossible, and the movement of targets between cameras creates monitoring blind spots. Finally, the storage strategy is inefficient. The system stores all raw video streams (daily data volume can reach several TB) and a small amount of key structured data equally. This may not only increase storage costs but also make it difficult to quickly locate effective information during post-event tracing. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method and system for real-time identification and data processing of camera surveillance video. Through environmental adaptive detection and deep feature abstraction, accurate identity verification, abnormal behavior identification and early warning, and cross-domain trajectory association and collaborative response, a complete intelligent processing link is realized from complex scene perception to global situation understanding and closed-loop management.

[0005] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0006] A first aspect includes a method for real-time identification data processing of camera surveillance video, the method comprising:

[0007] Based on a multi-dimensional environment state descriptor, the current video frame is extracted from the real-time video frame sequence and adaptive scene partitioning is performed to obtain the confidence weight of each region. Using the confidence weight of each region as input, the core parameters of the target detection network are jointly nonlinearly calibrated through a lightweight neural network model to obtain an optimized detection parameter set. Using the optimized detection parameter set, multi-target synchronous recognition and deep feature abstraction are performed on the current video frame to obtain a set of virtual feature contours containing people and vehicles.

[0008] Hierarchical feature parsing is performed on each contour in the virtual feature contour set to obtain a virtual feature vector; the virtual feature vector is then matched in real time with a preset virtual identity template library to obtain an identity matching vector;

[0009] Based on the identity matching vector, multi-source data fusion is performed on all target information within the same video frame to obtain a structured state vector containing virtual identity, status, and spatiotemporal coordinates; real-time pattern analysis is performed on the structured state vector to identify abnormal behavior patterns and generate early warning signals.

[0010] Based on the structured state vector and warning signal, virtual trajectory association and behavior chain reconstruction are performed across cameras to obtain a comprehensive behavior profile. Based on the comprehensive behavior profile, differentiated storage and linkage response are performed on all data generated during the processing that are associated with the target.

[0011] Secondly, a real-time recognition data processing system for camera surveillance video includes:

[0012] The target detection module is used to extract the current video frame from the real-time video frame sequence based on a multi-dimensional environment state descriptor and perform adaptive scene partitioning to obtain the confidence weight of each region. Using the confidence weight of each region as input, a lightweight neural network model is used to perform joint nonlinear calibration on the core parameters of the target detection network to obtain an optimized detection parameter set. Using the optimized detection parameter set, multi-target synchronous recognition and deep feature abstraction are performed on the current video frame to obtain a set of virtual feature contours containing people and vehicles.

[0013] The identity matching module is used to perform hierarchical feature parsing on each contour in the virtual feature contour set to obtain a virtual feature vector; and to perform real-time matching between the virtual feature vector and a preset virtual identity template library to obtain an identity matching vector.

[0014] The early warning module is used to perform multi-source data fusion on all target information within the same video frame based on identity matching vectors to obtain a structured state vector containing virtual identity, status, and spatiotemporal coordinates; and to perform real-time pattern analysis on the structured state vector to identify abnormal behavior patterns and generate early warning signals.

[0015] The response module is used to perform virtual trajectory association and behavior chain reconstruction across cameras based on the structured state vector and warning signal to obtain a comprehensive behavior profile; based on the comprehensive behavior profile, it performs differentiated storage and linkage response on all data generated during the processing that are associated with the target.

[0016] Thirdly, a computing device, comprising:

[0017] One or more processors;

[0018] A storage device for storing one or more programs that, when executed by one or more processors, cause the one or more processors to implement the method.

[0019] Fourthly, a computer-readable storage medium storing a program that, when executed by a processor, implements the method.

[0020] The above-described solution of the present invention has at least the following beneficial effects:

[0021] This system achieves adaptive scene partitioning to adapt to real-time environmental changes; obtains confidence weights for each region to ensure targeted parameter calibration; balances computational power consumption and parameter calibration effectiveness through a lightweight neural network model; optimizes the detection parameter set to improve the environmental adaptability of simultaneous multi-target recognition; deep feature abstraction preserves the core feature information of the targets, providing a high-quality data foundation for subsequent processing; hierarchical feature parsing improves the effectiveness and accuracy of virtual feature vectors; establishes association and matching between virtual feature vectors and identity templates to determine target identity information; generates standardized identity matching vectors to achieve accurate correspondence between feature information and identity information, providing a reliable identity basis for multi-source data fusion; and integrates multi-dimensional information of multiple targets within the same video frame through multi-source data fusion. The system generates structured state vectors to standardize data formats, improving data regularity and usability; real-time pattern analysis enables effective identification of abnormal behavior patterns; standardized early warning signals integrate key abnormal information, providing clear guidance for subsequent trajectory association and response processing; cross-camera virtual trajectory association enables trajectory connectivity across multiple monitoring areas, fully covering the target's activity range; behavior chain reconstruction restores the target's continuous activity process, clearly presenting the behavioral evolution; comprehensive behavior archives integrate trajectory and behavior information to form a comprehensive target data representation; differentiated storage optimizes resource allocation, improving data storage and retrieval efficiency; and coordinated response enables precise connection between data processing results and handling actions, ensuring the targeted and efficient nature of the processing flow. Attached Figure Description

[0022] Figure 1 This is a flowchart illustrating a real-time identification data processing method for camera surveillance video provided in an embodiment of the present invention.

[0023] Figure 2 This is a schematic diagram of a real-time recognition data processing system for camera surveillance video provided in an embodiment of the present invention. Detailed Implementation

[0024] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0025] like Figure 1 As shown, an embodiment of the present invention proposes a method for real-time identification data processing of camera surveillance video, the method comprising the following steps:

[0026] Step 100: Based on the multi-dimensional environment state descriptor, extract the current video frame from the real-time video frame sequence and perform adaptive scene partitioning to obtain the confidence weight of each region; using the confidence weight of each region as input, perform joint nonlinear calibration on the core parameters of the target detection network through a lightweight neural network model to obtain an optimized detection parameter set; using the optimized detection parameter set, perform multi-target synchronous recognition and deep feature abstraction on the current video frame to obtain a set of virtual feature contours containing people and vehicles;

[0027] Step 200: Perform hierarchical feature parsing on each contour in the virtual feature contour set to obtain a virtual feature vector; perform real-time matching of the virtual feature vector with a preset virtual identity template library to obtain an identity matching vector;

[0028] Step 300: Based on the identity matching vector, multi-source data fusion is performed on all target information within the same video frame to obtain a structured state vector containing virtual identity, status, and spatiotemporal coordinates; real-time pattern analysis is performed on the structured state vector to identify abnormal behavior patterns and generate early warning signals.

[0029] Step 400: Based on the structured state vector and the warning signal, perform virtual trajectory association and behavior chain reconstruction across cameras to obtain a comprehensive behavior profile; based on the comprehensive behavior profile, perform differentiated storage and linkage response on all data generated during the processing that are associated with the target.

[0030] In this embodiment of the invention, scene partitioning is achieved by adapting to multi-dimensional environmental states, improving the scene adaptability of regional processing; detection parameters are optimized through core parameters combined with nonlinear calibration, enhancing the adaptability of detection parameters to the current environment; multi-target synchronous recognition and deep feature abstraction can efficiently obtain virtual feature contours of personnel and vehicles, ensuring the integrity of basic data for subsequent processing; hierarchical feature parsing can comprehensively mine contour feature information, improving the integrity of virtual feature vectors; virtual feature vectors are matched with a pre-set template library in real time, improving the efficiency of identity association; ensuring the correspondence between identity matching vectors and targets; multi-source data fusion integrates multi-target information within the same video frame, improving the information comprehensiveness of structured state vectors; real-time pattern analysis enables timely identification of abnormal behavior, ensuring the timeliness of early warning signal generation; structured state vectors organize target information, facilitating subsequent data processing and analysis; cross-camera virtual trajectory association and behavior chain reconstruction can completely connect the target behavior process, improving the integrity of comprehensive behavior archives; differentiated storage rationally allocates storage resources, improving storage utilization efficiency; and linkage response enables efficient connection between data processing results and subsequent applications, ensuring the application effectiveness of processed data.

[0031] In a preferred embodiment of the present invention, before step 100, the method further includes:

[0032] The system adaptively receives and schedules the raw video streams captured by cameras to obtain real-time video frame sequences. It then performs environmental state analysis on these sequences, including at least light intensity abrupt change detection, background motion vector analysis, and rain, snow, and fog weather classification, to obtain corresponding environmental feature scalars. These environmental feature scalars are then fused into a multi-dimensional environmental state descriptor. Specifically, this involves: first, constructing an adaptive receiving module adapted to various camera transmission protocols. This module pre-stores transmission protocol parameters for different models of high-definition network cameras deployed at the smart park entrances and exits, including but not limited to adaptation parameters for RTSP real-time streaming protocol and HTTP streaming media protocol. During system operation, the adaptive receiving module actively establishes communication connections with each camera, real-time detecting link status parameters such as network bandwidth, data transmission rate, and packet loss rate. Finally, it addresses data processing latency issues in high-concurrency scenarios. To address the issue of excessive bandwidth, dynamic scheduling is implemented on the raw video stream based on detected link status parameters and preset video frame reception priority rules. The priority rule prioritizes receiving video streams from the main entrance panoramic camera, followed by video streams from various close-up cameras. Simultaneously, the receiving bitrate of the video stream is dynamically adjusted according to link bandwidth. When insufficient bandwidth is detected, the bitrate of the video stream from non-critical area cameras is appropriately reduced, while ensuring the video frame clarity meets recognition requirements, to avoid video frame loss or reception delays due to bandwidth congestion. Subsequently, frame extraction processing is performed on the received raw video stream. Complete video frames are extracted from the continuous video stream according to preset frame rate requirements (e.g., 30 frames / second), removing blurry or incomplete frames caused by transmission jitter. The filtered valid video frames are arranged in chronological order to form a real-time video frame sequence, providing continuous and effective data input for subsequent environmental status analysis and target recognition.

[0033] Based on the aforementioned real-time video frame sequence, an environmental state analysis process is initiated. This process utilizes three parallel analysis units to perform illumination intensity abrupt change detection, background motion vector analysis, and rain, snow, and fog weather classification, respectively. The specific implementation process of each analysis unit is as follows: Regarding illumination intensity abrupt change detection, multiple consecutive video frames in the real-time video frame sequence are selected as the detection window. The global pixel brightness mean of each video frame within the detection window is calculated. The difference between the global pixel brightness mean of the current frame and the global pixel brightness mean of the previous frame is calculated to obtain the brightness difference. Simultaneously, the standard deviation of the brightness difference between all adjacent frames within the detection window is calculated. When the brightness difference between the current frame and the previous frame exceeds a preset brightness threshold, and the brightness difference is greater than a preset multiple of the standard deviation of the brightness difference between adjacent frames within the detection window, an illumination intensity abrupt change is determined to exist. The brightness difference is then quantized to obtain an illumination intensity abrupt change quantization value, which is used to characterize the intensity level of the illumination abrupt change. If no illumination intensity abrupt change is detected, the illumination intensity abrupt change quantization value is set to zero.

[0034] Regarding background motion vector analysis, the background is first modeled for the current frame and the previous frame in the real-time video frame sequence. The motion region between the two frames is extracted using the frame difference method. Pre-defined fixed background regions, such as the regions corresponding to stationary objects like walls and gates at entrances and exits, are excluded to obtain the dynamic background region. Subsequently, the dynamic background region is divided into multiple sub-regions of preset size, and the pixel position offset of each sub-region in the current frame and the previous frame is calculated. This offset is used as the motion vector. The distribution of motion vectors in all sub-regions is statistically analyzed to obtain the background motion vector distribution feature, which is used to characterize the overall motion activity of the background in the current scene.

[0035] Regarding rain, snow, and fog weather classification, image features of real-time video frames are extracted, including global contrast, edge sharpness, and pixel grayscale value distribution variance. These extracted image features are compared with preset weather template features, which include standard image feature parameters corresponding to sunny, rainy, snowy, and foggy days. Based on the comparison results, the weather type corresponding to the current video frame is determined, generating a rain, snow, and fog weather classification label. If the weather is determined to be sunny, the classification label is set to 0; if it is determined to be rainy, snowy, or foggy, the classification label is set to 1, 2, or 3 respectively. This classification label is used to clarify whether rain, snow, or fog interference exists in the current environment and the type of interference. Through the collaborative work of the above three analysis units, the quantized value of light intensity abrupt change, background motion vector distribution characteristics, and rain, snow, and fog weather classification label are obtained respectively. These three are used as independent environmental feature scalars to complete the extraction of environmental feature scalars.

[0036] To achieve a comprehensive and integrated representation of the current environmental state, the aforementioned multiple environmental feature scalars need to be fused to form a multidimensional environmental state descriptor. Specifically, firstly, a descriptor fusion framework is constructed. This framework predetermines the fusion order of the environmental feature scalars, incorporating them sequentially according to the quantization value of sudden changes in illumination intensity, the distribution characteristics of background motion vectors, and the classification identifiers for rain, snow, and fog. Subsequently, each environmental feature scalar is standardized, mapping environmental feature scalars with different dimensions and numerical ranges to a predefined numerical interval, avoiding fusion deviations caused by differences in the inherent properties of the feature scalars. Based on this, the standardized environmental feature scalars are integrated into a structured feature set through feature concatenation. This feature set is the multidimensional environmental state descriptor. This multidimensional environmental state descriptor can comprehensively cover key environmental information such as changes in illumination, background motion, and weather interference in the current scene, providing complete and accurate environmental data support for subsequent adaptive scene partitioning and optimization of target detection parameters based on this descriptor.

[0037] In a preferred embodiment of the present invention, step 100 above involves extracting the current video frame from the real-time video frame sequence based on a multi-dimensional environment state descriptor and performing adaptive scene partitioning to obtain the confidence weights of each region; using the confidence weights of each region as input, performing joint nonlinear calibration on the core parameters of the target detection network through a lightweight neural network model to obtain an optimized detection parameter set; and using the optimized detection parameter set, performing multi-target synchronous recognition and deep feature abstraction on the current video frame to obtain a set of virtual feature contours containing personnel and vehicles, including:

[0038] Step 101: Based on the multi-dimensional environment state descriptor, extract the current video frame from the real-time video frame sequence. Specifically, this includes: first, retrieving the multi-dimensional environment state descriptor generated in the aforementioned preprocessing stage, extracting the timestamp information contained in the descriptor, and aligning and matching it with the timeline of the real-time video frame sequence; based on the alignment result, extracting a single frame image corresponding to the timestamp from the real-time video frame sequence as the current video frame; simultaneously, combining the relevant information representing environmental stability in the multi-dimensional environment state descriptor, performing a preliminary validity check on the extracted current video frame, eliminating blurry or incomplete frames caused by transmission jitter or environmental changes, ensuring that the extracted current video frame can truly reflect the scene state under the current environment, laying the foundation for subsequent scene partitioning based on environmental features.

[0039] Step 102 involves parsing the multidimensional environmental state descriptor to obtain its contained light intensity mutation quantization value, background motion vector distribution features, and rain, snow, and fog weather classification identifier. Specifically, this includes: First, according to the parsing rules, extracting the three core environmental features encapsulated in the multidimensional environmental state descriptor: light intensity mutation quantization value, background motion vector distribution features, and rain, snow, and fog weather classification identifier. During the extraction process, the format of each feature is simultaneously standardized, converting the light intensity mutation quantization value into a numerical form of uniform magnitude, standardizing the background motion vector distribution features into matrix-based distribution data, and converting the rain, snow, and fog weather classification identifier into a standardized category code, ensuring that the parsed environmental features can directly adapt to the calculation requirements of subsequent partitioning logic.

[0040] Step 103: Based on the quantized value of light intensity mutation, divide the current video frame into high dynamic range (HMR) and stable light regions. Specifically, this includes: first, comparing the quantized value of light intensity mutation with a preset light mutation threshold to determine the set of pixels in the current video frame whose quantized value exceeds the threshold, thus initially defining the approximate range of the light mutation region; then, using an image segmentation algorithm, based on the quantized value of light intensity mutation, initially dividing the HMR and stable light regions into preliminary shapes. The region with a quantized value of light intensity mutation greater than the preset threshold is the HMR region, encompassing unstable lighting areas such as backlighting and direct strong light; the remaining regions are the stable light regions. Based on this, a polygon region fitting algorithm is introduced for boundary correction. Specifically, the edge pixels of the HMR and stable light region prototypes are first extracted, and the extracted edge pixels are then processed... The process involves denoising and filtering to remove discrete outliers caused by image noise, retaining only valid edge points that accurately reflect the region boundaries. Next, polygon fitting calculations are performed based on these valid edge points. By traversing each valid edge point and assessing the rationality of connections between adjacent points, a polygon fitting model that closely matches the actual edge contours of the region is gradually constructed. After fitting, the model's effectiveness is validated to determine whether the fitted polygonal region completely covers the actual illumination abrupt change area without excessive redundancy. If the validation passes, the vertex coordinates of the polygon are used as the corrected region boundary coordinates. If the validation fails, the fitting parameters are readjusted and the fitting calculation is performed again. Finally, based on the corrected boundary coordinates, the precise ranges of the high dynamic illumination region and the stable illumination region are determined, the polygonal boundary vertex sequences of the two regions are identified, and the corresponding coordinate information is recorded to form the corrected illumination partitioning results, providing a basic framework for further refinement of the partitioning.

[0041] Step 104: Combining the background motion vector distribution characteristics, further define high-motion-activity sub-regions and static background sub-regions within the high-dynamic illumination region and the stable illumination region. Specifically, this includes: first, retrieving the background motion vector distribution characteristics and spatially superimposing them with the boundary coordinates of the high-dynamic illumination region and the stable illumination region obtained in step 103; for each superimposed illumination region, statistically analyzing the motion vector amplitude and direction distribution of each sub-pixel block within the region, and setting a motion activity threshold; aggregating sub-pixel blocks with motion vector amplitudes greater than the threshold to form high-motion-activity sub-regions, and aggregating sub-pixel blocks with motion vector amplitudes less than or equal to the threshold to form static background sub-regions; through the above method, the definition of high-motion-activity sub-regions and static background sub-regions is completed within both the high-dynamic illumination region and the stable illumination region, and the boundary coordinate information of each sub-region is updated synchronously, so that the partitioning results can simultaneously adapt to the illumination state and the background motion state.

[0042] Step 105: Based on the rain, snow, and fog weather classification identifiers, adaptively adjust the boundary divisions and relative priorities of the high dynamic illumination region, stable illumination region, high motion activity sub-region, and static background sub-region to obtain an initial partitioning structure that integrates the influence of multiple environmental factors. Specifically, this includes: First, obtaining the rain, snow, and fog weather classification identifiers obtained in step 102, and determining whether there is rain, snow, or fog interference in the current environment and the type of interference; if the classification identifier indicates that the current weather is sunny, then keep the partitioning structure and the priority of each region unchanged; if the classification identifier indicates that the current weather is rain, snow, or fog, then activate the partitioning adjustment mechanism, appropriately expand the boundary range of the high dynamic illumination region and the stable illumination region to adapt to the image blurring effect under rain, snow, and fog weather, and adjust the relative priorities of each region, raising the priority of the high motion activity sub-region above the static background sub-region, and further raising the priority of the high motion activity sub-region in the high dynamic illumination region to ensure that the region with dense targets and large environmental interference is prioritized during subsequent resource scheduling; after the adjustment is completed, integrate the boundary information and priority information of each region to form an initial partitioning structure that integrates multiple environmental factors such as illumination, motion, and weather.

[0043] Step 106: For each independent partition in the initial partitioning structure, retrieve the target recognition accuracy of that partition within a set time window from the historical log, and use it as the historical confidence benchmark for that partition. Specifically, this includes: first, assigning a unique spatial identifier to each independent partition in the initial partitioning structure. This identifier is generated using a composite coding rule, and the encoded content includes the partition's start and end coordinates in the current video frame, as well as environmental attribute encoding. The environmental attribute encoding is formed by a combination of lighting type, motion activity level, and weather adaptation type. This composite encoding ensures that the spatial identifier can uniquely associate the spatial location and environmental attributes of the partition, avoiding identifier confusion between different partitions; further... Next, a retrieval index is constructed based on the unique spatial identifier generated above. The corresponding historical data is retrieved from the system's preset historical log database. The retrieval is based on the spatial identifier as the core retrieval condition, while a time window constraint is superimposed. This time window can be dynamically configured according to the environmental stability of the actual application scenario. In scenarios with high environmental stability, the time window can be set to the last 60 days, while in scenarios with low environmental stability, it can be shortened to the last 15 days. The default configuration is usually the last 30 days. The retrieval scope is strictly limited to historical records within this time window that are consistent with the spatial identifier of the current partition and whose environmental feature similarity reaches a preset threshold. This ensures that the retrieved historical data is comparable to the environmental scenario of the current partition.

[0044] After the retrieval is completed, the target recognition accuracy data corresponding to each independent partition is extracted from the matched historical records to form an accuracy dataset. Then, anomaly removal calculations are performed on this accuracy dataset. Specifically, the overall distribution range of the dataset is first calculated, along with the mean and dispersion of all accuracy data. An anomaly threshold is determined based on the dispersion; accuracy data deviating from the mean by more than twice the dispersion is considered abnormal fluctuation data and removed from the dataset to avoid interference with the baseline results. After anomaly removal, a statistical average is calculated on the remaining valid accuracy data. Specifically, the number of valid data points is counted, and... All valid accuracy data are summed, and the sum is divided by the number of valid data points to obtain the average accuracy value. Finally, the validity of this average accuracy value is verified by comparing it with the baseline accuracy range of similar historical scenarios. If it falls within the range, the average accuracy value is confirmed as the historical confidence benchmark for that partition. If it falls outside the range, the process of anomaly removal and statistical averaging is repeated until a satisfactory result is obtained. This historical confidence benchmark can accurately characterize the reliability of target recognition in similar historical environments, providing a stable and reliable benchmark for subsequent dynamic weighted calibration based on the current environmental interference coefficient.

[0045] Step 107: Simultaneously, extract environmental feature scalars corresponding to the current video frame and the spatial location of the partition from the multi-dimensional environmental state descriptor, and calculate the environmental interference coefficient of the current environmental interference intensity. Specifically, this includes: first, based on the spatial location information of each independent partition, filtering out local environmental feature data corresponding to the spatial location from the environmental feature scalars. This data includes the local value of light intensity change, local background motion vector, and local rain, snow, and fog interference intensity at the corresponding location; based on the filtered local environmental feature data, allocating interference weights to each local environmental feature through a preset interference intensity evaluation logic, wherein the weights of light intensity change and rain, snow, and fog interference are higher than the weights of background motion vectors; and finally, performing a weighted summation of each local environmental feature data according to the allocated weights to obtain the environmental interference coefficient of the partition under the current environment. The larger the coefficient value, the more severe the interference of the current environment on the recognition effect of the partition.

[0046] Step 108 involves dynamically weighting and calibrating the historical confidence benchmark based on the environmental interference coefficient to calculate the confidence weight for each region. Specifically, this includes: first, executing a basic data validity verification process to ensure the reliability of the input data for the calibration calculation; then, retrieving the historical confidence benchmark and the corresponding environmental interference coefficient for each independent partition, and performing range verification on both types of data; the preset reasonable range for the historical confidence benchmark is 0 to 1, and the preset reasonable range for the environmental interference coefficient is also 0 to 1; if the historical confidence benchmark for a certain partition exceeds this range, the critical value of this range is taken as the corrected benchmark value—corrected to 1 if higher than 1, and corrected to 0 if lower than 0; if the environmental interference coefficient exceeds the reasonable range, the critical value is also corrected according to the above rules to avoid deviations in the calibration results caused by abnormal data.

[0047] Then, the calculation of the environment adaptation coefficient is initiated. For each independent partition, the environment interference coefficient after validity verification is used as the calculation object, and the environment adaptation coefficient is obtained through numerical deduction. The specific calculation process is to use the value 1 as a base constant and subtract the environment interference coefficient of the current partition from this constant. The result is the environment adaptation coefficient. The value of this coefficient is inversely related to the degree of environmental interference. The more severe the environmental interference, the smaller the environment adaptation coefficient, and vice versa. This provides a quantitative basis for subsequent weight adjustment.

[0048] Next, a weighted product calibration operation is performed. Using the historical confidence benchmark, after validity verification, as the base value, it is multiplied by the corresponding partition's calculated environment fit coefficient for each partition. Each partition undergoes a separate complete calculation to avoid cross-interference between data from different partitions. During the calculation, the specific values ​​of the current partition's base value and environment fit coefficient are first determined, and then the two values ​​are multiplied to obtain the initial confidence weight for that partition. For example, if a partition's historical confidence benchmark is 0.9 and its environment interference coefficient is 0.3, the environment fit coefficient is first calculated to be 0.7. Then, 0.9 and 0.7 are multiplied to obtain an initial confidence weight of 0.63, thus reducing the weight of partitions affected by environmental interference. If another partition has an environment interference coefficient of 0.1, an environment fit coefficient of 0.9, and a historical confidence benchmark of 0.85, the initial confidence weight after multiplication is 0.765, thus fine-tuning the weight of environmentally stable partitions.

[0049] After the calibration operation is completed, the result rationality verification mechanism is activated. For the initial confidence weight of each partition, it is determined whether it is within the preset reasonable range. This reasonable range is consistent with the reasonable range of the historical confidence benchmark, which is 0 to 1. If the initial confidence weight is within this range, it is considered a valid result. If it exceeds the range, the previous calculation steps are traced back to check the correction of the historical confidence benchmark and the calculation process of the environmental adaptation coefficient. If the result exceeds the standard due to an abnormal environmental interference coefficient, the environmental interference coefficient is corrected and the calculation is performed again. If it is due to improper correction of the base value, the correction rules of the historical confidence benchmark are adjusted and the calculation is recalculated until the confidence weight is within the reasonable range.

[0050] Finally, the effective confidence weights of all partitions are normalized and regularized. The sum of the confidence weights of all independent partitions is calculated, and the confidence weight of each partition is divided by the sum to obtain the final normalized confidence weight. This regularization process ensures that the sum of the confidence weights of all partitions is 1, thus determining the relative proportion of the processing priority of each partition. After calibration, the confidence weight of each independent partition is obtained, which directly reflects the recognition reliability and processing priority of each partition in the current environment.

[0051] Step 109 involves constructing ordered input feature vectors from the confidence weights of each region according to their spatial location, and then normalizing them to form standardized region confidence features. Specifically, this includes: first, arranging the confidence weights of each region sequentially according to their spatial location within the current video frame to construct a one-dimensional input feature vector; then, initiating the normalization process using the min-max normalization method to map each weight value in the input feature vector to a value range of 0 to 1; simultaneously recording the normalization parameters during the normalization process to provide a basis for subsequent inverse normalization adjustments; through the above processing, standardized region confidence features are formed, eliminating model training bias caused by differences in the magnitude of each weight value, and ensuring that the model can accurately learn the correlation between the confidence of each region and the detection parameters.

[0052] Step 110: Input the standardized region confidence features into a pre-trained lightweight neural network model. The lightweight neural network model, through its multi-layer fully connected network structure and non-linear activation function, performs forward propagation computation to map the input features into high-dimensional feature vectors. Specifically, this includes: achieving high-dimensional mapping of region confidence features through a pre-built and trained lightweight neural network model, controlling computational power consumption while mining deep feature correlations. The specific implementation process includes the entire process of building and training the lightweight neural network model and feature mapping, detailed below: First, the lightweight neural network model is built. This model adopts a simplified network architecture design to control computational power consumption, and includes an input layer, hidden layers, and an output layer. The number of neurons in the input layer matches the dimension of the standardized region confidence features generated in step 109, ensuring that the features can be completely input; the hidden... The network consists of 2 to 3 fully connected layers with a decreasing number of neurons in each layer. This gradual reduction in the number of neurons simplifies the network size. A pre-defined non-linear activation function is embedded after each fully connected layer. The activation function chosen is suitable for lightweight models. Its core function is to break the linear correlation between features, enhancing the model's ability to learn deep non-linear relationships between features. This allows the model to capture the complex intrinsic connections between confidence levels in different regions, thus adapting to the complex non-linear adaptation rules between region confidence features and subsequent core detection parameters. The output layer uses a dedicated mapping layer structure, with the number of neurons matching the number of core detection parameters requiring adjustment, ensuring the model outputs high-dimensional features corresponding to the core parameters. Furthermore, a parameter pruning strategy is introduced during network construction to remove redundant connection parameters, further compressing the model size and improving operational efficiency.

[0053] After the model is built, targeted training is conducted. The training data comes from regional confidence feature samples in historical scenarios and the optimal detection parameters verified in practice for the corresponding scenarios. First, the historical sample data is sorted and screened, and after removing abnormal samples, it is divided into training set and validation set according to a preset ratio. During the training process, the adaptation deviation between the high-dimensional feature vector output by the model and the corresponding optimal detection parameters is used as the optimization objective. An appropriate loss function is selected to measure the degree of deviation, and gradient descent is used for iterative optimization. The learning rate is dynamically adjusted during the training process. As the number of iterations increases, the learning rate is gradually reduced to improve training accuracy. At the same time, an early stopping mechanism is set. When the deviation index on the validation set no longer decreases for several consecutive rounds, training is stopped to avoid model overfitting. After training, the model performance is verified through the test set. The focus is on evaluating the feature mapping accuracy and running efficiency of the model in different environmental scenarios to ensure that the model has good generalization ability and lightweight running characteristics. Only after the verification is passed can it be put into practical application.

[0054] After training, standardized region confidence features are input into the lightweight neural network model. Once the model starts, it transforms the input features layer by layer through its multi-layer fully connected network structure. During each transformation, a preset non-linear activation function is used to non-linearly map the features, generating non-linearly transformed feature values. These values ​​serve as the input basis for subsequent layer feature transformations. Through layer-by-layer propagation and further transformation, deep correlation information is gradually extracted from the low-dimensional region confidence features, providing effective data support for multi-round forward propagation calculations. Through multiple rounds of forward propagation calculations, the low-dimensional region confidence features are mapped into high-dimensional feature vectors. These high-dimensional feature vectors comprehensively characterize the correlation between the confidence levels of each region and the adaptation rules with the detection parameters, providing high-dimensional feature support for the generation of subsequent core parameter adjustment quantities.

[0055] Step 111: Based on the high-dimensional feature vector, a joint adjustment amount for at least two core parameters in the object detection network is generated simultaneously through a dedicated mapping matrix of the output layer of the lightweight neural network model. The core parameters include the anchor box scale parameter controlling the size of the candidate box and the non-maximum suppression threshold parameter used to filter overlapping detection results. Specifically, this includes: First, based on the high-dimensional feature vector, calling the dedicated mapping matrix pre-stored in the output layer of the lightweight neural network model. This mapping matrix is ​​a parameter adjustment mapping relationship obtained during model training and is pre-configured to simultaneously associate the adjustment logic of the anchor box scale parameter and the non-maximum suppression threshold parameter. Its matrix dimension is pre-designed to adapt to the dimension of the high-dimensional feature vector and the number of parameters of the adjustment amount to be generated, ensuring that the matrix and the high-dimensional feature vector can be effectively operated. After calling, the dimension consistency of the dedicated mapping matrix and the high-dimensional feature vector is checked to confirm that the dimension of the high-dimensional feature vector is completely matched with the input dimension of the dedicated mapping matrix. If there is a dimension mismatch, dimension adaptation processing is started, and feature completion or dimension pruning is used to make the two meet the operation requirements to avoid dimension misalignment problems in subsequent calculations.

[0056] The linear transformation calculation process is then initiated. Specifically, each element of the high-dimensional feature vector is multiplied one by one with the corresponding element of the dedicated mapping matrix to obtain the product result at the corresponding position. All product results are then grouped and summed according to a preset grouping rule. One group of summed results corresponds to the adjustment calculation of the anchor frame scale parameter, and the other group corresponds to the adjustment calculation of the non-maximum suppression threshold parameter. For the anchor frame scale parameter adjustment group, the accumulated intermediate values ​​are scaled according to a preset scale coefficient. This scale coefficient is pre-set based on the value range of the anchor frame scale parameter to ensure that the scaled result is within a reasonable adjustment range, generating the initial adjustment amount for the anchor frame scale parameter. Similarly, for the non-maximum suppression threshold parameter adjustment group, the accumulated intermediate values ​​are scaled according to a preset threshold coefficient. This threshold coefficient adapts to the value characteristics of the non-maximum suppression threshold, generating the initial adjustment amount for the non-maximum suppression threshold parameter.

[0057] Next, a collaborative verification calculation is performed on the two initial adjustment values. Specifically, the optimal adaptation ratio of the anchor frame scale parameter and the non-maximum suppression threshold parameter in the historical scene is extracted. The ratio of the two currently generated initial adjustment values ​​is compared with this optimal adaptation ratio to calculate the ratio deviation value. If the ratio deviation value is within the preset allowable range, the two initial adjustment values ​​are directly combined to form a joint adjustment value. If the ratio deviation value exceeds the allowable range, the two initial adjustment values ​​are balanced and corrected based on the optimal adaptation ratio. During the correction process, the overall adjustment trend of the adjustment values ​​remains unchanged, and only the numerical value is finely adjusted proportionally until the ratio of the two meets the optimal adaptation requirements. Then, they are combined to form a joint adjustment value. This joint adjustment value can reflect the collaborative adaptation relationship between the two core parameters, ensuring that the parameter adjustment can jointly adapt to the current environmental state and confidence characteristics of each partition. After the calculation is completed, the joint adjustment value is formatted to meet the subsequent input requirements, thus completing the generation of the joint adjustment value.

[0058] Step 112 involves constraining and denormalizing the joint adjustment amount to adapt it to the parameter value range of the target detection network, thereby obtaining an optimized detection parameter set. Specifically, this includes: firstly, based on the hardware adaptation specifications of the target detection network, actual scenario requirements, and industry technical standards, a preset parameter value range is established. The width of the anchor frame scale parameter ranges from 10 to 500 pixels, and the height ranges from 20 to 600 pixels; the non-maximum suppression threshold ranges from 0.1 to 0.9, and these parameters are organized into a standard constraint and stored for later use. Then, parameter-by-parameter constraint verification is initiated. The adjustment amounts of the anchor frame scale and the non-maximum suppression threshold are separated from the joint adjustment amount and compared with their corresponding ranges. If the anchor frame scale adjustment amount exceeds the range, it is truncated to the critical value. If the deviation from the historical optimal range within the range is too large, it is corrected by using the median of the range. If the non-maximum suppression threshold adjustment amount exceeds the range, it is truncated. If the deviation is too large, it is fine-tuned towards the scene adaptation benchmark. The constrained joint adjustment amount is then obtained. Next, scale denormalization calculation is performed, calling the normalization parameters recorded in step 109, including the normalization upper and lower limits and the original numerical span. The constrained adjustment amount is then subtracted from the normalization value. The process involves minimizing the value, multiplying the difference by the original numerical range, and adding the original minimum value of the parameter to restore the normalized interval to the actual numerical range. After denormalization, the validity of the result is verified by comparing the restored adjustment amount with the preset value range again to avoid boundary overflow. At the same time, the result of superposition with the current network parameters is calculated to determine whether it is within a reasonable working range. If it is abnormal, the adjustment constraint or denormalization process is backtracked. Finally, the verified adjustment amount is superimposed with the initial network parameters. The initial parameters of the anchor box scale are added with the corresponding adjustment amount, and the initial parameters of the non-maximum suppression threshold are added with the corresponding adjustment amount to obtain the optimized parameters. The data precision is normalized according to the network input requirements, and a preset number of decimal places are retained to form a standardized detection parameter set, which can be directly applied to the target detection network parameter configuration to ensure that it is adapted to the current scene environment characteristics and target distribution.

[0059] Step 113: Receive the optimized detection parameter set, and set the anchor frame scale parameter and non-maximum suppression threshold parameter contained therein as the corresponding configurable operating parameters in the target detection network to obtain the target detection network after dynamic parameter configuration. Specifically, this includes: first, starting the parameter configuration interface of the target detection network and retrieving the optimized detection parameter set; extracting the anchor frame scale parameter and non-maximum suppression threshold parameter from the parameter set, and writing the two parameters into the corresponding configurable operating parameter bits of the network according to the parameter configuration protocol of the target detection network; after the parameters are written, starting the network parameter verification process to verify whether the written parameters meet the network operation requirements. If the verification passes, the parameter update of the target detection network is completed, and the target detection network after dynamic parameter configuration is obtained; if the verification fails, return to step 112 to re-perform constraint processing and inverse normalization to ensure that the target detection network after dynamic parameter configuration can run stably and adapt to the current environmental state.

[0060] Step 114: Drive the dynamically parameter-configured target detection network to perform forward propagation calculation on the current video frame to obtain a multi-scale feature space; based on the anchor box scale parameters, generate a scale-adaptive initial candidate box set at each spatial location in the multi-scale feature space. Specifically, this includes: firstly, driving the dynamically parameter-configured target detection network to perform forward propagation calculation on the current video frame extracted in step 101. The specific calculation process is as follows: the pixel matrix of the current video frame is used as input and sequentially passed to the feature extraction layer of the network for layer-by-layer processing; the first feature extraction layer uses a convolution of a preset size. The input pixel matrix is ​​checked and convolutional operations are performed pixel by pixel to generate the first-level feature map. At the same time, pooling operations are used to downsample the first-level feature map, compressing the feature map size and retaining key features. Subsequent feature extraction layers are based on the previous level feature map, and convolution and pooling operations are repeated. The size of the convolution kernel and the pooling window of each layer are gradually adjusted according to preset rules, so that the size of the output feature map of each layer decreases and the receptive field of the feature increases in turn. Finally, a multi-scale feature space containing feature information of low, medium and high scales is generated. Each scale feature space corresponds to the target feature extraction requirements of different sizes.

[0061] Then, the optimized anchor frame scale parameters configured in step 113 are retrieved. The parameters are first parsed to extract multiple sets of scale specification data, each set of data corresponding to an anchor frame width-to-height ratio. Based on the hierarchical characteristics of the multi-scale feature space, an adaptation mapping relationship between the anchor frame scale parameters and the feature space hierarchy is established. The specific calculation process is as follows: according to the size of the feature map at each level and the size of the receptive field, the target size range that the feature map at different levels can effectively cover is calculated. Each set of scale specifications in the anchor frame scale parameters is assigned to the corresponding feature space hierarchy according to the target size range. For example, small-sized anchor frames are assigned to shallow feature spaces with smaller feature map sizes and smaller receptive fields to adapt to small-sized targets such as people, while large-sized anchor frames are assigned to deep feature spaces with larger feature map sizes and larger receptive fields to adapt to large-sized targets such as vehicles.

[0062] After adaptation, initial candidate boxes are generated at various spatial locations in the multi-scale feature space. The specific calculation process is as follows: First, traverse every pixel of the feature map at each level, taking each pixel as the center point of the corresponding anchor box. Calculate the width and height values ​​of the anchor box according to the anchor box scale specifications assigned to that level. Based on the center point coordinates and width and height values, calculate the upper left and lower right corner coordinates of each anchor box. Specifically, subtract half of the anchor box width from the center point's x-coordinate to obtain the upper left x-coordinate, subtract half of the anchor box height from the center point's y-coordinate to obtain the upper left y-coordinate, add half of the anchor box width to the center point's x-coordinate to obtain the lower right x-coordinate, and add half of the anchor box height to the center point's y-coordinate to obtain the lower right y-coordinate. Following the above calculation method, anchor boxes of the corresponding scale are generated at all pixels in each feature space level. Summarize the anchor boxes generated at all levels to form a scale-adaptive initial candidate box set.

[0063] Step 115: Based on the initial candidate box set, calculate the targetability value of each candidate box and refine the bounding box position to obtain a candidate detection box set; apply the non-maximum suppression threshold parameter to perform overlap suppression and filtering on the candidate detection box set to obtain an effective detection box set. Specifically, this includes: First, based on a preset targetability evaluation criterion, calculate the targetability value of each candidate box in the initial candidate box set. The specific calculation process is as follows: extract feature information within the feature region corresponding to each candidate box, including feature grayscale distribution, edge gradient information, and local texture features; perform matching calculations between the extracted feature information and a preset target feature template, count the number of successfully matched feature dimensions, and simultaneously calculate the similarity between the features within the candidate box and the target feature template; set weights according to the number of matched dimensions and similarity, and perform a weighted summation of the two to obtain the targetability value. This targetability value is used to determine the probability that the candidate box contains the target.

[0064] Subsequently, a target value threshold is set, and the target value of each candidate box is compared with the threshold. Candidate boxes with target values ​​greater than the threshold are retained, while those with target values ​​less than or equal to the threshold are removed. The retained candidate boxes are then refined using a bounding box regression algorithm. The specific calculation process is as follows: First, the coordinate deviation between the candidate box and the real target region is calculated, including the horizontal coordinate deviation, vertical coordinate deviation, width deviation, and height deviation. Based on preset regression coefficients, the above deviations are weighted to obtain the deviation correction amount. The original coordinates of the candidate box are superimposed with the deviation correction amount to obtain the refined candidate box coordinates, thus completing the bounding box position refinement. All refined candidate boxes are then summarized to obtain the candidate detection box set.

[0065] Next, the optimized non-maximum suppression threshold parameter configured in step 113 is invoked. Based on the overlap threshold set by this parameter, the overlap of the candidate detection box set is calculated and suppression is screened. The specific calculation process is as follows: First, all detection boxes in the candidate detection box set are sorted from high to low according to their targetability value. The detection box at the top of the sorted list is selected as the reference box. The overlap between the reference box and all other detection boxes is calculated. The overlap is calculated using the ratio of the intersection area to the union area. That is, the area of ​​the overlapping region of the two detection boxes is calculated first, and then the union area of ​​the regions occupied by the two detection boxes is calculated. The overlap value is obtained by dividing the area of ​​the domain by the area of ​​the union. The calculated overlap value is compared with the preset nonmaximum suppression threshold, and detection boxes with overlap exceeding the threshold are removed. Then, the next detection box that is not removed from the ranking is taken as the new reference box, and the above overlap calculation and removal operation is repeated until all detection boxes in the candidate detection box set have been compared and filtered. Through the above processing, the detection boxes with the highest confidence and no excessive overlap in the candidate detection box set are retained to obtain the effective detection box set, which can accurately locate people and vehicle targets in the current video frame.

[0066] Step 116: For each detection box in the set of valid detection boxes, aggregate and abstract a deep feature vector representing the target category and attributes from the region features corresponding to the detection box in the multi-scale feature space. Specifically, this includes: for each detection box in the set of valid detection boxes, determining its corresponding feature region in the multi-scale feature space generated in step 114; using a feature aggregation algorithm to extract multi-dimensional feature information from the corresponding feature region, including the target's contour features, texture features, and local detail features; performing feature abstraction and fusion on the extracted multi-dimensional feature information, removing redundant and noisy features, and generating a deep feature vector that can uniquely represent the target category and attributes; simultaneously performing dimensional regularization on the deep feature vector during feature extraction to ensure that the deep feature vectors of all targets have a uniform dimensional specification, facilitating subsequent batch matching processing.

[0067] Step 117: Encapsulate the depth feature vector and detection boxes to construct a unified virtual feature contour data object; summarize all unified virtual feature contour data objects to form a virtual feature contour set containing people and vehicles. Specifically, this includes: First, defining a unified virtual feature contour data object structure, which includes a depth feature vector field and a detection box information field, wherein the detection box information field covers the coordinates, size, and other positional information of the detection box; filling the depth feature vector and the positional information of the corresponding detection box into the corresponding fields of the data object to complete the construction of the virtual feature contour data object for a single target; after the virtual feature contour data objects corresponding to all valid detection boxes are constructed, summarizing and integrating all data objects to form a virtual feature contour set containing all people and vehicle targets in the current video frame; this set adopts a standardized data format and can be directly input into the subsequent identity matching module for processing, realizing efficient flow of feature data.

[0068] In this embodiment of the invention, the current video frame is extracted based on a multi-dimensional environmental state descriptor, enabling precise association between the extracted video frame and environmental information, laying the foundation for subsequent scene adaptability processing; key environmental parameters are obtained by parsing the multi-dimensional environmental state descriptor, identifying the core environmental factors affecting target recognition, and providing accurate data support for scene partitioning; the illumination region is divided according to the quantization value of light intensity mutation, achieving precise partitioning of the illumination dimension, allowing regions with different illumination conditions to be processed differently; sub-regions are calibrated by combining background motion vector distribution features, further refining the partition granularity and improving the adaptability of partitions to target motion states; partition boundaries and priorities are adjusted based on weather classification labels, and the initial partition structure is optimized by integrating multiple environmental factors, enhancing the adaptability of partitions to complex weather environments; historical recognition accuracy is retrieved as a confidence benchmark, providing a reliable reference for partition confidence assessment using historical data, ensuring the rationality of the benchmark; environmental feature scalars are extracted to calculate the environmental interference coefficient, accurately quantifying the degree of interference of the current environment on each partition, providing a precise basis for confidence calibration; dynamic weighted calibration obtains confidence weights, making the weights of each partition conform to the actual current environment, improving the confidence weights. The system prioritizes timeliness and accuracy; it constructs and normalizes input feature vectors, standardizes input data formats, and improves the stability and efficiency of subsequent neural network model processing; a lightweight neural network generates high-dimensional feature vectors, reducing computational consumption while fully representing regional confidence correlation information; it generates joint adjustment quantities for core parameters, achieving multi-parameter collaborative optimization and ensuring the correlation and adaptability of parameter adjustments; it performs constraint processing and denormalization of joint adjustment quantities to ensure that the optimized parameters meet the requirements of the detection network operation, improving the reliability of parameter application; it configures optimized parameters to obtain a dynamic detection network, enabling the detection network parameters to adapt to the current environment and regional features, improving the adaptability of network operation; it generates scale-adaptive initial candidate boxes to adapt to different sizes of personnel and vehicle targets, improving the comprehensiveness of candidate box coverage; it filters to obtain a set of effective detection boxes, reducing overlapping box interference and improving the accuracy and effectiveness of detection boxes; it aggregates abstract deep feature vectors to fully mine target category and attribute information, providing high-quality feature support for subsequent identity matching; it encapsulates and summarizes virtual feature contour data objects, standardizes feature data formats, facilitates unified processing in subsequent steps, and improves data flow efficiency.

[0069] In a preferred embodiment of the present invention, step 200 above, which involves performing hierarchical feature parsing on each contour in the virtual feature contour set to obtain a virtual feature vector, and then performing real-time matching between the virtual feature vector and a preset virtual identity template library to obtain an identity matching vector, includes:

[0070] Step 201: Based on each virtual feature contour data object in the virtual feature contour set, separate the primary feature components related to appearance, texture, and geometry from the depth feature vector encapsulated by the virtual feature contour data object; combine the spatiotemporal sequence information of the detection box in the virtual feature contour data object to calculate the high-level feature components related to its motion trajectory and behavior pattern. Specifically, this includes: first, retrieving the virtual feature contour set; for each virtual feature contour data object in the set, starting the feature separation module. This module is pre-configured with feature classification rules, and according to these rules, separating the contour features related to the target appearance, the detail features related to the target texture, and the feature components related to the target geometry from the depth feature vector encapsulated by the data object one by one. The structural features related to the shape are integrated into the above three types of features into primary feature components. Further, the spatiotemporal sequence information of the detection box in the virtual feature contour data object is extracted, where the time information is the timestamp of the video frame corresponding to the detection box, and the spatial information is the coordinate position of the detection box in the video frame. Through the continuous correlation of the spatiotemporal sequence information, the position offset of the target in adjacent video frames is calculated, and then the motion trajectory of the target is fitted. At the same time, the change of the target's motion posture within a set time window is statistically analyzed, and high-level feature components related to the target's behavior pattern are extracted. Through the above processing, the primary feature components and high-level feature components corresponding to each virtual feature contour data object are extracted, realizing comprehensive coverage of the target's static attributes and dynamic behavior features.

[0071] Step 202 involves weighted fusion and dimensionality reduction of the primary and high-level feature components to obtain a virtual feature vector. The virtual feature vector is then normalized to obtain a normalized virtual feature vector. Specifically, this includes: First, determining the preset logic of the preset feature weight allocation rule. This rule is pre-defined based on the core identity recognition requirements of the target application scenario, and is preset through a feature contribution evaluation and verification optimization process. The specific preset process involves first analyzing the typical recognition requirements of target application scenarios such as smart park entrances and exits, determining the core judgment dimensions for personnel and vehicle identity recognition, and then determining the influence weights of static features such as appearance, texture, and geometric shape, and dynamic features such as motion trajectory and behavior patterns in different recognition scenarios. Subsequently, a large number of historical data are selected... For identity matching sample data, the number of correct matches and the number of incorrect matches contributed by different feature components in the sample matching process are statistically analyzed. The percentage of improvement in identity recognition accuracy of each feature component is calculated to quantitatively evaluate the identity recognition contribution of different feature components. Based on the evaluated contribution level, the weights are initially set, with higher weights assigned to feature components with higher contributions and lower weights assigned to feature components with lower contributions. The initially set weight rules are applied to feature fusion and matching tests on validation set samples, and the matching accuracy and stability indicators are statistically analyzed. If the indicators do not meet the preset requirements, the weight values ​​of each feature component are iteratively adjusted until the matching effect of the validation set meets the scene recognition accuracy requirements. Finally, a fixed feature weight allocation rule is formed and stored in the system for subsequent use.

[0072] Based on the aforementioned pre-defined feature weight allocation rules, the primary and advanced feature components are weighted and fused. Weight values ​​are assigned according to the contribution of features to identity recognition in different application scenarios. For example, in a smart park entrance / exit scenario, the weights of personnel appearance features and vehicle geometry features are higher than those of motion trajectory features. The two types of feature components are fused through weighted summation to obtain a fused feature vector. Subsequently, a dimensionality reduction process is initiated, employing a feature selection algorithm to remove redundant and noisy features from the fused feature vector, reducing its dimensionality and improving subsequent data processing efficiency, while retaining core features that accurately represent the target's identity. Finally, the dimensionality-reduced fused feature vector is normalized, mapping its values ​​to a pre-defined unified range to eliminate matching biases caused by differences in magnitude between different feature components, resulting in a normalized virtual feature vector that directly meets the needs of subsequent similarity calculations.

[0073] Step 203: Perform real-time similarity calculation between the normalized virtual feature vector and all template feature vectors in the pre-set virtual identity template library. The real-time similarity calculation includes at least the fusion of cosine similarity and Euclidean distance to obtain a comprehensive similarity value. Specifically, this includes: first, completing the construction and pre-setting of the pre-set virtual identity template library. The pre-setting process involves: pre-collecting standard sample data of authorized personnel and registered vehicles within the park. The authorized personnel sample data includes facial image sequences under different angles and lighting conditions, and the registered vehicle sample data includes clear image sequences of the front, side, and rear of the vehicle. The collected sample data is pre-processed to remove blurry, incomplete, or occluded invalid samples, retaining valid samples with clear image quality and complete features. The valid samples are then processed according to the preset feature extraction specifications. The extraction process involves extracting template feature vectors with dimensions consistent with the virtual feature vectors to ensure dimensionality adaptation for subsequent similarity calculations. Each extracted template feature vector is associated with a unique identity identifier. Personnel identity identifiers are associated with authorized information such as personnel name and employee ID, while vehicle identity identifiers are associated with registration information such as license plate number and vehicle type, establishing a one-to-one correspondence between template feature vectors and identity identifiers. All template feature vectors associated with identity identifiers are categorized and stored according to a pre-defined data structure to construct an initial virtual identity template library. A template library update and maintenance mechanism is also set up. When authorized personnel or registered vehicle information within the park is added, changed, or cancelled, the corresponding template feature vectors and identity identifier associations in the template library are updated synchronously to ensure the timeliness and accuracy of the template library information, thus completing the pre-configuration of the virtual identity template library.

[0074] After the template library is pre-set, it is retrieved. This template library pre-stores template feature vectors corresponding to vehicles registered by authorized personnel within the park, and each template feature vector is associated with a unique identity identifier. Then, the normalized virtual feature vectors are compared with all template feature vectors in the template library in real time. The calculation process adopts a fusion of cosine similarity and Euclidean distance. First, the cosine similarity value and Euclidean distance value between the virtual feature vector to be matched and each template feature vector are calculated separately. Then, the two similarity values ​​are weighted and summed according to the preset fusion weight to obtain the comprehensive similarity value. The fusion weight is set according to the reliability of the two similarity calculation methods in identity matching to ensure that the comprehensive similarity value can comprehensively and objectively reflect the degree of fit between the feature to be matched and the template feature.

[0075] Step 204: Based on the comparison result between the comprehensive similarity value and the preset confidence threshold, determine the target matching identity identifier and the corresponding confidence value, and combine them to generate the current target identity matching vector; wherein, if there is a comprehensive similarity value exceeding the threshold, the identity identifier and the confidence value are respectively taken as the template identity identifier corresponding to the highest value and the highest value; otherwise, a predetermined unknown identifier and a predefined minimum confidence value are taken. Specifically, this includes: firstly, performing an adaptation calculation of the preset confidence threshold. This threshold is not a fixed value, but is determined through a statistical calibration process in combination with the identity verification accuracy requirements of the actual application scenario. The specific calculation process is as follows: select a large number of historical identity matching samples in the target scenario, count the lowest comprehensive similarity value of the correctly matched cases and the highest comprehensive similarity value of the incorrectly matched cases in the samples, and calculate the intermediate critical value of the two types of values; based on this critical value, the threshold is fine-tuned in combination with the allowed false positive rate and false negative rate requirements of the scenario. If the scenario requires strict control of false positives, the critical value is appropriately increased; if strict control of false negatives is required, the critical value is appropriately decreased. Finally, the preset confidence threshold adapted to the current scenario is obtained and stored in the system.

[0076] Then, all comprehensive similarity values ​​are obtained, and a one-to-one comparison calculation process is initiated. Specifically, all comprehensive similarity values ​​are traversed, and each similarity value is compared with a preset reliability threshold. At the same time, the identity identifier associated with the template feature vector corresponding to each similarity value is recorded, forming an association list of similarity value and identity identifier. During the comparison process, the comparison results are marked simultaneously. Comprehensive similarity values ​​greater than the threshold are marked as valid matches, and those less than or equal to the threshold are marked as invalid matches, generating a list of valid match markers.

[0077] Next, the result determination calculation is performed based on the matching validity tag list. If there is a valid match in the tag list, the highest similarity value filtering calculation is initiated. The specific process is as follows: extract all valid comprehensive similarity values, sort them in descending order of value to form a descending similarity sequence; select the first value of the sequence as the highest comprehensive similarity value. If there are multiple identical highest values ​​in the sequence, further extract the identity identifiers corresponding to these values, and perform a secondary determination by querying the filing priority of the identity identifiers in the virtual identity template library. Select the identity identifier with the highest filing priority as the matching result, and establish an association between the identity identifier and the highest comprehensive similarity value. If all records in the tag list are invalid matches, the identity identifier is determined to be a predetermined unknown identifier, and the corresponding confidence value is a predefined minimum confidence value. This minimum confidence value is predefined by the following calculation logic: statistically analyze the distribution of comprehensive similarity values ​​of historical unmatched samples, take the minimum value of the distribution interval as the benchmark, and then adjust it down by a preset ratio to obtain the predefined minimum confidence value, ensuring that it is significantly lower than the confidence range of valid matches.

[0078] Finally, the identity matching vector is generated. Specifically, the determined identity identifier and confidence value are sequentially concatenated according to a preset standardized data format, with the identity identifier preceding the confidence value. The concatenated raw data undergoes format normalization to remove any redundant numerical bits, retains a preset number of decimal places for the confidence value, and encodes the identity identifier to meet the system's character length requirements. After normalization, the current target's identity matching vector is generated. This vector clearly represents the target's identity verification result and can be directly input into subsequent steps for multi-source data fusion processing. Furthermore, by setting unknown identifiers and minimum confidence levels, it ensures that even targets that do not match successfully can form complete identity matching data.

[0079] In this embodiment of the invention, primary feature components related to the appearance texture geometry are separated and acquired to comprehensively capture the static feature information of the target; high-level feature components related to motion trajectory and behavior pattern are calculated by combining spatiotemporal sequence information to supplement the dynamic feature dimension of the target; the collaborative extraction of static and dynamic features is realized to enrich the feature representation dimension and provide more comprehensive feature support for subsequent matching; the primary and high-level feature components are weighted and fused to strengthen the representation role of key features; redundant feature information is eliminated through dimensionality reduction to improve feature processing efficiency; virtual feature vectors are normalized to standardize feature data format and ensure the stability and consistency of subsequent matching calculations; cosine similarity and Euclidean distance are fused for real-time similarity calculation to measure the degree of feature matching from multiple dimensions; the comprehensiveness and reliability of similarity assessment are improved to ensure that matching relationships under different feature dimensions can be effectively captured; the identity identifier and confidence value are determined based on the comparison results of comprehensive similarity and confidence threshold to determine the matching judgment criteria; a standardized identity matching vector is generated to facilitate subsequent multi-source data fusion processing; the completeness of target identity matching is ensured by setting unknown identifiers and minimum confidence levels to avoid omissions in the processing of unmatched targets.

[0080] In a preferred embodiment of the present invention, step 300 above, based on the identity matching vector, performs multi-source data fusion on all target information within the same video frame to obtain a structured state vector containing virtual identity, state, and spatiotemporal coordinates; and performs real-time pattern analysis on the structured state vector to identify abnormal behavior patterns and generate early warning signals, including:

[0081] Step 301: Based on the identity matching vectors and corresponding depth feature vectors of all targets within the same video frame, feature alignment and spatial information fusion are performed to construct an initial state vector for each target. The initial state vectors of all targets are then aggregated to form a set of initial state vectors for the video frame. Specifically, this includes: first, retrieving the identity matching vectors of all targets, and simultaneously retrieving the depth feature vectors corresponding to each identity matching vector from step 100, establishing a one-to-one correspondence between identity information and feature information; then, according to preset feature alignment rules, matching and calibrating the feature dimensions of the identity matching vectors and the feature dimensions of the depth feature vectors to eliminate the two types of vectors. To address dimensional deviations and spatiotemporal misalignments caused by different feature acquisition links, the consistency of feature information is ensured. Based on this, the spatiotemporal coordinates of the detection boxes corresponding to each target are extracted and fused with the aligned identity matching vector and depth feature vector. The fusion process employs feature concatenation, integrating the identity information, feature information, and spatiotemporal information in a preset order to form the initial state vector for each target. Finally, the initial state vectors of all targets within the same video frame are summarized and arranged according to their spatial position within the video frame, forming the initial state vector set corresponding to that video frame. This provides well-organized foundational data for subsequent mining of interactions between targets.

[0082] Step 302: Based on the initial state vector set, construct interaction feature vectors by calculating the spatial and motion relationships between targets; fuse the interaction feature vectors with the initial state vectors of the corresponding targets to obtain the structured state vector of each target; summarize the structured state vectors of all targets to form a structured state vector set for the video frame, specifically including: firstly, based on the initial state vector set, extract the spatiotemporal coordinates and motion trend information from the initial state vector of each target in the set; construct the spatial correlation matrix and motion correlation matrix between targets by calculating the spatial distance, relative orientation, and motion direction consistency between any two targets; fuse the two matrices to extract a matrix that can characterize the target. The interaction feature vectors of the targets are obtained, which cover key interaction information such as whether the targets are approaching each other at close range and whether their movement trajectories intersect. Then, the interaction feature vectors of each target are weighted and fused with their own initial state vectors. The fusion weights are preset according to the degree of influence of the interaction information on the target state determination, so as to ensure the coordinated representation of interaction features and basic features of individual targets. After fusion, a structured state vector of each target is obtained, which fully contains virtual identity, feature information, spatiotemporal coordinates and interaction relationships. Finally, the structured state vectors of all targets in the same video frame are summarized to form a set of structured state vectors of the video frame, providing comprehensive state data support for subsequent temporal behavior analysis.

[0083] Step 303: Perform temporal pattern analysis on the structured state vector set, match and compare it with a preset normal behavior pattern library, and calculate the behavior deviation degree; based on the comparison of the behavior deviation degree with a preset abnormal threshold, identify abnormal behavior patterns. Specifically, this includes: firstly, perform temporal pattern analysis on the structured state vector set, extract the structured state vectors corresponding to multiple consecutive video frames, and construct a temporal sequence of the target's behavior within a set time window. This time window is configured according to the typical duration of the target's behavior in the scene, usually set to 5 to 10 frames, to ensure that the sequence can completely reflect the temporal evolution of the target's behavior.

[0084] Subsequently, a pre-built normal behavior pattern library is retrieved. This library is pre-constructed through standardized data collection, feature extraction, and classification storage processes. Specifically, the pre-built process involves: first, collecting video data of various authorized normal behaviors at smart park entrances and exits, including continuous video clips of typical behaviors such as normal pedestrian traffic, orderly vehicle parking, and visitor registration; pre-processing the collected video clips to remove blurry, occluded, or discontinuous clips, retaining only valid clips with complete behavior and clear image quality; for each valid clip, extracting structured state vectors from consecutive frames according to the structured state vector generation rules and constructing a behavior time sequence as a standard behavior sequence; and then processing all standard behaviors... Sequences are categorized by behavior type, with standard sequences of the same behavior type grouped together and labeled with a behavior category tag. The mean feature value of all standard sequences in each behavior category is calculated to generate the baseline time-series feature of that behavior category. At the same time, the feature fluctuation range of sequences within the category is statistically analyzed as a fault tolerance basis for subsequent matching and comparison. The categorized standard behavior sequences, baseline time-series features, and fault tolerance basis are stored according to a preset data structure to form an initial normal behavior pattern library. A library update mechanism is set up so that when a new normal behavior type is added in the scene or the original behavior norm is adjusted, new normal behavior data is collected synchronously and the above process is repeated to update the corresponding standard sequences and baseline features in the library, ensuring the timeliness and adaptability of the pattern library.

[0085] The target's behavioral time-series sequence is matched and compared with each standard sequence in the normal behavior pattern library one by one to calculate the behavioral deviation. The specific calculation process is as follows: First, the numerical difference between the target's behavioral time-series sequence and each standard sequence in the corresponding feature dimension is calculated. The sum of the squares of the differences in all dimensions is then averaged to obtain the mean of dimension differences. Next, combined with the fault tolerance criteria corresponding to the standard sequences, the temporal change synchronization degree between the target sequence and the standard sequences is calculated, that is, the percentage of times that the feature change trends of the two are consistent at the same time point is counted. Weights are set according to the mean of dimension differences and the temporal change synchronization degree, with the weight of the mean of dimension differences being higher than that of synchronization degree. The two are weighted and calculated to obtain the behavioral deviation degree. The larger the deviation degree value, the more obvious the difference between the target behavior and normal behavior.

[0086] Finally, the process of determining the preset anomaly threshold is executed. This threshold is preset through historical behavior data statistics and scene security level adaptation calculation. The specific calculation process is as follows: select a large amount of historical normal and abnormal behavior sample data within the scene, and calculate the behavior deviation of each sample according to the above method; statistically analyze the behavior deviation distribution of historical normal behavior samples, and take the upper limit of the distribution, i.e., the 95th quantile, as the initial threshold; fine-tune the initial threshold according to the scene security level requirements. For high-security-level scenes, the anomaly false negative rate needs to be strictly controlled, so the initial threshold is lowered by 10% to 20%. For ordinary-security-level scenes, the initial threshold is maintained, and for low-security-level scenes, the threshold is adjusted accordingly. For all scenarios, the threshold value can be appropriately increased to reduce the false positive rate. The fine-tuned value is then applied to the validation set samples for testing. The accuracy, false positive rate, and false negative rate of anomaly recognition are statistically analyzed. If the indicators do not meet the preset requirements, the threshold value is iteratively adjusted until the balance between scenario security and recognition accuracy is met. Finally, the preset anomaly threshold is determined and stored in the system. The calculated behavior deviation is compared with the preset anomaly threshold. If the behavior deviation is greater than the preset anomaly threshold, the corresponding behavior is determined to be an abnormal behavior pattern. If the behavior deviation is less than or equal to the preset anomaly threshold, it is determined to be a normal behavior pattern, thereby completing the accurate identification of abnormal behavior patterns.

[0087] Step 304: For the abnormal behavior pattern, extract its corresponding target structured state vector, timestamp of the associated video segment, and spatial location information, and encapsulate them to generate an early warning signal containing anomaly type, confidence level, target identifier, and handling suggestions. Specifically, this includes: First, for the identified abnormal behavior pattern, trace and extract its corresponding target structured state vector, and simultaneously extract the timestamp information of the associated video segment during the time period of the abnormal behavior, as well as the spatial coordinate information corresponding to the location of the abnormal behavior; Based on the extracted information, first determine the specific type of abnormal behavior, including preset categories such as unauthorized entry by personnel and unauthorized occupation of lanes by vehicles, and then retrieve the behavior deviation degree as the anomaly confidence level, and obtain the target identifier by associating the virtual identity identifier in the target structured state vector; Combined with the preset handling rules of the scenario, match corresponding handling suggestions for different types of abnormal behavior, including notifying security personnel to come to the scene for verification, triggering the barrier gate to intercept, etc.; Finally, encapsulate the above-mentioned anomaly type, confidence level, target identifier, timestamp, spatial location information, and handling suggestions according to a preset data format to generate a standardized early warning signal. This signal can be directly pushed to the backend management platform and the frontend security terminal to provide comprehensive and accurate information support for subsequent linkage response.

[0088] In this embodiment of the invention, feature alignment between identity matching vectors and deep feature vectors is achieved to ensure consistency of features from different sources; spatial information is fused to construct an initial state vector, integrating core basic information of the target; an initial state vector set is formed to achieve the orderly inclusion of multiple target information within the same video frame, laying a data foundation for subsequent interactive feature extraction; spatial and motion relationships between targets are calculated to construct interactive feature vectors, capturing the correlation information between targets; interactive feature vectors and initial state vectors are fused to enrich the information dimensions of structured state vectors; a structured state vector set is formed to achieve a comprehensive representation of target information and improve the completeness of target state description; temporal pattern analysis is performed on the structured state vectors to mine the temporal correlation features of target behavior; matching and comparison are performed based on a pre-set normal behavior pattern library to provide a reliable benchmark for behavior judgment; abnormal behavior patterns are identified by calculating behavior deviation, achieving standardization of abnormal judgment and improving the systematic nature of behavior analysis; core information related to abnormalities is extracted to ensure the information integrity of the warning signal; a warning signal containing the target identifier of the abnormality type confidence level and handling suggestions is encapsulated to determine the core elements of the warning; precise guidance is provided for subsequent linkage response, improving the practicality and operability of the warning signal and ensuring efficient connection of abnormal handling.

[0089] In a preferred embodiment of the present invention, step 400 above, based on the structured state vector and the warning signal, performs virtual trajectory association and behavior chain reconstruction across cameras to obtain a comprehensive behavior profile; based on the comprehensive behavior profile, performs differentiated storage and linkage response on all data generated during the processing and associated with the target, including:

[0090] Step 401: Obtain the structured state vector set and warning signals of the current camera and its associated cameras. Align and calibrate the spatiotemporal coordinates in the structured state vector set based on a unified spatiotemporal reference to form a spatiotemporally aligned multi-camera target state set. Specifically, this includes: First, retrieving all associated cameras corresponding to the current camera through a pre-set camera association mapping table. This mapping table pre-determines the coverage area and adjacent relationships of each camera according to the smart park monitoring layout plan. Then, synchronously acquire the structured state vector set and warning signals output by the current camera and all associated cameras, establishing a multi-source data convergence channel. Based on this, using a pre-set unified time server and spatial coordinate origin within the park as a reference, align and calibrate the timestamps and spatial coordinates in the structured state vector sets of each camera. Time calibration eliminates clock deviations between cameras through timestamp synchronization correction, while spatial calibration achieves accurate conversion from local coordinates to global coordinates through a fusion spatial coordinate mapping algorithm. The specific calculation process involves first obtaining the installation parameters of each camera, including intrinsic parameters, extrinsic parameters, installation height, and horizontal tilt angle. Based on these parameters, construct the local coordinates and the global coordinates of the park. The system establishes a coordinate mapping model. Local coordinate data is then extracted from the structured state vectors of each camera. Mapping calculations are performed on each local coordinate point. First, the camera's intrinsic parameters correct for coordinate deviations caused by lens distortion. Then, extrinsic parameters are used to convert the corrected local coordinates into intermediate coordinates based on the origin of the park's global coordinate system. Next, the intermediate coordinates are calibrated for elevation and azimuth based on the camera's installation height and horizontal tilt angle to eliminate coordinate offsets caused by different installation positions. After completing the single-coordinate point mapping, multiple preset fixed park markers are selected as verification benchmarks. The actual global coordinates of these markers are compared with the coordinates obtained from the mapping calculation to calculate the coordinate deviation. If the deviation exceeds a preset threshold, the mapping model parameters are adjusted, and the mapping calculation is re-executed. If the deviation is within the allowable range, the mapping result is confirmed to be valid. Through the series of calculations of the above spatial coordinate mapping algorithm, accurate mapping of all camera local coordinates to a unified park global coordinate system is achieved. After calibration, the calibrated structured state vectors and warning signals of all cameras are integrated to form a spatiotemporally aligned multi-camera target state set, ensuring spatiotemporal consistency of the same target data collected by different cameras.

[0091] Step 402: Based on the multi-camera target state set, cross-camera trajectory association and matching are performed according to the target's identity identifier, appearance features, and continuous spatiotemporal position. A global trajectory spanning multiple camera monitoring areas is generated for each target. Specifically, this includes: first, extracting the identity identifier, appearance features, and continuous spatiotemporal position information of each target from the multi-camera target state set to establish a target basic information index; then, using a multi-dimensional matching strategy, preliminary matching is performed first by identity identifier, associating target data with consistent identity identifiers as candidate trajectory segments; for targets with missing identity identifiers or unsuccessful matching, secondary matching is performed based on appearance feature similarity comparison and continuous spatiotemporal position smoothness analysis. The appearance feature comparison filters potential matching targets by ranking features similarity, and the spatiotemporal position analysis judges the continuity of the target's position at the boundary of adjacent camera coverage areas, while also judging the rationality of the movement direction, eliminating unreasonable matching results; through the above multi-dimensional matching, the target trajectory segments scattered across different cameras are concatenated and spliced ​​to generate a global trajectory covering multiple camera monitoring areas for each target. This trajectory completely records the target's movement path and time sequence information within the park.

[0092] Step 403: Based on the global trajectory and the behavioral states carried in the multi-camera target state set, a behavioral sequence reconstruction is performed, connecting and reconstructing discrete state points into a behavioral chain reflecting the continuous activities of the target. Specifically, this includes: firstly, extracting the temporal nodes corresponding to the global trajectory, and retrieving the target behavioral state information corresponding to each temporal node in the multi-camera target state set in chronological order. This behavioral state information includes key data such as the target's motion state and interaction state. Then, the behavioral sequence reconstruction engine is started, using the temporal nodes of the global trajectory as clues to connect the discrete behavioral state points in chronological order. At the same time, based on the target's motion trend and behavioral logic, behavioral transition information between adjacent state points is supplemented to eliminate the gaps between discrete state points. Through the above reconstruction process, the originally isolated target state data is transformed into a behavioral chain that can reflect the continuous activity process of the target. This behavioral chain completely covers all the target's behavioral activities from entering the monitoring area of ​​the first camera to leaving the monitoring area of ​​the last camera.

[0093] Step 404: Integrate the global trajectory and behavior chain to obtain a comprehensive behavior profile. This includes: First, defining a standardized data structure for the comprehensive behavior profile, which pre-defines four core modules: trajectory information area, behavior information area, identity information area, and associated warning area. Then, input the global trajectory data into the trajectory information area, including trajectory coordinate sequence, time sequence, and associated camera information. Input the reconstructed behavior chain data into the behavior information area, including behavior type sequence, behavior duration, and key behavior nodes. Simultaneously, extract the target's identity identifier, confidence level, and other information from the multi-camera target state set and input it into the identity information area. Extract the associated warning signals and input them into the associated warning area. Finally, perform consistency verification on the data of each module to ensure that the trajectory information and behavior information match each other in time, and that the identity information and associated warning information are accurately associated. After the verification is passed, a complete comprehensive behavior profile is formed.

[0094] Step 405: Based on the target's behavior category, risk level, and associated warning signals in the comprehensive behavior file, perform hierarchical classification and storage of cross-camera processing data associated with the target and the comprehensive behavior file; simultaneously, according to preset rules, trigger automated linkage response control commands corresponding to the risk level, specifically including: firstly, extracting the target's behavior category, risk level, and associated warning signals from the comprehensive behavior file, and dividing the data according to preset hierarchical classification and storage rules. These rules are preset through standardized requirements analysis, importance assessment, and adaptation verification processes. The specific preset process is as follows: firstly, sort out the core requirements of smart park security management and data traceability, and determine the role of different types of data in event handling and post-event verification; statistically analyze the historical data call frequency, traceability priority, and storage cost, and quantitatively assess the importance level of various types of data, among which data directly related to abnormal behavior and warning signals are prioritized. The data is categorized based on importance, with the highest priority given to global target trajectory and key behavioral data, followed by routine and basic data. This categorization determines the specific scope of core, critical, important, and ordinary data. For each data type, the appropriate storage devices and methods are matched to meet access speed requirements, storage cycle requirements, and cost budgets. Core data needs to meet the requirements of rapid retrieval and long-term traceability, so high-speed storage devices and long-term storage strategies are adopted. Important data balances storage efficiency and cost, using standard storage devices and medium-term storage strategies. Ordinary data has low access frequency, so compressed storage and short-term storage strategies are used. These data classification standards, storage device selection, and storage cycle settings are integrated into hierarchical storage rules, applied to the validation dataset for storage testing, and used to statistically analyze storage utilization, retrieval response speed, and cost consumption. The rule details are iteratively adjusted until the indicators meet the standards, and finally, the rules are solidified and stored in the system.

[0095] Based on the preset rules, the data is divided into three levels: core critical data, important data, and ordinary data. Core critical data includes comprehensive behavior profiles, warning signals, and video clips corresponding to abnormal behaviors. Important data includes the target's global trajectory and key behavior status data. Ordinary data includes basic status data corresponding to regular passage. Different storage strategies are adopted for different levels of data: core critical data is stored long-term using high-speed storage devices, important data is stored medium-term using standard storage devices, and ordinary data is stored short-term using compressed storage.

[0096] Simultaneously, based on the risk level in the comprehensive behavior profile, pre-set linkage response rules are matched. These rules are also pre-set through scenario requirement decomposition, risk level definition, and response measure adaptation processes. The specific pre-setting process is as follows: First, based on the smart park security level standards, the handling requirements of different risk scenarios are decomposed, and the judgment criteria for three risk levels—high-risk, medium-risk, and low-risk—are defined. High-risk corresponds to behaviors that may cause safety accidents or significant property damage; medium-risk corresponds to behaviors that affect the normal order of the park and require timely intervention; and low-risk corresponds to behaviors that involve minor violations and do not require emergency handling. For each risk level, the corresponding handling process and response measures are sorted out. Referring to historical security incident handling cases, efficient and feasible response methods are selected. High-risk levels are matched with emergency linkage response measures, medium-risk levels with general linkage response measures, and low-risk levels with suggestive response measures. The risk level judgment criteria, response measure list, and triggering logic are integrated into linkage response rules. Response tests are conducted by simulating different risk scenarios to evaluate the timeliness of the response, the adaptability of the measures, and the resource consumption. Based on the test results, the details of the response measures and triggering conditions are adjusted to ensure that the rules meet the balance requirements of scenario security control and reasonable resource allocation. Finally, the rules are solidified and stored in the system.

[0097] When the risk level is high, an emergency response command is triggered, including activating on-site audible and visual alarms, notifying security personnel for emergency response, and triggering gate interception. When the risk level is medium, a general response command is triggered, including pushing early warning information to security terminals and the trajectory of key monitored targets. When the risk level is low, a prompt response command is triggered, recording only relevant information for subsequent verification. Through the above-mentioned hierarchical storage and precise response, rational resource allocation and efficient risk management are achieved.

[0098] In this embodiment of the invention, structured state vectors and warning signals from multiple cameras are integrated to achieve the convergence of multi-source data; spatiotemporal coordinates are calibrated using a unified spatiotemporal reference to ensure spatiotemporal consistency of data from different cameras; a spatiotemporally aligned set of target states from multiple cameras is formed, providing a regular and compatible data foundation for cross-camera trajectory association; cross-camera trajectory association is performed by combining identity identification appearance features and continuous spatiotemporal positions to improve the accuracy of trajectory matching; a global trajectory is generated to achieve trajectory continuity across the monitoring area of ​​multiple cameras; monitoring breakpoints in target movement between cameras are eliminated to improve the integrity of target trajectories; and behavioral sequences are reconstructed based on the global trajectory and behavioral states, transforming discrete state points into continuous lines. This system serves as a chain; it fully reconstructs the continuous activity process of the target, clearly presenting the evolutionary trajectory of the target's behavior; it enhances the coherence and readability of behavioral information; it integrates the global trajectory and behavioral chain to form a comprehensive behavioral archive, comprehensively summarizing the target's trajectory information and behavioral data; it constructs a complete target behavior representation system, providing comprehensive decision-making basis for subsequent storage and response; it classifies and stores data according to behavioral category, risk level, and early warning signals, rationally allocating storage resources to improve storage utilization efficiency; it facilitates rapid retrieval and traceability of subsequent data; it triggers corresponding automated linkage response commands based on risk level, achieving precise risk-adaptive responses; it ensures the pertinence and efficiency of response measures, enhancing the application value of data processing results.

[0099] like Figure 2 As shown, embodiments of the present invention also provide a real-time recognition data processing system for camera surveillance video, comprising:

[0100] The target detection module is used to extract the current video frame from the real-time video frame sequence based on a multi-dimensional environment state descriptor and perform adaptive scene partitioning to obtain the confidence weight of each region. Using the confidence weight of each region as input, a lightweight neural network model is used to perform joint nonlinear calibration on the core parameters of the target detection network to obtain an optimized detection parameter set. Using the optimized detection parameter set, multi-target synchronous recognition and deep feature abstraction are performed on the current video frame to obtain a set of virtual feature contours containing people and vehicles.

[0101] The identity matching module is used to perform hierarchical feature parsing on each contour in the virtual feature contour set to obtain a virtual feature vector; and to perform real-time matching between the virtual feature vector and a preset virtual identity template library to obtain an identity matching vector.

[0102] The early warning module is used to perform multi-source data fusion on all target information within the same video frame based on identity matching vectors to obtain a structured state vector containing virtual identity, status, and spatiotemporal coordinates; and to perform real-time pattern analysis on the structured state vector to identify abnormal behavior patterns and generate early warning signals.

[0103] The response module is used to perform virtual trajectory association and behavior chain reconstruction across cameras based on the structured state vector and warning signal to obtain a comprehensive behavior profile; based on the comprehensive behavior profile, it performs differentiated storage and linkage response on all data generated during the processing that are associated with the target.

[0104] It should be noted that this system is a system corresponding to the above method. All implementation methods in the above method embodiments are applicable to this embodiment and can achieve the same technical effect.

[0105] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for real-time identification data processing of camera surveillance video, characterized in that, The method includes: Step 100: Based on the multi-dimensional environment state descriptor, extract the current video frame from the real-time video frame sequence and perform adaptive scene partitioning to obtain the confidence weight of each region; using the confidence weight of each region as input, perform joint nonlinear calibration on the core parameters of the target detection network through a lightweight neural network model to obtain an optimized detection parameter set; using the optimized detection parameter set, perform multi-target synchronous recognition and deep feature abstraction on the current video frame to obtain a set of virtual feature contours containing people and vehicles; Step 100 further includes: The confidence weights of each region are constructed into an ordered input feature vector according to their spatial location, and then normalized to form standardized regional confidence features. Standardized region confidence features are input into a pre-trained lightweight neural network model; the lightweight neural network model performs forward propagation calculations through its multi-layer fully connected network structure and non-linear activation function to map the input features into high-dimensional feature vectors. Based on the high-dimensional feature vector, a dedicated mapping matrix is ​​generated through the output layer of the lightweight neural network model, and a joint adjustment amount for at least two core parameters in the object detection network is generated. The core parameters include the anchor box scale parameter that controls the size of the candidate box and the non-maximum suppression threshold parameter used to filter overlapping detection results. The joint adjustment amount is constrained and denormalized to fit the parameter value range of the target detection network, so as to obtain the optimized detection parameter set. Step 200: Perform hierarchical feature parsing on each contour in the virtual feature contour set to obtain a virtual feature vector; perform real-time matching of the virtual feature vector with a preset virtual identity template library to obtain an identity matching vector; Step 300: Based on the identity matching vector, multi-source data fusion is performed on all target information within the same video frame to obtain a structured state vector containing virtual identity, status, and spatiotemporal coordinates; real-time pattern analysis is performed on the structured state vector to identify abnormal behavior patterns and generate early warning signals. Step 400: Based on the structured state vector and the warning signal, perform virtual trajectory association and behavior chain reconstruction across cameras to obtain a comprehensive behavior profile; based on the comprehensive behavior profile, perform differentiated storage and linkage response on all data generated during the processing that are associated with the target.

2. The method for real-time identification data processing of camera surveillance video according to claim 1, characterized in that, Before step 100, the method further includes: The system adaptively receives and schedules the raw video stream captured by the camera to obtain a real-time video frame sequence; it then performs environmental state analysis on the real-time video frame sequence, including at least light intensity abrupt change detection, background motion vector analysis, and rain, snow, and fog weather classification, to obtain corresponding environmental feature scalars; and finally, it fuses the environmental feature scalars into a multi-dimensional environmental state descriptor.

3. The real-time identification data processing method for camera surveillance video according to claim 2, characterized in that, Step 100 includes: Based on the multidimensional environment state descriptor, the current video frame is extracted from the real-time video frame sequence; The multidimensional environmental state descriptor is parsed to obtain its included light intensity abrupt change quantization value, background motion vector distribution characteristics, and rain, snow, and fog weather classification identifier; Based on the light intensity abrupt change quantization value, a high dynamic range lighting region and a stable lighting region are divided in the current video frame; Based on the background motion vector distribution characteristics, within the high dynamic illumination region and the stable illumination region, a high motion activity sub-region and a static background sub-region are further identified. Based on the rain, snow and fog weather classification labels, the boundary division and relative priority of the high dynamic illumination area, stable illumination area, high motion activity sub-region and static background sub-region are adaptively adjusted to obtain an initial partition structure that integrates the influence of multiple environmental factors. For each independent partition in the initial partitioning structure, the target recognition accuracy of that partition within a set time window is retrieved from the historical log and used as the historical confidence benchmark for that partition. Simultaneously, environmental feature scalars corresponding to the current video frame and the spatial location of the partition are extracted from the multidimensional environmental state descriptor, and the environmental interference coefficient of the current environmental interference intensity is calculated. The historical confidence benchmark is dynamically weighted and calibrated based on the environmental interference coefficient to calculate the confidence weight of each region.

4. The real-time identification data processing method for camera surveillance video according to claim 3, characterized in that, Step 100 further includes: The optimized detection parameter set is received, and the anchor frame scale parameter and non-maximum suppression threshold parameter contained therein are set as the corresponding configurable operating parameters in the target detection network, so as to obtain the target detection network after dynamic parameter configuration. The target detection network, configured with dynamic parameters, is driven to perform forward propagation calculation on the current video frame to obtain a multi-scale feature space; based on the anchor box scale parameters, an initial candidate box set with scale adaptation is generated at each spatial location in the multi-scale feature space. Based on the initial candidate box set, the objective value of each candidate box is calculated and the bounding box position is refined to obtain a candidate detection box set; the non-maximum suppression threshold parameter is applied to the candidate detection box set to suppress overlap and filter, so as to obtain an effective detection box set. For each detection box in the set of valid detection boxes, a deep feature vector representing the target category and attributes is aggregated and abstracted from the region features corresponding to the detection box in the multi-scale feature space. The depth feature vector and the detection box are encapsulated to construct a unified virtual feature contour data object; all unified virtual feature contour data objects are summarized to form a virtual feature contour set containing people and vehicles.

5. The method for real-time identification data processing of camera surveillance video according to claim 4, characterized in that, Step 200 includes: Based on each virtual feature contour data object in the virtual feature contour set, primary feature components related to appearance, texture, and geometry are separated from the depth feature vector encapsulated by the virtual feature contour data object; combined with the spatiotemporal sequence information of the detection box in the virtual feature contour data object, high-level feature components related to its motion trajectory and behavior pattern are calculated. The primary feature components and the high-level feature components are weighted, fused, and dimensionality reduced to obtain a virtual feature vector. The virtual feature vector is then normalized to obtain a normalized virtual feature vector. The normalized virtual feature vector is compared with all template feature vectors in the preset virtual identity template library in real time. The real-time similarity calculation includes at least the fusion of cosine similarity and Euclidean distance to obtain a comprehensive similarity value. Based on the comparison result between the comprehensive similarity value and the preset confidence threshold, the identity identifier and corresponding confidence value of the target match are determined, and the identity matching vector of the current target is generated by combining them; wherein, if there is a comprehensive similarity value that exceeds the threshold, the identity identifier and the confidence value are respectively taken as the template identity identifier corresponding to the highest value and the highest value; otherwise, a predetermined unknown identifier and a predefined minimum confidence value are taken.

6. The real-time identification data processing method for camera surveillance video according to claim 5, characterized in that, Step 300 includes: Based on the identity matching vectors and corresponding depth feature vectors of all targets within the same video frame, feature alignment and spatial information fusion are performed to construct the initial state vector of each target. The initial state vectors of all targets are then aggregated to form the initial state vector set of the video frame. Based on the initial state vector set, an interaction feature vector is constructed by calculating the spatial and motion relationships between targets; the interaction feature vector is fused with the initial state vector of the corresponding target to obtain the structured state vector of each target; the structured state vectors of all targets are summarized to form a set of structured state vectors for the video frame. Temporal pattern analysis is performed on the structured state vector set, and the behavior deviation is calculated by matching and comparing it with a preset normal behavior pattern library. Abnormal behavior patterns are identified based on the comparison between the behavior deviation and a preset abnormal threshold. For the aforementioned abnormal behavior pattern, the corresponding target structured state vector, timestamps of associated video segments, and spatial location information are extracted and encapsulated to generate an early warning signal containing the abnormality type, confidence level, target identifier, and handling suggestions.

7. The method for real-time identification data processing of camera surveillance video according to claim 6, characterized in that, Step 400 includes: Obtain the structured state vector set and warning signal of the current camera and its associated cameras, and align and calibrate the spatiotemporal coordinates in the structured state vector set based on a unified spatiotemporal reference to form a spatiotemporally aligned multi-camera target state set; Based on the multi-camera target state set, cross-camera trajectory association and matching are performed according to the target's identity, appearance features and continuous spatiotemporal location, generating a global trajectory for each target that spans the monitoring areas of multiple cameras; Based on the global trajectory and the behavioral states carried in the multi-camera target state set, behavioral sequence reconstruction is performed, and discrete state points are connected and reconstructed into a behavioral chain that reflects the continuous activity of the target. Integrate the global trajectory and behavior chain to obtain a comprehensive behavior profile; Based on the target's behavior category, risk level, and associated warning signals in the comprehensive behavior profile, the cross-camera processing data associated with the target and the comprehensive behavior profile are stored in a hierarchical and classified manner; at the same time, according to preset rules, automated linkage response control commands corresponding to the risk level are triggered.

8. A real-time recognition data processing system for camera surveillance video, wherein the system implements the method as described in any one of claims 1 to 7, characterized in that, include: The object detection module is used to extract the current video frame from the real-time video frame sequence based on a multi-dimensional environment state descriptor and perform adaptive scene partitioning to obtain the confidence weight of each region. Using the confidence weights of each region as input, a lightweight neural network model is used to perform joint nonlinear calibration on the core parameters of the target detection network to obtain an optimized detection parameter set. Using the optimized detection parameter set, multi-target synchronous recognition and deep feature abstraction are performed on the current video frame to obtain a set of virtual feature contours containing people and vehicles. The identity matching module is used to perform hierarchical feature parsing on each contour in the virtual feature contour set to obtain a virtual feature vector; and to perform real-time matching between the virtual feature vector and a preset virtual identity template library to obtain an identity matching vector. The early warning module is used to perform multi-source data fusion on all target information within the same video frame based on identity matching vectors to obtain a structured state vector containing virtual identity, status, and spatiotemporal coordinates; and to perform real-time pattern analysis on the structured state vector to identify abnormal behavior patterns and generate early warning signals. The response module is used to perform virtual trajectory association and behavior chain reconstruction across cameras based on the structured state vector and warning signal to obtain a comprehensive behavior profile; based on the comprehensive behavior profile, it performs differentiated storage and linkage response on all data generated during the processing that are associated with the target.

9. A computing device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the method as described in any one of claims 1 to 7.

Images