Method and apparatus for asynchronous task callback of hardware codec and GPU

Abstract

This application provides a method and apparatus for asynchronous task callback between a hardware codec and a GPU, belonging to the field of graphics processing technology. It involves acquiring video frame data and facial feature data of a target object; performing signal-to-noise estimation and latency analysis on the video frame data to generate frame quality data; performing multi-level overall mapping transformation on the facial feature data to generate surface features, enhanced features, and deep features, and measuring the similarity between the surface features and the enhanced features; evaluating the stability of the frame quality data, and weighting and fusing the stability evaluation results and similarity measurement results to generate weight factors; and weighting and judging incremental features based on the weight factors to generate callback results. This improves the dynamic adjustment capability of features at different layers, enhances real-time performance and accuracy in high-concurrency, multi-camera environments, and thus improves video processing efficiency and the stability of aggregation results.

Description

Technical Field

[0001] This application belongs to the field of graphics processing technology, and specifically relates to a method and apparatus for asynchronous task callback between a hardware codec and a GPU. Background Technology

[0002] In the field of modern video processing and computer vision, with the development of artificial intelligence and deep learning technologies, application scenarios such as video analysis, intelligent surveillance, and facial recognition have placed higher demands on computing performance and real-time performance. Especially in multi-camera, multi-task concurrent processing environments, the acquisition, feature extraction, aggregation, and analysis of video data rely on the collaborative work of hardware codecs and GPUs to complete high-throughput data processing and complex model inference. Hardware codecs are mainly responsible for efficient video frame encoding and decoding, while GPUs are responsible for feature extraction and analysis calculations for deep learning models. This division of labor has become a common architecture in large-scale video processing systems.

[0003] In existing technologies, the collaboration between hardware codecs and GPUs (graphics processing units) mainly relies on synchronous or blocking task submission methods. The advantages of these methods are simple implementation logic, easy data integrity assurance, and the ability to meet basic requirements in single-camera or low-concurrency scenarios. However, they lack the ability to dynamically adjust the contribution of features at different layers, which can easily lead to instability in feature aggregation results in complex scenarios such as changes in video frame quality, sudden changes in lighting conditions, partial occlusion, or inconsistent camera angles. At the same time, during the feature aggregation process, the difficulty in dynamically weighting multi-layer features directly affects the aggregation accuracy, while full aggregation further amplifies the computational latency. This makes it difficult to balance real-time performance and accuracy in high-concurrency, multi-camera environments, thus creating a dual constraint on the efficiency of video feature processing and the stability of aggregation results. Summary of the Invention

[0004] In view of the above problems, this application proposes an asynchronous task callback method and apparatus between hardware codec and GPU. This improves the dynamic adjustment capability of features at different layers, enhances real-time performance and accuracy in high-concurrency, multi-camera environments, and thus improves video processing efficiency and the stability of aggregation results.

[0005] One embodiment of this application provides an asynchronous task callback method between a hardware codec and a GPU, including:

[0006] Acquire video frame data and facial feature data of the target object;

[0007] Signal-to-noise estimation and delay analysis are performed on the video frame data to generate frame quality data;

[0008] The facial feature data is subjected to a multi-level overall mapping transformation to generate surface features, enhanced features and deep features respectively, and the similarity between the surface features and the enhanced features is measured.

[0009] The frame quality data is subjected to stability evaluation, and the stability evaluation results and similarity measurement results are weighted and fused to generate weight factors;

[0010] The incremental features are weighted according to the weighting factors to generate callback results.

[0011] Furthermore, the process of weighting the incremental features according to the weighting factors to generate a callback result also includes:

[0012] Based on the weighting factors, the surface features, the enhanced features, and the deep features are respectively subjected to hierarchical adjustment and unified projection mapping to generate an initial aggregate vector;

[0013] The surface features, the enhanced features, and the deep features are subjected to pairwise difference analysis, and the difference analysis results are weighted and adjusted according to the weighting factor. It is determined whether the weighted adjustment result is greater than the preset discrimination threshold. If so, the difference analysis, the weighted adjustment result, and the initial aggregation vector are encapsulated to generate incremental features.

[0014] The preset discrimination threshold is obtained by performing overall statistical analysis on the frame quality data.

[0015] Furthermore, the acquisition of video frame data and facial feature data of the target object includes:

[0016] Based on the single-frame structured image data packet of the target object acquired by the front-end camera device at a preset sampling frequency, video frame data is obtained based on the single-frame structured image data packet;

[0017] For the single-frame structured image data packet and the video frame data, data packets representing the identity information, texture information, structural information and deep semantic structure of the same target object in a single frame or consecutive frames are collected to obtain human image feature data.

[0018] Furthermore, the step of performing signal-to-noise estimation and delay analysis on the video frame data to generate frame quality data includes:

[0019] The signal-to-noise estimation of the video frame data includes:

[0020] The video frame data is used as the only input, and the process involves sequentially performing complete frame verification, overall signal-to-noise assessment, and read-in delay statistics.

[0021] The overall signal-to-noise ratio (SNR) assessment includes performing frame-wide statistical calculations on the brightness uniformity, pixel energy distribution, edge density variation, and background noise baseline of the video frame, and then using a whole-packet mapping function to fuse the statistical quantities into an SNR index.

[0022] Furthermore, the step of performing signal-to-noise estimation and delay analysis on the video frame data to generate frame quality data also includes:

[0023] Delay analysis of the video frame data includes:

[0024] Delay samples are formed by recording the difference between the arrival time of the frame and its timestamp, and the mean and variance of the delay are calculated.

[0025] Based on the indivisible mapping rules, the signal-to-noise ratio and delay analysis results are combined into a single quality score, generating frame quality data as a structured whole output.

[0026] Furthermore, the facial feature data undergoes a multi-level overall mapping transformation to generate surface features, enhanced features, and deep features, including:

[0027] Apply lightweight global projection mapping to facial feature data, including performing global low-dimensional linear transformation or global hash-compression mapping on the facial feature data;

[0028] After mapping, the entire packet output in multiple temporal neighborhoods is smoothed to generate surface features that express appearance contours and coarse-grained distinguishability.

[0029] Furthermore, the facial feature data undergoes a multi-level overall mapping transformation to generate surface features, enhanced features, and deep features, and also includes:

[0030] Using the same portrait feature data as input, a holistic augmentation mapping network is used to perform mid-dimensional holistic feature extraction.

[0031] The dimensional feature extraction performed through the overall augmentation mapping network includes:

[0032] The system performs overall spatial relationship modeling, overall strong and weak texture fusion, and overall contrast-sensitive mapping, and performs overall consistency calibration on the enhancement results within a time window to obtain enhanced features for pose changes and illumination fluctuations.

[0033] Furthermore, the facial feature data undergoes a multi-level overall mapping transformation to generate surface features, enhanced features, and deep features, and also includes:

[0034] The facial feature data is input into the deep reasoning unit for overall semantic abstraction and high-dimensional structure encoding, including overall contextual relationship inference, overall key region aggregation and overall semantic compression mapping, to obtain deep features that can represent the long-term stable characteristics of a person's identity.

[0035] Furthermore, after generating surface features, enhanced features, and deep features, the process also includes generating surface confidence.

[0036] Specifically, the facial feature data is used to generate an initial confidence through overall confidence extraction, and the time alignment table is used to generate a time series factor through time series consistency evaluation.

[0037] The initial confidence, the time series factor, and the frame quality data are weighted and fused to generate a surface confidence.

[0038] Furthermore, the method includes: generating the time alignment table by non-blocking time alignment of video frame data and the portrait feature data;

[0039] Overall confidence extraction refers to a data processing procedure that obtains a global confidence index that can measure the degree of confidence of a feature vector without disassembling the internal fields of the entire human face feature package.

[0040] Initial confidence refers to the degree of credibility of the current portrait feature data in terms of semantic matching for portrait recognition, independent of temporal factors and frame quality factors.

[0041] Furthermore, the method includes:

[0042] The numerical index, i.e., time series factor, is obtained by evaluating the non-blocking time alignment results between camera frame data and human facial feature data.

[0043] Without causing waiting or blocking in the current data processing flow, the system uses the timestamp field in the camera frame and the facial feature data packet to perform a non-blocking time alignment process by using a time-series sliding window and dynamic interpolation calculation to achieve time-related pairing of data.

[0044] Furthermore, the similarity measurement of the surface features and the enhanced features includes:

[0045] The surface features and the enhanced features are similarly measured using a similarity function;

[0046] The similarity functions include whole-packet cosine similarity, global Euclidean distance mapping, and matching functions based on the global projection space.

[0047] Furthermore, the stability assessment of the frame quality data includes:

[0048] The frame quality data is weighted using a sliding window, and the distribution of all weighted results within the sliding window is statistically analyzed to generate a stability evaluation result.

[0049] The distribution statistics refer to data processing that quantitatively models and statistically analyzes the variation characteristics of frame image quality data within a certain time range.

[0050] Furthermore, the step of performing distribution statistics on all overall weighted results within the sliding window to generate stability evaluation results also includes:

[0051] Each frame quality data in the sliding window is used as input, and weighting processing is performed on each frame quality data through overall weighting. The overall weighting function takes the overall quality value in the whole frame quality packet, the corresponding timestamp, and the relative position in the window as overall input parameters, and maps to a single weighting result with preset weight coefficients.

[0052] Perform distribution statistics processing on all weighted results generated within the entire sliding window. The distribution statistics include the unified calculation of the overall mean, overall variance, quantile interval and stability measure factor of the weighted results. All statistics are completed with the entire set of weighted results as an indivisible whole input.

[0053] The mean, dispersion, and stability metric obtained from statistical processing are fused together in an overall mapping manner to output a single stability assessment result.

[0054] Furthermore, the step of performing hierarchical adjustment and unified projection mapping on the surface features, the enhanced features, and the deep features according to the weighting factors to generate an initial aggregate vector includes:

[0055] The surface features, the enhancement features, and the deep features are respectively mapped using the weighting factor to generate surface adjustment coefficients, enhancement adjustment coefficients, and deep adjustment coefficients;

[0056] The surface features are used to generate surface-corrected features through the surface adjustment coefficients; the enhanced features are used to generate enhanced-corrected features through mid-dimensional mapping and the enhancement adjustment coefficients; and the deep features are used to generate deep-corrected features through deep-dimensional normalization and the deep adjustment coefficients.

[0057] An initial aggregate vector is generated by uniformly projecting the surface correction features, the enhanced correction features, and the deep correction features.

[0058] Furthermore, the surface adjustment coefficient is a single coefficient or vector set that weights and adjusts the overall numerical structure of the surface features;

[0059] The enhancement adjustment coefficient is a single coefficient or vector set that weights and adjusts the overall numerical structure of the enhancement feature;

[0060] The deep adjustment coefficient is a single coefficient or vector set that weights and adjusts the overall numerical structure of deep features;

[0061] Surface-modified features are a set of weighted modified feature vectors obtained by numerical adjustment, morphological remapping and structure preservation of surface features.

[0062] Enhanced modified features are a set of weighted modified feature vectors obtained by numerical adjustment, morphological remapping and structure preservation of enhanced features.

[0063] Deep modified features are a set of weighted modified feature vectors obtained after numerical adjustment, morphological remapping and structure preservation of deep features.

[0064] Unified projection is a global mapping mechanism based on a structured vector fusion model.

[0065] Furthermore, the difference analysis results are weighted and adjusted according to the weighting factors. It is then determined whether the weighted adjustment result is greater than a preset discrimination threshold. If so, the difference analysis, weighted adjustment results, and the initial aggregation vector are encapsulated to generate incremental features, including:

[0066] The preset discrimination threshold T is obtained by performing overall statistical analysis on the frame quality data, and its specific calculation formula is as follows:

[0067]

[0068] In the formula, T represents the preset discrimination threshold, and μQ represents the mean value of the frame quality data. Q represents the standard deviation of the frame quality data. The weighting coefficients represent the mean of the frame quality data. The mean of the frame quality data and the weighting coefficients are all normalized during the calculation.

[0069] Furthermore, the method also includes:

[0070] The preset hierarchical difference threshold is obtained by performing overall statistics on the surface confidence level:

[0071] Specifically, the entire surface confidence package is subjected to overall statistical processing, including overall mean, overall variance, and overall skewness of the entire surface confidence package, which are indivisible overall statistical measures. The overall statistics are then converted into preset hierarchical difference thresholds through a mapping function.

[0072] Furthermore, the step of weighting the incremental features according to the weighting factor and generating a callback result includes:

[0073] The incremental features and the initial aggregated vector are compared to generate a change amount through whole-pack consistency comparison, and the change amount and the weight factor are mapped and weighted to generate a weighted change amount.

[0074] The frame quality data and the surface confidence are processed by threshold adaptive processing to generate a constraint threshold, and the weighted change amount and the constraint threshold are compared by judgment to generate a callback result;

[0075] Among them, the change refers to a feature difference measure with the overall vector as the granularity;

[0076] The weighted change refers to a total numerical vector that comprehensively represents the degree of change in the aggregate state;

[0077] Threshold adaptive processing refers to the data processing process of dynamically calculating the judgment boundary value applicable to the current round of judgment based on the current frame quality data and surface confidence data, i.e., the "constraint threshold".

[0078] The constraint threshold refers to the dynamic judgment boundary value calculated by the system before each callback judgment, based on the specific quality and confidence information of the current input data, to control whether to trigger the generation of the callback result.

[0079] Based on the same inventive concept, another aspect of the embodiments of this application provides an asynchronous task callback device between a hardware codec and a GPU, including:

[0080] The data acquisition unit is used to acquire video frame data and facial feature data of the target object;

[0081] The data analysis unit is used to perform signal-to-noise estimation and delay analysis on the video frame data to generate frame quality data;

[0082] The feature transformation unit is used to perform multi-level overall mapping transformation on the portrait feature data, generate surface features, enhanced features and deep features respectively, and measure the similarity between the surface features and the enhanced features.

[0083] The weighted fusion unit is used to perform stability assessment on the frame quality data, and to perform weighted fusion on the stability assessment results and similarity measurement results to generate weight factors.

[0084] The feature determination unit is used to perform weighted determination of incremental features according to the weighting factor and generate callback results.

[0085] Based on the same inventive concept, another aspect of the embodiments of this application provides an electronic device, including: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0086] Memory, which stores computer programs;

[0087] When the processor executes a program stored in memory, it implements asynchronous task callback methods between the hardware codec and the GPU.

[0088] Based on the same inventive concept, another aspect of the embodiments of this application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements an asynchronous task callback method between a hardware codec and a GPU.

[0089] The beneficial effects of this application are as follows: Based on the above technical solution, by generating three types of adjustment coefficients for surface features and deep features according to weight factors, the contribution of each layer of features can be adaptively adjusted according to the camera source quality, temporal stability and cross-view consistency in the current scene. Then, the three types of modified features are fused through unified projection to generate an initial aggregation vector, so that the entire aggregation result has a fast response capability, thereby significantly improving the cross-view recognition accuracy of people in complex dynamic monitoring environments and the overall robustness of the system.

[0090] By classifying the weighted adjustment results and difference analysis results during the judgment process, and adaptively generating hierarchical difference thresholds based on the overall statistics of surface confidence, the system can trigger the encapsulation and output of incremental features only when there are truly significant changes in human images in a multi-camera asynchronous aggregation environment. This avoids erroneous updates caused by noise fluctuations, slight pose changes, or single-view quality anomalies in traditional methods, and enhances the stability, robustness, and real-time response capability of the entire asynchronous feature aggregation link.

[0091] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures pointed out in the description and the accompanying drawings. Attached Figure Description

[0092] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0093] Figure 1 A schematic diagram of an asynchronous task callback method between a hardware codec and a GPU is shown.

[0094] Figure 2This paper illustrates a schematic diagram of the callback result generation process in an embodiment of this application.

[0095] Figure 3 A schematic diagram of the surface confidence generation process in an embodiment of this application is shown;

[0096] Figure 4 A schematic diagram of the weight factor generation process in an embodiment of this application is shown;

[0097] Figure 5 This illustration shows a schematic diagram of the initial aggregation vector generation process in an embodiment of this application;

[0098] Figure 6 This illustration shows a schematic diagram of an asynchronous task callback device between a hardware codec and a GPU according to an embodiment of this application.

[0099] Figure 7 A schematic diagram of an electronic device according to an embodiment of this application is shown. Detailed Implementation

[0100] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0101] It should be noted that the terms "first," "second," etc., used in this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. In this application, the terms "upper," "lower," "left," "right," "front," "rear," "top," "bottom," "inner," "outer," "middle," "vertical," "horizontal," "lateral," "longitudinal," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings.

[0102] In traditional video processing architectures, the collaborative work between hardware codecs and GPUs relies on synchronous or blocking task submission mechanisms. These mechanisms lack the ability to dynamically adjust the contribution of multi-layer features, making it difficult for the feature aggregation process to adapt to environmental changes in complex scenarios such as fluctuations in video frame quality, sudden changes in lighting conditions, or local occlusion. This leads to instability in the output of feature vectors. This problem directly restricts the system's real-time response capability and feature analysis accuracy in high-concurrency multi-task environments, making it difficult for the video processing workflow to meet the needs of practical applications in terms of key performance indicators.

[0103] For example, in an urban intelligent traffic monitoring system, multiple cameras continuously collect real-time video streams from intersections. When encountering sudden changes in light intensity caused by rapid cloud movement, the signal-to-noise ratio of the video frames output by the hardware codec fluctuates non-linearly. When the GPU performs the human face feature extraction task, the similarity measurement value between surface features and enhanced features deviates significantly due to unstable frame quality. The feature aggregation module is forced to use fixed weights for full calculation, causing the aggregated vector to oscillate repeatedly in a short period of time. As a result, the system makes continuous misjudgments in the recognition results of pedestrian behavior, and the backlog of task queues leads to a continuous accumulation of response delays.

[0104] If the above problems are not addressed, the instability of feature aggregation will further spread in multi-camera concurrent processing scenarios, causing a continuous decline in the overall system throughput and potentially triggering cascading failures in the task processing chain. Ultimately, this will make it difficult to guarantee the reliability of the video analysis process, and in severe cases, it will affect the accuracy of real-time decision-making in critical application scenarios.

[0105] This application provides an asynchronous task callback method between a hardware codec and a GPU, as illustrated in this embodiment. Figure 1 ,include:

[0106] S101: Obtain video frame data and facial feature data of the target object;

[0107] S102: Perform signal-to-noise estimation and delay analysis on the video frame data to generate frame quality data;

[0108] S103: Perform multi-level overall mapping transformation on the portrait feature data to generate surface features, enhanced features and deep features respectively, and measure the similarity between the surface features and the enhanced features;

[0109] S104: Perform a stability assessment on the frame image quality data, and weight and fuse the stability assessment results and similarity measurement results to generate a weight factor;

[0110] S105: The incremental features are weighted according to the weighting factor to generate a callback result.

[0111] The asynchronous task callback method between the hardware codec and the GPU in the embodiments of this specification is typically used in servers, which can be cloud servers or local servers. Servers can be electronic devices such as laptops, desktop computers, tablets, and all-in-one computers.

[0112] Specifically, the process of weighting the incremental features according to the weighting factors to generate a callback result also includes:

[0113] Based on the weighting factors, the surface features, the enhanced features, and the deep features are respectively subjected to hierarchical adjustment and unified projection mapping to generate an initial aggregate vector;

[0114] The surface features, the enhanced features, and the deep features are subjected to pairwise difference analysis, and the difference analysis results are weighted and adjusted according to the weighting factor. It is determined whether the weighted adjustment result is greater than the preset discrimination threshold. If so, the difference analysis, the weighted adjustment result, and the initial aggregation vector are encapsulated to generate incremental features.

[0115] The preset discrimination threshold is obtained by performing overall statistical analysis on the frame quality data.

[0116] Human face feature data refers to data packets used to characterize the identity information, texture information, structural information, and deep semantic structure of the same target object in a single frame or consecutive frames;

[0117] Signal-to-noise estimation and delay analysis refers to the overall analysis of video frame data captured by the camera and preprocessed by the hardware codec, thereby evaluating the ratio between useful signals and background noise in the image, and statistically modeling the time difference experienced by each video frame data from the time of acquisition to the time it is read and processed by the system.

[0118] Frame quality data refers to a set of parameters used to measure the signal effectiveness, availability, and stability of a frame at a specific point in time.

[0119] Multi-level holistic mapping transformation refers to the system performing a three-level semantic transformation on the facial feature data based on a unified input structure without splitting the internal fields of the facial feature data.

[0120] Surface features refer to fast-response feature vectors obtained by the system performing a first-level overall mapping transformation on the input human facial feature data;

[0121] Enhanced features refer to the mid-level feature representation obtained by performing non-linear overall mapping and temporal context-aware optimization on the basis of the whole package of human portrait features;

[0122] Deep features refer to a class of high-level feature representations generated after high-dimensional, semantic-level extraction of human facial feature data;

[0123] Similarity measurement refers to the process by which the system compares two whole human image feature data packets (surface features and enhancement features, respectively) on the machine side in a non-splitting, whole vector manner.

[0124] Stability assessment refers to a comprehensive statistical calculation of the frame sequence's performance in terms of imaging quality, acquisition delay, illumination consistency, and inter-frame stability, based on the overall performance of the target object's video frame data within the same time window.

[0125] Weighting factor is a data structure used to characterize the overall confidence level between image frame quality and portrait matching confidence;

[0126] Hierarchical adjustment and unified projection mapping refer to the structural scale matching and weight recalibration of feature data from different extraction depths (i.e., surface features, enhanced features, and deep features) before feature fusion, so as to map the multi-level portrait features after hierarchical adjustment to the same semantic space.

[0127] The initial aggregation vector refers to a unified feature representation vector generated based on the structural computational relationship between multi-level feature data and weight factors, used to express the current state of the target object;

[0128] Differential analysis refers to the data processing procedure used to measure the differences in expression among surface features, augmented features, and deep features on the same target object;

[0129] The preset discrimination threshold refers to an integer or floating-point numerical threshold parameter used to determine whether the difference is significant, which is obtained by performing overall statistical analysis on the frame quality data;

[0130] Incremental features refer to structured result data used to characterize whether the current frame's identification state has undergone a valid change relative to the previous aggregated state;

[0131] Overall statistical analysis refers to the unified statistical derivation process performed on the quality data of multiple consecutively acquired frames;

[0132] The callback result refers to the unique output data entity generated by the system based on the weight factors and incremental features after multi-layer feature processing, difference analysis, and weighted judgment are completed between the hardware codec and the GPU.

[0133] This solution generates frame image quality data by performing overall signal-to-noise estimation and delay analysis on video frame data. This enables the system to dynamically adjust the intensity of subsequent feature processing based on image realism and temporal stability, thereby significantly reducing the misleading effects of noisy frames, blurred frames, and delayed frames on feature calculation.

[0134] Secondly, by performing multi-level overall mapping transformation on the facial feature data, including surface features, enhanced features, and deep features, and by measuring the similarity between surface features and enhanced features, the system can capture facial appearance information from different semantic levels and quickly identify local changes, thereby improving the effective fusion accuracy across different viewpoints and camera scenarios.

[0135] Furthermore, by generating weighting factors based on frame quality stability assessment and similarity measurement results, and using these weighting factors to perform hierarchical adjustment and unified projection mapping on multi-layer features to generate an initial aggregated vector, the fusion process becomes adaptive. This allows it to automatically increase the weights of high-confidence features in unstable scenarios such as sudden changes in illumination, occlusion, and blurring, thereby significantly improving the reliability of the aggregated representation.

[0136] Subsequently, by performing pairwise difference analysis on surface features, enhanced features, and deep features, and combining this with weighted adjustments based on weighting factors, the system can accurately perceive subtle changes in a person's state or appearance. Furthermore, by using a dynamic discrimination threshold obtained through overall statistics, false positives are reduced, making the generation of incremental features more robust. This achieves both high efficiency and accuracy in recognition updates.

[0137] Finally, by performing weighted judgment on incremental features to generate callback results, the system can quickly output accurate, low-noise, and traceable event results in the asynchronous task chain, which significantly improves the response speed and decision reliability in real-time monitoring, facial tracking, and risk identification scenarios.

[0138] Specifically, in step S101, acquiring the video frame data and facial feature data of the target object includes:

[0139] Based on the single-frame structured image data packet of the target object acquired by the front-end camera device at a preset sampling frequency, video frame data is obtained based on the single-frame structured image data packet;

[0140] For the single-frame structured image data packet and the video frame data, data packets representing the identity information, texture information, structural information and deep semantic structure of the same target object in a single frame or consecutive frames are collected to obtain human image feature data.

[0141] It should be noted that video frame data of the target object is acquired through intelligent camera devices; the video frame data includes, but is not limited to, timestamps, camera identifiers, and image numbers; and facial feature data of the target object is acquired through video capture devices; the facial feature data includes, but is not limited to, camera IDs and capture timestamps.

[0142] Specifically, in step S102, the step of performing signal-to-noise estimation and delay analysis on the video frame data to generate frame quality data includes:

[0143] The signal-to-noise estimation of the video frame data includes:

[0144] The video frame data is used as the only input, and the process involves sequentially performing complete frame verification, overall signal-to-noise assessment, and read-in delay statistics.

[0145] The overall signal-to-noise ratio (SNR) assessment includes performing frame-wide statistical calculations on the brightness uniformity, pixel energy distribution, edge density variation, and background noise baseline of the video frame, and then using a whole-packet mapping function to fuse the statistical quantities into an SNR index.

[0146] In step S102, the step of performing signal-to-noise estimation and delay analysis on the video frame data to generate frame quality data further includes:

[0147] Delay analysis of the video frame data includes:

[0148] Delay samples are formed by recording the difference between the arrival time of the frame and its timestamp, and the mean and variance of the delay are calculated.

[0149] Based on the indivisible mapping rules, the signal-to-noise ratio and delay analysis results are combined into a single quality score, generating frame quality data as a structured whole output.

[0150] Specifically, after receiving video frame data, the video frame data is used as the only input. The system performs three types of processing in sequence: complete frame verification, overall signal-to-noise ratio evaluation, and read-in delay statistics. The overall signal-to-noise ratio evaluation includes performing whole-frame statistical calculations on the brightness uniformity, pixel energy distribution, edge density changes, and background noise baseline of the video frame, and using a whole-packet mapping function to fuse the statistical quantities into a signal-to-noise ratio.

[0151] Delay analysis forms delay samples by recording the difference between the arrival time of the frame and its timestamp, and calculates the delay mean and variance. Then, the system combines the signal-to-noise ratio and the delay analysis results into a single quality score using an indivisible mapping rule, and finally generates frame quality data as a structured whole output.

[0152] In step S103, the facial feature data undergoes a multi-level overall mapping transformation to generate surface features, enhanced features, and deep features, including:

[0153] Apply lightweight global projection mapping to facial feature data, including performing global low-dimensional linear transformation or global hash-compression mapping on the facial feature data;

[0154] After mapping, the entire packet output in multiple temporal neighborhoods is smoothed to generate surface features that express appearance contours and coarse-grained distinguishability.

[0155] Specifically, in step S103, the facial feature data undergoes a multi-level overall mapping transformation to generate surface features, enhanced features, and deep features, and further includes:

[0156] Using the same portrait feature data as input, a holistic augmentation mapping network is used to perform mid-dimensional holistic feature extraction.

[0157] The dimensional feature extraction performed through the overall augmentation mapping network includes:

[0158] The system performs overall spatial relationship modeling, overall strong and weak texture fusion, and overall contrast-sensitive mapping, and performs overall consistency calibration on the enhancement results within a time window to obtain enhanced features for pose changes and illumination fluctuations.

[0159] Specifically, in step S103, the facial feature data undergoes a multi-level overall mapping transformation to generate surface features, enhanced features, and deep features, and further includes:

[0160] The facial feature data is input into the deep reasoning unit for overall semantic abstraction and high-dimensional structure encoding, including overall contextual relationship inference, overall key region aggregation and overall semantic compression mapping, to obtain deep features that can represent the long-term stable characteristics of a person's identity.

[0161] Specifically, when generating surface features, a lightweight global projection mapping is applied to the portrait feature data, including performing a global low-dimensional linear transformation or a global hash-based compression mapping on the portrait feature data, and after mapping, the entire packet output in multiple time neighborhoods is smoothed to generate surface features that express appearance contours and coarse-grained discriminability.

[0162] When generating enhanced features, the same portrait feature data is used as input, and the overall feature extraction of the mid-dimensional dimension is performed through the overall enhancement mapping network. This process includes whole bag spatial relationship modeling, overall strong and weak texture fusion and overall contrast sensitive mapping. The overall consistency calibration of the enhancement results is performed within the time window to obtain enhanced features that are more robust to pose changes and illumination fluctuations.

[0163] When generating deep features, the portrait feature data is input into the deep inference unit for overall semantic abstraction and high-dimensional structure encoding, including overall context relationship inference, overall key region aggregation and overall semantic compression mapping, and finally obtains deep features that can represent the long-term stable features of a person's identity.

[0164] Specifically, see Figure 3 After generating surface features, enhanced features, and deep features, the process also includes generating surface confidence.

[0165] Specifically, the facial feature data is used to generate an initial confidence through overall confidence extraction, and the time alignment table is used to generate a time series factor through time series consistency evaluation.

[0166] The initial confidence, the time series factor, and the frame quality data are weighted and fused to generate a surface confidence.

[0167] Specifically, the time alignment table is generated by non-blocking time alignment of video frame data and the human portrait feature data;

[0168] Overall confidence extraction refers to a data processing procedure that obtains a global confidence index that can measure the degree of confidence of a feature vector without disassembling the internal fields of the entire human face feature package.

[0169] Initial confidence refers to the degree of credibility of the current portrait feature data in terms of semantic matching for portrait recognition, independent of temporal factors and frame quality factors.

[0170] The timing factor is a numerical indicator obtained by evaluating the non-blocking time alignment results between camera frame data and human facial feature data.

[0171] Non-blocking time alignment refers to a data processing process in which the system, based on the timestamp fields in the camera frame and the facial feature data packet, performs time-related pairing calculations through a time-series sliding window and dynamic interpolation without causing waiting or blocking to the current data processing flow.

[0172] Furthermore, the method includes:

[0173] The numerical index, i.e., time series factor, is obtained by evaluating the non-blocking time alignment results between camera frame data and human facial feature data.

[0174] Without causing waiting or blocking in the current data processing flow, the system uses the timestamp field in the camera frame and the facial feature data packet to perform a non-blocking time alignment process by using a time-series sliding window and dynamic interpolation calculation to achieve time-related pairing of data.

[0175] Specifically, the overall integrity of the facial feature data is checked and its source timestamp and camera source identifier are confirmed to be valid. Then, the overall amplitude normalization of the facial feature data is performed to eliminate amplitude differences caused by different acquisition conditions. Subsequently, the overall confidence mapping is used to perform a one-time calculation on the facial feature data to generate initial confidence.

[0176] The time alignment table is evaluated for time series consistency to generate a time series factor Ft, and its specific calculation formula is as follows:

[0177]

[0178] In the formula, N represents the number of alignment records in the time alignment table, and Di represents the overall offset corresponding to the i-th alignment mapping record in the time alignment table. The value represents the overall average offset of the entire packet in the time alignment table, α represents the consistency sensitivity coefficient, and exp represents the natural exponential mapping. All of the above data have been normalized during the calculation.

[0179] The initial confidence, the time series factor, and the frame quality data are weighted and fused to generate a surface confidence.

[0180] The time alignment table is generated by non-blocking time alignment of the video frame data and the portrait feature data.

[0181] The timestamp and source identifier of the overall video frame data packet and the overall human face feature data packet are obtained respectively, and written into the alignment cache without splitting the internal fields of the two types of data.

[0182] Subsequently, a sliding time window is constructed based on the data packets that have arrived in the alignment cache. The absolute difference between the overall timestamp of the video frame data and the overall timestamp of the facial feature data is compared to determine whether they are within the same window range. The window width is adaptively adjusted by combining the statistics of the most recent data arrival intervals, so that the window automatically widens when the data delay fluctuates greatly and automatically shrinks when the timing is stable. During window alignment, the system uses an overall mapping interpolation method to establish a correspondence between two data packets that have not yet completely overlapped. That is, while maintaining the integrity of the data, the system maps the time alignment offset value of the two according to the time difference and generates a mapping entry containing video frame data reference, feature data reference, and offset. The above window search, overall mapping, and offset generation operations are continuously executed on all data packets entering the window in chronological order, and finally a time alignment table composed of multiple mapping entries is formed.

[0183] Specifically, in step S103, the similarity measurement of the surface features and the enhanced features includes:

[0184] The surface features and the enhanced features are similarly measured using a similarity function;

[0185] The similarity functions include whole-packet cosine similarity, global Euclidean distance mapping, and matching functions based on the global projection space.

[0186] This application's embodiments generate surface confidence by performing overall confidence extraction on facial feature data, temporal consistency evaluation on the time alignment table, and weighted fusion with frame quality data. This enables the system to comprehensively quantify the reliability of surface features from three dimensions—feature confidence, temporal correspondence, and frame imaging quality—under an asynchronous task submission architecture, thereby significantly improving the feature reliability judgment capability under cross-camera conditions. Specifically, the temporal factor obtained through non-blocking time alignment can effectively reduce feature deviations caused by inconsistent frame arrival order between different cameras, ensuring that surface confidence maintains temporal consistency in an asynchronous environment. Furthermore, the introduction of frame quality can further suppress unstable features caused by illumination, motion blur, or noise, making the generated surface confidence more robust and reliable, thereby improving the system's recognition performance and response efficiency in complex multi-source video environments.

[0187] Specifically, see Figure 1 and Figure 4 In step S104, the stability assessment of the frame quality data includes:

[0188] The frame quality data is weighted using a sliding window, and the distribution of all weighted results within the sliding window is statistically analyzed to generate a stability evaluation result.

[0189] The distribution statistics are data processing methods that quantitatively model and statistically analyze the variation characteristics of frame image quality data within a certain time range.

[0190] The step of performing distribution statistics on all weighted results within the sliding window to generate stability evaluation results also includes:

[0191] Each frame quality data in the sliding window is used as input, and weighting processing is performed on each frame quality data through overall weighting. The overall weighting function takes the overall quality value in the whole frame quality packet, the corresponding timestamp, and the relative position in the window as overall input parameters, and maps to a single weighting result with preset weight coefficients.

[0192] Perform distribution statistics processing on all weighted results generated within the entire sliding window. The distribution statistics include the unified calculation of the overall mean, overall variance, quantile interval and stability measure factor of the weighted results. All statistics are completed with the entire set of weighted results as an indivisible whole input.

[0193] The mean, dispersion, and stability metric obtained from statistical processing are fused together in an overall mapping manner to output a single stability assessment result.

[0194] The stability assessment results, similarity measurement results, and surface confidence are weighted, fused, and normalized to generate weight factors.

[0195] This application embodiment, through stability assessment of frame image quality data and weighted fusion processing of stability assessment results, similarity measurement results, and surface confidence, enables the system to adaptively identify more reliable feature sources and dynamically adjust weight factors in complex scenarios with asynchronous input from multiple cameras, changes in lighting, occlusion fluctuations, and uneven frame image quality. This effectively improves the robustness and discriminativeness of cross-camera feature aggregation. Specifically, the overall weighting and distribution statistics of the sliding window can automatically smooth short-term abnormal frames during quality fluctuations, achieving robust output of stability assessment results. Surface confidence and similarity measurement jointly participate in weight adjustment during the fusion process, enabling weight factors to more accurately reflect the credibility of each feature under the current time-series conditions. This reduces the probability of misjudgment and improves feature consistency in subsequent feature fusion, aggregation updates, and incremental judgments, ultimately achieving higher cross-view recognition accuracy and overall system anti-interference performance.

[0196] Specifically, see Figure 5 The step of performing hierarchical adjustment and unified projection mapping on the surface features, enhanced features, and deep features according to the weight factors to generate an initial aggregate vector includes:

[0197] The surface features, the enhancement features, and the deep features are respectively mapped using the weighting factor to generate surface adjustment coefficients, enhancement adjustment coefficients, and deep adjustment coefficients;

[0198] The surface features are used to generate surface-corrected features through the surface adjustment coefficients; the enhanced features are used to generate enhanced-corrected features through mid-dimensional mapping and the enhancement adjustment coefficients; and the deep features are used to generate deep-corrected features through deep-dimensional normalization and the deep adjustment coefficients.

[0199] An initial aggregate vector is generated by uniformly projecting the surface correction features, the enhanced correction features, and the deep correction features.

[0200] The surface adjustment coefficient is a single coefficient or vector set that weights and adjusts the overall numerical structure of the surface features.

[0201] The enhancement adjustment coefficient is a single coefficient or vector set that weights and adjusts the overall numerical structure of the enhancement feature;

[0202] The deep adjustment coefficient is a single coefficient or vector set that weights and adjusts the overall numerical structure of deep features;

[0203] Surface-modified features are a set of weighted modified feature vectors obtained by numerical adjustment, morphological remapping and structure preservation of surface features.

[0204] Enhanced modified features are a set of weighted modified feature vectors obtained by numerical adjustment, morphological remapping and structure preservation of enhanced features.

[0205] Deep modified features are a set of weighted modified feature vectors obtained after numerical adjustment, morphological remapping and structure preservation of deep features.

[0206] Unified projection is a global mapping mechanism based on a structured vector fusion model.

[0207] Specifically, taking surface features and weight factors as input, normalization, overall similarity evaluation and weight interpolation are performed through the overall mapping function, and the surface adjustment coefficient is output.

[0208] Then, using the enhanced features and the same weighting factor as input, nonlinear amplification suppression, stability correction, and range compression are performed through mapping paths with the same structure but different parameters, and the enhanced adjustment coefficient is output.

[0209] Using deep features and weighting factors as input, the system performs overall sensitivity adjustment and hierarchical bias compensation by utilizing deep correlation mapping, and outputs deep adjustment coefficients.

[0210] The surface features are multiplied using the surface adjustment coefficients to generate surface modified features; the enhanced features are multiplied using the mid-dimensional mapping and the enhancement adjustment coefficients to generate enhanced modified features; and the deep features are multiplied using deep-dimensional normalization and the deep adjustment coefficients to generate deep modified features.

[0211] An initial aggregate vector is generated by uniformly projecting the surface correction features, the enhanced correction features, and the deep correction features.

[0212] This application's embodiments generate surface adjustment coefficients, enhancement adjustment coefficients, and deep adjustment coefficients respectively under weighted factor control for surface features, enhancement features, and deep features. This enables the system to automatically adjust the influence of different feature layers based on real-time frame quality, feature confidence, and cross-layer consistency differences. This enhances the sensitivity of the aggregation results to high-confidence perspectives under conditions such as lighting changes, occlusion, and multi-view mixing. Secondly, by mapping surface features, enhancement features, and deep features to surface correction features, enhancement correction features, and deep correction features respectively, the differences in scale, noise, and distribution among the features of each layer are effectively offset, allowing multi-layer information to be uniformly incorporated into the same projection space. Thirdly, by generating an initial aggregation vector through unified projection, the three layers of corrected features achieve deep fusion in the global space, improving the discriminativeness and stability of cross-camera feature aggregation, reducing feature drift caused by perspective differences, and ultimately significantly improving the accuracy of the system in the recognition, tracking, and subsequent decision-making stages while effectively reducing the false trigger rate. Overall, this enhances the robustness and real-time performance of the asynchronous multi-source feature fusion system.

[0213] Furthermore, the difference analysis results are weighted and adjusted according to the weighting factors. It is then determined whether the weighted adjustment result is greater than a preset discrimination threshold. If so, the difference analysis, weighted adjustment results, and the initial aggregation vector are encapsulated to generate incremental features, including:

[0214] The preset discrimination threshold T is obtained by performing overall statistical analysis on the frame quality data, and its specific calculation formula is as follows:

[0215]

[0216] In the formula, T represents the preset discrimination threshold, and μQ represents the mean value of the frame quality data. Q represents the standard deviation of the frame quality data. The weighting coefficients represent the mean of the frame quality data. The mean of the frame quality data and the weighting coefficients are all normalized during the calculation.

[0217] The preset hierarchical difference threshold is obtained by performing overall statistics on the surface confidence level:

[0218] Specifically, the entire surface confidence package is subjected to overall statistical processing, including overall mean, overall variance, and overall skewness of the entire surface confidence package, which are indivisible overall statistical measures. The overall statistics are then converted into preset hierarchical difference thresholds through a mapping function.

[0219] Specifically, the following describes how to determine whether the weighted adjustment result is greater than a preset discrimination threshold. If so, the difference analysis result, the weighted adjustment result, and the initial aggregation vector are encapsulated to generate incremental features, specifically including:

[0220] If the weighted adjustment result is greater than a preset discrimination threshold, then the difference analysis result is further determined to be greater than a preset hierarchical difference threshold. If so, the difference analysis result, the weighted adjustment result, and the initial aggregation vector are encapsulated to generate incremental features.

[0221] The preset hierarchical difference threshold is obtained by performing overall statistics on the surface confidence.

[0222] Among them, the preset hierarchical difference threshold refers to the key discriminant parameter used to determine whether there is a significant change in the initial aggregation vector that needs to be updated, which is obtained by performing overall statistics on the surface confidence.

[0223] Overall statistics refers to the batch summary and analysis of the acquired surface confidence data, and the generation of a representative hierarchical difference threshold that can be used for subsequent discrimination through statistical feature extraction methods.

[0224] The above content will be described in detail below:

[0225] If the weighted adjustment result is greater than a preset discrimination threshold, then the difference analysis result is further determined to be greater than a preset hierarchical difference threshold. If so, the difference analysis result, the weighted adjustment result, and the initial aggregation vector are encapsulated to generate incremental features.

[0226] The preset discrimination threshold T is obtained by performing overall statistical analysis on the frame quality data, and its specific calculation formula is as follows:

[0227]

[0228] In the formula, μQ represents the mean value of the frame quality data. Q represents the standard deviation of the frame quality data. This represents the weighting coefficients; all the data above have been normalized during the calculation.

[0229] The preset hierarchical difference threshold is obtained by performing overall statistics on the surface confidence level.

[0230] Perform overall statistical processing on the entire surface confidence package, including overall mean, overall variance, and overall skewness of the entire surface confidence package, which are indivisible overall statistical measures. Then, convert the above overall statistics into preset hierarchical difference thresholds through a mapping function.

[0231] This application embodiment compares the weighted adjustment result with a preset discrimination threshold during the incremental judgment stage, and further performs dual screening using the difference analysis result and the hierarchical difference threshold obtained from the overall statistics of surface confidence. This enables more accurate change recognition within the data stream, allowing the system to encapsulate and output incremental features only when a significant shift in human features actually occurs, avoiding false triggers caused by noise, transient occlusion, or low-quality frames. Furthermore, by encapsulating the difference analysis result, weighted adjustment result, and initial aggregation vector into a unified incremental feature package, it ensures that the GPU and codec perform consistency analysis and version updates based on a single and reliable data entry point in subsequent data processing, reducing redundant calculations and invalid callbacks. Especially when the hierarchical difference threshold dynamically adapts to scene changes based on the overall statistics of surface confidence, the system can automatically suppress misjudgments under low confidence conditions and strengthen the contribution of high confidence inputs, thereby improving the stability, accuracy, and update efficiency of cross-camera human tracking. Ultimately, this achieves the technical effects of reduced latency, improved resource utilization, and more accurate aggregation library version management.

[0232] Specifically, the following describes the pairwise difference analysis of the surface features, the enhanced features, and the deep features, and the weighted adjustment of the difference analysis results according to the weighting factors, including:

[0233] Perform pairwise difference analysis on the surface features, the enhancement features, and the deep features;

[0234] The difference analysis results are weighted and adjusted based on the surface adjustment coefficient, the enhancement adjustment coefficient, and the deep adjustment coefficient to generate a weighted adjustment result.

[0235] The above content will be described in detail below:

[0236] Perform pairwise difference analysis on the surface features, the enhanced features, and the deep features:

[0237] Using the entire package of surface features and the entire package of enhanced features as input, a first difference index is calculated using an overall similarity measurement function (such as the cosine difference of the entire package or the overall hash difference); using the entire package of surface features and the entire package of deep features as input, a second difference index is calculated using the same overall difference measurement framework; using the entire package of enhanced features and the entire package of deep features as input, a third difference index is calculated while keeping the entire package input intact.

[0238] The first, second, and third difference indicators are combined in a structured manner to generate difference analysis results;

[0239] The difference analysis results are weighted and adjusted based on the surface adjustment coefficient, the enhancement adjustment coefficient, and the depth adjustment coefficient to generate the weighted adjustment result Radj. The specific calculation formula is as follows:

[0240]

[0241] In the formula, C represents the difference analysis result, δ represents the surface adjustment coefficient, β represents the enhancement adjustment coefficient, and γ represents the deep adjustment coefficient. All the above data have been normalized during the calculation.

[0242] In the process of performing pairwise difference analysis on surface features, enhanced features, and deep features and weighting them with weighting factors, this application can leverage the differences in information granularity, robustness, and spatiotemporal stability of different feature levels to achieve a refined measurement of the consistency of portrait features under cross-viewpoint, cross-illumination, and partial occlusion conditions. By applying hierarchical weights consisting of surface adjustment coefficients, enhanced adjustment coefficients, and deep adjustment coefficients to the difference results, the system can highlight more reliable or stable feature levels when feature conflicts or noise interference exist, causing the difference measurement results to converge towards the trend of the real target change. This significantly improves the judgment accuracy of multi-layer feature fusion, reduces erroneous aggregation and unnecessary computation backtracking caused by misjudgments, improves the overall recognition reliability in complex scenes, and provides a more robust input basis for subsequent aggregation updates and callback decisions.

[0243] Specifically, in step S105, the step of weighting the incremental features according to the weighting factor and generating a callback result includes:

[0244] The incremental features and the initial aggregated vector are compared to generate a change amount through whole-pack consistency comparison, and the change amount and the weight factor are mapped and weighted to generate a weighted change amount.

[0245] The frame quality data and the surface confidence are processed by threshold adaptive processing to generate a constraint threshold, and the weighted change amount and the constraint threshold are compared by judgment to generate a callback result;

[0246] Among them, the change refers to a feature difference measure with the overall vector as the granularity;

[0247] The weighted change refers to a total numerical vector that comprehensively represents the degree of change in the aggregate state;

[0248] Threshold adaptive processing refers to the data processing process of dynamically calculating the judgment boundary value applicable to the current round of judgment based on the current frame quality data and surface confidence data, i.e., the "constraint threshold".

[0249] The constraint threshold refers to the dynamic judgment boundary value calculated by the system before each callback judgment, based on the specific quality and confidence information of the current input data, to control whether to trigger the generation of the callback result.

[0250] The following describes how to weight the incremental features based on the weighting factors and generate callback results, specifically including:

[0251] The incremental features and the initial aggregated vector are compared to generate a change amount through whole-pack consistency comparison, and the change amount and the weight factor are mapped and weighted to generate a weighted change amount.

[0252] The frame quality data and the surface confidence are processed by threshold adaptive processing to generate a constraint threshold, and the weighted change amount and the constraint threshold are compared by judgment to generate a callback result.

[0253] Among them, the change refers to a feature difference measure with the overall vector as the granularity;

[0254] The weighted change refers to a total numerical vector that comprehensively represents the degree of change in the aggregate state;

[0255] Threshold adaptive processing refers to the data processing process that dynamically calculates the decision boundary value (i.e., "constraint threshold") applicable to the current round of decision based on the current frame quality data and surface confidence data (both of which are whole packet inputs);

[0256] The constraint threshold refers to the dynamic judgment boundary value calculated by the system before each callback judgment, based on the specific quality and confidence information of the current input data, to control whether to trigger the generation of the callback result.

[0257] The above content will be described in detail below:

[0258] The incremental features and the initial aggregated vector are subjected to full cosine similarity, full Euclidean distance or other full distance metric functions to generate changes;

[0259] The surface adjustment coefficient, the enhancement adjustment coefficient, and the deep adjustment coefficient are weighted and averaged, and the entire change amount is multiplied according to the weighted average result to generate a weighted change amount;

[0260] Constraint thresholds are generated by mapping the frame quality data and the surface confidence to the entire packet.

[0261] If the weighted change is greater than the constraint threshold, then the difference between the weighted change and the constraint threshold is calculated, and the initial aggregation vector is updated based on the difference. Then, the updated initial aggregation vector and the incremental feature are encapsulated together to generate a callback result.

[0262] This solution utilizes the change amount obtained from whole-packet consistency comparison to accurately reflect the overall differences between aggregated vectors of different periods without field splitting. Secondly, through the overall mapping and weighted correction of weighting factors, the weighted change amount can adaptively highlight the data contribution from high-quality perspectives and suppress interference from low-quality or noisy perspectives, thereby significantly improving the stability and reliability of cross-view feature change judgment. Thirdly, the constraint threshold generated by threshold adaptation based on frame quality data and surface confidence can dynamically adjust the judgment sensitivity according to the current video quality status and feature reliability, effectively avoiding false triggers caused by lighting, occlusion, or camera fluctuations. Finally, the callback result obtained by the system through the judgment comparison of weighted change amount and constraint threshold can indicate real human image change events with lower latency and higher accuracy, thereby improving the overall reliability and response efficiency of asynchronous aggregation and callback links, and significantly improving the accuracy and timeliness of cross-camera tracking, recognition, and protection decisions.

[0263] The above describes the weighted judgment of the incremental features based on the weighting factors to generate a callback result. The following describes the generation of change amounts by comparing the incremental features and the initial aggregation vector through a whole-pack consistency comparison, and the generation of weighted change amounts by overall mapping and weighting correction of the change amounts and the weighting factors. Specifically, this includes:

[0264] The incremental features and the initial aggregation vector are compared using a whole-pack consistency comparison to generate the change amount;

[0265] The surface adjustment coefficient, the enhancement adjustment coefficient, and the deep adjustment coefficient are weighted and averaged. Based on the weighted average result, the entire change amount is processed by global norm calculation and statistical normalization to generate a weighted change amount.

[0266] Among them, the weighted average refers to merging the values ​​of three whole scalars of three types of hierarchical adjustment coefficients—"surface adjustment coefficient", "enhanced adjustment coefficient" and "deep adjustment coefficient"—and assigning weights to the three separately by setting a weighting strategy, and finally outputting a single weighted result.

[0267] Overall norm calculation and statistical normalization refers to the full-dimensional measurement of the difference between two whole-pack vector data on the machine side, which is used to quantify the overall deviation between data. Its calculation is not based on a single dimension but covers all vector dimensions in the whole-pack structure. The calculated change is standardized according to a certain statistical distribution or dynamic context value.

[0268] This part has already been described in detail above, so I will not repeat it here.

[0269] This scheme performs an overall consistency comparison between incremental features and the initial aggregated vector, thereby accurately characterizing the variation range of cross-camera fusion results in the form of a whole-pack difference metric. Subsequently, it uses the weighted average result of the surface adjustment coefficient, enhancement adjustment coefficient, and deep adjustment coefficient to perform overall norm calculation and statistical normalization on the change, so that features at different levels obtain a dynamic and adaptive contribution ratio in the weight correction. This effectively suppresses the amplification effect of single-view noise or local feature anomalies on the final judgment, significantly improves the robustness of incremental judgment to environmental disturbances, lighting changes, and view inconsistencies, and improves the accuracy of aggregation update triggering and the overall system response reliability.

[0270] The above describes how the incremental features and the initial aggregated vector are compared to generate a change amount through whole-packet consistency comparison, and how the change amount and the weight factor are mapped and weighted to generate a weighted change amount. The following describes how the frame quality data and the surface confidence are processed using threshold adaptive processing to generate a constraint threshold, and how the weighted change amount and the constraint threshold are compared to generate a callback result, specifically including:

[0271] Constraint thresholds are generated by mapping the frame quality data and the surface confidence to the entire packet.

[0272] If the weighted change is greater than the constraint threshold, then the difference between the weighted change and the constraint threshold is calculated, and the initial aggregation vector is updated based on the difference. Then, the updated initial aggregation vector and the incremental feature are encapsulated together to generate a callback result.

[0273] This part has already been described in detail above, so I will not repeat it here.

[0274] This solution generates an adaptive constraint threshold by mapping frame quality data and surface confidence to a whole package. This threshold is then used to dynamically determine the weighted change, enabling the system to automatically adjust callback triggering conditions under different lighting, occlusion, jitter, and asynchronous cross-camera environments, achieving real-time sensitivity control to environmental fluctuations. When the weighted change exceeds the constraint threshold, the system performs a targeted update on the initial aggregated vector based on the difference and encapsulates it together with the incremental features into a "callback result." This ensures that the callback is only triggered when there is a real and significant change in cross-source information, avoiding false triggers and redundant callbacks. This process not only enhances the stability of cross-camera feature aggregation but also makes the callback chain robust and accurate, significantly improving the system's real-time response capability, recognition accuracy, and resource utilization efficiency in complex dynamic monitoring environments.

[0275] Based on the same inventive concept, another aspect of the embodiments of this application provides an asynchronous task callback device between a hardware codec and a GPU, see [link to relevant documentation]. Figure 6 ,include:

[0276] Data acquisition unit 201 is used to acquire video frame data and facial feature data of the target object;

[0277] Data analysis unit 202 is used to perform signal-to-noise estimation and delay analysis on the video frame data to generate frame quality data;

[0278] The feature transformation unit 203 is used to perform multi-level overall mapping transformation on the portrait feature data to generate surface features, enhanced features and deep features respectively, and to measure the similarity between the surface features and the enhanced features.

[0279] The weighted fusion unit 204 is used to perform stability assessment on the frame quality data, and to perform weighted fusion on the stability assessment results and similarity measurement results to generate weight factors.

[0280] The feature determination unit 205 is used to perform weighted determination of incremental features according to the weighting factor and generate a callback result.

[0281] Based on the same inventive concept, this disclosure also provides an electronic device 161, see [link to previous document]. Figure 7 It includes a processor 164, a communication interface 165, a memory 162, and a communication bus, wherein the processor 164, the communication interface 165, and the memory 162 communicate with each other through the communication bus;

[0282] Memory 162 stores computer program 163;

[0283] When processor 164 executes the program stored in memory 162, it implements asynchronous task callback methods between the hardware codec and the GPU.

[0284] The aforementioned communication bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc.

[0285] The communication interface 165 is used for communication between the aforementioned electronic device 161 and other devices.

[0286] The memory 162 may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory 162 may also be at least one storage device located remotely from the aforementioned processor 164.

[0287] The processor 164 mentioned above can be a general-purpose processor 164, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0288] Based on the same inventive concept, another aspect of the present disclosure provides a computer-readable storage medium storing a computer program 163, which, when executed by a processor 164, implements an asynchronous task callback method between a hardware codec and a GPU.

[0289] The computer-readable storage medium may be included in the device / apparatus described in the above embodiments; or it may exist independently and not assembled into the device / apparatus. The computer-readable storage medium carries one or more programs that, when executed, implement the asynchronous task callback method between the hardware codec and the GPU according to embodiments of this disclosure.

[0290] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for asynchronous task callback between a hardware codec and a GPU, characterized in that, include: Acquire video frame data and facial feature data of the target object; Signal-to-noise estimation and delay analysis are performed on the video frame data to generate frame quality data; The facial feature data is subjected to a multi-level overall mapping transformation to generate surface features, enhanced features and deep features respectively, and the similarity between the surface features and the enhanced features is measured. The frame quality data is subjected to stability evaluation, and the stability evaluation results and similarity measurement results are weighted and fused to generate weight factors; The incremental features are weighted according to the weighting factors to generate callback results.

2. The method according to claim 1, characterized in that, The process of weighting and determining the incremental features based on the weighting factors to generate a callback result also includes: Based on the weighting factors, the surface features, the enhanced features, and the deep features are respectively subjected to hierarchical adjustment and unified projection mapping to generate an initial aggregate vector; The surface features, the enhanced features, and the deep features are subjected to pairwise difference analysis, and the difference analysis results are weighted and adjusted according to the weighting factor. It is determined whether the weighted adjustment result is greater than the preset discrimination threshold. If so, the difference analysis, the weighted adjustment result, and the initial aggregation vector are encapsulated to generate incremental features. The preset discrimination threshold is obtained by performing overall statistical analysis on the frame quality data.

3. The method according to claim 1, characterized in that, The acquisition of video frame data and facial feature data of the target object includes: Based on the single-frame structured image data packet of the target object acquired by the front-end camera device at a preset sampling frequency, video frame data is obtained based on the single-frame structured image data packet; For the single-frame structured image data packet and the video frame data, data packets representing the identity information, texture information, structural information and deep semantic structure of the same target object in a single frame or consecutive frames are collected to obtain human image feature data.

4. The method according to claim 1, characterized in that, The step of performing signal-to-noise estimation and delay analysis on the video frame data to generate frame quality data includes: The signal-to-noise estimation of the video frame data includes: The video frame data is used as the only input, and the process involves sequentially performing complete frame verification, overall signal-to-noise assessment, and read-in delay statistics. The overall signal-to-noise ratio (SNR) assessment includes performing frame-wide statistical calculations on the brightness uniformity, pixel energy distribution, edge density variation, and background noise baseline of the video frame, and then using a whole-packet mapping function to fuse the statistical quantities into an SNR index.

5. The method according to claim 4, characterized in that, The step of performing signal-to-noise estimation and delay analysis on the video frame data to generate frame quality data further includes: Delay analysis of the video frame data includes: Delay samples are formed by recording the difference between the arrival time of the frame and its timestamp, and the mean and variance of the delay are calculated. Based on the indivisible mapping rules, the signal-to-noise ratio and delay analysis results are combined into a single quality score, generating frame quality data as a structured whole output.

6. The method according to claim 1, characterized in that, The facial feature data undergoes a multi-level overall mapping transformation to generate surface features, enhanced features, and deep features, including: Apply lightweight global projection mapping to facial feature data, including performing global low-dimensional linear transformation or global hash-compression mapping on the facial feature data; After mapping, the entire packet output in multiple temporal neighborhoods is smoothed to generate surface features that express appearance contours and coarse-grained distinguishability.

7. The method according to claim 6, characterized in that, The facial feature data undergoes a multi-level overall mapping transformation to generate surface features, enhanced features, and deep features, and also includes: Using the same portrait feature data as input, a holistic augmentation mapping network is used to perform mid-dimensional holistic feature extraction. The dimensional feature extraction performed through the overall augmentation mapping network includes: The system performs overall spatial relationship modeling, overall strong and weak texture fusion, and overall contrast-sensitive mapping, and performs overall consistency calibration on the enhancement results within a time window to obtain enhanced features for pose changes and illumination fluctuations.

8. The method according to claim 7, characterized in that, The facial feature data undergoes a multi-level overall mapping transformation to generate surface features, enhanced features, and deep features, and also includes: The facial feature data is input into the deep reasoning unit for overall semantic abstraction and high-dimensional structure encoding, including overall contextual relationship inference, overall key region aggregation and overall semantic compression mapping, to obtain deep features that can represent the long-term stable characteristics of a person's identity.

9. The method according to claim 8, characterized in that, After generating surface features, enhanced features, and deep features, the process also includes generating surface confidence; Specifically, the facial feature data is used to generate an initial confidence through overall confidence extraction, and the time alignment table is used to generate a time series factor through time series consistency evaluation. The initial confidence, the time series factor, and the frame quality data are weighted and fused to generate a surface confidence.

10. The method according to claim 9, characterized in that, The method includes: The time alignment table is generated by non-blocking time alignment of video frame data and the human portrait feature data; Overall confidence extraction refers to a data processing procedure that obtains a global confidence index that can measure the degree of confidence of a feature vector without disassembling the internal fields of the entire human face feature package. Initial confidence refers to the degree of credibility of the current portrait feature data in terms of semantic matching for portrait recognition, independent of temporal factors and frame quality factors.

11. The method according to claim 10, characterized in that, The method includes: The numerical index, i.e., time series factor, is obtained by evaluating the non-blocking time alignment results between camera frame data and human facial feature data. Without causing waiting or blocking in the current data processing flow, the system uses the timestamp field in the camera frame and the facial feature data packet to perform a non-blocking time alignment process by using a time-series sliding window and dynamic interpolation calculation to achieve time-related pairing of data.

12. The method according to claim 1, characterized in that, The similarity measurement of the surface features and the enhanced features includes: The surface features and the enhanced features are similarly measured using a similarity function; The similarity functions include whole-packet cosine similarity, global Euclidean distance mapping, and matching functions based on the global projection space.

13. The method according to claim 1, characterized in that, The stability assessment of the frame quality data includes: The frame quality data is weighted using a sliding window, and the distribution of all weighted results within the sliding window is statistically analyzed to generate a stability evaluation result. The distribution statistics refer to data processing that quantitatively models and statistically analyzes the variation characteristics of frame image quality data within a certain time range.

14. The method according to claim 13, characterized in that, The step of performing distribution statistics on all weighted results within the sliding window to generate stability evaluation results also includes: Each frame quality data in the sliding window is used as input, and weighting processing is performed on each frame quality data through overall weighting. The overall weighting function takes the overall quality value in the whole frame quality packet, the corresponding timestamp, and the relative position in the window as overall input parameters, and maps to a single weighting result with preset weight coefficients. Perform distribution statistics processing on all weighted results generated within the entire sliding window. The distribution statistics include the unified calculation of the overall mean, overall variance, quantile interval and stability measure factor of the weighted results. All statistics are completed with the entire set of weighted results as an indivisible whole input. The mean, dispersion, and stability metric obtained from statistical processing are fused together in an overall mapping manner to output a single stability assessment result.

15. The method according to claim 2, characterized in that, The step of performing hierarchical adjustment and unified projection mapping on the surface features, enhanced features, and deep features according to the weight factors to generate an initial aggregate vector includes: The surface features, the enhancement features, and the deep features are respectively mapped using the weighting factor to generate surface adjustment coefficients, enhancement adjustment coefficients, and deep adjustment coefficients; The surface features are used to generate surface-corrected features through the surface adjustment coefficients; the enhanced features are used to generate enhanced-corrected features through mid-dimensional mapping and the enhancement adjustment coefficients; and the deep features are used to generate deep-corrected features through deep-dimensional normalization and the deep adjustment coefficients. An initial aggregate vector is generated by uniformly projecting the surface correction features, the enhanced correction features, and the deep correction features.

16. The method according to claim 15, characterized in that, The surface adjustment coefficient is a single coefficient or vector set that weights and adjusts the overall numerical structure of the surface features. The enhancement adjustment coefficient is a single coefficient or vector set that weights and adjusts the overall numerical structure of the enhancement feature; The deep adjustment coefficient is a single coefficient or vector set that weights and adjusts the overall numerical structure of deep features; Surface-modified features are a set of weighted modified feature vectors obtained by numerical adjustment, morphological remapping and structure preservation of surface features. Enhanced modified features are a set of weighted modified feature vectors obtained by numerical adjustment, morphological remapping and structure preservation of enhanced features. Deep modified features are a set of weighted modified feature vectors obtained after numerical adjustment, morphological remapping and structure preservation of deep features. Unified projection is a global mapping mechanism based on a structured vector fusion model.

17. The method according to claim 16, characterized in that, The difference analysis results are weighted and adjusted according to the weighting factors. It is then determined whether the weighted adjustment result is greater than a preset discrimination threshold. If so, the difference analysis, weighted adjustment results, and the initial aggregation vector are encapsulated to generate incremental features, including: The preset discrimination threshold T is obtained by performing overall statistical analysis on the frame quality data, and its specific calculation formula is as follows: In the formula, T represents the preset discrimination threshold, and μQ represents the mean value of the frame quality data. Q represents the standard deviation of the frame quality data. The weighting coefficients represent the mean of the frame quality data. The mean of the frame quality data and the weighting coefficients are all normalized during the calculation.

18. The method according to claim 17, characterized in that, The method further includes: The preset hierarchical difference threshold is obtained by performing overall statistics on the surface confidence level: Specifically, the entire surface confidence package is subjected to overall statistical processing, including overall mean, overall variance, and overall skewness of the entire surface confidence package, which are indivisible overall statistical measures. The overall statistics are then converted into preset hierarchical difference thresholds through a mapping function.

19. The method according to claim 2, characterized in that, The step of weighting the incremental features according to the weighting factor and generating a callback result includes: The incremental features and the initial aggregated vector are compared to generate a change amount through whole-pack consistency comparison, and the change amount and the weight factor are mapped and weighted to generate a weighted change amount. The frame quality data and the surface confidence are processed by threshold adaptive processing to generate a constraint threshold, and the weighted change amount and the constraint threshold are compared by judgment to generate a callback result; Among them, the change refers to a feature difference measure with the overall vector as the granularity; The weighted change refers to a total numerical vector that comprehensively represents the degree of change in the aggregate state; Threshold adaptive processing refers to the data processing process of dynamically calculating the judgment boundary value applicable to the current round of judgment based on the current frame quality data and surface confidence data, i.e., the "constraint threshold". The constraint threshold refers to the dynamic judgment boundary value calculated by the system before each callback judgment, based on the specific quality and confidence information of the current input data, to control whether to trigger the generation of the callback result.

20. An asynchronous task callback device between a hardware codec and a GPU, characterized in that, include: The data acquisition unit is used to acquire video frame data and facial feature data of the target object; The data analysis unit is used to perform signal-to-noise estimation and delay analysis on the video frame data to generate frame quality data; The feature transformation unit is used to perform multi-level overall mapping transformation on the portrait feature data, generate surface features, enhanced features and deep features respectively, and measure the similarity between the surface features and the enhanced features. The weighted fusion unit is used to perform stability assessment on the frame quality data, and to perform weighted fusion on the stability assessment results and similarity measurement results to generate weight factors. The feature determination unit is used to perform weighted determination of incremental features according to the weighting factor and generate callback results.

21. An electronic device, characterized in that, include: The processor, communication interface, memory, and communication bus are connected, with the processor, communication interface, and memory communicating with each other via the communication bus. Memory, which stores computer programs; When a processor executes a program stored in memory, it implements the asynchronous task callback method between the hardware codec and the GPU as described in any one of claims 1 to 19.

22. A computer-readable storage medium, characterized in that, The device stores a computer program that, when executed by a processor, implements the asynchronous task callback method between the hardware codec and the GPU as described in any one of claims 1 to 19.

Images