Distributed face feature real-time extraction method and system based on edge calculation

Abstract

The invention provides a distributed face feature real-time extraction method and system based on edge calculation, and relates to the technical field of face recognition, and the method comprises the steps: carrying out the time sequence sampling of a collected face image, obtaining a multi-frame sequence, constructing a spatial-temporal feature association graph, and predicting and correcting a face position, dividing the optimized image sequence into task packets, distributing the task packets to edge computing equipment, constructing a feature point local tensor field, extracting feature description information, and finally selecting an optimal feature fusion matrix. According to the invention, the real-time performance and accuracy of face feature extraction are improved, and the consumption of computing resources is reduced.

Description

Technical Field

[0001] The present invention relates to the field of face recognition technology, and in particular to a distributed real-time facial feature extraction method and system based on edge computing. Background Art

[0002] With the rapid development of artificial intelligence (AI), facial recognition, as a key biometric technology, has been widely used in a variety of fields, including security monitoring, identity verification, and smart device unlocking. Traditional facial feature extraction is typically processed on cloud servers, requiring the collected image data to be uploaded to the cloud, where high-performance servers perform feature extraction and recognition. However, with the proliferation of IoT devices and the expansion of facial recognition applications, this centralized processing model has become unable to meet real-time requirements, privacy protection, and network bandwidth requirements.

[0003] Edge computing, as a computing paradigm, deploys computing and storage resources at the edge of the network, close to the data source. This effectively reduces data transmission latency, improves real-time processing, and alleviates computing pressure on the central cloud. Migrating facial feature extraction tasks from the cloud to the edge, leveraging distributed computing, can better meet the requirements of low latency and high reliability, making it particularly suitable for facial recognition applications in mobile scenarios.

[0004] However, in the existing technology, there are still some shortcomings in the facial feature extraction method. Most of the existing facial feature extraction methods are based on single-frame image processing, lack of effective utilization of the spatiotemporal information of the face in continuous video sequences, and difficult to cope with complex scenes such as changes in facial posture and lighting, affecting the accuracy and stability of feature extraction; the existing edge computing resource allocation strategy usually adopts a static method, and cannot dynamically adjust the task allocation according to the facial motion characteristics and edge device load, resulting in inefficient computing resource utilization and unstable processing delay; the feature point extraction process in the existing technology often adopts a fixed algorithm mode, lacks an adaptive mechanism based on local information, and cannot perform differentiated processing on the characteristics of different facial areas, affecting the accuracy and distinguishability of feature description. Summary of the Invention

[0005] The embodiments of the present invention provide a distributed real-time facial feature extraction method and system based on edge computing, which can solve the problems in the prior art.

[0006] A first aspect of an embodiment of the present invention provides a distributed real-time facial feature extraction method based on edge computing, comprising:

[0007] Perform time-series sampling on facial image data collected by the terminal device to obtain a continuous multi-frame image sequence;

[0008] Establishing an edge computing resource pool including multiple edge computing devices;

[0009] A spatiotemporal feature correlation map is constructed for continuous multi-frame image sequences to extract facial motion trajectories. The face position is predicted using a recursive Bayesian method and corrected using a reprojection error compensation method to generate a spatiotemporally aligned optimized facial image sequence.

[0010] Divide the optimized facial image sequence into continuous frame task packages, allocate the continuous frame task packages using a coloring marking method, establish a feature point motion prediction mechanism, and allocate the continuous frame task packages to edge computing devices to perform feature extraction tasks;

[0011] In the edge computing device, a local tensor field of feature points is constructed and potential energy values ​​are obtained. Multi-level seed points are established to determine the feature point extraction sequence. Feature description information is extracted based on the tensor field gradient, a tensor feature sequence is generated, and a feature fusion matrix is ​​constructed.

[0012] The feature fusion matrix generated by each edge computing device is time-series verified, the optimal feature fusion matrix is ​​selected according to the preset mutual information threshold, and the optimal feature fusion matrix is ​​sent to the terminal device.

[0013] In an optional embodiment, constructing a spatiotemporal feature correlation map for a continuous multi-frame image sequence and extracting a facial motion trajectory includes:

[0014] Extract a set of feature points from each frame in a continuous multi-frame image sequence, calculate the spatial distance between each feature point and surrounding feature points, adaptively determine the influence radius of the Gaussian kernel function based on the local distribution density of the feature points, and use the Gaussian kernel function to perform weighted calculation on the spatial distance to obtain the local density value of each feature point;

[0015] The spatial gradient of the local density value of each feature point is calculated to obtain the density gradient field pointing to the direction of increasing local density value. The cosine value of the angle between the density gradient direction and the direction of the feature point connection is calculated. The spatial distance is weighted by combining the Gaussian kernel function to obtain the spatial correlation strength between the feature points.

[0016] Calculating the ratio of local density values ​​of corresponding feature points in adjacent frame images as the density transfer ratio of the feature points, and establishing a temporal correlation relationship between the feature points in adjacent frames based on the density transfer ratio;

[0017] The spatiotemporal feature correlation graph is constructed with feature points as vertices, spatial correlation strength as intra-frame edge weight, and density transfer ratio as inter-frame edge weight.

[0018] In the spatiotemporal feature association graph, a density flow field is constructed based on the density gradient field and the density transfer ratio. The streamline trajectory is iteratively calculated along the intra-frame edge weight and inter-frame edge weight direction of the spatiotemporal feature association graph to obtain the face motion trajectory.

[0019] In an optional embodiment, in the spatiotemporal feature association graph, a density flow field is constructed based on the density gradient field and the density transfer ratio, and streamline trajectories are iteratively calculated along the intra-frame edge weight and inter-frame edge weight directions of the spatiotemporal feature association graph to obtain the face motion trajectory, including:

[0020] Extract the first direction vector of the density gradient field from the spatiotemporal feature correlation map, map the density transfer ratio to the second direction vector, and fuse the first direction vector and the second direction vector through adaptive weighting to construct a density flow field;

[0021] In the density flow field, the feature point is used as the initial point of the streamline. The intra-frame edge weight of the spatiotemporal feature association graph is used to determine the motion direction of the feature point in the spatial domain. The inter-frame edge weight is used to determine the motion direction of the feature point in the temporal domain. The feature point position is iteratively updated until convergence, generating a streamline trajectory.

[0022] Taking the area with the highest density of feature points as the starting position, the streamline trajectories are tracked in the forward and reverse directions in the density flow field to obtain the motion trajectory sequence of the feature point group. The motion trajectory of the face is obtained by curve fitting.

[0023] In an optional embodiment, the optimized facial image sequence is divided into continuous frame task packages, the continuous frame task packages are allocated using a coloring marking method, a feature point motion prediction mechanism is established, and the continuous frame task packages are allocated to an edge computing device to perform feature extraction tasks, including:

[0024] Obtaining the time interval of each frame image in the optimized face image sequence, calculating the motion displacement and motion direction of the image frame according to the time interval, and dividing the continuous frame images into continuous frame task packages of adaptive size based on the motion displacement and motion direction;

[0025] Counting the number of feature points and the spatial distribution area of ​​each frame image in the continuous frame task package, calculating the ratio of the number of feature points to the spatial distribution area, and using the ratio as the correlation strength between the continuous frame task packages;

[0026] Coloring and marking adjacent continuous frame task packets based on correlation strength, so that adjacent continuous frame task packets with correlation strength higher than a preset strength threshold are marked with different coloring marks, and the number of coloring marks is equal to the number of edge computing devices, to obtain a coloring mark for each continuous frame task packet;

[0027] A motion state model of feature points is established for consecutive frame task packages with the same coloring mark, and the migration direction of feature points between consecutive frame task packages corresponding to different coloring marks is predicted based on the motion state model, and feature point detection is performed in the area corresponding to the feature point migration direction;

[0028] Continuous frame task packages with the same color mark are assigned to the corresponding edge computing devices. Each edge computing device performs parallel feature extraction tasks based on the feature point migration direction, and dynamically adjusts the number of frames of the continuous frame task package according to the processor occupancy of the edge computing device.

[0029] In an optional embodiment, establishing a motion state model of feature points for consecutive frame task packets with the same coloring mark includes:

[0030] For the feature points in the continuous frame task package, a K-order neighborhood graph is constructed with the feature point as the center, the topological distance matrix and the angle difference matrix between the feature point and the neighboring feature points are calculated, and the distribution density of the feature points per unit area in the K-order neighborhood graph is obtained;

[0031] Constructing local motion constraints based on the topological distance matrix, the angle difference matrix, and the feature point distribution density, calculating the affine transformation relationship of the feature points in the continuous frame task package, and extracting the scale component, rotation component, and translation component of the affine transformation relationship;

[0032] The degree of violation of the local motion constraint condition and the residual of the affine transformation relationship are constructed as optimization targets, and the optimization targets are iterated by the gradient descent method. The motion state evolution law of the feature point and the actual observation correspondence are determined according to the optimization results to obtain the motion state model of the feature point.

[0033] In an optional embodiment, a local tensor field of feature points is constructed in an edge computing device and potential energy values ​​are obtained. A multi-level seed point is established to determine a feature point extraction sequence. Feature description information is extracted based on the tensor field gradient to generate a tensor feature sequence. Constructing a feature fusion matrix includes:

[0034] In the edge computing device, a local tensor descriptor is constructed between feature points, and the direction parameters, scale parameters, and intensity parameters of the corresponding feature points are extracted to establish a tensor field. The regional correlation and structural importance of the feature points in the tensor field are calculated to obtain the potential energy value of the feature points.

[0035] Select the feature point with the largest potential energy value as the first-level seed point, determine the multi-level seed point based on the potential energy difference, select the optimal traversal path, record the position index information of the feature points on the optimal traversal path, and generate a feature point extraction sequence;

[0036] Calculate the gradient distribution value of the tensor field, extract the first feature description information for the area where the gradient distribution value is greater than the preset gradient strength threshold, and extract the second feature description information for the area where the gradient distribution value is less than the preset gradient strength threshold, to obtain a multi-dimensional feature description of the feature point;

[0037] Determine the feature extraction order according to the feature point extraction sequence, extract the structural parameters and neighborhood data of the feature points, generate tensor feature vectors and organize them according to the order of the feature point extraction sequence to form a tensor feature sequence;

[0038] The phase difference values ​​of adjacent eigenvectors in the tensor feature sequence are calculated, a phase difference matrix is ​​constructed and the feature fusion weight coefficient is calculated to generate a feature fusion weight matrix. The tensor feature sequence is fused using the feature fusion weight matrix to obtain a feature fusion matrix.

[0039] In an optional embodiment, a feature point with the largest potential energy value is selected as a first-level seed point, multiple levels of seed points are determined based on the potential energy difference, an optimal traversal path is selected, and position index information of the feature points on the optimal traversal path is recorded. Generating a feature point extraction sequence includes:

[0040] Determine the feature point with the largest potential energy value in the tensor field as the first-level seed point, establish a traversal window with the first-level seed point as the center, and calculate the potential energy difference of the feature points in the traversal window;

[0041] Select the feature point with the smallest potential energy difference as the second-level seed point, and record the traversal direction and traversal distance from the first-level seed point to the second-level seed point;

[0042] Determine a search sector in the traversal direction, and select a corresponding number of feature points in the search sector sorted from high to low in terms of potential energy values ​​according to a preset optimal number as third-level seed points;

[0043] Taking the third-level seed points as nodes, connecting them to form a candidate traversal path, and calculating the potential energy gradient of adjacent feature points on the candidate traversal path;

[0044] The candidate traversal path with the smallest potential energy gradient is selected as the optimal traversal path, and the position index information of the feature points along the optimal traversal path is recorded to generate a feature point extraction sequence.

[0045] A second aspect of an embodiment of the present invention provides a distributed real-time facial feature extraction system based on edge computing, including:

[0046] The first unit is used to perform time-series sampling on the facial image data collected by the terminal device to obtain a continuous multi-frame image sequence;

[0047] The second unit is used to establish an edge computing resource pool including multiple edge computing devices;

[0048] The third unit is used to construct a spatiotemporal feature correlation map for a continuous multi-frame image sequence, extract the facial motion trajectory, predict the face position using a recursive Bayesian method, and use the reprojection error compensation method to correct it, thereby generating a spatiotemporally aligned optimized facial image sequence;

[0049] The fourth unit is used to divide the optimized facial image sequence into continuous frame task packages, allocate the continuous frame task packages using a coloring marking method, establish a feature point motion prediction mechanism, and allocate the continuous frame task packages to an edge computing device to perform a feature extraction task;

[0050] The fifth unit is used to construct a local tensor field of feature points in the edge computing device and obtain potential energy values, establish multi-level seed points to determine the feature point extraction sequence, extract feature description information based on the tensor field gradient, generate a tensor feature sequence, and construct a feature fusion matrix;

[0051] The sixth unit is used to perform time series verification on the feature fusion matrix generated by each edge computing device, select the optimal feature fusion matrix according to a preset mutual information threshold, and send the optimal feature fusion matrix to the terminal device.

[0052] According to a third aspect of an embodiment of the present invention, an electronic device is provided, including:

[0053] processor;

[0054] a memory for storing processor-executable instructions;

[0055] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0056] According to a fourth aspect of an embodiment of the present invention, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described above is implemented.

[0057] In an embodiment of the present invention, by constructing a spatiotemporal feature association graph and combining it with a recursive Bayesian method to predict the face position, and using reprojection error compensation for correction, the spatiotemporal alignment optimization of the face image in motion is achieved, effectively improving the accuracy and stability of face feature extraction; a coloring marking method is used to allocate continuous frame task packages, and a feature point motion prediction mechanism is established to achieve efficient scheduling and load balancing of edge computing resources, significantly reduce the system response time, and improve the real-time performance of feature extraction; a local tensor field of feature points is constructed in the edge computing device and the feature extraction sequence is determined based on multi-level seed points, feature description information is extracted according to the tensor field gradient, and the optimal feature fusion matrix is ​​selected through time series verification and mutual information threshold, which greatly improves the robustness and discrimination ability of feature expression and adapts to the needs of face recognition in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 This is a flow chart of a distributed real-time facial feature extraction method based on edge computing according to an embodiment of the present invention;

[0059] Figure 2 Construct a flow chart for the feature point motion state model;

[0060] Figure 3 This is the network structure diagram of the feature point extraction algorithm. DETAILED DESCRIPTION

[0061] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0062] The following specific embodiments are used to describe the technical solution of the present invention in detail. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0063] Figure 1 This is a flow chart of a distributed real-time facial feature extraction method based on edge computing according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0064] Perform time-series sampling on facial image data collected by the terminal device to obtain a continuous multi-frame image sequence;

[0065] Establishing an edge computing resource pool including multiple edge computing devices;

[0066] A spatiotemporal feature correlation map is constructed for continuous multi-frame image sequences to extract facial motion trajectories. The face position is predicted using a recursive Bayesian method and corrected using a reprojection error compensation method to generate a spatiotemporally aligned optimized facial image sequence.

[0067] Divide the optimized facial image sequence into continuous frame task packages, allocate the continuous frame task packages using a coloring marking method, establish a feature point motion prediction mechanism, and allocate the continuous frame task packages to edge computing devices to perform feature extraction tasks;

[0068] In the edge computing device, a local tensor field of feature points is constructed and potential energy values ​​are obtained. Multi-level seed points are established to determine the feature point extraction sequence. Feature description information is extracted based on the tensor field gradient, a tensor feature sequence is generated, and a feature fusion matrix is ​​constructed.

[0069] The feature fusion matrix generated by each edge computing device is time-series verified, the optimal feature fusion matrix is ​​selected according to the preset mutual information threshold, and the optimal feature fusion matrix is ​​sent to the terminal device.

[0070] In an optional embodiment, constructing a spatiotemporal feature association map for a continuous multi-frame image sequence and extracting a facial motion trajectory includes:

[0071] Extract a set of feature points from each frame in a continuous multi-frame image sequence, calculate the spatial distance between each feature point and surrounding feature points, adaptively determine the influence radius of the Gaussian kernel function based on the local distribution density of the feature points, and use the Gaussian kernel function to perform weighted calculation on the spatial distance to obtain the local density value of each feature point;

[0072] The spatial gradient of the local density value of each feature point is calculated to obtain the density gradient field pointing to the direction of increasing local density value. The cosine value of the angle between the density gradient direction and the direction of the feature point connection is calculated. The spatial distance is weighted by combining the Gaussian kernel function to obtain the spatial correlation strength between the feature points.

[0073] Calculating the ratio of local density values ​​of corresponding feature points in adjacent frame images as the density transfer ratio of the feature points, and establishing a temporal correlation relationship between the feature points in adjacent frames based on the density transfer ratio;

[0074] The spatiotemporal feature correlation graph is constructed with feature points as vertices, spatial correlation strength as intra-frame edge weight, and density transfer ratio as inter-frame edge weight.

[0075] In the spatiotemporal feature association graph, a density flow field is constructed based on the density gradient field and the density transfer ratio. The streamline trajectory is iteratively calculated along the intra-frame edge weight and inter-frame edge weight direction of the spatiotemporal feature association graph to obtain the face motion trajectory.

[0076] In one specific embodiment, a set of feature points is extracted for each frame of a sequence of multiple consecutive facial images. A feature point extraction algorithm such as SIFT or SURF can be used to extract key points from the image as the feature point set. Assume that the set of feature points extracted from the i-th frame is Pi = {p1, p2, ..., pn}, where each feature point contains position coordinates and descriptor information.

[0077] Calculate the spatial distance between each feature point and the surrounding feature points. For feature points pi and pj, calculate the Euclidean distance dij between them. For example, for feature point p1 at coordinates (10, 15) and feature point p2 at coordinates (13, 19), their spatial distance d12 is 5 pixels. Adaptively determine the influence radius dc of the Gaussian kernel function based on the local distribution density of the feature points. Determine the dc value by calculating the average distance of the k nearest neighbors of the feature point. For example, when k = 7, calculate the average distance from each feature point to its nearest 7 feature points, and then average this average distance of all feature points to obtain the dc value. For example, in a certain frame image, dc = 12 pixels.

[0078] The Gaussian kernel function is used to perform weighted calculation on the spatial distance. The Gaussian kernel function is expressed as exp(-dij 2 / dc 2 ), where dij is the distance between feature points and dc is the influence radius. When the distance between two points is 6 pixels and dc is 12 pixels, the Gaussian kernel function value is approximately 0.75. For each feature point pi, the sum of its Gaussian kernel function values ​​with all other feature points is calculated to obtain the local density value ρi of that feature point. For example, if the Gaussian kernel function values ​​of a feature point and the 10 surrounding feature points are [0.95, 0.86, 0.75, 0.64, 0.52, 0.42, 0.31, 0.22, 0.15, 0.09] respectively, the local density value of the feature point is the sum of these values, 4.91.

[0079] The spatial gradient of the local density value of each feature point is calculated to obtain a density gradient field pointing in the direction of increasing local density values. On the image plane, the local density values ​​of the feature points are interpolated to construct a density field, and then the gradient of this density field is calculated. For example, the density gradient direction at feature point p1 is (0.6, 0.8), indicating the direction of the fastest density increase at that point.

[0080] Calculate the cosine of the angle between the density gradient and the line connecting the feature points. For feature points pi and pj, calculate the cosine of the angle between the gradient direction vector and the line connecting the feature points, cosθij. For example, if the line connecting feature points p1 and p2 is (0.8, 0.6) and the gradient direction at p1 is (0.6, 0.8), then the cosine of the angle is 0.96. Use the Gaussian kernel function to weight the spatial distance and multiply cosθij by the Gaussian kernel value to obtain the spatial correlation strength sij between the feature points. When cosθij is 0.96 and the Gaussian kernel value is 0.75, the spatial correlation strength sij is 0.72.

[0081] Calculate the ratio of the local density values ​​of corresponding feature points in adjacent frame images. Use a feature matching algorithm such as FLANN to determine the corresponding feature point pairs in adjacent frames. For the feature point pi in the t-th frame and the corresponding feature point pi' in the t+1-th frame, calculate the density transfer ratio tij = ρi' / ρi. For example, if the local density value of pi is 4.91 and the local density value of pi' is 5.15, the density transfer ratio is 1.05, indicating that the density of the feature point has increased slightly. Based on the density transfer ratio, a temporal correlation relationship between feature points in adjacent frames is established.

[0082] A spatiotemporal feature association graph G is constructed, with feature points as vertices, spatial correlation strength as intra-frame edge weights, and density transfer ratio as inter-frame edge weights. In this graph, the feature points of each frame form a layer, with points within a layer connected by spatial correlation strength, and corresponding points between different layers connected by density transfer ratios. For example, for a sequence consisting of five frames with 100 feature points extracted per frame, the constructed spatiotemporal feature association graph contains 500 vertices, approximately 4,500 intra-frame edges, and approximately 400 inter-frame edges.

[0083] In the spatiotemporal feature correlation graph, a density flow field is constructed based on the density gradient field and the density transfer ratio. The density flow field defines the flow direction and intensity of feature points in space and time. For intra-frame flow, the direction is determined by the density gradient, and the intensity is determined by the spatial correlation strength. For inter-frame flow, the direction is determined by the time progression, and the intensity is determined by the density transfer ratio.

[0084] Streamline trajectories are iteratively calculated along the intra-frame and inter-frame edge weights of the spatiotemporal feature association graph. Starting from the initial feature point, the iterative calculation is guided by the density flow field. At each iteration, the next movement direction and step size are determined based on the intra-frame and inter-frame correlation strengths of the current point. For example, if the maximum intra-frame correlation strength between the current point and its neighbor is 0.72, and the corresponding inter-frame correlation strength is 1.05, then movement in that direction is prioritized, with the step size proportional to the correlation strength. Through multiple iterations, a complete streamline trajectory is formed, representing the motion trajectory of a specific facial part.

[0085] The extracted streamlines are filtered and smoothed to remove abnormal trajectories, resulting in a final set of facial motion trajectories. For example, in a 10-frame facial image sequence, 5-10 stable trajectories are extracted from key areas such as the corners of the eyes and mouth, accurately reflecting the movement of each part of the face during expression changes.

[0086] In an optional embodiment, in the spatiotemporal feature association graph, a density flow field is constructed based on the density gradient field and the density transfer ratio, and streamline trajectories are iteratively calculated along the intra-frame edge weight and inter-frame edge weight directions of the spatiotemporal feature association graph to obtain the face motion trajectory, including:

[0087] Extract the first direction vector of the density gradient field from the spatiotemporal feature correlation map, map the density transfer ratio to the second direction vector, and fuse the first direction vector and the second direction vector through adaptive weighting to construct a density flow field;

[0088] In the density flow field, the feature point is used as the initial point of the streamline. The intra-frame edge weight of the spatiotemporal feature association graph is used to determine the motion direction of the feature point in the spatial domain. The inter-frame edge weight is used to determine the motion direction of the feature point in the temporal domain. The feature point position is iteratively updated until convergence, generating a streamline trajectory.

[0089] Taking the area with the highest density of feature points as the starting position, the streamline trajectories are tracked in the forward and reverse directions in the density flow field to obtain the motion trajectory sequence of the feature point group. The motion trajectory of the face is obtained by curve fitting.

[0090] In a specific embodiment, a set of facial feature points in multiple consecutive frames of images is obtained to construct a spatiotemporal feature association graph. The association graph contains two types of edges: intra-frame association and inter-frame association. The intra-frame association edge represents the spatial relationship between feature points in the same frame, and the inter-frame association edge represents the temporal relationship between feature points between adjacent frames. For each feature point, its local density value in the feature space is calculated. This density value can be obtained by counting the number of other feature points within a preset radius centered on the feature point. For example, a circular area with a radius of 10 pixels is selected, and the number of feature points falling within the area is counted as the density value of the point.

[0091] When extracting the first direction vector of the density gradient field in the spatiotemporal feature association graph, for each feature point, the rate of change of its density value relative to the eight surrounding directions is calculated. Specifically, the adjacent points in 8 directions (0°, 45°, 90°, 135°, 180°, 225°, 270°, 315°) around the feature point are selected, the density differences in these directions are calculated, and normalized to unit vectors. For example, if the density value of a feature point is 15 and the density value of its adjacent points in the 45° direction is 20, then the density gradient in this direction is 5, and the direction points to 45°. The density gradients in the eight directions are calculated comprehensively, and the direction with the largest density change is obtained as the first direction vector.

[0092] The density transfer ratio is calculated by analyzing the migration of feature points between adjacent frames. For a feature point in the current frame, the corresponding feature point in the next frame is found using the inter-frame edges, and the density ratio between the two points is calculated. For example, if the feature point density in the current frame is 12 and the corresponding point density in the next frame is 18, the density transfer ratio is 1.5. This ratio is mapped to a second direction vector pointing in the direction of increasing density, with the magnitude of the vector proportional to the ratio.

[0093] When adaptively weighting the first and second direction vectors, the weights are adjusted based on the local density distribution of the feature points. In areas of drastic density changes, the first direction vector receives a larger weight (e.g., 0.7); in areas of gentle density changes, the second direction vector receives a larger weight (e.g., 0.7). This adaptive weighting allows the two direction vectors to be fused to construct a density flow field.

[0094] When generating streamline trajectories in a density flow field, feature points are selected as the initial points of the streamlines. For these initial points, their direction of motion in the spatial domain is determined by the intra-frame edge weights of the spatiotemporal feature association graph. The intra-frame edge weights are calculated based on the Euclidean distance between feature points, with closer distances giving higher weights. For example, if the distance between two feature points is 5 pixels, the intra-frame edge weight can be set to 0.8; if the distance is 15 pixels, the weight can be set to 0.3. The direction of motion of feature points in the temporal domain is determined by the inter-frame edge weights. The inter-frame edge weights are calculated based on the matching confidence of the feature points in adjacent frames, with higher matching confidences giving higher weights. For example, if the matching confidence of corresponding feature points in two frames is 0.9, the inter-frame edge weight can be set to 0.85.

[0095] Iteratively update the feature point position. Starting from the initial feature point, it moves to a new position with a fixed step size (e.g., 2 pixels) based on the direction of the density flow field. The flow field direction is recalculated at the new position, and the movement continues. When the feature point movement distance is less than a threshold (e.g., 0.1 pixels) or the maximum number of iterations (e.g., 100) is reached, convergence is achieved, and the generation of a streamline trajectory is complete.

[0096] To obtain a complete facial motion trajectory, it's necessary to identify the region with the highest feature point density as the starting position. For example, in facial images, areas like the tip of the nose and the corners of the eyes typically have a high feature point density. Therefore, the top 5% of feature points with the highest density can be selected as the starting position. From this starting position, streamlines are tracked in both forward and reverse directions within the density flow field. Forward tracking captures motion trends in future frames, while reverse tracking recovers motion history in past frames. Combining these two methods yields a complete sequence of feature point cluster motion trajectories.

[0097] The obtained motion trajectory sequence is subjected to curve fitting using cubic spline interpolation to ensure smooth transitions at the trajectory points. The control point interval is set to 5 frames, and the curve shape is adjusted at each control point based on the actual trajectory point position. For example, for a 50-frame trajectory sequence, control points are selected at frames 1, 6, 11, ..., and 46. A smooth, continuous curve is generated using cubic spline interpolation to obtain the facial motion trajectory.

[0098] The above process effectively captures subtle facial movements in video sequences, achieving high-precision facial motion trajectory extraction. This method, combining the advantages of density gradient fields and density transfer ratios, can adapt to facial motion characteristics in different scenarios, providing important technical support for applications such as facial expression analysis and emotion recognition.

[0099] In an optional embodiment, the optimized facial image sequence is divided into continuous frame task packages, the continuous frame task packages are allocated using a coloring marking method, a feature point motion prediction mechanism is established, and the continuous frame task packages are allocated to an edge computing device to perform feature extraction tasks, including:

[0100] Obtaining the time interval of each frame image in the optimized face image sequence, calculating the motion displacement and motion direction of the image frame according to the time interval, and dividing the continuous frame images into continuous frame task packages of adaptive size based on the motion displacement and motion direction;

[0101] Counting the number of feature points and the spatial distribution area of ​​each frame image in the continuous frame task package, calculating the ratio of the number of feature points to the spatial distribution area, and using the ratio as the correlation strength between the continuous frame task packages;

[0102] Coloring and marking adjacent continuous frame task packets based on correlation strength, so that adjacent continuous frame task packets with correlation strength higher than a preset strength threshold are marked with different coloring marks, and the number of coloring marks is equal to the number of edge computing devices, to obtain a coloring mark for each continuous frame task packet;

[0103] A motion state model of feature points is established for consecutive frame task packages with the same coloring mark, and the migration direction of feature points between consecutive frame task packages corresponding to different coloring marks is predicted based on the motion state model, and feature point detection is performed in the area corresponding to the feature point migration direction;

[0104] Continuous frame task packages with the same color mark are assigned to the corresponding edge computing devices. Each edge computing device performs parallel feature extraction tasks based on the feature point migration direction, and dynamically adjusts the number of frames of the continuous frame task package according to the processor occupancy of the edge computing device.

[0105] In a specific embodiment, the time interval of each frame image in the optimized facial image sequence is obtained, for example, the time difference between adjacent frames is 30 milliseconds. The motion displacement and motion direction of the image frame are calculated according to the time interval, specifically by comparing the coordinate changes of the facial feature points in adjacent frames. For example, the average displacement between frames is calculated to be 5 pixels, and the main motion direction is 45 degrees to the left. Based on these motion parameters, the continuous frame images are divided into continuous frame task packages of adaptive size. When the motion displacement is small (such as less than 3 pixels) and the direction is stable, 8-12 frames of images can be divided into a task package; when the motion displacement is large (such as greater than 10 pixels) or the direction changes significantly, 3-5 frames of images are divided into a task package to adapt to dynamically changing scenes.

[0106] For each consecutive frame task package, the number of feature points and the spatial distribution area of ​​each frame are counted. Feature points are obtained using the FAST corner detection algorithm. For example, if 128 feature points are detected in a given frame, these feature points are distributed within an area of ​​12,800 square pixels. The ratio of the number of feature points to the spatial distribution area is calculated: 128 ÷ 12,800 = 0.01. This ratio is used as the correlation strength between consecutive frame task packages. The difference in correlation strength between adjacent task packages reflects the degree of change in image content; smaller differences indicate stronger correlation.

[0107] Adjacent consecutive frame task packets are colored based on their correlation strength. A preset strength threshold of 0.003 is set. When the difference in correlation strength between adjacent task packets exceeds this threshold, they are assigned different coloring tags. The number of coloring tags is equal to the number of edge computing devices. For example, if there are four edge computing devices, red, green, blue, and yellow coloring tags are used. Thus, a task packet with a correlation strength of 0.01 might be marked red, while an adjacent task packet with a correlation strength of 0.015 might be marked green.

[0108] For consecutive frame task packages with the same coloring mark, the system builds a motion state model of the feature points. Taking the red-marked task package as an example, by analyzing the positional changes of the feature points in consecutive frames, a state model is established to describe the feature point motion trends. This model takes into account changes in the velocity and acceleration of the feature points. For example, if a trend of rightward rotation of the face is detected, the feature points in the next task package are predicted to move approximately 7 pixels to the right. Based on this motion state model, the migration direction of the feature points between consecutive frame task packages corresponding to different coloring marks is predicted, and detection is focused on areas where feature points are likely to appear, thereby improving detection efficiency.

[0109] Consecutive frame task packets with the same color mark are assigned to the corresponding edge computing device. For example, the red-marked task packet is assigned to device 1, the green-marked task packet is assigned to device 2, and so on. Each edge computing device performs parallel feature extraction tasks based on the predicted feature point migration direction. When the processor utilization rate of device 1 exceeds 85%, the system will dynamically reduce the number of frames of the red task packet from 10 to 6 frames. When the processor utilization rate of device 2 is less than 40%, the system will increase the number of frames of the green task packet from 6 to 9 frames to achieve balanced utilization of computing resources.

[0110] For example, for a 30fps facial video stream, this method can divide a 300-frame video sequence into approximately 30-40 consecutive frame task packages, which are then distributed to four edge computing devices for parallel processing. By using a feature point motion prediction mechanism, the accuracy of feature point detection is improved by approximately 15%, while computational redundancy is reduced by approximately 30%. Dynamically adjusting the task package size under varying load conditions maintains the processor utilization of each device within the ideal range of 50%-80%, achieving efficient utilization of computing resources and reducing overall processing time by approximately 25% compared to traditional methods.

[0111] In this embodiment, adaptive-sized continuous frame task package division is achieved through the motion displacement and direction information between image frames, so that the task package is more in line with the changes in image content, which helps to reduce computational redundancy; the correlation strength is calculated based on the density and distribution area of ​​feature points, and the coloring marking method is used for task allocation, so that the tasks processed by each edge device have better balance, avoiding single-point overload; the number of task package frames is dynamically adjusted using the processor occupancy rate, so that the task allocation can be flexibly adjusted according to the real-time load of the edge device, thereby improving the overall computing efficiency and response speed; by establishing a feature point motion state model to predict the migration direction of feature points, it is helpful to accurately detect the changes of feature points between different task packages and maintain the spatiotemporal continuity of feature extraction; each task package can be allocated to multiple edge devices for parallel processing, effectively reducing the feature extraction delay and meeting real-time requirements.

[0112] In an optional embodiment, establishing a motion state model of feature points for consecutive frame task packets with the same coloring mark includes:

[0113] For the feature points in the continuous frame task package, a K-order neighborhood graph is constructed with the feature point as the center, the topological distance matrix and the angle difference matrix between the feature point and the neighboring feature points are calculated, and the distribution density of the feature points per unit area in the K-order neighborhood graph is obtained;

[0114] Constructing local motion constraints based on the topological distance matrix, the angle difference matrix, and the feature point distribution density, calculating the affine transformation relationship of the feature points in the continuous frame task package, and extracting the scale component, rotation component, and translation component of the affine transformation relationship;

[0115] The degree of violation of the local motion constraint condition and the residual of the affine transformation relationship are constructed as optimization targets, and the optimization targets are iterated by the gradient descent method. The motion state evolution law of the feature point and the actual observation correspondence are determined according to the optimization results to obtain the motion state model of the feature point.

[0116] In one specific embodiment, a K-order neighborhood graph is constructed for a feature point in a continuous frame task package, centered around the feature point. The K nearest neighbors around the feature point are selected as its neighborhood points. Typically, the value of K can be set between 8 and 12 to balance computational complexity and model stability. For example, for a video frame with a resolution of 1920×1080, when the feature point density is approximately 0.001 per square pixel, a K value of 10 yields good results. After this construction is completed, a topological distance matrix is ​​calculated between the feature point and its neighborhood feature points. Each element in the distance matrix represents the Euclidean distance, in pixels, between the central feature point and its corresponding neighborhood point. An angle difference matrix is ​​also calculated, recording the angles formed between the central feature point and each pair of adjacent neighborhood points. For example, for 10 neighborhood points, the angle difference matrix contains 10 elements, each representing the angle, in degrees, between two adjacent neighborhood points and the central point. Furthermore, the density of feature points per unit area in the K-order neighborhood graph is obtained by dividing the number of feature points within the area covered by the neighborhood graph by the area of ​​that area. For example, if the neighborhood graph covers an area of ​​200 square pixels and contains 15 feature points, the distribution density is 0.075 per square pixel.

[0117] Based on the obtained topological distance matrix, angle difference matrix, and feature point distribution density, local motion constraints are constructed. These constraints primarily consist of three parts: distance preservation, angle preservation, and density consistency. The distance preservation constraint requires that the relative distance between a feature point and its neighbors remain essentially constant across consecutive frames, with the permitted variation typically not exceeding 10%. The angle preservation constraint requires that the angle formed by a feature point and its neighbors remain largely constant across consecutive frames, with the permitted variation typically within ±5 degrees. The density consistency constraint requires that the density variation of feature points in a local region be consistent with the motion type. For example, for rigid motion, the density variation should be less than 5%, while for non-rigid motion, the upper limit on density variation can be relaxed to 15%. Subsequently, the affine transformation relationships between feature points in consecutive frame task packets are calculated. Affine transformations describe how feature points transform from one frame to the next, including scaling, rotation, and translation. The least squares method is used to estimate the optimal affine transformation matrix based on the corresponding positions of the feature point and its neighbors in adjacent frames. The scale, rotation, and translation components are extracted from the affine transformation matrix. The scale component represents the scaling ratio of the feature point area, for example, from 1.0 to 1.05 represents a 5% magnification; the rotation component represents the rotation angle of the feature point area, such as 3 degrees clockwise rotation; the translation component represents the displacement vector of the feature point, such as 5 pixels horizontally and 2 pixels vertically.

[0118] The degree of violation of the local motion constraints and the residual of the affine transformation relationship are constructed as optimization objectives. The degree of violation refers to the difference between the actual observed feature point motion and the local motion constraints, and is quantified by a weighted squared difference. For example, the distance constraint violation can be weighted 0.6, the angle constraint violation 0.3, and the density constraint violation 0.1. The affine transformation residual refers to the Euclidean distance between the feature point positions predicted using the affine transformation and the actual observed positions. The average residual is typically required to be less than 2 pixels. The optimization objective function is a weighted sum of the degree of violation and the residual, with a weight ratio of 7:3. The optimization objective is iterated using gradient descent. The learning rate is initially set to 0.01 and gradually reduced to 0.001 over the course of the iteration. The number of iterations is typically set to 100 to 200, or terminated early if the objective function value changes by less than 0.001. In a real-world case, for a continuous frame task package containing 30 frames, the optimization process typically converges after 150 iterations, with the final objective function value decreasing to less than 5% of the initial value. The motion state evolution law of the feature points and the actual observation correspondence are determined based on the optimization results. The motion state evolution law includes the motion trajectory, velocity change and acceleration characteristics of the feature points. For example, the feature points may exhibit linear motion (displacement is proportional to time), accelerated motion (displacement is proportional to the square of time) or periodic motion (displacement exhibits sinusoidal characteristics). The actual observation correspondence records the mapping relationship between the actual position and predicted position of the feature points in each frame, which can be used to evaluate tracking accuracy and correct subsequent predictions. The final feature point motion state model is a time series prediction model that can predict the position and motion characteristics of the feature points in future frames based on their historical motion states. The prediction accuracy can reach more than 95% under normal scenarios. This model can be used in application scenarios such as feature point tracking, motion analysis and anomaly detection.

[0119] Typically, feature point motion analysis techniques are based on optical flow (such as the Lucas-Kanade algorithm), feature matching (such as SIFT and SURF matching), or tracking methods based on Kalman filtering. These methods have limitations when dealing with feature point motion analysis. Traditional optical flow methods rely primarily on the assumption of constant brightness, which easily fails in scenes with changing lighting or rapid motion. Feature matching methods typically process each feature point independently, ignoring the topological relationships between feature points and overall structural constraints. Filter-based methods often use simplified motion models (such as uniform velocity or uniform acceleration models) and are difficult to adapt to complex nonlinear motion. Existing methods lack effective utilization of changes in the distribution density of feature points in local areas, resulting in poor performance in scenes with deformation and partial occlusion.

[0120] The method of this embodiment constructs a feature point motion analysis framework based on local topological structure, and more comprehensively describes and constrains the motion characteristics of feature points by introducing K-order neighborhood graph and multiple constraints. The introduction of K-order neighborhood graph structure not only considers the motion of a single feature point, but also the overall motion pattern of its neighborhood feature points, thereby enhancing the perception of local structural changes; a multi-dimensional constraint system including topological distance matrix, angle difference matrix and distribution density is constructed to make feature point motion analysis more comprehensive and robust; the local motion constraint is combined with the affine transformation, which not only retains the simplicity of the affine transformation but also incorporates local structural information, thereby improving the model's adaptability to complex motion; the adoption of a gradient descent-based optimization strategy enables the model to minimize prediction errors while satisfying local constraints, thereby balancing local consistency and global accuracy.

[0121] like Figure 2 As shown in FIG, a flowchart of constructing a feature point motion state model is shown.

[0122] In an optional embodiment, a local tensor field of feature points is constructed in an edge computing device and potential energy values ​​are obtained. A multi-level seed point is established to determine a feature point extraction sequence. Feature description information is extracted based on the tensor field gradient to generate a tensor feature sequence. Constructing a feature fusion matrix includes:

[0123] In the edge computing device, a local tensor descriptor is constructed between feature points, and the direction parameters, scale parameters, and intensity parameters of the corresponding feature points are extracted to establish a tensor field. The regional correlation and structural importance of the feature points in the tensor field are calculated to obtain the potential energy value of the feature points.

[0124] Select the feature point with the largest potential energy value as the first-level seed point, determine the multi-level seed point based on the potential energy difference, select the optimal traversal path, record the position index information of the feature points on the optimal traversal path, and generate a feature point extraction sequence;

[0125] Calculate the gradient distribution value of the tensor field, extract the first feature description information for the area where the gradient distribution value is greater than the preset gradient strength threshold, and extract the second feature description information for the area where the gradient distribution value is less than the preset gradient strength threshold, to obtain a multi-dimensional feature description of the feature point;

[0126] Determine the feature extraction order according to the feature point extraction sequence, extract the structural parameters and neighborhood data of the feature points, generate tensor feature vectors and organize them according to the order of the feature point extraction sequence to form a tensor feature sequence;

[0127] The phase difference values ​​of adjacent eigenvectors in the tensor feature sequence are calculated, a phase difference matrix is ​​constructed and the feature fusion weight coefficient is calculated to generate a feature fusion weight matrix. The tensor feature sequence is fused using the feature fusion weight matrix to obtain a feature fusion matrix.

[0128] In a specific embodiment, the image data is preprocessed, including operations such as noise reduction, enhancement, and normalization. The preprocessed image data is input into a feature point detection module, and a differential filter is used to detect significant points in the image, such as corner points, edge points, etc. For each detected feature point, its direction parameter, scale parameter, and intensity parameter are extracted. The direction parameter is obtained by calculating the main direction of the pixel gradient in the neighborhood of the feature point, usually divided into 8 direction intervals; the scale parameter is determined by detecting the response value of the feature point in different scale spaces, and the typical value range is 1.0 to 4.0; the intensity parameter is obtained by calculating the size of the response value at the feature point, which indicates the significance of the feature point.

[0129] Based on these parameters, a local tensor descriptor is constructed around the feature point. The tensor descriptor is represented by a second-order symmetric tensor and contains the structural information of the feature point. The construction of the tensor field is achieved by calculating the pixel gradient covariance matrix in the neighborhood of the feature point. The neighborhood size is usually set to 15×15 pixels. For each feature point, its regional correlation with the surrounding feature points is calculated. The correlation is based on the similarity measure between tensors. The specific calculation uses the tensor inner product method. The larger the inner product value of the two tensors, the higher their correlation. In addition, the structural importance of the feature point is calculated. The structural importance is determined based on the eigenvalue size of the feature point tensor. The larger the eigenvalue difference, the more obvious the structural information and the higher the importance. Combining the regional correlation and structural importance, the potential energy value of the feature point is obtained. The potential energy value usually ranges from 0 to 1. The higher the potential energy value, the more important the feature point in the image structure.

[0130] The feature point with the largest potential energy value is selected as the first-level seed point. For example, if the potential energy value of a feature point is 0.95, which is higher than that of other feature points, it will be set as the first-level seed point. Multi-level seed points are determined based on the potential energy difference value. The potential energy difference threshold is set to 0.05. When the potential energy value of a feature point is greater than the potential energy difference between the previous level seed point and the feature point is greater than the threshold, the feature point is set as the next level seed point. For example, when a feature point with a potential energy value of 0.89 is detected, since the potential energy difference with the first level seed point is 0.06, which is greater than the threshold of 0.05, it is set as the second level seed point. For the third level seed point, its potential energy value is 0.83, and the potential energy difference with the second level seed point is 0.06, which is also greater than the threshold, so it is selected as the third level seed point, and so on.

[0131] After determining the multi-level seed points, the optimal traversal path is selected. This optimal traversal path is determined based on the connection cost between the seed points, which takes into account the spatial distance and potential energy differences between the seed points. The system uses a modified minimum spanning tree algorithm to find the optimal path, with the distance weighting factor in the connection cost calculation set to 0.7 and the potential energy difference weighting factor set to 0.3. After the optimal path is determined, the position index information of the feature points on the path is recorded, and a feature point extraction sequence is generated, which contains the feature point indices arranged in the optimal order.

[0132] Calculate the gradient distribution value of the tensor field. The gradient distribution value is obtained by calculating the tensor gradient size of each point in the tensor field. For areas where the gradient distribution value is greater than the preset gradient strength threshold, extract the first description information of the feature. The preset gradient strength threshold is set to 0.4. When the regional gradient distribution value is greater than 0.4, the structural features of the region are extracted, including the main direction, secondary direction, and direction strength ratio. The main direction represents the direction of the main eigenvector of the tensor, the secondary direction represents the direction of the secondary eigenvector of the tensor, and the direction strength ratio represents the ratio of the tensor eigenvalues, reflecting the degree of anisotropy of the structure. For areas where the gradient distribution value is less than the preset gradient strength threshold, extract the second description information of the feature, including the regional mean, variance, and entropy. These description information reflect the texture characteristics of the region. In this way, a multi-dimensional feature description of the feature points is obtained.

[0133] The feature extraction sequence is determined based on the feature point extraction sequence. The structural parameters and neighborhood data for each feature point are extracted. The structural parameters include the tensor eigenvalues ​​and eigenvectors of the feature point, while the neighborhood data includes the pixel value distribution in the area surrounding the feature point. This information is organized into a tensor feature vector, which has a dimension of 256 and contains a complete description of the feature point. These tensor feature vectors are organized in the order of the feature point extraction sequence to form a tensor feature sequence.

[0134] The phase difference between adjacent eigenvectors in the tensor feature sequence is calculated. The phase difference is obtained by calculating the angle between the eigenvectors. For example, the phase difference between the first and second eigenvectors is 15 degrees, and the phase difference between the second and third eigenvectors is 20 degrees. These phase difference values ​​are organized into a phase difference matrix, the size of which is related to the number of eigenvectors. Based on the phase difference matrix, the feature fusion weight coefficient is calculated. The weight coefficient is inversely proportional to the phase difference; the smaller the phase difference, the larger the weight coefficient. A feature fusion weight matrix is ​​generated and used to perform weighted fusion on the tensor feature sequence to obtain a feature fusion matrix. The feature fusion matrix contains a comprehensive description of the feature points and can be used for subsequent image matching and recognition tasks.

[0135] In an optional embodiment, a feature point with the largest potential energy value is selected as a first-level seed point, multiple levels of seed points are determined based on the potential energy difference, an optimal traversal path is selected, and position index information of the feature points on the optimal traversal path is recorded. Generating a feature point extraction sequence includes:

[0136] Determine the feature point with the largest potential energy value in the tensor field as the first-level seed point, establish a traversal window with the first-level seed point as the center, and calculate the potential energy difference of the feature points in the traversal window;

[0137] Select the feature point with the smallest potential energy difference as the second-level seed point, and record the traversal direction and traversal distance from the first-level seed point to the second-level seed point;

[0138] Determine a search sector in the traversal direction, and select a corresponding number of feature points in the search sector sorted from high to low in terms of potential energy values ​​according to a preset optimal number as third-level seed points;

[0139] Taking the third-level seed points as nodes, connecting them to form a candidate traversal path, and calculating the potential energy gradient of adjacent feature points on the candidate traversal path;

[0140] The candidate traversal path with the smallest potential energy gradient is selected as the optimal traversal path, and the position index information of the feature points along the optimal traversal path is recorded to generate a feature point extraction sequence.

[0141] In one specific embodiment, tensor field data to be processed is obtained. This tensor field can be multidimensional data such as image data, video data, or sound data. For each feature point, its potential energy value is calculated. The calculation of the potential energy value can be based on characteristics of the feature point, such as gradient, curvature, and entropy. For example, for image data, the Harris corner detection algorithm can be used to calculate the potential energy value of each pixel; for sound data, the energy distribution after short-time Fourier transform can be used as the potential energy value.

[0142] After the potential energy value calculation is completed, the feature point with the largest potential energy value is determined as the first-level seed point. For example, in a 100×100 two-dimensional tensor field, the feature point with coordinates (35, 42) is determined to have the largest potential energy value of 0.95, so this point is set as the first-level seed point.

[0143] A traversal window is established with the first-level seed point as the center. The size of the traversal window can be dynamically adjusted based on the complexity of the tensor field. For example, a 15×15 window can be set to cover a total of 225 feature points around the first-level seed point. Within this window, the potential energy difference between each feature point and the first-level seed point is calculated. The potential energy difference is calculated as the absolute difference between the potential energy value of the feature point and the potential energy value of the first-level seed point.

[0144] After calculating the potential energy differences between all feature points in the window and the first-level seed point, the feature point with the smallest potential energy difference is selected as the second-level seed point. For example, suppose the feature point with coordinates (30, 40) in the window has a potential energy value of 0.92 and a potential energy difference of 0.03 with the first-level seed point. This is the smallest potential energy difference among all feature points, so it is set as the second-level seed point.

[0145] Record the traversal direction and traversal distance from the first-level seed point to the second-level seed point. The traversal direction can be calculated by the coordinate difference between the two points. In the above example, the traversal direction from the first-level seed point (35, 42) to the second-level seed point (30, 40) is (-5, -2), and the traversal distance is sqrt(5 2 +2 2 )=sqrt(29)≈5.4 units.

[0146] The search sector is determined based on the traversal direction from the first-level seed point to the second-level seed point. The search sector is an area with the second-level seed point as the vertex and the traversal direction as the axis of symmetry, extending to both sides by a certain angle. For example, the search sector angle can be set to 60 degrees, which means that the search sector extends 30 degrees to both sides of the traversal direction.

[0147] Within the search sector, a set number of feature points are selected, sorted by their potential energy values ​​from high to low, according to the preset number of preferred points. For example, if the preset number of preferred points is 3, the three feature points with the highest potential energy values ​​within the search sector are selected as the third-level seed points. Assume that the coordinates of these three points are (25, 38), (28, 35), and (22, 37), and their potential energy values ​​are 0.88, 0.85, and 0.82, respectively.

[0148] The third-level seed points are used as nodes to connect and form candidate traversal paths. Since there are three third-level seed points, three candidate traversal paths can be formed: the path from the first-level seed point to the second-level seed point and then to the third-level seed point (25, 38); the path from the first-level seed point to the second-level seed point and then to the third-level seed point (28, 35); and the path from the first-level seed point to the second-level seed point and then to the third-level seed point (22, 37).

[0149] Calculate the potential energy gradient between adjacent feature points on each candidate traversal path. The potential energy gradient represents the rate of change of the potential energy value along the path and can be calculated by dividing the difference in the potential energy values ​​of adjacent feature points by the distance between the two points. For each candidate traversal path, calculate the potential energy gradient from the first-level seed point to the second-level seed point, and from the second-level seed point to the third-level seed point. The average of these values ​​is then used as the total potential energy gradient for the path.

[0150] For example, the potential energy gradient of the first path is: [(0.95-0.92) / 5.4+(0.92-0.88) / sqrt((30-25) 2 +(40-38) 2 )] / 2=[0.0056+0.0080] / 2=0.0068. Similarly, calculate the potential energy gradients of other paths.

[0151] The candidate traversal path with the smallest potential energy gradient is selected as the optimal traversal path. Assuming that the potential energy gradient of the second path is 0.0062 and the potential energy gradient of the third path is 0.0075, the second path has the smallest potential energy gradient and is selected as the optimal traversal path.

[0152] The position index information of the feature points along the optimal traversal path is recorded to generate a feature point extraction sequence. In this example, the position indexes of the feature points on the optimal traversal path are: (35, 42), (30, 40), (28, 35).

[0153] Through the above steps, we complete the process of determining multi-level seed points based on potential energy differences, selecting the optimal traversal path, and generating a feature point extraction sequence. This method can effectively extract representative feature point sequences from tensor fields, providing support for subsequent data processing and analysis.

[0154] Feature point extraction techniques typically include algorithms such as SIFT, SURF, and ORB, which detect feature points based on local gradient information or the statistical properties of image blocks. However, these methods have limitations when processing complex tensor fields. They typically evaluate and extract feature points independently, lacking consideration of the relationships between them. Traditional methods often employ fixed thresholds or fixed numbers of feature points, making them difficult to adapt to data of varying complexity. Furthermore, when processing multidimensional data, these methods struggle to effectively capture the topological relationships and potential energy variations between feature points.

[0155] The method of this embodiment establishes a multi-level seed point traversal mechanism based on potential energy difference. By considering the potential energy gradient relationship between feature points, a feature point extraction sequence is adaptively constructed. The introduction of the concept of potential energy difference allows the selection of feature points to consider not only the absolute potential energy value but also the relative change, enhancing sensitivity to local structural changes. A multi-level seed point strategy (first level, second level, and third level) is adopted to form a hierarchical feature point selection mechanism that effectively captures the structural relationship in the tensor field. The search sector is determined based on the traversal direction, and the potential energy gradient minimization criterion is used to select the optimal traversal path, so that the extracted feature point sequence can better reflect the essential characteristics of the tensor field.

[0156] like Figure 3 As shown in the figure, the network structure diagram of the feature point extraction algorithm is shown.

[0157] The distributed real-time facial feature extraction system based on edge computing in an embodiment of the present invention includes:

[0158] The first unit is used to perform time-series sampling on the facial image data collected by the terminal device to obtain a continuous multi-frame image sequence;

[0159] The second unit is used to establish an edge computing resource pool including multiple edge computing devices;

[0160] The third unit is used to construct a spatiotemporal feature correlation map for a continuous multi-frame image sequence, extract the facial motion trajectory, predict the face position using a recursive Bayesian method, and use the reprojection error compensation method to correct it, thereby generating a spatiotemporally aligned optimized facial image sequence;

[0161] The fourth unit is used to divide the optimized facial image sequence into continuous frame task packages, allocate the continuous frame task packages using a coloring marking method, establish a feature point motion prediction mechanism, and allocate the continuous frame task packages to an edge computing device to perform a feature extraction task;

[0162] The fifth unit is used to construct a local tensor field of feature points in the edge computing device and obtain potential energy values, establish multi-level seed points to determine the feature point extraction sequence, extract feature description information based on the tensor field gradient, generate a tensor feature sequence, and construct a feature fusion matrix;

[0163] The sixth unit is used to perform time series verification on the feature fusion matrix generated by each edge computing device, select the optimal feature fusion matrix according to a preset mutual information threshold, and send the optimal feature fusion matrix to the terminal device.

[0164] According to a third aspect of an embodiment of the present invention, an electronic device is provided, including:

[0165] processor;

[0166] a memory for storing processor-executable instructions;

[0167] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0168] According to a fourth aspect of an embodiment of the present invention, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described above is implemented.

[0169] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0170] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A distributed real-time facial feature extraction method based on edge computing, characterized in that: include: Perform time-series sampling on facial image data collected by the terminal device to obtain a continuous multi-frame image sequence; Establishing an edge computing resource pool including multiple edge computing devices; A spatiotemporal feature correlation map is constructed for continuous multi-frame image sequences to extract facial motion trajectories. The face position is predicted using a recursive Bayesian method and corrected using a reprojection error compensation method to generate a spatiotemporally aligned optimized facial image sequence. Divide the optimized facial image sequence into continuous frame task packages, allocate the continuous frame task packages using a coloring marking method, establish a feature point motion prediction mechanism, and allocate the continuous frame task packages to edge computing devices to perform feature extraction tasks; In the edge computing device, a local tensor field of feature points is constructed and potential energy values ​​are obtained. Multi-level seed points are established to determine the feature point extraction sequence. Feature description information is extracted based on the tensor field gradient, a tensor feature sequence is generated, and a feature fusion matrix is ​​constructed. The feature fusion matrix generated by each edge computing device is time-series verified, the optimal feature fusion matrix is ​​selected according to the preset mutual information threshold, and the optimal feature fusion matrix is ​​sent to the terminal device.

2. The method according to claim 1, characterized in that Constructing a spatiotemporal feature correlation graph for a continuous multi-frame image sequence and extracting the face motion trajectory includes: Extract a set of feature points from each frame in a continuous multi-frame image sequence, calculate the spatial distance between each feature point and surrounding feature points, adaptively determine the influence radius of the Gaussian kernel function based on the local distribution density of the feature points, and use the Gaussian kernel function to perform weighted calculation on the spatial distance to obtain the local density value of each feature point; The spatial gradient of the local density value of each feature point is calculated to obtain the density gradient field pointing to the direction of increasing local density value. The cosine value of the angle between the density gradient direction and the direction of the feature point connection is calculated. The spatial distance is weighted by combining the Gaussian kernel function to obtain the spatial correlation strength between the feature points. Calculating the ratio of local density values ​​of corresponding feature points in adjacent frame images as the density transfer ratio of the feature points, and establishing a temporal correlation relationship between the feature points in adjacent frames based on the density transfer ratio; The spatiotemporal feature correlation graph is constructed with feature points as vertices, spatial correlation strength as intra-frame edge weight, and density transfer ratio as inter-frame edge weight. In the spatiotemporal feature association graph, a density flow field is constructed based on the density gradient field and the density transfer ratio. The streamline trajectory is iteratively calculated along the intra-frame edge weight and inter-frame edge weight direction of the spatiotemporal feature association graph to obtain the face motion trajectory.

3. The method according to claim 2, characterized in that In the spatiotemporal feature association graph, a density flow field is constructed based on the density gradient field and the density transfer ratio. The streamline trajectory is iteratively calculated along the intra-frame edge weight and inter-frame edge weight direction of the spatiotemporal feature association graph to obtain the face motion trajectory, including: Extract the first direction vector of the density gradient field from the spatiotemporal feature correlation map, map the density transfer ratio to the second direction vector, and fuse the first direction vector and the second direction vector through adaptive weighting to construct a density flow field; In the density flow field, the feature point is used as the initial point of the streamline. The intra-frame edge weight of the spatiotemporal feature association graph is used to determine the motion direction of the feature point in the spatial domain. The inter-frame edge weight is used to determine the motion direction of the feature point in the temporal domain. The feature point position is iteratively updated until convergence, generating a streamline trajectory. Taking the area with the highest density of feature points as the starting position, the streamline trajectories are tracked in the forward and reverse directions in the density flow field to obtain the motion trajectory sequence of the feature point group. The motion trajectory of the face is obtained by curve fitting.

4. The method according to claim 1, wherein The optimized facial image sequence is divided into continuous frame task packages, the continuous frame task packages are allocated using a coloring marking method, a feature point motion prediction mechanism is established, and the continuous frame task packages are allocated to edge computing devices to perform feature extraction tasks, including: Obtaining the time interval of each frame image in the optimized face image sequence, calculating the motion displacement and motion direction of the image frame according to the time interval, and dividing the continuous frame images into continuous frame task packages of adaptive size based on the motion displacement and motion direction; Counting the number of feature points and the spatial distribution area of ​​each frame image in the continuous frame task package, calculating the ratio of the number of feature points to the spatial distribution area, and using the ratio as the correlation strength between the continuous frame task packages; Coloring and marking adjacent continuous frame task packets based on correlation strength, so that adjacent continuous frame task packets with correlation strength higher than a preset strength threshold are marked with different coloring marks, and the number of coloring marks is equal to the number of edge computing devices, to obtain a coloring mark for each continuous frame task packet; A motion state model of feature points is established for consecutive frame task packages with the same coloring mark, and the migration direction of feature points between consecutive frame task packages corresponding to different coloring marks is predicted based on the motion state model, and feature point detection is performed in the area corresponding to the feature point migration direction; Continuous frame task packages with the same color mark are assigned to the corresponding edge computing devices. Each edge computing device performs parallel feature extraction tasks based on the feature point migration direction, and dynamically adjusts the number of frames of the continuous frame task package according to the processor occupancy of the edge computing device.

5. The method according to claim 4, characterized in that The motion state model of feature points is established for the task package of consecutive frames with the same coloring mark, including: For the feature points in the continuous frame task package, a K-order neighborhood graph is constructed with the feature point as the center, the topological distance matrix and the angle difference matrix between the feature point and the neighboring feature points are calculated, and the distribution density of the feature points per unit area in the K-order neighborhood graph is obtained; Constructing local motion constraints based on the topological distance matrix, the angle difference matrix, and the feature point distribution density, calculating the affine transformation relationship of the feature points in the continuous frame task package, and extracting the scale component, rotation component, and translation component of the affine transformation relationship; The degree of violation of the local motion constraint condition and the residual of the affine transformation relationship are constructed as optimization targets, and the optimization targets are iterated by the gradient descent method. The motion state evolution law of the feature point and the actual observation correspondence are determined according to the optimization results to obtain the motion state model of the feature point.

6. The method according to claim 1, wherein In the edge computing device, a local tensor field of feature points is constructed and potential energy values ​​are obtained. Multi-level seed points are established to determine the feature point extraction sequence. Feature description information is extracted based on the tensor field gradient, and a tensor feature sequence is generated. The feature fusion matrix is ​​constructed, including: In the edge computing device, a local tensor descriptor is constructed between feature points, and the direction parameters, scale parameters, and intensity parameters of the corresponding feature points are extracted to establish a tensor field. The regional correlation and structural importance of the feature points in the tensor field are calculated to obtain the potential energy value of the feature points. Select the feature point with the largest potential energy value as the first-level seed point, determine the multi-level seed point based on the potential energy difference, select the optimal traversal path, record the position index information of the feature points on the optimal traversal path, and generate a feature point extraction sequence; Calculate the gradient distribution value of the tensor field, extract the first feature description information for the area where the gradient distribution value is greater than the preset gradient strength threshold, and extract the second feature description information for the area where the gradient distribution value is less than the preset gradient strength threshold, to obtain a multi-dimensional feature description of the feature point; Determine the feature extraction order according to the feature point extraction sequence, extract the structural parameters and neighborhood data of the feature points, generate tensor feature vectors and organize them according to the order of the feature point extraction sequence to form a tensor feature sequence; The phase difference values ​​of adjacent eigenvectors in the tensor feature sequence are calculated, a phase difference matrix is ​​constructed and the feature fusion weight coefficient is calculated to generate a feature fusion weight matrix. The tensor feature sequence is fused using the feature fusion weight matrix to obtain a feature fusion matrix.

7. The method according to claim 6, characterized in that The feature point with the largest potential energy value is selected as the first-level seed point. Multi-level seed points are determined based on the potential energy difference. The optimal traversal path is selected, and the position index information of the feature points on the optimal traversal path is recorded. The feature point extraction sequence is generated, including: Determine the feature point with the largest potential energy value in the tensor field as the first-level seed point, establish a traversal window with the first-level seed point as the center, and calculate the potential energy difference of the feature points in the traversal window; Select the feature point with the smallest potential energy difference as the second-level seed point, and record the traversal direction and traversal distance from the first-level seed point to the second-level seed point; Determine a search sector in the traversal direction, and select a corresponding number of feature points in the search sector sorted from high to low in terms of potential energy values ​​according to a preset optimal number as third-level seed points; Taking the third-level seed points as nodes, connecting them to form a candidate traversal path, and calculating the potential energy gradient of adjacent feature points on the candidate traversal path; The candidate traversal path with the smallest potential energy gradient is selected as the optimal traversal path, and the position index information of the feature points along the optimal traversal path is recorded to generate a feature point extraction sequence.

8. A distributed real-time facial feature extraction system based on edge computing, used to implement the method according to any one of claims 1 to 7, characterized in that: include: The first unit is used to perform time-series sampling on the facial image data collected by the terminal device to obtain a continuous multi-frame image sequence; The second unit is used to establish an edge computing resource pool including multiple edge computing devices; The third unit is used to construct a spatiotemporal feature correlation map for a continuous multi-frame image sequence, extract the facial motion trajectory, predict the face position using a recursive Bayesian method, and use the reprojection error compensation method to correct it, thereby generating a spatiotemporally aligned optimized facial image sequence; The fourth unit is used to divide the optimized facial image sequence into continuous frame task packages, allocate the continuous frame task packages using a coloring marking method, establish a feature point motion prediction mechanism, and allocate the continuous frame task packages to an edge computing device to perform a feature extraction task; The fifth unit is used to construct a local tensor field of feature points in the edge computing device and obtain potential energy values, establish multi-level seed points to determine the feature point extraction sequence, extract feature description information based on the tensor field gradient, generate a tensor feature sequence, and construct a feature fusion matrix; The sixth unit is used to perform time series verification on the feature fusion matrix generated by each edge computing device, select the optimal feature fusion matrix according to a preset mutual information threshold, and send the optimal feature fusion matrix to the terminal device.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images