Method and system for reconstructing continuous manifold of weak motion under dynamic low light frequency flash

Abstract

The application discloses a kind of dynamic weak light stroboscopic under transient weak motion feature's continuous manifold reconstruction method and system.For the problem of target trajectory break caused by stroboscopic environment, the application first generates threshold based on the brightness distribution in sliding window, and divides video frame into bright and dark field sequence;Subsequently, the broken trajectory fragments are extracted by mapping bright frame to low-dimensional manifold space, and the positive and negative geometric tangent lines are extracted at the breakpoint;By extending the tangent line to the dark field, the spatial intersection or midpoint of common perpendicular segment is calculated as a virtual geometric node;Finally, the broken trajectory is connected with the node as anchor point, and the topologically continuous trajectory skeleton is generated, and the continuous manifold surface of weak motion is reconstructed accordingly.The application effectively repairs the feature loss caused by dark field, and significantly improves the reconstruction accuracy of the real physical motion law of target.

Description

Technical Field

[0001] This invention relates to the field of information technology, and in particular to a method and system for continuous manifold reconstruction of transient weak motion characteristics under dynamic low-light flicker. Background Technology

[0002] Extracting the transient, subtle motion characteristics of target objects is a core requirement in specific scenarios such as industrial automation inspection, nighttime traffic monitoring, and precision physics experiments under pulsed light source illumination.

[0003] For example, in monitoring microsecond-level mechanical rebound vibrations or deformation of high-speed rotating components, the target often exhibits motion characteristics with small amplitude but extremely high frequency.

[0004] Existing techniques typically employ manifold reconstruction algorithms, which reconstruct the motion trajectory and dynamic features of a target by calculating the mapping of consecutive video frames from a high-dimensional pixel space to a low-dimensional manifold space.

[0005] However, in dynamic low-light environments with high-frequency flicker, existing technologies generally suffer from the technical defect of "trajectory breakage" and the resulting loss of transient characteristics.

[0006] Because stroboscopic light sources have a bright "sampling range" and a dark "dark field range," when a target undergoes transient motion and the "extreme point" where its motion changes direction falls within the dark field period, the camera will be unable to capture this crucial turning point.

[0007] In the existing manifold reconstruction space, this lack of physical information manifests as two trajectory segments with completely different directions of motion and discontinuous in the time dimension.

[0008] Traditional algorithms (such as Kalman filtering and spline curve smoothing) typically use complex mathematical functions to forcibly connect the two ends of the trajectory when dealing with such trajectory breakpoints.

[0009] While this approach achieves visual "connectivity," it directly loses the original "reversal" and "transient" characteristics of the object in terms of physical motion features. This results in severe distortion of the reconstructed manifold surface near extreme points, making it unable to accurately reflect the true physical motion laws of the target object. Summary of the Invention

[0010] This invention provides a continuous manifold reconstruction method for transient weak motion characteristics under dynamic low-light flicker, mainly comprising: S1, acquire the target motion video under dynamic low light flicker environment, calculate the brightness distribution of each video frame in the target motion video within a preset time sliding window, adaptively generate a brightness threshold, and divide each video frame into a bright frame sequence and a dark frame sequence accordingly. S2, map the bright frame sequence to a low-dimensional manifold space, and extract the weak motion trajectory of the target to obtain multiple discontinuous bright trajectory segments truncated in the time dimension by the dark frame sequence. The end point of the previous bright trajectory segment and the start point of the next bright trajectory segment, which are adjacent in time order, are jointly determined as a trajectory breakpoint pair to be spliced. S3, for each pair of trajectory breakpoints to be spliced, extract the forward geometric tangent at the end point of the previous bright trajectory segment and the reverse geometric tangent at the starting point of the next bright trajectory segment in the low-dimensional manifold space. S4, extend the forward geometric tangent and the reverse geometric tangent to the time interval of the corresponding dark field frame sequence, calculate the spatial intersection point or the midpoint of the common perpendicular segment of the two extended lines, and determine the obtained spatial intersection point or the midpoint of the common perpendicular segment as a virtual geometric node. S5, using the virtual geometric node as the trajectory connection anchor point, construct connecting line segments in the low-dimensional manifold space, connect the endpoint of the previous bright trajectory segment to the virtual geometric node, and connect the virtual geometric node to the starting point of the next bright trajectory segment, so as to achieve the continuous connection of the current trajectory breakpoint pair. S6, traverse all truncated bright trajectory segments in the target motion video, repeat steps S3 to S5, stitch all bright trajectory segments into a topologically continuous motion trajectory, and use the stitched topologically continuous motion trajectory as a topological skeleton to reconstruct the continuous manifold surface of the target's weak motion.

[0011] This invention provides a continuous manifold reconstruction system for transient weak motion characteristics under dynamic low-light flicker, mainly comprising: The brightness adaptive segmentation module is used to acquire target motion video under dynamic low light flicker environment, calculate the brightness distribution of each video frame in the target motion video within a preset time sliding window, adaptively generate a brightness threshold, and divide each video frame into a bright frame sequence and a dark frame sequence accordingly. The low-dimensional manifold mapping and trajectory breakpoint extraction module is used to map the bright frame sequence to a low-dimensional manifold space and extract the weak motion trajectory of the target to obtain multiple discontinuous bright trajectory segments truncated in the time dimension by the dark frame sequence. The endpoint of the previous bright trajectory segment and the starting point of the next bright trajectory segment, which are adjacent in time order, are jointly determined as a trajectory breakpoint pair to be spliced. The geometric tangent extraction module is used to extract, in the low-dimensional manifold space, the forward geometric tangent at the end point of the previous bright trajectory segment and the reverse geometric tangent at the start point of the next bright trajectory segment for each pair of trajectory breakpoints to be spliced. The virtual geometric node determination module is used to extend the forward geometric tangent and the reverse geometric tangent to the time interval of the corresponding dark field frame sequence, calculate the spatial intersection point or the midpoint of the common perpendicular segment of the two extended lines, and determine the obtained spatial intersection point or the midpoint of the common perpendicular segment as a virtual geometric node. The trajectory connection anchor point construction module is used to construct connecting line segments in the low-dimensional manifold space, using the virtual geometric node as the trajectory connection anchor point, connecting the endpoint of the previous bright trajectory segment to the virtual geometric node, and connecting the virtual geometric node to the starting point of the next bright trajectory segment, so as to realize the continuous connection of the current trajectory breakpoint pair. The topology continuous stitching and manifold surface reconstruction module is used to traverse all truncated bright trajectory segments in the target motion video, repeat steps S3 to S5, stitch all bright trajectory segments into a topology continuous motion trajectory, and use the stitched topology continuous motion trajectory as a topology skeleton to reconstruct the continuous manifold surface of the target's weak motion.

[0012] The technical solutions provided by the embodiments of the present invention may include the following beneficial effects: This invention discloses a continuous manifold reconstruction method for transient weak motion features under dynamic low-light flicker conditions. Addressing the problem of trajectory breakage caused by brightness variations in target motion videos under dynamic low-light flicker environments, it proposes an innovative solution for adaptive brightness threshold partitioning and trajectory stitching. This invention calculates the brightness distribution of video frames and adaptively generates thresholds to divide the frame sequence into bright and dark frames. The weak motion trajectory of the target is extracted from the bright frames. For trajectory breakpoints caused by dark frames, geometric tangent extensions and virtual node construction within a low-dimensional manifold space are used to accurately connect the broken trajectory fragments, ultimately reconstructing a topologically continuous motion trajectory and manifold surface. This method effectively solves the problem of discontinuous motion features under low-light flicker conditions, significantly improves the accuracy of capturing and reconstructing transient weak motion, and provides reliable technical support for target tracking and analysis under complex lighting conditions. Attached Figure Description

[0013] Figure 1 This is a flowchart of a continuous manifold reconstruction method for transient weak motion characteristics under dynamic low-light flickering according to the present invention.

[0014] Figure 2 This is a schematic diagram of the structure of a continuous manifold reconstruction system for transient weak motion characteristics under dynamic weak light flickering according to the present invention. Detailed Implementation

[0015] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] Terminology Explanation Transient weak motion characteristics: These refer to changes in the physical motion state of a target object within an extremely short timescale, characterized by extremely small spatial displacement amplitude but usually accompanied by high-frequency properties. In the dynamic low-light stroboscopic scene of this invention, this characteristic specifically manifests as microsecond-level mechanical rebound vibrations, instantaneous deformation or forced vibrations of high-speed rotating components, etc. Due to the dark field masking effect of the stroboscopic light source, the transitions of these minute movements are easily lost in the video frame sequence, and are the core physical properties that this invention needs to recover through manifold reconstruction.

[0017] Continuous manifold reconstruction refers to the data processing and modeling process of mapping high-dimensional discrete observation data to a low-dimensional topological space and restoring the inherent geometric coherence and physical smoothness between data within this space. In this invention, this technique is used to solve the problem of trajectory abrupt breaks caused by illumination flicker. By establishing geometric and topological connections between the broken trajectory segments, a continuous surface reflecting the true motion of the target is reconstructed. As a general technical term, its specific implementable techniques include Locally Linear Embedding (LLE) manifold reconstruction algorithms, Isomap reconstruction algorithms, and Laplacian Eigenmaps reconstruction algorithms.

[0018] Brightness distribution within a time-sliding window: refers to the statistical fluctuation pattern of the overall grayscale value or pixel brightness in a local video frame sequence that is extracted for a fixed number of frames on the timeline and slides at specific steps. In this invention, this distribution is used to dynamically capture the periodic alternation of light and dark caused by stroboscopic light sources, ensuring that the extracted brightness benchmark can adapt to local changes in light and shadow caused by target movement in the video, thereby preventing misjudgments caused by sudden changes in a single frame.

[0019] Sliding window algorithm: refers to an algorithm model for time series or streaming data that uses a fixed-length window that slides forward along the time axis at preset steps to capture local data for dynamic calculation and updating. In the brightness calculation stage of this invention, this algorithm is used to continuously update the statistical characteristics of a local video frame set. As a general technical term, its specific implementable techniques include fixed-step sliding window algorithms, event-triggered variable-length sliding window algorithms, etc.

[0020] A brightness distribution histogram is a two-dimensional mathematical chart plotted with the gray levels of image pixels on the x-axis and the frequency of occurrence of the corresponding gray level in the image sequence or the cumulative total number of pixels on the y-axis. In this invention, this histogram gathers the brightness information of all frames within a sliding window, intuitively highlighting the bright reflective peaks and dark background valleys caused by flickering environments, serving as the data foundation for adaptive binarization segmentation.

[0021] The maximum inter-class variance method is an image adaptive thresholding mechanism based on global statistical features. Its core principle is to find a gray-level segmentation point that maximizes the global variance between foreground and background pixels, thus achieving optimal binarization. In this invention, this mechanism is used to automatically extract the initial brightness discrimination threshold from the brightness distribution histogram. As a general technical term, its specific implementable techniques include the classic Otsu method, the two-dimensional Otsu algorithm, and the multi-threshold Otsu segmentation algorithm.

[0022] Brightness variance: refers to the statistical measure of the dispersion of the brightness value of each pixel or the average brightness value of each frame in a local video frame set from its mathematical expectation (mean). In this invention, high brightness variance usually indicates a strong abrupt change in the stroboscopic light source or localized high-brightness reflection on the target surface. The system uses this to determine the severity of the current illumination fluctuation and as a key input parameter for dynamically correcting the initial brightness threshold.

[0023] Dynamic threshold compensation algorithm: This refers to an algorithm that, based on an initial global classification threshold, uses local statistical features of the data (such as standard deviation and skewness) to construct an offset coefficient, and then adaptively increases or decreases the initial threshold in real time. In the application of this invention, this algorithm effectively prevents the misclassification of weak reflections in the target object as dark background, greatly improving the robustness of segmentation in low light. As a general technical term, its specific implementable techniques include the Niblack compensation algorithm based on local mean and standard deviation, the Sauvola dynamic threshold algorithm, and the adaptive offset algorithm based on histogram skewness.

[0024] Bright frame sequences and dark frame sequences: Bright frame sequences refer to a set of video frames where, after adaptive brightness threshold filtering, the overall brightness of the image or the brightness of the target's region of interest is higher than the threshold, clearly presenting the target's texture and motion state; dark frame sequences refer to a set of video frames where the brightness is lower than the threshold due to the strobe light source being turned off or obscured, resulting in a severe lack of target physical information. In this invention, the accurate division of these two sequences is a prerequisite for subsequent identification of trajectory break intervals.

[0025] Manifold learning algorithms and dimensionality reduction algorithms refer to a class of unsupervised or semi-supervised machine learning algorithms that aim to discover the low-dimensional intrinsic manifold structure of a high-dimensional original data space and project the data to a low-dimensional space through mathematical transformations to remove redundant noise and preserve key features. In the weak motion extraction stage of this invention, this mechanism is used to transform a complex video pixel matrix into a concise sequence of trajectory coordinates. As a higher-level technical term, its lower-level implementable techniques include Principal Component Analysis (PCA) dimensionality reduction algorithms, t-distributed random nearest neighbor embedding (t-SNE) algorithms, Local Preserving Projection (LPP) algorithms, and autoencoder nonlinear dimensionality reduction networks.

[0026] Inter-frame differencing: This refers to an image processing mechanism that extracts regions with large absolute differences by calculating the difference between corresponding pixel values ​​in two or more adjacent frames of a video sequence and setting a threshold, thereby highlighting moving targets. In the preprocessing stage of this invention, this method is often used in conjunction with target detection to quickly filter out static backgrounds and identify candidate regions with weak motion.

[0027] Phase correlation and feature point matching algorithms refer to alignment algorithms used to calculate spatial relative translation, rotation, or scaling transformation parameters between adjacent images. Phase correlation methods determine the translation amount through the cross power spectrum in the frequency domain; feature point matching algorithms perform registration by extracting local invariant features of the image. In this invention, these algorithms are used to offset camera jitter errors before manifold dimensionality reduction. As a general technical term, the specific implementable techniques under the feature point matching algorithm include Scale Invariant Feature Transform (SIFT) algorithms, Accelerated Robust Feature Transform (SURF) algorithms, and ORB feature matching algorithms.

[0028] Principal Component Analysis (PCA) and Eigenvalue Decomposition: PCA is a multivariate statistical analysis method that transforms a set of potentially correlated variables into a set of linearly uncorrelated variables (i.e., principal components) through orthogonal transformations. Eigenvalue decomposition, on the other hand, decomposes the covariance matrix into linear algebraic operations of eigenvalues ​​and corresponding eigenvectors. In the step of extracting the geometric tangent in this invention, the covariance matrix is ​​constructed for the local neighborhood point set, and eigenvalue decomposition is performed to extract the first or second principal component eigenvector corresponding to the largest eigenvalue. Mathematically, this vector represents the direction of the greatest spatial variation in the local point set, i.e., the instantaneous geometric tangent of the trajectory.

[0029] Optical flow algorithm: refers to an algorithmic mechanism that calculates the instantaneous velocity vector field of each pixel in the current frame by utilizing the temporal changes of pixels in an image sequence and the correlation between adjacent frames. In this invention, this algorithm is often used to track the fine motion displacement of a target within a bright frame in a low-dimensional manifold space or aligned image space. As a general technical term, its specific implementable techniques include the Lucas-Kanade (LK) sparse optical flow algorithm, the Horn-Schunck (HS) global dense optical flow algorithm, and the Farnebäck polynomial expansion optical flow algorithm.

[0030] Faint motion trajectories and bright trajectory segments: A faint motion trajectory is a three-dimensional or two-dimensional spatial curve that represents the change in the true position of a target, formed by connecting coordinate points in the manifold space in time sequence; a bright trajectory segment specifically refers to an independent and discontinuous local effective observation trajectory segment left behind after being truncated in the time dimension by the dark field frame sequence.

[0031] Spatiotemporal correlation constraints and spatiotemporal distance: Spatiotemporal correlation constraints refer to the logical judgment rules established when determining whether two independent trajectory segments originate from the same physical target, comprehensively considering the rationality of the time interval and the feasibility of the spatial Euclidean distance span; spatiotemporal distance is the specific joint penalty value that quantifies this constraint. In this invention, the system filters out breakpoint pairs that truly meet the physical continuity conditions by determining whether the spatiotemporal distance of the breakpoint pair is less than a set trajectory splicing threshold (i.e., the physical limit estimated based on the target's maximum possible acceleration and velocity).

[0032] A trajectory breakpoint pair refers to a set of mathematical point pairs that are strictly adjacent in a time series, consisting of the endpoint coordinates of the preceding bright trajectory segment and the starting coordinates of the following bright trajectory segment. In this invention, this breakpoint pair defines the spatial start and end positions of the information loss interval caused by the flickering dark field, and is the direct target of subsequent tangent extension and node insertion.

[0033] The first local neighborhood point set and the second local neighborhood point set refer to the sets of all historical or future manifold coordinate points intercepted within a preset spatial radius or time step in the low-dimensional manifold space, with the endpoint of the preceding segment and the starting point of the following segment as the centers of the spheres, respectively. In this invention, these two point sets contain the local geometric curvature structure and motion inertia of the trajectory before the break and at the beginning of recovery, and are the original data samples for constructing the covariance matrix to extract the spatial tangent.

[0034] Forward and reverse geometric tangents: These refer to the spatial direction vectors representing the instantaneous extension trend of the trajectory, extracted through mathematical regression or principal component analysis on both sides of the trajectory breakpoint. The forward geometric tangent is obtained by calculating the sign of the inner product between the first principal component eigenvector and the motion direction vector of the preceding bright trajectory segment, pointing towards the dark field region. The reverse geometric tangent is obtained by inverting the sign of the inner product between the second principal component eigenvector and the initial motion direction vector of the following bright trajectory segment, also pointing towards the dark field region. Together, they constitute the geometric guide for predicting the boundary of hidden motion trajectories in the dark field.

[0035] Linear extrapolation algorithms and trajectory extrapolation: Trajectory extrapolation refers to the calculation process of predicting the spatial coordinates of a target in an unknown time interval based on the transient kinematic characteristics (such as velocity vector and tangent direction) of a known trajectory segment using a mathematical model. In this invention, extending the trajectory along the tangent direction to find the intersection point is its specific manifestation. As a general technical term, its specific implementable techniques include linear extrapolation algorithms based on first derivatives, second-order parabolic extrapolation algorithms based on kinematic equations, and time-series extrapolation algorithms based on autoregressive moving average (ARMA) models.

[0036] Spatial intersection point or midpoint of common perpendicular segment and shortest spatial distance: In three-dimensional or high-dimensional manifold space, the forward and reverse extension lines generated by trajectory extrapolation are usually non-coplanar skew lines. The shortest spatial distance refers to the minimum perpendicular distance between these two skew lines calculated using the Euclidean distance formula. When this distance is less than a tolerance threshold, their approximate intersection point is defined as the spatial intersection point; if it is greater than the threshold, the geometric midpoint of the common perpendicular segment between the two skew lines is calculated. In this invention, this point represents the most probable spatial location where the target motion undergoes a physical reversal or extreme value change within the dark field interval.

[0037] Mapping transformation and inverse transformation function: Mapping transformation refers to the process of transforming data from one mathematical space (such as manifold space) to another mathematical space (such as high-dimensional pixel space or a unified physical reference frame); the inverse transformation function is the inverse analytical expression or neural network layer of this mapping process. In this invention, since the spatial intersection points are calculated in the abstract space of tangent extension, an inverse mapping transformation must be performed to accurately reset them back to the coordinate system of the original manifold skeleton, thereby establishing legitimate virtual geometric nodes. Specific implementable techniques include the isometric mapping inverse transformation function and a decoder network mapping mechanism based on a deep learning autoencoder.

[0038] Virtual geometric nodes and trajectory connection anchors: Virtual geometric nodes are supplementary data points constructed out of thin air through spatial geometric calculations, used to fill in key turning points lost during the dark field; when the node is actually used to guide and stabilize the smooth connection of the broken trajectories before and after, its role in the manifold topology is defined as the trajectory connection anchor.

[0039] Interpolation methods, linear interpolation algorithms, and Bézier curve fitting methods: Interpolation methods are a type of numerical analysis mechanism that uses a known set of discrete data points to supplement continuous data points between data points by constructing specific analytical functions. In the trajectory stitching stage of this invention, they are used to generate transition coordinates between breakpoints and anchor points. As a general technical term, its specific implementable techniques include first-order linear interpolation algorithms, higher-order Bézier curve fitting methods, and B-spline curve fitting algorithms based on control points.

[0040] The first and second associated trajectory data refer to the new spatial point coordinate sequences generated by using interpolation or fitting algorithms to connect the endpoint of the previous bright trajectory segment to a virtual geometric node (first associated trajectory) and to connect the virtual geometric node to the starting point of the next bright trajectory segment (second associated trajectory). These two sets of data achieve a smooth stitching of the physical motion cliffs caused by stroboscopic flicker.

[0041] Continuous connectivity and topological continuity, and thinning algorithms: Continuous connectivity, in a geometric and analytical mathematical sense, refers to the elimination of coordinate jumps at the junctions of trajectory segments, achieving a state of continuity everywhere ($C^0$ or even $C^1$ continuity); topological continuity further emphasizes that the overall trajectory composed of multiple segments has no topological holes or breaks in its spatial structure. A thinning algorithm is a morphological algorithm that iteratively erodes the edges of an image or data matrix, ultimately preserving a core topological skeleton with a single-pixel width. In this invention, the thinning algorithm is used to extract the topological skeleton reflecting the core motion laws from the stitched coarsely connected trajectory.

[0042] The local Frenet frame and its principal and secondary normal vectors: In differential geometry, the local Frenet frame is a locally active coordinate system defined at each point on a smooth curve in space, consisting of orthogonal basis vectors. In this frame, except for the tangent vector, the principal normal vector points in the direction of the most severe curvature of the curve (the center of curvature), while the secondary normal vector is perpendicular to the oscillating plane formed by the tangent and principal normal vectors. In this invention, constructing the frame and utilizing these two normal vectors is to determine the precise spatial orientation of the target manifold surface as it expands laterally and normally.

[0043] Object detection algorithms and physical geometric bounding boxes: Object detection algorithms refer to machine vision models that can automatically identify specific object categories in complex images or videos and output their spatial locations and bounding boxes; physical geometric bounding boxes are the two-dimensional rectangular or three-dimensional polyhedral boundary systems that tightly enclose the target object, as output by the algorithm. In this invention, by extracting the bounding box dimensions and mapping them to preset geometric dimensions in manifold space, physical boundary constraints are provided for the extension of one-dimensional trajectories to three-dimensional surfaces. As a higher-level technical term, the lower-level implementable technologies of object detection algorithms include the YOLO (YouOnlyLookOnce) series of deep learning models, the Faster R-CNN detection algorithm, and traditional detection mechanisms based on HOG features and SVM classifiers.

[0044] Spline functions and spline interpolation algorithms: A spline function is a piecewise mathematical function composed of multiple polynomials with a certain continuity (such as continuity of the first or second derivative). Spline interpolation algorithms utilize spline functions to fit and wrap discrete skeleton points and extended cross-section point sets onto a surface. In the step of extending the one-dimensional topological skeleton to a two-dimensional or three-dimensional manifold space in this invention, spline interpolation ensures that the generated reconstructed surface does not produce sharp, unnatural wrinkles in both visual and physical deformation analysis. As a general technical term, its sub-implementable techniques include cubic spline interpolation algorithms, non-uniform rational B-spline (NURBS) interpolation surface construction algorithms, and tensor product spline surface fitting algorithms.

[0045] Kalman filtering and spline smoothing: Kalman filtering is an algorithm that uses the state equations of a linear system to perform optimal recursive estimation of the system state using system input and output observation data. It is often used to filter transient trajectories with Gaussian noise. Spline smoothing, on the other hand, uses spline functions to constrain data points, employing a geometric smoothing technique to minimize curvature energy. In the continuous manifold reconstruction system of this invention, both can be used as auxiliary or comparative algorithms to reduce high-frequency observation jitter caused by system flicker, further improving the noise resistance of continuous manifold surface reconstruction.

[0046] One-dimensional observation vector: refers to a single-row column formed by flattening multi-dimensional raw data (such as a two-dimensional pixel matrix in a video frame or a region of interest image) according to a preset scanning logic (such as row-first or column-first). In this invention, this transformation converts the spatial topological distribution of the image into an algebraic vector form, enabling the system to perform batch matrix operations on the global features of bright frames using linear algebra methods. This serves as the basic input format for subsequent dimensionality reduction of manifold spatial features.

[0047] Dimensionality reduction algorithms refer to a class of computational mechanisms that use mathematical transformations to project sample points from a high-dimensional original feature space to a lower-dimensional subspace, while preserving as much of the essential structural relationships between data as possible (such as variance distribution, geodesic distance, or local nearest neighbor topology). In this invention, dimensionality reduction algorithms are used to remove sensor thermal noise and background redundancy information in low-light flicker environments, extracting the intrinsic manifold coordinates reflecting the true physical trajectory of the target object from complex pixel evolution. As a general technical term, specific implementable techniques include linear dimensionality reduction algorithms (such as Principal Component Analysis (PCA), Local Preserving Projection (LPP), and Independent Component Analysis (ICA)) and nonlinear dimensionality reduction algorithms (such as Local Linear Embedding (LLE), Isomap, t-distributed random nearest neighbor embedding (t-SNE), and Autoencoder dimensionality reduction networks).

[0048] Decoder Network: In a neural network model, this refers to the component used to perform the inverse mapping from a low-dimensional latent space to a high-dimensional original data space or target reconstruction space. In this invention, its main function is to receive the coordinates of virtual geometric nodes within the low-dimensional manifold space and, through pre-trained nonlinear transformation weights, restore them to coordinate points that conform to the physical deformation laws or physiological motion characteristics of the target object. As a higher-level technical term, its more specific implementable techniques include multilayer perceptron (MLP) decoding structures based on fully connected layers, reconstruction networks based on transposed convolution, and deep feature reconstruction networks with residual connections.

[0049] Feature Reconstruction Mapping (MRMAP) refers to the computational process of reprojecting abstract feature vectors, after dimensionality reduction, back onto discrete data points with clear physical meaning or geometric constraints using mathematical transformation functions or learned mapping operators. In this invention, this mapping is used to eliminate feature loss or nonlinear distortion during dimensionality reduction, ensuring that the spatial intersections or midpoints of common perpendicular segments calculated in the low-dimensional manifold space accurately reflect the transient, subtle motion state of the target when reverted to the original manifold skeleton coordinate system. This mapping process is typically used as the inverse operation of dimensionality reduction algorithms, such as the inverse transformation function of isomap or the decoder mapping logic in an autoencoder.

[0050] It should be further explained that, for nonlinear manifold dimensionality reduction algorithms such as LLE and t-SNE that lack explicit inverse analytical formulas, this invention, in addition to using the aforementioned autoencoder decoder network for inverse mapping, also provides an alternative interpolation and reconstruction scheme based on a radial basis function (RBF) neural network. Specifically, an RBF interpolation network is trained online using the known coordinates of bright trajectory segments in the low-dimensional manifold space as input and their corresponding high-dimensional physical space coordinates as output. Subsequently, the calculated low-dimensional virtual geometric nodes are input into this RBF network, and using the established local smoothing weights, the approximate coordinates of the virtual geometric nodes in the high-dimensional physical space are directly calculated, thus effectively solving the inverse calculation problem of parameterless dimensionality reduction algorithms.

[0051] The 'decoder network' or 'inverse transform function' mentioned in this invention has the following general implementation logic and training method: Since nonlinear dimensionality reduction algorithms typically lack explicit inverse mathematical analytical expressions, this invention constructs a multilayer perceptron (MLP) or deconvolutional decoder network to map virtual geometric nodes in a low-dimensional manifold space back to the original high-dimensional physical space. This network is trained through offline self-supervised learning, using continuous high-dimensional video frames without flicker as the label set, and their dimensionality reduction results as the network input. During training, mean squared error (MSE) and local preservation loss are jointly used as a comprehensive loss function to enforce that the reconstructed physical nodes conform to the objective laws of non-rigid continuous deformation of the target object in three-dimensional space, ensuring the physical authenticity of the feature reconstruction mapping.

[0052] Example 1 This embodiment provides a continuous manifold reconstruction method for transient weak motion characteristics under dynamic low-light stroboscopic illumination, primarily applied in industrial automation inspection scenarios, specifically for monitoring micro-vibration of high-speed rotating mechanical spindles under stroboscopic illumination. In this scenario, transient weak motion characteristics refer to the motion properties of the mechanical spindle that occur within an extremely short timescale (microseconds), exhibiting extremely small spatial displacements (micrometers) but containing key physical state changes such as mechanical fatigue or eccentricity faults. To accurately capture this characteristic, this embodiment employs continuous manifold reconstruction technology, a data processing procedure that maps high-dimensional discrete observation data to a low-dimensional mathematical space and restores the topological coherence and geometric smoothness of the data within that space.

[0053] During the monitoring process, the system first acquires video footage of the mechanical spindle movement under dynamic low-light flickering conditions using a high-speed industrial camera. Due to the high-frequency flickering of the light source, the system utilizes a sliding window algorithm (a dynamic processing algorithm for time-series data that extracts local data by setting a fixed-length interval that slides according to time steps; its input is a continuous video frame stream, and its output is a set of local video frames) to calculate the brightness distribution of each video frame in the target motion video within a preset time sliding window. This distribution specifically characterizes the statistical regularity of pixel grayscale values ​​within the window. Subsequently, a brightness distribution histogram of the image pixels is extracted (i.e., a two-dimensional statistical chart reflecting the frequency of occurrence of pixels at each grayscale level in the image). Based on this, the maximum inter-class variance method (a higher-level term, specifically implemented as the Otsu method, an adaptive image segmentation algorithm based on a global threshold) is used to extract an initial brightness threshold from the histogram. The Otsu method takes the brightness distribution histogram as input and outputs the initial threshold. Its core mathematical formula is to find the variance between classes. Maximum gray level t: in and These represent the foreground and background pixel ratios, divided according to a threshold t. and This corresponds to the average brightness. Next, the brightness variance (a statistic reflecting the severity of brightness fluctuations) of each video frame within the time sliding window is calculated, and a compensation factor is determined based on this brightness variance. This embodiment uses a dynamic threshold compensation algorithm (an algorithm that corrects the global threshold based on local statistical characteristics), the formula of which is... ,in S is a weighting adjustment coefficient related to the dynamic range of the camera sensor, and S is the histogram skewness. This represents the standard deviation of brightness. In this embodiment, The value range is strictly limited to between 0.1 and 0.5; specifically, the system obtains the dark current noise variance of the target motion video under predetermined extremely dark conditions. ,set up This allows the weighting adjustment coefficient to adaptively adjust according to the ambient floor noise. The initial brightness threshold is then dynamically corrected using this compensation factor to obtain an adaptive brightness threshold. Based on this threshold, each video frame is divided into a bright frame sequence (a set of frames with a mean brightness greater than T) that reflects clear reflections of the main axis and a dark frame sequence (a set of frames with a mean brightness less than or equal to T) that lacks background and lighting.

[0054] For the extracted bright frame sequence, the system employs a manifold learning algorithm (a higher-level term, specifically Principal Component Analysis (PCA) or Isomap, etc., for high-dimensional feature reduction; this embodiment specifically uses PCA) to map it to a low-dimensional manifold space (referring to the lower-dimensional, locally homeomorphic subspace of Euclidean space hidden beneath high-dimensional data) to obtain the manifold coordinate sequence. Before mapping, to align the target, the system uses inter-frame difference (an algorithm that extracts motion regions by calculating the absolute difference between pixels in two adjacent frames; the input is two adjacent frames, and the output is a difference binary image) combined with phase correlation (an algorithm that estimates image translation parameters in the frequency domain based on Fourier transform) for pixel alignment. Based on the manifold coordinate sequence, an optical flow algorithm (specifically the Lucas-Kanade optical flow method, which calculates the instantaneous velocity vector field of pixels by assuming constant local neighborhood optical flow) is used to extract the weak motion trajectory of the target. Due to the time truncation of the dark-field frame sequence, this trajectory breaks into multiple discontinuous bright trajectory segments. The system calculates the spatiotemporal correlation constraints between the endpoints of each segment (referring to a joint penalty function that comprehensively considers the time interval and spatial Euclidean distance). When the calculated spatiotemporal distance is less than a preset trajectory splicing threshold, the endpoint of the preceding bright trajectory segment and the starting point of the following bright trajectory segment, which are adjacent in time sequence, are jointly determined as a pair of trajectory breakpoints. The preset trajectory splicing threshold is not a fixed value, but a physical boundary dynamically determined based on the prior a priori dynamic limit of the target object. Specifically, if the maximum physical acceleration of the target object is known to be... The continuous time span caused by the dark frame sequence is The theoretical upper limit of the target's displacement within the dark field interval. The calculation is as follows: ,in Let be the instantaneous velocity of the target at the moment of fracture. The system will then define the displacement boundary values ​​in this physical space. The Jacobian matrix norm is converted into an upper bound for geodesic distance in the manifold space using a low-dimensional manifold mapping algorithm, and this upper bound is set as the trajectory stitching threshold. This mechanism ensures that the trajectory breakpoint stitching strictly conforms to classical kinematic laws and eliminates mismatches of non-homogeneous targets. Specifically, the process of conversion using the Jacobian matrix norm includes: in the low-dimensional manifold space, for the endpoint of the previous bright trajectory segment... The k nearest neighbors of the target object in the low-dimensional space are extracted. A local mapping linear operator J from the low-dimensional space to the high-dimensional observation space is constructed using Local Linear Embedding (LLE) weights or Local Least Squares fitting. The norm of the Jacobian matrix J is used as a scaling factor. The upper limit of geodesic distance within the manifold space; Calculated as This step solves the technical challenge of inconsistent physical scales during nonlinear dimensionality reduction.

[0055] For each pair of trajectory breakpoints to be spliced, in the low-dimensional manifold space, extract the first local neighborhood point set (the set of historical manifold coordinates within a preset radius centered at the endpoint) at the end of the previous bright trajectory segment and the second local neighborhood point set at the start of the next bright trajectory segment. Construct covariance matrices based on these two point sets and perform eigenvalue decomposition (decomposing the matrix into linear algebraic operations of eigenvectors and eigenvalues) to extract the first and second principal component eigenvectors corresponding to the largest eigenvalues. Calculate the inner product of the first principal component eigenvector and the motion direction vector of the previous bright trajectory segment. In this step, the specific method for obtaining the motion direction vector of the previous bright trajectory segment is as follows: extract the endpoint of the previous bright trajectory segment and its m immediately preceding manifold coordinate points (m≥2), and use the least squares method to... Linear fitting is performed on each point, and the direction vector of the fitted line is taken as the instantaneous motion direction vector at the endpoint. If the inner product is positive, it indicates that the directions are consistent, and it is directly determined as the positive geometric tangent representing the instantaneous motion trend at the endpoint; similarly, the inner product of the second principal component feature vector and the starting motion direction vector of the subsequent bright trajectory segment is calculated, and the direction is adjusted according to the sign of the inner product to determine the reverse geometric tangent. The instantaneous motion direction vector is obtained by extracting the corresponding breakpoint (endpoint or startpoint) and its m adjacent manifold coordinate points on the time axis ( Using the least squares method to analyze this... Linear regression or higher-order polynomial fitting is performed at each coordinate point. The tangent vector of the fitted curve at the break point is defined as the instantaneous motion direction vector. By performing an inner product operation between this vector and the principal component eigenvector, the polarity of the geometric tangent can be accurately corrected, ensuring that the tangent always points to the dark field region, thereby achieving accurate prediction of 'invisible' trajectories in the dark field.

[0056] After obtaining the forward and reverse geometric tangents, the system performs trajectory extrapolation (specifically, a linear extrapolation algorithm is used, with the input being endpoint coordinates, tangent vector, and time step, and the output being the predicted spatial coordinates for the future or history), generating forward and reverse extension lines extending into the dark-field frame time interval. Subsequently, the shortest spatial distance d between the two extension lines is calculated using the spatial Euclidean distance formula. in For online purposes, Let d be the direction vector. If d exceeds the allowable error range, then find the common perpendicular between the two skew lines and calculate its geometric center position, i.e., the spatial intersection point or the midpoint of the common perpendicular segment. This spatial intersection point or the midpoint of the common perpendicular segment is directly determined as a virtual geometric node, or a mapping transformation (specifically, an inverse transformation function from low-dimensional to high-dimensional) is performed on this spatial intersection point or the midpoint of the common perpendicular segment to determine it as a virtual geometric node used to fill the trajectory gaps.

[0057] The virtual geometric node is used as the trajectory connection anchor point (i.e., the pivot coordinates in the manifold space used to guide and stabilize the broken trajectories on both sides) to construct connecting line segments. Specifically, the system uses a linear interpolation algorithm (input is the coordinates of the two endpoints, output is a set of coordinates evenly distributed proportionally between the two points) to construct the first connecting line segment from the previous endpoint to the virtual node, generating the first associated trajectory data; similarly, the second associated trajectory data to the next starting point is generated. These two types of data are fused to complete the splicing of the previous and subsequent bright trajectory segments, achieving continuous connectivity of the current trajectory breakpoint pair (meaning it is mathematically differentiable everywhere or at least has...). (Continuous physical connection state).

[0058] The system iterates through all breakpoint pairs in the video, repeating the above operation to stitch together a topologically continuous (meaning the trajectory has no breaks or holes in its spatial structure and maintains its connectivity) motion trajectory, which is then used as the topological skeleton representing the main motion path. The principal normal vector of each discrete node on the skeleton is calculated. and binormal vector A local FrenetFrame (an orthogonal coordinate system defined on points of a spatial curve) is constructed. This is combined with the physical geometric bounding box of the mechanical spindle obtained through an object detection algorithm (specifically, the YOLOv8 deep learning object detection network). It should be noted that, to overcome the interference of low-light and dark fields on visual detection, the object detection algorithm is only executed on video frames divided into 'bright frame sequences' by an adaptive threshold, ensuring the accuracy of the physical bounding box boundary acquisition; subsequently, its size is mapped to obtain a preset geometric size value. In step S6, the specific implementation of mapping the size features of the bounding box to the low-dimensional manifold space is as follows: extract the pixel coordinates of the edge key points of the target bounding box in the physical image domain, and use the mapping transformation matrix or forward propagation network corresponding to the dimensionality reduction algorithm to project the pixel coordinates of the edge key points into the low-dimensional manifold space. The Euclidean distance or geodesic distance of each key point after projection in the manifold space is calculated, and this is used as the preset geometric size value when extending the one-dimensional topological skeleton to the three-dimensional manifold space. This operation effectively avoids spatial scale distortion caused by directly using pixel dimensions. A set of cross-sectional points is generated along the directions of the principal and secondary normal vectors according to the preset values. Specifically, using the preset geometric dimensions as distance constraints, two-dimensional discrete spatial coordinates are generated in a ring or polygonal distribution around the discrete nodes within the local normal plane spanned by the principal and secondary normal vectors. These coordinates serve as the cross-sectional contour point set of the tubular or strip-shaped surface at that location. Finally, a spline interpolation algorithm (specifically, non-uniform rational B-spline NURBS interpolation) is used to smoothly wrap the point set, completing the expansion from a one-dimensional topological skeleton to a three-dimensional manifold space. Ultimately, a continuous manifold surface (referring to a three-dimensional geometric model with a manifold structure and a smooth surface without singularities) reflecting the weak motion and deformation state of the principal axis is reconstructed.

[0059] Example 2 The continuous manifold reconstruction method and system of this embodiment are applied to nighttime traffic monitoring scenarios, aiming to solve the problem of loss of the slight movement trajectory of license plates of vehicles with fake license plates or vehicles involved in accidents when streetlights flash frequently or vehicles drive through shadow areas.

[0060] After acquiring nighttime surveillance video, the system needs to overcome the interference from vehicle headlight reflections and streetlight flicker. A sliding time window, twice the length of the flicker period, is established to extract the brightness distribution histogram of local frames. The Otsu's method (maximum inter-class variance method) is then used to find the optimal segmentation point between the two peaks. In this scenario, drastic fluctuations in brightness variance indicate the sudden appearance of a strong light source (such as high beams). After calculating this variance, the system sets a compensation factor and uses a dynamic threshold compensation algorithm to lower the initial threshold, generating an adaptive brightness threshold. This allows frames containing clear license plate features to be assigned to the bright frame sequence, while frames obscured by shadows are assigned to the dark frame sequence.

[0061] To extract trajectories from complex nighttime backgrounds, the system employs a feature point matching algorithm (a more general term, specifically the Scale Invariant Feature Transform (SIFT) algorithm, taking two images as input and outputting the coordinates of matched feature point pairs) to align license plates. Furthermore, it utilizes Local Linear Embedding (LLE, a dimensionality reduction algorithm) to map the bright frame sequence to a low-dimensional manifold space, obtaining the manifold coordinate sequence of vehicle motion. Due to streetlight flickering, the target appears as multiple discontinuous bright trajectory fragments. The system calculates the spatiotemporal correlation constraints between adjacent fragments. In this scenario, the trajectory stitching threshold is set as a dynamic boundary including the vehicle's maximum physical acceleration limit. If the spatiotemporal distance between fragments is within the boundary, they are considered a pair of trajectory breakpoints to be stitched together. In this scenario, the trajectory stitching threshold is strictly limited to the spatiotemporal boundary defined by the vehicle dynamics limit equation. Specifically, the system pre-obtains the legally mandated maximum speed limit for the monitored road segment. And the maximum physical braking / acceleration absolute value of conventional passenger cars (For example, take) When the duration of the dark frame sequence is... At that time, the system calculates the maximum possible spatial displacement boundary value within that time period using the dynamic limit equations. Subsequently, the displacement boundary values ​​of this physical space are... The Jacobian matrix norm is converted into an upper bound for geodesic distance within the manifold space using a low-dimensional manifold mapping algorithm, and this upper bound is directly set as the preset trajectory splicing threshold. This calculation logic completely eliminates the arbitrariness of threshold setting, ensuring that trajectory breakpoint splicing strictly conforms to classical kinematic laws.

[0062] To accurately connect broken vehicle trajectories, the system extracts the first and second local neighborhood point sets at the breakpoints. Instead of simple straight-line fitting, the system constructs a covariance matrix and performs Principal Component Analysis (PCA). This eigenvalue decomposition process effectively filters thermal noise from nighttime image sensors. After extracting the first principal component eigenvector, it is multiplied by the motion direction vector of the preceding bright trajectory segment. After confirming sign consistency, the positive geometric tangent, unaffected by noise, is obtained. Similarly, the reverse geometric tangent at the starting point of the next segment is obtained.

[0063] Entering the trajectory extrapolation stage, to simulate the vehicle's possible curved motion (such as lane changes and turns) in a dark field, the system uses the tangent as the initial derivative and combines it with the vehicle dynamics model to generate forward and reverse extension lines. The shortest spatial distance between the two extension lines in the manifold space is calculated. If the distance is less than the minimum tolerance, the spatial intersection point is directly obtained by solving the equations of the two lines simultaneously; if skew lines are formed due to coordinate drift, the midpoint of the common perpendicular segment is calculated. The system uses a mapping transformation (specifically, the inverse mapping function of LLE, with low-dimensional manifold coordinates as input and high-dimensional image pixel space coordinates as output) to determine this midpoint as a virtual geometric node.

[0064] Since vehicle lane-changing trajectories typically exhibit smooth nonlinear characteristics, the system uses virtual geometric nodes as trajectory connection anchors. Instead of linear interpolation, it introduces the Bézier curve fitting method (a mathematical method that defines a smooth parametric curve by controlling polygons). Inputting the previous endpoint, anchor point, and subsequent starting point, the system outputs smoothly transitioning first and second associated trajectory data, thus achieving continuous connectivity of the vehicle's nighttime monitoring trajectory in a dark environment.

[0065] Finally, the system repeats the above steps to complete the reconstruction of the continuous topological trajectory throughout the entire time period. A thinning algorithm (a mathematical morphology skeleton extraction algorithm that iteratively peels away edge pixels until a single-pixel-wide centerline is retained) is used to extract the topological skeleton of the trajectory. The principal and secondary normal vectors of the skeleton nodes are calculated to establish a local Flyner frame. The geometric dimensions are preset by manually calibrating the license plate size (i.e., the physical geometric contour bounding box). The contour is generated by expanding along the normal vector and wrapped with a spline function (specifically a cubic spline interpolation function) to complete the expansion from a one-dimensional skeleton to a three-dimensional continuous manifold surface, accurately reconstructing the complete dynamic deformation and displacement process of the vehicle at night.

[0066] Example 3 This embodiment describes a specific solution for high-frequency testing scenarios in microelectromechanical systems (MEMS), particularly for the reconstruction of the high-frequency forced vibration trajectory of a MEMS microcantilever beam (or microresonator) under stroboscopic cold light source microscopy. In this environment, the microcantilever beam generates microsecond-level mechanical rebound under piezoelectric actuation. Due to the duty cycle limitation of the stroboscopic light source used for observation, the transient physical characteristics at the displacement extreme point (i.e., the mechanical reversal point) are easily lost within the dark field period.

[0067] First, high frame rate microscopic video is acquired. Due to the strong dark current noise in the background, traditional brightness segmentation methods are prone to failure. The system extracts the brightness distribution within a sliding window and generates a brightness distribution histogram. After obtaining an initial threshold using the Otsu method, the highly sensitive brightness variance is calculated. Combined with a dynamic threshold compensation algorithm, due to the small reflective area of ​​the microcantilever beam and the extremely high skewness of the histogram, the compensation factor... It plays a key role by adaptively lowering the threshold to generate an adaptive brightness threshold, accurately extracting the bright frame sequence when the microcantilever beam is excited to vibrate, and defining the flicker gap of the cold light source as the dark field frame sequence.

[0068] To extract the subtle motion trajectories affected by high-frequency mechanical vibrations, t-SNE (t-distributed random nearest neighbor embedding, an advanced manifold learning algorithm) is used to reduce the dimensionality of the high-dimensional pixel space to a three-dimensional low-dimensional manifold space, obtaining a manifold coordinate sequence. Due to intermittent light sources, the trajectory breaks into high-density bright trajectory fragments. The system then introduces Kalman filtering to perform preliminary pre-smoothing of the fragments and determines the trajectory breakpoint pairs between each fragment based on spatiotemporal correlation constraints.

[0069] For understanding the motion characteristics of a microcantilever beam under forced vibration, tangent extraction at the break point is crucial. The system extracts the first and second local neighborhood point sets of the endpoint and starting point. Singular value decomposition (SVD) is used to perform PCA eigenvalue decomposition, extracting the first principal component eigenvector. By calculating the inner product with the motion direction vector of the preceding bright trajectory segment, the instantaneous momentum direction of the microcantilever beam at the moment before the light source is extinguished is accurately captured and defined as the positive geometric tangent. The same process is applied to the subsequent segment, and the reverse geometric tangent is obtained after direction correction.

[0070] When performing trajectory extrapolation, a microsecond-level linear extrapolation algorithm is used based on the mechanical inertia of the microcantilever beam to extend the trajectory and obtain the forward and reverse extension lines in space. By calculating the cross product and dot product of the two skew vectors, the shortest spatial distance in Euclidean geometry is calculated. After extracting the midpoint of the common perpendicular segment, since nonlinear algorithms such as t-SNE lack explicit mathematical inverse transformation analytical expressions, the system performs a mapping transformation through a trained decoder network (or an inverse transformation function based on radial basis functions) to determine a unique virtual geometric node.

[0071] By using it as the trajectory connection anchor point in the manifold space, spline curves are used to smoothly generate the first associated trajectory data connecting the previous endpoint and the anchor point, as well as the second associated trajectory data connecting the anchor point and the next starting point. This completely eliminates the "abrupt change" in physical characteristics of the micro cantilever beam caused by sampling loss, and achieves high-precision continuous connectivity.

[0072] After traversing and stitching, a topologically continuous skeleton with physical homology is extracted. The principal and secondary normal vectors of the skeleton's three-dimensional space curves are calculated. The physical geometric contour bounding box of the microcantilever beam is fitted with the designed linewidth or optical diffraction limit to obtain the corresponding preset geometric dimensions. The cross-section is expanded along the local Flyner frame, and the boundary conditions are fused using a spline interpolation algorithm to realize the expansion from the one-dimensional topological skeleton to the three-dimensional manifold space. Finally, a continuous manifold surface reflecting the weak forced vibration of the microcantilever beam is rendered in the host computer.

[0073] Example 4 The system and method of this embodiment are deployed in a medical endoscopic imaging device, focusing on the extraction of transient, weak motion features of high-frequency vocal cord vibration or microcapillary pulsation. To avoid tissue thermal damage, endoscopes typically employ a stroboscopic cold light source with an extremely short duty cycle, resulting in a significant alternation of light and dark in the observed image.

[0074] After capturing endoscopic video, the system considers the complex reflective properties of the internal environment (such as mucus reflection). When calculating the brightness distribution within the sliding window, it constructs a brightness distribution histogram and combines it with an improved Otsu's method for segmentation. Based on the drastic changes in brightness variance caused by the extracted mucus highlights, the system's built-in compensation factor is used to correct the initial threshold, generating an adaptive brightness threshold that conforms to the current physiological cavity lighting environment. The moments of vocal cord closure / opening with clear reflections are extracted as bright frame sequences, while the stroboscopic gaps are divided into dark frame sequences.

[0075] To eliminate the influence of slight vibrations in the endoscopic probe, the system utilizes a phase correlation algorithm based on cross-correlation for global translation compensation. Subsequently, the aligned image is mapped to a low-dimensional manifold space using an autoencoder (an autoencoder, a nonlinear dimensionality reduction algorithm based on neural networks), outputting a manifold coordinate sequence of tissue deformation. The faint motion trajectory of the vocal cord edge is extracted from this sequence. Due to the stroboscopic effect, this continuous physiological process is truncated into multiple bright trajectory segments. The system calculates the spatiotemporal distance between them; if the vocal cord vibration frequency is within the trajectory splicing threshold calculated from the physiological limit (e.g., 1000Hz), they are combined into trajectory breakpoint pairs.

[0076] The first and second local neighborhood point sets at the vocal cord edge of the breakpoint are extracted to construct a covariance matrix reflecting the local deformation tensor, and principal component analysis (PCA) is performed. By extracting the eigenvector of the first principal component and calculating its dot product with the motion direction vector of the preceding bright trajectory segment, not only is the tangent equation determined, but the positive geometric tangent of the vocal cord about to rebound is also accurately captured. Since the vocal cord often reaches its amplitude maximum and begins to rebound in the dark field, the reverse geometric tangent (obtained by sign correction through the dot product with the initial motion direction vector of the following bright trajectory segment) often forms a large angle with the positive tangent at this time.

[0077] When extrapolating the trajectory into the dark field region, the system extends the forward and reverse tangents. The shortest spatial distance of the extended lines within the manifold space is calculated to obtain the spatial intersection or the midpoint of the common perpendicular segment. To ensure the physiological significance of the solution, a mapping transformation is performed through the decoder network (inverse transform function) of the autoencoder to generate virtual geometric nodes that can represent the extreme points of the vocal cord's ultimate expansion state. The autoencoder network needs to be trained offline before being put into use. The training process is as follows: collecting the vocal cord motion manifold coordinate sequence under historical normal illumination as the training set; using the mean squared error (MSE) as the reconstruction loss function, updating the network weights through a gradient descent algorithm to minimize the error between the reconstructed coordinates output by the decoder network and the original input manifold coordinates. Specifically, the decoder network adopts a multilayer perceptron (MLP) or deconvolution structure, with the number of neurons in its input layer consistent with the dimension of the low-dimensional manifold space (e.g., 3D), and the output layer reconstructing a high-dimensional physiological feature space. As a preferred engineering implementation, when the decoder network is constructed using a multilayer perceptron (MLP), its network topology is configured to include at least one input layer, three hidden layers, and one output layer. Specifically, the input layer has a receiving dimension of K (e.g., ...). The low-dimensional manifold space coordinates are used; the number of neurons in the first hidden layer is expanded to 2K to 4K, and a linear fully connected layer with LeakyReLU activation function is used to preserve the negative topological gradient; the second and third hidden layers non-linearly increase the dimensionality of the feature space by increasing the number of nodes (e.g., set to 128 and 512 respectively), and a regularization layer with a dropout rate of 0.2 is introduced between these two layers to prevent overfitting; the number of neurons in the output layer is strictly equal to the dimension of the high-dimensional physiological feature space (or the flattened image pixel region), and a linear activation function is used to output the reconstructed physical coordinates or pixel values. This explicit hourglass-shaped inverse expansion architecture, combined with the aforementioned local structure preservation loss, This ensures that the virtual geometric nodes can stably converge to the real physical manifold surface. By locating the spatial intersection points in the low-dimensional latent space and then mapping them through the nonlinear activation function (such as ReLU) of the decoder network, it is ensured that the generated virtual geometric nodes are strictly located on the manifold surface that conforms to the prior laws of physiological tissue deformation. In order to overcome the technical difficulty of the lack of explicit mathematical inverse transformation analytical expression in nonlinear manifold dimensionality reduction algorithms (such as t-SNE, LLE), the decoder network in this embodiment adopts a self-supervised learning mechanism for offline training. The specific training steps include: First, under a continuous stable light source (non-flicker) environment, complete high-dimensional motion video frames of the target object are collected as a label set (Ground Truth); second, the label set is processed by the same dimensionality reduction algorithm to obtain a corresponding set of low-dimensional manifold coordinate sequences; then, a multilayer perceptron (MLP) is constructed as the decoder network, with the above low-dimensional manifold coordinate sequence as the network input and the original high-dimensional video frame pixel matrix or physical coordinates as the expected output. During training, mean squared error (MSE) combined with local preservation loss is used as the comprehensive loss function for backpropagation. The local preservation loss penalizes the deviation between the Euclidean distance between adjacent nodes in the reconstructed high-dimensional data and the distance between the actual high-dimensional data. Specifically, the local preservation loss... The calculation formula is defined as follows: Where N is the total number of samples of low-dimensional manifold coordinates. Represents the coordinates of the i-th low-dimensional manifold Local nearest neighbor set, This is the similarity weight matrix calculated based on the physical similarity between the i-th sample and the j-th sample in the original high-dimensional space. This represents the reconstruction mapping function of the decoder network. By introducing this loss function in conjunction with the mean squared error (MSE) for network gradient updates, the decoder network, when performing feature reconstruction mapping, can not only recover the absolute coordinates of individual virtual geometric nodes, but also enforce that the reconstructed surface nodes conform to the local non-rigid continuous deformation constraints of the target organization in physical space.

[0078] After iterative convergence, the decoder network possesses the ability to reliably reverse-engineer the low-dimensional spatial intersections back to the original physical space, thus ensuring the validity and accuracy of the physical coordinates of the virtual geometric nodes. During the mapping transformation stage, spatial intersections or the midpoints of common perpendicular segments are used as latent features input to the trained decoder network. After the network output layer reconstructs the high-dimensional original data space, the encoder or forward projection matrix corresponding to the low-dimensional manifold mapping algorithm is used to reproject the high-dimensional data back to the low-dimensional manifold space. Through this 'low-dimensional-high-dimensional-low-dimensional' closed-loop mapping, noise deviating from the manifold surface due to linear extrapolation is forcibly filtered out, establishing the final projected coordinates as virtual geometric nodes strictly located on the low-dimensional manifold surface.

[0079] The decoder network employs an hourglass-shaped inverse expansion architecture, configured to include at least one input layer, three hidden layers, and one output layer. Specifically: the input layer receives low-dimensional manifold coordinates of dimension K; the number of neurons in the first hidden layer is expanded to 2K to 4K, and a fully connected layer with a LeakyReLU activation function is used to preserve negative topological gradients; the second and third hidden layers non-linearly upscale the features by increasing the number of nodes (e.g., 128 and 512). During training, mean squared error (MSE) and local structure preservation loss (Local Preservation Loss) are jointly used as a comprehensive loss function for gradient updates. The Local Preservation Loss penalizes the distance deviation between the reconstructed high-dimensional data points and their original physical neighbors, ensuring that the virtual geometric nodes remain strictly located on the manifold surface that conforms to the prior laws of physical deformation after mapping.

[0080] Due to the non-rigid nature of physiological motion, direct straight-line connections lead to distortions in both vision and data analysis. The system uses this virtual node as the core trajectory connection anchor point, inputs the coordinates of adjacent endpoints, and calls an algorithm based on Bézier curve fitting (a type of non-linear interpolation method) to fit first and second associated trajectory data that perfectly match the damping characteristics of physiological soft tissue. This ensures the smoothness of the reconstructed trajectory in terms of derivatives and curvature, achieving continuous connectivity.

[0081] This operation is repeated for all vocal cord movement cycles, splicing them together to form a topologically continuous trajectory representing a complete vocalization cycle. The centerline of the trajectory is extracted using a refinement algorithm as the topological skeleton. The principal and secondary normal vectors for each stage are calculated to construct a local Flyner frame. The system calls a deep learning-based semantic segmentation and object detection algorithm (such as the U-Net model, with medical images as input and a target region mask as output) to automatically delineate the physical geometric bounding box of the vocal cords, converting it into preset geometric dimensions in the manifold space. Physiological sections are generated along the orthogonal direction of the frame. Using a non-uniform tensor spline interpolation algorithm commonly used in medical imaging, the one-dimensional topological skeleton is extended to a three-dimensional manifold space, reconstructing a continuous manifold surface pathological diagnostic model that allows clinicians to observe vocal cord closure abnormalities from multiple angles.

[0082] like Figure 1 The continuous manifold reconstruction method and system for transient weak motion characteristics under dynamic weak light flicker according to an embodiment of the present invention may specifically include: S1, acquire the target motion video under dynamic low light flicker environment, calculate the brightness distribution of each video frame in the target motion video within a preset time sliding window, adaptively generate a brightness threshold, and divide each video frame into a bright frame sequence and a dark frame sequence accordingly.

[0083] A luminance distribution histogram is obtained by extracting a set of local video frames from the target motion video within a time sliding window. An initial luminance threshold is extracted from the luminance distribution histogram using the Otsu's method. The luminance variance of the luminance of each video frame within the time sliding window is calculated, and a compensation factor is determined based on the luminance variance. The initial luminance threshold is dynamically corrected using the compensation factor to obtain an adaptive luminance threshold. The adaptive luminance threshold is then used to divide each video frame into a bright frame sequence and a dark frame sequence. Specifically, the logic for obtaining the compensation factor is as follows: calculate the standard deviation of the frame luminance within the time sliding window. The standard deviation is then compared with the system's preset reference illumination constant C (where C is the mean variance of the pixel brightness mean calculated by continuously acquiring at least 100 frames of black field images with the camera shutter closed, used to characterize the system's static background noise floor). When high-frequency strong light flicker interference is detected in the environment, the skewness S of the brightness distribution histogram is calculated. The empirical offset coefficient... The calculation formula is set as follows ,in This is the weighting adjustment coefficient, with a value ranging from 0.1 to 0.5.

[0084] In one embodiment, for target motion video in a dynamic low-light flicker environment, a local video frame set is first extracted using a time-sliding window, for example, with a window size of 10 frames. These frames are then continuously selected from the video sequence to capture brightness changes. It is important to note that the length of the preset time-sliding window is not arbitrarily set; its length must be greater than or equal to the ratio of the camera's frame rate to the flicker frequency of the ambient flicker source. This constraint ensures that the time-sliding window contains at least one complete bright-dark alternation cycle, thereby guaranteeing that the constructed brightness distribution histogram accurately reflects the extreme brightness characteristics of the flicker environment and preventing misjudgment of dark frames due to an excessively small sampling window.

[0085] For example, in the process of obtaining the brightness distribution histogram, the selected set of video frames is converted into a grayscale image, and then the brightness value distribution of each pixel is statistically analyzed.

[0086] Specifically, the luminance distribution histogram is constructed by calculating the frequency of pixel luminance ranging from 0 to 255 within a frame. For a set of frames, the average luminance is first calculated for each frame, and then the luminance histogram data of all frames are accumulated to form an overall distribution, thus reflecting the illumination fluctuations within a time window. This method ensures that the histogram can capture luminance peaks caused by flicker; for example, in nighttime surveillance videos, intermittent brightness spikes caused by vehicle headlights are highlighted. By introducing a sliding window algorithm, the histogram is dynamically updated, for example, the distribution is recalculated every frame, thus adapting to changes in light and shadow caused by target movement in the video. This facilitates adaptive adjustment of subsequent thresholds.

[0087] In one embodiment, the process of determining an adaptive brightness threshold based on the brightness distribution histogram involves extracting an initial brightness threshold and compensating for it.

[0088] Specifically, the initial brightness threshold can be automatically calculated from the histogram using the maximum inter-class variance method. This method assumes a bimodal brightness distribution and finds the optimal segmentation point by maximizing the inter-class variance. For example, the valley between the dark peak and the bright peak in the histogram can be identified as the initial threshold. Subsequently, compensation is performed to generate an adaptive threshold. Specifically, the brightness variation varies greatly depending on the application scenario (e.g., strong light flicker in surveillance or irregular reflections in industrial inspection). Therefore, the logic for obtaining the compensation factor is as follows: calculate the standard deviation of the frame brightness within the time sliding window and compare this standard deviation with the system's preset reference illumination constant. When the skewness of the brightness distribution histogram indicates the presence of sudden strong light spots in the overall image, an empirical offset coefficient is subtracted from the corresponding initial threshold. Its physical meaning is to prevent misjudging local highlight reflections on the target object's surface as a dark background, thus obtaining the final dynamic adaptive brightness threshold. Specifically, the logic for determining the compensation factor based on the brightness variance is as follows: let the brightness variance of the local video frame set extracted within the time sliding window be... The system's preset reference illumination constant is C. When When high-frequency strong light flicker interference is detected in the environment, the skewness S of the brightness distribution histogram is calculated. The empirical offset coefficient... The calculation formula is set as follows ,in This is a weighting adjustment coefficient, typically ranging from 0.1 to 0.5. The final adaptive brightness threshold... ,in The initial brightness threshold is calculated using the Otsu's method. Through dynamic adjustment of the aforementioned algebraic relationship, the misclassification of strong reflections as bright frames is effectively avoided. This dynamic adjustment based on statistical distribution patterns robustly adapts to dynamic low-light environments without relying on complex nonlinear fitting. The dynamic threshold compensation algorithm quantifies distribution deviation by analyzing the skewness or kurtosis parameters of the histogram. For example, a positive skewness value indicates a high brightness bias, so the threshold is adjusted downwards to avoid over-classifying dark frames. The resulting adaptive threshold effectively distinguishes frame sequences with uneven illumination.

[0089] For example, when using the adaptive brightness threshold to divide each video frame into a bright frame sequence and a dark frame sequence, the average brightness of each frame is calculated and compared with the threshold. If it is higher than the threshold, it is classified into the bright sequence; otherwise, it is classified into the dark sequence, thus completing the video segmentation.

[0090] In one embodiment, this partitioning improves target detection accuracy in low-light flickering video, for example by reducing the interference of light and shadow noise on motion tracking, thereby optimizing the performance of the surveillance system.

[0091] S2, the bright frame sequence is mapped to a low-dimensional manifold space, and the faint motion trajectory of the target is extracted to obtain multiple discontinuous bright trajectory segments truncated in the time dimension by the dark frame sequence. The endpoint of the previous bright trajectory segment and the starting point of the next bright trajectory segment, which are adjacent in time order, are jointly determined as a trajectory breakpoint pair to be spliced.

[0092] A sequence of bright frames is mapped to a low-dimensional manifold space to obtain a manifold coordinate sequence. Weak motion trajectories are extracted based on the manifold coordinate sequence to obtain multiple discontinuous bright trajectory segments truncated by the dark frame sequence. The endpoints of adjacent bright trajectory segments and the starting points of adjacent bright trajectory segments are identified as trajectory breakpoint pairs to be spliced. The spatiotemporal correlation constraints of these trajectory breakpoint pairs are calculated. If the spatiotemporal distance corresponding to the spatiotemporal correlation constraint is less than a preset trajectory splicing threshold, the trajectory breakpoint pair is deemed to meet the splicing condition. The discontinuous bright trajectory segments are connected based on the trajectory breakpoint pairs that meet the splicing condition to obtain a complete bright trajectory sequence, thereby restoring the integrity of the target motion truncated by the dark frame sequence.

[0093] In one embodiment, for target motion video processing in a dynamic low-light flicker environment, the bright frame sequence is first mapped to a low-dimensional manifold space to obtain a manifold coordinate sequence.

[0094] In one embodiment, for target motion video processing in dynamic low-light flickering environments, the bright frame sequence is first mapped to a low-dimensional manifold space to obtain a manifold coordinate sequence. Specifically, high-dimensional pixel data can be processed using manifold learning algorithms or dimensionality reduction algorithms. Before mapping, local regions of interest (ROIs) containing the target object in each bright frame are extracted using inter-frame difference or target detection algorithms, and the pixel matrix within the ROI is flattened into a one-dimensional observation vector. Before flattening into a one-dimensional observation vector, to eliminate the influence of camera shake or background noise on the manifold structure, a phase correlation algorithm or feature point matching algorithm between adjacent bright frames is used to spatially align the ROIs containing the target object to offset translation errors. Subsequently, the aligned ROI pixel matrix is ​​flattened to ensure that the principal component features of the constructed data matrix can truly and uniquely reflect the motion essence of the target object, rather than random background perturbations. The set of one-dimensional observation vectors corresponding to consecutive bright frames is arranged column-wise to construct a data matrix, which is then input as high-dimensional pixel data into the dimensionality reduction algorithm. Taking Principal Component Analysis (PCA) as an example, this embodiment calculates the covariance matrix of the data matrix and performs principal component eigenvalue decomposition. It then extracts the eigenvectors corresponding to the top K largest eigenvalues ​​whose cumulative variance contribution rate is greater than a preset percentage (e.g., between 85% and 95%), thereby constructing a K-dimensional (e.g., two-dimensional or three-dimensional) low-dimensional manifold space. This processing logic not only preserves the main physical structural features of the target's subtle motion but also naturally filters out high-frequency background noise in low-light environments. After projecting the high-dimensional pixel data onto this low-dimensional manifold space, a sequence of low-dimensional coordinate points for each bright frame can be obtained. The movement trajectories of these coordinate points in the manifold space intuitively and smoothly reflect the transient subtle motion changes of the target.

[0095] For example, continue to extract weak motion trajectories based on the manifold coordinate sequence.

[0096] Specifically, optical flow algorithms can be used to calculate the displacement vectors between adjacent coordinate points, and then these vectors can be connected to form a trajectory line. Due to the interference of dark field frame sequences, the trajectory is truncated into multiple discontinuous bright trajectory segments. For example, in surveillance videos, the trajectory of a target such as a pedestrian may appear broken when the lights are flashing.

[0097] In one embodiment, the endpoint of the preceding trajectory segment and the starting point of the following trajectory segment are determined as a pair of trajectory breakpoints to be spliced ​​based on the bright trajectory segment.

[0098] For example, in a business scenario, for vehicle trajectories in nighttime traffic monitoring videos, breakpoint pairs caused by street light flicker are identified, and their spatiotemporal correlation constraints are calculated. The specific process includes measuring the interval and spatial Euclidean distance of the breakpoint pairs on the time axis, and evaluating whether it is reasonable in combination with the velocity estimation model. If the spatiotemporal distance is less than a preset threshold, such as 5 frame intervals and 10 pixel distance, it is determined that the splicing conditions are met, thereby connecting the segments into a complete bright trajectory sequence.

[0099] Specifically, the spatiotemporal distance The calculation formula is: in, Let Euclidean distance be the distance between the starting point of the subsequent bright trajectory segment and the ending point of the previous bright trajectory segment in the low-dimensional manifold space. The time frame interval corresponding to the two points. and These are the normalized weighting coefficients. The preset trajectory stitching threshold is determined based on the upper limit of the target object's maximum physical motion speed.

[0100] It should be further explained that the normalized weighting coefficients ω1 and ω2 in the above formula are not arbitrarily set, but are dynamically balanced based on the dimensional differences between spatial displacement and time span in the actual physical scenario. Specifically, ω1 is the reciprocal of the variance of the spatial Euclidean distance, and ω2 is the reciprocal of the variance of the time frame interval, thereby unifying the two penalty values ​​to the same dimensionless order of magnitude. Furthermore, the preset trajectory stitching threshold is obtained through a priori dynamic limits of the target object. For example, in a traffic monitoring scenario, if the maximum physical acceleration of the vehicle is known... The time span of the dark field is The theoretical boundary of the maximum possible displacement of the vehicle in the dark field interval is: By increasing the theoretical boundary value of the spatiotemporal constraints by 10% to 15% with a system tolerance margin, the system can accurately calculate and determine the trajectory stitching threshold. This mechanism ensures the objectivity and repeatability of the threshold setting, avoiding the uncertainty caused by subjective parameter tuning.

[0101] For example, after the above connection, the integrity of the target motion truncated by the dark field frame sequence can be restored. In security systems, this helps to accurately track the suspect's path and improve identification efficiency.

[0102] S3, for each pair of trajectory breakpoints to be spliced, extract the forward geometric tangent at the end point of the previous bright trajectory segment and the reverse geometric tangent at the starting point of the next bright trajectory segment in the low-dimensional manifold space.

[0103] Obtain the pairs of breakpoints of the trajectories to be spliced ​​in the low-dimensional manifold space, and extract the first local neighborhood point set of the preceding bright trajectory segment and the second local neighborhood point set of the following bright trajectory segment from the breakpoint pairs; construct a covariance matrix based on the first local neighborhood point set and the second local neighborhood point set, and perform principal component analysis to obtain the first principal component eigenvector and the second principal component eigenvector; calculate the inner product of the first principal component eigenvector and the motion direction of the preceding bright trajectory segment, and adjust the direction of the first principal component eigenvector according to the sign of the inner product to make it consistent with the motion direction, thereby determining it as the positive geometric tangent at the endpoint; calculate the inner product of the second principal component eigenvector and the starting motion direction of the following bright trajectory segment. If the inner product is greater than zero, invert the second principal component eigenvector to determine it as the reverse geometric tangent at the starting point; if the inner product is less than zero, directly determine the second principal component eigenvector as the reverse geometric tangent.

[0104] In one possible implementation, the breakpoint pairs of the trajectory to be stitched are first obtained in the low-dimensional manifold space. For example, in a video sequence processing system, the high-dimensional image data is reduced to three-dimensional space by a manifold learning algorithm, thereby extracting these breakpoint pairs to support subsequent trajectory stitching operations.

[0105] In one possible implementation, a first local neighborhood point set of the preceding bright trajectory segment and a second local neighborhood point set of the following bright trajectory segment are extracted from the breakpoint pair. The first local neighborhood point set can be understood as several nearest neighbor points surrounding the endpoint of the preceding trajectory segment. For example, in practical particle tracking applications, assuming the endpoint coordinates of the preceding trajectory segment are (x1, y1, z1), all points within a spherical region centered on this point with a preset radius (e.g., 0.5 units) are selected to form a point set. These points reflect the local geometric structure. Similarly, the second local neighborhood point set is selected for the starting point of the following trajectory segment, thus providing a data foundation for subsequent analysis. Next, a covariance matrix is ​​constructed based on the first and second local neighborhood point sets, and principal component analysis is performed to obtain the first and second principal component eigenvectors.

[0106] Specifically, the covariance matrix is ​​constructed by calculating the mean and deviation of each point in the point set. For example, for the first point set, the average coordinates of all points are first taken as the center, and then the deviation vector of each point relative to the center is calculated. The matrix is ​​obtained by summing the outer products of these deviation vectors. Principal component analysis involves performing eigenvalue decomposition on this matrix and extracting the vector with the largest eigenvalue as the first principal component eigenvector. This represents the main direction of variation of the point set, thereby capturing the geometric characteristics of the local trajectory. The second point set is processed in the same way to obtain the second principal component eigenvector. If the first principal component feature vector is consistent with the motion direction of the previous bright trajectory segment, it is determined as the positive geometric tangent at the endpoint. For example, in a moving object tracking scenario, if the motion direction of the previous trajectory segment is a vector (vx, vy, vz) from point A to point B, and the dot product of the first principal component feature vector with it is greater than zero, it indicates that the direction is consistent, and it is considered as a positive tangent to evaluate the feasibility of splicing. For the next bright trajectory segment, a preset number of sampling points after the starting point are extracted to construct a reference direction vector reflecting the initial motion trend of the segment. Subsequently, the inner product of the second principal component feature vector and the reference direction vector is calculated. If the inner product is greater than zero, it indicates that the feature vector points to the direction of the target motion. At this time, the second principal component feature vector is inverted (i.e., multiplied by a negative sign) to make it point to the dark field interval where the trajectory breaks, and it is determined as the reverse geometric tangent at the starting point. If the inner product is less than zero, it indicates that it has already pointed to the dark field interval, and the second principal component feature vector is retained as the reverse geometric tangent. Through the above sign correction of the tangent, trajectory segments interrupted by flickering dark fields or noise can be effectively connected.

[0107] In one possible implementation, the geometric constraints of the trajectory are verified through the above process, thereby improving the accuracy of the stitching.

[0108] S4, extend the forward geometric tangent and the reverse geometric tangent to the time interval of the corresponding dark field frame sequence, calculate the spatial intersection point or the midpoint of the common perpendicular segment of the two extended lines, and determine the obtained spatial intersection point or the midpoint of the common perpendicular segment as a virtual geometric node.

[0109] The reason for using the midpoint of the common perpendicular segment instead of directly connecting the breakpoint is that the extreme points of transient motion (such as the highest point of rebound) often physically manifest as drastic changes in velocity direction. The intersection of the extensions of the positive and negative tangents represents, in a probabilistic statistical sense, the physical spatial limit position that the target is most likely to reach during dark periods. By introducing this midpoint as an anchor point, this 'reversal' characteristic can be forcibly constrained in the reconstructed trajectory, avoiding the loss of physical information caused by overfitting in traditional smoothing algorithms.

[0110] Obtain the forward and reverse geometric tangents and their corresponding time intervals; extrapolate the trajectories of the forward and reverse geometric tangents to obtain forward and reverse extension lines; calculate the shortest spatial distance value based on the forward and reverse extension lines; if the shortest spatial distance value is less than a preset threshold, calculate the spatial intersection point; otherwise, calculate the midpoint of the common perpendicular segment; perform mapping processing on the spatial intersection point or the midpoint of the common perpendicular segment to determine it as a virtual geometric node.

[0111] In one possible implementation, the forward and reverse geometric tangents and their corresponding time intervals are first obtained. For example, when processing satellite trajectory data in a low-dimensional manifold space, it is assumed that the forward tangent of the previous trajectory segment is vector A pointing in the future direction, and the reverse tangent is vector B pointing in the past direction. The time intervals, such as t1 to t2, are associated with this, thus providing a basis for subsequent extrapolation.

[0112] For example, extrapolating the trajectories of the positive and negative geometric tangents yields the positive and negative extension lines.

[0113] Specifically, in trajectory stitching, the extrapolation process can be implemented using a linear extrapolation algorithm: for a forward tangent, a distance is extended from the endpoint of the previous trajectory along the tangent direction to form a forward extension line; similarly, a reverse extension line is extended from the starting point of the next trajectory along the reverse tangent. In this process, the specific length of the extension distance is determined by both the time interval length of the dark-field frame sequence and the instantaneous velocity of the target before the breakpoint. That is, the extension distance equals the product of the instantaneous velocity and the dark-field time interval, thus mapping the missing time dimension to the extrapolated tangent length in the spatial dimension. This extrapolation is similar to extending a straight line segment in a two-dimensional plane to simulate a potential continuous path of the trajectory, thereby establishing a virtual connection between breakpoints.

[0114] It should be noted that the extrapolation length can be dynamically adjusted according to the time interval. For example, if the time interval is short, the extension distance can be set to a fixed ratio, such as 0.5 times the interval length, to avoid over-extension leading to calculation errors. In practical satellite trajectory applications, this step helps bridge breaks caused by signal loss, ensuring the overall continuity of the trajectory.

[0115] In one possible implementation, the shortest spatial distance is calculated based on the forward and reverse extension lines. In low-dimensional manifold spaces (e.g., three-dimensional space), due to noise and systematic errors, the forward and reverse extension lines often appear as skew lines and cannot intersect perfectly. In this case, an Euclidean spatial distance calculation algorithm is used to calculate the minimum spatial distance between these two skew extension lines, and the spatial intersection point or the midpoint of the common perpendicular segment is found accordingly. Considering the trajectory drift that may be caused by environmental noise, the preset intersection tolerance threshold is not an absolutely constant, but can be dynamically determined based on the system disturbance variance of the local neighborhood of the endpoint of the previous bright trajectory segment. If the shortest spatial distance is less than the intersection tolerance threshold, the two extension lines are considered to intersect approximately in the physical manifold sense, and the spatial intersection point is found by solving a system of spatial geometric equations; if the shortest spatial distance is greater than or equal to the preset threshold, the common perpendicular segment between the two skew extension lines is calculated, and the geometric midpoint of the common perpendicular segment is extracted. Subsequently, the extracted spatial intersections or midpoints of common perpendicular segments are mapped and determined as virtual geometric nodes for bridging trajectory cliffs.

[0116] In one possible implementation, a mapping process is performed on the spatial intersection points or the midpoints of the common perpendicular segments to determine them as virtual geometric nodes. For example, the point is mapped back to the original high-dimensional space, and its coordinates are recovered through an inverse transformation function, thereby generating virtual nodes that can be used for trajectory filling. Through the above process, precise bridging of trajectory breakpoints is achieved, thereby improving the accuracy of overall trajectory reconstruction.

[0117] S5, using the virtual geometric node as the trajectory connection anchor point, construct connecting line segments in the low-dimensional manifold space, connect the endpoint of the previous bright trajectory segment to the virtual geometric node, and connect the virtual geometric node to the starting point of the next bright trajectory segment, thereby achieving continuous connectivity of the current trajectory breakpoint pair.

[0118] Obtain the endpoint coordinates of the previous bright trajectory segment and the starting coordinates of the next bright trajectory segment mapped to the low-dimensional manifold space. Determine the position coordinates of virtual geometric nodes based on the endpoint coordinates and the starting coordinates. Construct a first connecting line segment based on the position coordinates of the virtual geometric nodes to obtain first associated trajectory data. Construct a second connecting line segment based on the first associated trajectory data to obtain second associated trajectory data. Combine the first associated trajectory data and the second associated trajectory data to stitch the previous bright trajectory segment and the next bright trajectory segment together to determine complete continuous trajectory data, thus achieving continuous connectivity of the current trajectory breakpoint pair.

[0119] In one embodiment, for trajectory processing in a low-dimensional manifold space, the endpoint coordinates of the previous bright trajectory segment and the starting coordinates of the next bright trajectory segment are first obtained. These coordinates are obtained through a previous mapping algorithm. For example, in video surveillance services, bright trajectory segments correspond to a continuous sequence of light spots of a moving target.

[0120] In one embodiment, the position coordinates of the virtual geometric node are determined based on the endpoint coordinates and the starting point coordinates.

[0121] Specifically, the process involves calculating the geometric midpoint or weighted average point between two coordinates to bridge trajectory breakpoints.

[0122] For example, in a traffic monitoring system, the endpoint coordinates of the previous trajectory segment are (x1, y1, z1), and the starting coordinates of the next trajectory segment are (x2, y2, z2). The midpoint coordinates can be calculated using formulas ((x1+x2) / 2, (y1+y2) / 2, (z1+z2) / 2). However, instead of direct numerical calculation, the influence of spatial curvature is considered, and low-dimensional manifold constraints are introduced to ensure that the node positions conform to the characteristics of the manifold surface. Thus, in practical applications such as autonomous driving path planning, virtual nodes can effectively fill the missing parts of the trajectory caused by occlusion. Further extending to multi-target tracking scenarios, node positions can be iteratively optimized and adjusted to minimize the overall trajectory curvature deviation, achieving a smoother connection. Furthermore, in extended solutions, if the trajectory segment involves a time dimension, temporal weights can be incorporated to make the node positions biased towards the movement trend of the previous trajectory, enriching the adaptability of the solution.

[0123] In one embodiment, a first connecting line segment is constructed based on the position coordinates of the virtual geometric node to obtain first associated trajectory data, and a second connecting line segment is constructed based on the first associated trajectory data to obtain second associated trajectory data.

[0124] Specifically, the first connecting line segment extends from the end point of the previous trajectory to the virtual node, generating associated data containing speed and direction; the second connecting line segment extends from the virtual node to the starting point of the next trajectory, similarly generating associated data.

[0125] For example, in security video analytics, the first associated trajectory data can be filled with the point set between nodes using a linear interpolation algorithm, while the second associated trajectory data is smoothed using a Bézier curve to ensure trajectory continuity.

[0126] In one embodiment, the previous bright trajectory segment and the next bright trajectory segment are spliced ​​together based on the first associated trajectory data and the second associated trajectory data to determine complete continuous trajectory data, thereby achieving continuity of the current trajectory breakpoint pair and improving the accuracy of trajectory tracking in business operations.

[0127] S6, traverse all truncated bright trajectory segments in the target motion video, repeat steps S3 to S5, stitch all bright trajectory segments into a topologically continuous motion trajectory, and use the stitched topologically continuous motion trajectory as a topological skeleton to reconstruct the continuous manifold surface of the target's weak motion.

[0128] Obtain the truncated endpoints and motion direction vectors of the bright trajectory segments; if the angle between the motion direction vectors is less than a preset threshold, determine that the truncated endpoints have connectivity and generate a splicing path; connect the bright trajectory segments according to the splicing path to obtain a topologically continuous trajectory, and extract the topological skeleton of the topologically continuous trajectory; calculate the principal normal vector and the secondary normal vector of each node on the topological skeleton, and expand the space along the directions of the principal normal vector and the secondary normal vector, combined with the preset value of the target's geometric dimensions, thereby reconstructing the continuous manifold surface of the target's weak motion.

[0129] In one embodiment, the truncation endpoints and motion direction vectors of the bright trajectory segments are first obtained.

[0130] For example, when processing vehicle movement trajectories in a video surveillance system, image processing algorithms are used to extract the coordinates of the end points of the trajectory and the corresponding direction vectors from consecutive frames. These vectors represent the extension trend of the trajectory in space, thus providing basic data for subsequent judgments.

[0131] For example, in the case of human motion capture, the angle between the motion direction vectors is calculated after obtaining the truncated endpoints.

[0132] Specifically, the cosine value is obtained by performing a dot product operation between the end direction vector of the previous trajectory segment and the start direction vector of the next trajectory segment. The angle is then obtained by using the inverse cosine function. If the angle is less than a preset threshold, such as 30 degrees, it is determined that these endpoints have connectivity. At this point, a splicing path is generated. For example, a Bézier curve fitting algorithm is used to connect the endpoints. The path generation takes into account the distance and direction consistency between the endpoints to ensure a smooth transition of the path and avoid abrupt changes.

[0133] For example, a topologically continuous trajectory can be obtained by connecting bright trajectory segments based on the generated splicing path.

[0134] Specifically, in robot navigation, multiple disconnected trajectory segments are merged into a single continuous trajectory through path insertion, and then its topological skeleton is extracted. For example, a thinning algorithm is used to gradually remove redundant pixels from the trajectory, retaining only the centerline structure as the skeleton. This helps to simplify the trajectory representation and highlight the topological relationships.

[0135] For example, a continuous manifold surface with weak target motion can be reconstructed using the surface normal vectors of the topological skeleton.

[0136] Specifically, in medical image analysis, the normal vectors of each point on the skeleton are calculated. These vectors are perpendicular to the surface and indicate the direction of local curvature. The normal vectors are then extended into a continuous surface using spline interpolation algorithms, thereby restoring the complete shape of subtle movements such as vascular pulsation.

[0137] In one embodiment, this reconstruction process improves the accuracy of motion analysis.

[0138] See Figure 2 The present invention also provides a continuous manifold reconstruction system for transient weak motion characteristics under dynamic weak light flicker, mainly comprising: The brightness adaptive segmentation module is used to acquire target motion video under dynamic low light flicker environment, calculate the brightness distribution of each video frame in the target motion video within a preset time sliding window, adaptively generate a brightness threshold, and divide each video frame into a bright frame sequence and a dark frame sequence accordingly. The low-dimensional manifold mapping and trajectory breakpoint extraction module is used to map the bright frame sequence to a low-dimensional manifold space and extract the weak motion trajectory of the target to obtain multiple discontinuous bright trajectory segments truncated in the time dimension by the dark frame sequence. The endpoint of the previous bright trajectory segment and the starting point of the next bright trajectory segment, which are adjacent in time order, are jointly determined as a trajectory breakpoint pair to be spliced. The geometric tangent extraction module is used to extract, in the low-dimensional manifold space, the forward geometric tangent at the end point of the previous bright trajectory segment and the reverse geometric tangent at the start point of the next bright trajectory segment for each pair of trajectory breakpoints to be spliced. The virtual geometric node determination module is used to extend the forward geometric tangent and the reverse geometric tangent to the time interval of the corresponding dark field frame sequence, calculate the spatial intersection point or the midpoint of the common perpendicular segment of the two extended lines, and determine the obtained spatial intersection point or the midpoint of the common perpendicular segment as a virtual geometric node. The trajectory connection anchor point construction module is used to construct connecting line segments in the low-dimensional manifold space, using the virtual geometric node as the trajectory connection anchor point, connecting the endpoint of the previous bright trajectory segment to the virtual geometric node, and connecting the virtual geometric node to the starting point of the next bright trajectory segment, so as to realize the continuous connection of the current trajectory breakpoint pair. The topology continuous stitching and manifold surface reconstruction module is used to traverse all truncated bright trajectory segments in the target motion video, repeat steps S3 to S5, stitch all bright trajectory segments into a topology continuous motion trajectory, and use the stitched topology continuous motion trajectory as a topology skeleton to reconstruct the continuous manifold surface of the target's weak motion.

[0139] Based on the embodiments of the present invention described above, and through the above description, those skilled in the art can make various changes and modifications without departing from the technical concept of the present invention. The technical scope of the present invention is not limited to the contents of the specification, but must be determined according to the scope of the claims.

Claims

1. A method for reconstructing continuous manifolds of transient weak motion characteristics under dynamic low-light flicker, characterized in that, The method includes: S1, acquire the target motion video under dynamic low light flicker environment, calculate the brightness distribution of each video frame in the target motion video within a preset time sliding window, adaptively generate a brightness threshold, and divide each video frame into a bright frame sequence and a dark frame sequence accordingly. S2, extract the local region of interest (ROI) of the target in each bright frame of the bright frame sequence, flatten the pixel matrix of the ROI corresponding to each bright frame into a one-dimensional observation vector, and use a dimensionality reduction algorithm to map multiple one-dimensional observation vectors to a low-dimensional manifold space. Extract the weak motion trajectory of the target in the low-dimensional manifold space to obtain multiple discontinuous bright trajectory segments truncated in the time dimension by the dark frame sequence. Determine the endpoint of the previous bright trajectory segment and the starting point of the next bright trajectory segment that are adjacent in time as a pair of candidate trajectory breakpoints. Extract the candidate trajectory breakpoint pairs that satisfy the spatiotemporal correlation constraints as the trajectory breakpoint pairs to be spliced. S3, for each pair of trajectory breakpoints to be spliced, extract the forward geometric tangent at the end point of the previous bright trajectory segment and the reverse geometric tangent at the starting point of the next bright trajectory segment in the low-dimensional manifold space. S4, extend the forward geometric tangent and the reverse geometric tangent to the time interval of the corresponding dark field frame sequence, calculate the spatial intersection point or the midpoint of the common perpendicular segment of the two extended lines, and determine the obtained spatial intersection point or the midpoint of the common perpendicular segment as a virtual geometric node. S5, using the virtual geometric node as a trajectory connection anchor point, construct connecting line segments in the low-dimensional manifold space, connect the endpoint of the previous bright trajectory segment to the virtual geometric node, and connect the virtual geometric node to the starting point of the next bright trajectory segment, thereby achieving continuous connectivity of the current trajectory breakpoint pair.

2. The method according to claim 1, characterized in that, Also includes: S6, traverse all truncated bright trajectory segments in the target motion video, repeat steps S3 to S5, stitch all bright trajectory segments into a topologically continuous motion trajectory, use the stitched topologically continuous motion trajectory as a topological skeleton, and reconstruct a continuous manifold surface corresponding to the weak motion trajectory.

3. The method according to claim 1, characterized in that, S1 includes: Extract the brightness distribution histogram of image pixels from a set of local video frames within a preset time sliding window of the target motion video; The initial brightness threshold is extracted from the brightness distribution histogram using the maximum inter-class variance method. Calculate the luminance variance of the luminance of each video frame within the time sliding window, determine a compensation factor based on the luminance variance, and use the compensation factor to dynamically correct the initial luminance threshold to obtain the adaptive luminance threshold; The adaptive brightness threshold is used to divide each video frame into a bright frame sequence and a dark frame sequence.

4. The method according to claim 1, characterized in that, The phrase "extracting candidate trajectory breakpoint pairs that satisfy spatiotemporal correlation constraints as trajectory breakpoint pairs to be spliced" includes: Calculate the spatiotemporal distance between the candidate trajectory breakpoint pairs; If the spatiotemporal distance is less than the preset trajectory splicing threshold, then the candidate trajectory breakpoint pair is determined to satisfy the spatiotemporal correlation constraint and is used as the trajectory breakpoint pair to be spliced.

5. The method according to claim 1, characterized in that, S3 includes: Obtain the pairs of breakpoints of the trajectories to be spliced ​​in the low-dimensional manifold space, and extract the first local neighborhood point set of the previous bright trajectory segment and the second local neighborhood point set of the next bright trajectory segment from the pairs of breakpoints; Covariance matrices are constructed for the first local neighborhood point set and the second local neighborhood point set respectively, and principal component analysis is performed to obtain the first principal component eigenvector and the second principal component eigenvector. Extract a predetermined number of trajectory nodes before the endpoint of the previous bright trajectory segment, and fit them to generate the instantaneous motion direction vector of the previous bright trajectory segment; extract a predetermined number of trajectory nodes after the starting point of the next bright trajectory segment, and fit them to generate the instantaneous motion direction vector of the next bright trajectory segment. Calculate the inner product of the first principal component feature vector and the instantaneous motion direction vector of the previous bright trajectory segment. If the inner product is greater than or equal to zero, the first principal component feature vector is directly determined as the positive geometric tangent at the endpoint. If the inner product is less than zero, the first principal component feature vector is inverted so that its direction is consistent with the instantaneous motion direction vector, and thus determined as the positive geometric tangent at the endpoint. Calculate the inner product of the second principal component eigenvector and the instantaneous motion direction vector of the subsequent bright trajectory segment. If the inner product is greater than zero, invert the second principal component eigenvector to determine the reverse geometric tangent at the starting point. If the inner product is less than or equal to zero, directly determine the second principal component eigenvector as the reverse geometric tangent at the starting point.

6. The method according to claim 1, characterized in that, S4 includes: Obtain the forward and reverse geometric tangents and the corresponding time intervals of the dark field frame sequence, and extrapolate the trajectory of the forward and reverse geometric tangents to obtain the forward and reverse extension lines; Calculate the shortest spatial distance value based on the forward and reverse extension lines; If the shortest spatial distance is less than a preset threshold, the spatial intersection point is calculated; otherwise, the midpoint of the common perpendicular segment is calculated. The spatial intersection point or the midpoint of the common perpendicular segment can be directly determined as a virtual geometric node in the low-dimensional manifold space; or, a feature reconstruction mapping can be performed on the spatial intersection point or the midpoint of the common perpendicular segment through a pre-trained decoder network to correct and determine it as the final virtual geometric node.

7. The method according to claim 1, characterized in that, S5 includes: Obtain the endpoint coordinates of the previous bright trajectory segment and the starting coordinates of the next bright trajectory segment mapped to the low-dimensional manifold space, and determine the position coordinates of the virtual geometric node based on the endpoint coordinates and the starting coordinates; Spatial interpolation or curve fitting is performed between the endpoint of the previous bright trajectory segment and the virtual geometric node to obtain the first associated trajectory data; spatial interpolation or curve fitting is performed between the virtual geometric node and the starting point of the next bright trajectory segment to obtain the second associated trajectory data. Based on the first associated trajectory data and the second associated trajectory data, the connecting line segment is constructed, and the previous bright trajectory segment is spliced ​​with the next bright trajectory segment to achieve continuous connection of the current trajectory breakpoint pair.

8. The method according to claim 2, characterized in that, S6 includes: Extract the centerline of the continuous motion trajectory of the spliced ​​topology and use it as the topological skeleton; Extract multiple discrete nodes on the topological skeleton, calculate the principal normal vector and the secondary normal vector of each discrete node, and construct a local Flyner frame based on the topological skeleton; Obtain the physical geometric contour bounding box of the target determined by the target detection algorithm, and map the size features of the physical geometric contour bounding box to the low-dimensional manifold space to obtain the preset value of the geometric size; Using each discrete node as a reference point, a cross-sectional profile point set is generated along the corresponding principal normal vector and sub-normal vector directions according to the preset geometric dimensions. Spline interpolation algorithm is used to smoothly connect adjacent cross-sectional contour point sets, completing the extension of the one-dimensional topological skeleton to the three-dimensional manifold space, and reconstructing a continuous manifold surface that reflects the weak motion deformation state of the target.

9. A continuous manifold reconstruction system for transient weak motion characteristics under dynamic low-light flicker, characterized in that, The system includes: The brightness adaptive segmentation module is used to acquire target motion video under dynamic low light flicker environment, calculate the brightness distribution of each video frame in the target motion video within a preset time sliding window, adaptively generate a brightness threshold, and divide each video frame into a bright frame sequence and a dark frame sequence accordingly. The low-dimensional manifold mapping and trajectory breakpoint extraction module is used to map the bright frame sequence to a low-dimensional manifold space and extract the weak motion trajectory of the target to obtain multiple discontinuous bright trajectory segments truncated in the time dimension by the dark frame sequence. The endpoint of the previous bright trajectory segment and the starting point of the next bright trajectory segment, which are adjacent in time order, are jointly determined as a trajectory breakpoint pair to be spliced. The geometric tangent extraction module is used to extract, in the low-dimensional manifold space, the forward geometric tangent at the end point of the previous bright trajectory segment and the reverse geometric tangent at the start point of the next bright trajectory segment for each pair of trajectory breakpoints to be spliced. The virtual geometric node determination module is used to extend the forward geometric tangent and the reverse geometric tangent to the time interval of the corresponding dark field frame sequence, calculate the spatial intersection point or the midpoint of the common perpendicular segment of the two extended lines, and determine the obtained spatial intersection point or the midpoint of the common perpendicular segment as a virtual geometric node. The trajectory connection anchor point construction module is used to construct connecting line segments in the low-dimensional manifold space, using the virtual geometric node as the trajectory connection anchor point, connecting the endpoint of the previous bright trajectory segment to the virtual geometric node, and connecting the virtual geometric node to the starting point of the next bright trajectory segment, so as to realize the continuous connection of the current trajectory breakpoint pair. The topology continuous stitching and manifold surface reconstruction module is used to traverse all truncated bright trajectory segments in the target motion video, repeat steps S3 to S5, stitch all bright trajectory segments into a topology continuous motion trajectory, and use the stitched topology continuous motion trajectory as a topology skeleton to reconstruct the continuous manifold surface of the target's weak motion.

Images