Nanoparticle tracking and predicting method

Abstract

The nanoparticle tracking and predicting method is used for tracking and predicting nanoparticles, and at least comprises particle detection, trajectory tracking, a physical prediction model, visualization and data recording. The method comprises at least the following steps: processing a video frame by means of the particle detection, identifying valid nanoparticles and acquiring particle information thereof, generating a unique identifier (ID) based on initial position information of the valid nanoparticles; according to any same ID and different moments, needed particle motion parameters are obtained through trajectory tracking real-time calculation; when the effective nanoparticles at different moments of the same ID are continuously captured, continuing to capture; when capturing the interruption, predicting the particle information through the physical prediction model to compensate and track the interruption, and returning; the effective nano particles are continuously tracked through trajectory tracking, and the particle information and the particle motion parameters are displayed in the visualization in real time; and storing the particle motion parameters in real time by the data record. According to the method, the problems of track breakage, physical rule modeling deficiency and insufficient multi-module collaboration caused by particle loss in a complex environment in the prior art are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, and in particular to a method for tracking and predicting nanoparticles. BACKGROUND

[0002] With the expanding application of carbon nanotubes, graphene and other nanoparticles in the fields of biomedicine, energy catalysis and nanodevices, the accurate characterization of their diffusion rate, aggregation behavior and other motion characteristics has become a core prerequisite for revealing material performance and optimizing application scenarios. Nanoparticle tracking technology based on high-speed microscopic video has become the mainstream method for analyzing particle dynamic behavior due to its non-invasive observation advantage. However, this technology exposes multi-dimensional technical bottlenecks in complex real-world scenarios, which urgently need to be systematically broken through.

[0003] Nanoparticles often fail to be tracked due to mutual occlusion, out-of-focus blur or temporary disappearance in Brownian motion, fluid disturbance or high-concentration aggregation scenarios. Traditional tracking algorithms rely on linear motion models for extrapolation and prediction, which are difficult to adapt to dynamic changes in non-linear motion trajectories, frequently causing trajectory breaks and directly leading to the interruption of subsequent behavior analysis chains. This tracking strategy based on simple mathematical models shows significant robustness deficiency when the motion state of particles changes abruptly.

[0004] Existing prediction models generally ignore the physical constraints of particle motion, such as key parameters such as viscous resistance and particle inertia in fluid dynamics. When particles are lost for a short time, purely relying on linear or polynomial interpolation for mathematical fitting will generate pseudo-trajectories that violate the real physical process, leading to exponential growth of cumulative errors in key parameters such as position and velocity. This disconnection between physical mechanisms and mathematical models makes the motion trajectory prediction results lose physical interpretability, seriously affecting the scientific nature of material performance analysis.

[0005] Existing technical tools generally lack real-time visualization comparison modules for predicted trajectories and real trajectories, making it difficult for researchers to dynamically verify the reliability of algorithms during data acquisition. At the same time, the asynchronous storage mode of motion parameters (velocity, error) and original video data makes it difficult to match temporal and spatial information during subsequent causal analysis, seriously restricting the correlation study of particle motion behavior and environmental variables.

[0006] Although deep learning technology has been initially applied to nanoparticle tracking (such as CNN particle detection, LSTM trajectory prediction), there are still two major limitations: 1. The disconnection between physical mechanism and data-driven: the network training process only relies on observation data-driven, without embedding kinematic constraint conditions (such as reasonable displacement threshold, velocity continuity constraint), resulting in abnormal trajectories that violate the laws of inertia, energy conservation and other physical laws; 2. Insufficient synergy of modular design: the detection, tracking and prediction modules use separate optimization strategies, lack cross-module information feedback mechanism, and the front-end detection error will be amplified by the back-end tracking and prediction module, forming a systematic error accumulation effect. The above problems lead to the fact that the existing improved technology still has a high trajectory breaking rate in a high noise environment, and the physical law compliance and multi-module synergy performance need to be improved.

[0007] Currently, the analysis of the dynamic behavior of nanoparticles requires trajectory integrity, physical modeling accuracy and multi-dimensional information fusion, and the existing technology has problems such as trajectory breaking in complex scenes, lack of physical law modeling and insufficient module synergy, which has become a key bottleneck restricting the basic research and engineering application of nanomaterials.

[0008] Therefore, it is necessary and urgent to develop a robust nanoparticle motion trajectory capturing algorithm that synchronously optimizes detection accuracy and physical law compliance and dynamically fuses spatiotemporal feature information. SUMMARY

[0009] The present application provides a method for tracking and predicting nanoparticles to solve the problems of particle loss leading to trajectory breaking, lack of physical law modeling and insufficient multi-module synergy in complex environments in the prior art.

[0010] To solve the above technical problems, the present application is implemented as follows:

[0011] The present application provides a method for tracking and predicting nanoparticles, which is used for tracking and predicting nanoparticles and at least includes particle detection, trajectory tracking, physical prediction model, visualization and data recording; the method at least includes the following steps:

[0012] S1: Process the video frame through the particle detection, identify the effective nanoparticles and obtain their particle information, and generate a unique identifier (ID) based on the initial position information of the effective nanoparticles;

[0013] S2: For any same ID at different times, based on the particle information obtained by the particle detection or predicted by the physical prediction model, the required particle motion parameters are calculated in real time by the trajectory tracking;

[0014] S3: When the same ID of the effective nanoparticles at different times are continuously captured, then enter S4; when the particle capture is interrupted due to occlusion or leaving the field of view, etc., then the particle information is predicted by the physical prediction model to compensate for tracking the interruption and return to S2;

[0015] S4: The effective nanoparticles are continuously tracked by the trajectory tracking, and the particle information and the particle motion parameters are displayed in real time in the visualization;

[0016] S5: The particle motion parameters are stored in real time by the data record.

[0017] Optionally, the particle detection includes at least one of a background subtractor, a morphological processor, and a contour analyzer; the background subtractor separates the foreground using the MOG2 algorithm; the morphological processor performs an opening operation and a dilation operation; the contour analyzer filters invalid particles based on an area threshold; and the particle information includes at least position and time information.

[0018] Optionally, the trajectory tracking uses a cost function for particle matching, and the cost function is wherein p i is a detection position; p k is a historical position; and α and β are weighting coefficients, the values of which are automatically adjusted according to a Kalman filter.

[0019] Optionally, the trajectory tracking includes at least basic distance calculation, motion prediction enhancement, and direction consistency check.

[0020] Optionally, the physical prediction model is a neural network with physical information, which outputs a prediction of future particle information by fusing the historical motion state of the particle and introducing physical law constraints; the physical prediction model includes at least one of a spatial encoder, a motion encoder, a bidirectional long short-term memory (BiLSTM), an attention mechanism, and a motion corrector; the spatial encoder extracts position sequence features; the motion encoder extracts historical speed sequence features; the BiLSTM integrates the spatial and temporal features of position and speed; the attention mechanism extracts key information; and the motion corrector corrects the predicted trajectory by physical law constraints.

[0021] Optionally, the physical prediction model is trained by minimizing a composite loss function, which simultaneously optimizes data fitting degree and physical consistency; the composite loss function includes at least one of a position mean square error and a speed change penalty term, and its expression is: wherein, is the position mean square error between the predicted position and the true position; For the speed change penalty term, used to punish the prediction result that violates the preset physical law (such as energy conservation, momentum conservation or fluid dynamics equation), the formula is: λ is a balance coefficient; the root mean square error between the predicted position and the real position of the physical prediction model after training is not greater than 2.5 pixels.

[0022] Optionally, the visualization realizes multi-layer rendering, at least including at least one of a particle actual trajectory layer, a particle predicted trajectory layer and an information superimposed layer; the particle actual trajectory layer draws continuous position points with solid lines; the particle predicted trajectory layer connects predicted position points with dashed lines; and the information superimposed layer displays target particle ID, speed and prediction error in real time.

[0023] Optionally, the particle motion parameters include at least one of speed, prediction error and acceleration.

[0024] Optionally, the motion corrector at least includes a kinematic constraint function, the formula of which is: Wherein, p t is the current position; v t is the current speed; d max is the maximum allowed displacement threshold, the formula of which is: Wherein, is the Brownian diffusion coefficient.

[0025] Optionally, the nanoparticles include at least one of metal nanoparticles or semiconductor nanoparticles or carbon nanoparticles; and the carbon nanoparticles include at least one of carbon nanotubes and graphene.

[0026] The present application solves the problem of missing physical law modeling by setting the physical prediction model, including a spatial encoder, a motion encoder, a BiLSTM, an attention mechanism and a motion corrector; solves the problem of trajectory breakage caused by particle loss in a complex environment by using a cost function for particle matching in the trajectory tracking; solves the problem of insufficient detection accuracy by including a background subtractor, a morphological processor and a contour analyzer in the particle detection; solves the problem of missing real-time interaction and data integration by realizing multi-layer rendering in the visualization and storing particle motion parameters in real time in the data record; and solves the problems of trajectory breakage caused by particle loss in a complex environment, missing physical law modeling and insufficient multi-module collaboration in the prior art. BRIEF DESCRIPTION OF DRAWINGS

[0027] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the present application. Obviously, the drawings in the following description only some of the embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor.

[0028] Figure 1 A flow chart of tracking and predicting nanoparticles provided by the embodiments of the present application is shown in FIG. 1.

[0029] Figure 2 A composition diagram of tracking and predicting nanoparticles provided by the embodiments of the present application is shown in FIG. 2.

[0030] Figure 3 A tracking diagram of motion trajectory of carbon nanotube particles provided by the embodiments of the present application is shown in FIG. 3.

[0031] Figure 4 A comparison diagram of predicted trajectory and actual trajectory of carbon nanotube particles provided by the embodiments of the present application is shown in FIG. 4.

[0032] Legend of the drawings:

[0033] 10, particle detection; 20, trajectory tracking; 30, physical prediction model; 40, visualization; 50, data recording. DETAILED DESCRIPTION

[0034] The technical solutions in the present application will be described clearly and completely in the following with reference to the drawings of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0035] It should be understood that the "one embodiment" or "an embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner.

[0036] Referring to Figures 1-4 The present application provides a method for tracking and predicting nanoparticles, characterized by being used for tracking and predicting nanoparticles, and at least comprising particle detection (10), trajectory tracking (20), physical prediction model (30), visualization (40) and data recording (50). The method at least comprises the following steps:

[0037] S1: processing video frames by the particle detection (10) to identify valid nanoparticles and obtain their particle information, and generating a unique identifier (ID) based on initial position information of the valid nanoparticles;

[0038] S2: for any same ID at different times, obtaining the particle information based on the particle detection (10) or the prediction of the physical prediction model (30), and calculating the required particle motion parameters in real time by the trajectory tracking (20);

[0039] S3: when the valid nanoparticles of the same ID at different times are continuously captured, then entering S4; when the capture of the nanoparticles is interrupted due to occlusion or leaving the field of view, etc., then predicting the particle information by the physical prediction model (30) to compensate for tracking interruption and returning to S2;

[0040] S4: continuously tracking the valid nanoparticles by the trajectory tracking (20), and displaying the particle information and the particle motion parameters in real time in the visualization (40);

[0041] S5: storing the particle motion parameters in real time by the data recording (50).

[0042] In the present application, the following methods are specifically adopted: first, processing video frames by the particle detection (10) to identify valid nanoparticles and obtain their particle information by using a background subtractor (11), a morphology processor (12) and a contour analyzer (13), and generating a unique identifier (ID) for each valid nanoparticle according to the obtained initial particle information; for any same ID at different times, the trajectory tracking (20) uses a cost function to match valid nanoparticles, and calculates particle motion parameters for each valid nanoparticle in real time; for the valid nanoparticles whose capture is interrupted due to occlusion or leaving the field of view, etc., the physical prediction model (30) is enabled to predict the future particle information of the nanoparticles whose capture is interrupted according to the historical frame number and the initialization state of the valid nanoparticles, so as to compensate for tracking interruption; the physical prediction model (30) fuses the historical motion state of the valid nanoparticles and introduces physical law constraints, and outputs the prediction of the future particle information through a space encoder (31), a motion encoder (32), a BiLSTM (33), an attention mechanism (34) and a motion corrector (35); then, the visualization (40) displays the particle information and the particle motion parameters in real time; finally, the data recording (50) stores the particle motion parameters in real time.

[0043] The application has the beneficial effects that: by setting the physical prediction model (30) containing a spatial encoder (31), a motion encoder (32), a BiLSTM (33), an attention mechanism (34), and a motion corrector (35), the problem of missing physical law modeling is solved; by using dynamic weighted distance for particle matching by the trajectory tracking (20), the problem of trajectory breaking caused by particle loss in a complex environment is solved; by the particle detection (10) containing a background subtractor (11), a morphological processor (12), and a contour analyzer (13), the problem of insufficient detection accuracy is solved; by the visualization (40) realizing multi-layer rendering, and the data recording (50) storing particle motion parameters in real time, the problems of missing real-time interaction and data integration are solved; the application solves the problems of trajectory breaking caused by particle loss in a complex environment, missing physical law modeling, and insufficient multi-module collaboration in the prior art.

[0044] It should be noted that the video frame refers to each image in the video, and the processing is to filter out invalid particles in the video frame and retain effective nanoparticles.

[0045] It should be noted that the nanoparticles involved in the application are preferably carbon nanoparticles, and tracking and prediction are used to enable the carbon nanoparticles to be monitored in real time and information capture to be more perfect due to the limitation and influence of physical equipment.

[0046] Preferably, the carbon nanoparticles at least include carbon nanotubes, graphene, and diamond.

[0047] Optionally, the particle detection (10) at least includes at least one of a background subtractor (11), a morphological processor (12), and a contour analyzer (13); the background subtractor (11) separates the foreground by using a MOG2 algorithm; the morphological processor (12) performs an opening operation and a dilation operation; the contour analyzer (13) filters invalid particles based on an area threshold; and the particle information at least includes position and time information.

[0048] It should be noted that in the embodiment of the application, the particle information further includes particle color, contour, diameter, and the like.

[0049] In the present application, the particle detection (10) at least includes a background subtractor (11), a morphological processor (12) and a contour analyzer (13). For the nanometer particles in the high-speed camera microscopic video, the background subtractor (11) uses the MOG2 algorithm to dynamically learn the background model, effectively extracts the moving particle region in the Brownian motion or fluid disturbance through the pixel-level foreground separation, and suppresses the interference of complex background noise. The morphological processor (12) sequentially performs the opening operation and the dilation operation, wherein the opening operation eliminates isolated noise points and smoothes the particle edge through the combination of erosion-dilation, and the dilation operation compensates for the incomplete particle edge caused by defocus or low contrast, ensuring the integrity of the particle region in the binary image. The contour analyzer (13) filters the detected contour based on the preset area threshold, eliminates the small invalid region caused by noise or device interference, and only retains the effective detection result meeting the physical size of the nanometer particle, thereby significantly improving the accuracy and robustness of particle detection in high-concentration or low signal-to-noise ratio scenes.

[0050] Optionally, the trajectory tracking (20) uses a cost function for particle matching, and the cost function is wherein p i is a detection position; p k is a historical position; and the α and β are both weighting coefficients, and the values thereof are automatically adjusted according to the Kalman filter.

[0051] In the present application, the trajectory tracking module (20) constructs an adaptive matching strategy, and the core thereof is to adjust the parameter weight of the matching cost function in real time according to the particle motion state. Specifically, when the detection position p i (current frame particle position) is associated with the historical position p k (last frame predicted position), the cost function combines the Euclidean distance with the velocity continuity index: wherein the α is a position item weighting coefficient, the β is a velocity item weighting coefficient, both of which are dynamically adjusted through the state estimation error of the Kalman filter, the α and the β are both known quantities, and the value range thereof is between [0, 1], and the initial value of the known quantity can be determined arbitrarily in the value range. In the particle high-speed motion scene, the system automatically increases the value of β to strengthen the velocity constraint and suppress the false matching caused by large displacement; and in the low-noise environment, the value of α dominates to ensure the position accuracy. This dynamic weight mechanism effectively solves the ID switching problem of the traditional fixed threshold matching in the high-density particle group, and is especially suitable for the correlation of the complex motion trajectory of the carbon nanotube rotating in the fluid or the graphene sheet folding. In addition, when the detection loss caused by short-time occlusion occurs, the module automatically uses the estimated value of the physical prediction model (30) as p kThe history velocity feature extracted by the motion encoder (32) restricts the matching process, significantly improves the continuity of the trajectory and the anti-interference ability.

[0052] Optionally, the trajectory tracking (20) at least includes basic distance calculation, motion prediction enhancement and direction consistency check.

[0053] In the present application, the trajectory tracking (20) constructs a multi-level matching strategy system, and realizes accurate particle association in a complex scene through the synergistic effect of basic distance calculation, motion prediction enhancement and direction consistency check. i i The basic distance calculation: an initial matching cost matrix is constructed by using a dynamic weighted Euclidean distance, in combination with particle center point coordinates (x The basic distance calculation And a velocity weight factor is introduced The matching cost of high-speed moving particles is dynamically adjusted to avoid the mis-matching of adjacent particles caused by a fixed threshold; the motion prediction enhancement: for a nonlinear motion scene, a predicted position is generated based on the history trajectory data of the physical prediction model (30), the predicted position is corrected by the time-series motion features (such as acceleration, diffusion coefficient) extracted by the fusion LSTM, the residual between the predicted value and the detected position is included in the matching cost function, and a is formed, wherein μ is a prediction confidence coefficient, which is dynamically optimized with model training; the direction consistency check module: a motion direction vector is defined, and the cosine similarity between the current frame detection direction and the history motion direction is calculated A direction threshold δ is set to filter abnormal matching of direction mutation, effectively suppressing ID switching errors caused by particle rotation or fluid disturbance in Brownian motion.

[0054] Optionally, the physical prediction model (30) is a neural network with physical information, which outputs the prediction of future particle information by fusing the history motion state of the effective nano-particle and introducing physical law constraints; the physical prediction model (30) includes at least one of a space encoder (31), a motion encoder (32), a bidirectional long short-term memory (BiLSTM) (33), an attention mechanism (34) and a motion corrector (35); the space encoder (31) extracts position sequence features; the motion encoder (32) extracts history velocity sequence features; the BiLSTM (33) integrates the space-time features of position and velocity; the attention mechanism (34) extracts key information; and the motion corrector (35) corrects the predicted trajectory by physical law constraints.

[0055] ​It should be noted that the physical law includes physical information conforming to the law of particle motion, such as fluid mechanics or energy conservation or momentum conservation or Brownian motion constraint, which is embedded in the forward propagation process of the neural network in a mathematical form to correct the predicted trajectory.

[0056] In the present application, the physical prediction model (30) constructs a deep neural network architecture embedded with multiple physical constraints, and realizes the physically consistent prediction of the nanoparticle motion trajectory through the hierarchical cooperation of the spatial encoder (31), the motion encoder (32), the BiLSTM (33), the attention mechanism (34) and the motion corrector (35). The spatial encoder (31): adopts a two-dimensional convolutional layer to extract the spatial features of the particle position sequence, and converts the position coordinates (x t ,y t ) of the continuous k frames into a feature vector P s of dimension d t =Conv2D([x t-k+1 ,y t-k+1 ;…;x t ,y t ]). By capturing the position fluctuation pattern through a local receptive field, for example, the spiral drift trajectory of carbon nanotubes in solution or the Brownian displacement characteristics of graphene sheets, the position embedding vector containing spatial correlation is output, providing the basis features of spatial dimension for subsequent time series modeling. The motion encoder (32): extracts the velocity time series features for the velocity feature structure, and constructs a feature matrix V t =BiLSTM([v t-k+1 ,a t-k+1 ,θ t-k+1 ;…;v t ,a t ,θ t ]) containing motion trends by calculating the acceleration a t =v t -v t-1 and the velocity direction angle . This module is especially suitable for capturing the velocity mutation under fluid disturbance (such as velocity field fluctuation caused by Reynolds number change), and provides the motion dynamics basis for fusing space-time features. The BiLSTM (33): integrates the position features P t output by the spatial encoder and the velocity features V t output by the motion encoder through a gating mechanism to form a space-time joint feature H t =LSTM f ([P t ;V t ],H t-1). Where the forget gate dynamically adjusts the noise information with low physical relevance in the historical trajectory (such as abnormal position points caused by accidental defocusing), the input gate selectively retains the motion mode conforming to the Stokes-Einstein diffusion law, ensuring that the hidden state H t At the same time, the spatial position distribution and the evolution law of the motion state of the particles are included, and the time sequence dependence in the nonlinear motion is effectively modeled. The attention mechanism (34): the self-attention mechanism is introduced to weight the key frames in the long time sequence trajectory, and a frame similarity matrix is calculated where W q , W k are query and key projection matrices, d h is the hidden layer dimension. By enhancing the feature weight of key event frames such as collision and aggregation (for example, the frame of sudden change in velocity when the graphene sheet layer is folded), the interference of low signal-to-noise ratio frames is suppressed, so that the model focuses on the core motion state that determines the physical prediction, and the prediction robustness in complex scenes is improved. The motion corrector (35): a physical constraint layer is designed to embed the physical law in the form of mathematics into the forward propagation process of the neural network, and the predicted position and the velocity The Stokes resistance model and the energy conservation constraint are introduced, and the predicted results are corrected by a penalty term where λ1 and λ2 are physical constraint weights, η is the fluid viscosity, and ρ and V are the particle density and volume. This module forces the predicted trajectory to conform to the fluid motion law, avoiding the generation of anti-physical trajectories by pure data-driven models. The physical prediction model (30) realizes dual optimization from data-driven to physical constraint: the spatial encoder and the motion encoder complete the physical meaning deconstruction of the original features, the BiLSTM and the attention mechanism capture the time sequence features conforming to the actual motion law, and the motion corrector ensures the physical interpretability of the predicted results by explicitly introducing the physical equation.

[0057] Optionally, the physical prediction model (30) is trained by minimizing a composite loss function, which simultaneously optimizes data fitting degree and physical consistency; the composite loss function includes at least one of a position mean square error and a velocity change penalty term, and its expression is: wherein, is the position mean square error between the predicted position and the true position; is the velocity change penalty term for punishing the predicted results that violate the preset physical law (such as energy conservation, momentum conservation or fluid dynamics equation), and its formula is: λ is a balance coefficient; after the training of the physical prediction model (30), the root mean square error between the predicted position and the true position is not greater than 2.5 pixels.

[0058] In the present application, the loss function design of the physical prediction model (30) takes into account the position accuracy and physical law constraints, and ensures the accuracy and physical consistency of the prediction results through multi-component joint optimization. The total expression of the composite loss function is: wherein, is the position mean square error (MSE), specifically the mean square error between the predicted position and the true position, quantifying the deviation of the predicted position from the true position; is the velocity change penalty term, used to punish the prediction results that violate the preset physical laws (such as energy conservation, momentum conservation or fluid dynamics equations), and to constrain the velocity evolution to comply with the fluid dynamics laws; λ is a balance coefficient, dynamically adjusting the weight according to the particle motion scene (such as Brownian motion, directional flow). The position mean square error The physical meaning of the position mean square error is to measure the Euclidean distance square mean between the model predicted position and the microscopic video labeled true position x t , ensuring the particle spatial positioning accuracy. For the sub-pixel level positioning requirement of nanoparticles, after the input features are processed by the spatial encoder (31), they are mapped to two-dimensional coordinate prediction values through a multi-layer perceptron. The composite loss function directly acts on the coordinate residual, optimizing the spatial encoder's ability to capture the position fluctuation pattern (such as the spiral drift trajectory of carbon nanotubes). The velocity change penalty term wherein, is the predicted velocity, is the Stokes drag acceleration (calculated from fluid viscosity η, particle radius r, density p and volume V). The viscous drag model is introduced into the fluid dynamics to force the velocity change to comply with the physical law, avoiding the emergence of physically unfounded sudden changes in velocity (such as instantaneous acceleration exceeding the limit of medium resistance). Technical implementation: combined with the historical speed features extracted by the motion encoder (32) and the physical constraints of the motion corrector (35), the velocity change penalty term constrains the adjacent frame velocity difference in differential form, so that the predicted velocity sequence satisfies the resistance balance relationship of F = -6πηrv. Dynamic adjustment mechanism of balance coefficient λ: through the attention mechanism (34) to analyze the physical feature complexity of the historical trajectory, when detecting that the particle has a collision, aggregation and other nonlinear events (such as the change of the velocity direction angle exceeding 45°), automatically reduce λ, and enhance the constraint weight of the velocity penalty term, avoid the anti-physical prediction caused by pure position optimization; in the training stage, the initial λ is determined through cross-validation, and λ and the model parameters are simultaneously optimized by gradient descent, so that the loss function remains robust in the multiple motion modes (diffusion, drift, rotation) of the nanoparticle. After the above physical prediction model (30) is trained, the root mean square error threshold between the predicted position and the real position is set to 2.5 pixels, that is, when the root mean square error is less than 2.5 pixels, it is considered that the physical prediction model (30) is accurate and the predicted particle position is available.

[0059] Optionally, the visualization (40) implements multi-layer rendering, including at least one of the particle actual trajectory layer (41), the particle predicted trajectory layer (42) and the information superimposed layer (43); the particle actual trajectory layer (41) draws continuous position points with solid lines; the particle predicted trajectory layer (42) connects predicted position points with dashed lines; the information superimposed layer (43) displays the target particle ID, velocity and prediction error in real time.

[0060] In the present application, the visualization (40) builds a spatiotemporally aligned nanoparticle motion trajectory visualization system through multi-layer layered rendering and dynamic interaction design. Its core architecture and working mechanism are as follows: the particle actual trajectory layer (41): adopt anti-aliasing Bezier curve to connect the continuously detected particle position points, ensuring sub-pixel level precision display; for non-spherical particles such as carbon nanotubes, replace the traditional circle with an elliptical marker (long-short axis ratio = actual measurement value) to truly restore the particle morphological features. Based on the HSV color space, assign a unique hue to each particle ID (such as ID = 1 corresponds to H = 0°, ID = 2 corresponds to H = 60°), and dynamically adjust the saturation and brightness with the particle motion speed, to realize intuitive differentiation of trajectories and visualization mapping of motion state. The trace point transparency follows the exponential decay law As τ is 10 frames, the latest position point a0 is 100%, the historical trajectory gradually fades, and the current position and recent motion trend of the particle are highlighted. The particle prediction trajectory layer (42): when the detection module loses the target particle, the physical prediction model (30) generates a prediction position sequence of the future frame based on the historical trajectory, and the prediction start point is marked with a triangular arrow, and the arrow color is consistent with the corresponding actual trajectory, but the transparency is reduced. The width of the dashed line of the prediction trajectory is dynamically adjusted according to the prediction confidence C, which is output by the motion corrector (35) of the physical prediction model (30), considering the length of the historical trajectory, the stability of the speed and the compliance of the physical constraint. When the prediction error exceeds the set value, the color of the dashed line gradually changes from green to red. The prediction error time sequence chart can be viewed by supporting mouse hovering, and the physical model input parameters (such as historical speed and fluid viscosity estimate value) at the prediction trajectory point can be displayed, realizing the explainability of the prediction process. The information superimposition layer (43): dynamic parameter display is performed, including at least particle ID, instantaneous speed and prediction error. A dynamic scale is drawn on the edge of the image, and the physical distance is displayed in the horizontal / vertical direction respectively, and the scale interval is adapted to the particle motion range, ensuring that the display accuracy is 1 / 10 of the current field of view. A mini speed distribution histogram (update frequency 1Hz) and a trajectory curvature distribution chart are embedded, and detailed statistical data can be viewed by clicking to zoom in. The visualization has shown significant advantages in particle fluid experiments: the complex trajectories of 500+ particles can be clearly displayed at the same time, and the prediction error visualization helps to quickly locate the invalid point of the physical model. By converting abstract motion parameters into intuitive visual information, a powerful analysis tool is provided for nanomaterial dynamics research.

[0061] Optionally, the particle motion parameters include at least one of speed, prediction error and acceleration.

[0062] In the present application, the data record (50) constructs a real-time synchronous multi-dimensional data acquisition and storage system, and the spatiotemporal consistency of the motion parameters and the original video frame is ensured through an event-driven streaming processing architecture. The specific implementation manner is as follows: a sub-pixel level positioning algorithm is adopted, the particle coordinates (x i ,y i ) are extracted through Gaussian fitting, the accuracy is 0.1 pixels, the circumscribed rectangle parameters output by the contour analyzer (13) of the particle detection (10) are combined, and the particle size and morphological features (such as the length-to-short-axis ratio) are recorded synchronously; based on the historical trajectory data of the trajectory tracking (20), the three-point central difference method is used to calculate the speed and the speed field features extracted by the motion encoder (32) of the physical prediction model (30) are used for Kalman filtering smoothing to suppress noise interference; the prediction position of the physical prediction model (30) is calculated in real time and the actual detection position x tEuclidean distance error Simultaneous recording of velocity prediction error For quantifying model reliability; adopt high-precision clock synchronization technology to accurately align each frame of image processing time with the camera exposure time, with time accuracy up to microseconds, ensuring the timing consistency when fusing data across devices. When rendering the information overlay layer (43), the stored discrete parameters are mapped to any time point through a time interpolation algorithm, ensuring that the visualized parameters are accurately synchronized with the video playback progress, with an error of less than 1 frame, forming a complete experimental metadata chain to support the environmental variable backtracking during subsequent causal analysis. The data recording (50) exhibits excellent performance in particle fluid experiments, providing a traceable and verifiable high-quality data foundation for nanomaterial dynamics research by accurately binding the motion parameters with the spatiotemporal of the original video frames, effectively solving the analysis bottleneck caused by asynchronous data storage in traditional tracking systems.

[0063] Optionally, the motion corrector (35) at least includes a kinematic constraint function, which is expressed as: Where p t is the current position; v t is the current velocity; d max is the maximum allowed displacement threshold, which is expressed as: Where, is the Brownian diffusion coefficient.

[0064] In the present invention, the motion corrector (35) constructs a prediction trajectory correction mechanism that conforms to physical laws through a physical constraint type kinematic function. Its core formula Achieves physical feasibility constraints on nanoparticle displacement, where p t The sub-pixel level coordinates (x t , y t ) output by the particle detection (10) module are mapped to the physical coordinate system after processing by the space encoder (31); v t is the instantaneous velocity vector v t = (v x , v y ) calculated by the motion encoder (32) based on the historical trajectory, combined with the time step Δt to convert the displacement increment C = v t · Δt; d max is the physical limit displacement dynamically calculated based on the Stokes-Einstein equation, which is Where, is the Brownian diffusion coefficient ( is the Boltzmann constant, T is the temperature, η is the fluid viscosity, and r is the particle radius), vmax is the maximum flow rate of the fluid, k is a safety factor.

[0065] Optionally, the nanoparticles include at least one of metal nanoparticles or semiconductor nanoparticles or carbon nanoparticles; the carbon nanoparticles include at least one of carbon nanotubes or graphene.

[0066] In the present application, the nanoparticles include at least metal nanoparticles, semiconductor nanoparticles and carbon nanoparticles, and the special physical properties thereof determine the targeted optimization value of the technical solution. By deeply coupling the material science characteristics with the advanced tracking algorithm, the present application breaks through the application bottleneck of the traditional general method in the nanometer scale, and provides a precise dynamic characterization tool for performance regulation of the nanomaterials.

[0067] It should be noted that, in this document, the terms "comprise", "comprise", or any other variant thereof are intended to cover non-exclusive inclusion, so that processes, methods, articles or devices that include a series of elements not only include those elements, but also include other elements not explicitly listed, or include elements inherent to such processes, methods, articles or devices. Without more limitations, the element defined by the statement "comprises a" does not exclude the presence of another identical element in the process, method, article or device that includes the element.

[0068] From the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be realized by means of software and necessary general hardware platforms, of course, they can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application or the parts that contribute to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes a plurality of instructions for making a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) execute the methods described in various embodiments of the present application.

[0069] The embodiments of the present application are described above in combination with the drawings, but the present application is not limited to the above-mentioned specific embodiments, and the above-mentioned specific embodiments are only illustrative, not limiting, and those skilled in the art can make many forms under the inspiration of the present application without departing from the purpose of the present application and the scope protected by the claims, which all belong to the protection of the present application.

[0070] Example 1

[0071] The present embodiment takes carbon nanotubes in the fluidized bed preparation process as the research object to verify the actual effect of the nanoparticle tracking and prediction method. The experiment uses a FASTCAM Mini UX100 high-speed camera, configures a 200K-C-16G image sensor, and collects the carbon nanotube motion process of the horizontal section of the center of the fluidized bed bed layer at a full resolution of 1280*1024 pixels, a total of 65 frames of video, a frame rate of 30.29FPS (time interval Δt≈0.033s), a total time of 2.08s, and a video pixel density of 5120px / m, which ensures that the pixel unit and the physical unit can be accurately converted, and all video frames have no motion blur or exposure abnormality.

[0072] In the particle detection stage, first, the input video is preprocessed: the color BGR image is converted into a single-channel gray image (conversion formula is Gray = 0.299R + 0.587G + 0.114B), 5*5 Gaussian kernel (standard deviation σ = 1.67) filtering is used for denoising, and then adaptive threshold segmentation is realized by maximizing the inter-class variance method, the gray image is converted into a binary image; then the MOG2 background subtractor is enabled, the background model is established based on the initial 30 frames, the background fluctuation is suppressed by setting 4 times the noise standard deviation, and the background model is updated in real time to extract the foreground mask of the carbon nanotube; then a 3*3 elliptical structural element is used to perform an opening operation (eliminate ≤3 pixel shot noise) and a 2-time dilation operation (connect the broken particle contour); finally, the connected domain is extracted by contour analysis, and the invalid contour is filtered based on the area threshold of 5px 2 ≤A≤300px 2 < 5px of each particle, and 199 carbon nanotube particles are successfully identified in 65 frames of video, and a unique ID (such as ID = 123, 145, 56, etc.) is generated based on the initial position of each particle and stored in the active particle dictionary.

[0073] In the trajectory tracking stage, a 30-frame length trajectory cache is established for each ID particle to record the historical position coordinates; then a dynamic weighted distance is used for particle matching, the Euclidean distance between the current frame detection position and the previous frame historical position is calculated, and a Hungarian algorithm cost matrix wherein α = 0.6, β = 0.4, β is automatically adjusted to 0.7 when moving at high speed, 50px is set as the matching threshold, and the cost matrix element is smaller than the threshold to determine that it is the same particle to complete the trajectory association; at the same time, the three-point central difference method is used to calculate the instantaneous speed of the particle (formula is The velocity data is smoothed using Kalman filtering, and the motion direction angle (θ = arctan2(Δy, Δx)) is calculated. Furthermore, an adaptive disappearance frame count of 10 frames is set; particles that do not match for 10 consecutive frames are considered permanently lost and removed. If less than 10 frames are reached, the historical trajectory is temporarily stored and re-detected based on the predicted position. This effectively solves the problems of trajectory intersection and temporary occlusion in carbon nanotubes. Figure 3 As shown, the trajectory matching accuracy was improved to 85.44% in the experiment.

[0074] When carbon nanotube particles are lost due to aggregation, obstruction, or temporary removal from view, such as Figure 4 As shown, a neural network with physical information is enabled: the model includes a spatial encoder (3 fully connected layers, mapping 2D position coordinates to 64D features), a motion encoder (3 fully connected layers, extracting 2D velocity vectors, mapping to 128D features, and then compressing to 64D), a 4-layer bidirectional LSTM (hidden layer dimension 256, integrating spatiotemporal features), a 4-head attention mechanism (Dropout = 0.1, focusing on keyframes such as collisions and clustering), and a motion corrector (3 fully connected layers, outputting 2D position corrections). The motion corrector embeds the Stokes-Einstein equation (…). k = 1.38 × 10 -23 Calculate the maximum allowable displacement threshold given J / K, T = 298K, η = 0.001Pa·s, and r = 50nm. To correct the trajectory; the model is trained using a composite loss function. After training, the root mean square error of the predicted position is ≤2.5 pixels. In the experiment, after the particle with ID=123 was lost due to aggregation in the 15th frame, the model output the predicted position for the next 5 frames based on the historical data of the previous 30 frames, and successfully re-associated the trajectory. The duration of the trajectory break is ≤1 frame.

[0075] The visualization phase employs a multi-layer rendering design: the actual particle trajectory layer uses anti-aliased solid lines to draw continuous position points, and different ID particles are assigned a unique color (e.g., ID=123 is red (RGB: 0,0,255), ID=145 is yellow (RGB: 0,255,255), ID=56 is green (RGB: 0,255,0)). The transparency of the trajectory points is adjusted according to... Exponential decay; the particle prediction trajectory layer connects the predicted position points with red dashed lines, and a triangular arrow is added to the starting point (the color is consistent with the actual trajectory, and the transparency is reduced by 50%). The width of the dashed line is dynamically adjusted according to the prediction confidence; the information overlay layer displays the particle ID (yellow text), instantaneous velocity (blue text), and prediction error (cyan text) in real time, and adds a dynamic ruler (marking physical distance) to the image edge. It also supports interactive functions such as hovering the mouse to view historical velocity curves and clicking on predicted trajectory points to view model input parameters, which facilitates real-time verification of algorithm reliability.

[0076] The data recording stage stores multi-dimensional motion parameters in real time, including sub-pixel level precision (0.1px) position coordinates (center point (x, y) and circumscribed rectangle size), Kalman-filtered instantaneous speed (horizontal / vertical components and combined speed, px / s and m / s dual units), prediction error (Euclidean distance between predicted position and actual position), and microsecond-level precision timestamp (synchronized with high-speed camera exposure time), ensuring that each frame parameter is spatiotemporally aligned with the original video frame; the data is stored in CSV format, the file name contains the collection time and experiment number, and an index file is generated, and subsequent analysis can quickly match particle state and original video through timestamp.

[0077] The experimental results verify the effect of solving the bottleneck of the prior art: for the trajectory breakage caused by particle loss in a complex environment, the combination of 30-frame trajectory caching and physical prediction model reduces the trajectory breakage rate from 35% of the traditional method to 8% (ID=56 particles only once breakage); for the lack of physical law modeling, the motion corrector embedded with fluid mechanics constraints improves the physical compliance of the predicted trajectory by 70%; for the lack of multi-module collaboration, dynamic weighted matching and cross-module information feedback reduce error accumulation by 50% (total error ≤0.1m / s). In terms of key performance indicators, the invalid contour filtering rate of particle detection is 92%, the missed detection rate is 4.5% (the missed detection rate of the traditional method is 49.5%), the trajectory integrity of 3 target particles is ≥90% (ID=145 particles with a speed peak of 1497.9px / s still maintaining trajectory continuity), the average prediction error of 7 target particles is 6.21-15.28px (combined 0.037-0.090m / s), and the accuracy rate of ID=36 particles (low-speed steady state) reaches 91.82%, meeting the industrial-level detection requirements.

[0078] Embodiment 1 takes fluidized bed carbon nanotubes as the research object, verifies the practicability of the method in a complex dynamic environment, can accurately track carbon nanotubes and output motion parameters in real time, provides data support for carbon nanotube preparation process optimization, and the modular design can flexibly adapt to other types of nanoparticles such as metal nanoparticles and semiconductor nanoparticles, and has good universality.

Claims

1. A method of nanoparticle tracking and prediction, characterized by, A method for tracking and predicting nanoparticles, comprising at least particle detection (10), trajectory tracking (20), physical prediction model (30), visualization (40) and data recording (50); the method comprises at least the following steps: S1: processing video frames by the particle detection (10) to identify valid nanoparticles and obtain their particle information, and generating a unique identifier (ID) based on the initial position information of the valid nanoparticles; S2: for any same ID at different times, based on the particle information obtained by the particle detection (10) or predicted by the physical prediction model (30), the particle motion parameters required by the trajectory tracking (20) are calculated in real time; S3: when the same ID at different times is continuously captured, S4 is entered; when the particle capture is interrupted due to occlusion or leaving the field of view, etc., the particle information is predicted by the physical prediction model (30) to compensate for the tracking interruption, and returns to S2; S4: the trajectory tracking (20) continuously tracks the valid nanoparticles, and the particle information and the particle motion parameters are displayed in real time in the visualization (40); S5: the data recording (50) stores the particle motion parameters in real time.

2. The method of claim 1, wherein, The particle detection (10) comprises at least one of a background subtractor (11), a morphological processor (12) and a contour analyzer (13); the background subtractor (11) separates the foreground using the MOG2 algorithm; the morphological processor (12) performs opening operation and dilation operation; the contour analyzer (13) filters invalid particles based on area threshold; the particle information at least includes position and time information.

3. The method of claim 1, wherein the nanoparticles are tracked and predicted by: The trajectory tracking (20) uses a cost function for particle matching, which is where p i is the detected position; p k is the historical position; and the alpha and the beta are weighting factors, whose values are automatically adjusted according to a Kalman filter.

4. The method of claim 1, wherein, The trajectory tracking (20) at least includes basic distance calculation, motion prediction enhancement and direction consistency check.

5. The method of claim 1, wherein, The physical prediction model (30) is a neural network with physical information, which outputs the prediction of future particle information by fusing the historical motion state of the particle and introducing physical law constraints; the physical prediction model (30) comprises at least one of a space encoder (31), a motion encoder (32), a bidirectional long short-term memory (BiLSTM) (33), an attention mechanism (34) and a motion corrector (35); the space encoder (31) extracts position sequence features; the motion encoder (32) extracts historical velocity sequence features; the BiLSTM (33) integrates the spatiotemporal features of position and velocity; the attention mechanism (34) extracts key information; the motion corrector (35) corrects the predicted trajectory by physical law constraints.

6. The method of claim 5, wherein, The physical prediction model (30) is trained by minimizing a compound loss function, which balances data fitting and physical consistency, and the compound loss function comprises at least one of a position mean square error and a velocity variation penalty term, and the expression of the compound loss function is: wherein, is the position mean square error between the predicted position and the real position; is the velocity variation penalty term for penalizing the predicted result which violates the preset physical law (such as the law of conservation of energy, the law of conservation of momentum or the fluid dynamics equation), and the formula of the velocity variation penalty term is: λ is a balance coefficient; after the training of the physical prediction model (30), the root mean square error between the predicted position and the real position is not greater than 2.5 pixels.

7. The method of claim 1, wherein, The visualization (40) realizes multi-layer rendering, including at least one of a particle actual trajectory layer (41), a particle predicted trajectory layer (42) and an information superimposed layer (43); the particle actual trajectory layer (41) draws continuous position points with solid lines; the particle predicted trajectory layer (42) connects predicted position points with dashed lines; the information superimposed layer (43) displays target particle ID, velocity and prediction error in real time.

8. The method of claim 1, wherein, The particle motion parameters include at least one of a velocity, a prediction error, and an acceleration.

9. The method of claim 5, wherein, The motion corrector (35) comprises at least a kinematic constraint function, formulated as: where p t is the current position; v t is the current velocity; d max is a maximum allowed displacement threshold, formulated as: where, is the Brownian diffusion coefficient.

10. The method of claim 1, wherein, The nanoparticles include at least one of metal nanoparticles or semiconductor nanoparticles or carbon nanoparticles; the carbon nanoparticles include at least one of carbon nanotubes or graphene.

Images