Real-time video localization method based on three-dimensional imaging simulation of ferromagnetic material magnetic field signals

Abstract

This invention discloses a real-time video localization method based on three-dimensional imaging simulation of magnetic field signals of ferromagnetic materials, relating to the detection of ferromagnetic materials in a medical MRI room. The method includes: piecewise fitting of the magnetic field gradient using B-spline basis functions; omnidirectional calibration of the array using a rotating platform to establish a nonlinear mapping matrix between the sensor output and the actual magnetic field strength; extraction of target shape features; optical flow estimation of video frames to compensate for the magnetic field measurement delay caused by target motion; establishment of an association matrix between the target state vector and the measured value; reconstruction of the three-dimensional spatial distribution of the ferromagnetic material using an inversion method based on a perspective transformation model; and fusion of the three-dimensional imaging results with the real-time video. Optical flow estimation of video frames compensates for the magnetic field measurement delay caused by target motion, and estimation of multiple modes in parallel improves the convergence speed of state estimation.

Description

Technical Field

[0001] This invention relates to the field of ferromagnetic material detection in MRI chambers in medical settings, specifically to a real-time video positioning method based on three-dimensional imaging simulation of ferromagnetic material magnetic field signals. Background Technology

[0002] In aerospace and energy equipment applications such as nuclear power plant pipelines and wind turbines, real-time monitoring of ferromagnetic components, including welds, bolts, and pipes, is required to detect cracks, corrosion, or foreign object intrusion. In urban underground pipe networks (such as metal pipes and cables), archaeological exploration (such as ancient tomb artifacts), and military counter-terrorism (such as buried explosives), rapid location of ferromagnetic targets and assessment of their spatial distribution are essential. In complex industrial environments such as mines and nuclear power plants, robots need to identify ferromagnetic obstacles, such as fallen tools and metal fragments, and plan their paths. In magnetic nanoparticle-targeted therapy, real-time monitoring of the distribution and concentration of ferromagnetic particles in the body is crucial.

[0003] Current medical MRI equipment for detecting ferromagnetic materials can only determine the approximate location of the ferromagnetic material by measuring the magnetic field signal intensity of multiple fluxgate sensors and identifying the sensor with the strongest signal. Typically, three fluxgate sensors are distributed on each side of the MRI room door to roughly determine the location of the object being measured: upper left, middle left, lower left, upper right, middle right, and lower right. If the target is located between two sensors, multiple locations will be determined simultaneously, making it impossible to provide a specific location. Furthermore, fluxgate sensors typically transmit analog signals or convert them to digital signals. However, regardless of whether the signal is analog or digital, the location, distance, and angle of the measured object remain abstract. Therefore, all such products on the market can only roughly indicate a few directions based on the amplitude of the signal changes measured by the sensor, such as "upper left, middle left, lower left, upper right, middle right, and lower right." Although these are accompanied by a display screen, lights, and voice prompts, strictly speaking, they still provide two-dimensional prompts for a few fixed segmented areas. They have poor anti-interference capabilities and cannot distinguish the source of the target object's signal. If there is a large object with a significant magnetic field signal outside the target area, it is impossible to distinguish it because the sensor collects waveforms or digital signals in the same form. Summary of the Invention

[0004] To address the aforementioned technical problems, a real-time video positioning method based on three-dimensional imaging simulation of magnetic field signals from ferromagnetic materials is provided. This technical solution solves the problems mentioned in the background section.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A real-time video localization method based on three-dimensional imaging simulation of magnetic field signals from ferromagnetic materials includes:

[0007] Based on the target's velocity, acceleration range, and the intensity of magnetic field gradient changes, the order of the B-spline is determined. The B-spline basis function is used to fit the magnetic field gradient piecewise, enabling the grid cells to deform in real time with the target's trajectory.

[0008] The array is omnidirectionally calibrated by rotating the platform to establish a nonlinear mapping matrix between the sensor output and the actual magnetic field strength.

[0009] The magnetic field gradient tensor is decomposed into an isotropic part and a partial tensor part to extract the target shape features;

[0010] Optical flow estimation is performed on video frames to compensate for the magnetic field measurement delay caused by target motion, and a parametric perspective transformation model from the image coordinate system to the magnetic field coordinate system is established.

[0011] Establish the correlation matrix between the target state vector and the measured value, and perform parallel estimation of the target's uniform velocity, uniform acceleration and random motion modes. The target state vector includes position, velocity and magnetic moment vector.

[0012] By utilizing the mapping relationship between magnetic field signals and spatial positions, and combining the characteristics of real-time acquired magnetic field signals, the three-dimensional spatial distribution of ferromagnetic materials is reconstructed through an inversion method based on a perspective transformation model, thereby achieving three-dimensional imaging.

[0013] The three-dimensional imaging results are fused with real-time video, and the position of ferromagnetic materials is marked in real time in the video frame based on the magnetic field positioning information.

[0014] Preferably, the step of estimating optical flow in video frames, compensating for the magnetic field measurement delay caused by target motion, and establishing a parametric perspective transformation model from the image coordinate system to the magnetic field coordinate system specifically includes:

[0015] Based on the local consistency assumption, the motion vectors of all pixels are the same within a neighborhood of an image.

[0016] The corner detection algorithm is used to detect feature points with gradients in the image, and a neighborhood window is selected for each feature point.

[0017] Within the window, an optical flow equation is established using the gray-level conservation assumption, ensuring that the gray-level values ​​of pixels within the window remain constant during movement.

[0018] The motion vectors of the feature points are obtained by solving the optical flow equation using the least squares method.

[0019] Based on the optical flow estimation results, predict the target's trajectory during the magnetic field measurement delay;

[0020] By combining the predicted motion trajectory and the measurement data from the magnetic field sensor, the magnetic field measurement value is corrected, and the magnetic field measurement data is aligned with the timestamp of the video frame to determine the delay time.

[0021] Based on the predicted motion trajectory, the magnetic field measurement values ​​are mapped to the target's actual position at the current moment;

[0022] The compensated magnetic field measurement data is fused with data from other sensors;

[0023] Using the collected corresponding point data, for each set of corresponding points, two equations are established based on the parametric perspective transformation model, and a system of equations is established to solve for the parameters of the parametric perspective transformation model.

[0024] Preferably, the step of establishing the correlation matrix between the target state vector and the measured values, and performing parallel estimation of the target's uniform velocity, uniform acceleration, and random motion patterns specifically includes:

[0025] Based on the estimation and uncertainty of the target's initial state, an initial state vector and covariance matrix are defined;

[0026] For each motion mode, a state transition model is defined to describe the change of the target state over time.

[0027] Define a uniform motion model where acceleration is zero, and state transitions involve only changes in position and velocity.

[0028] Define a uniformly accelerated motion model where acceleration is constant, and state transitions involve changes in position, velocity, and acceleration;

[0029] Define a random motion model where acceleration is a random process and the state transition needs to take into account the influence of random noise.

[0030] Assign an initial probability to each motion pattern and predict the target state at the current moment based on its state transition model;

[0031] For each motion pattern, the predicted state is updated by combining the measurement value at the current moment. The state vector and covariance matrix are adjusted by calculating the residual between the measurement value and the predicted state.

[0032] The likelihood value is calculated based on the residual, and the probability of the motion pattern is updated.

[0033] Based on the probabilities of each motion mode, their estimation results are weighted and averaged, and the fused state estimate is used as the final estimate for the current moment.

[0034] Preferably, the step of utilizing the mapping relationship between magnetic field signals and spatial positions, combined with the characteristics of real-time acquired magnetic field signals, to reconstruct the three-dimensional spatial distribution of ferromagnetic materials through an inversion method based on a perspective transformation model, and achieving three-dimensional imaging specifically includes:

[0035] Based on the projected magnetic field signal characteristics, an initial three-dimensional distribution model is generated in the target coordinate system. The magnetic field signal characteristics include gradient extrema and intensity peaks.

[0036] The target area is divided into a regular three-dimensional grid, and each grid is assigned a value based on the magnetic field signal strength.

[0037] The central framework of ferromagnetic materials is extracted through morphological manipulation, and then expanded outward to generate a complete distribution.

[0038] The initial distribution is cropped and corrected based on the geometric boundary of the target area, and the edge contour of the distribution is optimized by utilizing the gradient features of the magnetic field signal at the boundary.

[0039] Weights are assigned based on the signal-to-noise ratio of each sensor data, and the distribution results are weighted and superimposed.

[0040] For regions with conflicting multi-view distributions, a confidence-weighted approach is used to determine the final value.

[0041] By combining other physical field signals of the gravitational and electric fields, the robustness of the inversion method can be improved.

[0042] A joint forward model of gravity, electric field, and magnetic field is constructed to describe the comprehensive response of the target material to at least two field signals.

[0043] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0044] Based on the target velocity, acceleration range, and the severity of magnetic field gradient changes, the B-spline order is adaptively adjusted to perform piecewise fitting of the magnetic field gradient, enabling real-time deformation of the grid cells along the target trajectory. The sensor array is omnidirectionally calibrated using a rotating platform to establish a nonlinear mapping matrix between the sensor output and the actual magnetic field strength, eliminating environmental interference and individual differences. Optical flow estimation is performed on video frames to compensate for the magnetic field measurement delay caused by target motion. In random motion scenarios, parallel estimation is performed for uniform, uniformly accelerated, and random motion modes, with dynamic weight adjustment, improving the convergence speed of state estimation. Attached Figure Description

[0045] Figure 1 This is a flowchart of the real-time video positioning method based on three-dimensional imaging simulation of magnetic field signals of ferromagnetic materials according to the present invention.

[0046] Figure 2 This is a flowchart of the method for piecewise fitting of magnetic field gradients according to the present invention;

[0047] Figure 3 The flowchart of the method for establishing the nonlinear mapping matrix between sensor output and real magnetic field strength in this invention is shown below.

[0048] Figure 4This is a flowchart of the target shape feature extraction method of the present invention;

[0049] Figure 5 This is a flowchart of the optical flow estimation method for video frames according to the present invention;

[0050] Figure 6 This is a flowchart of the parallel estimation method for uniform velocity, uniform acceleration, and random motion patterns of a target according to the present invention.

[0051] Figure 7 This is a flowchart of the method for reconstructing the three-dimensional spatial distribution of ferromagnetic materials according to the present invention. Detailed Implementation

[0052] The following description is intended to disclose the invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art.

[0053] Reference Figure 1 As shown, a real-time video positioning method based on three-dimensional imaging simulation of magnetic field signals of ferromagnetic materials includes:

[0054] Based on the target's velocity, acceleration range, and the intensity of magnetic field gradient changes, the order of the B-spline is determined. The B-spline basis function is used to fit the magnetic field gradient piecewise, enabling the grid cells to deform in real time with the target's trajectory.

[0055] The array is omnidirectionally calibrated by rotating the platform to establish a nonlinear mapping matrix between the sensor output and the actual magnetic field strength.

[0056] The magnetic field gradient tensor is decomposed into an isotropic part and a partial tensor part to extract the target shape features;

[0057] Optical flow estimation is performed on video frames to compensate for the magnetic field measurement delay caused by target motion, and a parametric perspective transformation model from the image coordinate system to the magnetic field coordinate system is established.

[0058] Establish the correlation matrix between the target state vector and the measured value, and perform parallel estimation of the target's uniform velocity, uniform acceleration and random motion modes. The target state vector includes position, velocity and magnetic moment vector.

[0059] By utilizing the mapping relationship between magnetic field signals and spatial positions, and combining the characteristics of real-time acquired magnetic field signals, the three-dimensional spatial distribution of ferromagnetic materials is reconstructed through an inversion method based on a perspective transformation model, thereby achieving three-dimensional imaging.

[0060] The three-dimensional imaging results are fused with real-time video, and the position of ferromagnetic materials is marked in real time in the video frame based on the magnetic field positioning information.

[0061] Reference Figure 2As shown, based on the target's velocity, acceleration range, and the drastic change in the magnetic field gradient, the order of the B-spline is determined. The magnetic field gradient is piecewise fitted using B-spline basis functions to achieve real-time deformation of the mesh elements according to the target's trajectory. Specifically, this includes:

[0062] Motion sensors are installed on the target object to collect speed and acceleration data during the target's movement in real time. The collected motion data is then filtered to remove noise interference.

[0063] A magnetic field sensor array is arranged around the target motion area to collect magnetic field gradient data in real time. The collected magnetic field gradient data is calibrated and normalized to eliminate the sensor's own error.

[0064] Statistically analyze the maximum speed, minimum speed, average speed, and rate of change of speed parameters during the target's motion, and plot a speed-time curve to show the changes in the target's speed;

[0065] Calculate the maximum acceleration, minimum acceleration, average acceleration, and rate of change of acceleration parameters during the target's motion, and analyze the fluctuation range and trend of acceleration.

[0066] Based on the changes in velocity and acceleration, the target's motion state is divided into a uniform motion stage, an accelerated motion stage, and a decelerated motion stage.

[0067] Calculate the mean, variance, maximum and minimum statistics of the magnetic field gradient to assess the overall degree of change in the magnetic field gradient;

[0068] Plot the curves of magnetic field gradient variation over time and space, analyze the trend and local characteristics of magnetic field gradient variation, and observe whether there are abrupt or periodic changes in magnetic field gradient.

[0069] When the target's trajectory is relatively smooth and the magnetic field gradient changes slowly, a quadratic B-spline is chosen for fitting.

[0070] When the target's trajectory is complex and the magnetic field gradient changes drastically, a fourth-order B-spline is selected for fitting.

[0071] Based on the determined B-spline order and node vector, calculate the B-spline basis functions within each fitting interval;

[0072] To minimize the sum of squared errors between the fitted curve and the actual magnetic field gradient data, the fitting coefficients are obtained by solving a system of linear equations.

[0073] The obtained fitting coefficients are multiplied by the corresponding B-spline basis functions and summed to obtain the magnetic field gradient fitting curves in each fitting interval. The fitting curves of each interval are connected to obtain the piecewise fitting result of the entire magnetic field gradient.

[0074] Calculate the mean, maximum, and minimum values ​​of the magnetic field gradient, and record the ratio of the difference between the maximum and minimum values ​​to the mean as the overall degree of change of the magnetic field gradient. When the overall degree of change of the magnetic field gradient is less than 0.3 and the motion state is uniform, a quadratic B-spline is selected for fitting. When the overall degree of change of the magnetic field gradient is greater than or equal to 0.3 and the motion state is acceleration or deceleration, a quadratic B-spline is selected for fitting.

[0075] Reference Figure 3 As shown, the array is omnidirectionally calibrated using a rotating platform to establish a nonlinear mapping matrix between the sensor output and the actual magnetic field strength. Specifically, this includes:

[0076] S101: Fix the sensor array on the rotating platform and set the initial rotation angle to 0 degrees;

[0077] S102: Start the rotating platform and rotate it in predetermined 15-degree increments;

[0078] S103: At each rotation angle, wait for the sensor output to stabilize and collect the output data of the sensor array;

[0079] S104: Determine whether the rotating platform has completed a full 360-degree rotation. If yes, no output is made; otherwise, return to step S102.

[0080] S105: Select the nonlinear function form according to the characteristics of the sensor and the application scenario. The nonlinear function form includes polynomial model, exponential model and neural network model.

[0081] S106: Combine the mapping functions of each sensor into a matrix form. For M sensors and N data points, construct an M×N mapping matrix, where each element represents the mapping value of each sensor at each data point.

[0082] The sensor spatial distribution is based on the three-dimensional space of the MRI room entrance area. Facing the door, the X-axis has a width of 2 meters from left to right, the Y-axis has a height of 2 meters vertically upwards from the ground, and the Z-axis has a depth of 1.5 meters outwards from the door frame. There are 2 sets of X-axis sensors, 3 sets of Y-axis magnetic sensors, and 3 sets of Z-axis magnetic sensors (the number of magnetic sensors is variable; the more hardware devices, the higher the physical resolution and the resolution of the acquired data). Data is collected at 10 cubic centimeters per coordinate point along the X, Y, and Z axes (each set consists of data from 6 to 9 sensors), totaling 6000 sets of data. Based on this, a three-dimensional simulation data visual model is established. The simulation visual model has two states: relatively static... The system employs both static and dynamic models. When no target object enters the detection range, the data visual model remains relatively stable and stationary. When a ferromagnetic target object enters, the data model measures changes in the data. Based on the algorithm, the XYZ coordinates of the changing data are obtained and visualized as two-dimensional and three-dimensional graphics. These are then marked with red from dark to light at each rotation angle. After waiting for more than 5 seconds for the sensor output to stabilize, the three-axis outputs of all sensors are collected synchronously at a sampling rate greater than 10Hz. The average value of 10 measurements is taken. After the rotating platform completes a 360° rotation, the data integrity is checked. When selecting a model, the polynomial model is suitable for scenarios where the magnetic field strength and sensor output have a smooth nonlinear relationship, while the exponential model is suitable for scenarios with significant magnetic saturation effects.

[0083] Reference Figure 4 As shown, the magnetic field gradient tensor is decomposed into an isotropic part and a partial tensor part. Extracting the target shape features specifically includes:

[0084] The partial tensor is recorded as a quadratic form of an ellipsoid, and the shape features of the target are extracted by analyzing the shape of the ellipsoid.

[0085] Calculate the eigenvalues ​​of the deviator tensor, analyze their magnitude and sign, and obtain the characteristics of the magnetic field gradient change of the target in each direction;

[0086] By fitting an ellipsoid, the ratio of the major and minor axes is extracted to reflect the elongation and compression of the target.

[0087] Analyze the direction of the feature vectors to extract the main directional features of the target;

[0088] By combining higher-order information from the magnetic field gradient tensor, the curvature features of the target are extracted to describe its shape characteristics.

[0089] By analyzing the ratio of the major and minor axes, principal directions, and curvature of the ellipsoid, geometric features such as the ratio of the major and minor axes and principal directions are mapped to the actual shape of the target, such as rod-shaped, sheet-shaped, or spherical. The shape features are then superimposed onto the real-time video image, and the target position and geometric attributes are labeled. The isotropic part reflects the uniform variation in the magnetic field gradient tensor that is independent of the target shape, while the partial tensor part contains information related to the target shape. Through this decomposition, the influence of the target shape on the magnetic field gradient can be highlighted, laying the foundation for subsequent extraction of target shape features.

[0090] Reference Figure 5 As shown, optical flow estimation is performed on video frames to compensate for the magnetic field measurement delay caused by target motion. A parametric perspective transformation model from the image coordinate system to the magnetic field coordinate system is established, specifically including:

[0091] Based on the local consistency assumption, the motion vectors of all pixels are the same within a neighborhood of an image.

[0092] The corner detection algorithm is used to detect feature points with gradients in the image, and a neighborhood window is selected for each feature point.

[0093] Within the window, an optical flow equation is established using the gray-level conservation assumption, ensuring that the gray-level values ​​of pixels within the window remain constant during movement.

[0094] The motion vectors of the feature points are obtained by solving the optical flow equation using the least squares method.

[0095] Based on the optical flow estimation results, predict the target's trajectory during the magnetic field measurement delay;

[0096] By combining the predicted motion trajectory and the measurement data from the magnetic field sensor, the magnetic field measurement value is corrected, and the magnetic field measurement data is aligned with the timestamp of the video frame to determine the delay time.

[0097] Based on the predicted motion trajectory, the magnetic field measurement values ​​are mapped to the target's actual position at the current moment;

[0098] The compensated magnetic field measurement data is fused with data from other sensors;

[0099] Using the collected corresponding point data, for each set of corresponding points, two equations are established based on the parametric perspective transformation model, and a system of equations is established to solve for the parameters of the parametric perspective transformation model.

[0100] The optical flow equation is:

[0101]

[0102] In the formula, Gray is the grayscale value function of the pixel within the window, x is the horizontal coordinate of the pixel, y is the vertical coordinate of the pixel, and t is the time parameter;

[0103] Optical flow refers to the temporal motion of each pixel in an image. The goal of optical flow estimation is to calculate the motion vector of each pixel between two consecutive frames based on image information. Optical flow estimation can be represented in two ways: dense optical flow and sparse optical flow. Dense optical flow means that the optical flow vector is calculated for every pixel in the image, while sparse optical flow only selects a subset of pixels to calculate the optical flow vector. The basic assumption of optical flow estimation is that the light intensity is constant, that is, the light intensity does not change in the area around a pixel. Based on this assumption, we can infer the motion information of an object by observing changes in pixel values.

[0104] Reference Figure 6 As shown, the establishment of the correlation matrix between the target state vector and the measured values, and the parallel estimation of the target's uniform velocity, uniform acceleration, and random motion modes specifically include:

[0105] Based on the estimation and uncertainty of the target's initial state, an initial state vector and covariance matrix are defined;

[0106] For each motion mode, a state transition model is defined to describe the change of the target state over time.

[0107] Define a uniform motion model where acceleration is zero, and state transitions involve only changes in position and velocity.

[0108] Define a uniformly accelerated motion model where acceleration is constant, and state transitions involve changes in position, velocity, and acceleration;

[0109] Define a random motion model where acceleration is a random process and the state transition needs to take into account the influence of random noise.

[0110] Assign an initial probability to each motion pattern and predict the target state at the current moment based on its state transition model;

[0111] For each motion pattern, the predicted state is updated by combining the measurement value at the current moment. The state vector and covariance matrix are adjusted by calculating the residual between the measurement value and the predicted state.

[0112] The likelihood value is calculated based on the residual, and the probability of the motion pattern is updated.

[0113] Based on the probabilities of each motion mode, their estimation results are weighted and averaged, and the fused state estimate is used as the final estimate for the current moment.

[0114] The time step is set according to the sensor sampling rate and must be much smaller than the target motion characteristic time. The fused position, velocity, acceleration and magnetic moment vectors are output for 3D imaging and video annotation. The probability of each motion mode is displayed in real time to help judge the target motion characteristics. By estimating three motion modes in parallel—uniform velocity, uniform acceleration and random motion—it can adapt to different motion states that the target may have at different times. When the actual motion mode of the target is close to a certain assumed model, the estimation result corresponding to that model will be more accurate. By weighted averaging, the advantages of each model can be combined to reduce the error caused by the estimation of a single model, thereby improving the accuracy of target state estimation.

[0115] Reference Figure 7 As shown, by utilizing the mapping relationship between magnetic field signals and spatial positions, combined with the characteristics of real-time acquired magnetic field signals, the three-dimensional spatial distribution of ferromagnetic materials is reconstructed through an inversion method based on a perspective transformation model, achieving three-dimensional imaging. Specifically, this includes:

[0116] Based on the projected magnetic field signal characteristics, an initial three-dimensional distribution model is generated in the target coordinate system. The magnetic field signal characteristics include gradient extrema and intensity peaks.

[0117] The target area is divided into a regular three-dimensional grid, and each grid is assigned a value based on the magnetic field signal strength.

[0118] The central framework of ferromagnetic materials is extracted through morphological manipulation, and then expanded outward to generate a complete distribution.

[0119] The initial distribution is cropped and corrected based on the geometric boundary of the target area, and the edge contour of the distribution is optimized by utilizing the gradient features of the magnetic field signal at the boundary.

[0120] Weights are assigned based on the signal-to-noise ratio of each sensor data, and the distribution results are weighted and superimposed.

[0121] For regions with conflicting multi-view distributions, a confidence-weighted approach is used to determine the final value.

[0122] By combining other physical field signals of the gravitational and electric fields, the robustness of the inversion method can be improved.

[0123] A joint forward model of gravity, electric field, and magnetic field is constructed to describe the comprehensive response of the target material to at least two field signals.

[0124] The joint forward model of gravity-electric field-magnetic field is as follows:

[0125]

[0126] In the formula, Δg(r) represents the forward gravity model, G is the gravitational constant, ρ(r') is the density of matter at the source location, φ(r) represents the forward electric field model, ε is the dielectric constant, σ(r') is the conductivity at the source location, and r', r, ... Let B(r) be the source position, current position, and theoretical position, respectively; B(r) be the forward model of the magnetic field; μ0 be the free permeability; and M(r') be the magnetization at the source position.

[0127] The software algorithm fits the simulated image from the magnetic sensor and the real-time image from the camera, overlaying the coordinates of the two images. The red markers indicating disturbances in the simulated magnetic field are then displayed on the coordinates corresponding to the real-time image. The real-time camera image has the functions of target distance, movement trajectory, and visual recognition of the human torso. The coordinates of the red markers indicating magnetic field disturbances are used in conjunction with the camera to make judgments. This allows for the accurate identification and marking of items carried on a person that are invisible to the naked eye, such as mobile phones, coins, keys, small folding knives, and lighters, thereby improving the accuracy, performance, and reliability of the inspection.

[0128] Furthermore, this solution also proposes a computer-readable storage medium storing a computer-readable program, which, when invoked, executes the aforementioned real-time video positioning method based on three-dimensional imaging simulation of ferromagnetic material magnetic field signals.

[0129] It is understandable that the storage medium can be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; an optical medium, such as a DVD; or a semiconductor medium, such as a solid-state drive (SSD).

[0130] In summary, the advantages of this invention are as follows: based on the target velocity, acceleration range, and the degree of change in the magnetic field gradient, the order of the B-spline is adaptively adjusted to fit the magnetic field gradient piecewise, enabling the grid cells to deform in real time with the target trajectory; the sensor array is omnidirectionally calibrated through a rotating platform to establish a nonlinear mapping matrix between the sensor output and the actual magnetic field strength, eliminating environmental interference and individual differences; optical flow estimation is performed on video frames to compensate for the magnetic field measurement delay caused by target motion; in random motion scenarios, parallel estimation is performed for uniform velocity, uniform acceleration, and random motion modes, and the weights are dynamically adjusted, thus improving the convergence speed of state estimation.

[0131] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention. The scope of protection claimed by the appended claims and their equivalents is defined.

Claims

1. A real-time video positioning method based on three-dimensional imaging simulation of magnetic field signals of ferromagnetic materials, characterized in that, include: Based on the target's velocity, acceleration range, and the intensity of magnetic field gradient changes, the order of the B-spline is determined. The B-spline basis function is used to fit the magnetic field gradient piecewise, enabling the grid cells to deform in real time with the target's trajectory. The array is omnidirectionally calibrated by rotating the platform to establish a nonlinear mapping matrix between the sensor output and the actual magnetic field strength. The magnetic field gradient tensor is decomposed into an isotropic part and a partial tensor part to extract the target shape features; Optical flow estimation is performed on video frames to compensate for the magnetic field measurement delay caused by target motion, and a parametric perspective transformation model from the image coordinate system to the magnetic field coordinate system is established. Establish the correlation matrix between the target state vector and the measured value, and perform parallel estimation of the target's uniform velocity, uniform acceleration and random motion modes. The target state vector includes position, velocity and magnetic moment vector. By utilizing the mapping relationship between magnetic field signals and spatial positions, and combining the characteristics of real-time acquired magnetic field signals, the three-dimensional spatial distribution of ferromagnetic materials is reconstructed through an inversion method based on a perspective transformation model, thereby achieving three-dimensional imaging. The three-dimensional imaging results are fused with real-time video, and the position of ferromagnetic material is marked in real time in the video frame based on the magnetic field positioning information. The determination of the order of the B-spline based on the target's velocity, acceleration range, and the drastic change in the magnetic field gradient specifically includes: Motion sensors are installed on the target object to collect velocity and acceleration data during the target's movement in real time. The collected motion data is then filtered to remove noise interference. A magnetic field sensor array is arranged around the target motion area to collect magnetic field gradient data in real time. The collected magnetic field gradient data is calibrated and normalized to eliminate the sensor's own error. Statistically analyze the maximum speed, minimum speed, average speed, and rate of change of speed parameters during the target's motion, and plot a speed-time curve to show the changes in the target's speed; Calculate the maximum acceleration, minimum acceleration, average acceleration, and rate of change of acceleration parameters during the target's motion, and analyze the fluctuation range and trend of acceleration. Based on the changes in velocity and acceleration, the target's motion state is divided into a uniform motion stage, an accelerated motion stage, and a decelerated motion stage. Calculate the mean, variance, maximum and minimum statistics of the magnetic field gradient to assess the overall degree of change in the magnetic field gradient; Plot the curves of magnetic field gradient variation over time and space, analyze the trend and local characteristics of magnetic field gradient variation, and observe whether there are abrupt or periodic changes in magnetic field gradient. When the target's trajectory is relatively smooth and the magnetic field gradient changes slowly, a quadratic B-spline is chosen for fitting. When the target's trajectory is complex and the magnetic field gradient changes drastically, a fourth-order B-spline is selected for fitting.

2. The real-time video positioning method based on three-dimensional imaging simulation of ferromagnetic material magnetic field signals according to claim 1, characterized in that, The step of using B-spline basis functions to piecewise fit the magnetic field gradient to achieve real-time deformation of the grid cells according to the target's motion trajectory specifically includes: Based on the determined B-spline order and node vector, calculate the B-spline basis functions within each fitting interval; To minimize the sum of squared errors between the fitted curve and the actual magnetic field gradient data, the fitting coefficients are obtained by solving a system of linear equations. The obtained fitting coefficients are multiplied by the corresponding B-spline basis functions and summed to obtain the magnetic field gradient fitting curves in each fitting interval. The fitting curves of each interval are connected to obtain the piecewise fitting result of the entire magnetic field gradient.

3. The real-time video positioning method based on three-dimensional imaging simulation of ferromagnetic material magnetic field signals according to claim 2, characterized in that, The step of omnidirectionally calibrating the array using a rotating platform to establish a nonlinear mapping matrix between the sensor output and the actual magnetic field strength specifically includes: S101: Fix the sensor array on the rotating platform and set the initial rotation angle to 0 degrees; S102: Start the rotating platform and rotate it in predetermined 15-degree increments; S103: At each rotation angle, wait for the sensor output to stabilize and collect the output data of the sensor array; S104: Determine whether the rotating platform has completed a full 360-degree rotation. If yes, no output is made; otherwise, return to step S102. S105: Select the nonlinear function form according to the characteristics of the sensor and the application scenario. The nonlinear function form includes polynomial model, exponential model and neural network model. S106: Combine the mapping functions of each sensor into a matrix form. For M sensors and N data points, construct an M×N mapping matrix, where each element represents the mapping value of each sensor at each data point.

4. The real-time video positioning method based on three-dimensional imaging simulation of ferromagnetic material magnetic field signals according to claim 3, characterized in that, The specific steps of decomposing the magnetic field gradient tensor into an isotropic component and a partial tensor component to extract target shape features include: The partial tensor is recorded as a quadratic form of an ellipsoid, and the shape features of the target are extracted by analyzing the shape of the ellipsoid. Calculate the eigenvalues ​​of the deviator tensor, analyze their magnitude and sign, and obtain the characteristics of the magnetic field gradient change of the target in each direction; By fitting an ellipsoid, the ratio of the major and minor axes is extracted to reflect the elongation and compression of the target. Analyze the direction of the feature vectors to extract the main directional features of the target; By combining higher-order information from the magnetic field gradient tensor, the curvature features of the target are extracted to describe its shape characteristics.

5. The real-time video positioning method based on three-dimensional imaging simulation of ferromagnetic material magnetic field signals according to claim 4, characterized in that, The process of estimating optical flow in video frames, compensating for the magnetic field measurement delay caused by target motion, and establishing a parametric perspective transformation model from the image coordinate system to the magnetic field coordinate system specifically includes: Based on the local consistency assumption, the motion vectors of all pixels are the same within a neighborhood of an image. The corner detection algorithm is used to detect feature points with gradients in the image, and a neighborhood window is selected for each feature point. Within the window, an optical flow equation is established using the gray-level conservation assumption, ensuring that the gray-level values ​​of pixels within the window remain constant during movement. The motion vectors of the feature points are obtained by solving the optical flow equation using the least squares method. Based on the optical flow estimation results, predict the target's trajectory during the magnetic field measurement delay; By combining the predicted motion trajectory and the measurement data from the magnetic field sensor, the magnetic field measurement value is corrected, and the magnetic field measurement data is aligned with the timestamp of the video frame to determine the delay time. Based on the predicted motion trajectory, the magnetic field measurement values ​​are mapped to the target's actual position at the current moment; The compensated magnetic field measurement data is fused with data from other sensors; Using the collected corresponding point data, for each set of corresponding points, two equations are established based on the parametric perspective transformation model, and a system of equations is established to solve for the parameters of the parametric perspective transformation model.

6. The real-time video positioning method based on three-dimensional imaging simulation of ferromagnetic material magnetic field signals according to claim 5, characterized in that, The establishment of the correlation matrix between the target state vector and the measured values, and the parallel estimation of the target's uniform velocity, uniform acceleration, and random motion modes specifically include: Based on the estimation and uncertainty of the target's initial state, an initial state vector and covariance matrix are defined; For each motion mode, a state transition model is defined to describe the change of the target state over time. Define a uniform motion model where acceleration is zero, and state transitions involve only changes in position and velocity. Define a uniformly accelerated motion model where acceleration is constant, and state transitions involve changes in position, velocity, and acceleration; Define a random motion model where acceleration is a random process and the state transition needs to take into account the influence of random noise. Assign an initial probability to each motion pattern and predict the target state at the current moment based on its state transition model; For each motion pattern, the predicted state is updated by combining the measurement value at the current moment. The state vector and covariance matrix are adjusted by calculating the residual between the measurement value and the predicted state. The likelihood value is calculated based on the residual, and the probability of the motion pattern is updated. Based on the probabilities of each motion mode, their estimation results are weighted and averaged, and the fused state estimate is used as the final estimate for the current moment.

7. The real-time video positioning method based on three-dimensional imaging simulation of ferromagnetic material magnetic field signals according to claim 6, characterized in that, The method of utilizing the mapping relationship between magnetic field signals and spatial positions, combined with the characteristics of real-time acquired magnetic field signals, and reconstructing the three-dimensional spatial distribution of ferromagnetic materials through an inversion method based on a perspective transformation model to achieve three-dimensional imaging specifically includes: Based on the projected magnetic field signal characteristics, an initial three-dimensional distribution model is generated in the target coordinate system. The magnetic field signal characteristics include gradient extrema and intensity peaks. The target area is divided into a regular three-dimensional grid, and each grid is assigned a value based on the magnetic field signal strength. The central framework of ferromagnetic materials is extracted through morphological manipulation, and then expanded outward to generate a complete distribution. The initial distribution is cropped and corrected based on the geometric boundary of the target area, and the edge contour of the distribution is optimized by utilizing the gradient features of the magnetic field signal at the boundary. Weights are assigned based on the signal-to-noise ratio of each sensor data, and the distribution results are weighted and superimposed. For regions with conflicting multi-view distributions, a confidence-weighted approach is used to determine the final value. By combining other physical field signals of the gravitational and electric fields, the robustness of the inversion method can be improved. A joint forward model of gravity, electric field, and magnetic field is constructed to describe the comprehensive response of the target material to at least two field signals.

Images