Real-time video positioning method based on ferromagnetic substance magnetic field signal three-dimensional imaging simulation

Abstract

The invention discloses a real-time video positioning method based on ferromagnetic substance magnetic field signal three-dimensional imaging simulation, and relates to the field of ferromagnetic substance detection in a medical scene nuclear magnetic resonance room, and the method comprises the steps: carrying out the segmentation fitting of a magnetic field gradient through employing a B-spline basis function; performing omni-directional calibration on the array through a rotating platform, and establishing a nonlinear mapping matrix of sensor output and real magnetic field intensity; extracting target shape features; performing optical flow estimation on the video frame, and compensating magnetic field measurement delay caused by target motion; establishing an incidence matrix of the target state vector and the measured value; reconstructing the three-dimensional space distribution of the ferromagnetic substance through an inversion method based on a perspective transformation model; and fusing the three-dimensional imaging result with the real-time video. Magnetic field measurement delay caused by target motion is compensated by performing optical flow estimation on a video frame, and a state estimation convergence speed is improved by estimating a multi-mode parallel state.

Description

Technical Field

[0001] The present invention relates to the field of ferromagnetic material detection in nuclear magnetic resonance rooms in medical scenarios, and in particular to a real-time video positioning method based on three-dimensional imaging simulation of magnetic field signals of ferromagnetic materials. Background Art

[0002] In scenarios such as aerospace and energy equipment, such as nuclear power plant pipelines and wind turbines, real-time monitoring of cracks, corrosion, or foreign object intrusion in ferromagnetic components such as welds, bolts, and pipes is required. In urban underground pipelines such as metal pipes and cables, archaeological exploration such as iron artifacts in ancient tombs, or military counter-terrorism such as buried explosives, it is necessary to quickly locate ferromagnetic targets and evaluate their spatial distribution. In complex industrial environments such as mines and nuclear power plants, robots need to identify ferromagnetic obstacles such as fallen tools and metal fragments and plan paths. In magnetic nanoparticle targeted therapy, the distribution and concentration of ferromagnetic particles in the body need to be monitored in real time.

[0003] Existing medical MRI ferromagnetic material detection equipment can only roughly determine the approximate direction of the ferromagnetic material by measuring the magnetic field signal strength through multiple fluxgate sensors and the direction of the sensor with the stronger signal strength. The usual practice is to disperse three fluxgate sensors on both sides of the MRI room door to roughly determine the direction of the object being measured: upper left, left center, lower left, upper right, right center, and lower right. If the target is between two sensors, multiple directions will be determined simultaneously, and the specific direction cannot be given. Moreover, fluxgate sensors generally provide analog signals or convert them into digital signals. However, regardless of whether they are analog or digital, the position, distance, and angle of the object being measured are still abstract. Therefore, all such products currently on the market can only roughly provide a few positions according to the amplitude of the signal change measured by the sensor, that is, "upper left, left center, lower left, upper right, right center, and lower right." Although equipped with a display screen, lights, and voice prompts, strictly speaking, they are still two-dimensional and have fixed segmented areas. They have poor anti-interference capabilities and cannot distinguish the source of the target object's signal. If there is an object with a large volume and magnetic field signal outside the target area, it cannot be distinguished because the waveforms or digital signals collected by the sensor are all in the same form. Summary of the Invention

[0004] In order to solve the above technical problems, a real-time video positioning method based on three-dimensional imaging simulation of magnetic field signals of ferromagnetic materials is provided. This technical solution solves the problems raised in the above background technology.

[0005] In order to achieve the above objects, the technical solution adopted by the present invention is:

[0006] A real-time video positioning method based on three-dimensional imaging simulation of magnetic field signals of ferromagnetic materials includes:

[0007] Based on the target's motion speed, acceleration range, and intensity of magnetic field gradient changes, the order of the B-spline is determined, and the magnetic field gradient is piecewise fitted using B-spline basis functions to achieve real-time deformation of the grid cells along the target's motion trajectory.

[0008] The array is omnidirectionally calibrated by a rotating platform to establish a nonlinear mapping matrix between the sensor output and the actual magnetic field strength;

[0009] Decompose the magnetic field gradient tensor into the isotropic part and the deviatoric tensor part to extract the target shape features;

[0010] Perform optical flow estimation on video frames, compensate for the magnetic field measurement delay caused by target motion, and establish a parametric perspective transformation model from the image coordinate system to the magnetic field coordinate system;

[0011] Establishing a correlation matrix between a target state vector and measurement values, and performing parallel estimation of uniform velocity, uniform acceleration, and random motion modes of the target, wherein the target state vector includes position, velocity, and magnetic moment vector;

[0012] By using the mapping relationship between magnetic field signals and spatial positions, combined with the real-time acquired magnetic field signal characteristics, the three-dimensional spatial distribution of ferromagnetic materials is reconstructed through an inversion method based on a perspective transformation model to achieve three-dimensional imaging.

[0013] The three-dimensional imaging results are integrated with the real-time video, and the position of the ferromagnetic material is marked in real time in the video according to the magnetic field positioning information.

[0014] Preferably, performing optical flow estimation on the video frames, compensating for the magnetic field measurement delay caused by target motion, and establishing a parametric perspective transformation model from the image coordinate system to the magnetic field coordinate system specifically include:

[0015] Based on the local consistency assumption, the motion vectors of all pixels in a neighborhood of an image are the same;

[0016] Use the corner detection algorithm to detect feature points with gradients in the image, and select a neighborhood window for each feature point;

[0017] Within the window, the optical flow equation is established using the grayscale conservation assumption, and the grayscale value of the pixels within the window remains unchanged during the movement;

[0018] Solve the optical flow equation by the least square method to obtain the motion vector of the feature point;

[0019] Based on the optical flow estimation results, the target's motion trajectory during the magnetic field measurement delay is predicted;

[0020] Combining the predicted motion trajectory with the measurement data from the magnetic field sensor, the magnetic field measurement value is corrected, and the magnetic field measurement data is aligned with the timestamp of the video frame to determine the delay time;

[0021] Based on the predicted motion trajectory, the magnetic field measurement value is mapped to the actual position of the target at the current moment;

[0022] Fusing the compensated magnetic field measurement data with other sensor data;

[0023] Using the collected corresponding point data, for each set of corresponding points, two equations are established according to the parametric perspective transformation model, and a set of equations is established to solve the parameters of the parametric perspective transformation model.

[0024] Preferably, the establishing of the correlation matrix between the target state vector and the measurement value and the parallel estimation of the uniform velocity, uniform acceleration and random motion modes of the target specifically include:

[0025] Based on the estimation and uncertainty of the target initial state, the initial state vector and covariance matrix are set;

[0026] For each motion mode, define a state transition model to describe the change of the target state over time;

[0027] Define a uniform motion model where the acceleration is zero and the state transition only involves changes in position and velocity;

[0028] Define a uniform acceleration motion model where the acceleration is a constant and the state transition involves changes in position, velocity, and acceleration;

[0029] Define a random motion model, where acceleration is a random process and state transitions must take into account the influence of random noise;

[0030] Assign an initial probability to each motion mode and predict the target state at the current moment based on its state transition model;

[0031] For each motion mode, the predicted state is updated based on the current measurement value by calculating the residual between the measurement value and the predicted state and adjusting the state vector and covariance matrix.

[0032] Calculate the likelihood value based on the residual and update the probability of the motion mode;

[0033] Based on the probability of each motion mode, their estimation results are weighted averaged, and the fused state estimation is used as the final estimation result at the current moment.

[0034] Preferably, the three-dimensional imaging is achieved by utilizing the mapping relationship between the magnetic field signal and the spatial position, combining the magnetic field signal characteristics collected in real time, and reconstructing the three-dimensional spatial distribution of the ferromagnetic material through an inversion method based on a perspective transformation model. Specifically, the method includes:

[0035] generating an initial three-dimensional distribution model in a target coordinate system based on the projected magnetic field signal characteristics, wherein the magnetic field signal characteristics include gradient extreme points and intensity peaks;

[0036] Divide the target area into regular three-dimensional grids and assign values ​​to each grid based on the magnetic field signal strength;

[0037] The central skeleton of the ferromagnetic material is extracted through morphological operations and expanded outward to generate the complete distribution;

[0038] The initial distribution is cropped and corrected based on the geometric boundaries of the target area, and the edge contour of the distribution is optimized using the gradient characteristics of the magnetic field signal at the boundary.

[0039] Assign weights based on the signal-to-noise ratio of each sensor data and perform weighted superposition on the distribution results;

[0040] For areas with conflicts in multi-view distribution, confidence weighting is used to determine the final value;

[0041] Integrate other physical field signals such as gravity and electric fields to improve the robustness of the inversion method;

[0042] A gravity-electric field-magnetic field joint forward model is constructed to describe the comprehensive response of the target material to at least two field signals.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] Based on the target speed, acceleration range, and the intensity of the magnetic field gradient change, the B-spline order is adaptively adjusted, and the magnetic field gradient is piecewise fitted to achieve real-time deformation of the grid cells along the target trajectory. The sensor array is omnidirectionally calibrated using a rotating platform, and a nonlinear mapping matrix between the sensor output and the actual magnetic field intensity is established to eliminate environmental interference and individual differences. Optical flow estimation is performed on video frames to compensate for the magnetic field measurement delay caused by target motion. In random motion scenarios, uniform speed, uniform acceleration, and random motion modes are estimated in parallel, and the weights are dynamically adjusted to improve the convergence speed of state estimation. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is a flow chart of the real-time video positioning method based on three-dimensional imaging simulation of magnetic field signals of ferromagnetic materials of the present invention;

[0046] Figure 2 This is a flow chart of the segmented fitting method for magnetic field gradient of the present invention;

[0047] Figure 3 A flow chart of the method for establishing a nonlinear mapping matrix between sensor output and true magnetic field strength according to the present invention;

[0048] Figure 4This is a flow chart of the method for extracting target shape features of the present invention;

[0049] Figure 5 This is a flow chart of the method for estimating optical flow of a video frame according to the present invention;

[0050] Figure 6 This is a flow chart of a method for parallel estimation of uniform velocity, uniform acceleration and random motion patterns of a target according to the present invention;

[0051] Figure 7 This is a flow chart of the method for reconstructing the three-dimensional spatial distribution of ferromagnetic materials according to the present invention. DETAILED DESCRIPTION

[0052] The following description is intended to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are merely examples, and those skilled in the art may conceive of other obvious variations.

[0053] Reference Figure 1 As shown, the real-time video positioning method based on three-dimensional imaging simulation of magnetic field signals of ferromagnetic materials includes:

[0054] Based on the target's motion speed, acceleration range, and intensity of magnetic field gradient changes, the order of the B-spline is determined, and the magnetic field gradient is piecewise fitted using B-spline basis functions to achieve real-time deformation of the grid cells along the target's motion trajectory.

[0055] The array is omnidirectionally calibrated by a rotating platform to establish a nonlinear mapping matrix between the sensor output and the actual magnetic field strength;

[0056] Decompose the magnetic field gradient tensor into the isotropic part and the deviatoric tensor part to extract the target shape features;

[0057] Perform optical flow estimation on video frames, compensate for the magnetic field measurement delay caused by target motion, and establish a parametric perspective transformation model from the image coordinate system to the magnetic field coordinate system;

[0058] Establishing a correlation matrix between a target state vector and measurement values, and performing parallel estimation of uniform velocity, uniform acceleration, and random motion modes of the target, wherein the target state vector includes position, velocity, and magnetic moment vector;

[0059] By using the mapping relationship between magnetic field signals and spatial positions, combined with the real-time acquired magnetic field signal characteristics, the three-dimensional spatial distribution of ferromagnetic materials is reconstructed through an inversion method based on a perspective transformation model to achieve three-dimensional imaging.

[0060] The three-dimensional imaging results are integrated with the real-time video, and the position of the ferromagnetic material is marked in real time in the video according to the magnetic field positioning information.

[0061] Reference Figure 2As shown in the figure, based on the target's motion speed, acceleration range, and intensity of the magnetic field gradient change, the order of the B-spline is determined, and the magnetic field gradient is piecewise fitted using the B-spline basis function to achieve real-time deformation of the grid unit along the target's motion trajectory. Specifically, the following steps are involved:

[0062] The motion sensor is installed on the target object to collect the speed and acceleration data of the target in real time during the movement process, and the collected motion data is filtered to remove noise interference;

[0063] Arrange a magnetic field sensor array around the target motion area to collect magnetic field gradient data in real time, calibrate and normalize the collected magnetic field gradient data to eliminate the sensor's own errors;

[0064] Count the maximum speed, minimum speed, average speed and speed change rate parameters of the target during movement, draw a speed-time curve, and show the change of target speed;

[0065] Calculate the maximum acceleration, minimum acceleration, average acceleration, and acceleration rate of change during target motion, and analyze the fluctuation range and change trend of acceleration;

[0066] Based on the changes in speed and acceleration, the target motion state is divided into uniform motion stage, accelerated motion stage and decelerated motion stage;

[0067] Calculate the mean, variance, maximum and minimum statistics of the magnetic field gradient to evaluate the overall variation of the magnetic field gradient;

[0068] Draw the curve of magnetic field gradient changing with time and space, analyze the changing trend and local characteristics of magnetic field gradient, and observe whether there are sudden changes or periodic changes in magnetic field gradient;

[0069] When the target's motion trajectory is relatively smooth and the magnetic field gradient changes slowly, the quadratic B-spline is selected for fitting;

[0070] When the target's motion trajectory is complex and the magnetic field gradient changes dramatically, the quartic B-spline is selected for fitting;

[0071] Based on the determined B-spline order and knot vector, the B-spline basis function in each fitting interval is calculated;

[0072] Minimize the sum of square errors between the fitting curve and the actual magnetic field gradient data, and obtain the fitting coefficients by solving the linear equations;

[0073] The obtained fitting coefficient is multiplied by the corresponding B-spline basis function and summed to obtain the magnetic field gradient fitting curve in each fitting interval. The fitting curves of each interval are connected to obtain the segmented fitting result of the entire magnetic field gradient.

[0074] The mean, maximum, and minimum values ​​of the magnetic field gradient were calculated, and the ratio of the difference between the maximum and minimum values ​​to the mean was recorded as the overall change of the magnetic field gradient. When the overall change of the magnetic field gradient was less than 0.3 and the motion state was uniform, the quadratic B-spline was selected for fitting. When the overall change of the magnetic field gradient was greater than or equal to 0.3 and the motion state was acceleration or deceleration, the quadratic B-spline was selected for fitting.

[0075] Reference Figure 3 As shown, the array is omnidirectionally calibrated by rotating the platform, and the nonlinear mapping matrix between the sensor output and the actual magnetic field strength is established, which specifically includes:

[0076] S101: Fix the sensor array on the rotating platform and set the initial rotation angle to 0 degrees;

[0077] S102: Start the rotating platform and rotate it according to a predetermined 15-degree angle step;

[0078] S103: At each rotation angle, wait for the sensor output to stabilize and collect output data of the sensor array;

[0079] S104: Determine whether the rotating platform has completed a full 360-degree rotation. If so, no output is made; if not, return to step S102;

[0080] S105: selecting a nonlinear function form according to the characteristics and application scenario of the sensor, wherein the nonlinear function form includes a polynomial model, an exponential model, and a neural network model;

[0081] S106: The mapping functions of the sensors are combined into a matrix form. For M sensors and N data points, an M×N mapping matrix is ​​constructed, where the elements represent the mapping values ​​of each sensor on each data point.

[0082] The sensor space is distributed in the three-dimensional space of the door area of ​​the MRI room. Facing the door from left to right is the X-axis width of 2 meters, the vertical direction from the ground is the Y-axis height of 2 meters, and the door frame is outwardly deep to the Z-axis of 1.5 meters. There are 2 groups of X-axis sensors, 3 groups of Y-axis magnetic sensors, and 3 groups of Z-axis magnetic sensors (the number of magnetic sensors is variable. The more hardware devices there are, the higher the physical resolution and the resolution of the collected data). According to the X, Y, and Z coordinates, a group of data is collected at one coordinate point every 10 cubic centimeters (each group of data is composed of data measured by 6 to 9 sensors), totaling 6,000 groups of data. Based on this, a three-dimensional simulation data visual model is established. The simulation visual model is divided into two states: relative static and relative static. The static state model and the dynamic model are as follows: when no target enters the detection range, the data vision model is relatively stable and static; when a ferromagnetic target enters, the data model measures the data change. According to the algorithm, the XYZ coordinates of the changing data are obtained, and the two-dimensional and three-dimensional graphics are simulated by vision, and displayed with red marks from dark to light. At each rotation angle, wait for more than 5 seconds for the sensor output to stabilize, and synchronously collect the three-axis output of all sensors with a sampling rate greater than 10Hz. The average value of 10 measurements is taken. After the rotating platform completes a 360° rotation, check the data integrity. When selecting the model, the polynomial model is suitable for scenes with a smooth nonlinear relationship between the magnetic field intensity and the sensor output, and the exponential model is suitable for scenes with obvious magnetic saturation effects.

[0083] Reference Figure 4 As shown in the figure, the magnetic field gradient tensor is decomposed into an isotropic part and a deviatoric tensor part, and the target shape features are extracted specifically including:

[0084] The partial tensor is recorded as a quadratic representation of an ellipsoid, and the shape features of the target are extracted by analyzing the shape of the ellipsoid;

[0085] Calculate the eigenvalue of the bias tensor, analyze its magnitude and sign, and obtain the target's magnetic field gradient variation characteristics in all directions;

[0086] By fitting the ellipsoid, the ratio of the major and minor axes is extracted to reflect the degree of elongation and compression of the target;

[0087] Analyze the direction of the feature vector and extract the main directional features of the target;

[0088] Combined with the high-order information of the magnetic field gradient tensor, the curvature characteristics of the target are extracted to describe the target shape characteristics.

[0089] The ellipsoid's major-minor axis ratio, main direction, and curvature are analyzed, and geometric features such as the major-minor axis ratio and main direction are mapped to the actual shape of the target, such as a rod, a sheet, or a sphere. The shape features are superimposed on the real-time video screen, and the target position and geometric properties are marked. The isotropic part reflects the uniformly changing part of the magnetic field gradient tensor that is not related to the target shape, while the bias tensor part contains information related to the target shape. Through this decomposition, the influence of the target shape on the magnetic field gradient can be highlighted, laying the foundation for the subsequent extraction of target shape features.

[0090] Reference Figure 5 As shown in the figure, the optical flow of the video frame is estimated, the magnetic field measurement delay caused by the target motion is compensated, and a parametric perspective transformation model from the image coordinate system to the magnetic field coordinate system is established. Specifically, the following steps are involved:

[0091] Based on the local consistency assumption, the motion vectors of all pixels in a neighborhood of an image are the same;

[0092] Use the corner detection algorithm to detect feature points with gradients in the image, and select a neighborhood window for each feature point;

[0093] Within the window, the optical flow equation is established using the grayscale conservation assumption, and the grayscale value of the pixels within the window remains unchanged during the movement;

[0094] Solve the optical flow equation by the least square method to obtain the motion vector of the feature point;

[0095] Based on the optical flow estimation results, the target's motion trajectory during the magnetic field measurement delay is predicted;

[0096] Combining the predicted motion trajectory with the measurement data from the magnetic field sensor, the magnetic field measurement value is corrected, and the magnetic field measurement data is aligned with the timestamp of the video frame to determine the delay time;

[0097] Based on the predicted motion trajectory, the magnetic field measurement value is mapped to the actual position of the target at the current moment;

[0098] Fusing the compensated magnetic field measurement data with other sensor data;

[0099] Using the collected corresponding point data, for each set of corresponding points, two equations are established according to the parametric perspective transformation model, and a set of equations is established to solve the parameters of the parametric perspective transformation model.

[0100] The optical flow equation is:

[0101]

[0102] Where Gray is the gray value function of the pixel in the window, x is the horizontal coordinate of the pixel, y is the vertical coordinate of the pixel, and t is the time parameter;

[0103] Optical flow refers to the temporal motion of each pixel in an image. The goal of optical flow estimation is to calculate the motion vector of each pixel between two frames based on the image information between consecutive frames. Optical flow estimation can be represented in two ways: dense optical flow and sparse optical flow. Dense optical flow means that the optical flow vector is calculated at each pixel point in the image, while sparse optical flow only selects some pixels to calculate the optical flow vector. The basic assumption of optical flow estimation is that the light intensity is constant, that is, the light intensity will not change in the area around a pixel point. Based on this assumption, we can infer the motion information of the object through the change of pixel values.

[0104] Reference Figure 6 As shown, the establishment of the correlation matrix between the target state vector and the measurement value and the parallel estimation of the uniform speed, uniform acceleration and random motion modes of the target specifically include:

[0105] Based on the estimation and uncertainty of the target initial state, the initial state vector and covariance matrix are set;

[0106] For each motion mode, define a state transition model to describe the change of the target state over time;

[0107] Define a uniform motion model where the acceleration is zero and the state transition only involves changes in position and velocity;

[0108] Define a uniform acceleration motion model where the acceleration is a constant and the state transition involves changes in position, velocity, and acceleration;

[0109] Define a random motion model, where acceleration is a random process and state transitions must take into account the influence of random noise;

[0110] Assign an initial probability to each motion mode and predict the target state at the current moment based on its state transition model;

[0111] For each motion mode, the predicted state is updated based on the current measurement value by calculating the residual between the measurement value and the predicted state and adjusting the state vector and covariance matrix.

[0112] Calculate the likelihood value based on the residual and update the probability of the motion mode;

[0113] Based on the probability of each motion mode, their estimation results are weighted averaged, and the fused state estimation is used as the final estimation result at the current moment.

[0114] The time step is set according to the sensor sampling rate, which must be much smaller than the target motion characteristic time. The fused position, velocity, acceleration and magnetic moment vector are output for 3D imaging and video annotation, and the probability of each motion mode is displayed in real time to assist in judging the target motion characteristics. By estimating the three motion modes of uniform speed, uniform acceleration and random in parallel, it can adapt to the different motion states that may appear in the target at different times. When the actual motion mode of the target is close to a certain hypothetical model, the estimation result corresponding to the model will be more accurate. The weighted average can comprehensively utilize the advantages of each model to reduce the error caused by single model estimation, thereby improving the accuracy of target state estimation.

[0115] Reference Figure 7 As shown in the figure, the mapping relationship between magnetic field signals and spatial positions is used, combined with the real-time acquired magnetic field signal characteristics, and the three-dimensional spatial distribution of ferromagnetic materials is reconstructed through an inversion method based on a perspective transformation model. The three-dimensional imaging is specifically implemented as follows:

[0116] generating an initial three-dimensional distribution model in a target coordinate system based on the projected magnetic field signal characteristics, wherein the magnetic field signal characteristics include gradient extreme points and intensity peaks;

[0117] Divide the target area into regular three-dimensional grids and assign values ​​to each grid based on the magnetic field signal strength;

[0118] The central skeleton of the ferromagnetic material is extracted through morphological operations and expanded outward to generate the complete distribution;

[0119] The initial distribution is cropped and corrected based on the geometric boundaries of the target area, and the edge contour of the distribution is optimized using the gradient characteristics of the magnetic field signal at the boundary.

[0120] Assign weights based on the signal-to-noise ratio of each sensor data and perform weighted superposition on the distribution results;

[0121] For areas with conflicts in multi-view distribution, confidence weighting is used to determine the final value;

[0122] Integrate other physical field signals such as gravity and electric fields to improve the robustness of the inversion method;

[0123] A gravity-electric field-magnetic field joint forward model is constructed to describe the comprehensive response of the target material to at least two field signals.

[0124] The gravity-electric field-magnetic field joint forward model is:

[0125]

[0126] Where Δg(r) is the gravity forward model, G is the universal gravitational constant, ρ(r') is the density of the material at the source position, φ(r) is the electric field forward model, ε is the dielectric constant, σ(r') is the conductivity at the source position, r', r, are the source position, current position and theoretical position respectively, B(r) is the magnetic field forward model, μ0 is the vacuum permeability, and M(r') is the magnetization intensity at the source position;

[0127] The software algorithm fits the magnetic sensor simulation imaging and the camera real-time image, overlaps the coordinates of the two images, and displays the disturbed red mark of the magnetic field simulation graph on the coordinates of the real-time image. The camera real-time image has the target person distance, motion trajectory, and human torso visual recognition functions. The coordinates of the red mark of the magnetic field disturbance and the camera assist each other in judgment, so that it can accurately judge and mark invisible items carried by the human body, such as mobile phones, coins, keys, small folding knives, lighters, etc., thereby improving the accuracy, performance and reliability of the inspection.

[0128] Furthermore, the present solution also proposes a computer-readable storage medium on which a computer-readable program is stored. When the computer-readable program is called, the real-time video positioning method based on three-dimensional imaging simulation of magnetic field signals of ferromagnetic materials is executed.

[0129] It is understandable that the storage medium may be a magnetic medium, such as a floppy disk, a hard disk, or a magnetic tape; an optical medium, such as a DVD; or a semiconductor medium, such as a solid state disk (SSD).

[0130] In summary, the advantages of the present invention are: based on the target speed, acceleration range and the intensity of the magnetic field gradient change, the B-spline order is adaptively adjusted, the magnetic field gradient is segmentedly fitted, the grid unit is deformed in real time with the target trajectory, the sensor array is omnidirectionally calibrated by a rotating platform, a nonlinear mapping matrix between the sensor output and the actual magnetic field intensity is established, environmental interference and individual differences are eliminated, optical flow estimation is performed on video frames, and the magnetic field measurement delay caused by target motion is compensated. In random motion scenarios, uniform speed, uniform acceleration and random motion modes are estimated in parallel, the weights are dynamically adjusted, and the state estimation convergence speed is improved.

[0131] The above shows and describes the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions merely illustrate the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.

Claims

1. A real-time video positioning method based on three-dimensional imaging simulation of magnetic field signals of ferromagnetic materials, characterized in that: include: Based on the target's motion speed, acceleration range, and intensity of magnetic field gradient changes, the order of the B-spline is determined, and the magnetic field gradient is piecewise fitted using B-spline basis functions to achieve real-time deformation of the grid cells along the target's motion trajectory. The array is omnidirectionally calibrated by a rotating platform to establish a nonlinear mapping matrix between the sensor output and the actual magnetic field strength; Decompose the magnetic field gradient tensor into the isotropic part and the deviatoric tensor part to extract the target shape features; Perform optical flow estimation on video frames, compensate for the magnetic field measurement delay caused by target motion, and establish a parametric perspective transformation model from the image coordinate system to the magnetic field coordinate system; Establishing a correlation matrix between a target state vector and measurement values, and performing parallel estimation of uniform velocity, uniform acceleration, and random motion modes of the target, wherein the target state vector includes position, velocity, and magnetic moment vector; By using the mapping relationship between magnetic field signals and spatial positions, combined with the real-time acquired magnetic field signal characteristics, the three-dimensional spatial distribution of ferromagnetic materials is reconstructed through an inversion method based on a perspective transformation model to achieve three-dimensional imaging. The three-dimensional imaging results are integrated with the real-time video, and the position of the ferromagnetic material is marked in real time in the video according to the magnetic field positioning information.

2. The real-time video positioning method based on three-dimensional imaging simulation of magnetic field signals of ferromagnetic materials according to claim 1 is characterized in that: The method of determining the order of the B-spline based on the target's motion speed, acceleration range, and intensity of the magnetic field gradient change, and using the B-spline basis function to perform piecewise fitting on the magnetic field gradient to achieve real-time deformation of the grid unit along the target's motion trajectory specifically includes: The motion sensor is installed on the target object to collect the speed and acceleration data of the target in real time during the movement process, and the collected motion data is filtered to remove noise interference; Arrange a magnetic field sensor array around the target motion area to collect magnetic field gradient data in real time, calibrate and normalize the collected magnetic field gradient data to eliminate the sensor's own errors; Count the maximum speed, minimum speed, average speed and speed change rate parameters of the target during movement, draw a speed-time curve, and show the change of target speed; Calculate the maximum acceleration, minimum acceleration, average acceleration, and acceleration rate of change during target motion, and analyze the fluctuation range and change trend of acceleration; Based on the changes in speed and acceleration, the target motion state is divided into uniform motion stage, accelerated motion stage and decelerated motion stage; Calculate the mean, variance, maximum and minimum statistics of the magnetic field gradient to evaluate the overall variation of the magnetic field gradient; Draw the curve of magnetic field gradient changing with time and space, analyze the changing trend and local characteristics of magnetic field gradient, and observe whether there are sudden changes or periodic changes in magnetic field gradient; When the target's motion trajectory is relatively smooth and the magnetic field gradient changes slowly, the quadratic B-spline is selected for fitting; When the target's motion trajectory is complex and the magnetic field gradient changes dramatically, the quartic B-spline is selected for fitting; Based on the determined B-spline order and knot vector, the B-spline basis function in each fitting interval is calculated; Minimize the sum of square errors between the fitting curve and the actual magnetic field gradient data, and obtain the fitting coefficients by solving the linear equations; The obtained fitting coefficient is multiplied by the corresponding B-spline basis function and summed to obtain the magnetic field gradient fitting curve in each fitting interval. The fitting curves of each interval are connected to obtain the segmented fitting result of the entire magnetic field gradient.

3. The real-time video positioning method based on three-dimensional imaging simulation of ferromagnetic material magnetic field signals according to claim 2 is characterized in that: The omnidirectional calibration of the array by the rotating platform and the establishment of a nonlinear mapping matrix between the sensor output and the actual magnetic field strength specifically include: S101: Fix the sensor array on the rotating platform and set the initial rotation angle to 0 degrees; S102: Start the rotating platform and rotate it according to a predetermined 15-degree angle step; S103: At each rotation angle, wait for the sensor output to stabilize and collect output data of the sensor array; S104: Determine whether the rotating platform has completed a full 360-degree rotation. If so, no output is made; if not, return to step S102; S105: selecting a nonlinear function form according to the characteristics and application scenario of the sensor, wherein the nonlinear function form includes a polynomial model, an exponential model, and a neural network model; S106: The mapping functions of the sensors are combined into a matrix form. For M sensors and N data points, an M×N mapping matrix is ​​constructed, where the elements represent the mapping values ​​of each sensor on each data point.

4. The real-time video positioning method based on three-dimensional imaging simulation of magnetic field signals of ferromagnetic materials according to claim 3 is characterized in that: Decomposing the magnetic field gradient tensor into an isotropic part and a deviatoric tensor part and extracting target shape features specifically includes: The partial tensor is recorded as a quadratic representation of an ellipsoid, and the shape features of the target are extracted by analyzing the shape of the ellipsoid; Calculate the eigenvalue of the bias tensor, analyze its magnitude and sign, and obtain the target's magnetic field gradient variation characteristics in all directions; By fitting the ellipsoid, the ratio of the major and minor axes is extracted to reflect the degree of elongation and compression of the target; Analyze the direction of the feature vector and extract the main directional features of the target; Combined with the high-order information of the magnetic field gradient tensor, the curvature characteristics of the target are extracted to describe the target shape characteristics.

5. The real-time video positioning method based on three-dimensional imaging simulation of magnetic field signals of ferromagnetic materials according to claim 4 is characterized in that: The optical flow estimation of the video frame, compensating for the magnetic field measurement delay caused by the target motion, and establishing a parametric perspective transformation model from the image coordinate system to the magnetic field coordinate system specifically include: Based on the local consistency assumption, the motion vectors of all pixels in a neighborhood of an image are the same; Use the corner detection algorithm to detect feature points with gradients in the image, and select a neighborhood window for each feature point; Within the window, the optical flow equation is established using the grayscale conservation assumption, and the grayscale value of the pixels within the window remains unchanged during the movement; Solve the optical flow equation by the least square method to obtain the motion vector of the feature point; Based on the optical flow estimation results, the target's motion trajectory during the magnetic field measurement delay is predicted; Combining the predicted motion trajectory with the measurement data from the magnetic field sensor, the magnetic field measurement value is corrected, and the magnetic field measurement data is aligned with the timestamp of the video frame to determine the delay time; Based on the predicted motion trajectory, the magnetic field measurement value is mapped to the actual position of the target at the current moment; Fusing the compensated magnetic field measurement data with other sensor data; Using the collected corresponding point data, for each set of corresponding points, two equations are established according to the parametric perspective transformation model, and a set of equations is established to solve the parameters of the parametric perspective transformation model.

6. The real-time video positioning method based on three-dimensional imaging simulation of magnetic field signals of ferromagnetic materials according to claim 5 is characterized in that: The establishment of the correlation matrix between the target state vector and the measurement value and the parallel estimation of the uniform velocity, uniform acceleration and random motion modes of the target specifically include: Based on the estimation and uncertainty of the target initial state, the initial state vector and covariance matrix are set; For each motion mode, define a state transition model to describe the change of the target state over time; Define a uniform motion model where the acceleration is zero and the state transition only involves changes in position and velocity; Define a uniform acceleration motion model where the acceleration is a constant and the state transition involves changes in position, velocity, and acceleration; Define a random motion model, where acceleration is a random process and state transitions must take into account the influence of random noise; Assign an initial probability to each motion mode and predict the target state at the current moment based on its state transition model; For each motion mode, the predicted state is updated based on the current measurement value by calculating the residual between the measurement value and the predicted state and adjusting the state vector and covariance matrix. Calculate the likelihood value based on the residual and update the probability of the motion mode; Based on the probability of each motion mode, their estimation results are weighted averaged, and the fused state estimation is used as the final estimation result at the current moment.

7. The real-time video positioning method based on three-dimensional imaging simulation of magnetic field signals of ferromagnetic materials according to claim 6 is characterized in that: The three-dimensional imaging is achieved by utilizing the mapping relationship between magnetic field signals and spatial positions, combining the magnetic field signal characteristics collected in real time, and reconstructing the three-dimensional spatial distribution of ferromagnetic materials through an inversion method based on a perspective transformation model. Specifically, the following steps are performed: generating an initial three-dimensional distribution model in a target coordinate system based on the projected magnetic field signal characteristics, wherein the magnetic field signal characteristics include gradient extreme points and intensity peaks; Divide the target area into regular three-dimensional grids and assign values ​​to each grid based on the magnetic field signal strength; The central skeleton of the ferromagnetic material is extracted through morphological operations and expanded outward to generate the complete distribution; The initial distribution is cropped and corrected based on the geometric boundaries of the target area, and the edge contour of the distribution is optimized using the gradient characteristics of the magnetic field signal at the boundary. Assign weights based on the signal-to-noise ratio of each sensor data and perform weighted superposition on the distribution results; For areas with conflicts in multi-view distribution, confidence weighting is used to determine the final value; Integrate other physical field signals such as gravity and electric fields to improve the robustness of the inversion method; A gravity-electric field-magnetic field joint forward model is constructed to describe the comprehensive response of the target material to at least two field signals.

Images