Three-dimensional tracking registration-based nuclear radiation dose augmented reality interactive visualization method and system

Abstract

The application discloses a kind of nuclear radiation dose augmented reality interaction visualization method, system, electronic equipment and storage medium based on three-dimensional tracking registration, method includes: in monitoring area, nuclear radiation value is monitored;According to nuclear radiation value, nuclear radiation value in future period is predicted;According to the nuclear radiation value of prediction interpolation generates nuclear pollution area radiation intensity video, and it is integrated in three-dimensional virtual environment, realizes three-dimensional scene fusion;According to the pose and position in real scene and the three-dimensional scene of fusion, realizes three-dimensional tracking registration;Based on the three-dimensional scene fusion of three-dimensional tracking registration, intelligent terminal realizes augmented reality interaction visualization.The method provided by the application, by obtaining the position and attitude information of emergency rescuer, the nuclear radiation intensity image based on the visual angle of rescuer is displayed by intelligent terminal, so that emergency rescuer can intuitively obtain global, three-dimensional nuclear radiation intensity information, to provide auxiliary decision support for emergency disposal.

Description

Technical Field

[0001] The present invention belongs to the technical field of safety emergency disposal, and in particular relates to a nuclear radiation dose augmented reality interactive visualization method, system, electronic device and storage medium based on three-dimensional tracking and registration. Background Art

[0002] As economic development continues to increase the demand for energy, nuclear energy, as a clean and economical energy source, has received more attention. Nuclear power has become the best choice for baseload power supply due to its extremely strong power supply stability. Nuclear power safety emergency decision support is an indispensable part of nuclear emergency plans, providing a scientific basis for guiding early protection and minimizing the consequences of radioactive contamination and nuclear hazards.

[0003] In nuclear contamination monitoring, nuclear radiation sensors are typically used to sample the air, measure its volumetric radioactivity, and evaluate the monitoring results. To monitor for unexpected air contamination, continuous sampling and alarm monitoring instruments can be used, automatically issuing an alarm signal when radiation levels exceed standards. Air sampling is a relatively convenient method for nuclear radiation monitoring, offering advantages such as wide coverage, high accuracy, and high sensitivity. Subsequently, based on the time series of on-site nuclear contamination values, effective predictions of future levels can be made, effectively ensuring that nuclear power plants and their personnel can take appropriate protective measures in a timely manner. Consequently, the current mainstream method for numerical monitoring of nuclear radiation contamination relies primarily on point-based monitoring, which is difficult to effectively monitor the entire radiation coverage area of ​​a nuclear power plant. Furthermore, data processing, data conversion, and data visualization cannot meet the real-time requirements of pollutant dispersion analysis. Summary of the Invention

[0004] To improve the emergency response capabilities of nuclear power plants, the present invention provides a method, system, electronic device, and storage medium for augmented reality interactive visualization of nuclear radiation dose based on three-dimensional tracking and registration technology. The method monitors nuclear radiation values ​​within a monitoring area; predicts nuclear radiation values ​​within a certain period of time based on the nuclear radiation values; interpolates and generates a radiation intensity video of the nuclear contaminated area based on the predicted nuclear radiation values, and integrates the radiation intensity video of the nuclear contaminated area into a three-dimensional virtual environment to achieve three-dimensional scene fusion; performs three-dimensional tracking and registration based on the acquired posture and position in the real scene and the fused three-dimensional scene; and based on the three-dimensional scene fusion of three-dimensional tracking and registration, an intelligent terminal implements augmented reality interactive visualization. By obtaining the position and posture information of emergency rescue personnel, the intelligent terminal generates thematic image videos, which facilitates emergency rescue personnel to intuitively obtain global and three-dimensional nuclear power plant nuclear radiation intensity information, which is used to guide emergency evacuation of personnel in nuclear radiation scenarios and provide auxiliary decision support for nuclear power plant emergency response.

[0005] The first object of the present invention is to provide a nuclear radiation dose augmented reality interactive visualization method based on three-dimensional tracking and registration technology.

[0006] The second object of the present invention is to provide a nuclear radiation dose augmented reality interactive visualization system based on three-dimensional tracking and registration technology.

[0007] A third object of the present invention is to provide an electronic device.

[0008] A fourth object of the present invention is to provide a storage medium.

[0009] The first object of the present invention can be achieved by adopting the following technical solutions:

[0010] A nuclear radiation dose augmented reality interactive visualization method based on three-dimensional tracking and registration, the method comprising:

[0011] Monitoring nuclear radiation levels within the monitoring area; and predicting nuclear radiation levels within a future period of time based on the nuclear radiation levels;

[0012] interpolating the predicted nuclear radiation value to generate a radiation intensity video of the nuclear contaminated area, and integrating the radiation intensity video of the nuclear contaminated area into a three-dimensional virtual environment to achieve three-dimensional scene fusion;

[0013] According to the posture and position in the real scene and the fused 3D scene, 3D tracking and registration are realized;

[0014] Based on 3D tracking and registration, 3D scene fusion is used to enable augmented reality interactive visualization on smart terminals.

[0015] Furthermore, the step of integrating the radiation intensity video of the nuclear contaminated area into a three-dimensional virtual environment to achieve three-dimensional scene fusion includes:

[0016] The radiation intensity video images of the nuclear contaminated area are integrated into the three-dimensional virtual environment in the form of real-time texture according to the location, realizing the effective fusion of video and three-dimensional scene and improving the information carrying capacity of the virtual scene.

[0017] Furthermore, the step of integrating the radiation intensity video image of the nuclear contaminated area into the three-dimensional virtual environment in a real-time texture manner according to the position includes:

[0018] Establish a one-to-one correspondence between the vertices of the three-dimensional model and the texture coordinates of the video image of the radiation intensity of the nuclear contaminated area;

[0019] Obtain the texture coordinate values ​​of the vertices of the 3D model and the radiation intensity video image of the nuclear contaminated area, and use the texture mapping function to bind the texture and coordinates;

[0020] Through rasterization, the texture coordinates corresponding to each vertex of the 3D model are converted into the texture coordinates corresponding to each pixel on the screen;

[0021] Texture sampling is performed according to the texture coordinates corresponding to each pixel to calculate the shading result.

[0022] Furthermore, the three-dimensional tracking registration is realized based on the acquired posture and position in the real scene and the fused three-dimensional scene, including:

[0023] Use GPS / Beidou combined with an inertial test unit, and then pass the error state Kalman filter to obtain the attitude and position in the real scene;

[0024] According to the posture and position in the real scene, the corresponding screen coordinates in the fused three-dimensional scene are obtained.

[0025] Furthermore, the inertial test unit includes a three-axis acceleration sensor and a three-axis gyroscope angular velocity meter;

[0026] The method uses GPS / Beidou combined with an inertial test unit and then passes the error state Kalman filter to obtain the attitude and position in the real scene, including:

[0027] The triaxial accelerometer is used to measure acceleration in three-dimensional space. Positioning is achieved by integrating the three acceleration values ​​twice to obtain displacement. Because the calculated position information has deviations, an error state Kalman filter algorithm is used to predict the errors, and then the deviations are corrected based on the actual position information obtained.

[0028] The three-axis gyroscope angular velocity meter is used to measure the rotational angular velocity around the three coordinate axes in three-dimensional space; and the attitude information is obtained by integrating the angular velocity values.

[0029] Furthermore, the predicting of the nuclear radiation value within a future period of time based on the nuclear radiation value includes:

[0030] Take any point in the monitoring area, input the nuclear radiation values ​​and other information recorded in the recent period of time at that location into the LSTM neural network, and output the set of nuclear radiation values ​​of that location in the future period of time;

[0031] By integrating the set of nuclear radiation values ​​of the entire monitoring area in the future period, the nuclear radiation value in the future period is used.

[0032] Furthermore, monitoring the nuclear radiation value within the monitoring area includes:

[0033] The nuclear island area is designated as a specific area. Nuclear radiation detectors are installed at intervals in non-specific areas within the monitoring area, and more densely deployed in specific areas.

[0034] According to the arranged nuclear radiation detectors, obtain the nuclear radiation value corresponding to each position;

[0035] The nuclear radiation value is interpolated using a cubic spline function to achieve monitoring of the nuclear radiation value.

[0036] Furthermore, the interpolation processing of the nuclear radiation value using a cubic spline function includes:

[0037] Establishing a three-dimensional array according to the nuclear radiation value and the corresponding position information;

[0038] The nuclear radiation value is subjected to cubic interpolation processing according to the three-dimensional array to achieve regional refined interpolation.

[0039] The second object of the present invention can be achieved by adopting the following technical solutions:

[0040] A nuclear radiation dose augmented reality interactive visualization system based on three-dimensional tracking and registration, the system comprising:

[0041] The nuclear radiation pollutant numerical monitoring module is used to monitor the nuclear radiation value in the monitoring area; based on the nuclear radiation value, predict the nuclear radiation value in the future;

[0042] A 3D scene fusion module is used to generate a radiation intensity video of the nuclear contaminated area based on the predicted nuclear radiation value, and integrate the radiation intensity video of the nuclear contaminated area into a 3D virtual environment to achieve 3D scene fusion;

[0043] A 3D tracking and registration module is used to implement 3D tracking and registration based on the acquired posture and position in the real scene and the fused 3D scene;

[0044] The augmented reality interactive visualization module is used to fuse three-dimensional scenes based on three-dimensional tracking and registration, generate real-time video streams after three-dimensional fusion, and realize augmented reality interactive visualization through 4G / 5G on smart terminals.

[0045] The third object of the present invention can be achieved by adopting the following technical solutions:

[0046] An electronic device includes a processor and a memory for storing a program executable by the processor. When the processor executes the program stored in the memory, the above-mentioned nuclear radiation dose augmented reality interactive visualization method is implemented.

[0047] The fourth object of the present invention can be achieved by adopting the following technical solutions:

[0048] A storage medium stores a program, which, when executed by a processor, implements the above-mentioned nuclear radiation dose augmented reality interactive visualization method.

[0049] The present invention has the following beneficial effects compared to the prior art:

[0050] The method provided by the present invention monitors nuclear radiation levels within a monitoring area; predicts nuclear radiation levels within a future period based on the nuclear radiation levels; generates a radiation intensity video of the nuclear contaminated area based on the predicted nuclear radiation levels, integrates the radiation intensity video of the nuclear contaminated area into a three-dimensional virtual environment, and achieves three-dimensional scene fusion; performs three-dimensional tracking and registration based on the acquired posture and position in the real scene and the fused three-dimensional scene; and enables interactive visualization of augmented reality on smart terminals based on the three-dimensional scene that achieves three-dimensional tracking and registration. By employing technologies based on virtual reality and augmented reality, combined with three-dimensional virtual scenes and three-dimensional tracking and registration, nuclear radiation dose fields are visualized and analyzed in different dimensions, and invisible nuclear radiation dose fields are visualized, providing auxiliary decision support for nuclear power plant emergency response, improving the nuclear power plant's nuclear accident emergency preparedness and response level, and strengthening its ability to respond to emergencies. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0052] Figure 1 This is a flowchart of the nuclear radiation dose augmented reality interactive visualization method based on three-dimensional tracking and registration technology according to Example 1 of the present invention.

[0053] Figure 2 This is a schematic diagram of the three-dimensional scene fusion implementation process of Example 1 of the present invention.

[0054] Figure 3 Schematic diagram of the coordinate system conversion process of Example 1 of the present invention.

[0055] Figure 4 This is a diagram of the conversion relationship from the world coordinate system to the camera coordinate system in Example 1 of the present invention.

[0056] Figure 5 Schematic diagram of three rotation transformations of a vector around a coordinate axis according to embodiment 1 of the present invention.

[0057] Figure 6 This is a diagram showing the conversion relationship from the camera coordinate system to the image coordinate system in Example 1 of the present invention.

[0058] Figure 7 This is a framework diagram of a nuclear radiation dose augmented reality interactive visualization system based on three-dimensional tracking and registration technology according to Example 2 of the present invention.

[0059] Figure 8 This is a structural block diagram of an electronic device according to embodiment 3 of the present invention. DETAILED DESCRIPTION

[0060] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention. It should be understood that the specific embodiments described are only used to explain this application and are not used to limit this application.

[0061] Example 1:

[0062] like Figure 1 As shown, the nuclear radiation dose augmented reality interactive visualization method based on three-dimensional tracking and registration provided in this embodiment includes the following steps:

[0063] S101. Realize numerical monitoring and prediction of nuclear pollutants within the monitoring area.

[0064] Numerical monitoring of sudden nuclear leaks involves the use of a specific number of nuclear radiation detectors, installed at specific locations within nuclear power plants and surrounding areas, to ensure accurate monitoring across the entire covered area. Numerical monitoring of sudden nuclear leaks is a form of quantitative monitoring, and particularly so when high-precision spatiotemporal resolution and timeliness are required, placing extremely high demands on computational timeliness and accuracy. Predicting nuclear pollution values ​​is a form of time series prediction. In this example, an LSTM (Long Short-Term Memory) neural network is used to predict nuclear radiation doses.

[0065] (1) Realize numerical monitoring of nuclear pollutants within the monitoring area.

[0066] Numerical monitoring of nuclear contaminant diffusion primarily consists of three components: nuclear radiation detector layout design, data acquisition, and data processing. The primary function of this module is to enable large-scale nuclear contamination monitoring, ensuring real-time updates of monitoring results. Furthermore, through differential processing, it calculates the numerical value of nuclear radiation contamination within the monitored area.

[0067] (1-1) Layout design of nuclear radiation detectors.

[0068] The function of nuclear radiation detection instruments (i.e., sensors) is to convert nuclear radiation signals into electrical signals, thereby detecting the intensity and changes of radiation. The working principle of such detectors is based on the luminescence effect of certain substances under the influence of nuclear radiation, or the ionization effect of gases emitted when nuclear radiation passes through them.

[0069] Because nuclear radiation affects a large area, comprehensive regional monitoring is difficult, requiring targeted installation of radiation detectors in specific areas. A nuclear power plant primarily consists of the nuclear island, conventional island, supporting facilities, and safety measures. Gas radiation detectors are deployed throughout the plant according to plan.

[0070] In one embodiment, the nuclear island area is monitored as a key area, where more radiation detectors are installed than in other areas. Radiation detectors are installed at regular intervals in non-specific areas, and more densely deployed in the nuclear island area.

[0071] (1-2) According to the arranged nuclear radiation detectors, obtain the nuclear radiation value J corresponding to each position.

[0072] When radiation passes through the object being tested, it is accompanied by energy loss. As long as the exact amount of loss is known, the characteristics of the object being tested can be accurately understood. The change in its radioactivity intensity over time is shown in the following formula:

[0073] J=J0*e -λt

[0074] Where J0 is the initial radioactivity intensity, J is the radioactivity intensity after time t, and λ is the radioactive decay constant. Nuclear radiation sensors convert all or part of the energy of incident nuclear radiation particles into a measurable electrical signal. By collecting the nuclear radiation values ​​detected by the corresponding coordinates of the nuclear radiation detectors deployed in the previous step, we can subsequently calculate the overall nuclear radiation situation around the nuclear power plant.

[0075] (1-3) According to the nuclear radiation value, the nuclear pollution monitoring data is interpolated to realize the numerical monitoring of nuclear pollutants.

[0076] Based on the detection values ​​of the nuclear radiation sensor and the location information of the sensor, a three-dimensional array is established on the back end for storage, and the nuclear pollution monitoring data is interpolated three times to achieve regional fine-grained interpolation.

[0077] The process of interpolation processing of nuclear pollution monitoring data is as follows:

[0078] First, the nuclear power plant is divided into regions, with each adjacent nuclear radiation detector location being divided into an interval. Assuming there are n intervals in total, the function curve for calculating the nuclear radiation value in each small interval is a cubic equation, and the cubic equation satisfies the following conditions:

[0079] (I) In each small interval [x i ,x i+1 ],S(x)=S i (x) is a cubic equation;

[0080] Where i = 0, 1, 2, ..., n, [x i ,x i+1 ] represents the two endpoints of the i-th interval, namely the starting position and the ending position, S i (x) represents the nuclear radiation value corresponding to position x in the i-th interval;

[0081] (II) satisfies the interpolation condition, that is, S(x i )=y i ;y i Indicates position x i The nuclear radiation value at (obtained from the sensor);

[0082] (III) The curve is smooth, that is, S(x), S′(x), and S′′(x) are continuous.

[0083] The above cubic equation can be constructed in the following form:

[0084] y=a i +b i x+c i x 2 +d i

[0085] The above cubic equation is the cubic spline function S i (x).

[0086] There are four unknowns in each small interval of the above cubic equation (a i 、b i 、c i d i ), and there are n intervals in total, so there are 4n unknown items.

[0087] The steps to solve the above cubic equation are as follows:

[0088] 4n unknowns require 4n equations to solve:

[0089] First, all points satisfy the interpolation condition S i (x) = y i , except for the two endpoints, each of the n-1 interior points satisfies Si (x i+1 )=y i+1 、S i+1 (x i+1 )=y i+1 The two piecewise cubic equations before and after, there are 2(n-1) equations; plus the two endpoints satisfying the cubic equations, there are a total of 2n equations;

[0090] Secondly, the first-order derivatives of n-1 internal points are continuous, that is, the end point of the i-th interval and the starting point of the i+1-th interval are the same point, and their first-order derivatives are also equal, that is, S′ i (x i+1 )=S′ i+1 (x i+1 ), then there are n-1 equations. In addition, the second-order derivatives of the interior points are also continuous, that is, S″ i (x i+1 )=S″ i+1 (x i+1 ), there are also n-1 equations.

[0091] There are 4n-2 equations in total, and there are two equations to solve all the unknowns, which can be obtained through boundary conditions.

[0092] There are three types of boundary conditions: natural boundary, fixed boundary, and non-nodal boundary, where:

[0093] Natural boundary: Specify the second-order derivative of the endpoint to be 0, S″(x0)=0=S″(x n );

[0094] Fixed boundary: specify the first-order derivatives of the endpoints, here they are defined as A and B. That is, S′0(x0)=A,S′ n-1 (n) = B;

[0095] Non-kinked boundary: force the third-order derivative value of the first interpolation point to be equal to the third-order derivative value of the second point, and finally the third-order derivative value of the first point to be equal to the third-order derivative value of the second-to-last point. That is, S"′0(x0)=S"′1(x1) and S"′ n-2 (x n-1 )=S"′ n-1 (x n ).

[0096] The specific derivation process is as follows:

[0097] S i (x)= a i +b i (xx i )+c i (xx i ) 2+d i (xx i ) 3 (1)

[0098] S i ′(x)=b i +2c i (xx i )+3d i (xx i ) 2 (2)

[0099] S″ i (x) = 2c i +6d i (xx i ) (3)

[0100] (1-3-1) by S i (x i )=a i +b i (xx i )+c i (xx i ) 2 +d i (xx i ) 3 =y i You can get a i =y i ;

[0101] (1-3-2) Use h i =x i+1 -x i Indicates the step size, S i (x i+1 )=y i+1 roll out

[0102]

[0103] (1-3-3) By S′ i (x i+1 )=S′ i+1 (x i+1 )roll out:

[0104] S′ i (x i+1 )=b i +2c i (x i+1 -x i )+3d i (x i+1 -x i ) 2 =bi +2c i h+3d i h 2

[0105] S′ i+1 (x i+1 )=b i+1 +2c i (x i+1 -x i+1 )+3d i (x i+1 -x i+1 ) 2 =b i+1

[0106] We can get:

[0107]

[0108] (1-3-4) by S″ i (x i+1 )=S″ i+1 (x i+1 ) Launched 2c i +6h i d i =2c i+1 ;

[0109] Let m i =S″ i (x i )=2c i Then 2c i +6h i d i =2c i+1 , which can be rewritten as m i +6h i d i =m i+1 , we can get

[0110] (1-3-5)a i 、b i 、c i and d i Can be expressed as a second-order derivative relationship, and brought into

[0111] a i +b i h i +c i h i 2 +d i h i 3 =y i+1 Available

[0112] (1-3-6) will a i 、b i 、c i and d i Substitution We can get:

[0113]

[0114] The following linear equations can be listed:

[0115]

[0116] in:

[0117] a′ i =h i ,b′ i =2(h i +h i+1 ),c′ i =h i+1 ,f i =6(d i+1 -d i )

[0118] The corresponding construction matrix is:

[0119]

[0120] 1) When clamping boundary conditions:

[0121] b′0=2h0, c′0=h0, d0=0, f0=6(d0-A);

[0122] c′0=0,a′ n =h n-1 ,b′ n =2h n-1 ,f n =6(Bd n-1 );

[0123] 2) Under natural boundary conditions:

[0124] b′0=1, c′0=0, c′0=0, f0=0;

[0125] c′ n =0,a′ n =0,b′0=1,f n =0;

[0126] 3) Under non-kinked boundary conditions:

[0127] b′0=h1, c′0=-(h0+h1), a′0=h0, f0=0;

[0128] c′ n =h n-1 ,a′ n =-(h n-2 +h n-1 ),b′ n =h n-2 ,f n =0;

[0129] Since the necessary and sufficient condition for the non-homogeneous equation system Ax=B with n unknowns to have a unique solution is R(A)=R(B)=n, it can be seen that the matrix equation has a unique solution in the above three cases.

[0130] The matrix equation can be solved by Gaussian column principal element elimination method to obtain m i .

[0131] Finally, substituting into formula (1) we can get:

[0132]

[0133] Based on a certain i 、b i 、c i and d i The value of nuclear pollutants is calculated according to the location of the nuclear pollutants, thereby realizing comprehensive nuclear pollutant value monitoring.

[0134] (2) Predict the nuclear pollution situation based on nuclear pollution monitoring data.

[0135] This step uses the LSTM long short-term memory neural network to predict the nuclear radiation situation of the nuclear power plant in the next time period based on the nuclear radiation changes in the past period of time counted by the nuclear radiation numerical monitoring module and other characteristics of the area in the past time period.

[0136] (2-1)LSTM neural network.

[0137] As a variant of the RNN, the LSTM neural network effectively addresses the vanishing gradient problem that often occurs during the training of traditional RNNs for long time series. Compared to traditional RNNs, the neural network within the LSTM unit is more complex. It processes information about the current unit through input, forget, and output gates, choosing to "forget" or "remember" information from previous units.

[0138] The LSTM unit structure has two important states, the cell state c(t) and the hidden state h(t). The cell state transmits information through different gates, enhancing the dependency between long-term sequence information.

[0139] First, the forget gate f(t) selects which information at the current moment needs to be retained or discarded, and passes it through the sigmoid function, converting it into a value between 0 and 1. The closer it is to 0, the easier it is to discard; otherwise, it is easier to retain. The calculation formula is as follows:

[0140] f(t)=σ(W fh h(t-1)+W fx x(t)+b f )

[0141] Where σ(·) is the sigmoid activation function, W fh and W fx represents the corresponding weight parameter, x(t) represents the input at time t, h(t-1) represents the hidden state of the cell at time t-1, and b f is the bias term.

[0142] Secondly, the function of the input gate i(t) is to selectively memorize the candidate cell states and update the current cell state. The current input and the previous hidden state are converted to values ​​between 0 and 1 through the sigmoid function. The current input and the previous hidden state are compressed to between -1 and 1 through the tanh activation function:

[0143] i(t)=σ(W ih h(t-1)+W ix x(t)+b i )

[0144]

[0145] Among them, W ih 、W ix 、W ch and W cx Represents the corresponding weight parameter, x(t) represents the input at time t, h(t-1) represents the hidden state of the cell at time t-1, represents the candidate state at the current moment, b i and b c is the bias term, Represents element-wise multiplication.

[0146] Then, the output gate o(t) determines the output component of the cell state through the sigmoid function, and then the cell state is matrix-multiplied with it point by point through tanh to obtain the new hidden state h(t):

[0147] o(t)=σ(W oh h(t-1)+W ox x(t)+b o

[0148]

[0149] Among them, W oh and W ox Represents the corresponding weight parameter, x(t) represents the input at time t, b o represents the bias term.

[0150] (2-2) Based on nuclear pollution monitoring data, use LSTM neural network to predict nuclear radiation values.

[0151] The set of nuclear radiation values ​​and other information recorded in the recent period at any point in the monitoring area is input into the trained LSTM neural network, and the set of nuclear radiation values ​​at that location in the future is output. By integrating the set of nuclear radiation values ​​in the entire monitoring area in the future, the nuclear pollution situation can be predicted, including:

[0152] First define the input matrix A at each position c , A c It is the nuclear radiation value recorded at point c in the nuclear power plant in the recent period of time and a collection of other information of the location during this period of time (such as wind direction and speed, temperature and humidity, etc.).

[0153]

[0154] Where n is the number of input features; k is the maximum time step; J t,c is the radiation value of the position c at time step t, t = 1, 2, ... k; F t,j (t=1, 2, ..., k; j=1, 2, ..., n) is the jth input feature at time step t. For example, the information included at point c at time t is wind speed, temperature, humidity, etc.

[0155] The final output is a collection of radiation values ​​for that location over the next period of time. This is then integrated to determine the radiation situation for the entire nuclear power plant over the next period of time. Specifically, by inputting the previously collected radiation values ​​for the entire area of ​​the nuclear power plant, along with data from various sensors, into a pre-trained LSTM neural network, the resulting radiation situation for the plant over the next period of time can be determined.

[0156] S102. Generate radiation intensity image and video of the nuclear contaminated area according to the predicted nuclear radiation value, and integrate the image and video into the three-dimensional virtual environment according to the position in a real-time texture manner to achieve three-dimensional scene fusion.

[0157] The virtual-real fusion of nuclear radiation prediction thematic maps and three-dimensional scenes uses global nuclear pollution values ​​predicted by interpolation calculation or neural network to generate radiation intensity videos of nuclear contaminated areas, and integrates video images into a three-dimensional virtual environment according to location in the form of real-time textures.

[0158] Video fusion technology in a 3D setting combines nuclear radiation trend images with a 3D virtual scene. Using a 3D scene fusion algorithm, the 3D static scene and dynamic video footage are geographically reconstructed to create a new scene. The fusion of 3D models and video enhances the interaction between the virtual scene and the real world, increasing the virtual scene's information carrying capacity and building a bridge between reality and virtuality, effectively improving information acquisition capabilities.

[0159] First, the regional radiation values ​​obtained from the nuclear power plant's future radiation forecast obtained in step S101 are visualized and converted into a continuously updated video. The video image is then integrated into the 3D virtual environment using real-time textures based on location, effectively integrating the video with the 3D scene. 3D scene fusion technology plays an important role in achieving a seamless integration of virtual and real visual perception and enhancing a truly immersive visual experience.

[0160] The fusion steps specifically include:

[0161] First, a one-to-one correspondence between 3D model vertices and video texture coordinates is established. The model vertex and video texture coordinate values ​​are obtained, and then the 3DGIS texture mapping function is used to bind the texture and coordinates. Subsequently, through rasterization in the GPU rendering pipeline, the texture coordinates corresponding to each model vertex are converted into texture coordinates corresponding to each pixel on the screen. Texture sampling is performed based on the texture coordinates corresponding to each pixel to calculate the shading result.

[0162] The whole process of texture mapping to obtain texture coordinates is as follows Figure 2 shown.

[0163] S103: Implement three-dimensional tracking and registration based on the acquired posture and position in the real scene and the fused three-dimensional scene.

[0164] 3D tracking and registration technology is a key technology in augmented reality. By tracking and locating images, objects, or people in a real scene, virtual objects are superimposed onto the real scene with the correct spatial perspective. This allows for synchronization between the real and virtual environments, enabling users to more intuitively and conveniently access multi-source information, enhancing the user experience.

[0165] To achieve seamless superposition of virtual and real scenes, the virtual information and the real environment must be aligned in three-dimensional space using tracking and registration technology. This embodiment uses GPS / Beidou combined with an IMU inertial measurement unit (IMU) and then uses an ESKF error state Kalman filter to obtain high-precision position and attitude information. The global positioning system (GPS) and Beidou satellite navigation system obtain the latitude and longitude position information of the current location, but the GPS / Beidou update frequency is relatively low, approximately 10Hz. The inertial measurement unit (IMU) has a higher update frequency, generally reaching 1KHz, and generally includes a three-axis accelerometer and a three-axis gyroscope angular velocity meter. The former can measure the acceleration of a three-dimensional space, and the latter can measure the rotation speed around the three coordinate axes of the three-dimensional space. Using the three acceleration values, displacement can be obtained through two integrations to achieve position positioning. The angular velocity value integration can obtain attitude information, and combined together, the actual state of the object can be obtained. Although the GPS / Beidou update frequency is low, there is no error accumulation with each update because each information is independent of the previous value. IMUs update frequently, but this process can lead to cumulative errors. Because position and attitude information are obtained through integration, they complement each other and are then filtered through an error-state Kalman filter (ESKF) to achieve relatively accurate positioning. The performance of the algorithms involved in tracking and registration technology directly determines the ultimate effectiveness of augmented reality systems.

[0166] (1) Calculate the posture and position in the real scene.

[0167] In this embodiment, the three sensors used for viewing angle and position calculation are an accelerometer, a gyroscope, and GPS / Beidou. The spatial position of the target can be obtained by performing a quadratic integration on the accelerometer measurement; the rotation angle output of the target can be obtained by integrating the angular rate gyroscope's measurement value once. However, the measurement errors of these two inertial sensors drift over time and there is a zero-point deviation, so they need to be regularly corrected using absolute position information. The error state Kalman filter algorithm uses an IMU to predict the error and observes through GPS / Beidou to correct the position information. The error-state Kalman filter (ESKF) is an indirect filter that processes the system error state. It has a simple structure and is widely used in multi-sensor data fusion.

[0168] Since the system in this embodiment deals with a nonlinear system, it is necessary to use an error state Kalman filter to fuse multi-source measurement information from the inertial gyroscope and accelerometer and then use GPS / Beidou for measurement. This information is fused according to the minimum mean square error criterion to give the optimal estimate of the state.

[0169] (1-1) First, define the error prediction process.

[0170] Error state transition process:

[0171] δ x ←f(x,δ x ,u m ,i)=F x (x,u m )·δ x +F i ·i

[0172] The position error δ is defined during the prediction process p , speed error δ v , the error of posture δ θ , acceleration bias error Angular velocity bias error and gravitational acceleration; δ in the above formula x Represents the updated error, that is, x represents p, v, θ, a respectively b and ω b ; where u m is the acceleration and angular velocity measured by the IMU, which is a vector of 6 rows and 1 column; X is the six states to be estimated, which is a vector of 18 rows and 1 column; i is the noise vector of 12 rows and 1 column; F x is the state transition matrix; F i is the noise transfer matrix:

[0173]

[0174]

[0175] Update of variance:

[0176] P←F x PF x T +F i Q i F i T

[0177] The above formula is the update process of the variance of the error in the prediction stage. In the first update, the initial value of the variance P needs to be set, where Q i It is a 12*12 matrix, and the expression is as follows:

[0178]

[0179] Where V i 、Θ i 、A i ,Ω i It is derived from the following formula:

[0180]

[0181] in, The noise of the accelerometer is expressed in m / s 2 , the square of the accelerometer noise multiplied by Δt 2 Multiply it by the unit matrix to get the noise covariance of the error velocity δV, the unit is m 2 / s 2 Similarly, we can get the covariance Θ of δΘ noise i .

[0182] (1-2) Observation process:

[0183] First, define the observation equation:

[0184] y=h(x t )+v

[0185] Update of Kalman gain:

[0186] K=PH T (HPH T +V) -1

[0187] Corrected error after observation:

[0188]

[0189] Covariance after observation:

[0190] P←(I-KH)P

[0191] The H matrix is:

[0192]

[0193] This process can obtain accurate and real-time updated personnel position and viewing angle information after correction.

[0194] (2) Obtain the corresponding screen coordinates based on the posture and position in the real scene.

[0195] The geographic registration in this step refers to the process of converting world coordinates to camera coordinates and then converting camera coordinates to screen coordinates.

[0196] (2-1) Convert world coordinates to camera coordinates.

[0197] Depend on Figure 4 It can be seen that the change from the world coordinate system to the camera coordinate system is a rigid body change, that is, only a geometric object is rotated and translated, and the object does not deform. w ,Y w ,Z w ) coordinates in the camera coordinate system are:

[0198]

[0199] Where R represents the rotation matrix, R = R1(α)R2(β)R3(γ), where R1(α), R2(β), and R3(γ) are calculated as follows. The three rotation transformations of the vector around the coordinate axis are as follows: Figure 5 As shown, T represents the translation matrix.

[0200]

[0201]

[0202] (2-2) Convert camera coordinates to image coordinates.

[0203] From the camera coordinate system to the image coordinate system, it is a transformation from a three-dimensional coordinate system to a two-dimensional coordinate system. It belongs to the perspective projection transformation, that is, the object is projected onto the projection surface using the central projection method, thereby obtaining a single-sided projection image that is closer to the visual effect.

[0204] Depend on Figure 6 It can be seen that:

[0205] △ABO c ~△oCO c

[0206]

[0207] S104. Based on the three-dimensional tracking and registration, the three-dimensional scene is integrated and the smart terminal realizes the augmented reality interactive visualization.

[0208] The geographical location and posture information of the smart terminal are obtained based on the sensor, and the radiation thematic map is obtained by combining it with three-dimensional tracking and registration to realize augmented reality interactive visualization that integrates real scenes and three-dimensional virtual scenes.

[0209] Smart terminals are emerging internet hardware products that possess information collection, processing, and connectivity capabilities, and enable intelligent perception, interaction, and big data services. They are key enablers of "Internet Plus" and artificial intelligence. With the development of technologies like 5G, big data, and the Internet of Things, my country's smart terminal market holds broad prospects.

[0210] Specifically, the smart terminal includes CPU, operating system, display optical machine, GPS / Beidou positioning module, accelerometer, gyroscope sensor, infrared imager, Bluetooth, battery, and WIFI / 4G / 5G communication modules.

[0211] Augmented reality, also known as augmented reality, is a subtype of virtual reality. Unlike traditional virtual reality, which aims for complete immersion, augmented reality technology fuses computer-generated virtual world information with real-world information to enhance real-world scenes. The resulting scenes can be displayed through devices such as monitors, projectors, and head-mounted displays, enhancing and expanding human perception of the environment.

[0212] Specifically, the final augmented reality visualization stage is mainly based on the current location and posture information of the emergency personnel's smart terminal, generating a real-time video stream that has undergone three-dimensional fusion on the 3DGIS backend. Emergency personnel use the smart terminal to interactively visualize the video stream and understand the nuclear radiation trend and radiation intensity near themselves.

[0213] Those skilled in the art will appreciate that all or part of the steps in the method for implementing the above embodiments may be completed by instructing related hardware through a program, and the corresponding program may be stored in a computer-readable storage medium.

[0214] It should be noted that although the method operations of the above embodiments are described in a particular order in the accompanying drawings, this does not require or imply that the operations must be performed in this particular order, or that all of the illustrated operations must be performed to achieve the desired results. Rather, the depicted steps may be performed in a different order. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into a single step, and / or a single step may be broken down into multiple steps.

[0215] Example 2:

[0216] like Figure 7 As shown, this embodiment provides a nuclear radiation dose augmented reality interactive visualization system based on three-dimensional tracking and registration. The system includes a nuclear radiation pollutant numerical monitoring module 701, a three-dimensional scene fusion module 702, a three-dimensional tracking and registration module 703, and an augmented reality interactive visualization module 704, wherein:

[0217] The nuclear radiation pollutant value monitoring module 701 is used to monitor the nuclear radiation value in the monitoring area and predict the nuclear radiation value in the future based on the nuclear radiation value;

[0218] A 3D scene fusion module 702 is configured to generate a radiation intensity video of the nuclear contaminated area based on the predicted nuclear radiation value, and integrate the radiation intensity video of the nuclear contaminated area into a 3D virtual environment to achieve 3D scene fusion.

[0219] The 3D tracking and registration module 703 is used to implement 3D tracking and registration based on the acquired posture and position in the real scene and the fused 3D scene;

[0220] The augmented reality interactive visualization module 704 is used to fuse three-dimensional scenes based on three-dimensional tracking and registration, generate a real-time video stream after three-dimensional fusion, and realize augmented reality interactive visualization through 4G / 5G on smart terminals.

[0221] The specific implementation of each module in this embodiment can be found in the above-mentioned embodiment 1, and will not be described one by one here; it should be noted that the system provided in this embodiment is only illustrated by the division of the above-mentioned functional modules. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure can be divided into different functional modules to complete all or part of the functions described above.

[0222] Example 3:

[0223] This embodiment provides an electronic device, which may be a computer, such as Figure 8 As shown, a processor 802, a memory, an input device 803, a display 804, and a network interface 805 are connected via a system bus 801. The processor is used to provide computing and control capabilities. The memory includes a non-volatile storage medium 806 and an internal memory 807. The non-volatile storage medium 806 stores an operating system, a computer program, and a database. The internal memory 807 provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. When the processor 802 executes the computer program stored in the memory, the nuclear radiation dose augmented reality interactive visualization method of the above-mentioned embodiment 1 is implemented as follows:

[0224] Monitoring nuclear radiation levels within the monitoring area; and predicting nuclear radiation levels within a future period of time based on the nuclear radiation levels;

[0225] Interpolating the predicted nuclear radiation value to generate a radiation intensity video of the nuclear contaminated area, and integrating the nuclear contamination radiation intensity video into a three-dimensional virtual environment to achieve three-dimensional scene fusion;

[0226] According to the posture and position in the real scene and the fused 3D scene, 3D tracking and registration are realized;

[0227] Based on 3D tracking and registration, 3D scene fusion is used to enable augmented reality interactive visualization on smart terminals.

[0228] Example 4:

[0229] This embodiment provides a storage medium, which is a computer-readable storage medium and stores a computer program. When the computer program is executed by a processor, the nuclear radiation dose augmented reality interactive visualization method of the above-mentioned embodiment 1 is implemented as follows:

[0230] Monitoring nuclear radiation levels within the monitoring area; and predicting nuclear radiation levels within a future period of time based on the nuclear radiation levels;

[0231] Interpolating the predicted nuclear radiation value to generate a radiation intensity video of the nuclear contaminated area, and integrating the nuclear contamination radiation intensity video into a three-dimensional virtual environment to achieve three-dimensional scene fusion;

[0232] According to the posture and position in the real scene and the fused 3D scene, 3D tracking and registration are realized;

[0233] Based on 3D tracking and registration, 3D scene fusion is used to enable augmented reality interactive visualization on smart terminals.

[0234] It should be noted that the computer-readable storage medium of the present embodiment may be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0235] In summary, the method provided by the present invention includes numerical monitoring and prediction of nuclear pollutants, three-dimensional virtual-real fusion, three-dimensional tracking and registration, and augmented reality visualization. Among them, the numerical monitoring and prediction of nuclear pollutants ensures that the covered area can be accurately monitored by installing a certain number of nuclear radiation detectors at specific locations in the nuclear power plant and the surrounding area; the numerical monitoring of sudden nuclear leakage pollutants is quantitative monitoring, especially under the requirements of high-precision spatiotemporal resolution and timeliness, which places extremely high demands on the timeliness and accuracy of calculations; the prediction of nuclear pollution values ​​is a time series prediction, and the present invention uses an LSTM (Long Short-Term Memory) long short-term memory neural network to predict nuclear radiation dose; three-dimensional virtual-real fusion is to use the global nuclear pollution values ​​predicted by interpolation calculation or neural network to generate a radiation intensity video of the nuclear contaminated area, and integrate the video image into a three-dimensional virtual environment in a real-time texture manner according to the position; three-dimensional scene fusion technology plays an important role in achieving a seamless combination of virtual and real visual senses and improving a real immersive visual experience; three-dimensional tracking and registration is a key technology for achieving augmented reality effects. In order to achieve seamless superposition of virtual scenes and real scenes, it is required that virtual information and real environments are aligned in three-dimensional spatial positions using tracking and registration technology. The present invention uses GPS / Beidou combined with an IMU inertial measurement unit and then passes through an ESKF error state Kalman filter to obtain high-precision position and attitude information. The performance of the relevant algorithms in the tracking and registration technology directly determines the final use effect of the augmented reality system. The augmented reality visualization is mainly based on the position and attitude of the smart terminal, and the back-end fusion scene is presented to the smart terminal, which is convenient for emergency rescue personnel to understand the intensity and trend of nuclear radiation near their location. For easy viewing, the present invention uses a display light machine to display the results in augmented reality, which does not hinder the observation of the real world while allowing the understanding of radiation information. The present invention obtains the position and attitude information of the emergency rescue personnel, and the smart terminal generates thematic image videos, which facilitates emergency rescue personnel to intuitively obtain global and three-dimensional nuclear power plant nuclear radiation intensity information, which is used to guide the emergency evacuation of personnel in nuclear radiation scenarios and provide auxiliary decision support for nuclear power plant emergency response.

[0236] The above is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes based on the technical solution and inventive concept of the present invention within the scope disclosed by the present invention, which falls within the scope of protection of the present invention.

Claims

1. A nuclear radiation dose augmented reality interactive visualization method based on three-dimensional tracking and registration, characterized in that: The method comprises: Monitoring nuclear radiation levels within the monitoring area; and predicting nuclear radiation levels within a future period of time based on the nuclear radiation levels; interpolating the predicted nuclear radiation value to generate a radiation intensity video of the nuclear contaminated area, and integrating the radiation intensity video of the nuclear contaminated area into a three-dimensional virtual environment to achieve three-dimensional scene fusion; According to the posture and position in the real scene and the fused 3D scene, 3D tracking and registration are realized; Based on 3D tracking and registration, 3D scene fusion enables augmented reality interactive visualization on smart terminals; The monitoring of nuclear radiation values ​​includes: Design of the layout of nuclear radiation detectors; According to the arranged nuclear radiation detectors, the nuclear radiation value corresponding to each position is obtained as follows: J=J0*e -λt Where J0 is the radioactivity intensity at the beginning, J is the radioactivity intensity after time t, and λ is the radioactivity decay constant; According to the nuclear radiation value, the nuclear pollution monitoring data is interpolated to realize the numerical monitoring of nuclear pollutants; The method of predicting the nuclear radiation value within a future period of time based on the nuclear radiation value includes: Take any point in the monitoring area, input the nuclear radiation values ​​and other information recorded in the recent period of time at that location into the LSTM neural network, and output the set of nuclear radiation values ​​of that location in the future period of time; By integrating the set of nuclear radiation values ​​of the entire monitoring area in the future period of time as the nuclear radiation value in the future period of time; The step of integrating the radiation intensity video of the nuclear contaminated area into a three-dimensional virtual environment to achieve three-dimensional scene fusion includes: The radiation intensity video images of the nuclear contaminated area are integrated into the three-dimensional virtual environment in the form of real-time texture according to the location, realizing the effective fusion of video and three-dimensional scene.

2. The nuclear radiation dose augmented reality interactive visualization method according to claim 1, characterized in that: The method of integrating the radiation intensity video image of the nuclear contaminated area into the three-dimensional virtual environment in a real-time texture manner according to the position includes: Establish a one-to-one correspondence between the vertices of the three-dimensional model and the texture coordinates of the video image of the radiation intensity of the nuclear contaminated area; Obtain the texture coordinate values ​​of the vertices of the 3D model and the radiation intensity video image of the nuclear contaminated area, and use the texture mapping function to bind the texture and coordinates; Through rasterization, the texture coordinates corresponding to each vertex of the 3D model are converted into the texture coordinates corresponding to each pixel on the screen; Texture sampling is performed according to the texture coordinates corresponding to each pixel to calculate the shading result.

3. The nuclear radiation dose augmented reality interactive visualization method according to claim 1, characterized in that: The three-dimensional tracking registration is realized based on the acquired posture and position in the real scene and the fused three-dimensional scene, including: Use GPS or BeiDou, combined with an inertial test unit, and then pass the error state Kalman filter to obtain the attitude and position in the real scene; According to the posture and position in the real scene, the corresponding screen coordinates in the fused three-dimensional scene are obtained.

4. The nuclear radiation dose augmented reality interactive visualization method according to claim 3, characterized in that: The inertial test unit includes a three-axis acceleration sensor and a three-axis gyroscope angular velocity meter; The method uses GPS or BeiDou, combined with an inertial test unit, and then passes the error state Kalman filter to obtain the attitude and position in the real scene, including: The triaxial accelerometer is used to measure acceleration in three-dimensional space. Positioning is achieved by integrating the three acceleration values ​​twice to obtain displacement. Because the calculated position information has deviations, an error state Kalman filter algorithm is used to predict the errors, and then the deviations are corrected based on the actual position information obtained. The three-axis gyroscope angular velocity meter is used to measure the rotational angular velocity around the three coordinate axes in three-dimensional space; and the attitude information is obtained by integrating the angular velocity values.

5. The nuclear radiation dose augmented reality interactive visualization method according to any one of claims 1 to 4, characterized in that: Monitoring nuclear radiation values ​​within the monitoring area includes: The nuclear island area is designated as a specific area. Nuclear radiation detectors are installed at intervals in non-specific areas within the monitoring area, and more densely deployed in specific areas. According to the arranged nuclear radiation detectors, obtain the nuclear radiation value corresponding to each position; The nuclear radiation value is interpolated using a cubic spline function to achieve monitoring of the nuclear radiation value.

6. The nuclear radiation dose augmented reality interactive visualization method according to claim 5, characterized in that: The interpolation processing of the nuclear radiation value using a cubic spline function includes: Establishing a three-dimensional array according to the nuclear radiation value and the corresponding position information; The nuclear radiation value is subjected to cubic interpolation processing according to the three-dimensional array to achieve regional refined interpolation.

7. A nuclear radiation dose augmented reality interactive visualization system based on three-dimensional tracking and registration, characterized in that: The system comprises: The nuclear radiation pollutant numerical monitoring module is used to monitor the nuclear radiation value in the monitoring area; based on the nuclear radiation value, predict the nuclear radiation value in the future; A 3D scene fusion module is used to interpolate and generate a radiation intensity video of the nuclear contaminated area based on the predicted nuclear radiation value, and integrate the radiation intensity video of the nuclear contaminated area into a 3D virtual environment to achieve 3D scene fusion; A 3D tracking and registration module is used to implement 3D tracking and registration based on the acquired posture and position in the real scene and the fused 3D scene; The augmented reality interactive visualization module is used to fuse 3D scenes based on 3D tracking and registration, generating a 3D-fused real-time video stream. Smart terminals can achieve augmented reality interactive visualization via 4G or 5G. The monitoring of nuclear radiation values ​​includes: Design of the layout of nuclear radiation detectors; According to the arranged nuclear radiation detectors, the nuclear radiation value corresponding to each position is obtained as follows: J=J0*e -λt Where J0 is the radioactivity intensity at the beginning, J is the radioactivity intensity after time t, and λ is the radioactivity decay constant; According to the nuclear radiation value, the nuclear pollution monitoring data is interpolated to realize the numerical monitoring of nuclear pollutants; The method of predicting the nuclear radiation value within a future period of time based on the nuclear radiation value includes: Take any point in the monitoring area, input the nuclear radiation values ​​and other information recorded in the recent period of time at that location into the LSTM neural network, and output the set of nuclear radiation values ​​of that location in the future period of time; By integrating the set of nuclear radiation values ​​of the entire monitoring area in the future period of time as the nuclear radiation value in the future period of time; The step of integrating the radiation intensity video of the nuclear contaminated area into a three-dimensional virtual environment to achieve three-dimensional scene fusion includes: The radiation intensity video images of the nuclear contaminated area are integrated into the three-dimensional virtual environment in the form of real-time texture according to the location, realizing the effective fusion of video and three-dimensional scene.

8. A storage medium storing a program, characterized in that: When the program is executed by a processor, the nuclear radiation dose augmented reality interactive visualization method according to any one of claims 1 to 6 is implemented.

Images