Four-dimensional space-time field reconstruction method based on phase shift time coding and imaging space consistency

Abstract

The invention discloses a four-dimensional space-time field reconstruction method based on phase shift time coding and imaging space consistency, and relates to a four-dimensional space-time field reconstruction method. The objective of the invention is to solve the hidden danger that the image acquisition time of the existing super-generalized stereopair is uncertain and the fitting is difficult. And due to the difference between imaging geometric model parameter representation forms and precision among images, the mapping direction of a light sample is inaccurate, and the geometric constraint of light joint construction has deviation. The method comprises the following steps: step 1, constructing a space-time field function; 2, mapping the remote sensing image imaging geometric model to a space-time field coordinate system; optimizing parameters of the remote sensing image imaging geometric model under the space-time field coordinate system to obtain the remote sensing image imaging geometric model with the optimal parameters under the space-time field coordinate system; and step 3, completing reconstruction of a four-dimensional space-time field based on the constructed space-time field function and the remote sensing image imaging geometric model of the optimal parameters under the space-time field coordinate system. The method is applied to the field of space-time field reconstruction.

Description

Technical Field

[0001] The present invention relates to a four-dimensional space-time field reconstruction method. Background Art

[0002] Many software products in the field of remote sensing, such as Google Earth of Google, SkylineGlobe of Skyline Company, Makai Voyager of Makai Ocean Engineering Company, etc., all have the function of displaying four-dimensional or quasi-four-dimensional spatio-temporal data, support loading the three-dimensional reconstruction results with time information, and can display the three-dimensional geometric forms of the scene at different times. Since they are not oriented to the three-dimensional reconstruction task of time-varying scenes based on ultra-generalized stereo image pairs, most of the similar software only stays at the level of displaying results. For ultra-generalized stereo image pairs, image acquisition is discrete and at non-fixed intervals in the time dimension. Therefore, although the three-dimensional reconstruction of time-varying scenes is not a four-dimensional reconstruction with continuous time, the intervention of a four-dimensional space-time representation method is still required, that is, a four-dimensional space-time field function containing spatio-temporal information needs to be constructed. If only the three-dimensional space is implicitly represented at discrete time sampling points, the space-time field function is still equivalent to the three-dimensional implicit representation, which is difficult to provide effective time features and is not conducive to the time-varying analysis of the scene; if a four-dimensional space-time implicit function is directly constructed based on the neural implicit representation method, due to the lack of continuous time data support, it is easy to cause the space-time field function to be difficult to fit. Therefore, when introducing the time dimension, it is necessary to take into account the characteristics of the image observation time and the strategy of spatial implicit representation. The relevant research on the time encoding method customized for time-varying scenes is the key to realizing the organic combination of the time dimension and the space dimension and reducing the complexity of the spatio-temporal representation of the scene. However, there is currently a lack of a reliable four-dimensional space-time implicit representation method for time-varying scenes.

[0003] In terms of the complexity of the spatio-temporal representation of the scene and the imaging geometric model differences of multi-source images, how to introduce the time dimension and construct an implicit representation function of the space-time field; on this basis, how to reconstruct the imaging geometric model of ultra-generalized stereo image pairs into a spatio-temporal field imaging geometric model with a unified form and optimized model parameters;

[0004] Study the construction method of the space-time field, including the construction of the space-time field function and the reconstruction of the imaging geometric model in the space-time field. As Figure 2 shown, first study a space-time field function for representing time-varying scenes and its neural implicit representation method, providing a prerequisite for the three-dimensional reconstruction of time-varying scenes.

[0005] The image acquisition time of ultra-generalized stereo image pairs is uncertain. An improperly constructed space-time field function is difficult to perceive the time correlation contained in the images and the periodic physical imaging laws at different time unit levels such as year, month, day, and hour, resulting in a hidden danger of fitting difficulties.

[0006] The differences in the parameter representation forms and accuracies of the imaging geometric models between images lead to inaccurate mapping directions of light samples. As shown in Figure 3 , this causes problems with deviations in the geometric constraints constructed by the combination of light rays. Summary of the Invention

[0007] The objective of the present invention is to solve the problems that the image acquisition time of existing ultra - generalized stereo image pairs is uncertain, posing a potential risk of difficult fitting; and the differences in the parameter representation forms and accuracies of the imaging geometric models between images result in inaccurate mapping directions of light samples, leading to deviations in the geometric constraints constructed by the combination of light rays. Thus, a four - dimensional spatio - temporal field reconstruction method based on phase - shift time encoding and imaging space consistency is proposed.

[0008] The specific process of the four - dimensional spatio - temporal field reconstruction method based on phase - shift time encoding and imaging space consistency is as follows:

[0009] Step 1: Construct a spatio - temporal field function;

[0010] Step 2: Map the remote - sensing image imaging geometric model to the spatio - temporal field coordinate system;

[0011] Optimize the parameters of the remote - sensing image imaging geometric model in the spatio - temporal field coordinate system to obtain the remote - sensing image imaging geometric model with optimal parameters in the spatio - temporal field coordinate system;

[0012] Step 3: Based on the constructed spatio - temporal field function and the remote - sensing image imaging geometric model with optimal parameters in the spatio - temporal field coordinate system, complete the reconstruction of the four - dimensional spatio - temporal field.

[0013] The beneficial effects of the present invention are as follows:

[0014] The present invention proposes a four - dimensional spatio - temporal field construction technology based on phase - shift time encoding and imaging space consistency to construct a spatio - temporal field for a time - varying scene. It generally includes two parts of tasks: constructing a spatio - temporal field function and reconstructing an imaging geometric model. First, a spatio - temporal field function is jointly constituted based on a time - varying signed distance function and a time - varying radiation function to implicitly represent the four - dimensional spatio - temporal field. The time dimension is introduced based on the phase - shift time encoding method, which helps the spatio - temporal field function perceive the time correlations contained between images and the periodic physical imaging laws at different time unit levels such as year, month, day, and hour, facilitating the exploration of the imaging laws of time - varying scenes, optimizing the learning rules, and reducing the fitting difficulty of the spatio - temporal field function. Second, a reconstruction strategy for the imaging geometric model based on imaging space consistency is proposed. The original imaging geometric model of the image is mapped to the spatio - temporal field imaging geometric model, and a model parameter optimization strategy is designed to address the potential risk of geometric constraint deviations caused by the differences in the imaging geometric models.

[0015] The present invention proposes a four - dimensional spatio - temporal field construction technology based on phase - shift time encoding and imaging space consistency to solve the problems of the complexity of scene spatio - temporal representation and the differences in the imaging geometric models of multi - source images.

[0016] The present invention normalizes the original imaging geometric model of each image of the ultra-generalized stereo image pair, uniformly maps it to the spatio-temporal field imaging geometric model, and designs a parameter optimization strategy for the model, providing necessary conditions for generating a ray sample set based on the ultra-generalized stereo image pair and establishing geometric constraint rules. Description of the Drawings

[0017] Figure 1 is a flowchart for constructing a four-dimensional spatio-temporal field based on phase-shift time coding and imaging space consistency of the present invention;

[0018] Figure 2 is a flowchart for constructing a spatio-temporal field for an ultra-generalized remote sensing stereo image pair;

[0019] Figure 3 is a diagram of the geometric constraint deviation problem caused by the difference in the imaging geometric model. Detailed Description of the Invention

[0020] Detailed Embodiment 1: The specific process of the four-dimensional spatio-temporal field reconstruction method based on phase-shift time coding and imaging space consistency in this embodiment is as follows:

[0021] This part of the technology aims to build the basic framework of the four-dimensional spatio-temporal scene. The overall technical block diagram is as Figure 1 shown, mainly including two components: the construction of the spatio-temporal field function and the reconstruction of the imaging geometric model.

[0022] Step 1: Construct the spatio-temporal field function;

[0023] Step 2: Map the imaging geometric model of the remote sensing image to the spatio-temporal field coordinate system;

[0024] Optimize the parameters of the imaging geometric model of the remote sensing image in the spatio-temporal field coordinate system to obtain the optimal parameter imaging geometric model of the remote sensing image in the spatio-temporal field coordinate system;

[0025] Step 3: Based on the constructed spatio-temporal field function and the optimal parameter imaging geometric model of the remote sensing image in the spatio-temporal field coordinate system, complete the reconstruction of the four-dimensional spatio-temporal field.

[0026] Detailed Embodiment 2: The difference between this embodiment and Detailed Embodiment 1 is that in Step 1, the spatio-temporal field function is constructed; the specific process is as follows:

[0027] Step 1-1: Model the spatio-temporal field function;

[0028] Step 1-2: Initialize the spatio-temporal field function.

[0029] Other steps and parameters are the same as those in Detailed Embodiment 1.

[0030] Specific Embodiment 3: The difference between this embodiment and Embodiment 1 or 2 is as follows: In step 11, the spatio-temporal field function is modeled. The specific process is as follows:

[0031] The time-varying scene is set as a spatio-temporal field F, which is jointly composed of a time-varying geometric field G and a time-varying radiation field R.

[0032] The time-varying geometric field represents the spatial geometric form of the scene at different times and reflects the spatial distribution characteristics of the scene.

[0033] The time-varying radiation field represents the radiation values when observing the corresponding spatial geometric form from different perspectives at different times and reflects the light reflection characteristics of the scene.

[0034] On this basis, the spatio-temporal field function is represented jointly by the time-varying geometric function and the time-varying radiation function.

[0035] Step 111: Construct a time encoding function.

[0036] Step 112: Based on the time encoding function, construct a time-varying signed distance function TVSDF.

[0037] Step 113: Based on the time encoding function and the time-varying signed distance function TVSDF, construct a time-varying spectral radiation function TVSRF.

[0038] Step 114: Based on the time-varying signed distance function TVSDF and the time-varying spectral radiation function TVSRF, construct a spatio-temporal field function STF (Spatio-Temporal Function, STF).

[0039] Other steps and parameters are the same as those in Embodiment 1 or 2.

[0040] Specific Embodiment 4: The difference between this embodiment and any one of Embodiments 1 to 3 is as follows: In step 111, the time encoding function is constructed. The specific process is as follows:

[0041] Time encoding module: A time feature extraction method based on phase shift time encoding TSE (Time Shift Encoding) is proposed. As shown in Equation (5), the time dimension is introduced, and the observation time series corresponding to the input image is given in the order of year, month, day, and hour.

[0042]

[0043] In the formula, represents the vector concatenation operator, which can concatenate multiple individual vectors into a one-dimensional vector;

[0044] respectively represent the normalized results of the year, the day within the year, and the hour within the day;

[0045] Y number represents the current year, Y min represents the minimum year of the input time series, Y max represents the maximum year of the input time series;

[0046] M number is the current month; M sum is the total number of observable days within a month, which is fixed and consistent for each month after being set; D number corresponds to the date within the month, Y sum is the number of observable days within a year;

[0047] H number is the current hour, H sum is the total number of observable hours within a day, which is fixed and consistent for each day after being set; H sum is fixed and consistent.

[0048] In view of the periodic law of time units, the sin(·) function and its phase-shifted form cos(·) are used to extract the time characteristics in the time characteristic function. The time encoding function fully considers the irradiance difference of solar illumination within a day for encoding the intra-day time information, while fully considering the periodicity of seasonal changes for encoding the intra-year time information. The time characteristics are helpful for the radiation field to output more accurate TVSRF values according to the illumination characteristics at different times during the fitting process of the spatio-temporal field function, thereby optimizing the learning rule and reducing the fitting difficulty of the spatio-temporal field function.

[0049] Other steps and parameters are the same as those in any one of the specific implementation manners one to three.

[0050] Specific implementation manner five: The difference between this implementation manner and any one of the specific implementation manners one to four is that: in step 112, a time-varying signed distance function TVSDF is constructed based on the time encoding function; the specific process is as follows:

[0051] A time-varying signed distance function (Time-Varying Signed Distance Function, TVSDF) is proposed to represent the time-varying geometric field G;

[0052] The time-varying signed distance function takes the three-dimensional spatial coordinates p = (x, y, z) in the spatial rectangular coordinate system and the time encoding feature Φ i of the i-th remote sensing image at the observation time t i as the input, and outputs the signed distance function value g i corresponding to the position p at the moment t i , as shown in Equations (1) and (2):

[0053] G: TVSDF(p, Φ i ) → g i (1)

[0054]

[0055] Among them, G represents a time-varying geometric field;

[0056] Ω i represents the space occupied by ground objects in three-dimensional space at time t; i i

[0057] γ i represents the ground object surface at time t; i i

[0058] q represents the coordinates of a point on the ground object surface at time t; i i

[0059] ξ(p) represents an indicator function.

[0060] Other steps and parameters are the same as those in any one of the specific embodiments 1 to 4.

[0061] Specific embodiment 6: The difference between this embodiment and any one of the specific embodiments 1 to 5 is that: in step 113, a time-varying spectral radiance function TVSRF is constructed based on a time encoding function and a time-varying signed distance function TVSDF; the specific process is as follows:

[0062] A time-varying spectral radiance function (Time-Varying Spectrum Radiance Function, TVSRF) is proposed to represent the time-varying radiation field R.

[0063] The time-varying spectral radiance function TVSRF takes the three-dimensional space coordinate p, the time encoding feature Φ i , the image viewing direction v i (the viewing angle of the camera for the image, an inherent property of the image), and the output g of the time-varying geometric function i as inputs, and outputs the radiation value r i that can be obtained by observing the scene at this position along the viewing direction at this time, as shown in Equation (3):

[0064] R: TVSRF(g i , p, Φ i , v i ) → r i (3)

[0065] Among them, R represents the time-varying radiation field, and r i represents the spectral channel intensity value (taking the tagged p, Φ i , v i , g i as the input of the neural network, and the tagged spectral channel intensity value r i as the output of the neural network to obtain a trained neural network; p, Φi , v i , g i Four known quantities are input into the trained neural network, and the trained neural network outputs the predicted spectral channel intensity value r i ).

[0066] Other steps and parameters are the same as those in any one of the first to fifth specific embodiments.

[0067] Specific Embodiment Seven: The difference between this embodiment and any one of the first to sixth specific embodiments is that in step 1-14, a spatio-temporal field function STF (Spatio-Temporal Function, STF) is constructed based on the time-varying signed distance function TVSDF and the time-varying spectral radiation function TVSRF, as shown in Equation (4):

[0068] F: STF(p, v i , Φ i ) → (g i , r i ) (4)

[0069] Among them, F represents the spatio-temporal field.

[0070] Other steps and parameters are the same as those in any one of the first to sixth specific embodiments.

[0071] Specific Embodiment Eight: The difference between this embodiment and any one of the first to seventh specific embodiments is that in step 1-2, the spatio-temporal field function is initialized; the specific process is as follows:

[0072] After the representation method of the spatio-temporal field function is determined, the spatio-temporal field function can be initialized according to the scene range to be reconstructed in the time-varying three-dimensional reconstruction, which mainly includes two parts: scene space normalization and initialization of the time-varying signed distance function. For the time-varying spectral radiation function, the time-varying spectral radiation function is randomly initialized;

[0073] Initialization of the time-varying spectral radiation function: Randomly initialize the time-varying spectral radiation function;

[0074] Scene space normalization: Initially delimit the scene space range of the area to be reconstructed according to the geographical coordinates. To facilitate the unification of coordinates and subsequent data operations, the coordinate values of the scene area are normalized to the range of [0, 1], construct the space range of the spatio-temporal field coordinate system, and stipulate that the coordinate system in this coordinate space is O xyz Spatial rectangular coordinate system.

[0075] Scene space normalization: Normalize the coordinate values of the scene area to the range of [0, 1], construct the space range of the spatio-temporal field coordinate system, and stipulate that the spatio-temporal field coordinate system is O xyz Spatial rectangular coordinate system;

[0076] Initialization of time-varying signed distance function: Set the initial geometric form of the scene, that is, initialize the form distribution of the initial time-varying signed distance function TVSDF as a set of isosurfaces parallel to the x-y plane of the space rectangular coordinate system:

[0077] TVSDF(x,y,z,t i )=(g max -g min )×z+g min ,i∈{1...N} (6)

[0078] where x, y, and z represent the three-axis coordinate values of the space rectangular coordinate system, 0 ≤ x ≤ 1, 0 ≤ y ≤ 1, 0 ≤ z ≤ 1;

[0079] i represents the i-th frame, and N represents the total number of frames;

[0080] g max 、g min are preset hyperparameters, meaning the maximum and minimum g i values that the TVSDF field has at the initial moment;

[0081] Compared with traditional methods such as ellipsoids, this initialization method makes the initial form of the time-varying geometric function closer to the geometric characteristics of the real remote sensing scene, which is more conducive to fitting the geometric form of the remote sensing scene.

[0082] Other steps and parameters are the same as those in any one of the first to seventh specific implementation manners.

[0083] Specific implementation manner nine: The difference between this implementation manner and any one of the first to eighth specific implementation manners is that in step two, the imaging geometric model of the remote sensing image is mapped to the spatio-temporal field coordinate system; the parameters of the imaging geometric model of the remote sensing image in the spatio-temporal field coordinate system are optimized to obtain the imaging geometric model of the remote sensing image with the optimal parameters in the spatio-temporal field coordinate system;

[0084] The specific process is as follows:

[0085] In order to generate ray samples in the spatio-temporal field based on the super-generalized stereo image pair and construct geometric constraints, it is necessary to reconstruct the imaging geometric models of the spaceborne image and the airborne image in the spatio-temporal field based on the principle of spatial consistency, which mainly includes two tasks: the normalization of the parameter representation form of the imaging geometric model and the parameter optimization strategy.

[0086] Step 2-1. Map the imaging geometric model of the remote sensing image to the spatio-temporal field coordinate system; the specific process is as follows:

[0087] Since some scholars have analyzed and verified the feasibility of approximating the spaceborne RFM imaging model with a pinhole imaging model, the present invention adopts the pinhole model as the normalized representation form of the imaging geometric model parameters, approximating the spaceborne image imaging geometric model as a pinhole model, so as to keep the form unified with the airborne imaging geometric model and facilitate the implementation of backpropagation based on a unified neural network structure to optimize the model parameters, as shown in Equation (7). Given that the spatio-temporal field coordinate system is a normalized rectangular coordinate system, first map the latitude-longitude coordinate system corresponding to the original imaging geometric models of the spaceborne image and the airborne image to a transitional rectangular coordinate system, such as the topocentric coordinate system, etc., and then perform normalization processing on the transitional rectangular coordinate system and map it to the spatio-temporal field coordinate system.

[0088] Map the latitude-longitude coordinate system corresponding to the remote sensing image imaging geometric model (RFM(lat, lon, alt)) to a transitional rectangular coordinate system, such as the topocentric coordinate system, etc., and then perform normalization processing on the transitional rectangular coordinates and map it to the spatio-temporal field coordinate system; it is expressed as:

[0089] RFM(lat, lon, alt) = (u, v) → P(x, y, z, 1) = k(u, v, 1) (7)

[0090] Where lat represents latitude, lon represents longitude, alt represents altitude, (u, v) represents two-dimensional pixel coordinates, (x, y, z) are the spatial position coordinates in the spatio-temporal field coordinate system, P is the projection matrix representing the perspective imaging of the pinhole camera, and k is a constant coefficient;

[0091] RFM(lat, lon, alt) represents the latitude, longitude, and altitude of the remote sensing image imaging geometric model in the latitude-longitude coordinate system;

[0092] Step Two: Optimize the remote sensing image imaging geometric model parameters in the spatio-temporal field coordinate system to obtain the remote sensing image imaging geometric model with the optimal parameters in the spatio-temporal field coordinate system; the specific process is as follows:

[0093] The transformation of the model form, the transformation of the coordinate system, and the parameter accuracy error of the model itself can all lead to geometric mapping deviations in the spatio-temporal field imaging geometric model. After the representation forms of the imaging geometric model parameters are unified, set a parameter optimization strategy that maintains spatial consistency, that is, set the model parameters as learnable parameters, and gradually optimize the imaging geometric model parameters by minimizing the loss function in the subsequent spatio-temporal field function fitting process, so as to solve the geometric constraint deviation problem caused by the differences in the imaging geometric models.

[0094] Suppose N remote sensing images Ψ = {I i |I 1 ,..., I N} and the imaging geometric model parameters Θ cam= {P i | i = 1...N}, the optimization of the imaging geometric model parameters is expressed as:

[0095]

[0096] Where

[0097] Loss patch represents the projection consistency loss, and Loss color represents the pixel consistency loss;

[0098] Θ cam * represents the optimal parameters of the imaging geometric model; P i represents the spatial position coordinates of the i-th remote sensing image in the spatio-temporal field coordinate system;

[0099] I i represents the i-th remote sensing image; I 1 represents the first remote sensing image; I N represents the N-th remote sensing image.

[0100] Other steps and parameters are the same as those in the first to eighth specific embodiments.

[0101] Specific embodiment ten: The difference between this embodiment and one of the first to ninth specific embodiments is that the projection consistency loss is as shown in Equation (9);

[0102] The projection consistency loss is a calculation for optimizing the imaging parameters by improving the consistency between the image patches (patches) obtained by projecting the same spatial point onto each image;

[0103]

[0104] Where

[0105] patch(I i (p)) represents the image patch corresponding to the three-dimensional spatial coordinate p in the i-th remote sensing image I i (each remote sensing image I i is pre-cut into blocks of the same size);

[0106] patch(I j (p)) represents the image patch corresponding to the three-dimensional spatial coordinate p in the j-th remote sensing image I j (each remote sensing image I j is pre-cut into blocks of the same size);

[0107] NCC(·) represents calculating the similarity between image patches through normalized covariance;

[0108] Losspatch Denotes the projection consistency loss;

[0109] The pixel consistency loss is shown in Equation (10);

[0110] Pixel consistency loss Loss color Optimize the imaging parameters by optimizing the difference between the view synthesized by the spatio-temporal field and the input view;

[0111]

[0112] Wherein, Denotes the predicted view rendered by the spatio-temporal field function according to the imaging model, Loss color Denotes the pixel consistency loss, |||| 2 Denotes the square of the norm;

[0113] The said The acquisition process is:

[0114] Based on the ambient light E i , the illumination intensity distribution map S i , the object albedo A i , construct the predicted view of the remote sensing image I i The expression is: The expression is:

[0115]

[0116] The said ambient light E i The acquisition process is:

[0117] Input the remote sensing image into the convolutional neural network CNN, and the convolutional neural network CNN outputs the feature vector F in the image sun ;

[0118] Read the image parameter file to obtain the solar elevation angle, and calculate the solar direction v based on the solar elevation angle sun ;

[0119] Input the feature vector F sun , the solar direction v sun and the time feature Φ i into the multi-layer perceptron network MLP, and the multi-layer perceptron network MLP outputs the global ambient light variable E at time t i ; as shown in the following formula: i ; as shown in the following formula:

[0120] E i = MLP(F sun , v sun , Φ i ).

[0121] The other steps and parameters are the same as those in the first to ninth specific embodiments.

[0122] The present invention may also have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art can make various corresponding changes and modifications according to the present invention. However, these corresponding changes and modifications should fall within the protection scope of the appended claims of the present invention.

Claims

1. A four-dimensional space-time field reconstruction method based on phase-shift time coding and imaging space consistency, characterized by: The specific process of the method is: Step 1: Construct space-time field function; Step 2: Map the remote sensing image imaging geometric model to the space-time field coordinate system; Optimize the parameters of the remote sensing image imaging geometric model in the space-time field coordinate system to obtain the remote sensing image imaging geometric model with the optimal parameters in the space-time field coordinate system; Step 3: Based on the constructed space-time field function and the remote sensing image imaging geometric model with optimal parameters in the space-time field coordinate system, complete the reconstruction of the four-dimensional space-time field.

2. The four-dimensional space-time field reconstruction method based on phase shift time coding and imaging space consistency according to claim 1 is characterized in that: In step 1, a space-time field function is constructed; the specific process is as follows: Step 1: Modeling of space-time field function; Step 1 and 2: Initialize the space-time field function.

3. The four-dimensional space-time field reconstruction method based on phase shift time coding and imaging space consistency according to claim 2 is characterized in that: The space-time field function modeling in step 1 is as follows: Step 111, construct a time encoding function; Step 112: Based on the time coding function, construct a time-varying signed distance function TVSDF; Step 113: construct a time-varying spectral radiation function TVSRF based on the time coding function and the time-varying signed distance function TVSDF; Step 114: Construct the space-time field function STF based on the time-varying signed distance function TVSDF and the time-varying spectral radiance function TVSRF.

4. The four-dimensional space-time field reconstruction method based on phase shift time coding and imaging space consistency according to claim 4 is characterized in that: In the steps one by one, a time coding function is constructed; the specific process is as follows: In the formula, Represents a vector connector; Represent the normalized results of year, day within the year, and hour within the day respectively; Y number Indicates the current year, Y min Indicates the minimum year of the input time series, Y max Indicates the maximum year of the input time series; M number is the current month; M sum is the total number of observable days in a month; D number Corresponding to the day of the month, Y sum is the number of observable days in a year; H number is the current hour, H sum is the total number of observable hours in a day.

5. The four-dimensional space-time field reconstruction method based on phase shift time coding and imaging space consistency according to claim 4 is characterized in that: In the step 112, a time-varying signed distance function TVSDF is constructed based on the time coding function; The specific process is: The time-varying signed distance function is based on the three-dimensional spatial coordinates p = (x, y, z) in the spatial rectangular coordinate system and the observation time t of the i-th remote sensing image. i The temporal encoding feature Φ i is the input, and the output is at t i The signed distance function value g corresponding to the position at time p i , as shown in formula (1) and formula (2): G:TVSDF(p,φ i )→g i (1) in, Where G represents the time-varying geometric field; Ω i Indicates t i The space occupied by objects in three-dimensional space at the time; γ i Indicates t i The surface of the terrain at the moment; q means t i The coordinates of a point on the surface of the object at the time; ξ(p) represents the indicator function.

6. The four-dimensional space-time field reconstruction method based on phase shift time coding and imaging space consistency according to claim 5 is characterized in that: In the step 113, a time-varying spectral radiation function TVSRF is constructed based on the time coding function and the time-varying signed distance function TVSDF; the specific process is: The time-varying spectral radiation function TVSRF is encoded with three-dimensional spatial coordinates p and time characteristics Φ i , image viewing direction v i , and the output of the time-varying geometric function g i is the input, and the output is r i , as shown in formula (3): R:TVSRF(g i ,p,Φ i ,v i )→r i (3) Where R represents the time-varying radiation field, r i Indicates the spectral channel intensity value.

7. The four-dimensional space-time field reconstruction method based on phase shift time coding and imaging space consistency according to claim 6 is characterized by: In the step 114, based on the time-varying signed distance function TVSDF and the time-varying spectral radiance function TVSRF, a space-time field function STF is constructed, as shown in formula (4): F:STF(p,v i ,φ i )→(h i ,r i ) (4) Among them, F represents the space-time field.

8. The four-dimensional space-time field reconstruction method based on phase-shift time coding and imaging space consistency according to claim 7 is characterized in that: The space-time field function is initialized in step 1 and step 2; the specific process is: Time-varying spectral radiation function initialization: randomly initialize the time-varying spectral radiation function; Normalization of scene space: Construct the spatial range of the space-time field coordinate system and define the space-time field coordinate system as O xyz Space rectangular coordinate system; Time-varying signed distance function initialization: Set the initial geometric shape of the scene, that is, initialize the shape distribution of the initial time-varying signed distance function TVSDF to a set of isosurfaces parallel to the xy plane of the spatial rectangular coordinate system: TVSDF(x,y,z,t i )=(g max -g min )×z+g min ,i∈{1...N} (6) Among them, x, y, z represent the three-axis coordinate values ​​of the spatial rectangular coordinate system, 0≤x≤1, 0≤y≤1, 0≤z≤1; i represents the i-th frame, and N represents the total number of frames; g max , g min To set the hyperparameters in advance.

9. The four-dimensional space-time field reconstruction method based on phase-shift time coding and imaging space consistency according to claim 8 is characterized in that: In the step 2, the remote sensing image imaging geometric model is mapped to the space-time field coordinate system; the parameters of the remote sensing image imaging geometric model in the space-time field coordinate system are optimized to obtain the remote sensing image imaging geometric model with the optimal parameters in the space-time field coordinate system; The specific process is: Step 21: Map the remote sensing image imaging geometric model to the space-time field coordinate system; the specific process is: The latitude and longitude coordinate system corresponding to the remote sensing image imaging geometric model is mapped to the transition rectangular coordinate system, and then the transition rectangular coordinate is normalized and mapped to the space-time field coordinate system; it is expressed as: RFM(lat,lon,alt)=(u,v)→P(x,y,z,1)=k(u,v,1)(7) Among them, lat represents latitude, lon represents longitude, alt represents elevation, (u,v) represents two-dimensional pixel coordinates, (x,y,z) represents the spatial position coordinates in the space-time field coordinate system, P is the projection matrix representing the perspective imaging of the pinhole camera, and k is a constant coefficient; RFM (lat, lon, alt) represents the latitude, longitude, and elevation of the remote sensing image imaging geometric model in the latitude and longitude coordinate system; Step 22: Optimize the parameters of the remote sensing image imaging geometric model in the space-time field coordinate system to obtain the remote sensing image imaging geometric model with the optimal parameters in the space-time field coordinate system; the specific process is: Assume that N remote sensing images Ψ={I i |I1,...,I N } and the imaging geometry model parameters Θ corresponding to the remote sensing image cam = {P i |i=1...N}, the optimization of imaging geometric model parameters is expressed as: in, Loss patch Represents projection consistency loss, Loss color represents pixel consistency loss; Θ cam * represents the optimal parameters of the imaging geometric model; P i Represents the spatial position coordinates of the i-th remote sensing image in the space-time field coordinate system; Ii represents the i-th remote sensing image; I1 represents the first remote sensing image; I N Represents the Nth remote sensing image.

10. The four-dimensional space-time field reconstruction method based on phase shift time coding and imaging space consistency according to claim 9, characterized in that: The projection consistency loss is shown in formula (9); in, patch(I i (p)) represents the three-dimensional space coordinate p in the i-th remote sensing image I i The corresponding image block in patch(I j (p)) represents the three-dimensional space coordinate p in the jth remote sensing image I j The corresponding image block in NCC(·) means the similarity between image patches is calculated by normalized covariance; Loss patch represents the projection consistency loss; The pixel consistency loss is shown in formula (10); in, Represents the prediction view, Loss color represents pixel consistency loss, || || 2 represents the square of the norm; Said The acquisition process is: Based on ambient light E i , light intensity distribution diagram S i , object albedo A i , construct remote sensing image I i Forecast view The expression is: The ambient light E i The acquisition process is: The remote sensing image is input into the convolutional neural network CNN, and the convolutional neural network CNN outputs the feature vector F in the image sun ; Read the image parameter file to obtain the sunlight pitch angle, and calculate the sunlight direction v based on the sunlight pitch angle sun ; The feature vector F sun 、Direction of sunlight v sun and time characteristics | i Input into the multilayer perceptron network MLP, the multilayer perceptron network MLP output t i The global ambient light variable E at the moment i ; As shown below: E i =MLP(F sun ,v sun ,φ i )。

Images