A spatiotemporal integrated modeling system and method based on 4D temporal information

Abstract

This application relates to the field of image processing technology and discloses a spatiotemporal integrated modeling system and method based on 4D temporal information. This application performs spatial registration and temporal synchronization on spatiotemporal sensing data to generate a 4D dataset; constructs a four-dimensional spatiotemporal representation based on the 4D dataset, accurately separates and extracts guiding information that spatial features dominate temporal changes, and constraint information that temporal features limit spatial applications, generating spatiotemporal coupling features; establishes spatial neighborhood relationships and state transfer relationships to form a heterogeneous graph; performs directional inference operations on the heterogeneous graph through a graph attention network, outputting the spatiotemporal state covering the target scene; maps the spatiotemporal state to a continuous spatiotemporal field through implicit continuous representation, outputting the reconstruction result of the target scene at arbitrary spatiotemporal coordinates, effectively expanding the applicability and implementation effect of spatiotemporal modeling technology in practical industrial scenarios such as smart warehousing, site operation and maintenance, and situational awareness.

Description

Technical Field

[0001] This application relates to the field of image processing technology, specifically to a spatiotemporal integrated modeling system and method based on 4D temporal information. Background Technology

[0002] In the fields of smart industry and warehousing operation and maintenance, digital modeling technology for physical scenarios has formed two mainstream development paths: static spatial structure modeling technology and dynamic temporal evolution analysis technology. Static spatial modeling mainly uses methods such as point mapping and regional grid division to complete the digital replication of solid spatial information such as building structure, fixed facilities and functional zoning within the target scenario, realizing the static digital restoration of the physical layout of the scenario. Dynamic temporal analysis, on the other hand, uses methods such as time series fitting and data trend analysis to conduct pattern analysis and trend prediction on dynamic data such as the flow of goods, equipment operation and environmental status within the scenario that change over time.

[0003] As the demand for integrated management and control of complex scenarios continues to increase, the industry has begun to explore the combined application of spatial data and temporal data. The mainstream implementation method is to simply concatenate the two types of data to complete shallow linkage. At the same time, conventional network modeling, basic linear interpolation reconstruction and other technologies are also gradually being applied to the restoration of spatiotemporal information in scenarios to meet the basic situation expression needs of low-complexity scenarios. They have been initially implemented and used in scenarios such as simple parks and small storage sites.

[0004] Traditional modeling schemes generally separate spatial and temporal dimension operations and carry them out independently, only realizing unidirectional information transmission within a single dimension. They cannot build a four-dimensional integrated spatiotemporal representation architecture that can simultaneously carry the joint operation of spatial and temporal information. Traditional spatiotemporal graph modeling schemes can only construct a single type of topological relationship and cannot simultaneously build a dual topological structure of spatial and temporal relationships according to the actual operating logic of the scene. Existing technologies rely solely on discrete sampling points to perform basic linear interpolation to complete the continuous extension of data, resulting in problems such as spatial distortion and temporal evolution deviation.

[0005] In view of the shortcomings of the aforementioned prior art, the technical problem to be solved by this application is how to process 4D temporal information to achieve accurate spatiotemporal integrated modeling of the target scene. Summary of the Invention

[0006] To address the shortcomings of existing technologies, the purpose of this application is to provide a spatiotemporal integrated modeling system and method based on 4D temporal information, so as to effectively expand the application scope and implementation effect of spatiotemporal modeling technology in practical industrial scenarios such as smart warehousing, site operation and maintenance and situation control.

[0007] To achieve the above objectives, this application adopts the following technical solution:

[0008] Firstly, this application provides a spatiotemporal integrated modeling method based on 4D temporal series information, including:

[0009] Acquire spatiotemporal perception data of the target scene, perform time synchronization and spatial registration on the spatiotemporal perception data, and generate a 4D dataset;

[0010] A four-dimensional spatiotemporal representation is constructed on the 4D dataset. Bidirectional information interaction is carried out simultaneously along the spatial and temporal dimensions on the four-dimensional spatiotemporal representation. The guiding information of spatial features on temporal evolution and the constraint information of temporal features on spatial structure are extracted. Spatiotemporal coupling features are generated through bidirectional fusion.

[0011] Based on the spatiotemporal coupling feature, a heterogeneous graph including spatial neighborhood relations and state transit relations is constructed. Inference is performed on the heterogeneous graph through a graph attention network to output the spatiotemporal state of the target scene.

[0012] The spatiotemporal state is mapped to a continuous spatiotemporal field through implicit continuous representation, and the reconstruction result of the target scene at arbitrary spatiotemporal coordinates is output.

[0013] Furthermore, the generation of the 4D dataset includes:

[0014] Acquire spatiotemporal perception data of the target scene, classify it according to spatial granularity and temporal change rate, and distinguish between static spatial data and dynamic temporal data of the target scene;

[0015] Fixed entities in the target scene are selected as spatial references, and periodic fluctuation characteristics of the target scene are selected as time references. Spatial registration is performed on static spatial data, and time synchronization is performed on dynamic time series data.

[0016] The statistical spatial registration residuals and temporal offset residuals are analyzed and iteratively corrected. The corrected static spatial data and dynamic temporal data are then fused and encapsulated according to the spatial and temporal dimensions to generate a 4D dataset.

[0017] Furthermore, the spatial registration refers to selecting fixed entities within the target scene whose deformation rate is less than a preset deformation threshold as spatial references, and matching and correcting the spatial coordinates of different sampling devices through an iterative nearest-point algorithm;

[0018] The time synchronization refers to extracting the periodic fluctuation characteristics within the target scene as a time reference, and correcting the timestamp deviation of data with different sampling frequencies through time delay estimation.

[0019] Furthermore, the generated spatiotemporal coupling features include:

[0020] Spatial grid partitioning and temporal coding mapping are performed on the 4D dataset to construct a four-dimensional spatiotemporal representation;

[0021] In the four-dimensional spatiotemporal representation, information is transmitted along the spatial dimension to the corrected static spatial data, and information is permeated along the temporal dimension to the corrected dynamic temporal data, so as to achieve two-way information interaction.

[0022] Extracting information on how spatial features guide temporal evolution and how temporal features constrain spatial structures;

[0023] Based on the dynamic allocation and fusion ratio of the grid deformation coefficient and the temporal fluctuation coefficient, the guiding information and constraint information are spliced ​​together across dimensions to generate spatiotemporal coupled features.

[0024] Furthermore, the four-dimensional spatiotemporal representation involves dividing the 4D dataset into equidistant spatial grids, matching the spatial grids with time-delay-estimated and corrected temporal codes, and forming a four-dimensional representation matrix corresponding to the spatial grids and the temporal codes.

[0025] The bidirectional information interaction is achieved by traversing different spatial grids through spatial convolution to transmit information in the spatial dimension, and by recursive deduction based on the periodic fluctuation characteristics to achieve information penetration in the time dimension.

[0026] Furthermore, the spatiotemporal state of the output target scene includes:

[0027] The basic structure of heterogeneous graphs is constructed using spatial grids as graph nodes and spatiotemporal coupling features as node features.

[0028] Spatial neighborhood edges are constructed based on the physical distance between spatial grids, and temporal state transfer edges are constructed based on the periodic fluctuation characteristics, forming a heterogeneous graph topology that includes spatial neighborhood relationships and state transfer relationships;

[0029] Spatial topological weights are configured based on the mesh deformation coefficient, and temporal evolution weights are configured based on the temporal fluctuation coefficient. Feature inference is then performed on spatial neighborhood edges and state propagation edges on the heterogeneous graph through a graph attention network.

[0030] Based on the inference results, the node features of the graph nodes are iteratively corrected, and the corrected node features of all graph nodes are integrated to output the spatiotemporal state of the target scene.

[0031] Furthermore, the heterogeneous graph uses spatial grids bound with time-series codes as graph nodes, and assigns the spatiotemporal coupling features corresponding to the spatial grids as node features; the feature inference performed on the spatial neighborhood edges is to perform weighted aggregation of the guiding information of adjacent spatial grids according to the spatial topology weights, and complete the feature deviation calibration of adjacent spatial grids under the same time dimension; the feature inference performed on the state-transfer edges is to complete the recursive mapping of constraint information according to the arrangement order of periodic fluctuation features, and correct the evolutionary offset of different constraint information according to the temporal evolution weights.

[0032] Furthermore, the reconstruction result of the target scene at any spatiotemporal coordinates includes:

[0033] Extract the spatiotemporal state corresponding to each spatial grid, and use the physical coordinates and temporal codes of the spatial grid as implicit query variables to construct a spatiotemporal sample set;

[0034] The physical coordinates of the spatial grid within the spatiotemporal sample set are arranged in an ordered manner, and the temporal codes are sorted temporally according to the periodic fluctuation characteristics to generate a spatiotemporal sequence.

[0035] By performing coordinate interpolation to supplement points on the spatiotemporal sequence, and expanding it to generate continuous spatiotemporal coordinates, an implicit continuous representation is obtained;

[0036] Based on continuous spatiotemporal coordinates, all spatiotemporal states are integrated to fit and generate a continuous spatiotemporal field;

[0037] Within a continuous spatiotemporal field, arbitrary spatiotemporal coordinates are selected, and the spatiotemporal state corresponding to the spatiotemporal coordinates is solved through neighborhood grid weighted operation, outputting the reconstruction result of the target scene.

[0038] Furthermore, the implicit continuous representation uses the physical coordinates of the spatial grid as spatial input and the temporal encoding as temporal input to fill in the missing spatiotemporal samples;

[0039] The continuous spatiotemporal field is obtained by combining the grid deformation coefficient to correct spatial interpolation distortion and by combining the temporal fluctuation coefficient to compensate for temporal evolution offset.

[0040] The reconstruction result of the target scene is to retrieve the neighboring spatial grids of any spatiotemporal coordinates, assign weights according to the spatiotemporal coordinate distance, aggregate the spatiotemporal states corresponding to the neighboring spatial grids, and solve for the reconstruction result under the corresponding spatiotemporal coordinates; the spatiotemporal coordinate distance includes the physical distance between spatial grids and the time interval between temporal codes.

[0041] Secondly, this application provides a spatiotemporal integrated modeling system based on 4D temporal information, including: a perception module, a fusion module, an image analysis module, and a mapping module;

[0042] The perception module is used to acquire spatiotemporal perception data of the target scene, perform time synchronization and spatial registration on the spatiotemporal perception data, and generate a calibrated 4D dataset.

[0043] The fusion module is used to construct a four-dimensional spatiotemporal representation of the 4D dataset. It simultaneously performs bidirectional information interaction along the spatial and temporal dimensions on the four-dimensional spatiotemporal representation, extracts the guiding information of spatial features on temporal evolution and the constraint information of temporal features on spatial structure, and generates spatiotemporal coupled features through bidirectional fusion.

[0044] The graph analysis module constructs a heterogeneous graph based on spatiotemporal coupling features, including spatial neighborhood relationships and state transit relationships. It then performs inference on the heterogeneous graph through a graph attention network and outputs the spatiotemporal state of the target scene.

[0045] The mapping module is used to map the spatiotemporal state into a continuous spatiotemporal field through implicit continuous representation, and output the reconstruction result of the target scene in arbitrary spatiotemporal coordinates.

[0046] Compared with the prior art, the beneficial effects achieved by this application are as follows:

[0047] Spatial registration and temporal synchronization of spatiotemporal perception data are performed to generate a 4D dataset, laying a solid data foundation for subsequent full-process technology implementation. Based on the 4D dataset, a four-dimensional spatiotemporal representation is constructed to open up a two-way information interaction channel between the spatial and temporal dimensions. The guiding information that spatial features dominate temporal changes and the constraint information that temporal features limit spatial applications are accurately separated and extracted to generate spatiotemporal coupling features, effectively improving the scene adaptability and information expression completeness of spatiotemporal fusion features.

[0048] By establishing spatial neighborhood relationships and state transfer relationships, a heterogeneous graph adapted to complex scenarios is formed. Targeted reasoning operations are performed on the heterogeneous graph through a graph attention network to output the spatiotemporal state covering the target scenario, significantly improving the completeness of the representation of the overall scenario situation and the accuracy of inference.

[0049] By mapping spatiotemporal states to continuous spatiotemporal fields through implicit continuous representation, the reconstruction results of the target scene at arbitrary spatiotemporal coordinates are output. The real-world operating status corresponding to any spatial location and any time node within the scene can be flexibly obtained, effectively expanding the application scope and implementation effect of spatiotemporal modeling technology in practical industrial scenarios such as smart warehousing, site operation and maintenance and situational control. Attached Figure Description

[0050] Figure 1 This is a flowchart of a spatiotemporal integrated modeling method based on 4D temporal information.

[0051] Figure 2 A flowchart for generating spatiotemporal coupling features;

[0052] Figure 3 This is a schematic diagram of a spatiotemporal integrated modeling system based on 4D temporal information. Detailed Implementation

[0053] The technical solution of this application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments and specific features in the embodiments are detailed descriptions of the technical solution of this application, rather than limitations thereof.

[0054] Example 1

[0055] like Figure 1 As shown, this embodiment provides a spatiotemporal integrated modeling method based on 4D temporal information, including:

[0056] S1. Acquire spatiotemporal perception data of the target scene, perform time synchronization and spatial registration on the spatiotemporal perception data, and generate a 4D dataset.

[0057] Specifically, generating the 4D dataset includes:

[0058] Acquire spatiotemporal perception data of the target scene, classify it according to spatial granularity and temporal change rate, and distinguish between static spatial data and dynamic temporal data of the target scene;

[0059] Fixed entities in the target scene are selected as spatial references, and periodic fluctuation characteristics of the target scene are selected as time references. Spatial registration is performed on static spatial data, and time synchronization is performed on dynamic time series data.

[0060] The statistical spatial registration residuals and temporal offset residuals are analyzed and iteratively corrected. The corrected static spatial data and dynamic temporal data are then fused and encapsulated according to the spatial and temporal dimensions to generate a 4D dataset.

[0061] Spatial registration refers to selecting fixed entities within the target scene whose deformation rate is less than a preset deformation threshold as spatial references, and matching and correcting the spatial coordinates of different sampling devices through an iterative nearest point algorithm.

[0062] Time synchronization refers to extracting the periodic fluctuation characteristics within the target scene as a time reference and correcting the timestamp deviation of data with different sampling frequencies through time delay estimation.

[0063] After acquiring basic spatiotemporal perception data by deploying spatial positioning sensors, goods status sampling equipment, and environmental monitoring equipment in the factory warehouse, the first step is to classify and categorize the warehouse spatiotemporal perception data. The perception data acquired inside the factory warehouse includes both fixed building layout information and dynamic flow information of stored goods. The stability of the two types of data and their temporal variation patterns are fundamentally different. If a unified calibration operation is carried out directly without differentiation, the warehouse spatial coordinate correction process and the goods temporal status alignment process will interfere with each other, making it impossible to achieve targeted and accurate calibration.

[0064] Spatial granularity range refers to the actual area of ​​the warehouse area covered by a single set of warehouse spatial perception data and the fine level of warehouse space information division. Temporal change rate refers to the speed at which the state of goods and the state of the warehouse environment change over time. In the factory warehouse scenario, a unified quantitative division standard is set, defining 3m as the spatial granularity division boundary for fine warehouse storage, 15m as the spatial granularity division boundary for large-scale warehouse aggregation, and setting 1 minute as the judgment threshold for low-speed temporal change in the warehouse and 10s as the judgment threshold for high-speed temporal change in the warehouse.

[0065] Based on the above quantitative standards, data attributes were identified. Sensing data with a fixed spatial location and a temporal state fluctuation interval exceeding 1 minute were classified as static spatial data, while sensing data with a fixed spatial location and a self-state change interval of less than 10 seconds were classified as dynamic temporal data.

[0066] For example, the structural elements of the warehouse, such as load-bearing storage columns, fixed multi-layer shelving, and warehouse floor markings, remain unchanged in position for a long period of time. The corresponding data spatial granularity is stably maintained at the 3m level, with no short-term state changes, and is uniformly classified as static spatial data. The real-time location data of goods stacking in the warehouse, the operating status data of automated handling equipment, and the data on the flow of goods entering and leaving the warehouse are updated frequently, with the interval between state changes generally within 10 seconds, and are uniformly classified as dynamic time-series data.

[0067] This application sorts out two types of heterogeneous sensing data in the warehouse from the data source, clarifies the processing objects corresponding to spatial registration and time synchronization operations in the warehouse scenario, defines a clear data processing scope for subsequent selection of warehouse spatiotemporal references for directional calibration, and determines the corresponding execution relationship between warehouse static spatial data and spatial references and warehouse dynamic temporal data and time references.

[0068] After completing the classification of warehouse perception data attributes, the next stage is to select the warehouse spatiotemporal benchmark and implement multi-dimensional data calibration. The installation coordinates of sampling equipment deployed at different locations inside the factory warehouse are biased, and the data sampling frequency settings of various sampling equipment are not consistent. As a result, the final output warehouse spatiotemporal data has objective problems such as inconsistent spatial coordinate system and misaligned data collection time nodes. Warehouse data that has not undergone benchmark alignment processing cannot achieve full-area information linkage and integration in the warehouse area.

[0069] Fixed entities refer to rigid storage structures in factory warehouses that have stable internal structures and are unlikely to undergo deformation or displacement. The preset deformation threshold is a quantitative criterion for determining whether such structures can serve as a spatial benchmark for the entire warehouse area. Within the industrial warehousing implementation standard, 2% is set as the preset deformation threshold. Those skilled in the art can adjust the preset deformation threshold accordingly in the actual process. During implementation, only structures with an internal deformation rate of less than 2% are selected as a unified spatial benchmark.

[0070] Within the warehouse scenario, the iterative nearest point algorithm is used to complete the coordinate matching and coordinate system unification of the spatial point information output by sampling devices in different warehouse areas. Specifically, based on multiple iterations of the iterative nearest point algorithm, the coordinate normalization and correction of all static spatial data in the warehouse is completed. Simultaneously, the construction of the warehouse time benchmark and the synchronization of dynamic time-series data are promoted. The periodic fluctuation characteristics refer to the production and warehousing patterns with fixed and repetitive operating rhythms within the warehouse, such as the rhythm of goods entering the warehouse at fixed time periods, the rhythm of timed warehouse inventory operations, and the start-up and shutdown rhythms of automated warehousing equipment. A unified time benchmark for the warehouse is constructed based on these regular characteristics.

[0071] Time delay estimation is mainly used to measure the time misalignment difference between multiple sets of dynamic time series data in the warehouse and to correct the deviation, so as to achieve unified correction of the timestamps of dynamic data output by warehouse sampling equipment with different sampling frequencies. For example, the main reinforced concrete load-bearing columns of the warehouse have long-term structural stability, and the actual measured deformation is only 1.1%, which is lower than the preset deformation threshold of 2%, meeting the selection conditions for warehouse spatial benchmarks and serving as a unified reference standard for spatial coordinate calibration of the entire warehouse area. The concentrated entry and exit rhythm of goods in the warehouse at fixed times in the morning, noon and evening can be used as a time benchmark for periodic fluctuations. The warehouse dynamic data output by some low-frequency sampling equipment initially has a time misalignment deviation of 1.3s, which is corrected by time delay estimation.

[0072] This application eliminates spatial coordinate deviations and temporal node deviations caused by cross-device data acquisition within the warehouse at the source, establishes a warehouse spatial reference system and a temporal reference system, completes the preliminary dimensional standard processing of the two types of basic warehouse data, and provides data samples for subsequent warehouse data residual statistics and iterative correction work.

[0073] After the initial spatial registration and temporal synchronization of warehouse data are completed, the statistical analysis and iterative correction of warehouse spatial registration residuals and temporal offset residuals are carried out. The warehouse spatiotemporal data after initial calibration based on the unified spatiotemporal benchmark will retain slight spatial location offsets and misalignments of goods status temporal nodes. These residual deviations will continue to propagate along the modeling process, gradually amplifying the computational errors in subsequent warehouse spatiotemporal feature extraction and warehouse spatiotemporal state inference, reducing the accuracy of the warehouse scenario modeling results.

[0074] Spatial registration residual refers to the offset value of the remaining spatial position of the warehouse static spatial data after the initial coordinate unification is completed, compared with the warehouse's established spatial reference. Temporal offset residual refers to the offset value of the remaining temporal node misalignment of the warehouse dynamic temporal data after the initial timestamp alignment is completed, compared with the warehouse's established time reference. In the actual process, a deviation control range is set, with 3cm set as the maximum allowable limit for warehouse spatial registration residual and 0.2s set as the maximum allowable limit for warehouse temporal offset residual. Those skilled in the art can adjust the deviation control range accordingly in the actual process.

[0075] Specifically, using the warehouse's established time-space benchmark as a fixed comparison reference, the residual deviation values ​​corresponding to all warehousing data in the warehouse area are statistically analyzed in batches. Deviation correction operations are repeatedly executed according to a unified correction logic, and continuous iterative adjustments are made until both types of residual values ​​fall within the preset limit range. For example, the data of shelf attachment points in the edge area of ​​the warehouse area are prone to an initial spatial registration residual of about 4.2cm after initial registration. After two rounds of iterative correction, the residual offset is reduced to 2.8cm, which meets the 3cm spatial deviation control standard. The high-frequency acquisition of goods handling equipment operation status data in the warehouse is prone to generating an initial time series offset residual of 0.4s. After multiple rounds of time series correction, the misalignment time is reduced to 0.16s, which meets the 0.2s time series deviation control requirement.

[0076] This application eliminates subtle dimensional deviations retained within the warehouse spatiotemporal data layer by layer, continuously improves the fit between the warehouse static layout data, the dynamic time-series data of goods, and the unified warehousing spatiotemporal benchmark, and completes multiple rounds of iterative deviation correction of the two types of warehouse spatiotemporal data. The spatiotemporal dimension matching degree reaches the standard for warehousing data integration and application, and has the practical conditions for carrying out fusion and encapsulation according to the established dimension rules.

[0077] After completing all deviation correction work for warehouse storage data, the two types of calibrated warehouse spatiotemporal data are subjected to dimensional fusion and encapsulation processing to finally generate a 4D dataset adapted to the storage scenario. The pre-processed warehouse static spatial data and dynamic temporal data only complete the accurate calibration and optimization of their own dimensions. The data storage architecture is independently separated and cannot be connected to the subsequent implementation process of constructing the four-dimensional spatiotemporal representation of the warehouse.

[0078] Using the warehouse space dimension that has been uniformly calibrated as the basic carrier of the overall data architecture, all static spatial layout data of the warehouse after residual correction are centrally collected. The warehouse time dimension after regularization and alignment is used as the temporal extension carrier of the spatial architecture. According to the 3m basic spatial granularity and 10s high-speed temporal update frequency previously defined for the warehouse, the fixed spatial location information of the warehouse and the dynamic flow information of goods within the corresponding time period are bound together. Finally, a 4D dataset covering the fixed storage structure information of the entire warehouse and the dynamic storage operation information of the warehouse at all times is formed.

[0079] From the perspective of practical warehouse applications, the integrated and packaged 4D dataset can completely restore the fixed spatial form of the entire warehouse, such as the shelf layout, aisle planning, and infrastructure layout, while also carrying dynamic time-series information such as the entry and exit of goods, equipment operation, and changes in goods stacking at all times.

[0080] This application realizes the integrated reorganization and integration of heterogeneous spatiotemporal perception data in factory warehouses, constructs a standard warehouse dataset with unified benchmark, standardized format, and clear dimensional hierarchy, and comprehensively completes all spatiotemporal data preprocessing related work in the warehouse scenario. The 4D dataset serves as a standardized input data source for the four-dimensional spatiotemporal representation construction stage, realizing seamless connection between the warehouse data preprocessing stage and the subsequent warehouse spatiotemporal feature extraction stage, and ensuring the entire set of 4D time-series spatiotemporal integrated modeling technology for factory warehouses.

[0081] S2. Construct a four-dimensional spatiotemporal representation of the 4D dataset. Simultaneously, perform bidirectional information interaction along the spatial and temporal dimensions on the four-dimensional spatiotemporal representation. Extract the guiding information of spatial features on temporal evolution and the constraint information of temporal features on spatial structure. Generate spatiotemporal coupled features through bidirectional fusion.

[0082] Specifically, such as Figure 2 As shown, the spatiotemporal coupling features generated include:

[0083] Spatial grid partitioning and temporal coding mapping are performed on the 4D dataset to construct a four-dimensional spatiotemporal representation;

[0084] In the four-dimensional spatiotemporal representation, information is transmitted along the spatial dimension to the corrected static spatial data, and information is permeated along the temporal dimension to the corrected dynamic temporal data, so as to achieve two-way information interaction.

[0085] Extracting information on how spatial features guide temporal evolution and how temporal features constrain spatial structures;

[0086] Based on the dynamic allocation and fusion ratio of the grid deformation coefficient and the temporal fluctuation coefficient, the guiding information and constraint information are spliced ​​together across dimensions to generate spatiotemporal coupled features.

[0087] Among them, the four-dimensional spatiotemporal representation is to divide the 4D dataset into equidistant spatial grids, match the spatial grids with time-delay estimation and correction of the temporal codes, and form a four-dimensional representation matrix corresponding to the spatial grids and the temporal codes.

[0088] Two-way information interaction achieves spatial dimension information transmission by traversing different spatial grids through spatial convolution, and achieves temporal dimension information penetration by recursive deduction based on periodic fluctuation characteristics.

[0089] Based on the 4D dataset preprocessed from the warehouse scene, spatial grid division and temporal coding mapping are first carried out to construct a four-dimensional spatiotemporal representation adapted to the warehousing scene. The warehouse 4D dataset obtained after the initial integration and encapsulation is a holistic global spatiotemporal data set. There are no standardized independent computing units, and it is impossible to carry out targeted information computing and temporal correlation matching for local storage areas of the warehouse. Only by decomposing the overall data into regular and unified spatial units and matching each spatial unit with a unified standard temporal identifier can a basic spatiotemporal representation architecture supporting subsequent bidirectional information flow computing be built.

[0090] An equidistant spatial grid refers to an independent spatial computing unit formed after the physical storage space of the entire warehouse is equally divided according to a unified spatial scale; the time sequence code after time delay estimation and correction refers to the time sequence identifier formed after time delay estimation and time sequence deviation correction in the previous warehouse data synchronization process; the four-dimensional representation matrix is ​​a two-dimensional data matrix formed by combining the spatial coordinate information of the spatial grid with the corresponding bound time sequence code information according to the row and column regular arrangement rules.

[0091] This implementation selected a conventional medium-sized planar warehouse as the implementation platform. A uniform 3m fixed side length was used as the standard for equidistant spatial grid division throughout the entire area. The warehouse was divided into 20 horizontal columns and 15 vertical rows, resulting in a total of 300 identical spatial grids. The plane coordinates of the grid centers were sequentially arranged according to the reference origin. Typical grid center coordinates included fixed points such as (3,3)m, (3,6)m, and (15,12)m. The timing coding followed the previous timing synchronization correction accuracy standard, uniformly completing the timing deviation correction. A 10s increment was set as the basic timing coding progression unit, generating sequential timing codes. All codes completed a 0.1s timing deviation correction. The reference code corresponds to a cumulative timing value of 0s, the basic interval code corresponds to 10s, and the periodic node code corresponds to 60s. A set of timing codes was bound to each spatial grid within the warehouse. Finally, the grid plane layout coordinates were used as the row dimension of a matrix, and the bound timing codes were used as the column dimension, combining to construct a four-dimensional representation matrix covering the warehouse. The correspondence between all grids and codes was fixed and unified.

[0092] This application decomposes the overall 4D warehouse dataset into standardized, independently callable spatiotemporal basic units, unifying the spatial division scale and temporal identification specifications within the entire warehouse area, eliminating the problem of inconsistent data operation standards in local areas, and determining the basic implementation carrier for subsequent targeted information transmission and information penetration work based on spatial and temporal dimensions, respectively.

[0093] After completing the construction of the four-dimensional spatiotemporal representation, information is transmitted along the spatial dimension of the matrix system to the corrected static spatial data, and information is permeated along the temporal dimension to the corrected dynamic time-series data. This enables two-way spatiotemporal information interaction in the warehousing scenario. The static spatial layout data of the warehouse and the dynamic warehousing operation data stored within the four-dimensional spatiotemporal representation matrix are in a mutually isolated storage state. No linkage or correlation relationship has been established between the two types of data. It is impossible to explore the regulatory effect of the fixed spatial layout structure of the warehouse on the timing of goods flow and the operating rhythm of warehousing equipment, nor can it be judged the limiting effect of the warehousing operation status at different times on the stacking form and space usage range of the warehousing space. Only by building a two-way interconnected information flow path can the linkage and correlation of the two types of heterogeneous warehousing data be realized.

[0094] Spatial convolution is a computational method that relies on a fixed-size convolution kernel to complete the traversal of the neighborhood grid, connect the data paths between adjacent spatial grids, and realize the full-domain flow of static spatial data under the same standard time series conditions. Recursive deduction is an execution method that follows the changing rhythm of the scene's established periodic fluctuation characteristics, and deduces layer by layer at fixed time intervals to realize the layer-by-layer transmission of dynamic time series data between different time series nodes. The periodic fluctuation characteristics in the warehouse scene refer to the warehousing operation rhythm with repetitive operating rules, such as the entry and exit of goods and the start and stop of equipment in the warehouse at fixed time periods.

[0095] For example, for static spatial data such as the overall layout of warehouse shelves, fixed access routes in the warehouse area, and ground reference positioning structure, which have already undergone 3cm residual correction, a 3×3 convolution kernel is uniformly used to traverse all adjacent spatial grids within the warehouse, completing the unimpeded flow and transmission of static spatial structure information within the entire warehouse grid system. For dynamic time-series data such as warehouse goods inbound and outbound flow and the operating status of automated handling equipment, which have undergone 0.2s time-series offset residual correction, a 60-minute centralized goods inbound cycle and a 90-minute centralized goods outbound cycle are set as periodic fluctuation characteristics. Based on this time cycle, a recursive time-series dimension deduction calculation is carried out. Dynamic warehousing operation information, according to the previously set 10s time-series coding unit, completes the time-series permeation of information between different time-series nodes layer by layer, completing the time-series dimension connection of dynamic data across the entire domain.

[0096] This application completely breaks down the barriers between static spatial structure information and dynamic warehousing time-series operation information in warehouses, enabling two-way communication and preliminary association and integration of the two types of warehousing information with different attributes. In the process of information interaction and flow, the inherent linkage pattern between spatial and temporal data is discovered, providing a data association foundation for subsequent accurate separation and extraction of feature information of the two types of functional attributes.

[0097] Based on the completion of two-way information exchange, the guiding information of spatial characteristics on the evolution of time and the constraining information of time characteristics on the spatial structure are extracted from the already established warehouse spatiotemporal information system. The warehouse spatiotemporal information formed after two-way interaction and fusion is a hybrid fusion information. It has not been classified according to the information function attributes, and it is impossible to distinguish the information content that is dominated by the spatial structure in the temporal change, nor can it define the information content that is restricted by the temporal operation rules in the form of space use.

[0098] The guiding information of spatial characteristics on temporal evolution refers to the information formed by the inherent spatial characteristics such as the distribution of warehouse space grids, the division structure of storage areas, and the density of shelf arrangement, which can dominate the changes in the turnover rate of goods in the warehouse area and the temporal rhythm of warehousing operations. The constraining information of temporal characteristics on spatial structure refers to the information formed by the inherent temporal characteristics such as the rhythm of warehouse goods turnover, the timed warehousing operation arrangement, and the operating sequence of equipment, which can limit the specifications of goods stacking in the storage space, the scope of area space utilization, and the form of space use.

[0099] Within the corresponding warehouse's overall grid system, a total of 75 spatial grids for warehouse access channels are defined, accounting for 25% of the total number of grids in the entire area. The inherent spatial characteristics of these grids, such as their orientation and width layout, dominate the rate of goods movement and the overall operational sequence. Guiding information is formed by integrating the spatial correlation data of these grids. A total of 165 spatial grids for fixed goods stacking and storage are defined, accounting for 55% of the total number of grids in the entire area. The main operational time interval for the warehouse is set from 8:00 to 18:00 every day. The rules for inbound, inventory, and transfer operations within this fixed time interval limit the stacking height and area of ​​goods within the grid area. Complete constraint information is formed by integrating these time sequence rules. The boundaries between the two types of information are clear and quantifiable.

[0100] This application completes the attribute decomposition and sorting of mixed warehousing spatiotemporal information, clearly distinguishes the control objects and directions of action corresponding to the two types of characteristic information, and provides the prerequisites for adjusting the ratio according to the actual operating status of the scenario and carrying out cross-dimensional feature splicing and integration.

[0101] After extracting the two types of feature information, the fusion ratio of the two types of information is determined based on the grid deformation coefficient and the temporal fluctuation coefficient. The information content is integrated through cross-dimensional feature splicing to generate the spatiotemporal coupling features of the warehousing scenario. The structural stability of different storage areas within the factory warehouse varies significantly, and the activity of the flow of stored goods and the magnitude of operational changes also differ in different operating periods. Using a fixed and uniform ratio to complete the splicing of the two types of information cannot adapt to the differentiated spatiotemporal operating states of local areas of the warehouse. The fusion weight is dynamically adjusted according to the coefficients that can reflect the actual state of the scenario so that the final generated fusion features conform to the real warehousing operation rules of the warehouse.

[0102] The grid deformation coefficient is a parameter used to measure the stability of the independent spatial grid structure within the warehouse and the magnitude of the physical deformation. This parameter continues the deformation threshold judgment logic of the previous spatial benchmark screening process, and the general value range is set to 0.02 to 0.15. The temporal fluctuation coefficient is a parameter used to measure the magnitude of changes in the dynamic storage operation status and the level of circulation activity within different operating periods of the warehouse. Combined with the previous control standard of 0.2s temporal offset residual, the general value range is set to 0.1 to 0.8. Those skilled in the art can adjust the grid deformation coefficient and the temporal fluctuation coefficient accordingly in the actual process.

[0103] For example, if the measured deformation of the grid in the load-bearing goods storage area at the bottom of the warehouse is low, the grid deformation coefficient is set to 0.03; if the measured deformation of the grid in the upper layer of the warehouse storing light materials is high, the grid deformation coefficient is set to 0.12. During the peak period of goods circulation in the warehouse from 9:00 to 11:00 every day, the time-series fluctuation coefficient is set to 0.75; during the stable and static storage period in the early morning, the time-series fluctuation coefficient is set to 0.15. The fusion ratio is determined by the relative value of the two types of coefficients, guiding the fusion ratio P of information. g =μ / (μ+ν), the proportion of constraint information fusion P c =ν / (μ+ν), where μ represents the mesh deformation coefficient and ν represents the time series fluctuation coefficient.

[0104] Substituting the corresponding values ​​to complete the calculation, when the grid deformation coefficient is 0.03 and the temporal fluctuation coefficient is 0.75, the fusion ratio of guiding information is approximately 0.0385 and the fusion ratio of constraint information is approximately 0.9615. When the grid deformation coefficient is 0.12 and the temporal fluctuation coefficient is 0.15, the fusion ratio of guiding information is approximately 0.4444 and the fusion ratio of constraint information is approximately 0.5556. Based on this fusion ratio, the spatial dimension guiding information and the temporal dimension constraint information are sequentially spliced ​​across dimensions and integrated into a single structured feature data, ultimately generating a spatiotemporal coupled feature.

[0105] This application combines the structural characteristics of warehouse storage areas with the characteristics of time-sharing operations to complete differentiated feature fusion processing, so that the final spatiotemporal coupling features are highly consistent with the actual storage operation of the warehouse site. As the core node features in the subsequent process of constructing heterogeneous graph structures, it realizes the seamless connection between the spatiotemporal coupling feature generation process and the warehouse spatiotemporal state reasoning process, and ensures the logical consistency and unity of the entire modeling technology process.

[0106] S3. Construct a heterogeneous graph based on spatiotemporal coupling features, including spatial neighborhood relationships and state transfer relationships. Perform inference on the heterogeneous graph through a graph attention network to output the spatiotemporal state of the target scene.

[0107] Specifically, the spatiotemporal state of the output target scene includes:

[0108] The basic structure of heterogeneous graphs is constructed using spatial grids as graph nodes and spatiotemporal coupling features as node features.

[0109] Spatial neighborhood edges are constructed based on the physical distance between spatial grids, and temporal state transfer edges are constructed based on the periodic fluctuation characteristics, forming a heterogeneous graph topology that includes spatial neighborhood relationships and state transfer relationships;

[0110] Spatial topological weights are configured based on the mesh deformation coefficient, and temporal evolution weights are configured based on the temporal fluctuation coefficient. Feature inference is then performed on spatial neighborhood edges and state propagation edges on the heterogeneous graph through a graph attention network.

[0111] Based on the inference results, the node features of the graph nodes are iteratively corrected, and the corrected node features of all graph nodes are integrated to output the spatiotemporal state of the target scene.

[0112] Among them, the heterogeneous graph uses spatial grids bound with time-series codes as graph nodes, and assigns the spatiotemporal coupling features corresponding to the spatial grids as node features; the feature inference of spatial neighborhood edges is to perform weighted aggregation of the guiding information of adjacent spatial grids according to the spatial topology weights, and complete the feature deviation calibration of adjacent spatial grids under the same time dimension; the feature inference of state-transfer edges is to complete the recursive mapping of constraint information according to the arrangement order of periodic fluctuation features, and correct the evolution offset of different constraint information according to the temporal evolution weights.

[0113] Based on the warehouse space grid units assigned values ​​in the previous process and the generated spatiotemporal coupling features, the construction of the heterogeneous graph infrastructure is carried out first. The spatiotemporal coupling features distributed in each space grid of the warehouse are in a discrete storage state, and there is no unified graph structure data organization form. It is impossible to complete the linkage inference of warehouse space association and temporal state based on the graph model. Only by determining the standardized space grid as the basic computing node and simultaneously completing the node feature binding can a unified carrier adapted to the graph reasoning operation of the warehouse scenario be built.

[0114] Using a spatial grid bound with a time-series code as a graph node refers to a warehouse space unit that has been uniformly divided into 3m side lengths, and has completed 0.1s time delay deviation correction and matched with time-series codes; assigning the spatiotemporal coupling features corresponding to the spatial grid as node features means matching and recording the fusion features formed by cross-dimensional splicing of guiding information and constraint information into the corresponding graph node.

[0115] For example, a medium-sized warehouse is divided into 300 standard spatial grids, consisting of 20 columns horizontally and 15 rows vertically. All grids are uniformly bound to a T value that increments by 10 seconds. 001 To T 999The temporal coding is arranged in sequence, with each physical space grid corresponding to a different temporal code at different operating times. For example, the same grid corresponds to code T at time T=0s. 001 The corresponding code T at time T=10s 002 Similarly, the coordinates of the center of the global grid are arranged sequentially according to the site reference origin (0,0)m. The typical reference grid coordinates are (3,3)m, and the grid coordinates of the warehouse transfer area are (15,12)m. According to the fixed arrangement order of the physical area of ​​the warehouse from front to back and from left to right, the spatiotemporal coupling features corresponding to each grid are retrieved in sequence, and the graph node features are assigned one by one. The feature data is accurately assigned to the graph node corresponding to the spatiotemporal identifier (X,Y,T).

[0116] X represents the center coordinates of the spatial grid in the horizontal (east-west) direction under the warehouse reference coordinate system, in meters (m), and its value range is the grid center coordinate sequence within the horizontal boundary of the warehouse; Y represents the center coordinates of the spatial grid in the vertical (north-south) direction under the warehouse reference coordinate system, in meters (m), and its value range is the grid center coordinate sequence within the vertical boundary of the warehouse; T represents the cumulative time series duration bound to the spatial grid, in seconds (s), which is calculated from the time series coding after a 0.1s delay deviation correction, and its value range covers the time series coding interval of the entire operating period of the warehouse.

[0117] This application incorporates the spatiotemporal fusion features of the entire warehouse into the graph structure data system, unifies the basic unit specifications and feature input rules for graph operations in the warehousing scenario, and clearly defines the scope of operation nodes and the inherent feature content of nodes within the heterogeneous graph, thus laying the foundation for determining the node arrangement when building two types of topological association edges with different attributes.

[0118] After completing the configuration of basic nodes and node features of the heterogeneous graph, spatial neighborhood edges and temporal state transmission edges are constructed according to the established physical distance standard and temporal pattern standard, generating a heterogeneous graph topology structure with dual association relationships. Simply setting graph nodes cannot map the actual operation logic of the factory warehouse. It cannot reflect the actual physical adjacent position relationship between warehouse shelves and storage aisles, nor can it restore the state change pattern formed by the operation behaviors such as goods entering and leaving the warehouse and the start and stop of storage equipment over time. Only by establishing a two-layer topological association relationship can the graph structure model fit the layout of the warehouse entity and the normalized operation temporal logic.

[0119] The physical distance between spatial grids refers to the measured straight-line distance between the geometric center points of two different spatial grids. The periodic fluctuation characteristic refers to the fixed-duration warehousing operation rhythm formed by the long-term stable operation of the warehouse. The standard spatial grid of the warehouse has a single side length of 3m. The straight-line distance between the geometric centers is uniformly set to be less than or equal to 6m as the threshold for constructing spatial neighborhood edges. Spatial neighborhood edges are built between adjacent grid nodes that meet this distance condition. Following the established warehousing operation rhythm, the 60-minute goods inbound cycle and the 90-minute goods outbound cycle are used as the periodic fluctuation characteristics. The time sequence is built sequentially between nodes with different time sequence codes in the same physical grid, according to a fixed 10s time sequence code. Finally, a heterogeneous graph topology network that includes both spatial neighborhood relationships and state transmission relationships is formed.

[0120] This application fully reconstructs the real-world relational logic of the warehouse from both the physical space dimension and the time-series operation dimension, and divides it into two independent feature information transmission paths. It clearly distinguishes the application boundaries of the spatial dimension relational path and the time dimension relational path, and defines a clear execution scope for configuring corresponding computational weights for the two types of paths and performing differentiated feature reasoning.

[0121] Based on a two-layer heterogeneous topology graph, spatial topology weights are determined by converting grid deformation coefficients, and temporal evolution weights are determined by converting temporal fluctuation coefficients. Feature inference operations corresponding to the two types of topological edges are performed on the two types of topological edges through a graph attention network. The structural stability of different storage areas in the warehouse has fixed differences, and the activity of warehouse goods circulation has fixed distinctions in different operating periods. Using a uniform fixed weight for feature inference cannot fit the actual working conditions of the warehouse. The aforementioned scenario feature coefficients are used to complete the weight conversion, distinguish the feature inference objects corresponding to the two types of topological edges, and realize the directional calibration and temporal correction of feature information.

[0122] The spatial neighborhood edge feature inference based on spatial topology weights is essentially a weighted aggregation operation performed on the guiding information stored within adjacent grids, thereby calibrating the spatial feature deviation between adjacent storage units under the same temporal conditions. The state transfer edge feature inference based on temporal evolution weights is essentially a temporal recursive mapping of constraint information according to the fixed arrangement order of periodic fluctuation features, using predetermined weights to correct the evolutionary offset of constraint information in different temporal stages.

[0123] Where the spatial topology weight W s It is obtained by conversion from the mesh deformation coefficient μ, and the conversion relationship is set as W. s =1-μ, used to control the weighted aggregation degree of guiding information in spatial neighborhood edge feature inference; temporal evolution weight W t It is obtained by conversion from the time series fluctuation coefficient ν, and the conversion relationship is set as W. t=ν, used to control the evolution offset correction magnitude of constraint information in state transfer edge feature inference. In this scenario, the mesh deformation coefficient of the heavy cargo storage area at the bottom of the warehouse is μ1=0.03. Substituting this value into the equation yields the corresponding spatial topology weight W. s1 =1-0.03=0.97; The mesh deformation coefficient of the upper-level light material storage area of ​​the warehouse is μ2=0.12, and the corresponding spatial topology weight W is calculated. s2 =1-0.12=0.88; In terms of time series, during the peak period of goods turnover in the warehouse from 9:00 to 11:00 every day, the time series fluctuation coefficient is ν1=0.75, and the time series evolution weight W is determined. t1 =0.75; During the stable warehousing period with low inventory turnover in the early morning, the time series fluctuation coefficient is taken as ν2=0.15, and the time series evolution weight W is determined. t2 =0.15.

[0124] In the practical calculation of spatial neighborhood edge feature inference, two adjacent grid nodes with coordinates (3,3)m and (6,3)m were selected at the same time series value T=600s. The straight-line distance between the centers of the two nodes is 3m, which meets the 6m judgment threshold. The built-in guidance information of the two nodes is denoted as G1 and G2, respectively. G1 is the guidance information of the spatial feature extracted from the grid node with coordinate (3,3)m after bidirectional information interaction on the temporal evolution, and G2 is the guidance information corresponding to the grid node with coordinate (6,3)m. Weighted aggregation is completed using the spatial topology weight of 0.97 corresponding to the underlying storage area, i.e., G avg =W s1 ×G1+(1-W s1 )×G2, to complete the fusion calculation of adjacent grid guidance information, and simultaneously complete the unified calibration of spatial feature deviations in the same time dimension, G avg It is the fusion guidance information corresponding to the neighborhood edges of the space after the weighted aggregation is completed.

[0125] In the practical calculation of state-transfer edge feature reasoning, different time-series nodes corresponding to the same spatial coordinates (15, 12)m are selected, and the constraint information is recursively deduced layer by layer according to the arrangement order of the 60-minute warehousing cycle. Let the constraint information of the preceding time-series node be C. pre That is, in the grid corresponding to the time-series encoding of the previous cycle, the constraint information formed by the temporal characteristics on the spatial structure, and the reference constraint information of the current time-series node is C. cur That is, the baseline value of the constraint information already existing in the grid corresponding to the current temporal encoding is used to complete the offset correction by incorporating the temporal evolution weight, i.e., C. new =C cur +W t ×(C pre -C cur To quantitatively eliminate the constraint information evolution offset generated during the time series extrapolation process, C newThis is the constraint information of the current time node after being corrected by the temporal evolution weight.

[0126] This application achieves precise binding between inference weights and inherent parameters of the warehousing scenario, accurately completes targeted optimization processing of two types of core feature information, and obtains node feature data optimized by a single round of standardized inference, providing initial computational data in a fixed format for subsequent multi-round iterative convergence optimization.

[0127] Based on the optimized data obtained from a single round of feature inference, the internal features of all graph nodes are iteratively corrected according to unified rules. All corrected node features are integrated to output the spatiotemporal state of the target scene of the factory warehouse. A single edge feature inference can only complete the feature optimization of a local area and cannot achieve the overall collaborative unification of the features of 300 grid nodes in the warehouse. Local nodes still have independent feature deviations. Fixed iterative operation rules are set to complete feature convergence, and then the data is summarized according to a fixed integration order to form a spatiotemporal state that can characterize the overall storage operation status of the warehouse.

[0128] Repeat the aforementioned weight configuration and edge feature inference process to carry out iterative calculations, setting a fixed iteration limit of 8 rounds, and setting a node feature data fluctuation convergence threshold of 0.01. The convergence threshold of 0.01 represents the average L2 norm change of all graph node feature vectors in two consecutive iterations. When the overall change of node features in two consecutive iterations is less than the convergence threshold, the iteration process is terminated early. After the iteration process is completed, according to the arrangement order of the physical areas of the warehouse from left to right horizontally and from front to back vertically, the feature data of 300 spatial grid nodes after correction are summarized and integrated in sequence. After being organized and summarized, a unified and dimensionally complete warehouse spatiotemporal state is formed.

[0129] This application completes the collaborative convergence optimization of the node features of the entire warehouse, eliminates the isolation bias of local grid features, unifies the overall warehouse status representation standard, and provides input data for the subsequent mapping of the spatiotemporal state to a continuous spatiotemporal field through implicit continuous representation and solving the reconstruction results of arbitrary spatiotemporal coordinates of the warehouse.

[0130] S4. Map the spatiotemporal state to a continuous spatiotemporal field through implicit continuous representation, and output the reconstruction result of the target scene at arbitrary spatiotemporal coordinates.

[0131] Specifically, the reconstruction results of the target scene at arbitrary spatiotemporal coordinates include:

[0132] Extract the spatiotemporal state corresponding to each spatial grid, and use the physical coordinates and temporal codes of the spatial grid as implicit query variables to construct a spatiotemporal sample set;

[0133] The physical coordinates of the spatial grid within the spatiotemporal sample set are arranged in an ordered manner, and the temporal codes are sorted temporally according to the periodic fluctuation characteristics to generate a spatiotemporal sequence.

[0134] By performing coordinate interpolation to supplement points on the spatiotemporal sequence, and expanding it to generate continuous spatiotemporal coordinates, an implicit continuous representation is obtained;

[0135] Based on continuous spatiotemporal coordinates, all spatiotemporal states are integrated to fit and generate a continuous spatiotemporal field;

[0136] Within a continuous spatiotemporal field, arbitrary spatiotemporal coordinates are selected, and the spatiotemporal state corresponding to the spatiotemporal coordinates is solved through neighborhood grid weighted operation, outputting the reconstruction result of the target scene.

[0137] Among them, implicit continuous representation uses the physical coordinates of the spatial grid as spatial input and temporal encoding as temporal input to fill in the missing spatiotemporal samples;

[0138] The continuous spatiotemporal field combines grid deformation coefficients to correct spatial interpolation distortion and temporal fluctuation coefficients to compensate for temporal evolution migration.

[0139] The reconstruction result of the target scene is to retrieve the neighboring spatial grids of any spatiotemporal coordinates, assign weights according to the spatiotemporal coordinate distance, aggregate the spatiotemporal states corresponding to the neighboring spatial grids, and solve for the reconstruction result under the corresponding spatiotemporal coordinates; the spatiotemporal coordinate distance includes the physical distance between spatial grids and the time interval between temporal codes.

[0140] The spatial state of the warehouse grid, which has been optimized through multiple rounds of iterative correction of the heterogeneous graph, is stored independently within a single grid cell. Various types of spatial state data have not been structurally and uniformly collected, and there is a lack of a unified spatial and temporal positioning identification system. The discrete data storage structure cannot support large-scale orderly arrangement and continuous numerical extrapolation calculations.

[0141] The physical coordinates of the spatial grid are the two-dimensional planar coordinates (X,Y) incorporated into the S(X,Y,T) representation system, which are fixed numerical bases for calibrating the location of the warehouse entity. The temporal encoding is the cumulative time series duration T calculated after completing the 0.1s time delay deviation correction. The combination of the two constitutes the implicit query variable. The spatiotemporal sample set is a standardized dataset formed by summarizing the spatiotemporal state of a single grid, the grid calibration (X,Y) coordinates, and the corresponding time series duration T according to a fixed field structure. S(X,Y,T) is the identifier for any spatiotemporal state in the warehouse scenario. S is the spatiotemporal state identifier, X is the horizontal center coordinate of the spatial grid, Y is the vertical center coordinate of the spatial grid, and T is the cumulative time series duration. This identifier system carries both spatial positioning and temporal positioning functions.

[0142] Using the warehouse entrance origin (0,0)m as the baseline, a standard spatial grid with a side length of 3m is uniformly divided, with 20 columns horizontally and 15 rows vertically, totaling 300 standard grids. The spatiotemporal status of all grids is retrieved one by one, and the grid center coordinates accurately calibrated on-site are entered synchronously. For example, the center coordinates of the baseline grid in the first row and column are (3,3)m, the center coordinates of the core grid in the warehouse transfer area are (15,12)m, and the center coordinates of the grid at the end of the outbound area are (57,42)m. The cumulative time series duration after time delay correction is matched, with the baseline starting time series corresponding to 0s, the basic interval time series to 10s, and the hourly cycle time series to 60s. S(X,Y,T) is used as the implicit query identifier, and data is entered one by one according to the entity arrangement order of the vertical row number and horizontal column number of the grid, to build a warehouse spatiotemporal sample set with unified fields, unified coordinate format, and unified time series identifier.

[0143] This application completes the centralized organization of discrete spatiotemporal data across the entire domain, unifies the storage format and location retrieval standards of all data, and thoroughly streamlines the underlying data architecture. It provides a unified, coordinate-representation-standardized, and numerically clear original data source for subsequent ordered sorting based on coordinate and time-series values, which is an indispensable prerequisite for generating standardized spatiotemporal sequences.

[0144] After completing the construction of the spatiotemporal sample set, the data within the sample set only completed the collection of basic information. There was no unified numerical progression rule for the spatial coordinate numerical arrangement, and the temporal duration arrangement did not conform to the inherent cyclical fluctuation operation rules of the warehouse. The disordered numerical arrangement structure could not meet the basic execution conditions for continuous operations such as linear interpolation. Based on the coordinate numerical magnitude and temporal operation rules, bidirectional orderly regularization was completed to form a standard spatiotemporal sequence with fixed numerical evolution logic.

[0145] The physical coordinates of the spatial grid are arranged in an orderly manner, with the origin (0,0)m as the reference. They are arranged first in ascending order of vertical Y-coordinate values ​​from smallest to largest, and then in ascending order of horizontal X-coordinate values ​​within the same vertical dimension. The temporal coding and sorting are completed according to the periodic fluctuation characteristics. The time sequence intervals are divided according to the warehouse's 60-minute inbound cycle and 90-minute outbound cycle. The cumulative time sequence duration T within the same operation cycle interval is arranged in ascending order of value. The spatiotemporal sequence is the ordered spatiotemporal data arrangement system formed after the two-dimensional numerical sorting is completed.

[0146] For example, the above numerical sorting rules are applied to all S(X,Y,T) format data in the sample set, and the coordinates (3,3)m, (6,3)m, and (9,3)m are arranged in order. The time series data from T=0s to T=3600s in the inbound operation interval are centrally and neatly arranged, and the time series data from T=5400s to T=9000s in the outbound operation interval are uniformly classified and sorted.

[0147] This application clarifies the numerical distribution hierarchy of spatiotemporal data in the spatial dimension and the numerical evolution hierarchy in the temporal dimension, establishes a unified numerical regularization execution standard across the entire domain, avoids subsequent calculation deviations caused by disordered arrangement from the root, and forms a standardized spatiotemporal sequence with clear coordinate order and regular temporal numerical arrangement, which fully meets the hard prerequisite requirement of ordered numerical sequence for subsequent linear coordinate interpolation and point supplementation operations.

[0148] The standard spatiotemporal sequence formed by bidirectional numerical sorting only covers the coordinate points of the standard grid with a 3m interval and the standard time sequence with a 10s interval as previously defined. There is no corresponding spatiotemporal data to support the warehouse grid gaps, temporary storage points, and non-corresponding transition time sequence nodes, so it is impossible to achieve full coverage of information for the entire warehouse space and the entire operating period.

[0149] Implicit continuous representation uses the calibrated grid (X,Y) coordinates as the spatial input reference and the standardized cumulative time series duration T as the time input reference. It forms a continuous spatiotemporal information expression by filling in the gaps between discrete points through linear interpolation. The coordinate interpolation point filling uses linear interpolation to complete the numerical solution of the points, with a spatial dimension of X. mid =X1+(X2-X1) / L×l,Y mid =Y1+(Y2-Y1) / L×l, where the time dimension is T. mid =T1+(T2-T1) / M×γ, where L is the coordinate interval of the original adjacent spatial grids in the corresponding dimension, here the side length of the spatial grid is 3m, M is the time interval corresponding to the original adjacent temporal codes, here the basic temporal code increment unit is 10s, l is the spatial interpolation increment step size, here set l=1m, γ is the temporal interpolation increment step size, here set γ=2s, X mid Y mid T mid The intermediate transition coordinates and time values ​​generated by interpolation are X1 and X2, which are the coordinate values ​​of the two ends of the original adjacent spatial grid in the X direction, Y1 and Y2, which are the coordinate values ​​of the two ends of the original adjacent spatial grid in the Y direction, and T1 and T2, which are the two time duration values ​​corresponding to the original adjacent time codes.

[0150] For example, setting the spatial interpolation step size to 1m and the temporal interpolation step size to 2s, and substituting the actual coordinates to complete the calculation, taking the original two adjacent standard grid coordinates (6,9)m and (9,9)m, with a lateral coordinate interval of 3m, and interpolating them sequentially with a step size of 1m, the intermediate transition coordinates (7,9)m and (8,9)m are obtained; taking the original adjacent temporal durations T=10s and T=20s, with a temporal interval of 10s, and interpolating them sequentially with a step size of 2s, the temporal identifiers corresponding to T=12s, T=14s, T=16s, and T=18s are generated sequentially, and all the S(X,Y,T) spatiotemporal points obtained by the interpolation calculation are added to the spatiotemporal sequence, filling the data gaps of non-sampling locations and non-standard temporal nodes in the warehousing scenario, and completing the technical conversion of discrete spatiotemporal data to implicit continuous representation.

[0151] This application produces a global continuous spatiotemporal coordinate system with uniform coordinate value distribution and complete temporal node values, providing a sufficient and accurately traceable coordinate carrier for subsequent spatiotemporal field fitting work.

[0152] Continuous spatiotemporal coordinate data generated solely from linear interpolation will be affected by differences in the structural stability of the warehouse area and the activity level of time-sharing warehousing operations, resulting in spatial interpolation numerical distortion and temporal extrapolation numerical offset. Direct data integration will cause overall numerical distortion of the spatiotemporal field. Therefore, it is necessary to use the grid deformation coefficient and temporal fluctuation coefficient used uniformly in the whole scheme to complete the deviation correction calculation. After the correction is completed, the continuous spatiotemporal coordinates are bound to the corresponding spatiotemporal state to fit and form a standard continuous spatiotemporal field.

[0153] Spatial interpolation distortion correction operation uses the grid deformation coefficient as the correction weight coefficient to perform linear calibration on the spatial coordinate values ​​generated by interpolation. Temporal evolution offset compensation operation uses the temporal fluctuation coefficient as the compensation weight coefficient to perform gradient calibration on the temporal corresponding state values ​​generated by interpolation. The continuous spatiotemporal field is a global four-dimensional data field formed by binding and integrating each set of corrected S(X,Y,T) spatiotemporal coordinates with the corresponding spatiotemporal state after completing the two-dimensional numerical calibration.

[0154] The deformation coefficient of the grid in the heavy goods storage area at the bottom of the warehouse is set to 0.03. This coefficient is used in the calibration formula to correct the numerical deviation of the interpolation coordinates with a step size of 1m. The deformation coefficient of the grid in the light materials storage area at the top of the warehouse is set to 0.12. The corresponding spatial interpolation numerical calibration is completed. The temporal fluctuation coefficient during the peak period of goods circulation from 9:00 to 11:00 is set to 0.75. The temporal state numerical offset compensation is completed for the interpolation with a step size of 2s. The temporal fluctuation coefficient during the low-activity storage period from 0:00 to 6:00 is set to 0.15. The temporal data calibration is completed for the flat period. After all deviation correction calculations are completed, the data is integrated according to the binding rules corresponding to the coordinates. Each corrected spatiotemporal coordinate point and its corresponding spatiotemporal state value are used as field nodes. The discrete field representation of the whole domain is constructed by directly assigning the state value of the nearest neighbor grid. Finally, a continuous spatiotemporal field covering the entire area of ​​the factory warehouse and the entire operating time is fitted.

[0155] This application eliminates the numerical deviation inherent in interpolation through explicit coefficient calibration calculations, ensuring that all spatiotemporal coordinates and corresponding state values ​​within the spatiotemporal field conform to the actual operating conditions of the warehouse, forming a continuous spatiotemporal field with accurate values, complete dimensions, and sufficient calculation basis, possessing all the technical conditions for scene reconstruction for any custom S(X,Y,T) format spatiotemporal coordinates.

[0156] In actual warehouse operation and maintenance application scenarios, business personnel need to retrieve warehouse scene data corresponding to any custom S(X,Y,T) format spatiotemporal coordinates, filter the neighboring standard grids according to the defined fixed search range, and accurately obtain the reconstruction result corresponding to the target point through the composite spatiotemporal distance calculation formula and neighborhood feature weighted aggregation operation process.

[0157] Spatiotemporal coordinate distance is divided into physical distance in a straight line in space and distance in a time series interval. (Spatial straight line distance) , X0 represents the linear physical distance between the target spatiotemporal coordinates and the adjacent spatial grids, and Y0 represents the horizontal coordinate value of the target spatiotemporal coordinates. i Y is the lateral coordinate value of the i-th neighboring spatial grid center. i The vertical coordinate value of the i-th neighboring spatial grid center, and the time interval distance. , T represents the time interval distance between the target spatiotemporal coordinates and the neighboring spatial grids, T0 represents the temporal duration value of the target spatiotemporal coordinates, and T... i The cumulative time series duration value is bound to the i-th neighboring spatial grid, and the two together form the composite spatiotemporal distance.

[0158] The neighborhood grid weighting operation follows the principle that the smaller the distance, the higher the weight. The overall weight of a single grid is calculated. normalized weights The final reconstructed state aggregation calculation formula is as follows: ,in The final reconstructed state value corresponding to the queried spatiotemporal coordinates S(X0,Y0,T0) is the reconstruction result of the target scene under the spatiotemporal coordinates obtained by weighted aggregation of neighborhood grids; n is the number of neighboring spatial grids selected within the search range. The spatiotemporal state values ​​corresponding to neighboring grids are the spatiotemporal state values ​​output by the i-th neighboring grid after iterative correction by graph attention network under the corresponding temporal encoding. The spatiotemporal state aggregation operation follows the principle that the closer the distance, the higher the weight. The spatiotemporal state values ​​of all neighboring grids are weighted and summed according to normalized weights to obtain the reconstructed state of the target spatiotemporal coordinates.

[0159] The search execution standard is fixed, with the maximum spatial search radius set to 6m and the maximum temporal search interval set to 20s. Taking the target spatiotemporal coordinates S(7.5,10.5,15) as an example, a complete calculation is carried out. First, the neighboring standard grids S(6,9,12) and S(9,9,10) are selected within the search range. The distance values ​​are then calculated by substituting them into the formula. The spatial distance between the target coordinates and the first neighboring grid is... =2.12m, timing interval =3s, distance from the second adjacent grid space =2.12m, timing interval =5s; Continue substituting into the formula to calculate the comprehensive weight of the first neighboring grid. =1 / (2.12+3)≈0.1953, the combined weight of the second neighboring grid. =1 / (2.12+5)≈0.1404;

[0160] Normalization is used to determine the weights. =0.1953 / (0.1953+0.1404)≈0.5817, =0.1404 / (0.1953+0.1404)≈0.4183; For example, the first neighboring grid S(6,9,12) corresponds to the spatiotemporal state. =0.72, the second neighboring grid S(9,9,10) corresponds to the spatiotemporal state. =0.58, substitute into the formula =0.5817×0.72+0.4183×0.58≈0.6614, that is, the reconstructed state value of the warehouse storage scene under the spatiotemporal coordinate S(7.5,10.5,15) is approximately 0.6614.

[0161] This application breaks through the data retrieval limitations of discrete standard grid points, enabling accurate solution of scene information in any spatiotemporal coordinates within the entire warehouse area. It completes the closed-loop technical implementation work from the acquisition of raw sensing data to the reconstruction of arbitrary spatiotemporal coordinate scenes in the final data output stage of the entire modeling technology process.

[0162] Example 2

[0163] like Figure 3 As shown, this embodiment provides a spatiotemporal integrated modeling system based on 4D temporal information, including a perception module, a fusion module, a graph analysis module, and a mapping module.

[0164] The perception module is used to acquire spatiotemporal perception data of the target scene, perform time synchronization and spatial registration on the spatiotemporal perception data, and generate a calibrated 4D dataset.

[0165] The fusion module is used to construct a four-dimensional spatiotemporal representation of the 4D dataset. It simultaneously performs bidirectional information interaction along the spatial and temporal dimensions on the four-dimensional spatiotemporal representation, extracts the guiding information of spatial features on temporal evolution and the constraint information of temporal features on spatial structure, and generates spatiotemporal coupled features through bidirectional fusion.

[0166] The graph analysis module constructs a heterogeneous graph based on spatiotemporal coupling features, including spatial neighborhood relationships and state transit relationships. It then performs inference on the heterogeneous graph through a graph attention network and outputs the spatiotemporal state of the target scene.

[0167] The mapping module is used to map the spatiotemporal state into a continuous spatiotemporal field through implicit continuous representation, and output the reconstruction result of the target scene in arbitrary spatiotemporal coordinates.

[0168] The above description is merely a preferred embodiment of this application. The scope of protection of this application is not limited to the above embodiments. All technical solutions falling within the scope of this application's concept are within the scope of protection of this application. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of this application should also be considered within the scope of protection of this application.

Claims

1. A spatiotemporal integrated modeling method based on 4D temporal information, characterized in that, include: Acquire spatiotemporal perception data of the target scene, perform time synchronization and spatial registration on the spatiotemporal perception data, and generate a 4D dataset; A four-dimensional spatiotemporal representation is constructed on the 4D dataset. Bidirectional information interaction is carried out simultaneously along the spatial and temporal dimensions on the four-dimensional spatiotemporal representation. The guiding information of spatial features on temporal evolution and the constraint information of temporal features on spatial structure are extracted. Spatiotemporal coupling features are generated through bidirectional fusion. Based on the spatiotemporal coupling feature, a heterogeneous graph including spatial neighborhood relations and state transit relations is constructed. Inference is performed on the heterogeneous graph through a graph attention network to output the spatiotemporal state of the target scene. The spatiotemporal state is mapped to a continuous spatiotemporal field through implicit continuous representation, and the reconstruction result of the target scene at arbitrary spatiotemporal coordinates is output.

2. The spatiotemporal integrated modeling method based on 4D temporal information according to claim 1, characterized in that, The generated 4D dataset includes: Acquire spatiotemporal perception data of the target scene, classify it according to spatial granularity and temporal change rate, and distinguish between static spatial data and dynamic temporal data of the target scene; Fixed entities in the target scene are selected as spatial references, and periodic fluctuation characteristics of the target scene are selected as time references. Spatial registration is performed on static spatial data, and time synchronization is performed on dynamic time series data. The statistical spatial registration residuals and temporal offset residuals are analyzed and iteratively corrected. The corrected static spatial data and dynamic temporal data are then fused and encapsulated according to the spatial and temporal dimensions to generate a 4D dataset.

3. The spatiotemporal integrated modeling method based on 4D temporal information according to claim 2, characterized in that, The spatial registration refers to selecting fixed entities within the target scene whose deformation rate is less than a preset deformation threshold as spatial references, and matching and correcting the spatial coordinates of different sampling devices through an iterative nearest point algorithm. The time synchronization refers to extracting the periodic fluctuation characteristics within the target scene as a time reference, and correcting the timestamp deviation of data with different sampling frequencies through time delay estimation.

4. The spatiotemporal integrated modeling method based on 4D temporal information according to claim 3, characterized in that, The generated spatiotemporal coupling features include: Spatial grid partitioning and temporal coding mapping are performed on the 4D dataset to construct a four-dimensional spatiotemporal representation; In the four-dimensional spatiotemporal representation, information is transmitted along the spatial dimension to the corrected static spatial data, and information is permeated along the temporal dimension to the corrected dynamic temporal data, so as to achieve two-way information interaction. Extracting information on how spatial features guide temporal evolution and how temporal features constrain spatial structures; Based on the dynamic allocation and fusion ratio of the grid deformation coefficient and the temporal fluctuation coefficient, the guiding information and constraint information are spliced ​​together across dimensions to generate spatiotemporal coupled features.

5. The spatiotemporal integrated modeling method based on 4D temporal information according to claim 4, characterized in that, The four-dimensional spatiotemporal representation is to divide the 4D dataset into equidistant spatial grids, match the spatial grids with time-delay estimation and correction time-series codes, and form a four-dimensional representation matrix corresponding to the spatial grids and time-series codes. The bidirectional information interaction is achieved by traversing different spatial grids through spatial convolution to transmit information in the spatial dimension, and by recursive deduction based on the periodic fluctuation characteristics to achieve information penetration in the time dimension.

6. The spatiotemporal integrated modeling method based on 4D temporal information according to claim 5, characterized in that, The spatiotemporal state of the output target scene includes: The basic structure of heterogeneous graphs is constructed using spatial grids as graph nodes and spatiotemporal coupling features as node features. Spatial neighborhood edges are constructed based on the physical distance between spatial grids, and temporal state transfer edges are constructed based on the periodic fluctuation characteristics, forming a heterogeneous graph topology that includes spatial neighborhood relationships and state transfer relationships; Spatial topological weights are configured based on the mesh deformation coefficient, and temporal evolution weights are configured based on the temporal fluctuation coefficient. Feature inference is then performed on spatial neighborhood edges and state propagation edges on the heterogeneous graph through a graph attention network. Based on the inference results, the node features of the graph nodes are iteratively corrected, and the corrected node features of all graph nodes are integrated to output the spatiotemporal state of the target scene.

7. The spatiotemporal integrated modeling method based on 4D temporal information according to claim 6, characterized in that, The heterogeneous graph uses spatial grids bound with time-series codes as graph nodes, and assigns the spatiotemporal coupling features corresponding to the spatial grids as node features. The feature inference performed on the spatial neighborhood edge is to perform weighted aggregation of the guiding information of adjacent spatial grids according to the spatial topology weight, and to complete the feature deviation calibration of adjacent spatial grids under the same time dimension. The feature inference performed on the state transfer edge is to complete the recursive mapping of constraint information according to the arrangement order of periodic fluctuation features, and to correct the evolution offset of different constraint information according to the temporal evolution weight.

8. The spatiotemporal integrated modeling method based on 4D temporal information according to claim 7, characterized in that, The reconstruction results of the target scene at any spatiotemporal coordinate include: Extract the spatiotemporal state corresponding to each spatial grid, and use the physical coordinates and temporal codes of the spatial grid as implicit query variables to construct a spatiotemporal sample set; The physical coordinates of the spatial grid within the spatiotemporal sample set are arranged in an ordered manner, and the temporal codes are sorted temporally according to the periodic fluctuation characteristics to generate a spatiotemporal sequence. By performing coordinate interpolation to supplement points on the spatiotemporal sequence, and expanding it to generate continuous spatiotemporal coordinates, an implicit continuous representation is obtained; Based on continuous spatiotemporal coordinates, all spatiotemporal states are integrated to fit and generate a continuous spatiotemporal field; Within a continuous spatiotemporal field, arbitrary spatiotemporal coordinates are selected, and the spatiotemporal state corresponding to the spatiotemporal coordinates is solved through neighborhood grid weighted operation, outputting the reconstruction result of the target scene.

9. The spatiotemporal integrated modeling method based on 4D temporal information according to claim 8, characterized in that, The implicit continuous representation uses the physical coordinates of the spatial grid as spatial input and the temporal encoding as temporal input to fill in the missing spatiotemporal samples; The continuous spatiotemporal field is obtained by combining the grid deformation coefficient to correct spatial interpolation distortion and by combining the temporal fluctuation coefficient to compensate for temporal evolution offset. The reconstruction result of the target scene is to retrieve the neighboring spatial grids of any spatiotemporal coordinates, assign weights according to the spatiotemporal coordinate distance, aggregate the spatiotemporal states corresponding to the neighboring spatial grids, and solve for the reconstruction result under the corresponding spatiotemporal coordinates; the spatiotemporal coordinate distance includes the physical distance between spatial grids and the time interval between temporal codes.

10. A spatiotemporal integrated modeling system based on 4D temporal information, used to implement the spatiotemporal integrated modeling method based on 4D temporal information as described in any one of claims 1-9, characterized in that, include: The module consists of a perception module, a fusion module, a graph analysis module, and a mapping module. The perception module is used to acquire spatiotemporal perception data of the target scene, perform time synchronization and spatial registration on the spatiotemporal perception data, and generate a calibrated 4D dataset. The fusion module is used to construct a four-dimensional spatiotemporal representation of the 4D dataset. It simultaneously performs bidirectional information interaction along the spatial and temporal dimensions on the four-dimensional spatiotemporal representation, extracts the guiding information of spatial features on temporal evolution and the constraint information of temporal features on spatial structure, and generates spatiotemporal coupled features through bidirectional fusion. The graph analysis module constructs a heterogeneous graph based on spatiotemporal coupling features, including spatial neighborhood relationships and state transit relationships. It then performs inference on the heterogeneous graph through a graph attention network and outputs the spatiotemporal state of the target scene. The mapping module is used to map the spatiotemporal state into a continuous spatiotemporal field through implicit continuous representation, and output the reconstruction result of the target scene in arbitrary spatiotemporal coordinates.

Images