Artificial intelligence space-time cube online reasoning method, service system and equipment

Abstract

The present application relates to the field of data analysis and service technology, and in particular to an artificial intelligence spatiotemporal cube online reasoning method, service system and device, wherein the method includes: establishing a cube organization model based on a spatiotemporal grid, wherein the cube organization model uniformly organizes and manages spatiotemporal data and geographic artificial intelligence models based on a spatiotemporal benchmark; matching tasks and models based on explicit matching and implicit matching, wherein explicit matching is to convert the user's reasoning request into a multi-dimensional query condition of the cube organization model to retrieve a set of candidate models, and implicit matching is to achieve model matching by calculating the feature similarity of the tile to be inferred and the candidate model; and a workflow for distributed reasoning using an artificial intelligence model for inference tile data based on a distributed reasoning framework. Thus, the problems of low reasoning accuracy and slow reasoning speed of a single model in multi-spatial or large-scale scenarios in the related technology are solved.

Description

Technical Field

[0001] The present application relates to the field of data analysis and service technology, and in particular to an artificial intelligence space-time cube online reasoning method, service system and equipment. Background Art

[0002] With the rapid growth of spatiotemporal data such as earth observation, the processing methods of spatiotemporal big data are also changing. Existing spatiotemporal big data computing platforms, such as GEE (Google Earth Engine) and ODC (OpenData Cube), have successfully transformed the traditional local data analysis mode into an efficient online processing mode. These platforms have achieved seamless integration of multi-source data such as raster, vector, and point cloud by building spatiotemporal data cubes, and support online processing and analysis of large-scale, long-time series data under a unified spatiotemporal framework.

[0003] However, although these online cloud computing platforms have made significant progress in data processing, when dealing with artificial intelligence reasoning in multi-temporal or large-scale scenarios, model performance is highly dependent on specific time and space conditions, making it difficult for a single model to fully cover all application scenarios; secondly, the reasoning process of massive remote sensing images is not only computationally complex, but also has a large amount of data and a wide dimension, which places extremely high demands on computing resources and data processing capabilities. Summary of the invention

[0004] The present application provides an artificial intelligence space-time cube online reasoning method, service system and equipment to solve the problems of low single model reasoning accuracy and slow reasoning speed in multi-space or large-scale scenarios in related technologies.

[0005] The first aspect of the present application provides an artificial intelligence spatiotemporal cube online reasoning method, comprising the following steps: establishing a cube organization model based on a spatiotemporal grid, wherein the cube organization model uniformly organizes and manages spatiotemporal data and geographic artificial intelligence models based on a spatiotemporal benchmark; performing task and model matching based on a combination of explicit matching and implicit matching, wherein explicit matching is to convert a user's reasoning request into a multidimensional query condition of a cube organization model to retrieve a set of candidate models, and implicit matching is to achieve model matching by calculating the feature similarity between a tile to be inferred and a candidate model; performing a distributed reasoning workflow using an artificial intelligence model on the inference tile data based on a distributed reasoning framework, wherein the tile data to be inferred is converted into a distributed memory object, and the artificial intelligence model is deployed on each computing node to execute the reasoning workflow in parallel on the inference tile data.

[0006] Optionally, when spatiotemporal data enters the cube, it needs to undergo reprojection, resampling and tile partitioning operations. The cube organization model includes dimension tables and fact tables. The fact table stores tile information and model information. The tile information includes the dimension index and the file path where the tile is located, the number of bytes of the tile and the byte offset. The model information includes the dimension index and the id of the model metadata record. The dimension table describes the time, space, product, band attributes, time applicability range, spatial applicability range, supported input data products and bands, applicable tasks and classification categories involved of the tile fact.

[0007] Optionally, tasks and models are matched based on explicit matching and implicit matching, including: obtaining an inference request input by a user, the inference request including a task type, a classification category, a spatial range, a spatial resolution and a time range; converting the user's inference request into a dimension query condition, performing a dimension query based on the dimension query condition and a cube organization model, and returning a collection result, the collection result including tile data under the spatiotemporal grid and a list of models available under the current inference task; calculating the comprehensive quality of each model in the model list, selecting a model with a comprehensive quality exceeding a given threshold as a candidate model, and if the candidate model does not have training data set information, selecting a model and tile data with the highest comprehensive quality as the optimal combination under the spatiotemporal grid; if the candidate model is accompanied by training data set information, implicitly matching the candidate model, for each tile data, traversing the model corresponding to the tile data to calculate the similarity between the tile data and each model, and selecting the model with the highest similarity as the final model; for different tile data under the same spatiotemporal grid, selecting a group of tile data and models with the highest model quality as the final combination under the corresponding grid.

[0008] Optionally, model quality evaluation comprehensively considers model performance , computational complexity and generalization ability , comprehensive quality The calculation formula is:

[0009]

[0010] in, is the F1 score index, is the GFLOPs index, is the computational complexity of the candidate model The maximum value of is the computational complexity of the candidate model The minimum value of for The calculation weight of for The calculation weight of .

[0011] Optionally, implicit matching is performed on the candidate model, including: traversing the training data set of the model, using the pre-trained deep learning model to extract the feature vector of each sample, where the feature is a comprehensive representation of the sample's texture, color, semantics, etc. dimensional vector, and hash mapping is performed on each feature vector to build a feature library. The hash function used conforms to the characteristics of the local sensitive hash function. For each set result, the pre-trained model is used to extract the feature vector of the tile data, and then the model list is traversed. For each model, the hash value of the feature vector of the tile data is calculated using the corresponding hash function, and the samples with the same hash value are found in the feature library, and the feature distance between the tile data and the sample is calculated. When the feature distance is greater than the threshold, the sample is determined to be a similar sample of the tile data, and the proportion of similar samples in the entire sample set is calculated, and the model with the largest proportion is taken as the best model for the current tile data.

[0012] Optionally, the distributed reasoning framework includes a Ray framework, wherein the distributed reasoning workflow based on the Ray framework includes reading cube tile data, conversion of memory objects, tile feature extraction and implicit matching, preprocessing of tiles to be inferred, distributed reasoning and post-processing; wherein, the reading and implicit matching of cube tile data are based on the parallelism of multiple CPU cores, and the tile feature extraction, preprocessing of tiles to be inferred and distributed reasoning are based on the parallelism of multiple GPUs; to avoid exceeding the hardware memory limit, each processing will be performed according to the maximum number of tiles under the CPU / GPU memory capacity; the reasoning workflow will automatically perform different preprocessing and post-processing links for different reasoning tasks; for semantic segmentation tasks, the preprocessing link includes mean normalization and standard deviation normalization, and the post-processing includes color mapping and splicing edge optimization; for target detection tasks, the preprocessing link is consistent with the semantic segmentation tasks, and the post-processing is the removal of duplicate detection boxes based on weighted box fusion.

[0013] Optionally, it also includes: providing a graphical interactive interface and a service interface, based on which the user is allowed to interactively input reasoning conditions on a preset page, and providing a real-time reasoning mode and a batch reasoning mode, based on which external programs are allowed to use the data services and reasoning services provided by the integrated system; wherein the real-time reasoning mode performs reasoning on tile data at the spatial level and range of the current viewport, and each time the user zooms the map or moves a layer, a new reasoning will be triggered; the batch mode allows the user to perform reasoning tasks on large-scale data, and allows the user to export reasoning results at a preset spatiotemporal range and resolution.

[0014] The second aspect of the present application provides an artificial intelligence spatiotemporal cube online reasoning service system, including: a data module, used to establish a cube organization model based on a spatiotemporal grid, wherein the cube organization model uniformly organizes and manages spatiotemporal data and geographic artificial intelligence models based on a spatiotemporal benchmark; a matching module, used to match tasks and models based on explicit matching and implicit matching, wherein explicit matching is to convert the user's reasoning request into a multi-dimensional query condition of the cube organization model to retrieve a set of candidate models, and implicit matching is to achieve model matching by calculating the feature similarity between the tile to be inferred and the candidate model; an application module, used for a workflow for distributed reasoning using an artificial intelligence model on the tile data to be inferred based on a distributed reasoning framework, wherein the tile data to be inferred is converted into a distributed memory object, and the artificial intelligence model is deployed on each computing node to execute the reasoning workflow in parallel on the tile data to be inferred.

[0015] The third aspect of the present application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement an artificial intelligence space-time cube online reasoning method as described in the above embodiment.

[0016] The fourth aspect of the present application provides a computer-readable storage medium on which a computer program is stored. The program is executed by a processor to implement the artificial intelligence space-time cube online reasoning method as described in the above embodiment.

[0017] Therefore, this application includes the following beneficial effects:

[0018] The embodiment of the present application achieves accurate matching of tasks and models by integrating spatiotemporal data and geographic artificial intelligence models, and by means of efficient explicit and implicit matching mechanisms. Under the distributed reasoning framework, the model is closely associated with the data, and tile data is processed in parallel, which significantly improves the efficiency and accuracy of reasoning. At the same time, the model optimizes system performance, enhances scalability, and helps promote the application and development of spatiotemporal big data and geographic intelligence in the future. As a result, technical problems such as low reasoning accuracy and slow reasoning speed of single models in multi-spatiotemporal or large-scale scenarios in related technologies are solved.

[0019] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0021] Figure 1A flowchart of an artificial intelligence space-time cube online reasoning method provided according to an embodiment of the present application;

[0022] Figure 2 A flowchart of an artificial intelligence space-time cube online reasoning method provided according to an embodiment of the present application;

[0023] Figure 3 This is an example diagram of an artificial intelligence space-time cube online reasoning service system provided according to an embodiment of the present application;

[0024] Figure 4 This is an example diagram of an artificial intelligence space-time cube online reasoning service system provided according to an embodiment of the present application;

[0025] Figure 5 Schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0027] The following describes the artificial intelligence spatiotemporal cube online reasoning method, service system and equipment of the embodiment of the present application with reference to the accompanying drawings. In response to the problem of lack of artificial intelligence reasoning ability in multi-spacetime or large-scale scenarios mentioned in the above background technology, the present application provides an artificial intelligence spatiotemporal cube online reasoning method, in which the precise matching of tasks and models is achieved by uniformly integrating spatiotemporal data and geographic artificial intelligence models, and by means of efficient explicit and implicit matching mechanisms. Under the distributed reasoning framework, the model is closely related to the data, and tile data is processed in parallel, which significantly improves the reasoning efficiency and accuracy. At the same time, the model optimizes system performance, enhances scalability, and helps promote the application and development of spatiotemporal big data and geographic intelligence in the future. As a result, the problems of low reasoning accuracy and slow reasoning speed of single models in multi-spacetime or large-scale scenarios in related technologies are solved.

[0028] Specifically, Figure 1 A flowchart of the artificial intelligence space-time cube online reasoning method provided in an embodiment of the present application.

[0029] like Figure 1 As shown, the artificial intelligence space-time cube online reasoning method includes the following steps:

[0030] In step S101, a cube organization model is established based on the space-time grid, wherein the cube organization model uniformly organizes and manages the space-time data and the geographic artificial intelligence model based on the space-time benchmark.

[0031] Among them, the space-time grid can be a grid system for organizing and indexing space-time data, the cube organization model can be a data and model organization framework based on the space-time grid, and the space-time benchmark can be a reference framework for defining the organization and indexing of space-time data and models, including space benchmark and time benchmark.

[0032] It can be understood that the embodiment of the present application realizes the unified and efficient management of spatiotemporal data and geographic artificial intelligence models through a cube organization model established based on the spatiotemporal grid, providing strong technical support and data foundation for geographic intelligence applications.

[0033] In an embodiment of the present application, when spatiotemporal data enters a cube, it needs to undergo reprojection, resampling, and tile partitioning operations. The cube organization model includes a dimension table and a fact table. The fact table stores tile information and model information. The tile information includes the dimension index and the file path where the tile is located, the number of bytes of the tile, and the byte offset. The model information includes the dimension index and the id of the model metadata record. The dimension table describes the time, space, product, and band attributes of the tile fact, the time applicable scope of the model fact, the spatial applicable scope, the data products and bands supported for input, as well as the applicable tasks and the classification categories involved.

[0034] It can be understood that the embodiment of the present application integrates spatiotemporal data into a cube organization model through reprojection, resampling, and tile partitioning. The model consists of a dimension table and a fact table. The fact table stores tile and model information for fast access and transmission; the dimension table provides rich context information to support accurate matching and reasoning analysis.

[0035] Specifically, data products from different remote sensing satellites are collected, including Gaofen-2, Beijing-2 and Sentinel-2. At the same time, a variety of artificial intelligence models for geospatial tasks are integrated, covering multiple fields such as land use classification, specific land object extraction, target detection and change detection, and each model is equipped with unified description information. A spatiotemporal data-artificial intelligence model cube framework is constructed, with EPSG:3857 as the unified spatial reference, and a total of 0-24 pyramid levels are divided. In the time dimension, the world standard time is used as the reference, and the time grid is refined to the second level to capture fine time changes.

[0036] During the data import process, the original remote sensing data is first reprojected to ensure that it matches the selected pyramid level spatial benchmark; then, the data is resampled using efficient methods such as nearest neighbor interpolation or bilinear interpolation to match the nearest pyramid level resolution; finally, according to the tile division rules of the level, the processed data is divided into several tiles and stored in the high-performance MinIO object using the COG (Cloud Optimized GeoTIFF, cloud-optimized labeled image file format) format.

[0037] When data and models are imported into the cube, fact tables and dimension tables will be constructed through tile information and model description information. The cube constructs two fact tables, one for storing tile data and the other for storing model information. In addition, three dimension tables are set up to describe in detail the time, spatial location, product, and band attributes of tile facts, comprehensively describing the temporal and spatial scope of application of the model, and applicable spatial data products and bands; at the same time, the task types served by the model and the classification system involved are added;

[0038] The embodiment of the present application uses the geoprocessing toolkit GDAL to implement tile reprojection, resampling and segmentation processing, and uses MinIO as a storage solution to ensure efficient data access and scalability. The model file is also stored in MinIO, and its metadata is structured in the relational database Postgresql. The model fact table is responsible for associating the dimension index with the model metadata record ID, thereby achieving fast indexing and access to model information.

[0039] In step S102, task and model matching is performed based on explicit matching combined with implicit matching, wherein explicit matching is to convert the user's reasoning request into a multi-dimensional query condition of the cube organization model to retrieve a set of candidate models, and implicit matching is to achieve model matching by calculating the feature similarity between the tile to be inferred and the candidate model.

[0040] Among them, explicit matching can be a method of directly converting user reasoning requests into query conditions to retrieve a set of candidate models, and implicit matching can be a method of selecting models based on feature similarity.

[0041] It is understandable that in the embodiment of the present application, explicit matching converts the user's reasoning request into a multi-dimensional query condition to quickly retrieve a set of candidate models; implicit matching further calculates the feature similarity between the candidate model and the artificial intelligence model to screen out the optimal model. By combining explicit matching with implicit matching, accurate matching between tasks and models is achieved, ensuring the reliability of reasoning results.

[0042] In an embodiment of the present application, task and model matching is performed based on explicit matching and implicit matching, including: obtaining an inference request input by a user, the inference request including a task type, a classification category, a spatial range, a spatial resolution, and a time range; converting the user's inference request into a dimension query condition, performing a dimension query based on the dimension query condition and a cube organization model, and returning a collection result, the collection result including tile data under a spatiotemporal grid and a list of models available under the current inference task; calculating the comprehensive quality of each model in the model list, selecting a model whose comprehensive quality exceeds a given threshold as a candidate model, and if the candidate model does not have training data set information, selecting a model and tile data with the highest comprehensive quality as the optimal combination under the spatiotemporal grid; if the candidate model is accompanied by training data set information, implicitly matching the candidate model, for each tile data, traversing the model corresponding to the tile data to calculate the similarity between the tile data and each model, and selecting the model with the highest similarity as the final model; for different tile data under the same spatiotemporal grid, selecting a group of tile data and models with the highest model quality as the final combination under the corresponding grid.

[0043] It is understandable that the embodiment of the present application converts the multi-dimensional reasoning request input by the user into the dimensional query condition of the cube organization model to efficiently retrieve the tile data and available models under the spatiotemporal grid. The model is initially screened based on its comprehensive quality, and further implicitly matched based on whether the training data set information is attached to select the most suitable combination of model and tile data. Finally, the optimal combination of data and model is selected for different tile data under the same spatiotemporal grid to improve the efficiency and accuracy of the execution of geographic intelligence tasks.

[0044] Specifically, the specific steps for matching tasks and models based on explicit matching and implicit matching are as follows:

[0045] Step 1.1, the user inputs the reasoning request, including the task type, classification category, spatial range, spatial resolution and time range, where the classification category can be set to empty for non-classification tasks;

[0046] Step 1.2: The user's reasoning request will be converted into dimensional query conditions, namely, task-category conditions, spatial range conditions, and time range conditions. Based on these dimensional conditions, a multi-dimensional query is performed on the cube, and a (tile, models) set is returned, where tile is tile data under a certain spatiotemporal grid, and models is a list of models available for the tile under the current reasoning task.

[0047] Step 1.3, calculate the comprehensive quality of each model in the model list, and select the model whose comprehensive quality exceeds a given threshold as the candidate model;

[0048] If the candidate model does not have training data set information, the model and tile with the highest comprehensive quality are selected as the optimal combination under the spatiotemporal grid; if the candidate model has training data set information, proceed to step 1.4;

[0049] Step 1.4: Perform implicit matching on the (tile, models) after step 1.3. For each tile, traverse its corresponding models to calculate the similarity between the tile and each model, and select the model with the highest similarity as the final model.

[0050] Step 1.5: For different tiles in the same space-time grid, select the group (tile, model) with the highest model quality as the final combination in the grid.

[0051] In the embodiment of the present application, the model quality evaluation comprehensively considers the model performance , computational complexity and generalization ability , comprehensive quality The calculation formula is:

[0052]

[0053] in, is the F1 score index, is the GFLOPs index, is the computational complexity of the candidate model The maximum value of is the computational complexity of the candidate model The minimum value of for The calculation weight of for The calculation weight of

[0054] It should be noted that in real-time inference mode , , in batch inference mode , .

[0055] In the embodiment of the present application, the candidate model is implicitly matched, including: traversing the training data set of the model, using the pre-trained deep learning model to extract the feature vector of each sample, the feature is a comprehensive representation of the texture, color, semantics and other features of the sample, and the feature is a dimensional vector, and hash mapping is performed on each feature vector to build a feature library. The hash function used conforms to the characteristics of the local sensitive hash function. For each set result, the pre-trained model is used to extract the feature vector of the tile data, and then the model list is traversed. For each model, the hash value of the feature vector of the tile data is calculated using the corresponding hash function, and the samples with the same hash value are found in the feature library, and the feature distance between the tile data and the sample is calculated. When the feature distance is greater than the threshold, the sample is determined to be a similar sample of the tile data, and the proportion of similar samples in the entire sample set is calculated, and the model with the largest proportion is taken as the best model for the current tile data.

[0056] It can be understood that the embodiment of the present application extracts feature vectors through a deep learning model, builds a feature library in combination with hash mapping, and uses cosine distance to evaluate the similarity between tile data and model training samples, and finally selects the model with the highest similarity as the best matching model, thereby achieving an efficient and accurate model and data matching process.

[0057] Specifically, the specific steps for implicitly matching candidate models are as follows:

[0058] Step 2.1: When the model is imported into the cube, the model features are first extracted and stored in the feature library. The specific method is to traverse the training data set of the model and use the pre-trained deep learning model to extract the features of each sample. The feature is a comprehensive representation of the sample's texture, color, semantics and other features. Each feature is a dimensional vector;

[0059] Step 2.2, then hash mapping is performed on each feature vector, and random projection is used as the hash function of the local sensitive hashing of the cosine distance, which is defined as follows:

[0060] Random generation conforms to normal distribution of indivual Dimensional vector , for each feature vector , whose hash value for:

[0061]

[0062] in, is the Heaviside step function used to extract real numbers Symbols:

[0063]

[0064] The resulting hash value is The length is The random vector and the hash value of each sample are stored in the feature library corresponding to the model;

[0065] Step 2.3: For each (tile, models), use the pre-trained model to extract the feature vector of the tile, then traverse the models, and for each model first use the corresponding indivual The hash value of the tile feature vector is calculated using a random vector, and then samples with the same hash value are found in the feature library, and the cosine distance between the tile and the sample is calculated:

[0066]

[0067] Given a distance threshold, when the cosine distance is greater than the threshold, the two are considered similar, and the proportion of similar samples in the entire sample set is calculated; after obtaining the fitness of all models, the model with the largest proportion is taken as the best model for the current tile, and (tile, model) is obtained.

[0068] In step S103, a distributed reasoning workflow is performed on the tile data to be reasoned using an artificial intelligence model based on a distributed reasoning framework, wherein the tile data to be reasoned is converted into a distributed memory object, and the artificial intelligence model is deployed on each computing node to execute the reasoning workflow in parallel on the tile data to be reasoned.

[0069] Among them, the distributed reasoning framework can be an inference system that supports parallel computing and distributed processing, and the tile data to be inferred refers to the spatiotemporal data fragments that require geographic artificial intelligence reasoning.

[0070] It can be understood that the embodiments of the present application realize parallel reasoning workflows by converting tile data into distributed memory objects and distributing artificial intelligence models on multiple computing nodes, thereby efficiently processing geospatial data, improving data processing efficiency and accuracy, and providing a new approach for the intelligent processing of geospatial data.

[0071] In an embodiment of the present application, a distributed reasoning framework includes a Ray framework, wherein a distributed reasoning workflow based on the Ray framework includes reading cube tile data, converting memory objects, extracting and implicitly matching tile features, preprocessing tiles to be inferred, distributed reasoning, and post-processing, wherein the reading and implicit matching of cube tile data are based on the parallelism of multiple CPU cores, and the tile feature extraction, preprocessing of tiles to be inferred, and distributed reasoning are based on the parallelism of multiple GPUs; to avoid exceeding the hardware memory limit, each processing will be performed according to the maximum number of tiles under the CPU / GPU memory capacity; the reasoning workflow will automatically perform different preprocessing and post-processing links for different reasoning tasks; for semantic segmentation tasks, the preprocessing link includes mean normalization and standard deviation normalization, and the post-processing includes color mapping and splicing edge optimization; for target detection tasks, the preprocessing link is consistent with the semantic segmentation tasks, and the post-processing is the removal of duplicate detection boxes based on weighted box fusion.

[0072] It can be understood that in the embodiment of the present application, an inference workflow is formulated based on the distributed inference framework Ray, which optimizes different inference tasks through automated pre-processing and post-processing, and flexibly allocates GPU resources to improve processing efficiency and scalability, which is suitable for intelligent analysis of large-scale geospatial data.

[0073] It should be noted that the GPU resource allocation method includes: the Ray framework allows a single working node to be allocated part of the resources of a GPU, and the set parameters are only logical values. When the task exceeds the set resources, it will automatically occupy the available resources. Based on this feature, an optimal GPU resource allocation method is implemented, specifically:

[0074] First, a lower number of GPU resources is allocated to each worker node, and the number of worker nodes is gradually increased. When the execution time begins to stabilize, it indicates that the GPU resources are fully utilized under the current number of worker nodes. The optimal number of GPU resources for a single worker node is approximately the total number of GPU resources divided by the number of worker nodes.

[0075] In an embodiment of the present application, it also includes: providing a graphical interaction interface and a service interface, based on the graphical interaction interface, allowing the user to interactively input reasoning conditions on a preset page, and providing real-time reasoning mode and batch reasoning mode, based on the service interface, allowing external programs to use the data services and reasoning services provided by the integrated system; wherein the real-time reasoning mode reasons about the tile data under the spatial level and range of the current viewport, and each time the user zooms the map or moves the layer, a new reasoning will be triggered; the batch mode allows the user to perform reasoning tasks on large-scale data, allowing the user to export reasoning results with a preset spatiotemporal range and resolution.

[0076] It can be understood that the embodiment of the present application provides real-time and batch reasoning modes through a graphical interactive interface, which facilitates users to interactively input reasoning conditions; at the same time, it supports external programs to integrate data services and reasoning services through a service interface to achieve flexible data processing and reasoning functions.

[0077] According to the artificial intelligence spatiotemporal cube online reasoning method proposed in the embodiment of the present application, the precise matching of tasks and models is achieved by uniformly integrating spatiotemporal data and geographic artificial intelligence models, and by means of efficient explicit and implicit matching mechanisms. Under the distributed reasoning framework, the model is closely associated with the data, and tile data is processed in parallel, which significantly improves the reasoning efficiency and accuracy. At the same time, the model optimizes system performance, enhances scalability, and helps promote the application and development of spatiotemporal big data and geographic intelligence in the future. As a result, the problems of low reasoning accuracy and slow reasoning speed of single models in multi-spatiotemporal or large-scale scenarios in related technologies are solved.

[0078] The following will be combined Figure 2 The online reasoning method of artificial intelligence space-time cube is specifically explained. The input reasoning request is: "Get the land use classification results of the entire target space in July 2021 with a spatial resolution of 10 meters. The categories include forest land, cultivated land, building land, bare land, and water bodies." The request is initiated through the user interaction interface of the Web platform and the content is as follows:

[0079] S1. The user inputs an inference request, the task type is "land use classification", the classification category is "forest land, cultivated land, building land, bare land, water body", the spatial range is "the entire target space", the spatial resolution is "10 meters" and the time range is "2021.07.01-2021.07.31";

[0080] S2. The user's reasoning request will be converted into a dimension query condition. Based on the dimension query condition, a cube query is performed on the cube, and the result is a set of (tiles, models) that meets the above query conditions.

[0081] S3. Calculate the comprehensive quality of each model in the model list and select the models whose comprehensive quality exceeds the given threshold as candidate models; model quality evaluation comprehensively considers model performance , computational complexity and generalization ability , comprehensive quality The calculation formula is:

[0082]

[0083] in, is the F1 score index, is the GFLOPs index, is the computational complexity of the candidate model The maximum value of is the computational complexity of the candidate model The minimum value of for The calculation weight of for The calculation weight of , this embodiment is a batch processing mode, that is ;

[0084] All candidate patterns are accompanied by training datasets and enter S4;

[0085] S4: Perform implicit matching on the (tile, models) after S3. For each tile, traverse its corresponding models to calculate the similarity between the tile and each model, and select the model with the highest similarity as the final model. The similarity here is defined as the total proportion of the number of samples in the model training data set that are similar to the tile to be inferred in all training data sets. To speed up the matching, the local sensitive hashing method is used to convert it into a nearest neighbor search for the model training data set.

[0086] It should be noted that the selection of the final model includes the following steps:

[0087] S4.1. When the model is imported into the cube, the model features are first extracted and stored in the feature library. The specific method is to traverse the training data set of the model and use the pre-trained deep learning model ResNet to extract the features of each sample. Each feature is a dimension vector, where ;

[0088] S4.2. Then, a hash map is performed on each feature vector, and random projection is used as the hash function of the local sensitive hashing of the cosine distance, which is defined as follows:

[0089] Random generation conforms to normal distribution of indivual Dimensional vector , for each feature vector , whose hash value for:

[0090]

[0091] in, is the Heaviside step function used to extract real numbers Symbols:

[0092]

[0093] The resulting hash value is The length is The binary code of the combination of 0 or 1, here ; The random vector and the hash value of each sample are stored in the feature library corresponding to the model;

[0094] S4.3: For each (tile, models), use the pre-trained model to extract the feature vector of the tile, then traverse the models, and for each model first use the corresponding indivual The hash value of the tile feature vector is calculated using a random vector, and then samples with the same hash value are found in the feature library, and the cosine distance between the tile and the sample is calculated:

[0095]

[0096] The distance threshold is set to 0.9. When the cosine distance is greater than the threshold, the two are considered similar, and the proportion of similar samples in the entire sample set is calculated. After obtaining the fitness of all models, the model with the largest proportion is taken as the best model for the current tile, and (tile, model) is obtained.

[0097] S5. For different tiles in the same spatiotemporal grid, select the group (tile, model) with the highest model quality as the final combination in the grid;

[0098] S6: Use the selected model to perform distributed reasoning on each tile; the reasoning engine is based on a set of reasoning workflows developed based on the distributed reasoning framework Ray, including reading cube tile data, converting memory objects, tile feature extraction and implicit matching, preprocessing of tiles to be reasoned, distributed reasoning and post-processing. The distributed reading and implicit matching of cube tiles are based on the parallelism of multiple CPU cores, and the tile feature extraction, preprocessing and distributed reasoning are based on the parallelism of multiple GPUs; to avoid exceeding the hardware memory limit, each processing will be processed according to the maximum number of tiles under the CPU / GPU memory capacity. This embodiment is a task of semantic segmentation. The preprocessing stage includes mean normalization and standard deviation normalization, and the post-processing includes color mapping and splicing edge optimization; 0.2 GPU resources are allocated to each distributed GPU computing node;

[0099] S7. The inference result is output in PNG format, and the file URL is returned to the front end for visualization.

[0100] In summary, the embodiment of the present application uses a snowflake schema cube to manage spatiotemporal data and models, automatically selects the optimal data and models through an intelligent matching algorithm, and relies on a distributed reasoning framework to achieve efficient parallel reasoning, thereby improving the speed and accuracy of large-scale spatiotemporal reasoning.

[0101] Next, the artificial intelligence space-time cube online reasoning service system proposed according to the embodiment of the present application is described with reference to the accompanying drawings.

[0102] Figure 3 It is a block diagram of the artificial intelligence space-time cube online reasoning service system of the embodiment of the present application.

[0103] like Figure 3 As shown, the artificial intelligence space-time cube online reasoning service system 10 includes: a data module 100, a matching module 200 and an application module 300.

[0104] Among them, the data module 100 is used to establish a cube organization model based on the spatiotemporal grid, wherein the cube organization model uniformly organizes and manages spatiotemporal data and geographic artificial intelligence models based on a spatiotemporal benchmark; the matching module 200 is used to match tasks and models based on explicit matching and implicit matching, wherein explicit matching is to convert the user's reasoning request into a multidimensional query condition of the cube organization model to retrieve a set of candidate models, and implicit matching is to achieve model matching by calculating the feature similarity between the tile to be inferred and the candidate model; the application module 300 is used to perform a distributed reasoning workflow on the tile data to be inferred using an artificial intelligence model based on a distributed reasoning framework, wherein the tile data to be inferred is converted into a distributed memory object, and the artificial intelligence model is deployed on each computing node to execute the reasoning workflow in parallel on the tile data to be inferred.

[0105] It should be noted that the aforementioned explanation of the embodiment of the artificial intelligence space-time cube online reasoning method is also applicable to the artificial intelligence space-time cube online reasoning service system of this embodiment, and will not be repeated here.

[0106] According to the artificial intelligence spatiotemporal cube online reasoning service system proposed in the embodiment of the present application, the precise matching of tasks and models is achieved by unifying and integrating spatiotemporal data with geographic artificial intelligence models and using efficient explicit and implicit matching mechanisms. Under the distributed reasoning framework, the model is closely associated with the data, and tile data is processed in parallel, which significantly improves the reasoning efficiency and accuracy. At the same time, the model optimizes system performance, enhances scalability, and helps promote the application and development of spatiotemporal big data and geographic intelligence in the future. As a result, the problems of low reasoning accuracy and slow reasoning speed of single models in multi-spatiotemporal or large-scale scenarios in related technologies are solved.

[0107] The following will be combined Figure 4The artificial intelligence space-time cube online reasoning service system is specifically described as follows:

[0108] The artificial intelligence space-time cube online reasoning service system can include infrastructure layer, data layer, computing layer and application layer. The layers work together to support powerful online reasoning service capabilities.

[0109] The infrastructure layer consists of high-performance CPU / GPU computing clusters, high-speed fiber switches, and large-capacity high-performance storage arrays, which not only ensure the efficient storage of massive spatiotemporal data and models, but also provide solid hardware support for more complex computing tasks, ensuring the smoothness and efficiency of the reasoning process.

[0110] The data layer is a physical implementation of the spatiotemporal data-AI model cube based on the snowflake model. The relational database Postgresql is responsible for storing dimension tables, fact tables, and model metadata tables to ensure that the data is structured and easy to query; the object storage MinIO is responsible for the storage of tile files and model files, optimizing the flexibility and scalability of data access; the non-relational database MongoDB is specifically used to store the training data set features of the model, and its flexible data model meets the storage requirements of complex data;

[0111] The computing layer is based on the reasoning engine of the distributed reasoning framework Ray, which provides cube query, model quality assessment, feature extraction, and nearest neighbor search functions in terms of data; and provides model deployment, data preprocessing, model reasoning, and post-processing functions in terms of reasoning.

[0112] The application layer mainly provides three online reasoning service modes: Web interactive platform, open geographic information service and online programming development; among them, the Web interactive platform uses the React framework to build intuitive and easy-to-use Web pages, supports users to enter reasoning conditions in a variety of ways such as map frame selection, check box selection, drop-down box selection, and uses Cesium components to achieve map visualization of results, which greatly improves the user experience; open geographic information services provide standard OGC API-Processes interface to achieve service standardization and easy integration, which is convenient for users to connect with external systems. In addition, the online programming development module also provides users with a professional online programming development environment and Python script library, supports users to call reasoning services through scripts and visualize calculation results in real time, and further enhances the flexibility and scalability of the system.

[0113] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include:

[0114] A memory 501 , a processor 502 , and a computer program stored in the memory 501 and executable on the processor 502 .

[0115] When the processor 502 executes the program, the artificial intelligence space-time cube online reasoning method provided in the above embodiment is implemented.

[0116] Furthermore, the electronic device further comprises:

[0117] The communication interface 503 is used for communication between the memory 501 and the processor 502 .

[0118] The memory 501 is used to store computer programs that can be executed on the processor 502 .

[0119] The memory 501 may include a high-speed RAM (Random Access Memory) memory, and may also include a non-volatile memory, such as at least one disk memory.

[0120] If the memory 501, the processor 502 and the communication interface 503 are implemented independently, the communication interface 503, the memory 501 and the processor 502 can be connected to each other through a bus and communicate with each other. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0121] Optionally, in a specific implementation, if the memory 501, the processor 502 and the communication interface 503 are integrated on a chip, the memory 501, the processor 502 and the communication interface 503 can communicate with each other through an internal interface.

[0122] The processor 502 may be a CPU (Central Processing Unit), or an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application.

[0123] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned artificial intelligence space-time cube online reasoning method.

[0124] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0125] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0126] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0127] It should be understood that the various parts of the present application can be implemented in hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, the steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array, a field programmable gate array, etc.

[0128] A person of ordinary skill in the art may understand that all or part of the steps carried by the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the above-mentioned program may be stored in a computer-readable storage medium, which, when executed, includes one of the steps of the method embodiment or a combination thereof.

[0129] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. An artificial intelligence space-time cube online reasoning method, characterized in that: The following steps are involved: Establishing a cube organization model based on the space-time grid, wherein the cube organization model uniformly organizes and manages the space-time data and the geographic artificial intelligence model based on the space-time benchmark; Task and model matching is performed based on explicit matching and in combination with implicit matching, wherein the explicit matching is to convert the user's reasoning request into a multidimensional query condition of the cube organization model to retrieve a set of candidate models, and the implicit matching is to achieve model matching by calculating the feature similarity between the tile to be inferred and the candidate model; the task and model matching based on explicit matching and in combination with implicit matching includes: obtaining a reasoning request input by the user, the reasoning request including a task type, a classification category, a spatial range, a spatial resolution and a time range; converting the user's reasoning request into a dimensional query condition, performing a dimensional query based on the dimensional query condition and the cube organization model, and returning a collection result, the collection result including a spatiotemporal network The tile data under the grid and the list of models available under the current inference task; calculate the comprehensive quality of each model in the model list, select the model with a comprehensive quality exceeding a given threshold as the candidate model, if the candidate model has no training data set information, select the model and tile data with the highest comprehensive quality as the optimal combination under the spatiotemporal grid; if the candidate model comes with training data set information, implicitly match the candidate model, for each tile data, traverse the model corresponding to the tile data to calculate the similarity between the tile data and each model, and select the model with the highest similarity as the final model; for different tile data under the same spatiotemporal grid, select a group of tile data and models with the highest model quality as the final combination under the corresponding grid; model quality evaluation comprehensively considers model performance , computational complexity and generalization ability , comprehensive quality The calculation formula is: in, is the F1 score index, is the GFLOPs index, is the computational complexity of the candidate model The maximum value of is the computational complexity of the candidate model The minimum value of for The calculation weight of for The calculation weight of A workflow for performing distributed reasoning on the tile data to be reasoned using an artificial intelligence model based on a distributed reasoning framework, wherein the tile data to be reasoned is converted into a distributed memory object, and the artificial intelligence model is deployed on each computing node to execute the reasoning workflow in parallel on the tile data to be reasoned.

2. The artificial intelligence space-time cube online reasoning method according to claim 1 is characterized in that: When the spatiotemporal data enters the cube, it needs to go through the operations of reprojection, resampling and tile division. The cube organization model includes a dimension table and a fact table. The fact table stores tile information and model information. The tile information includes the dimension index and the file path where the tile is located, the number of bytes of the tile and the byte offset. The model information includes the dimension index and the id of the model metadata record; the dimension table describes the time, space, product, band attributes, time applicability range, spatial applicability range, supported input data products and bands, applicable tasks and classification categories involved of the tile fact.

3. The artificial intelligence space-time cube online reasoning method according to claim 1 is characterized in that: The implicit matching of the candidate models includes: Traverse the training data set of the model and use the pre-trained deep learning model to extract the feature vector of each sample. The feature is a comprehensive representation of the sample's texture, color, semantics and other features. This feature is a dimensional vector, hash mapping is performed on each feature vector to construct a feature library, and the hash function used conforms to the characteristics of the local sensitive hash function; For each set result, use the pre-trained model to extract the feature vector of the tile data, then traverse the model list, use the corresponding hash function for each model to calculate the hash value of the feature vector of the tile data, find samples with the same hash value in the feature library, and calculate the feature distance between the tile data and the sample; When the feature distance is greater than a threshold, the sample is determined to be a similar sample of the tile data, and the proportion of similar samples in the entire sample set is calculated, and the model with the largest proportion is taken as the best model suitable for the current tile data.

4. The artificial intelligence space-time cube online reasoning method according to claim 1 is characterized in that: The distributed reasoning framework includes a Ray framework, wherein the distributed reasoning workflow based on the Ray framework includes reading of cube tile data, conversion of memory objects, tile feature extraction and implicit matching, preprocessing of tiles to be inferred, distributed reasoning and post-processing, wherein the reading of cube tile data and the implicit matching are based on the parallelism of multiple CPU cores, and the tile feature extraction, the preprocessing of tiles to be inferred and the distributed reasoning are based on the parallelism of multiple GPUs; to avoid exceeding the limitation of hardware memory, each processing will be processed according to the maximum number of tiles under the CPU / GPU memory capacity; The reasoning workflow will automatically perform different preprocessing and postprocessing steps for different reasoning tasks; for semantic segmentation tasks, the preprocessing step includes mean normalization and standard deviation normalization, and the postprocessing includes color mapping and splicing edge optimization; for target detection tasks, the preprocessing step is consistent with the semantic segmentation task, and the postprocessing is the removal of duplicate detection frames based on weighted frame fusion.

5. The artificial intelligence space-time cube online reasoning method according to claim 1 is characterized in that: Also includes: Provide a graphical interactive interface and a service interface, based on which the graphical interactive interface allows the user to interactively input reasoning conditions on a preset page, and provides a real-time reasoning mode and a batch reasoning mode, and based on the service interface allows external programs to use the data services and reasoning services provided by the integrated system; wherein, The real-time reasoning mode infers the tile data under the spatial level and range of the current viewport. Each time the user zooms the map or moves the layer, a new reasoning will be triggered; Batch mode allows users to perform reasoning tasks on large-scale data and allows users to export reasoning results at a preset spatiotemporal range and resolution.

6. An artificial intelligence space-time cube online reasoning service system, characterized in that: include: A data module, used to establish a cube organization model based on a spatiotemporal grid, wherein the cube organization model uniformly organizes and manages spatiotemporal data and geographic artificial intelligence models based on a spatiotemporal reference; A matching module is used to match tasks and models based on explicit matching and implicit matching, wherein the explicit matching is to convert the user's reasoning request into a multidimensional query condition of the cube organization model to retrieve a set of candidate models, and the implicit matching is to achieve model matching by calculating the feature similarity between the tile to be inferred and the candidate model; the task and model matching based on explicit matching and implicit matching includes: obtaining an reasoning request input by the user, the reasoning request including a task type, a classification category, a spatial range, a spatial resolution and a time range; converting the user's reasoning request into a dimensional query condition, performing a dimensional query based on the dimensional query condition and the cube organization model, and returning a set result, the set result including The tile data under the spatiotemporal grid and the list of models available under the current reasoning task; calculate the comprehensive quality of each model in the model list, select the model with a comprehensive quality exceeding a given threshold as the candidate model, and if the candidate model has no training data set information, select the model and tile data with the highest comprehensive quality as the optimal combination under the spatiotemporal grid; if the candidate model comes with training data set information, implicitly match the candidate model, for each tile data, traverse the model corresponding to the tile data to calculate the similarity between the tile data and each model, and select the model with the highest similarity as the final model; for different tile data in the same spatiotemporal grid, select a set of tile data and models with the highest model quality as the final combination under the corresponding grid; model quality evaluation comprehensively considers model performance , computational complexity and generalization ability , comprehensive quality The calculation formula is: in, is the F1 score index, is the GFLOPs index, is the computational complexity of the candidate model The maximum value of is the computational complexity of the candidate model The minimum value of for The calculation weight of for The calculation weight of An application module is used to perform a distributed reasoning workflow on the tile data to be reasoned using an artificial intelligence model based on a distributed reasoning framework, wherein the tile data to be reasoned is converted into a distributed memory object, and the artificial intelligence model is deployed on each computing node to execute the reasoning workflow in parallel on the tile data to be reasoned.

7. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the artificial intelligence space-time cube online reasoning method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed, the artificial intelligence space-time cube online reasoning method described in any one of claims 1 to 5 is implemented.

Images