Large-scale intelligence data analysis method and system based on multi-modal data

Abstract

The invention discloses a multi-modal data-based large-scale information data analysis method and system, and the method comprises the steps: obtaining multi-modal data which comprises optical image data, radar image data, thermal infrared data, hyperspectral data, magnetic field data and navigation data; preprocessing the multi-modal data to generate preprocessed multi-modal data, and performing data fusion on the preprocessed multi-modal data to generate a data set containing a plurality of data points; a plurality of clusters are initialized, a multi-modal data clustering model is set, a data set containing a plurality of data points is aggregated according to the clusters, the data points are divided into the corresponding clusters, and a division result is visualized.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligence analysis, and more specifically, relates to a large-scale intelligence data analysis method and system based on multimodal data. Background Art

[0002] Satellite data can include many types of data, depending on the satellite’s sensors and mission. In general, satellite data can include the following main types of information: Optical imagery data: This is one of the most common types of satellite data, using optical sensors to capture images of the Earth's surface in the visible and infrared spectrum.

[0003] Radar image data: Radar satellites use microwave radiation to obtain images of the Earth’s surface. They can penetrate clouds and weather interference and have a high level of insight for observing the ground and ocean.

[0004] Thermal infrared data: These data are obtained by thermal infrared sensors and are used to measure the temperature distribution of the surface and objects, which helps to study surface temperature changes and environmental thermal characteristics.

[0005] Hyperspectral data: This type of data is obtained through spectral imaging technology and can provide more detailed spectral information. It has important applications in vegetation classification, soil composition analysis, etc.

[0006] Gravity and magnetic field data: Some satellites can also measure the Earth's gravity field and magnetic field, which is of great significance for geological exploration, geophysical research and other fields.

[0007] Position and navigation data: Precise position and navigation services are also provided by satellite systems, such as the Global Positioning System (GPS).

[0008] These data types are typically transmitted via satellite to ground stations, where they are then processed and analyzed for a variety of applications such as scientific research, resource management, environmental monitoring, and national defense security.

[0009] However, there is no technical solution in the prior art that can intelligently and efficiently analyze the above multi-modal data. Summary of the invention

[0010] In order to solve the above technical problems, the present invention proposes a large-scale intelligence data analysis method based on multimodal data, wherein the multimodal data is satellite data, including: Acquiring multimodal data, wherein the multimodal data includes: optical image data, radar image data, thermal infrared data, hyperspectral data, magnetic field data and navigation data; Preprocessing the multimodal data to generate preprocessed multimodal data, and fusing the preprocessed multimodal data to generate a data set containing multiple data points; Initialize multiple clusters and set up a multimodal data clustering model. Aggregate data sets containing multiple data points according to clusters, divide the data points into corresponding clusters, and visualize the division results.

[0011] Furthermore, the multimodal data clustering model includes: , in, is the number of clusters, is the first cluster weight, For the Cluster The Data points In time The weight of For the Cluster The data points, For the Data points and Data points The standard deviation of is the second clustering weight, For the Cluster The data points, For the Data points and Data points The standard deviation of is the third weight of clustering, For the Cluster The cluster center, For the Cluster The symmetric positive definite weight matrix is ​​used to adjust the weights and correlations of each dimension of the data point. For the Cluster and Cluster The connection weight between .

[0012] Furthermore, calculate the Cluster and Cluster The connection weight between include: , in, For the Cluster The cluster center, is an adjustment factor to control the distance between cluster centers, is the adjustment factor to control the similarity between data points, For the Data points and Data points The inner product of .

[0013] Further, calculate the third cluster weight include: , in, is the adjustment factor of the third weight of clustering, For the Cluster Dynamic weight of .

[0014] Furthermore, the clusters are: vegetation cluster, water cluster, building cluster, geological activity cluster and mountain cluster.

[0015] The present invention also proposes a large-scale intelligence data analysis system based on multimodal data, wherein the multimodal data is satellite data, and is characterized by comprising: A data acquisition module, used to acquire multimodal data, wherein the multimodal data includes: optical image data, radar image data, thermal infrared data, hyperspectral data, magnetic field data and navigation data; A data fusion module is used to preprocess the multimodal data to generate preprocessed multimodal data, and to fuse the preprocessed multimodal data to generate a data set containing multiple data points; The partitioning module is used to initialize multiple clusters and set up a multimodal data clustering model, aggregate a data set containing multiple data points according to clusters, partition the data points into corresponding clusters, and visualize the partitioning results.

[0016] Furthermore, the multimodal data clustering model includes: , in, is the number of clusters, is the first cluster weight, For the Cluster The Data points In time The weight of For the Cluster The data points, For the Data points and Data points The standard deviation of is the second clustering weight, For the Cluster The data points, For the Data points and Data points The standard deviation of is the third weight of clustering, For the Cluster The cluster center, For the Cluster The symmetric positive definite weight matrix is ​​used to adjust the weights and correlations of each dimension of the data point. For the Cluster and Cluster The connection weight between .

[0017] Furthermore, calculate the Cluster and Cluster The connection weight between include: , in, For the Cluster The cluster center, is an adjustment factor to control the distance between cluster centers, is the adjustment factor to control the similarity between data points, For the Data points and Data points The inner product of .

[0018] Further, calculate the third cluster weight include: , in, is the adjustment factor of the third weight of clustering, For the Cluster Dynamic weight of .

[0019] Furthermore, the clusters are: vegetation cluster, water cluster, building cluster, geological activity cluster and mountain cluster.

[0020] In general, the above technical solution conceived by the present invention has the following beneficial effects compared with the prior art: The invention preprocesses the multimodal data to generate preprocessed multimodal data, fuses the preprocessed multimodal data, generates a data set containing multiple data points, initializes multiple clusters, and sets a multimodal data clustering model, aggregates the data set containing multiple data points according to the clusters, divides the data points into corresponding clusters, and visualizes the division results. The invention can perform intelligent analysis on complex multimodal satellite data through the above technical scheme. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a flow chart of the method of embodiment 1 of the present invention; Figure 2 It is a structural diagram of the system of embodiment 2 of the present invention. DETAILED DESCRIPTION

[0022] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0023] The method provided by the present invention can be implemented in the following terminal environment, and the terminal may include one or more of the following components: a processor, a storage medium, and a display screen. The storage medium stores at least one instruction, and the instruction is loaded and executed by the processor to implement the method described in the following embodiment.

[0024] The processor may include one or more processing cores. The processor uses various interfaces and lines to connect various parts of the entire terminal, and executes various functions of the terminal and processes data by running or executing instructions, programs, code sets or instruction sets stored in the storage medium, and calling data stored in the storage medium.

[0025] The storage medium may include a random access memory (RAM) or a read-only memory (ROM). The storage medium may be used to store instructions, programs, codes, code sets or instructions.

[0026] The display screen is used to display the user interface of each application.

[0027] In addition, those skilled in the art can understand that the structure of the above terminal does not constitute a limitation on the terminal, and the terminal may include more or fewer components, or combine certain components, or arrange the components differently. For example, the terminal also includes components such as a radio frequency circuit, an input unit, a sensor, an audio circuit, and a power supply, which will not be described in detail here.

[0028] Example 1 like Figure 1 As shown, an embodiment of the present invention provides a large-scale intelligence data analysis method based on multimodal data, wherein the multimodal data is satellite data, including: Step 101, acquiring multimodal data, wherein the multimodal data includes: optical image data, radar image data, thermal infrared data, hyperspectral data, magnetic field data and navigation data; Step 102, preprocessing the multimodal data to generate preprocessed multimodal data, performing data fusion on the preprocessed multimodal data (for example, in this embodiment, data fusion is performed by support vector machines (SVM)) to generate a data set containing multiple data points; Specifically, the multimodal data clustering model includes: , in, is the number of clusters, is the first cluster weight, For the Cluster The Data points In time The weight of For the Cluster The data points, For the Data points and Data points The standard deviation of is the second clustering weight, For the Cluster The data points, For the Data points and Data points The standard deviation of is the third weight of clustering, For the Cluster The cluster center, For the Cluster The symmetric positive definite weight matrix is ​​used to adjust the weights and correlations of each dimension of the data point. For the Cluster and Cluster The connection weight between .

[0029] Specifically, calculate Cluster and Cluster The connection weight between include: , in, For the Cluster The cluster center, is an adjustment factor to control the distance between cluster centers, is the adjustment factor to control the similarity between data points, For the Data points and Data points The inner product of .

[0030] Specifically, calculate the third weight of clustering include: , in, is the adjustment factor of the third weight of clustering, For the Cluster Dynamic weight of .

[0031] Step 103, initialize multiple clusters, set a multimodal data clustering model, aggregate a data set containing multiple data points according to the clusters, divide the data points into corresponding clusters, and visualize the division results.

[0032] Specifically, the clusters are: vegetation cluster, water cluster, building cluster, geological activity cluster and mountain cluster.

[0033] Example 2 like Figure 2 As shown, an embodiment of the present invention further provides a large-scale intelligence data analysis system based on multimodal data, wherein the multimodal data is satellite data, including: A data acquisition module, used to acquire multimodal data, wherein the multimodal data includes: optical image data, radar image data, thermal infrared data, hyperspectral data, magnetic field data and navigation data; A data fusion module is used to preprocess the multimodal data to generate preprocessed multimodal data, and to fuse the preprocessed multimodal data to generate a data set containing multiple data points; Specifically, the multimodal data clustering model includes: , in, is the number of clusters, is the first cluster weight, For the Cluster The Data points In time The weight of For the Cluster The data points, For the Data points and Data points The standard deviation of is the second clustering weight, For the Cluster The data points, For the Data points and Data points The standard deviation of is the third weight of clustering, For the Cluster The cluster center, For the Cluster The symmetric positive definite weight matrix is ​​used to adjust the weights and correlations of each dimension of the data point. For the Cluster and Cluster The connection weight between .

[0034] Specifically, calculate Cluster and Cluster The connection weight between include: , in, For the Cluster The cluster center, is an adjustment factor to control the distance between cluster centers, is the adjustment factor to control the similarity between data points, For the Data points and Data points The inner product of .

[0035] Specifically, calculate the third weight of clustering include: , in, is the adjustment factor of the third weight of clustering, For the Cluster Dynamic weight of .

[0036] The partitioning module is used to initialize multiple clusters and set up a multimodal data clustering model, aggregate a data set containing multiple data points according to clusters, partition the data points into corresponding clusters, and visualize the partitioning results.

[0037] Specifically, the clusters are: vegetation cluster, water cluster, building cluster, geological activity cluster and mountain cluster.

[0038] Example 3 An embodiment of the present invention further proposes a storage medium storing a plurality of instructions, wherein the instructions are used to implement the large-scale intelligence data analysis method based on multimodal data.

[0039] Optionally, in this embodiment, the above storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

[0040] Optionally, in this embodiment, the storage medium is configured to store program codes for executing the following steps: Step 101, acquiring multimodal data, wherein the multimodal data includes: optical image data, radar image data, thermal infrared data, hyperspectral data, magnetic field data, and navigation data; Step 102, preprocessing the multimodal data to generate preprocessed multimodal data, and fusing the preprocessed multimodal data to generate a data set containing multiple data points; Specifically, the multimodal data clustering model includes: , in, is the number of clusters, is the first cluster weight, For the Cluster The Data points In time The weight of For the Cluster The data points, For the Data points and Data points The standard deviation of is the second clustering weight, For the Cluster The data points, For the Data points and Data points The standard deviation of is the third weight of clustering, For the Cluster The cluster center, For the Cluster The symmetric positive definite weight matrix is ​​used to adjust the weights and correlations of each dimension of the data point. For the Cluster and Cluster The connection weight between .

[0041] Specifically, calculate Cluster and Cluster The connection weight between include: , in, For the Cluster The cluster center, is an adjustment factor to control the distance between cluster centers, is the adjustment factor to control the similarity between data points, For the Data points and Data points The inner product of .

[0042] Specifically, calculate the third weight of clustering include: , in, is the adjustment factor of the third weight of clustering, For the Cluster Dynamic weight of .

[0043] Step 103, initialize multiple clusters, set a multimodal data clustering model, aggregate a data set containing multiple data points according to the clusters, divide the data points into corresponding clusters, and visualize the division results.

[0044] Specifically, the clusters are: vegetation cluster, water cluster, building cluster, geological activity cluster and mountain cluster.

[0045] Example 4 An embodiment of the present invention also proposes an electronic device, including a processor and a storage medium connected to the processor, wherein the storage medium stores a plurality of instructions, and the instructions can be loaded and executed by the processor so that the processor can execute the large-scale intelligence data analysis method based on multimodal data.

[0046] Specifically, the electronic device of this embodiment may be a computer terminal, and the computer terminal may include: one or more processors, and a storage medium.

[0047] Among them, the storage medium can be used to store software programs and modules, such as a large-scale intelligence data analysis method based on multimodal data in an embodiment of the present invention, corresponding program instructions / modules, and the processor executes various functional applications and data processing by running the software programs and modules stored in the storage medium, that is, realizing the above-mentioned large-scale intelligence data analysis method based on multimodal data. The storage medium may include a high-speed random storage medium, and may also include a non-volatile storage medium, such as one or more magnetic storage systems, flash memory, or other non-volatile solid-state storage media. In some instances, the storage medium may further include a storage medium remotely arranged relative to the processor, and these remote storage media may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0048] The processor may call the information and application program stored in the storage medium through the transmission system to perform the following steps: Step 101, acquiring multimodal data, wherein the multimodal data includes: optical image data, radar image data, thermal infrared data, hyperspectral data, magnetic field data and navigation data; Step 102, preprocessing the multimodal data to generate preprocessed multimodal data, and fusing the preprocessed multimodal data to generate a data set containing multiple data points; Specifically, the multimodal data clustering model includes: , in, is the number of clusters, is the first cluster weight, For the Cluster The Data points In time The weight of For the Cluster The data points, For the Data points and Data points The standard deviation of is the second clustering weight, For the Cluster The data points, For the Data points and Data points The standard deviation of is the third weight of clustering, For the Cluster The cluster center, For the Cluster The symmetric positive definite weight matrix is ​​used to adjust the weights and correlations of each dimension of the data point. For the Cluster and Cluster The connection weight between .

[0049] Specifically, calculate Cluster and Cluster The connection weight between include: , in, For the Cluster The cluster center, is an adjustment factor to control the distance between cluster centers, is the adjustment factor to control the similarity between data points, For the Data points and Data points The inner product of .

[0050] Specifically, calculate the third weight of clustering include: , in, is the adjustment factor of the third weight of clustering, For the Cluster Dynamic weight of .

[0051] Step 103, initialize multiple clusters, set a multimodal data clustering model, aggregate a data set containing multiple data points according to the clusters, divide the data points into corresponding clusters, and visualize the division results.

[0052] Specifically, the clusters are: vegetation cluster, water cluster, building cluster, geological activity cluster and mountain cluster.

[0053] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0054] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0055] In the several embodiments provided by the present invention, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the system embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0056] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0057] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0058] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only storage medium (ROM, Read-Only Memory), random access storage medium (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0059] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the invention.

Claims

1. A large-scale intelligence data analysis method based on multimodal data, wherein the multimodal data is satellite data, characterized in that: include: Acquiring multimodal data, wherein the multimodal data includes: optical image data, radar image data, thermal infrared data, hyperspectral data, magnetic field data and navigation data; Preprocessing the multimodal data to generate preprocessed multimodal data, and fusing the preprocessed multimodal data to generate a data set containing multiple data points; Initialize multiple clusters and set up a multimodal data clustering model. Aggregate data sets containing multiple data points according to clusters, divide the data points into corresponding clusters, and visualize the division results.

2. A large-scale intelligence data analysis method based on multimodal data as claimed in claim 1, characterized in that: The multimodal data clustering model includes: ， in, is the number of clusters, is the first cluster weight, For the Cluster The Data points In time The weight of For the Cluster The data points, For the Data points and Data points The standard deviation of is the second clustering weight, For the Cluster The data points, For the Data points and Data points The standard deviation of is the third weight of clustering, For the Cluster The cluster center, For the Cluster The symmetric positive definite weight matrix is ​​used to adjust the weights and correlations of each dimension of the data point. For the Cluster and Cluster The connection weight between .

3. A large-scale intelligence data analysis method based on multimodal data as claimed in claim 2, characterized in that: Calculate the Cluster and Cluster The connection weight between include: ， in, For the Cluster The cluster center, is an adjustment factor to control the distance between cluster centers, is the adjustment factor to control the similarity between data points, For the Data points and Data points The inner product of .

4. The large-scale intelligence data analysis method based on multimodal data as claimed in claim 2, characterized in that: Calculate the third weight of clustering include: ， in, is the adjustment factor of the third weight of clustering, For the Cluster Dynamic weight of .

5. The large-scale intelligence data analysis method based on multimodal data as claimed in claim 1, characterized in that: The clusters are: vegetation cluster, water cluster, building cluster, geological activity cluster and mountain cluster.

6. A large-scale intelligence data analysis system based on multimodal data, wherein the multimodal data is satellite data, characterized in that: include: A data acquisition module, used to acquire multimodal data, wherein the multimodal data includes: optical image data, radar image data, thermal infrared data, hyperspectral data, magnetic field data and navigation data; A data fusion module is used to preprocess the multimodal data to generate preprocessed multimodal data, and to fuse the preprocessed multimodal data to generate a data set containing multiple data points; The partitioning module is used to initialize multiple clusters and set up a multimodal data clustering model, aggregate a data set containing multiple data points according to clusters, partition the data points into corresponding clusters, and visualize the partitioning results.

7. A large-scale intelligence data analysis system based on multimodal data as claimed in claim 6, characterized in that: The multimodal data clustering model includes: ， in, is the number of clusters, is the first cluster weight, For the Cluster The Data points In time The weight of For the Cluster The data points, For the Data points and Data points The standard deviation of is the second clustering weight, For the Cluster The data points, For the Data points and Data points The standard deviation of is the third weight of clustering, For the Cluster The cluster center, For the Cluster The symmetric positive definite weight matrix is ​​used to adjust the weights and correlations of each dimension of the data point. For the Cluster and Cluster The connection weight between .

8. A large-scale intelligence data analysis system based on multimodal data as claimed in claim 7, characterized in that: Calculate the Cluster and Cluster The connection weight between include: ， in, For the Cluster The cluster center, is an adjustment factor to control the distance between cluster centers, is the adjustment factor to control the similarity between data points, For the Data points and Data points The inner product of .

9. A large-scale intelligence data analysis system based on multimodal data as claimed in claim 7, characterized in that: Calculate the third weight of clustering include: ， in, is the adjustment factor of the third cluster weight, For the Cluster Dynamic weight of .

10. A large-scale intelligence data analysis system based on multimodal data as claimed in claim 6, characterized in that: The clusters are: vegetation cluster, water cluster, building cluster, geological activity cluster and mountain cluster.

Images