Situational information visualization analysis methods, devices, electronic equipment and storage media

Abstract

This invention relates to a method, apparatus, electronic device, and storage medium for situational information visualization analysis. The method includes: acquiring data sources by monitoring data in a message consumption queue using a computing engine; cleaning and preprocessing the data sources; and storing the preprocessed data sources in a database. An algorithm platform is invoked to perform calculations and in-depth analysis on the data sources in the database to obtain a situational information profile, which is then stored in a data profile information database for real-time retrieval and mapping to the information visualization layer. The data sources include multi-source, multi-type comprehensive situational data, including structured and unstructured data. The algorithm platform is built upon multiple basic algorithms and machine learning frameworks. The basic algorithms include at least text mining algorithms, classification algorithms, regression algorithms, clustering algorithms, and recommendation algorithms, improving the comprehensiveness, accuracy, and visual intuitiveness of multi-source situational data analysis.

Description

Technical Field

[0001] This invention relates to the field of integrated situational awareness technology, and in particular to a situational information visual analysis method, apparatus, electronic device, and storage medium. Background Technology

[0002] The field of integrated situational awareness provides a comprehensive, multi-dimensional display of multi-source situational information from different angles and levels. This supports a complete and timely understanding of the current capabilities in the spatial environment domain, serves as the decision-making basis for typical tasks such as mission planning and implementation, and is a crucial guarantee for the accurate achievement of missions. With the rapid development of information technology, integrated situational awareness has been elevated to a strategic level and has become a focal point in the field of integrated information display. Furthermore, integrated situational awareness is an organic combination of various situational information. Only by standardizing and systematically processing multi-source situational data resources and extracting them in an orderly manner to form situational information, and by employing specific organizational and management methods, can the integrity and accuracy of the situational information be ensured during processing, analysis, and display.

[0003] Currently, amidst the surging data deluge of the big data era, situational awareness data is becoming increasingly diverse and massive, exhibiting distinct characteristics such as high volume, multiple types, difficulty in identification, and high value. With the deepening integration of business and information technology, the challenge for integrated situational awareness technology is no longer data scarcity, but rather how to extract value from massive amounts of data and make it usable. However, traditional integrated situational awareness technologies are often hampered by the sheer volume and variety of situational data, resulting in incomplete analysis, low accuracy, and insufficiently intuitive visualization. Summary of the Invention

[0004] Therefore, it is necessary to provide a situation information visual analysis method, device, electronic device, and storage medium that can improve the comprehensiveness, accuracy, and visual intuitiveness of situation data from multiple sources and of multiple types, in order to address the above-mentioned technical problems.

[0005] This invention provides a situational information visual analysis method, the method comprising:

[0006] The computing engine obtains the data source by listening to the data in the message consumption queue, and performs cleaning and preprocessing on the data source so as to store the cleaned and preprocessed data source in the database.

[0007] The algorithm platform is invoked to perform calculations and in-depth analysis on the data sources in the database to obtain a situation information profile, and the situation information profile is stored in the data profile information database so that the situation information profile can be retrieved in real time and mapped to the information visualization layer.

[0008] The data source includes multi-source, multi-type comprehensive situational data, which includes structured and unstructured data. The algorithm platform is built based on a variety of basic algorithms and machine learning frameworks, and the basic algorithms include at least text mining algorithms, classification algorithms, regression algorithms, clustering algorithms, and recommendation algorithms.

[0009] In one embodiment, the step of obtaining a data source by listening to data in the message consumption queue through a computing engine, and performing cleaning and preprocessing on the data source to store the cleaned and preprocessed data source in a database includes:

[0010] The unstructured data is extracted from the data source, and the unstructured data is filtered for data quality using a defined filtering model or filtering rules to remove low-quality data in the unstructured data that does not meet the set requirements.

[0011] The unstructured data after filtering is deduplicated using Bloom filter technology to obtain preprocessed unstructured data, and the structured data and the preprocessed unstructured data are stored in the database.

[0012] In one embodiment, the algorithm platform performs calculations and in-depth analysis on the data source in the database to obtain a situational information profile, and stores the situational information profile in a data profile information database for real-time retrieval of the situational information profile and mapping it to the information visualization layer. This includes the following steps:

[0013] The algorithm platform is built based on the aforementioned text mining algorithm, classification algorithm, regression algorithm, clustering algorithm, and recommendation algorithm, combined with the machine learning framework Flink.

[0014] The algorithm platform is used to abstract calculation and in-depth analysis algorithms for multi-type integrated situational data from various sources, so as to optimize the algorithm platform and obtain a calculation framework for each type of integrated situational data.

[0015] In one embodiment, the algorithm platform performs calculations and in-depth analysis on the data source in the database to obtain a situational information profile, and stores the situational information profile in a data profile information database to retrieve the situational information profile and map it to the information visualization layer in real time, including:

[0016] The algorithm platform is invoked to establish an analysis and processing model corresponding to each type of comprehensive situation data based on different types of comprehensive situation data.

[0017] The analysis and processing model is used to fit multiple processing and analysis tasks corresponding to the same type of comprehensive situation data to obtain processing and analysis results.

[0018] Among them, the integrated situation data of the same type share data, and each processing and analysis task corresponding to the integrated situation data of the same type has shared parameters. The processing and analysis result is obtained by fusing the analysis results output of multiple sub-tasks of each processing and analysis task.

[0019] In one embodiment, the algorithm platform performs calculations and in-depth analysis on the data sources in the database to obtain a situational information profile, and stores the situational information profile in a data profile information database to retrieve the situational information profile and map it to the information visualization layer in real time. The method also includes:

[0020] The fusion weight of each subtask in the processing and analysis task is determined based on the data dispersion and data scale of the comprehensive situation data of the corresponding type, and the fusion formula is called to fuse the analysis results of multiple subtasks according to the fusion weight of each subtask.

[0021] The formula for calculating the fusion weight is as follows:

[0022] ;

[0023] In the formula, The degree of data dispersion of the comprehensive situation data. The data size of the comprehensive situational data is... For weighting;

[0024] The fusion formula is as follows:

[0025] ;

[0026] In the formula, result is the total score after fusion, n is the number of subtasks, and result i Output the analysis results for the i-th subtask. Let be the fusion weight of the i-th subtask.

[0027] In one embodiment, the algorithm platform performs calculations and in-depth analysis on the data sources in the database to obtain a situational information profile, and stores the situational information profile in a data profile information database to retrieve the situational information profile and map it to the information visualization layer in real time. The method also includes:

[0028] Acquire the comprehensive situational data from multiple sources and divide the comprehensive situational data into offline data and real-time data;

[0029] The offline processing layer and near-line analysis layer in the algorithm platform are invoked to process and analyze the offline and real-time data, and the online computing layer is invoked to perform calculations and in-depth analysis on the processed and analyzed offline and real-time data to generate the situation information profile.

[0030] The algorithm platform includes an offline processing layer, a near-line analysis layer, and an online computing layer.

[0031] In one embodiment, the algorithm platform further includes the information visualization layer;

[0032] The algorithm platform performs calculations and in-depth analysis on the data sources in the database to obtain a situational information profile, and stores the situational information profile in a data profile information database. This allows for real-time retrieval of the situational information profile and mapping it to the information visualization layer. The method also includes:

[0033] The situation information profile generated by the online computing layer is retrieved in real time from the data profile information database and sent to the information visualization layer of the algorithm platform to map the situation information profile to the information visualization layer.

[0034] The present invention also provides a situational information visual analysis device, the device comprising:

[0035] The data preprocessing module is used to obtain the data source by listening to the data in the message consumption queue through the computing engine, and to clean and preprocess the data source so as to store the cleaned and preprocessed data source in the database.

[0036] The information visualization analysis module is used to call the algorithm platform to perform calculations and in-depth analysis on the data sources in the database, obtain situation information profiles, and store the situation information profiles in the data profile information database, so as to retrieve the situation information profiles in real time and map them to the information visualization layer.

[0037] The data source includes multi-source, multi-type comprehensive situational data, which includes structured and unstructured data. The algorithm platform is built based on a variety of basic algorithms and machine learning frameworks, and the basic algorithms include at least text mining algorithms, classification algorithms, regression algorithms, clustering algorithms, and recommendation algorithms.

[0038] The present invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the situation information visual analysis method as described above.

[0039] The present invention also provides a computer storage medium storing a computer program, which, when executed by a processor, implements the situation information visualization analysis method as described above.

[0040] The aforementioned situational information visualization analysis methods, devices, electronic equipment, and storage media, by establishing integrated management of various types of situational data resources and using big data processing and visualization analysis technologies for situational awareness, can comprehensively analyze different types of multi-source situational information. This fully leverages the advantages of intelligent detection and deep big data analysis technologies to form comprehensive situational information with complete elements and rich content. The analysis results of multiple types of situational data are organically combined to ultimately form a comprehensive situational information visualization view, enabling a comprehensive grasp and precise control of the spatial environment situation. This improves the comprehensiveness, accuracy, and intuitiveness of the analysis of various types of situational data from multiple sources. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0042] Figure 1 This is one of the flowcharts of the situation information visual analysis method provided by the present invention;

[0043] Figure 2 The second schematic diagram of the situation information visual analysis method provided by the present invention;

[0044] Figure 3 The third schematic diagram of the situation information visual analysis method provided by the present invention;

[0045] Figure 4 The fourth schematic diagram of the situation information visual analysis method provided by the present invention;

[0046] Figure 5 Fifth schematic diagram of the situation information visual analysis method provided by the present invention;

[0047] Figure 6 A schematic diagram of the overall architecture of the comprehensive situation information visual analysis system for a specific embodiment of the situation information visual analysis method provided by the present invention;

[0048] Figure 7 A schematic diagram of the overall situation information visual analysis system framework for the situation information visual analysis method in a specific embodiment of the present invention;

[0049] Figure 8 A schematic diagram of the comprehensive situational data preprocessing flow of the situational information visual analysis method in a specific embodiment of the present invention;

[0050] Figure 9 This is a schematic diagram illustrating the process of constructing a profile based on a comprehensive situational dataset in a specific embodiment of the situational information visualization analysis method provided by the present invention.

[0051] Figure 10 This is a schematic diagram of the multi-task data processing and analysis model structure of the situation information visualization analysis method in a specific embodiment of the present invention;

[0052] Figure 11 A schematic diagram illustrating the situation data processing and analysis visualization information flow of the situation information visualization analysis method in a specific embodiment of the present invention;

[0053] Figure 12 A schematic diagram of the situation information visual analysis device provided by the present invention;

[0054] Figure 13 An internal structural diagram of the computer device provided by the present invention. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0056] The following is combined Figures 1-13 The present invention describes a situation information visualization analysis method, apparatus, electronic device, and storage medium.

[0057] like Figure 1 As shown, in one embodiment, a situational information visual analysis method includes the following steps:

[0058] In step S110, the computing engine obtains the data source by listening to the data in the message consumption queue, and performs cleaning and preprocessing on the data source so as to store the cleaned and preprocessed data source in the database.

[0059] Specifically, the server obtains data sources by listening to the data in the message consumption queue through the computing engine, and cleans and preprocesses the data sources to store the cleaned and preprocessed data sources in the database. The data sources include multi-type comprehensive situational data from various sources, including structured data and unstructured data.

[0060] Step S120: Call the algorithm platform to perform calculations and in-depth analysis on the data source in the database to obtain a situation information profile, and store the situation information profile in the data profile information database so as to retrieve the situation information profile and map it to the information visualization layer in real time.

[0061] The algorithm platform is built upon a variety of basic algorithms and machine learning frameworks. The basic algorithms include, but are not limited to, text mining algorithms, classification algorithms, regression algorithms, clustering algorithms, and recommendation algorithms.

[0062] Specifically, the server calls an algorithm platform built on a variety of basic algorithms and machine learning frameworks to perform calculations and in-depth analysis on the data sources in the database, obtains a situation information profile, and stores the situation information profile in the data profile information database, so as to retrieve the situation information profile in real time and map it to the information visualization layer.

[0063] The aforementioned situational information visualization analysis method, by establishing integrated management of various types of situational data resources and using big data processing and visualization analysis technologies for situational awareness, can comprehensively analyze different types of multi-source situational information. It fully leverages the advantages of intelligent detection and deep big data analysis technologies to form comprehensive situational information with complete elements and rich content. By organically combining the analysis results of multiple types of situational data, a comprehensive situational information visualization view is ultimately formed. This enables a comprehensive grasp and precise control of the spatial environment situation, improving the comprehensiveness, accuracy, and intuitiveness of the analysis of various types of situational data from multiple sources.

[0064] like Figure 2 As shown, in one embodiment, the situational information visual analysis method provided by the present invention obtains the data source by listening to the data in the message consumption queue through a computing engine, and performs cleaning and preprocessing on the data source to store the cleaned and preprocessed data source in a database. Specifically, it includes the following steps:

[0065] Step S112: Extract unstructured data from the data source and perform data quality filtering on the unstructured data using a defined filtering model or filtering rules to filter out low-quality data in the unstructured data that does not meet the set requirements.

[0066] Specifically, the server extracts unstructured data from the data source and performs data quality filtering on the unstructured data using a defined filtering model or filtering rules to filter out low-quality data in the unstructured data that does not meet the set requirements.

[0067] Step S114: Use Bloom filter technology to deduplicate the filtered unstructured data, obtain preprocessed unstructured data, and store the structured data and preprocessed unstructured data in the database.

[0068] Specifically, the server uses Bloom filter technology to deduplicate the filtered unstructured data, obtaining pre-processed unstructured data, and then stores the structured data and the pre-processed unstructured data in the database.

[0069] Among them, the Bloom filter is a very space-efficient probabilistic data structure that runs fast and occupies little memory. It is actually composed of a long binary vector and a series of random mapping functions, and is mainly used to determine whether an element is in a set.

[0070] like Figure 3 As shown, in one embodiment, the situation information visualization analysis method provided by the present invention calls an algorithm platform to perform calculations and in-depth analysis on the data source in the database to obtain a situation information profile, and stores the situation information profile in a data profile information database, so as to retrieve the situation information profile and map it to the information visualization layer in real time. The preceding steps include:

[0071] Step S310: Build an algorithm platform based on text mining algorithms, classification algorithms, regression algorithms, clustering algorithms, and recommendation algorithms, combined with the machine learning framework Flink.

[0072] Specifically, the server is an algorithm platform built based on text mining algorithms, classification algorithms, regression algorithms, clustering algorithms, and recommendation algorithms, combined with the machine learning framework Flink.

[0073] Flink is a unified computing framework that combines batch and stream processing. Its core is a stream data processing engine that provides data distribution and parallel computing.

[0074] Step S320: Abstract the calculation and in-depth analysis algorithms for multi-type integrated situational data from multiple sources through the algorithm platform, so as to optimize the algorithm platform and obtain the calculation framework for each type of integrated situational data.

[0075] Specifically, the server uses the algorithm platform built in step S310 to abstract calculation and deep analysis algorithms for multi-type integrated situational data from multiple sources, so as to optimize the algorithm platform and obtain the calculation framework for each type of integrated situational data.

[0076] like Figure 4As shown, in one embodiment, the situation information visualization analysis method provided by the present invention calls an algorithm platform to perform calculations and in-depth analysis on the data source in the database to obtain a situation information profile, and stores the situation information profile in a data profile information database to retrieve the situation information profile and map it to the information visualization layer in real time. Specifically, it includes the following steps:

[0077] Step S121: Call the algorithm platform to establish an analysis and processing model corresponding to each type of comprehensive situation data based on different types of comprehensive situation data.

[0078] Specifically, the server calls the algorithm platform to establish an analysis and processing model corresponding to each type of comprehensive situational data based on different types of comprehensive situational data.

[0079] Step S123: By analyzing and processing the model, multiple processing and analysis tasks corresponding to the same type of comprehensive situational data are fitted to obtain the processing and analysis results.

[0080] Specifically, the server uses an analysis and processing model to fit multiple processing and analysis tasks corresponding to the same type of comprehensive situational data, and obtains the processing and analysis results.

[0081] It should be noted that during the multi-task analysis process, the server determines the fusion weight of each subtask in the analysis task based on the data dispersion and data scale of the corresponding type of comprehensive situational data, and calls the fusion formula to fuse the analysis results of multiple subtasks according to the fusion weight of each subtask.

[0082] The formula for calculating the fusion weight is as follows:

[0083] ;

[0084] In the formula, The degree of data dispersion of the comprehensive situation data. The data size of the comprehensive situational data is... For weight fusion.

[0085] The fusion formula is:

[0086] ;

[0087] In the formula, result is the total score after fusion, n is the number of subtasks, and result i Output the analysis results for the i-th subtask. Let be the fusion weight of the i-th subtask.

[0088] In addition, data is shared among integrated situation data of the same type, and each processing and analysis task corresponding to the same type of integrated situation data has shared parameters. The processing and analysis results are obtained by fusing the analysis results output of multiple sub-tasks of each processing and analysis task.

[0089] like Figure 5 As shown, in one embodiment, the situation information visualization analysis method provided by the present invention calls an algorithm platform to perform calculations and in-depth analysis on the data source in the database to obtain a situation information profile, and stores the situation information profile in a data profile information database to retrieve the situation information profile and map it to the information visualization layer in real time. Specifically, it also includes the following steps:

[0090] Step S122: Obtain comprehensive situational data from multiple sources and divide the comprehensive situational data into offline data and real-time data.

[0091] Specifically, the server acquires comprehensive situational data from multiple sources and divides the comprehensive situational data into offline data and real-time data.

[0092] Step S124: The offline processing layer and near-line analysis layer in the algorithm platform are called to process and analyze the offline and real-time data, and the online computing layer is called to perform calculations and in-depth analysis on the processed offline and real-time data to generate a situation information profile.

[0093] Specifically, the server calls the offline processing layer and near-line analysis layer in the algorithm platform to process and analyze offline and real-time data, and calls the online computing layer to perform calculations and in-depth analysis on the processed offline and real-time data to generate a situational information profile.

[0094] Step S126: Retrieve the situation information image generated by the online computing layer in real time from the data profile information database, and send it to the information visualization layer of the algorithm platform to map the situation information image to the information visualization layer.

[0095] Specifically, the server retrieves the situation information profile generated by the online computing layer in real time from the previously obtained data profile information database and sends it to the information visualization layer of the algorithm platform to map the situation information profile to the information visualization layer.

[0096] See Figure 6 As shown in the specific embodiment, the present invention provides a situational information visual analysis method. This method is mainly implemented by a comprehensive situational information visual analysis system. The comprehensive situational information visual analysis system is a highly integrated system with a four-layer architecture, including a support environment, a data processing and analysis layer, a service layer, and a comprehensive situational profile visualization layer. Through multi-layer design, the processing flow of specific functions is completed in each layer.

[0097] The support environment provides basic hardware and software resources. The data processing and analysis layer includes two main modules: situational information big data assessment and analysis, and situational information profile generation. The service layer provides services such as manpower situational awareness, material situational awareness, equipment and facility situational awareness, network situational awareness, data situational awareness, and mission situational awareness. The comprehensive situational awareness visualization layer provides functions such as manpower situational awareness, material situational awareness, equipment and facility situational awareness, network situational awareness, data situational awareness, and mission situational awareness.

[0098] In this embodiment, the supporting environment provides servers, databases for structured, semi-structured, and unstructured data, and hardware and software resources for accessing the system, enabling access to massive amounts of data. The data processing and analysis layer is used for comprehensive situational awareness monitoring, analysis, and assessment / prediction of human resources, materials, equipment, networks, data, and tasks. The core of this layer revolves around these data contents, collecting comprehensive situational data from various data sources in real time. Based on the big data system, it provides the ability to accept various types of data and stores the data in the big data processing platform, laying the foundation for analyzing and processing massive, multimodal data. The situational information big data assessment and analysis module provides the ability to analyze, assess, and predict various aspects of situational information. The entire data lifecycle includes data acquisition, data processing, data transmission, data storage, and usage. Data acquisition and processing are the foundation for data analysis and assessment; only based on objective, accurate, and comprehensive data can effective and precise situational information results be obtained. To ensure the usability and high quality of the collected data and meet the analysis objectives, situational data can be divided into two types: static data and dynamic data. Static information data mainly comes from user input and system-level acquisition. Because the collected static information is uncertain, it will be modeled, judged, and improved. Dynamic information data has the characteristic of being hidden, referring to constantly changing data with big data characteristics and real-time features, which needs to be accurately extracted through data analysis and data mining.

[0099] After collecting static and dynamic situational data, and under the premise of fully protecting data privacy and ensuring the authenticity and validity of the data, redundant data and abnormal information that are irrelevant to the characteristics of various situational data are first filtered out. Then, the cleaned situational data is processed to make it usable for modeling various situational information such as networks, human resources, and data.

[0100] The situational information profiling generation module is responsible for generating situational profiles for various situational data based on the results data generated by the evaluation and analysis module. These profiles can directly and indirectly reflect data capabilities. By combining static and dynamic information such as networks, manpower, materials, equipment and facilities, and data, relatively three-dimensional and accurate network profile models, manpower profile models, material profile models, equipment and facilities profile models, and data profile models are constructed respectively. Referring to existing factual data, algorithms are used to evaluate situational information to derive predictive information and explore potential capabilities.

[0101] In this embodiment, the service layer, based on a well-established situational information profiling model, designs various service engines in conjunction with specific business needs, such as an equipment and facility situational query engine and a network situational query engine. These engines are responsible for acquiring the profiling data after real-time calculation and analysis to support various applications of the comprehensive situational profiling visualization layer. The comprehensive situational profiling visualization layer addresses the complex data exploration and mining needs of the comprehensive situation. By developing a comprehensive situational information visualization system, it helps users implement relatively complete data analysis in the field of comprehensive situational awareness. The comprehensive situational profiling visualization layer, combined with an open-source big data visualization engine, visualizes the analyzed and processed knowledge-based comprehensive situational information. Using relatively simple visual graphics, it conveys a very high density of situational information, helping users better understand patterns, regularities, or anomalies in the data and make judgments and decisions.

[0102] The comprehensive situational awareness visualization layer utilizes a customized WebGL-based geographic information platform to implement a geospatial big data visualization and analysis framework. By connecting to the platform engine and framework, it achieves map-related capabilities and enables the overlay of various visual symbols onto the map to handle different analytical and presentation tasks. First, the engine accesses the base map for representing geographic information. Then, using map overlay with a situational location dataset, it accurately plots points of interest on a precise map, emphasizing the most critical geographical location information of these points, thus realizing a visualization analysis function that combines points of interest with geographic information.

[0103] In addition, the comprehensive situational profiling visualization layer, based on the data profiling and evaluation results of various situational information such as manpower, materials, equipment and facilities, networks, data, and tasks, obtains situational information data through various services connected to the service layer, performs intuitive data visualization as needed, integrates and empowers comprehensive situational information visualization, and ultimately applies it to reality, enabling users to quickly and accurately understand the true form of data, discover the value in the data, and assist in data analysis and data mining.

[0104] It should be noted that comprehensive situational data analysis involves multiple dimensions and is quite complex. Understanding comprehensive situational data through appropriate visualization methods is a crucial objective of comprehensive situational information visualization analysis systems. Common data visualization categories such as comparison, trend, proportion, distribution, correlation, hierarchy, and association are insufficient to fully represent the intended purpose of multi-source and multi-type situational data. Therefore, it is necessary to adhere to Web standards and use SVG, Canvas, and HTML to bring the data to life. By combining powerful visualization and interactive technologies with data-driven DOM manipulation methods, composite charts such as stacked bar charts and line charts, and tables and time axes, can be designed to allow for the free design of suitable visualization interfaces for multi-source and multi-type situational data. Ultimately, a simple view outlines the situation of manpower, networks, equipment and facilities, materials, and tasks. The intuitive data presentation assists users in analyzing diverse data, fully leveraging the advantages of visualization in aiding perception.

[0105] Combination Figure 7 As shown, in this embodiment, the integrated situational information visual analysis system aims to extract principles and discover patterns from existing situational data, serving as a basis for behavior and judgment. The integrated situational information visual analysis system framework integrates data acquisition, data storage and management, data analysis and calculation, data visualization, and data security and privacy functions. It possesses precise data statistical analysis and data mining capabilities, and its conclusions are typically decision-making in nature.

[0106] First, the real-time computing engine listens to data in the message consumption queue, collecting objective and rational data, and then writes the collected data sources into the database for storage in a unified data format. After collecting sufficient information, it runs components such as natural language processing, classification, and clustering to perform real-time calculations and in-depth analysis on the data in the database, obtaining more real-time and accurate situational information profiles. This enables real-time calculation of massive amounts of data, and the results are often factual information. After a series of deep reasoning and integration, the situational data resources are refined into knowledge and stored in the data profile information database for real-time retrieval and use, supporting various precision services. Finally, the data is mapped to the visualization layer, using graphs to represent the information contained in the data.

[0107] In comprehensive situational information visualization and analysis systems, massive datasets on comprehensive situational awareness are introduced. While a large amount of data can improve the accuracy of data mining and analysis, the widely sourced unstructured text often contains a significant amount of low-quality data. Data quality is the foundation of big data visualization and analysis; ensuring high data quality is crucial for obtaining accurate and reliable analytical results. Before analysis, the collected data must undergo meticulous preprocessing and cleaning. To improve data quality, optimizations were made in data sources, cleaning rules, storage, and backup, and appropriate data verification and validation methods were adopted.

[0108] Specifically, comprehensive situational data from various sources is collected and stored on a database or distributed file system. Unstructured data undergoes preprocessing such as quality filtering and content deduplication to cleanse the dataset, transforming it from unstructured to structured data. This standardizes the information and stores it in HBase and relational databases, making the data more readable and allowing for further offline or online value extraction. Data quality filtering is crucial for data preprocessing and can be performed according to defined rules or models. Often, data content is repetitive; Bloom filter technology is used to quickly deduplicate the data, forming a valid dataset and accelerating data processing efficiency.

[0109] Combination Figure 8 As shown in this embodiment, due to the rich variety of situational data types and their diverse characteristics, the processing logic varies. An algorithm platform is constructed based on fundamental algorithms such as text mining, classification, regression, clustering, and recommendation algorithms, as well as machine learning frameworks like Flink. For different data scenarios, the algorithm platform further abstracts more detailed algorithmic methods to handle specific business data processing, enabling refined mining of specific data types and continuous algorithm optimization until effective information is extracted. The data profiles and knowledge generated by each computing framework are exchanged through NoSQL intermediaries and quickly provided externally via an online service engine, demonstrating data value and ultimately achieving the goal of data mining.

[0110] Combination Figure 9 As shown, this embodiment combines big data technology with data profiling, improving the accuracy of the profiling while significantly increasing the scale of processable data, providing more refined and dimensionally rich visualization information. Corresponding data analysis and processing models are established based on different types of datasets. Data of the same type is shared, allowing a single model to simultaneously fit multiple processing and analysis tasks. Tasks share some parameters, reducing the overall size of the model, lowering memory consumption, saving resources, and improving the model's generalization and expressive capabilities.

[0111] Combination Figure 10 As shown in this embodiment, the data processing and analysis is in the form of multiple tasks. Each sub-task outputs analysis results, which need to be merged to form the final processing and analysis result. For analysis tasks related to scoring and recommendation, such as data contribution, ranking, and sorting, the score results need to be merged. First, the fusion weight of each sub-task must be determined, and then the result data is merged using a fusion formula. Taking into account the characteristics or attributes of the data itself, the dispersion of the data, and the data scale, the fusion weight of each sub-task is automatically calculated. The formula for calculating the fusion weight is:

[0112] ;

[0113] In the formula, The degree of data dispersion of the comprehensive situation data. The data size of the comprehensive situational data is... This is the fusion weight. Similarly, we can understand the relative importance of each data metric, then determine the fusion weight of each subtask, and subsequently use the fusion formula method to calculate the fusion result of the task output values. The fusion formula is: ;

[0114] In the formula, result is the total score after fusion, n is the number of subtasks, and result i Output the analysis results for the i-th subtask. Let be the fusion weight of the i-th subtask.

[0115] In this embodiment, real-time tasks require rapid real-time data processing. Therefore, to ensure the rapid generation of results from real-time data processing, time-consuming calculations cannot be performed within the real-time task; otherwise, any delay in data processing will cause channel congestion. The results from real-time and offline tasks may differ for the same data. Offline tasks yield more accurate results, while real-time tasks provide approximately correct results, and data from the most recent period is typically the most useful. Therefore, we employ a collaborative approach between offline and real-time tasks, utilizing a computing framework that supports offline batch data processing and online real-time streaming data processing to perform data reading and other processing tasks.

[0116] Throughout the comprehensive situational awareness analysis process, the visualization of information based on raw data is subdivided into processing layers according to task type. Data undergoes complex processing and analysis before final visualization, optimizing task processing efficiency and improving the real-time performance and accuracy of data processing. Multi-source comprehensive situational data is processed offline, analyzed in real-time, and processed by online computing services to generate comprehensive situational information suitable for visualization. Situational data passes through four layers: Offline processing, Nearline analysis, Online computing, and Visual information visualization. The offline processing layer handles massive amounts of data offline; the nearline analysis layer primarily provides real-time services, utilizing streaming technologies such as Flink and Spark Streaming to process real-time data; the online computing layer handles relatively simple computational logic, ensuring low latency, such as online service engines; and the information visualization layer renders the results of situational data analysis, providing an intuitive and clear comprehensive presentation of the information mined from the situational data. Through processing at each layer, the raw data is transformed into visualized information.

[0117] Combination Figure 11 As shown in this embodiment, the integrated situational information visual analysis system starts with raw data, deciphers the hidden data characteristics behind the situational data, and obtains useful valuable information after processing, realizing a complete analysis and drawing process from data to visualization step by step. It brings comprehensive situational awareness and visualization, allowing users to fully understand the overall picture of integrated situational data, predict threats, and improve their ability to discover, identify, understand, analyze, and respond to security threats from a global perspective.

[0118] The aforementioned situational information visualization analysis method, based on situational data and a comprehensive display framework, establishes integrated management of various situational data resources, including manpower, materials, equipment, networks, data, and tasks. By utilizing big data and visualization analysis technologies to develop situational awareness functions, it comprehensively analyzes, merges, and processes diverse multi-source situational information. Leveraging the advantages of intelligent detection and deep big data mining technologies, it generates comprehensive and rich situational information, organically combining manpower, materials, equipment, networks, data, and tasks to form a comprehensive situational information visualization view. This enables a comprehensive grasp and precise control of the space environment situation. This facilitates full management and comprehensive perception of the allocation status of resources in the space environment domain, establishes precise control over related activities in the space environment domain, and ensures efficient resource allocation, thereby guaranteeing the effectiveness of the execution of various related tasks.

[0119] The situation information visualization analysis device provided by the present invention is described below. The situation information visualization analysis device described below and the situation information visualization analysis method described above can be referred to in correspondence.

[0120] like Figure 12 As shown, in one embodiment, a situational information visual analysis device includes a data preprocessing module 1210 and an information visual analysis module 1220.

[0121] The data preprocessing module 1210 is used to obtain the data source by listening to the data in the message consumption queue through the computing engine, and to clean and preprocess the data source so as to store the cleaned and preprocessed data source in the database.

[0122] The information visualization analysis module 1220 is used to call the algorithm platform to perform calculations and in-depth analysis on the data source in the database, obtain the situation information profile, and store the situation information profile in the data profile information database, so as to retrieve the situation information profile and map it to the information visualization layer in real time.

[0123] The data sources include multi-type comprehensive situational data from various sources, including structured and unstructured data. The algorithm platform is built based on a variety of basic algorithms and machine learning frameworks. The basic algorithms include at least text mining algorithms, classification algorithms, regression algorithms, clustering algorithms, and recommendation algorithms.

[0124] In this embodiment, the situational information visual analysis device provided by the present invention includes a data preprocessing module specifically used for:

[0125] Extract unstructured data from the data source and filter the unstructured data using a defined filtering model or filtering rules to filter out low-quality data that does not meet the set requirements.

[0126] The Bloom filter technique is used to deduplicate the filtered unstructured data, resulting in preprocessed unstructured data. The structured data and the preprocessed unstructured data are then stored in the database.

[0127] In this embodiment, the situational information visual analysis device provided by the present invention further includes an algorithm platform construction module, used for:

[0128] An algorithm platform is built based on text mining, classification, regression, clustering, and recommendation algorithms, combined with the machine learning framework Flink.

[0129] By abstracting computation and in-depth analysis algorithms for multi-type integrated situational data from various sources through an algorithm platform, the algorithm platform is optimized to obtain a computation framework for each type of integrated situational data.

[0130] In this embodiment, the situational information visual analysis device provided by the present invention has an information visual analysis module specifically used for:

[0131] The algorithm platform is invoked to establish analysis and processing models corresponding to each type of comprehensive situation data based on different types of comprehensive situation data.

[0132] By fitting the analysis and processing model to multiple processing and analysis tasks corresponding to the same type of comprehensive situational data, the processing and analysis results are obtained.

[0133] Among them, comprehensive situation data of the same type share data, and each processing and analysis task corresponding to the same type of comprehensive situation data has shared parameters. The processing and analysis results are obtained by fusing the analysis results output of multiple sub-tasks of each processing and analysis task.

[0134] In this embodiment, the situational information visual analysis device provided by the present invention further includes an information visual analysis module specifically used for:

[0135] The fusion weight of each subtask in the processing and analysis task is determined based on the data dispersion and data scale of the corresponding type of comprehensive situational data. The fusion formula is then called to fuse the analysis results of multiple subtasks according to the fusion weight of each subtask.

[0136] The formula for calculating the fusion weight is as follows:

[0137] ;

[0138] In the formula, The degree of data dispersion of the comprehensive situation data. The data size of the comprehensive situational data is... For weight fusion.

[0139] The fusion formula is:

[0140] ;

[0141] In the formula, result is the total score after fusion, n is the number of subtasks, and result i Output the analysis results for the i-th subtask. Let be the fusion weight of the i-th subtask.

[0142] In this embodiment, the situational information visual analysis device provided by the present invention further includes an information visual analysis module specifically used for:

[0143] Acquire comprehensive situational data from multiple sources and divide the comprehensive situational data into offline data and real-time data.

[0144] The algorithm platform calls the offline processing layer and near-line analysis layer to process and analyze offline and real-time data, and calls the online computing layer to perform calculations and in-depth analysis on the processed offline and real-time data to generate a situational information profile.

[0145] The algorithm platform includes an offline processing layer, a near-line analysis layer, and an online computing layer.

[0146] In this embodiment, the situational information visual analysis device provided by the present invention further includes an information visual analysis module specifically used for:

[0147] The situation information profile generated by the online computing layer is retrieved in real time from the data profile information database and sent to the information visualization layer of the algorithm platform to map the situation information profile to the information visualization layer.

[0148] Figure 13 This example illustrates a schematic diagram of the physical structure of an electronic device, which can be a smart terminal. Its internal structure diagram can be as follows: Figure 13 As shown. The electronic device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a situational information visual analysis method, which includes:

[0149] The computing engine obtains the data source by listening to the data in the message consumption queue, and performs cleaning and preprocessing on the data source in order to store the cleaned and preprocessed data source in the database.

[0150] The algorithm platform is invoked to perform calculations and in-depth analysis on the data sources in the database to obtain a situation information profile, which is then stored in the data profile information database so that the situation information profile can be retrieved in real time and mapped to the information visualization layer.

[0151] The data sources include multi-type comprehensive situational data from various sources, including structured and unstructured data. The algorithm platform is built based on a variety of basic algorithms and machine learning frameworks. The basic algorithms include at least text mining algorithms, classification algorithms, regression algorithms, clustering algorithms, and recommendation algorithms.

[0152] Those skilled in the art will understand that Figure 13The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the electronic device to which the present invention is applied. A specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0153] On the other hand, the present invention also provides a computer storage medium storing a computer program, which, when executed by a processor, implements a situational information visual analysis method, the method comprising:

[0154] The computing engine obtains the data source by listening to the data in the message consumption queue, and performs cleaning and preprocessing on the data source in order to store the cleaned and preprocessed data source in the database.

[0155] The algorithm platform is invoked to perform calculations and in-depth analysis on the data sources in the database to obtain a situation information profile, which is then stored in the data profile information database so that the situation information profile can be retrieved in real time and mapped to the information visualization layer.

[0156] The data sources include multi-type comprehensive situational data from various sources, including structured and unstructured data. The algorithm platform is built based on a variety of basic algorithms and machine learning frameworks. The basic algorithms include at least text mining algorithms, classification algorithms, regression algorithms, clustering algorithms, and recommendation algorithms.

[0157] In another aspect, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and when the processor executes the computer instructions, it implements a situational information visual analysis method, which includes:

[0158] The computing engine obtains the data source by listening to the data in the message consumption queue, and performs cleaning and preprocessing on the data source in order to store the cleaned and preprocessed data source in the database.

[0159] The algorithm platform is invoked to perform calculations and in-depth analysis on the data sources in the database to obtain a situation information profile, which is then stored in the data profile information database so that the situation information profile can be retrieved in real time and mapped to the information visualization layer.

[0160] The data sources include multi-type comprehensive situational data from various sources, including structured and unstructured data. The algorithm platform is built based on a variety of basic algorithms and machine learning frameworks. The basic algorithms include at least text mining algorithms, classification algorithms, regression algorithms, clustering algorithms, and recommendation algorithms.

[0161] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory.

[0162] By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0163] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0164] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A method for visual analysis of situational information, characterized in that, The method includes: The computing engine obtains the data source by listening to the data in the message consumption queue, and performs cleaning and preprocessing on the data source so as to store the cleaned and preprocessed data source in the database. The algorithm platform is invoked to perform calculations and in-depth analysis on the data sources in the database to obtain a situation information profile, and the situation information profile is stored in the data profile information database so that the situation information profile can be retrieved in real time and mapped to the information visualization layer. The data source includes multi-source, multi-type comprehensive situational data, which includes structured and unstructured data. The algorithm platform is built based on multiple basic algorithms and machine learning frameworks. The basic algorithms include at least text mining algorithms, classification algorithms, regression algorithms, clustering algorithms, and recommendation algorithms. The algorithm platform performs calculations and in-depth analysis on the data sources in the database to obtain a situational information profile, and stores the situational information profile in a data profile information database. This allows for real-time retrieval of the situational information profile and mapping it to the information visualization layer, including: The algorithm platform is invoked to establish an analysis and processing model corresponding to each type of comprehensive situation data based on different types of comprehensive situation data. The analysis and processing model is used to fit multiple processing and analysis tasks corresponding to the same type of comprehensive situational data to obtain processing and analysis results. Among them, the integrated situation data of the same type share data, and each processing and analysis task corresponding to the integrated situation data of the same type has shared parameters. The processing and analysis result is obtained by fusing the analysis results output of multiple sub-tasks of each processing and analysis task.

2. The situation information visual analysis method according to claim 1, characterized in that, The process of obtaining a data source by listening to data in the message consumption queue through a computing engine, cleaning and preprocessing the data source, and storing the cleaned and preprocessed data source in the database includes: The unstructured data is extracted from the data source, and the unstructured data is filtered for data quality using a defined filtering model or filtering rules to remove low-quality data in the unstructured data that does not meet the set requirements. The unstructured data after filtering is deduplicated using Bloom filter technology to obtain preprocessed unstructured data, and the structured data and the preprocessed unstructured data are stored in the database.

3. The situation information visual analysis method according to claim 1, characterized in that, The algorithm platform performs calculations and in-depth analysis on the data sources in the database to obtain a situational information profile, and stores the situational information profile in a data profile information database. This allows for real-time retrieval of the situational information profile and mapping it to the information visualization layer. Prior to this, the following steps are included: The algorithm platform is built based on the aforementioned text mining algorithm, classification algorithm, regression algorithm, clustering algorithm, and recommendation algorithm, combined with the machine learning framework Flink. The algorithm platform is used to abstract calculation and in-depth analysis algorithms for multi-type integrated situational data from various sources, so as to optimize the algorithm platform and obtain a calculation framework for each type of integrated situational data.

4. The situation information visual analysis method according to claim 1, characterized in that, The algorithm platform performs calculations and in-depth analysis on the data sources in the database to obtain a situational information profile, and stores the situational information profile in a data profile information database. This allows for real-time retrieval of the situational information profile and mapping it to the information visualization layer. The method also includes: The fusion weight of each subtask in the processing and analysis task is determined based on the data dispersion and data scale of the comprehensive situation data of the corresponding type, and the fusion formula is called to fuse the analysis results of multiple subtasks according to the fusion weight of each subtask. The formula for calculating the fusion weight is as follows: ； In the formula, The degree of data dispersion of the comprehensive situation data. The data size of the comprehensive situational data is... For weighting; The fusion formula is as follows: ； In the formula, result is the total score after fusion, n is the number of subtasks, and result... i Output the analysis results for the i-th subtask. Let be the fusion weight of the i-th subtask.

5. The situational information visual analysis method according to claim 4, characterized in that, The algorithm platform performs calculations and in-depth analysis on the data sources in the database to obtain a situational information profile, and stores the situational information profile in a data profile information database. This allows for real-time retrieval of the situational information profile and mapping it to the information visualization layer. The method also includes: Acquire the comprehensive situational data from multiple sources and divide the comprehensive situational data into offline data and real-time data; The offline processing layer and near-line analysis layer in the algorithm platform are invoked to process and analyze the offline and real-time data, and the online computing layer is invoked to perform calculations and in-depth analysis on the processed and analyzed offline and real-time data to generate the situation information profile. The algorithm platform includes an offline processing layer, a near-line analysis layer, and an online computing layer.

6. The situation information visual analysis method according to claim 5, characterized in that, The algorithm platform also includes the information visualization layer; The algorithm platform performs calculations and in-depth analysis on the data sources in the database to obtain a situational information profile, and stores the situational information profile in a data profile information database. This allows for real-time retrieval of the situational information profile and mapping it to the information visualization layer. The method also includes: The situation information profile generated by the online computing layer is retrieved in real time from the data profile information database and sent to the information visualization layer of the algorithm platform to map the situation information profile to the information visualization layer.

7. A situational information visual analysis device, characterized in that, The device includes: The data preprocessing module is used to obtain the data source by listening to the data in the message consumption queue through the computing engine, and to clean and preprocess the data source so as to store the cleaned and preprocessed data source in the database. The information visualization analysis module is used to call the algorithm platform to perform calculations and in-depth analysis on the data sources in the database, obtain situation information profiles, and store the situation information profiles in the data profile information database, so as to retrieve the situation information profiles in real time and map them to the information visualization layer. The data source includes multi-source, multi-type comprehensive situational data, which includes structured and unstructured data. The algorithm platform is built based on multiple basic algorithms and machine learning frameworks. The basic algorithms include at least text mining algorithms, classification algorithms, regression algorithms, clustering algorithms, and recommendation algorithms. The information visualization analysis module is specifically used for: The algorithm platform is invoked to establish an analysis and processing model corresponding to each type of comprehensive situation data based on different types of comprehensive situation data. The analysis and processing model is used to fit multiple processing and analysis tasks corresponding to the same type of comprehensive situational data to obtain processing and analysis results. Among them, the integrated situation data of the same type share data, and each processing and analysis task corresponding to the integrated situation data of the same type has shared parameters. The processing and analysis result is obtained by fusing the analysis results output of multiple sub-tasks of each processing and analysis task.

8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images