A multi-source meteorological data access and processing visualization method and system

Abstract

The application discloses a kind of multi-source meteorological data access and processing visualization method and system, the method includes: the core configuration parameter of meteorological data to be accessed is configured, and initialization verification and data reading are carried out, and heterogeneous multi-source meteorological raw data are constructed;Format standardization and splitting optimization processing are carried out to heterogeneous multi-source meteorological raw data, obtain several layer particles;A plurality of layer particles are sequentially processed multi-layer flexible assembly and standardized service release, and corresponding SDK special renderer is matched to render, obtain multi-source meteorological data visualization result.The application can realize high-definition, high-flow, low-threshold meteorological data visualization display.The application is a kind of multi-source meteorological data access and processing visualization method and system, and can be widely applied in data visualization technical field.

Description

Technical Field

[0001] This invention relates to the field of data visualization technology, and in particular to a visualization method and system for multi-source meteorological data access and processing. Background Technology

[0002] With the continuous enrichment of meteorological observation and model products, the increasing variety of meteorological data types, and the continuous improvement of data resolution, traditional data processing methods can no longer meet the business demands of high concurrency, high timeliness, and high visualization. Existing technologies generally suffer from problems such as single access format, fixed scheduling methods, weak high-resolution data processing capabilities, and low client rendering efficiency. Moreover, they mostly focus on raw data acquisition and basic processing, and have not formed a full-link optimization solution from access to visualization.

[0003] Related technologies include collecting, analyzing, and processing raw data from meteorological equipment, focusing on data acquisition and preprocessing, without involving flexible access to multi-source heterogeneous data, dynamic time scheduling, high-resolution data compression optimization, granular layer assembly, or professional client-side rendering.

[0004] However, existing meteorological data access and processing technologies suffer from three major shortcomings: Limited access and scheduling capabilities: Limited supported data formats, fixed access methods, and inflexible scheduling strategies make them difficult to adapt to multi-source and dynamically updated scenarios. Insufficient high-resolution data processing capabilities: Lack of dedicated optimization algorithms leads to high data transmission pressure, low loading and rendering efficiency, and an inability to meet the demands for refined display. Fragmented end-to-end system: A lack of standardized layer services and efficient rendering mechanisms results in high client integration difficulty and weak scalability and reusability. Summary of the Invention

[0005] To address the aforementioned technical problems, the purpose of this invention is to provide a visualization method and system for multi-source meteorological data access and processing, which can achieve high-definition, highly fluid, and low-threshold meteorological data visualization.

[0006] The first technical solution adopted in this invention is: a visualization method for multi-source meteorological data access and processing, comprising the following steps: Configure the core configuration parameters of the meteorological data to be accessed, perform initialization verification and data reading, and construct heterogeneous multi-source meteorological raw data; The heterogeneous multi-source meteorological raw data were standardized and split for optimization to obtain several layer particles. The system flexibly assembles and publishes standardized services for several layers of particles in sequence, and then matches them with the corresponding SDK-specific renderer for rendering to obtain multi-source meteorological data visualization results.

[0007] Furthermore, the step of configuring the core configuration parameters of the meteorological data to be accessed, performing initialization verification and data reading, and constructing heterogeneous multi-source meteorological raw data specifically includes: Perform data access metadata initialization. By querying the system configuration database table, obtain the core configuration parameters of all meteorological data to be accessed. The core configuration parameters include data identifier, format type, access method, access address, cron scheduling expression, spatial range and element code. The core configuration parameters of all meteorological data to be accessed are sequentially checked for cron expression syntax, access address connectivity, and format type compliance to obtain the checked configuration parameters. Based on the verified configuration parameters, the data access method for meteorological data is determined, data is read, and heterogeneous multi-source meteorological raw data is constructed.

[0008] Furthermore, the step of determining the data access method for meteorological data based on the verified configuration parameters, reading the data, and constructing heterogeneous multi-source meteorological raw data, wherein the heterogeneous multi-source meteorological raw data is either the first meteorological data or the second meteorological data, specifically includes: Based on the verified configuration parameters, the data access method for meteorological data is determined, including external API interface mode and local file mode. Based on the external API interface mode, a data request is initiated with authentication information, spatiotemporal range and element parameters. After the integrity verification is passed, the first meteorological data is output. Based on the local file mode, the system extracts timestamp, feature type, and resolution information by parsing the file name using regular expressions, and then sorts and locates the second meteorological data by timestamp to read it.

[0009] Furthermore, the step of standardizing and splitting the heterogeneous multi-source meteorological raw data to obtain several layer particles specifically includes: Heterogeneous multi-source meteorological raw data are subjected to unified parsing of multi-format meteorological data to obtain semantically aligned heterogeneous multi-source meteorological data. After semantic alignment, the heterogeneous multi-source meteorological data are sequentially subjected to specialized optimization processing and layer granularization splitting processing to obtain several layer granules.

[0010] Furthermore, the step of performing unified parsing of heterogeneous multi-source meteorological raw data in multiple formats to obtain semantically aligned heterogeneous multi-source meteorological data specifically includes: Automatic format identification is performed on heterogeneous multi-source meteorological raw data to determine the format classification results of heterogeneous multi-source meteorological raw data; Based on the format classification results of heterogeneous multi-source meteorological raw data, the corresponding parsing algorithm is called to extract the core data fields of the heterogeneous multi-source meteorological raw data, and the key fields of the heterogeneous multi-source meteorological raw data are obtained. By using a pre-defined mapping relationship, the key fields of the heterogeneous multi-source meteorological raw data are converted into a standard structure format and semantically aligned to obtain semantically aligned heterogeneous multi-source meteorological data.

[0011] Furthermore, the step of sequentially performing specialized optimization and layer granularization processing on the semantically aligned heterogeneous multi-source meteorological data to obtain several layer particles specifically includes: Redundant data removal is performed on the semantically aligned heterogeneous multi-source meteorological data to obtain the removed heterogeneous multi-source meteorological data. Based on the lossless compression algorithm, the heterogeneous multi-source meteorological data after removal is subjected to data compression optimization processing to obtain compressed heterogeneous multi-source meteorological data. The compressed heterogeneous multi-source meteorological data is optimized by time-series and spatial simplification. The time-series data in the heterogeneous multi-source meteorological data is reasonably thinned, and the spatial coordinate storage format of the spatial data in the heterogeneous multi-source meteorological data is optimized to obtain the optimized heterogeneous multi-source meteorological data. Based on the meteorological element types, spatial ranges, and temporal characteristics in the heterogeneous multi-source meteorological data, the optimized heterogeneous multi-source meteorological data is split into layers into granular layers, resulting in several layer granules.

[0012] Furthermore, the step of flexibly assembling and standardizing the service deployment of several layers of particles sequentially, and then matching them with the corresponding SDK-specific renderer for rendering to obtain the multi-source meteorological data visualization result, specifically includes: The system obtains the preset service requirements, performs multi-layer flexible assembly and standardized service publishing on several layer particles in sequence, and obtains a service dataset in a callable state. The service dataset that is in a callable state is parsed, the features of each layer are extracted and divided into vector data, raster data and high-precision raster data; Vector data, raster data, and high-precision raster data are rendered using the corresponding SDK-specific renderers to obtain multi-source meteorological data visualization results.

[0013] Furthermore, the step of obtaining preset service requirements, sequentially performing multi-layer flexible assembly and standardized service publishing on several layer granularities to obtain a service dataset in a callable state specifically includes: Obtain the preset service requirements, parse them, and obtain the requirements parsing results; Based on the results of the requirements analysis, several layer particles are filtered to obtain the filtered layer particles. According to the preset assembly rules, the filtered layer particles are combined, adapted, and validated in sequence, and the assembled service dataset is output. The assembled service dataset is encapsulated into standardized services, and the corresponding interface configuration is called for registration and publication, resulting in a service dataset that is in a callable state.

[0014] Furthermore, the step of matching vector data, raster data, and high-precision raster data with corresponding SDK-specific renderers for rendering specifically includes: For vector data, a grayscale compression transmission strategy is adopted. The server linearly maps the raster values ​​to preset grayscale values ​​for transmission, and the client calculates and restores the original values ​​and then matches the color marks to complete the color rendering. For raster data, based on grayscale compression transmission, an 8-neighbor pixel two-dimensional interpolation algorithm is used to perform spatial smoothing on the grid data to complete color rendering; For high-precision raster data, it is divided into blocks according to a fixed size, and the rendering efficiency is improved by multi-threaded parallel processing. Spatial interaction operations are realized by combining latitude and longitude with screen pixel coordinate mapping to complete color rendering.

[0015] The second technical solution adopted in this invention is: a visualization system for multi-source meteorological data access and processing, comprising: The first module is used to configure the core configuration parameters of the meteorological data to be accessed, and to perform initialization verification and data reading to construct heterogeneous multi-source meteorological raw data; The second module is used to perform format standardization and splitting optimization on heterogeneous multi-source meteorological raw data to obtain several layer particles. The third module is used to flexibly assemble and standardize the services of several layers of particles, and match them with the corresponding SDK dedicated renderer for rendering to obtain multi-source meteorological data visualization results.

[0016] The beneficial effects of the method and system of this invention are as follows: This invention constructs heterogeneous multi-source meteorological raw data by configuring the core configuration parameters of the meteorological data to be accessed, and performing initialization verification and data reading. It eliminates the need for hard-coding and modifying scheduling rules, and ensures stable and reliable data access through time matching judgment and integrity verification. This decouples the scheduling strategy from business logic, significantly improving the system's adaptability. Furthermore, it standardizes and optimizes the format of the heterogeneous multi-source meteorological raw data, resulting in several layer particles. An assembly and publishing method using these layer particles as the smallest unit breaks down the meteorological data into the smallest layer particles according to elements, time, type, and quality status, supporting multi-level free combination and transparent overlay rendering to form a standardized visualization service. Finally, it flexibly assembles and publishes the several layer particles in multiple layers, matching them with the corresponding SDK-specific renderer for rendering, resulting in multi-source meteorological data visualization results. This significantly reduces transmission volume and improves rendering efficiency while ensuring data accuracy, solving the problems of slow loading and rendering stuttering of large-size satellite, radar, and gridded data. Attached Figure Description

[0017] Figure 1 This is a flowchart of the steps of a visualization method for multi-source meteorological data access and processing according to the present invention; Figure 2 This is a structural block diagram of a visualization system for multi-source meteorological data access and processing according to the present invention; Figure 3 This is a schematic diagram of the overall process for multi-source meteorological data access and processing provided in a specific embodiment of the present invention. Detailed Implementation

[0018] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. The step numbers in the following embodiments are only for ease of explanation and do not limit the order of the steps. The execution order of each step in the embodiments can be adapted according to the understanding of those skilled in the art.

[0019] First, it should be noted that current meteorological data is characterized by multiple sources, heterogeneity, high resolution, and multiple time phases, covering formats such as nc, tiff, grib, grib2, radar, satellite, shp, gpk, and geojson, and is widely used in weather forecasting, climate analysis, disaster prevention and mitigation, and meteorological services. Traditional meteorological data access and processing technologies have significant shortcomings and cannot meet the high-timeliness, high-precision, and high-integration requirements of modern meteorological operations. For example: 1) Insufficient data source access capabilities: It only supports a single or limited number of formats and cannot be compatible with multiple types of heterogeneous meteorological data; it only supports local file access and does not support flexible modes such as network interfaces and message queues, making it difficult to meet the needs of multi-source data aggregation scenarios.

[0020] 2) The scheduling mechanism is rigid and inefficient: it relies on fixed parameters and hard-coded time configuration, and cannot customize the scheduling cycle. It has poor adaptability to scenarios such as batch backtracking, timed updates, and incremental retrieval.

[0021] 3) Weak high-resolution data processing: The processing logic for large-size satellite imagery, high-density grid points, and high-resolution radar data is simple, resulting in problems such as high loading latency, large transmission volume, and rendering lag, which cannot support refined meteorological display.

[0022] 4) Weak data service and rendering capabilities: lack of standardized layer particles and flexible assembly mechanism, poor universality of client rendering framework, low rendering efficiency, poor compatibility and rough display effect of massive point, line and surface and high precision grid data.

[0023] 5) Lack of end-to-end integrated capabilities: Data access, processing, publishing, and rendering are fragmented, lacking unified standards and closed-loop processes, making client access difficult, integration costs high, and maintenance complex.

[0024] Therefore, the purpose of the embodiments of the present invention is to overcome the defects of the prior art, such as... Figure 3 As shown, a complete system is constructed that supports unified access to multi-source heterogeneous data, dynamic cron scheduling, efficient optimization of high-resolution data, flexible granular assembly of layers, and high-performance visualization on the client side. This system achieves standardization, efficiency, and lightweighting of the entire process of meteorological data from access to display, improves data processing capabilities, reduces transmission costs, optimizes rendering effects, and simplifies business integration. It meets the core needs of modern meteorological services for multi-source data, high resolution, high efficiency, and high-quality display, significantly improving meteorological data processing efficiency, transmission efficiency, and visualization effects, reducing business access and integration costs, and providing stable, efficient, and scalable data support for various meteorological services.

[0025] Reference Figure 1 This invention provides a visualization method for multi-source meteorological data access and processing, which includes the following steps: S100: Configure the core configuration parameters of the meteorological data to be accessed, perform initialization verification and data reading, and construct heterogeneous multi-source meteorological raw data; S110. Perform data access metadata initialization. By querying the system configuration database table, obtain the core configuration parameters of all meteorological data to be accessed. The core configuration parameters include data identifier, format type, access method, access address, cron scheduling expression, spatial range and element code. In this embodiment, the core parameters include: 1) Data unique identifier ID, data type and format (radar / satellite / nc / tiff / shp / gpk / geojson, etc.).

[0026] 2) Data access method (external API interface / local file).

[0027] 3) Data access address (API interface address / local file storage path).

[0028] 4) Cron scheduling expression (defines the data retrieval period, supporting minutes / hours / days / months / custom periods).

[0029] 5) Data metadata such as data storage path, output format, spatial range, element code, and quality threshold.

[0030] S120. Perform cron expression syntax verification, access address connectivity verification, and format type compliance verification on the core configuration parameters of all meteorological data to be accessed in sequence to obtain the verified configuration parameters. In this embodiment, the system performs legality checks on the read configuration parameters, including: cron expression syntax verification, access address connectivity verification, and format type compliance verification, to ensure that all parameters comply with system specifications, complete the initialization of the data access task, and prepare for subsequent scheduling triggering.

[0031] The formula for calculating the syntax validation of cron expressions is: ; in: This represents the result of the cron expression syntax validation. It is a boolean parameter with values ​​of {true (compliant) and false (in violation)}. This represents a set of fields split from the original expression, obtained by separating the original expression with spaces. Each field corresponds to a time dimension in sequence: (Second), (point), (hour), (day), (moon), (week), (Year, optional). This parameter represents the field number compliance determination parameter. It is a boolean type. The expression evaluates to true, while fields other than 6 / 7 and fields whose delimiters are not spaces are evaluated as false. This parameter represents a single-field syntax compliance check parameter. It is a boolean type and must simultaneously satisfy the field's value range and the rules for using special characters. If either rule is not satisfied, the result is false.

[0032] The value ranges for each field are: seconds [0-59], minutes [0-59], hours [0-23], days [1-31], months [1-12 / JAN-DEC], weeks [1-7 / SUN-SAT], and years [1970-2099]. Values ​​outside these ranges will be considered violations.

[0033] Access address connectivity verification is calculated using the following formula: ; in, This represents the access address connectivity verification result, a boolean parameter with values ​​of {true (connected)} and {false (not connected)}. This represents the request-response timeout threshold. It is an integer parameter in milliseconds and is set according to the business scenario. The typical value is 1000ms-5000ms, which is a fixed threshold parameter. This represents the actual request-response time. It is a floating-point parameter obtained by initiating a probe request and is a dynamic parameter calculated in real time. This parameter represents the address reachability determination parameter. It is a boolean value and is determined as true if the domain name can be resolved normally, the IP address format is valid, and the route is reachable. This parameter indicates whether the target port is open. It is a boolean value and is set to true if the target port is in a listening state and a network connection can be established.

[0034] The format type compliance check is calculated using the following formula: ; in: This parameter represents the format type compliance check result. It is a boolean parameter with values ​​of {true (compliant) and false (non-compliant)}. This represents the target data to be verified, which can be of any type, including character, numeric, and structured data. A preset standard format template is provided. This represents a preset standard data type, an enumerated parameter, including {integer, floating-point, string, boolean, JSON, XML, date, etc.}. This parameter represents the data format matching parameter. It is a Boolean value. If the data encoding, writing format, and character rules completely match the standard template, it is judged as true.

[0035] S130. Based on the verified configuration parameters, determine the data access method for meteorological data, read the data, and construct heterogeneous multi-source meteorological raw data.

[0036] Specifically, based on the verified configuration parameters, the data access method for meteorological data is determined. The data access method includes external API interface mode and local file mode. Based on external API interface mode, a data request is initiated carrying authentication information, spatiotemporal range and element parameters. After the integrity verification is passed, the first meteorological data is output. Based on local file mode, the timestamp, element type and resolution information are extracted by parsing the file name using regular expressions. The second meteorological data is then sorted by timestamp, located and read.

[0037] In this embodiment, the system automatically matches the corresponding data acquisition logic according to the configured access method, realizes parallel support for both external API pull and local file parsing, covers the access requirements of multi-source meteorological data, and the two branches ultimately output raw data in a unified format.

[0038] For network-side meteorological data, the system retrieves data by calling external API interfaces via HTTP / HTTPS protocols. The core logic is as follows: 1) First, schedule and trigger verification: Based on the configured cron expression, determine whether the current system time meets the trigger condition. The trigger formula is as follows: ; in, The current system time (accurate to the second). This is the set of scheduled trigger times after parsing the cron expression (all trigger times within the next hour). When the time tolerance (default 10 seconds, used to compensate for system time errors) is met, the system will automatically trigger this data retrieval task.

[0039] 2) Further API requests and data retrieval are performed. The system sends a request to the configured API address, carrying the authentication token and time parameters ( / Request bodies such as spatial range (bbox) and element codes are used to retrieve meteorological data for the corresponding time period and region.

[0040] 3) Finally, perform data integrity verification: Perform an integrity check on the retrieved data. The check formula is as follows: ; in, Number of valid data entries / number of pixels Total number of data entries / number of pixels After the integrity threshold (default 0.95, i.e. 95% data integrity) is passed, the data enters the subsequent parsing stage; if the verification fails, a retry mechanism is triggered, with a maximum of 3 retries. If it still fails, an exception log is recorded.

[0041] For locally stored meteorological data files, the system automatically locates the latest data through filename parsing rules. The core logic is as follows: 1) First, perform filename rule matching: The system parses information such as timestamp, feature type, station code, and resolution in the file name according to the preset meteorological data file name format (such as feature code_timestamp_station ID_resolution format, e.g., PRE_202510101200_54511_0.1) using regular expressions.

[0042] 2) Then, locate the latest data: For all files with the same element, the same site, and the same resolution, sort them by timestamp, automatically filter out the latest version data with the largest timestamp, and complete the data reading and loading. The positioning formula is as follows: ; in, This is the collection of all files for this element / site. The timestamp parsed from file F. The latest data file located.

[0043] S200: The heterogeneous multi-source meteorological raw data is standardized and split and optimized to obtain several layer particles; S210. Perform unified parsing of heterogeneous multi-source meteorological raw data in multiple formats to obtain semantically aligned heterogeneous multi-source meteorological data. Specifically, the format of heterogeneous multi-source meteorological raw data is automatically identified to determine the format classification result of the heterogeneous multi-source meteorological raw data; based on the format classification result of the heterogeneous multi-source meteorological raw data, the corresponding parsing algorithm is called to extract the core data fields of the heterogeneous multi-source meteorological raw data to obtain the key fields of the heterogeneous multi-source meteorological raw data; through a preset mapping relationship, the key fields of the heterogeneous multi-source meteorological raw data are converted into a standard structure format and semantic alignment processing is performed to obtain semantically aligned heterogeneous multi-source meteorological data.

[0044] In this embodiment, the system determines the heterogeneous data from multiple sources obtained through the data access method and performs unified parsing based on the data source acquisition method. This converts the original data with different structures and encodings into a unified internal standard structure, thereby achieving semantic alignment of the multi-source data.

[0045] First, format identification and classification are performed. Based on the determined data access method, the acquired multi-source data is automatically identified in terms of format. According to data suffix, file header identifier, encoding characteristics, etc., the data is classified into categories such as radar data, satellite data, nc format, tiff format, shp format, gpk format, and geojson format, providing a classification basis for subsequent format parsing.

[0046] Then, format-specific parsing and standard conversion are performed. For each data format, a corresponding parsing algorithm is used to analyze the storage structure and encoding rules of the original data, extract the core data fields, and then convert them into the system's internal standard structure through a preset mapping relationship, while simultaneously completing semantic alignment. The parsing of the mainstream data formats involved is explained below: Radar / satellite data: The main analysis focuses on the observation time, observation area, observation parameters (such as radar reflectivity, satellite brightness temperature), and geographic projection information. The key processing steps include the binary encoding of the data, band information, and coordinate correction. nc (NetCDF) format: Parse its dimension information, variable information, attribute information, extract raster or time series data, and process its compression encoding and multidimensional storage structure; TIFF format: Parse its raster matrix, pixel depth, georegistration information, extract pixel values ​​and spatial reference information of raster data, and process its compression format (such as LZW, DEFLATE). shp / gpk format: Parse its vector features (points, lines, polygons), attribute tables, and spatial indexes; extract the coordinate and attribute information of the features; and process their topological relationships. The geojson format parses the feature types, coordinate arrays, and attribute dictionaries in its JSON structure, extracts vector feature information, and unifies the coordinate reference system.

[0047] The core conversion formula is as follows: ; in, This indicates the data format type identifier, used to distinguish the format of the raw data and provide a basis for the selection of parsing algorithms. This represents raw, multi-source heterogeneous data, obtained through two access methods, and is the original, unparsed data, including the data body and format identifier. Representation format The parsing parameter set, including the format The parsing rules, encoding methods, data extraction thresholds, and other key parameters are crucial for the parsing algorithm. The core input. It represents semantic mapping rules, defining the correspondence between original data fields and system standard fields, data type conversion rules, and semantic ambiguity elimination rules. This represents standardized data output after parsing and semantic alignment of standard data within the system. It includes spatial information, attribute information, temporal information, etc., and has a unified structure and consistent semantics.

[0048] S220. After semantic alignment, the heterogeneous multi-source meteorological data are sequentially subjected to special optimization processing and layer granularization splitting processing to obtain several layer granules.

[0049] Specifically, redundant data removal is performed on the semantically aligned heterogeneous multi-source meteorological data to obtain the removed heterogeneous multi-source meteorological data. Based on a lossless compression algorithm, the removed heterogeneous multi-source meteorological data is compressed and optimized to obtain compressed heterogeneous multi-source meteorological data. The compressed heterogeneous multi-source meteorological data is then optimized for temporal and spatial simplification. The temporal data in the heterogeneous multi-source meteorological data is reasonably thinned, and the spatial coordinate storage format of the spatial data in the heterogeneous multi-source meteorological data is optimized to obtain optimized heterogeneous multi-source meteorological data. Based on the meteorological element types, spatial ranges, and temporal characteristics in the heterogeneous multi-source meteorological data, the optimized heterogeneous multi-source meteorological data is split into layers into granular layers to obtain several layer granules.

[0050] In this embodiment, standardized meteorological data is obtained from multi-format meteorological data through unified parsing. This standardized data has undergone format unification and semantic alignment, and contains complete spatial, attribute, and temporal information. This data is then broken down into "minimum data units" that can be directly used for subsequent service assembly. Furthermore, specialized optimizations are used to eliminate data redundancy, compress data volume, and improve data transmission rates and front-end rendering efficiency.

[0051] First, a specific optimization process was performed on the standardized meteorological data. Addressing redundancy, transmission bottlenecks, and rendering challenges, this study aims to compress data volume to the maximum extent possible without losing critical meteorological information. Improve data availability.

[0052] Then, redundant data is removed, eliminating redundant fields and abnormal data from the standardized data that are irrelevant to subsequent services, retaining key information, and reducing data volume. The formula is as follows: ; in, Rules for removing redundant data include criteria for identifying redundant fields and abnormal data. An abnormal data threshold is set to identify outliers in meteorological data. To standardize meteorological data, the output standardized data after parsing and semantic alignment includes spatial, attribute, and temporal information, with a unified structure and consistent semantics. This indicates that the optimized data has been removed, retaining only key meteorological information.

[0053] Furthermore, data compression optimization is performed. Considering the numerical characteristics of meteorological data, a lossless compression algorithm is adopted to compress the data storage and transmission volume while ensuring no loss of data accuracy, thereby reducing transmission bandwidth usage and improving transmission efficiency. The expression is as follows: ; ; in, This represents the compressed data volume. For optimized data The original volume of the data. Compression ratio (0 < <1), the smaller the compression ratio, the more obvious the data volume reduction effect. The compression algorithm aims to reduce data precision without sacrificing accuracy. Compress to This reduces data size and improves transmission efficiency. This is the compressed data.

[0054] Then, temporal / spatial simplification and optimization are carried out. While ensuring that the temporal resolution meets the service requirements, the temporal data is reasonably thinned. For meteorological data with a large spatial range, the spatial coordinate storage format is optimized to reduce spatial data redundancy and improve the smoothness of front-end rendering.

[0055] Layer granularization is performed, based on the element type, spatial range, and temporal characteristics of meteorological data, to optimize the data. The data is broken down into "minimum service units" to achieve refined data management and meet the flexibility requirements of subsequent service assembly. The specific breakdown rules are as follows: Splitting by meteorological element type: The optimized data is split into independent basic granular layers according to meteorological elements. Each layer contains only a single type of meteorological element data and the corresponding spatial and temporal information.

[0056] Splitting by Spatial Range: For a single meteorological element, the granular layer is split into smaller spatial granular units according to a preset spatial grid. Each unit corresponds to a fixed spatial range, which facilitates precise positioning and local rendering.

[0057] Splitting by temporal features: For granular layers containing temporal information, they are split into temporal granular units according to time resolution. Each unit corresponds to a fixed time node, which facilitates flexible calling by time dimension.

[0058] The specific formula is as follows: ; in, It is a granular splitting rule (including splitting criteria based on feature type, spatial range, and temporal characteristics). Define the splitting dimensions for feature type parameters to distinguish granular layers for different features. The spatial partitioning parameter defines the range and size of the spatial particle unit, which determines the fineness of the spatial particles. The timing decomposition parameter defines the temporal resolution of the timing granular unit, which determines the fineness of the timing granularity. It is a set of granular layers, containing n smallest granular units, each unit corresponding to a single feature, fixed space, and fixed time sequence.

[0059] The S300 performs multi-layer flexible assembly and standardized service publishing on several layers of particles in sequence, and matches the corresponding SDK dedicated renderer for rendering to obtain multi-source meteorological data visualization results.

[0060] S310. Obtain the preset service requirements, perform multi-layer flexible assembly and standardized service publishing on several layer particles in sequence, and obtain a service dataset in a callable state. Specifically, the system acquires and parses preset service requirements to obtain requirement parsing results; based on the requirement parsing results, it filters several layer particles to obtain filtered layer particles; according to preset assembly rules, it performs combination operations, layer adaptation, and verification on the filtered layer particles in sequence to output the assembled service dataset; it performs service standardization encapsulation on the assembled service dataset and calls the corresponding interface configuration for registration and publication to obtain a service dataset in a callable state.

[0061] In this embodiment, a single layer particle output from data processing, optimization, and granularization is used as the smallest assembly unit. Based on service requirements and combined with preset assembly rules, multiple layer particles are flexibly combined and layered to form a service dataset that meets the needs of a specific scenario. At the same time, a standardized service publishing process is performed on the assembled service dataset, which is then encapsulated into a visualization service that can be directly called by the client.

[0062] First, a multi-layered, flexible assembly based on layered particles is performed. Assembly requirements are analyzed by receiving pre-set service requirements from clients or the system and analyzing the core elements within those requirements.

[0063] Layer particle filtering: Based on the requirements analysis results, select the layer particles that meet the requirements from the output layer particle set L.

[0064] Multi-layer flexible combination and layer adaptation: The selected multiple layer particles are combined according to the preset assembly rules, and the combined layers are adapted to the layer hierarchy.

[0065] Assembly result verification: The assembled service dataset is verified to check the rationality of the layer granularity combination and the data integrity, ensuring that the assembly result can be used for subsequent standardized service deployments and avoiding client call anomalies due to assembly errors. The core formula is as follows: ; in, Define the filtering rules for layer particles. To meet the needs of clients, it includes core information such as meteorological elements, spatial range, and temporal resolution. Define the filtering precision (such as feature type matching threshold, spatial range error threshold). Used to define assembly rule parameters (such as layer overlay priority and transparency threshold). This represents a set of layer particles, which are selected based on the parameters mentioned above to find the layer particles that meet the requirements. For multi-layer assembly rules, the filtered layer particles are flexibly combined and adapted to different levels. A service dataset to meet client requirements.

[0066] Then, standardized service publishing is carried out. The assembled service dataset is packaged according to the system's preset service standards and published as a visual service that can be directly called by the client, clarifying the service call specifications, interface parameters, and data return format.

[0067] Service standardization encapsulation: The assembled service dataset is encapsulated according to the system's preset service standards, which clearly defines the service name, type, and core functions to ensure the standardization of services and adapt to the calling needs of different clients.

[0068] API call configuration: Configure the API interface for client calls to the service, specifying the interface address, request method, request parameters, and return format, and also configure interface call permissions and call frequency limits.

[0069] Service Registration and Publication: The standardized, encapsulated services are registered with the system service registry, updating the system service list and clearly defining the service's unique identifier, publication status, and corresponding assembly dataset. This ensures clients can quickly locate and query services through the service list. Simultaneously, the service publication operation is completed, making the service available for invocation. The core formula is as follows: ; in, For encapsulation rules, To incorporate encapsulation parameters (such as service interface specifications and data transmission format standards). The complete dataset, obtained by multi-layer assembly, is a service dataset containing multiple layer particle combinations that meet client requirements. To publish rules. For publishing parameters (such as service registration address, call permission parameters). It is a visualization service that can be directly invoked.

[0070] S320. Parse the service dataset that is in a callable state, extract the features of each layer and divide it into vector data, raster data and high-precision raster data; S330 uses the corresponding SDK-specific renderer to render vector data, raster data, and high-precision raster data respectively, to obtain multi-source meteorological data visualization results.

[0071] In this embodiment, the result of receiving layer assembly and service publication (i.e., the assembled service dataset corresponding to the standardized visualization service, denoted as...) is... Based on different layer data types, the system automatically matches and calls the SDK's built-in dedicated layer renderer to complete layered rendering of various types of data. For raster data, it innovatively adopts a "grayscale image compression transmission + decompression restoration" strategy to reduce transmission volume, while using an 8-neighbor pixel 2D interpolation algorithm to achieve high-precision smooth rendering. Ultimately, while significantly reducing data transmission volume, it ensures clear, continuous, and jagged visualization effects, providing clients with high-quality meteorological data visualization.

[0072] Perform layer data type identification and output. The data is analyzed and the core features of each layer are extracted, dividing the layer data into three categories: vector data, raster data, and high-precision raster data.

[0073] S331. For vector data, a grayscale compression transmission strategy is adopted. The server linearly maps the raster values ​​to preset grayscale values ​​for transmission. The client calculates and restores the original values ​​and then matches the color marks to complete the color rendering. In this embodiment, for vector data, a dedicated vector renderer adapts to the rendering requirements of vector elements, focusing on rendering the outline, color, and attribute annotations of the elements to ensure that the vector elements are clearly identifiable.

[0074] S332. For raster data, based on grayscale compression transmission, an 8-neighbor pixel two-dimensional interpolation algorithm is used to perform spatial smoothing on the raster data to complete color rendering. In this embodiment, for raster data, a dedicated raster renderer is used to adapt to the rendering requirements of ordinary raster data. Combined with a grayscale image compression and transmission strategy, it balances rendering efficiency and basic visualization effects, including the following steps: 1) Read the grayscale image as a WebGL material. The server compresses the original raster data into a grayscale image for transmission. The client reads the grayscale image and loads it as a WebGL texture material to complete the data input.

[0075] 2) The fragment shader retrieves the pixel RGB values ​​and calculates the grid point values. In the WebGL fragment shader, the pixel RGB values ​​of the grayscale image (R=G=B=Gray) are read and converted and compressed. The core formula is as follows: ; in, This is the grayscale value converted from the weighted average of the RGB three-channel brightness values. R, G, and B represent the brightness values ​​of the red, green, and blue channels, respectively, with values ​​ranging from 0 to 255.

[0076] 3) Match color values ​​according to the legend. Match the calculated feature values ​​with the configured color code legend, mapping them to the corresponding RGB color values, thus completing the data-to-visual color conversion. Specifically, this includes: The server-side compression mapping has the following expression: ; Where V represents the original meteorological element value. This represents the range of values ​​for this element. To ensure rounding to the nearest integer, the grayscale value must be an integer between 0 and 255.

[0077] The client-side inverse calculation and restoration is expressed as follows: ; This algorithm significantly reduces data transmission volume and improves loading speed while ensuring negligible loss of data accuracy (error ≤ 0.4%).

[0078] 4) Perform map rendering, write the matched color values ​​into the WebGL frame buffer, complete the final rendering of the raster layer, and output a clear and continuous visualization of meteorological elements.

[0079] 5) Perform spatial coordinate mapping. Spatial coordinate mapping and interaction enable precise mapping between geographic latitude and longitude and screen pixel coordinates, ensuring the spatial accuracy of data display. The mapping formula is as follows: ; in, Geographic latitude and longitude coordinates These are screen pixel coordinates (with the top left corner of the canvas as the origin). This mapping, which defines the width and height (in pixels) of the canvas, supports interactive operations such as click query, spatial positioning, zooming, and panning, ensuring a precise correspondence between user actions and data spatial locations.

[0080] S333. For high-precision raster data, block division is performed according to fixed size. Rendering efficiency is improved through multi-threaded parallel processing. Spatial interactive operation is achieved by combining latitude and longitude with screen pixel coordinate mapping to complete color rendering.

[0081] In this embodiment, for high-precision raster data, a dedicated high-precision raster renderer is used: adapting to the rendering requirements of high-resolution raster data, and combining an 8-neighbor pixel 2D interpolation algorithm to achieve smooth rendering and eliminate jagged edges.

[0082] To address the pain points of large file size, slow transmission, and jagged edges in rendering of raster data (including ordinary raster and high-precision raster), a combined optimization strategy of "grayscale image compression and transmission + inverse compression and restoration + 8-neighbor pixel 2D interpolation" is adopted to reduce the amount of data transmitted while ensuring the clarity, continuity, and smoothness of the rendering effect.

[0083] For 8-neighbor 2D interpolation smoothing, it performs spatial smoothing on raster / grid data to avoid visual jaggedness and abrupt color block changes, while preserving data details. The interpolation formula is as follows: ; ; in, The normalized coordinates of the target point within the local grid. . The original values ​​are for the 8 neighborhood points around the target point plus the center point, totaling 9 grid points. These are the weighting coefficients.

[0084] Data partitioning and multithreaded processing. Large raster / grid data is partitioned into fixed-size blocks, and efficiency is improved through multithreaded parallel processing. The formula for the number of blocks is as follows: ; in, This represents the data width (number of pixels / number of grid points). The data height (number of pixels / number of grid points). This is the block size (default 256×256, adapted to WebGL texture size requirements). To round up and ensure complete data coverage, the data is processed in parallel using a thread pool after being divided into blocks. This can improve processing efficiency by 3-5 times, adapting to the real-time processing needs of large-size data.

[0085] The standardized services to be released will be transformed into visual effects that can be directly viewed by the client through high-quality rendering, building a bridge between service release and client presentation, improving the client user experience, and meeting the core needs of meteorological data visualization and business decision-making.

[0086] In summary, the embodiments of the present invention have the following distinguishing technical features compared with the prior art: 1) The integrated access mechanism for multi-source heterogeneous meteorological data supports unified access to all types of meteorological data, including nc, tiff, grib, grib2, radar, satellite, shp, gpk, and geojson. It is compatible with both external API interfaces and local file access modes. The access parameters can be flexibly defined through the database configuration table, which breaks through the shortcomings of traditional access methods that are single and limited in format, and has universality and scalability.

[0087] 2) The data access dynamic scheduling method based on cron expressions allows users to customize the data access period using cron expressions, enabling scheduled data retrieval for minutes, hours, days, months, and batch backtracking without the need for hard-coded modification of scheduling rules. Through time matching judgment and integrity verification, it ensures stable and reliable data access, decouples scheduling strategies from business logic, and significantly improves system adaptability.

[0088] 3) The lightweight optimization algorithm system for high-resolution meteorological data includes a raster data grayscale image compression mapping and inverse calculation restoration algorithm, an 8-neighborhood two-dimensional interpolation smoothing algorithm, and a data block parallel processing algorithm. While ensuring data accuracy, it significantly reduces transmission volume and improves rendering efficiency, solving the problems of slow loading and rendering stuttering of large-size satellite, radar, and grid data.

[0089] 4) The assembly and publishing method based on layer particles as the smallest unit breaks down meteorological data into the smallest layer particles according to elements, time, type, and quality status, supports multi-level free combination and transparent overlay rendering, forming a standardized visualization service; unified output of service metadata enables services to be configurable, reusable, and quickly published, meeting the needs of multi-scenario thematic display.

[0090] 5) The client SDK's integrated rendering architecture includes dedicated vector, raster, and grid renderers. It achieves hardware acceleration based on WebGL, completes spatial interaction through precise mapping between geographic coordinates and pixel coordinates, and supports multi-threaded calculation, progressive loading, and adaptive color scale adjustment, enabling high-definition, high-smoothness, and low-threshold meteorological data visualization.

[0091] Therefore, the embodiments of the present invention have the following advantages compared with the prior art: 1) Enhanced data access capabilities: Existing technologies only collect basic raw data from devices. This invention supports unified access to meteorological data in all formats, from multiple sources, and using multiple protocols, covering various types of data such as model products, satellites, radar, and vector geography, making it more widely applicable.

[0092] 2) The scheduling mechanism is more flexible. Existing technologies rely on fixed processes and preset parameters. This invention adopts dynamic scheduling using cron expressions, which can configure access cycles and strategies as needed, support real-time updates and historical backtracking, and adapt to changing business needs without modifying the program.

[0093] 3) High-resolution processing is more professional. Existing technologies lack dedicated optimization mechanisms. This invention provides professional algorithms such as grayscale compression, 8-neighbor interpolation, and data block division, which significantly improve the efficiency of processing and rendering large files and high-precision data, and solve the performance bottleneck of traditional solutions.

[0094] 4) More complete services and visualization: Existing technologies only complete data processing. This invention constructs a full-link "processing-optimization-granulation-assembly-release-rendering", realizes flexible service orchestration with layer granules, and reduces client access costs with a dedicated SDK.

[0095] 5) Enhanced visualization and interaction: Existing technologies lack professional rendering design. This invention, based on WebGL acceleration, precise coordinate mapping, and multi-threaded parallel computing, achieves high-quality visualization and spatial interaction, better meeting the actual needs of meteorological operations.

[0096] Reference Figure 2 A visualization system for multi-source meteorological data access and processing, comprising: The first module 201 is used to configure the core configuration parameters of the meteorological data to be accessed, and to perform initialization verification and data reading to construct heterogeneous multi-source meteorological raw data; The second module 202 is used to perform format standardization and splitting optimization on heterogeneous multi-source meteorological raw data to obtain several layer particles. The third module 203 is used to flexibly assemble and standardize the services of several layers of particles in sequence, and match the corresponding SDK dedicated renderer for rendering to obtain the visualization results of multi-source meteorological data.

[0097] The content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0098] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this is not intended to limit the scope of the embodiments of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the embodiments of the present application.

Claims

1. A visualization method for multi-source meteorological data access and processing, characterized in that, Includes the following steps: Configure the core configuration parameters of the meteorological data to be accessed, perform initialization verification and data reading, and construct heterogeneous multi-source meteorological raw data; The heterogeneous multi-source meteorological raw data were standardized and split for optimization to obtain several layer particles. The system flexibly assembles and publishes standardized services for several layers of particles in sequence, and then matches them with the corresponding SDK-specific renderer for rendering to obtain multi-source meteorological data visualization results.

2. The visualization method for multi-source meteorological data access and processing according to claim 1, characterized in that, The step of configuring the core configuration parameters of the meteorological data to be accessed, performing initialization verification and data reading, and constructing heterogeneous multi-source meteorological raw data specifically includes: Perform data access metadata initialization. By querying the system configuration database table, obtain the core configuration parameters of all meteorological data to be accessed. The core configuration parameters include data identifier, format type, access method, access address, cron scheduling expression, spatial range and element code. The core configuration parameters of all meteorological data to be accessed are sequentially checked for cron expression syntax, access address connectivity, and format type compliance to obtain the checked configuration parameters. Based on the verified configuration parameters, the data access method for meteorological data is determined, data is read, and heterogeneous multi-source meteorological raw data is constructed.

3. The visualization method for multi-source meteorological data access and processing according to claim 2, characterized in that, The step of determining the data access method for meteorological data based on the verified configuration parameters, reading the data, and constructing heterogeneous multi-source meteorological raw data, wherein the heterogeneous multi-source meteorological raw data is either first meteorological data or second meteorological data, specifically includes: Based on the verified configuration parameters, the data access method for meteorological data is determined, including external API interface mode and local file mode. Based on the external API interface mode, a data request is initiated with authentication information, spatiotemporal range and element parameters. After the integrity verification is passed, the first meteorological data is output. Based on the local file mode, the system extracts timestamp, feature type, and resolution information by parsing the file name using regular expressions, and then sorts and locates the second meteorological data by timestamp to read it.

4. The visualization method for multi-source meteorological data access and processing according to claim 3, characterized in that, The step of standardizing and splitting heterogeneous multi-source meteorological raw data to obtain several layer particles specifically includes: Heterogeneous multi-source meteorological raw data are subjected to unified parsing of multi-format meteorological data to obtain semantically aligned heterogeneous multi-source meteorological data. After semantic alignment, the heterogeneous multi-source meteorological data are sequentially subjected to specialized optimization processing and layer granularization splitting processing to obtain several layer granules.

5. The visualization method for multi-source meteorological data access and processing according to claim 4, characterized in that, The step of performing unified parsing of heterogeneous multi-source meteorological raw data in multiple formats to obtain semantically aligned heterogeneous multi-source meteorological data specifically includes: Automatic format identification is performed on heterogeneous multi-source meteorological raw data to determine the format classification results of heterogeneous multi-source meteorological raw data; Based on the format classification results of heterogeneous multi-source meteorological raw data, the corresponding parsing algorithm is called to extract the core data fields of the heterogeneous multi-source meteorological raw data, and the key fields of the heterogeneous multi-source meteorological raw data are obtained. By using a pre-defined mapping relationship, the key fields of the heterogeneous multi-source meteorological raw data are converted into a standard structure format and semantically aligned to obtain semantically aligned heterogeneous multi-source meteorological data.

6. The visualization method for multi-source meteorological data access and processing according to claim 5, characterized in that, The step of sequentially performing specialized optimization and layer granularization processing on the semantically aligned heterogeneous multi-source meteorological data to obtain several layer particles specifically includes: Redundant data removal is performed on the semantically aligned heterogeneous multi-source meteorological data to obtain the removed heterogeneous multi-source meteorological data. Based on the lossless compression algorithm, the heterogeneous multi-source meteorological data after removal is subjected to data compression optimization processing to obtain compressed heterogeneous multi-source meteorological data. The compressed heterogeneous multi-source meteorological data is optimized by time-series and spatial simplification. The time-series data in the heterogeneous multi-source meteorological data is reasonably thinned, and the spatial coordinate storage format of the spatial data in the heterogeneous multi-source meteorological data is optimized to obtain the optimized heterogeneous multi-source meteorological data. Based on the meteorological element types, spatial ranges, and temporal characteristics in the heterogeneous multi-source meteorological data, the optimized heterogeneous multi-source meteorological data is split into layers into granular layers, resulting in several layer granules.

7. The visualization method for multi-source meteorological data access and processing according to claim 6, characterized in that, The step of flexibly assembling and standardizing the service deployment of several layers of particles sequentially, and then matching them with the corresponding SDK-specific renderer for rendering to obtain multi-source meteorological data visualization results, specifically includes: The system obtains the preset service requirements, performs multi-layer flexible assembly and standardized service publishing on several layer particles in sequence, and obtains a service dataset in a callable state. The service dataset that is in a callable state is parsed, the features of each layer are extracted and divided into vector data, raster data and high-precision raster data; Vector data, raster data, and high-precision raster data are rendered using the corresponding SDK-specific renderers to obtain multi-source meteorological data visualization results.

8. The visualization method for multi-source meteorological data access and processing according to claim 7, characterized in that, The step of obtaining preset service requirements, sequentially performing multi-layer flexible assembly and standardized service publishing on several layer granularities to obtain a service dataset in a callable state specifically includes: Obtain the preset service requirements, parse them, and obtain the requirements parsing results; Based on the results of the requirements analysis, several layer particles are filtered to obtain the filtered layer particles. According to the preset assembly rules, the filtered layer particles are combined, adapted, and validated in sequence, and the assembled service dataset is output. The assembled service dataset is encapsulated into standardized services, and the corresponding interface configuration is called for registration and publication, resulting in a service dataset that is in a callable state.

9. The visualization method for multi-source meteorological data access and processing according to claim 8, characterized in that, The step of matching vector data, raster data, and high-precision raster data with their respective SDK-specific renderers for rendering specifically includes: For vector data, a grayscale compression transmission strategy is adopted. The server linearly maps the raster values ​​to preset grayscale values ​​for transmission, and the client calculates and restores the original values ​​and then matches the color marks to complete the color rendering. For raster data, based on grayscale compression transmission, an 8-neighbor pixel two-dimensional interpolation algorithm is used to perform spatial smoothing on the grid data to complete color rendering; For high-precision raster data, it is divided into blocks according to a fixed size, and the rendering efficiency is improved by multi-threaded parallel processing. Spatial interaction operations are realized by combining latitude and longitude with screen pixel coordinate mapping to complete color rendering.

10. A visualization system for multi-source meteorological data access and processing, characterized in that, Includes the following modules: The first module is used to configure the core configuration parameters of the meteorological data to be accessed, and to perform initialization verification and data reading to construct heterogeneous multi-source meteorological raw data; The second module is used to perform format standardization and splitting optimization on heterogeneous multi-source meteorological raw data to obtain several layer particles. The third module is used to flexibly assemble and standardize the services of several layers of particles, and match them with the corresponding SDK dedicated renderer for rendering to obtain multi-source meteorological data visualization results.

Images