A meteorological grid algorithm access assessment method, system and device

Abstract

The present invention belongs to the field of meteorological services, and specifically provides a meteorological grid algorithm access assessment method, system and device, the method comprising: S1, completing the splitting and registration of the API algorithm according to the algorithm splitting rules; S2, using the asynchronous event response flow technology designed based on the observer pattern, and freely arranging the algorithm in series and parallel to form a new task according to the business logic; S3, evaluating the registered API algorithm according to the assessment model. The technology of obtaining heterogeneous database data sources based on visual configuration in the present invention solves the user's learning pressure on the new database. Ordinary users can obtain data without directly connecting to the database, avoiding the problem of too many connections affecting the normal production performance of the business database during high concurrent requests; using mirrored container deployment, the server automatically scales the capacity according to the user's long SQL, CPU and memory usage, which helps to improve the performance of the entire cluster.

Description

Technical Field

[0001] The present invention belongs to the technical field of meteorological services, and in particular relates to a meteorological grid algorithm access assessment method, system and device. Background Art

[0002] To promote data intensiveness and algorithm sharing, there is an urgent need to establish an evaluation system to assess the performance of algorithms. Because meteorological algorithms process large amounts of data and have high spatiotemporal resolution, the input data sources must be stored locally. Urgent challenges exist: efficiently acquiring local data, decoupling algorithm logic from input, freely combining disassembled algorithms to form new ones, and designing an entry-level evaluation method for applications with varying input and output scales. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a meteorological grid algorithm access evaluation method, system and device, which can freely connect upstream and downstream algorithms to form a new algorithm according to business needs, and conduct a comprehensive and objective evaluation of the algorithm through multi-dimensional indicators.

[0004] The present invention is implemented as follows: a meteorological grid algorithm access assessment method comprising:

[0005] S1. Complete the splitting and registration of API algorithms according to the algorithm splitting rules;

[0006] S2. Adopting asynchronous event response flow technology based on the observer pattern, the algorithm is freely arranged in series and parallel to form a new task according to the business logic;

[0007] S3. Evaluate the registered API algorithm based on the evaluation model.

[0008] Furthermore, the S1 further includes the following sub-steps:

[0009] S101, splitting meteorological algorithms of different business types according to preset algorithm granularity;

[0010] S102, enter relevant information according to the page configuration and instantiate the data source;

[0011] S103. According to the HTTP request sent by the user, the fields in the data table are concatenated into the request URL using the & symbol;

[0012] S104. The server creates a basic image of the Xugu connection pool API, NoSql database connection pool API, and interface API according to the preset, and stores it in the image warehouse;

[0013] S105. After responding to the instructions of S102 and S103, the server extracts the basic image generated by S104 and generates a container of scalable Xugu connection pool API, NoSql connection pool API, and interface connection pool API;

[0014] S106, automatically connect to the Xugu database or NoSql database according to the parameters of the Http request in the interface connection pool API container;

[0015] S107. Register the split API algorithm in the business platform.

[0016] Furthermore, the step S2 further includes the following sub-steps:

[0017] S201. Connect all registered API algorithms in series according to business logic to form an algorithm flow;

[0018] S202, determining the initial algorithm of each algorithm flow;

[0019] S203, determining input parameters of upstream and downstream algorithms in each algorithm flow;

[0020] S204: Save all algorithm flows to form tasks.

[0021] Furthermore, the step S3 further includes the following sub-steps:

[0022] S301. Obtain information related to the API algorithm;

[0023] S302, inputting the information into the evaluation model to calculate the algorithm evaluation index;

[0024] S303. Determine the API algorithm evaluation result according to the algorithm evaluation index.

[0025] Furthermore, the information includes: the amount of algorithm input data, the amount of algorithm theoretical output data, the amount of algorithm output data, the total algorithm time, the time it takes for the server to obtain data, the time it takes from the end of the algorithm calculation to the user end, the number of CPU cores requested by the algorithm, the amount of memory requested by the algorithm, the number of users who call the algorithm, the number of daily visits, the number of successful algorithm deployments per day, the number of algorithms, and the number of units to which the users who call the algorithm belong;

[0026] The algorithm evaluation indicators include algorithm single-second data volume indicator, accuracy indicator, algorithm call status indicator, algorithm resource utilization indicator, and algorithm decoupling degree indicator;

[0027] The evaluation model includes an algorithm single-second data volume model, an algorithm accuracy model, an algorithm call index model, an algorithm resource utilization index model, and a decoupling index model.

[0028] Furthermore, the algorithm single-second data volume model calculates the algorithm single-second data volume indicator based on the algorithm input data volume, algorithm output data volume, algorithm total time consumption, server-side data acquisition time consumption, time consumption from algorithm calculation completion to user end consumption, number of CPU cores requested by the algorithm, and amount of memory requested by the algorithm;

[0029] The algorithm accuracy model calculates the accuracy index based on the algorithm theory output data volume and the algorithm output data volume;

[0030] The algorithm call index model calculates the algorithm call status index based on the number of users who call the algorithm, the number of daily visits, and the number of successful daily algorithm deployments;

[0031] The algorithm resource utilization index model calculates the highest CPU and memory usage within a certain period of time based on the number of daily visits;

[0032] The decoupling index model calculates the decoupling degree according to the amount of data output by the algorithm and the number of units to which the algorithm calls the user.

[0033] Furthermore, the expression of the single-second data volume model of the algorithm is:

[0034] ;

[0035] Where S1 represents the amount of input data to the algorithm, S2 represents the amount of output data from the algorithm, T represents the total time taken by the algorithm, T1 represents the time taken by the server to obtain data, T2 represents the time taken from the end of the algorithm calculation to the end of the user, C represents the number of CPU cores requested by the algorithm, and G represents the amount of memory requested by the algorithm.

[0036] The expression of the algorithm accuracy model is:

[0037] ;

[0038] Among them, S2 represents the amount of algorithm output data recorded by the gateway, and S3 represents the theoretical output data of the algorithm for a single successful call;

[0039] The expression of the algorithm calling indicator model is:

[0040] ;

[0041] ;

[0042] ;

[0043] ;

[0044] ;

[0045] Among them, E represents the number of successful calls to the algorithm, n represents the number of users who call the algorithm, and E i represents the number of successful calls of each algorithm, excluding the maximum and minimum access times within k days, σ represents the standard deviation of the algorithm call times, represents the average number of successful calls of the algorithm, C j represents the number of successful algorithm visits per day within k-2 days; D j Indicates the number of suspicious visits per day by the algorithm. The statistical rule is the total number of successful calls with the same IP and input within the sliding time period. In each sliding time period, the first successful call is not counted, and the rest are counted as suspicious calls. a Indicates the success rate of algorithm call, C a Indicates the number of daily algorithm calls within k-2 days;

[0046] The expression of the algorithm resource utilization index model is:

[0047] ;

[0048] ;

[0049] Among them, U_memory represents memory utilization, G i Indicates the actual maximum memory usage of algorithm i, U_cpu indicates the CPU utilization, C i Indicates the actual maximum CPU usage of algorithm i;

[0050] The expression of the decoupling index model is:

[0051] ;

[0052] Where γ represents the Pearson correlation coefficient, V β Indicates the standard deviation of the algorithm output data volume, represents the average amount of data output by the algorithm, m represents the number of units to which the algorithm calls the user, It represents the average number of units, and g represents the number of algorithms.

[0053] The present invention also provides a meteorological grid algorithm access evaluation system, comprising:

[0054] Algorithm registration module, used to complete the registration of API algorithms according to user instructions;

[0055] The task orchestration module is used to connect registered API algorithms in series according to business logic and form tasks;

[0056] The algorithm evaluation module is used to evaluate the registered API algorithms based on the evaluation model.

[0057] Furthermore, it also includes:

[0058] Algorithm management module, used to edit API algorithms according to user instructions and generate algorithm logs;

[0059] The algorithm call module is used for permission management, providing task load, request forwarding, and accessing algorithm logs.

[0060] The present invention also provides a meteorological grid algorithm admission evaluation device, comprising a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to execute the computer program to implement the steps of the above method.

[0061] The beneficial effects brought by the present invention are:

[0062] 1. The technology of obtaining heterogeneous database data sources based on visual configuration in this invention alleviates the pressure on users to learn new databases. Ordinary users can obtain data without directly connecting to the database, avoiding the problem of excessive number of connections affecting the normal production performance of the business database during high-concurrency requests. By adopting image container deployment, the server automatically scales the capacity according to the user's long SQL, CPU and memory usage, which helps to improve the performance of the entire cluster.

[0063] 2. The algorithm evaluation model in this invention is suitable for the access assessment of high-longitude grid meteorological data. It avoids the problem of simply counting the number of algorithm calls to reflect algorithm activity, which helps to avoid the problem of repeated calls. Based on practical business practices, it introduces the concept of algorithm decoupling and calculates the correlation coefficient based on the size of the algorithm output and the number of units to which the user belongs, which has certain promotional value. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 Flowchart of the method of the present invention.

[0065] Figure 2 The operation demonstration diagram configured for the page.

[0066] Figure 3 This is a diagram demonstrating the operation of algorithm concatenation.

[0067] Figure 4 This is a demonstration diagram of the series connection of three API algorithms.

[0068] Figure 5 This is a schematic diagram of the results of the Kriging interpolation algorithm.

[0069] Figure 6 This is a data interaction flow chart of the present invention.

[0070] Figure 7 This is a flowchart of S106 in the present invention. DETAILED DESCRIPTION

[0071] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention.

[0072] like Figures 1 to 7 As shown, this embodiment discloses a meteorological grid algorithm admission evaluation method, including steps S1-S3.

[0073] S1. Complete the splitting and registration of the API algorithm according to the algorithm splitting rules. This step is based on multi-process and multi-threaded hybrid programming, as well as block and KD tree query technology, and adopts a mirror container method to achieve fast and elastic data acquisition. Specifically, S1 also includes the following sub-steps:

[0074] S101. Split meteorological algorithms of different business types according to the preset algorithm granularity. In this step, meteorological algorithms can be divided into different types such as site real-time statistics, grid mode processing, radar processing, etc. according to the business type. The split granularity is preferably one or more related products corresponding to one algorithm. Taking the site real-time statistics algorithm as an example, the meteorological element value x={precipitation, temperature, humidity, pressure, visibility, weather phenomenon}, x=1...6; spatial range y={province-city-county-township four-level statistics, latitude and longitude range statistics, watershed statistics}, y=1,2,3; time range z={fixed time period (5 / 10 / 15 / 30 / 60...minutes) statistics, sliding time period (5 / 10 / 15 / 30 / 60...minutes) statistics}, z=1,2. The designed algorithm granularity is as follows:

[0075] ;

[0076] Among them, each time interval in the fixed time period is designed as an algorithm, and the same is true for the sliding time period statistics.

[0077] S102, input relevant information according to the page configuration and instantiate the data source. Figure 2 As shown, based on the page configuration, the user fills in the account, password, IP address, data table, interface and other information to instantiate the data source, corresponding to Figure 7 The data source instantiation module in .

[0078] S103, such as Figure 6 As shown in the figure, a user searches for weather data. Based on the HTTP request sent by the user, the server concatenates the fields in the data table into the request URL using the & symbol and dynamically converts the API into database SQL.

[0079] S104. The server creates a basic image of the database connection pool API (Xugu connection pool API, NoSql database connection pool API) and the interface API according to the preset, and stores it in the image warehouse.

[0080] S105. After responding to the instructions in S102 and S103, the server extracts the base image generated in S104 and generates scalable containers for the Xugu connection pool API, NoSql connection pool API, and interface connection pool API. The connection pool API container scaling rules are as follows: 1) Define each time window as 30 minutes, with 00:00:00-00:30:00 UTC as the first time window, and the same applies to the remaining time windows. 2) Calculate the number of HTTP requests, container CPU usage, and container memory usage within each time window container. 3) Calculate the average of the three sliding time windows. If a single user search spans more than 10 days in HTTP requests, CPU utilization is greater than 60%, and memory utilization is greater than 80%, if any of the three conditions are met, expand one node. If a single user search spans less than 24 hours in HTTP requests, CPU utilization is less than 20%, and memory utilization is less than 40%, recycle one node.

[0081] S106, the parameters of the Http request in the connection pool API container automatically connect to the Xugu database, NoSql database or TianQing API interface, and return the response result. Figure 7 As shown, in this step, the response result is returned in JSON format after data acceleration processing. The data acceleration processing process includes the following steps:

[0082] S1061. Obtain parameters of the HTTP request, including the grid starting latitude lat1, ending latitude lat2, starting longitudes lon1 and lon2, and grid resolution res (decimal, such as 0.01°).

[0083] S1062. Calculate the number of grid blocks according to the parameters of the HTTP request, and divide the grid blocks according to the number of grid blocks. The formula for calculating the number of grid blocks is:

[0084] ;

[0085] Among them, part_num represents the number of grid blocks, and step represents the step size, which is generally an integer multiple of the resolution res.

[0086] S1063. Query grid meteorological data and site meteorological data based on the grid's starting latitude, ending latitude, and starting longitude. Perform data quality control and remove site meteorological data with missing data, illegal characters, or time / space inconsistencies. Site meteorological data comes from the Xugu database, and grid meteorological data comes from a NoSQL database.

[0087] S1064. The grid meteorological data of each quality-controlled grid block is integrated with the station meteorological data of the corresponding area. A "multi-process + multi-thread" hybrid programming technique is used. For the grid meteorological data, grid meteorological data at the same latitude is processed using the same process, while different grid blocks at the same latitude are processed using different threads. The formula for calculating the number of process threads in the computer memory is:

[0088] ;

[0089] ;

[0090] Among them, process_num represents the number of processes, and threads_num represents the number of threads.

[0091] In each thread, a KD tree is constructed using the longitude and latitude values of the station, and the feature values are used as attribute values to achieve fast nearest neighbor search. The grid meteorological data are compared with each other. The station meteorological data is used as the true value for the same geographical location, and the grid meteorological data is used for the other locations.

[0092] S1065: The data of each grid block after data fusion is spliced together to obtain a data query result. The data query result includes three-dimensional information: longitude, latitude, and elements.

[0093] S107: Register the split API algorithm on the business platform. During this process, you'll need to enter information such as the algorithm's English name, the business system it belongs to, the development language, input parameters, output parameters, and the startup command. For approved algorithms, dual instances are automatically created, and load balancing is implemented.

[0094] It should be noted that, in step S1, steps S102 and S103 are completed by the user on the client side, and the remaining steps are completed on the server side.

[0095] S2. Connect the registered API algorithms in series according to the business logic and form tasks. This step uses the asynchronous event response stream (RxJava) technology designed based on the observer pattern, which can effectively avoid the occurrence of callback hell. The biggest difference from the conventional standard observer pattern is that it takes data flow as the core, handles the input, processing and output of data, and finally realizes the connection and combination of multiple existing API algorithms by dragging and dropping to form a new API algorithm. In the same task, the running results of the upstream algorithm do not need to be written to the disk, but are directly transmitted to the downstream algorithm in the computer memory. The biggest advantage of this solution is that it avoids the dependence of the intermediate results of the algorithm running in the same task on the storage medium, effectively improves the convenience of algorithm registration, and reduces the frequent IO interaction of data between the database / storage disk and the computer memory. Specifically, S2 includes the following sub-steps:

[0096] S201, connect all registered API algorithms in series according to business logic to form an algorithm flow. Figure 3 As shown, on the operation interface, click the right side of the algorithm box and drag an arrow to point to the next API algorithm to be executed. And so on, complete the series connection of all algorithms.

[0097] S202: Determine the initial algorithm for each algorithm flow. This step requires selecting the algorithm data input, that is, only the first algorithm in series can select the data source prepared by the algorithm registration module for input, and the input data format is JSON.

[0098] S203: Determine the input parameters for the upstream and downstream algorithms in each algorithm flow. This step requires modifying the algorithm input parameters, meaning that parameters can be modified for each algorithm. The type and number of modifiable parameters remain the same as during registration. Data flows between upstream and downstream algorithms are transmitted in JSON format. As can be seen, the results of upstream and downstream algorithms running the same task are transferred directly to computer memory (RAM), eliminating reliance on storage media.

[0099] S204: Save all algorithm flows as a task. When saving an algorithm, enter the name of the concatenated algorithm to create a task. After approval, the server instantiates the algorithm workflow by pulling the image repository for the task.

[0100] In step S2, when algorithms are combined in parallel, a synchronization barrier mechanism is used. After the algorithms at the same level are executed, status information is sent to the Kafka message queue. The downstream algorithm can only start after receiving the status return codes of all upstream algorithms.

[0101] like Figure 4 and Figure 5 As shown in the figure, a temperature color map algorithm for a specific province is used as an example. The data source is obtained through the TianQing API (obtaining hourly surface temperature data across China by time and administrative division). The algorithm involves the concatenation of three API algorithms: 1) quality control of the acquired temperature data using a quality control algorithm; 2) interpolation of station meteorological data into gridded meteorological data with a resolution of 5 kilometers using a kriging interpolation algorithm; and 3) color map filling of the gridded meteorological data using a color map filling algorithm. These three algorithms are packaged and combined into a new algorithm and released as an API interface for other users to use.

[0102] S3. Evaluate the registered API algorithm based on the evaluation model. This step evaluates the algorithm's operational efficiency, stability, reliability, and business applicability based on the algorithm's single-second data volume, accuracy, number of calls, call success rate, algorithm resource utilization, and the degree of decoupling between the algorithm's processing logic and algorithm input. Specifically, S3 also includes the following sub-steps:

[0103] S301. Obtain information related to the API algorithm. This information includes: the amount of algorithm input data, the amount of algorithm output data, the total algorithm time, the time it takes for the server to obtain data, the time it takes from the algorithm's completion to the user's end, the number of CPU cores requested by the algorithm, the amount of memory requested by the algorithm, the number of users calling the algorithm, the number of daily visits, the number of successful algorithm visits per day, the number of suspicious visits per day, the number of algorithms, and the number of organizations to which the users calling the algorithm belong.

[0104] S302: Input the information into the evaluation model to calculate the algorithm evaluation index. In this step, the algorithm evaluation index includes the algorithm single-second data volume index, accuracy index, algorithm call status index, algorithm resource utilization index, and algorithm decoupling index.

[0105] The evaluation model includes the algorithm single-second data volume model, algorithm accuracy model, algorithm call index model, algorithm resource utilization index model, and decoupling index model.

[0106] Specifically, the algorithm's single-second data volume model calculates the algorithm's single-second data volume based on the algorithm's input data volume, algorithm's output data volume, algorithm's total time consumption, server-side data acquisition time consumption, time consumption from algorithm calculation completion to user end consumption, algorithm's requested CPU core count, and algorithm's requested memory volume. The expression of the algorithm's single-second data volume model is:

[0107] ;

[0108] Where S1 represents the amount of input data for the algorithm, S2 represents the amount of output data for the algorithm, T represents the total time taken by the algorithm, T1 represents the time taken by the server to obtain data, T2 represents the time taken from the end of the algorithm calculation to the end of the user, C represents the number of CPU cores requested by the algorithm, and G represents the amount of memory requested by the algorithm.

[0109] The algorithm accuracy model is calculated based on the output data volume of the algorithm theory and the output data volume S2 recorded by the gateway algorithm. Taking a province's smart grid file as an example, the latitude range is 25°N-32°N, and the longitude range is 117°E-125°E. If 0.01° is used for interpolation calculation, the final result is The factor values are accurate to 6 decimal places. Then, the number of elements is multiplied by the space size of each data to estimate the theoretical output data volume S3 of the algorithm for a single successful call. The expression of the algorithm accuracy model is:

[0110] .

[0111] The algorithm call indicator model calculates the algorithm call status indicator based on the number of users who call the algorithm, the number of daily visits, the number of successful algorithm visits per day, and the number of suspicious algorithm visits per day. The expression of the algorithm call indicator model is:

[0112] ;

[0113] ;

[0114] ;

[0115] ;

[0116] ;

[0117] Among them, E represents the number of successful calls to the algorithm, n represents the number of users who call the algorithm, and E i represents the number of successful calls of each algorithm, excluding the maximum and minimum access times within k days, σ represents the standard deviation of the algorithm call times, represents the average number of successful calls of the algorithm, C j represents the number of successful algorithm visits per day within k-2 days; D j Indicates the number of suspicious visits per day by the algorithm. The statistical rule is the total number of successful calls with the same IP and input within the sliding time period. In each sliding time period, the first successful call is not counted, and the rest are counted as suspicious calls. a Indicates the success rate of algorithm call, C a Indicates the number of daily algorithm calls within k-2 days.

[0118] The algorithm resource utilization indicator model calculates the highest CPU and memory usage within a certain period of time based on the number of daily visits. The expression of the algorithm resource utilization indicator model is:

[0119] ;

[0120] ;

[0121] Among them, U_memory represents memory utilization, G i Indicates the actual maximum memory usage of algorithm i, U_cpu indicates the CPU utilization, C i Indicates the actual maximum CPU usage of algorithm i.

[0122] The decoupling index model calculates the decoupling degree based on the amount of data output by the algorithm and the number of units corresponding to the algorithm call user. The expression of the decoupling index model is:

[0123] ;

[0124] Where γ represents the Pearson correlation coefficient, V β Indicates the standard deviation of the algorithm output data volume, represents the average amount of data output by the algorithm, m represents the number of units to which the algorithm calls the user, Indicates the average value of the number of units, g represents the number of algorithms. In the case of a successful call, the standard deviation of the algorithm output within k days is V β ; Users call the grid algorithm according to administrative divisions or longitude and latitude ranges to generate different numbers of feature points, and the algorithm output is also different; V β Moderately positively correlated with m, the larger the m value, the higher the V β The larger it is, the higher the degree of decoupling between the algorithm processing logic and data input.

[0125] S303. Determine the API algorithm evaluation results based on the algorithm evaluation indicators. This model evaluates the algorithm's operational efficiency, stability, reliability, and business applicability based on five indicators: algorithm data volume per second, algorithm accuracy, algorithm call indicators, algorithm resource utilization, and algorithm decoupling. Each of these five indicators is scored 20 points, with a maximum score of 100. The scoring calculation process is as follows:

[0126] a) The scoring rules for evaluating the three metrics of data volume processed per second, algorithm accuracy, and algorithm resource utilization for g algorithms are as follows: the maximum value of each metric for the g algorithms is calculated, and the algorithm corresponding to the maximum value has a score of 20. The scores of the g algorithms are as follows:

[0127] ;

[0128] in x i Represents the value of each indicator, max ( x i ) represents the maximum value of the indicator. After rounding, the score is accurate to 2 decimal places.

[0129] b) Evaluate the algorithm call index scores of g algorithms using the following rules: take the maximum number of algorithm calls within k days and the maximum value of the inverse of the standard deviation of the algorithm call number, and use the product of the two as the benchmark. The scores of g algorithms are as follows:

[0130] ;

[0131] in, Represents the inverse of the standard deviation of the number of calls to each algorithm, max ( E i ) represents the maximum number of successful calls of each algorithm, Represents the maximum value of the reciprocal of the standard deviation of the number of times each algorithm is called. After rounding, the score is accurate to 2 decimal places.

[0132] c) The scoring rule for evaluating the decoupling index of g algorithms is as follows: take the standard deviation V of the algorithm output within k days β The maximum value and the maximum number of units m corresponding to the algorithm are multiplied as the benchmark. The scores of g algorithms are as follows:

[0133] ;

[0134] After rounding, the score is accurate to 2 decimal places.

[0135] Finally, the scores of steps a), b), and c) are summed to get the total score. SC = SC 1 + SC 2 + SC 3 , the algorithm recommendation level is determined based on the total score, as shown in the following table:

[0136] Algorithm recommendation level Score range Related Notes 1 [90,100] The algorithm runs stably, has high concurrency, low coupling, and is worth recommending. 2 [80,90） The algorithm runs relatively stably, has high resource utilization, and is generally recommended. 3 [70,80） The algorithm stability needs to be improved and the processing capability is average. You can try to apply 4 [60,70） The algorithm wastes a lot of resources and needs further improvement. 5 [0,60） Algorithm removal processing

[0137] Based on the same inventive concept, the present invention also provides a meteorological grid algorithm access evaluation system, including an algorithm registration module, a task scheduling module, an algorithm evaluation module, an algorithm management module and an algorithm calling module.

[0138] The algorithm registration module is used to complete the registration of the API algorithm according to user instructions. Specifically, this module performs the following sub-steps:

[0139] S101. Split meteorological algorithms of different business types according to preset algorithm granularity.

[0140] S102: Enter relevant information according to the page configuration and instantiate the data source.

[0141] S103: According to the HTTP request sent by the user, the fields in the data table are concatenated into the request URL using the & symbol.

[0142] S104. The server creates the basic images of Xugu connection pool API, NoSql database connection pool API, and interface API according to the presets, and stores them in the image warehouse.

[0143] S105. After responding to the instructions of S102 and S103, the server extracts the basic image generated by S104 and generates scalable containers of Xugu connection pool API, NoSql connection pool API, and interface connection pool API.

[0144] S106. Automatically connect to the Xugu database or NoSql database according to the parameters of the Http request in the interface connection pool API container.

[0145] S107. Register the split API algorithm in the business platform.

[0146] The task orchestration module is used to connect registered API algorithms according to business logic and form tasks. Specifically, this module performs the following sub-steps:

[0147] S201. Connect all registered API algorithms in series according to business logic to form an algorithm flow.

[0148] S202: Determine the initial algorithm of each algorithm flow.

[0149] S203: Determine input parameters of upstream and downstream algorithms in each algorithm flow.

[0150] S204: Save all algorithm flows to form tasks.

[0151] The algorithm evaluation module is used to evaluate registered API algorithms based on the evaluation model. Specifically, the module performs the following sub-steps:

[0152] S301. Obtain information related to the API algorithm.

[0153] S302: Input the information into the evaluation model to calculate the algorithm evaluation index.

[0154] S303. Determine the API algorithm evaluation result according to the algorithm evaluation index.

[0155] The algorithm management module is used to edit API algorithms based on user instructions and generate algorithm logs. Specifically, this module supports online modification of algorithm startup parameters, algorithm name, algorithm description, algorithm code, algorithm version number, and other algorithm information. Algorithm upgrades are notified to downstream algorithms via internal messaging, and algorithm upgrades do not affect existing services of older algorithm versions. Algorithm log printing is developed in Java, using Java to retrieve Kafka system metrics and Kafka to retrieve keys for log acquisition and analysis.

[0156] The algorithm call module is responsible for permission management, providing task load, request forwarding, and accessing algorithm logs. Specifically, this module includes a service gateway, load balancing, service routing, user authentication, and behavior logging. The interface gateway is responsible for unified permission management, load balancing is responsible for providing task load capacity during high-incidence situations, service routing is responsible for accurately and efficiently forwarding requests to different service nodes, and behavior logging is responsible for providing interface access logs to facilitate the tracing of data service interface usage.

[0157] Based on the same inventive concept, the present invention also provides a meteorological grid algorithm access assessment device, comprising a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to execute the computer program to implement the steps of the above method.

[0158] The above are only preferred embodiments of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A meteorological grid algorithm access assessment method, characterized in that: The steps include: S1. Complete the splitting and registration of the API algorithm according to the algorithm splitting rules; S1 also includes the following sub-steps: S101, splitting meteorological algorithms of different business types according to preset algorithm granularity; S102, enter relevant information according to the page configuration and instantiate the data source; S103. According to the HTTP request sent by the user, the fields in the data table are concatenated into the request URL using the & symbol; S104. The server creates a basic image of the Xugu connection pool API, NoSql database connection pool API, and interface API according to the preset, and stores it in the image warehouse; S105. After responding to the instructions of S102 and S103, the server extracts the basic image generated by S104 and generates a container of scalable Xugu connection pool API, NoSql connection pool API, and interface connection pool API; S106, automatically connecting to the Xugu database or NoSql database according to the parameters of the Http request in the interface connection pool API container; S106 also includes the following sub-steps: S1061. Obtain parameters of the HTTP request, including the grid starting latitude lat1, ending latitude lat2, starting longitude lon1, lon2, and grid resolution res; S1062. Calculate the number of grid blocks according to the parameters of the HTTP request, and divide the meteorological grid blocks according to the number of grid blocks; S1063. Query grid meteorological data and site meteorological data based on the grid starting latitude, ending latitude, and starting longitude, and perform data quality control to eliminate site meteorological data with missing data, illegal characters, or time / space inconsistencies; the site meteorological data comes from the Xugu database, and the grid meteorological data comes from the NoSql database; S1064. The grid meteorological data of each quality-controlled meteorological grid block is integrated with the station meteorological data of the corresponding area. A "multi-process + multi-thread" hybrid programming technique is used. For the grid meteorological data, the grid meteorological data at the same latitude are processed by the same process, while different meteorological grid blocks at the same latitude are processed by different threads. Within each thread, a KD tree is constructed using the station longitude and latitude values, with feature values used as attribute values to achieve fast nearest neighbor search. For comparisons with the same grid meteorological data, the station meteorological data is used as the true value for the same geographical location, and the grid meteorological data is used for other locations. S1065, splicing the data of each meteorological grid block after data fusion to obtain a data query result; The data query results include three dimensional information: longitude, latitude, and elements; S107. Register the split API algorithm in the business platform; S2. Using asynchronous event response flow technology based on the observer pattern, freely arrange the algorithms in series and parallel to form a new task according to the business logic; S2 also includes the following sub-steps: S201. Connect all registered API algorithms in series according to business logic to form an algorithm flow; S202, determining the initial algorithm of each algorithm flow; S203, determining input parameters of upstream and downstream algorithms in each algorithm flow; S204, save all algorithm flows to form tasks; S3. Evaluate the registered API algorithm according to the evaluation model; S3 also includes the following sub-steps: S301. Obtain information related to the API algorithm; S302, inputting the information into the evaluation model to calculate the algorithm evaluation index; S303. Determine the API algorithm evaluation result according to the algorithm evaluation index.

2. A meteorological grid algorithm access assessment method according to claim 1, characterized in that: The information includes: the amount of algorithm input data, the amount of algorithm theoretical output data, the amount of algorithm output data, the total algorithm time, the time it takes for the server to obtain data, the time it takes from the end of the algorithm calculation to the user end, the number of CPU cores requested by the algorithm, the amount of memory requested by the algorithm, the number of users who call the algorithm, the number of daily visits, the number of successful algorithm deployments per day, the number of algorithms, and the number of units to which the users who call the algorithm belong; The algorithm evaluation indicators include algorithm single-second data volume indicator, accuracy indicator, algorithm call status indicator, algorithm resource utilization indicator, and algorithm decoupling degree indicator; The evaluation model includes an algorithm single-second data volume model, an algorithm accuracy model, an algorithm call index model, an algorithm resource utilization index model, and a decoupling index model.

3. A meteorological grid algorithm access assessment method according to claim 2, characterized in that: The algorithm single-second data volume model calculates the algorithm single-second data volume indicator based on the algorithm input data volume, algorithm output data volume, algorithm total time consumption, server-side data acquisition time consumption, algorithm calculation end time consumption to user end consumption, algorithm application CPU core number, algorithm application memory volume; The algorithm accuracy model calculates the accuracy index based on the algorithm theory output data volume and the algorithm output data volume; The algorithm call index model calculates the algorithm call status index based on the number of users who call the algorithm, the number of daily visits, and the number of successful daily algorithm deployments; The algorithm resource utilization index model calculates the highest CPU and memory usage within a certain period of time based on the number of daily visits; The decoupling index model calculates the decoupling degree according to the amount of data output by the algorithm and the number of units to which the algorithm calls the user.

4. A meteorological grid algorithm access assessment method according to claim 3, characterized in that: The expression of the single-second data volume model of the algorithm is: ； Where S1 represents the amount of input data to the algorithm, S2 represents the amount of output data from the algorithm, T represents the total time taken by the algorithm, T1 represents the time taken by the server to obtain data, T2 represents the time taken from the end of the algorithm calculation to the end of the user, C represents the number of CPU cores requested by the algorithm, and G represents the amount of memory requested by the algorithm. The expression of the algorithm accuracy model is: ； Among them, S2 represents the amount of algorithm output data recorded by the gateway, and S3 represents the theoretical output data of the algorithm for a single successful call; The expression of the algorithm calling indicator model is: ； ； ； ； ； Among them, E represents the number of successful calls to the algorithm, n represents the number of users who call the algorithm, and E i represents the number of successful calls of each algorithm, excluding the maximum and minimum access times within k days, σ represents the standard deviation of the algorithm call times, represents the average number of successful calls of the algorithm, C j represents the number of successful algorithm visits per day within k-2 days; D j Indicates the number of suspicious visits per day by the algorithm. The statistical rule is the total number of successful calls with the same IP and input within the sliding time period. In each sliding time period, the first successful call is not counted, and the rest are counted as suspicious calls. a Indicates the success rate of algorithm call, C a Indicates the number of daily algorithm calls within k-2 days; The expression of the algorithm resource utilization index model is: ； ； Among them, U_memory represents memory utilization, G i Indicates the actual maximum memory usage of algorithm i, U_cpu indicates the CPU utilization, C i Indicates the actual maximum CPU usage of algorithm i; The expression of the decoupling index model is: ； Where γ represents the Pearson correlation coefficient, V β Indicates the standard deviation of the algorithm output data volume, represents the average amount of data output by the algorithm, m represents the number of units to which the algorithm calls the user, It represents the average number of units, and g represents the number of algorithms.

5. A meteorological grid algorithm access evaluation system, used in the meteorological grid algorithm access evaluation method according to claim 1, characterized in that: include: Algorithm registration module, used to complete the registration of API algorithms according to user instructions; The task orchestration module is used to connect registered API algorithms in series according to business logic and form tasks; The algorithm evaluation module is used to evaluate the registered API algorithms based on the evaluation model.

6. A meteorological grid algorithm access evaluation system according to claim 5, characterized in that: Also includes: Algorithm management module, used to edit API algorithms according to user instructions and generate algorithm logs; The algorithm call module is used for permission management, providing task load, request forwarding, and accessing algorithm logs.

7. A meteorological grid algorithm access assessment device, characterized in that: The method comprises a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to execute the computer program to implement the steps of the method according to claim 1.

Images