Meteorological grid algorithm access evaluation method, system and device

Abstract

The invention belongs to the field of meteorological services, and particularly provides a meteorological grid algorithm access assessment method, system and device, and the method comprises the steps: S1, completing the splitting and registration of an API algorithm according to an algorithm splitting rule; s2, adopting an asynchronous event response flow technology designed based on an observer mode, and freely arranging algorithms in series-parallel connection according to business logic to form a new task; and S3, evaluating the registered API algorithm according to the evaluation model. According to the technology for obtaining the data source of the heterogeneous database based on the visual configuration, the problem that the learning pressure of a user on a novel database is solved, a common user can obtain data without directly connecting the database, and the problem that the normal production performance of a business database is influenced by too many connections during high-concurrency requests is solved; mirror image containerization deployment is adopted, and a server performs automatic capacity expansion and contraction according to long SQL, CPU and memory use conditions of a user, so that the performance of the whole cluster is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of meteorological services, and particularly relates to a method, a system and a device for evaluating the access of meteorological grid algorithms. Background Art

[0002] In order to promote data intensification and algorithm sharing, it is urgent to establish a set of evaluation systems to evaluate the advantages and disadvantages of algorithms. Due to the characteristics of large data volume and high spatio-temporal resolution in meteorological algorithms, the input data sources of algorithms need to be stored locally. How to efficiently obtain local data, how to decouple the algorithm logic from the algorithm input, how to freely combine the split algorithms to form a new algorithm, and how to design an access evaluation method under the condition of different input and output scales are the problems that need to be solved urgently at present. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a method, a system and a device for evaluating the access of meteorological grid algorithms, which can freely connect upstream and downstream algorithms according to business needs to form a new algorithm, and comprehensively and objectively evaluate the algorithm through multi-dimensional indicators.

[0004] The present invention is implemented as follows: A method for evaluating the access of meteorological grid algorithms includes: S1. Complete the splitting and registration of API algorithms according to the algorithm splitting rules; S2. Adopt the asynchronous event response flow technology designed based on the observer pattern, and freely arrange the algorithms in series and parallel according to the business logic to form a new task; S3. Evaluate the registered API algorithms according to the evaluation model.

[0005] Further, the S1 further includes the following sub-steps: S101. Split the meteorological algorithms of different business types according to the preset algorithm granularity; S102. Instantiate the data source according to the information entered in the page configuration; S103. According to the HTTP request sent by the user, splice the fields in the data table in the request URL through the & symbol; S104. The server creates the basic images of the virtual valley connection pool API, the NoSql database connection pool API, and the interface API according to the preset and stores them in the image repository; S105. After the server responds to the instructions of S102 and S103, extract the basic images generated by S104 to generate containers of the scalable virtual valley connection pool API, the NoSql connection pool API, and the interface connection pool API; S106. Automatically connect to the Xugu database or the NoSql database according to the parameters of the Http request in the interface connection pool API container; S107. Register the split API algorithms into the business platform.

[0006] Furthermore, the S2 further includes the following sub-steps: S201. Concatenate all the registered API algorithms according to the business logic to form an algorithm flow; S202. Determine the initial algorithm of each algorithm flow; S203. Determine the input parameters of the upstream and downstream algorithms in each algorithm flow; S204. Save all the algorithm flows to form a task.

[0007] Furthermore, the S3 further includes the following sub-steps: S301. Obtain the information related to the API algorithm; S302. Input the information into the evaluation model to calculate the algorithm evaluation metrics; S303. Determine the evaluation result of the API algorithm according to the algorithm evaluation metrics.

[0008] Furthermore, the information includes: algorithm input data volume, algorithm theoretical output data volume, algorithm output data volume, total algorithm time consumption, server data acquisition time consumption, time consumption from algorithm calculation end to user side, number of CPU cores applied by the algorithm, memory amount applied by the algorithm, number of users calling the algorithm, daily access times, daily successful algorithm invocation times, number of algorithms, number of affiliated units corresponding to the users calling the algorithm; The algorithm evaluation metrics include algorithm single-second data volume metric, accuracy metric, algorithm call situation metric, algorithm resource utilization situation metric, algorithm decoupling degree metric; The evaluation model includes algorithm single-second data volume model, algorithm accuracy model, algorithm call metric model, algorithm resource utilization rate metric model, decoupling degree metric model.

[0009] Furthermore, the algorithm single-second data volume model calculates the algorithm single-second data volume metric according to the algorithm input data volume, algorithm output data volume, total algorithm time consumption, server data acquisition time consumption, time consumption from algorithm calculation end to user side, number of CPU cores applied by the algorithm, and memory amount applied by the algorithm; The algorithm accuracy model calculates the accuracy metric according to the algorithm theoretical output data volume and the algorithm output data volume; The algorithm call metric model calculates the algorithm call situation metric according to the number of users calling the algorithm, daily access times, and daily successful algorithm invocation times; The algorithm resource utilization rate index model statistically calculates the highest utilization rates of CPU and memory within a certain period of time based on the daily access times; The decoupling degree index model calculates the decoupling degree based on the algorithm output data volume and the number of affiliated units corresponding to the algorithm - calling users.

[0010] Furthermore, the expression of the algorithm single - second data volume model is: ; where S 1 represents the algorithm input data volume, S 2 represents the algorithm output data volume, T represents the total algorithm execution time, T 1 represents the time taken by the server to obtain data, T 2 represents the time from the end of algorithm calculation to the user side, C represents the number of CPU cores requested by the algorithm, and G represents the amount of memory requested by the algorithm; The expression of the algorithm accuracy model is: ; where S 2 represents the algorithm output data volume recorded by the gateway, S 3 represents the theoretical output data volume of the algorithm for a single successful call; The expression of the algorithm call index model is: ; ; ; ; ; where E represents the number of successful algorithm calls, n represents the number of users calling the algorithm, E i represents the number of successful calls for each algorithm, the maximum and minimum access times are excluded within k days, σ represents the standard deviation of the algorithm call times, represents the average value of the number of successful algorithm calls, C j represents the number of daily successful algorithm accesses within k - 2 days; D j represents the number of daily suspicious algorithm accesses. The statistical rule is that within a sliding time period, for the same IP, with the same input, and the total number of successful call situations. In each sliding time period, the first successful case is not counted, and the rest are counted as suspicious times; V a represents the algorithm call success rate, C a represents the number of daily algorithm calls within k - 2 days; The expression of the algorithm resource utilization rate index model is: ; ; Among them, U_memory represents the memory utilization rate, and G i represents the highest actual memory usage of algorithm i, U_cpu represents the CPU utilization rate, and C i represents the highest actual CPU usage of algorithm i; The expression of the decoupling degree index model is: ; Among them, γ represents the Pearson correlation coefficient, and V β represents the standard deviation of the algorithm output data volume, represents the average value of the algorithm output data volume, m represents the number of affiliated units corresponding to the users called by the algorithm, represents the average value of the number of affiliated units, and g represents the number of algorithms.

[0011] The present invention also provides a meteorological grid algorithm access evaluation system, including: An algorithm registration module, used to complete the registration of API algorithms according to user instructions; A task scheduling module, used to concatenate the registered API algorithms according to business logic and form tasks; An algorithm evaluation module, used to evaluate the registered API algorithms according to the evaluation model.

[0012] Furthermore, it also includes: An algorithm management module, used to edit API algorithms according to user instructions and generate algorithm logs; An algorithm call module, used for permission management, providing task loads, request forwarding, and accessing algorithm logs.

[0013] The present invention also provides a meteorological grid algorithm access evaluation device, including a processor and a memory. The memory is used to store a computer program, and the processor is used to execute the computer program to implement the steps of the above method.

[0014] The beneficial effects brought by the present invention are: 1. The technology of obtaining heterogeneous database data sources based on visual configuration in the present invention solves the learning pressure of users on new databases. Ordinary users can obtain data without directly connecting to the database, avoiding the problem that too many connection numbers affect the normal production performance of business databases during high-concurrency requests; adopting mirror containerization deployment, the server automatically scales according to the user's long SQL, CPU, and memory usage conditions, which helps to improve the performance of the entire cluster.

[0015] 2. The algorithm evaluation model in the present invention is adapted to the scenario of meteorological high-longitude grid data access evaluation, avoiding the problem of simply counting the number of algorithm calls to feedback algorithm activity, and is conducive to avoiding the problem of repeated calls. Starting from the actual business, the concept of algorithm decoupling degree is introduced, and the correlation coefficient is statistically analyzed according to the size of the algorithm output and the number of user-owned units, which has a certain popularization value. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a flowchart of the method in the present invention.

[0017] Figure 2 It is an operation demonstration diagram of page configuration.

[0018] Figure 3 It is an operation demonstration diagram of algorithm concatenation.

[0019] Figure 4 It is a concatenation demonstration diagram of three API algorithms.

[0020] Figure 5 It is a result schematic diagram of the Kriging interpolation algorithm.

[0021] Figure 6 It is a data interaction flowchart in the present invention.

[0022] Figure 7 It is a flowchart of S106 in the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0023] The present invention will be further described below with reference to the drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and cannot be used to limit the protection scope of the present invention.

[0024] As Figures 1 to 7 shown, this embodiment discloses a meteorological grid algorithm access evaluation method, including steps S1 - S3.

[0025] S1. According to the algorithm splitting rule, complete the splitting and registration of the API algorithm. This step is based on multi-process and multi-thread hybrid programming, as well as block and KD-tree query technologies, and adopts the mirror container method to achieve rapid and elastic data acquisition. Specifically, S1 further includes the following sub-steps: S101. Split meteorological algorithms of different business types according to the preset algorithm granularity. In this step, meteorological algorithms can be divided into different types such as site real-time statistics, grid model processing, radar processing, etc. according to business types. It is advisable that the splitting granularity is such that one or more associated products correspond to one algorithm. Taking the site real-time statistical algorithm as an example, meteorological element values x = {precipitation, temperature, humidity, pressure, visibility, weather phenomenon}, x = 1...6; spatial range y = {province-city-county-township four-level statistics, longitude and latitude range statistics, basin statistics}, y = 1, 2, 3; time range z = {fixed time period (5 / 10 / 15 / 30 / 60... minutes) statistics, sliding time period (every 5 / 10 / 15 / 30 / 60... minutes) statistics}, z = 1, 2. Design the algorithm granularity as follows: ; Among them, each time interval in the fixed time period is designed as 1 algorithm, and the same applies to the sliding time period statistics.

[0026] S102. Enter relevant information according to the page configuration and perform data source instantiation. As Figure 2 shown, based on the page configuration, the user fills in information such as account, password, IP address, data table, interface, etc. to perform data source instantiation, corresponding to the data source instantiation module in Figure 7 .

[0027] S103. As Figure 6 shown, the user conducts meteorological data retrieval. According to the HTTP request sent by the user, the server concatenates the fields in the data table in the request URL through the & symbol and dynamically converts the API into a database SQL.

[0028] S104. The server creates basic images of database connection pool APIs (Xugu connection pool API, NoSql database connection pool API) and interface APIs according to the preset and stores them in the image repository.

[0029] S105. After the server responds to the instructions of S102 and S103, it extracts the basic image generated by S104 and generates scalable containers of Xugu connection pool API, NoSql connection pool API, and interface connection pool API. The connection pool API container scaling rules are as follows: 1) Define each time window as 30 minutes, with 00:00:00-00:30:00 UTC as the first time window, and the remaining time is similar. 2) Take the usage of the number of HTTP requests, container CPU, and container memory in each time window container. 3) Calculate the average value of the sliding 3 time windows. If the single user retrieval time span in the HTTP request exceeds 10 days, the CPU utilization rate is greater than 60%, and the memory utilization rate is greater than 80%, if any of the three conditions are met, expand 1 node; if the single user retrieval time span in the HTTP request does not exceed 24 hours, the CPU utilization rate is less than 20%, and the memory utilization rate is less than 40%, recycle 1 node.

[0030] S106, the parameters of the Http request in the connection pool API container automatically connect to the Xugu database, NoSql database or TianQing API interface, and return the response result. Figure 7 As shown, in this step, the response result is returned in JSON format after data acceleration processing. The data acceleration processing process includes the following steps: S1061. Get the parameters of the Http request, including the grid starting latitude lat 1 , end latitude lat 2 , starting longitude lon 1 ,lon 2 , grid resolution res (decimal, such as 0.01°).

[0031] S1062, calculate the number of grid blocks according to the parameters of the Http request, and divide the grid blocks according to the number of grid blocks. The calculation formula for the number of grid blocks is: ; Among them, part_num represents the number of grid blocks, and step represents the step size, which is generally an integer multiple of the resolution res.

[0032] S1063. Query the grid data and station data according to the grid starting latitude, ending latitude, and starting longitude, and perform data quality control to remove station data with missing stations, illegal characters, and inconsistent time / space. The station data comes from the Xugu database, and the grid data comes from the NoSql database.

[0033] S1064. Integrate the grid data of each quality-controlled grid block with the station data of its corresponding regional area. Using the "multi-process + multi-thread" hybrid programming technique, in the grid data, the grid data of the same latitude is processed by the same process, and different grid blocks of the same latitude are processed by different threads. The calculation formula for the number of processes and threads in the computer memory is: ; ; Among them, process_num represents the number of processes, and threads_num represents the number of threads.

[0034] Within each thread, build a KD tree with the longitude and latitude values of the stations, and use the element values as attribute values to achieve fast nearest neighbor search. Compare with the grid data. For the same geographical location, use the station data as the true value, and use the grid data for the remaining locations.

[0035] S1065. Stitch the data of each grid block after data integration to obtain the data query result. The data query result contains three-dimensional information: longitude, latitude, and elements.

[0036] S107. Register the split API algorithm into the business platform. During this process, information such as the English name of the algorithm, the business system to which the algorithm belongs, the development language, input parameters, output parameters, and startup commands need to be filled in. For the algorithms that pass the review, double instances are automatically created, and the load balancing function is realized.

[0037] It should be noted that in step S1, steps S102 and S103 are completed by the user's operation on the client side, and the remaining steps are completed on the server side.

[0038] S2. Connect the registered API algorithms in series according to the business logic to form a task. This step uses the asynchronous event response stream (RxJava) technology designed based on the observer pattern. This technology can effectively avoid the generation of callback hell. The biggest difference from the conventional standard observer pattern is that it takes the data stream as the core, processes the input, processing, and output of the data, and finally realizes that multiple existing API algorithms are connected in series by dragging to form a new API algorithm. In the same task, the running result of the upstream algorithm does not need to be written to disk and is directly transmitted to the downstream algorithm in the computer memory. The biggest advantage of this solution is to avoid the dependence of the intermediate results of algorithm operation in the same task on the storage medium, effectively improve the convenience of algorithm registration, and reduce the frequent IO interaction between the database / storage disk and the computer memory. Specifically, S2 includes the following sub-steps: S201. Connect all the registered API algorithms in series according to the business logic to form an algorithm stream. Such as Figure 3As shown in the figure, click on the right side of the algorithm box in the operation interface and drag out an arrow pointing to the next API algorithm to be executed, and so on, to complete the concatenation of all algorithms.

[0039] S202. Determine the initial algorithm for each algorithm stream. In this step, the data input of the algorithm needs to be selected. That is, only the first algorithm in the concatenation can select the data source prepared by the algorithm registration module for input, and the input data format is JSON.

[0040] S203. Determine the input parameters of the upstream and downstream algorithms in each algorithm stream. In this step, the input parameters of the algorithm need to be modified. That is, for each algorithm, the parameters can be modified, and the types and quantities of the modifiable parameters are the same as those during registration. The data stream between the upstream and downstream algorithms is transmitted in JSON format. It can be seen that the operation results of the upstream and downstream algorithms in the same task are directly transmitted in the computer memory (RAM), avoiding the dependence on the storage medium.

[0041] S204. Save all algorithm streams to form a task. When saving the algorithm, the name after the concatenation of the algorithms needs to be filled in to form a task. After passing the review, the server instantiates the algorithm workflow by pulling the image repository of the task.

[0042] In step S2, when the algorithms are combined in parallel, a synchronous barrier mechanism is adopted. After the algorithms at the same level are executed, status information is sent to the Kafka message queue. The downstream algorithm can be started only after receiving the status return codes of all upstream algorithms.

[0043] As Figure 4 and Figure 5 shown, taking the temperature color patch map algorithm of a certain province as an example, the data source is obtained through the Tianqing API interface (obtain the hourly temperature data of the Chinese ground by time and administrative region). This algorithm includes the concatenation of three API algorithms: 1) Use the quality control algorithm to perform quality control processing on the obtained temperature data; 2) Use the Kriging interpolation algorithm to interpolate the station data into grid data with a resolution of 5 kilometers; 3) Use the color patch map filling algorithm to perform color patch map filling processing on the grid data. The three algorithms are encapsulated and combined into 1 new algorithm and published as an API interface for other users to use.

[0044] S3. Evaluate the registered API algorithms according to the evaluation model. This step evaluates the operation efficiency, stability, reliability, and business applicability of the algorithms from the aspects of the amount of data per second of the algorithm, accuracy, number of calls, call success rate, algorithm resource utilization rate, and decoupling degree between the algorithm processing logic and the algorithm input. Specifically, S3 also includes the following sub-steps: S301. Obtain information related to the API algorithm. In this step, the information includes: the amount of algorithm input data, the amount of algorithm output data, the total algorithm execution time, the time taken by the server to obtain data, the time from the end of algorithm calculation to the client, the number of CPU cores requested by the algorithm, the amount of memory requested by the algorithm, the number of users invoking the algorithm, the number of daily accesses, the number of successful daily algorithm accesses, the number of suspicious daily accesses, the number of algorithms, and the number of affiliated units corresponding to the users invoking the algorithms.

[0045] S302. Input the information into the evaluation model to calculate the algorithm evaluation metrics. In this step, the algorithm evaluation metrics include the algorithm's data volume per second metric, the accuracy metric, the algorithm invocation situation metric, the algorithm resource utilization situation metric, and the algorithm decoupling degree metric.

[0046] The evaluation model includes an algorithm data volume per second model, an algorithm accuracy model, an algorithm invocation metric model, an algorithm resource utilization rate metric model, and a decoupling degree metric model.

[0047] Specifically, the algorithm data volume per second model calculates the algorithm's data volume per second based on the algorithm input data volume, the algorithm output data volume, the total algorithm execution time, the time taken by the server to obtain data, the time from the end of algorithm calculation to the client, the number of CPU cores requested by the algorithm, and the amount of memory requested by the algorithm. The expression of the algorithm data volume per second model is: ; where S 1 represents the algorithm input data volume, S 2 represents the algorithm output data volume, T represents the total algorithm execution time, T 1 represents the time taken by the server to obtain data, T 2 represents the time from the end of algorithm calculation to the client, C represents the number of CPU cores requested by the algorithm, and G represents the amount of memory requested by the algorithm.

[0048] The algorithm accuracy model calculates based on the theoretically output data volume of the algorithm and the algorithm output data volume S 2 recorded by the gateway. Taking the intelligent grid file of a certain province as an example, the latitude range is 25°N - 32°N, and the longitude range is 117°E - 125°E. If interpolation calculation is performed at 0.01°, finally element values are obtained, and the element values are accurate to 6 decimal places. Then, multiplying the number of element quantities by the spatial size of each data can estimate the theoretically output data volume S 3 of a single successful algorithm invocation. The expression of the algorithm accuracy model is: .

[0049] The algorithm call metric model calculates the algorithm call situation metrics based on the number of users calling the algorithm, the number of daily accesses, the number of successful daily algorithm accesses, and the number of suspicious daily algorithm accesses. The expression of the algorithm call metric model is: ; ; ; ; ; Among them, E represents the number of successful algorithm calls, n represents the number of users calling the algorithm, E i represents the number of successful calls for each algorithm. The maximum and minimum access times are excluded within k days, σ represents the standard deviation of the algorithm call times, represents the average value of the number of successful algorithm calls, C j represents the number of successful daily algorithm accesses within k - 2 days; D j represents the number of suspicious daily algorithm accesses. The statistical rule is that within the sliding time period, for the same IP, with the same input, and the total number of successful call situations. In each sliding time period, the first successful situation is not counted, and the rest are counted as suspicious times; V a represents the algorithm call success rate, C a represents the number of daily algorithm calls within k - 2 days.

[0050] The algorithm resource utilization metric model calculates the highest utilization rates of CPU and memory within a certain period of time based on the number of daily accesses. The expression of the algorithm resource utilization metric model is: ; ; Among them, U _memory represents the memory utilization rate, G i represents the highest actual memory usage of algorithm i, U_cpu represents the CPU utilization rate, C i represents the highest actual CPU usage of algorithm i.

[0051] The decoupling degree metric model calculates the decoupling degree based on the amount of data output by the algorithm and the number of affiliated units corresponding to the algorithm call users. The expression of the decoupling degree metric model is: ; Among them, γ represents the Pearson correlation coefficient, V β represents the standard deviation of the amount of data output by the algorithm, represents the average value of the amount of data output by the algorithm, m represents the number of affiliated units corresponding to the algorithm call users, Represents the average number of affiliated units, and g represents the number of algorithms. In the case of successful invocation, within k days, take the standard deviation V of the algorithm output volume β ; Users call the grid algorithm according to the administrative division or the range of longitude and latitude to generate different numbers of feature points, and the algorithm output volume is also different; V β Is moderately positively correlated with m. The larger the m value, the larger V β Is also larger, indicating that the decoupling degree between the algorithm processing logic and the data input is relatively high.

[0052] S303. Determine the API algorithm evaluation result according to the algorithm evaluation index. This model evaluates the operation efficiency, stability, reliability, and business applicability of the algorithm from five indicators: the amount of data processed per second by the algorithm, the algorithm accuracy, the algorithm call index, the algorithm resource utilization rate, and the algorithm decoupling degree. For the above five indicators, each indicator is scored 20 points, with a full score of 100 points. The scoring calculation process is as follows: a) The scoring rules for evaluating the three indicators of the amount of data processed per second, algorithm accuracy, and algorithm resource utilization rate of g algorithms are as follows: Statistically calculate the maximum value of each indicator of g algorithms, and the algorithm corresponding to the maximum value gets 20 points. For g algorithms, the scores are as follows: ;

[0053] Among them x i Represents the value of each indicator, max ( x i ) Represents the maximum value of this indicator. After rounding, the score is accurate to two decimal places.

[0054] b) The scoring rules for evaluating the algorithm call index of g algorithms are as follows: Take the maximum value of the algorithm call times within k days and the maximum value of the reciprocal of the standard deviation of the algorithm call times, and the product of the two is used as the benchmark. For g algorithms, the scores are as follows: ;

[0055] Among them, Represents the reciprocal of the standard deviation of the call times of each algorithm, max ( E i ) Represents the maximum value of the successful call times of each algorithm, Represents the maximum value of the reciprocal of the standard deviation of the call times of each algorithm. After rounding, the score is accurate to two decimal places.

[0056] c) The scoring rules for evaluating the decoupling degree index of g algorithms are as follows: Take the maximum value of the standard deviation V of the algorithm output volume within k days β And the maximum value of the corresponding number m of affiliated units of the algorithm, and the product of the two is used as the benchmark. For g algorithms, the scores are as follows: ; After rounding, the score is accurate to two decimal places.

[0057] Finally, perform an arithmetic sum of the scores in the above three steps a), b), and c) to obtain the total score. SC = SC 1 + SC 2 + SC 3 , and determine the algorithm recommendation level according to the total score as shown in the following table: Algorithm recommended level Score range Related description 1 [90,100] The algorithm runs stably, has high concurrency, low coupling, and is worth recommending 2 [80,90） The algorithm runs relatively stably, has high resource utilization rate, and is generally recommended 3 [70,80） The stability of the algorithm needs to be improved, and the processing ability is average. It can be tried for application 4 [60,70） The algorithm wastes resources seriously and needs to be further improved 5 [0,60） The algorithm is taken off the shelf for processing

[0058] Based on the same inventive concept, the present invention also provides a meteorological grid algorithm access evaluation system, including an algorithm registration module, a task scheduling module, an algorithm evaluation module, an algorithm management module, and an algorithm call module.

[0059] The algorithm registration module is used to complete the registration of the API algorithm according to the user instruction. Specifically, this module executes the following sub-steps: S101. Split the meteorological algorithms of different business types according to the preset algorithm granularity.

[0060] S102. Enter relevant information according to the page configuration for data source instantiation.

[0061] S103. According to the HTTP request sent by the user, splice the fields in the data table in the request URL through the & symbol.

[0062] S104. The server creates the basic images of the virtual valley connection pool API, the NoSql database connection pool API, and the interface API according to the preset and stores them in the image repository.

[0063] S105. After the server responds to the instructions of S102 and S103, extract the basic images generated in S104 to generate containers of the scalable virtual valley connection pool API, the NoSql connection pool API, and the interface connection pool API.

[0064] S106. Automatically connect to the virtual valley database or the NoSql database according to the parameters of the Http request in the interface connection pool API container.

[0065] S107. Register the split API algorithm into the business platform.

[0066] The task scheduling module is used to concatenate the registered API algorithms according to the business logic to form tasks. Specifically, this module executes the following sub-steps: S201. Concatenate all the registered API algorithms according to the business logic to form an algorithm flow.

[0067] S202. Determine the initial algorithm for each algorithm flow.

[0068] S203. Determine the input parameters of the upstream and downstream algorithms in each algorithm flow.

[0069] S204. Save all algorithm flows to form a task.

[0070] The algorithm evaluation module is used to evaluate the registered API algorithms according to the evaluation model. Specifically, this module executes the following sub-steps: S301. Obtain the information related to the API algorithm.

[0071] S302. Input the information into the evaluation model to calculate the algorithm evaluation metrics.

[0072] S303. Determine the API algorithm evaluation result according to the algorithm evaluation metrics.

[0073] The algorithm management module is used to edit the API algorithm according to the user instruction and generate an algorithm log. Specifically, this module supports online modification of algorithm information such as startup parameters, algorithm name, algorithm description, algorithm code, algorithm version number, etc. When the algorithm is upgraded, the downstream algorithms are notified in the form of an in-site message. After the algorithm is upgraded, it does not affect the existing old version algorithm services. The algorithm log printing is developed using the Java language, and the Java language is used to obtain the Kafka system metrics, and the Kafka method of obtaining the key is used to implement the acquisition and analysis of the log.

[0074] The algorithm call module is used for permission management, providing task load, request forwarding, and accessing the algorithm log. Specifically, this module includes a service gateway, load balancing, service routing, user authentication, behavior recording, etc. Among them, the interface gateway is responsible for unified permission control, the load balancing is responsible for providing the task load capacity during high traffic, the service routing is responsible for accurately and efficiently forwarding the request to different service nodes, and the behavior recording is responsible for providing the interface access log record to facilitate tracing the usage of the service interface of the data.

[0075] Based on the same inventive concept, the present invention also provides a meteorological grid algorithm admission evaluation device, including a processor and a memory. The memory is used to store a computer program, and the processor is used to execute the computer program to implement the steps of the above method.

[0076] The above is only the preferred embodiment of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the technical principle of the present invention, several improvements and deformations can still be made, and these improvements and deformations should also be regarded as the protection scope of the present invention.

Claims

1. A meteorological grid algorithm access assessment method, characterized in that: The steps include: S1. Complete the splitting and registration of API algorithms according to the algorithm splitting rules; S2, using the asynchronous event response flow technology designed based on the observer pattern, freely arranging the algorithms in series and parallel to form a new task according to the business logic; S3. Evaluate the registered API algorithm according to the evaluation model.

2. A meteorological grid algorithm access assessment method according to claim 1, characterized in that: The S1 further comprises the following sub-steps: S101, splitting meteorological algorithms of different business types according to preset algorithm granularity; S102, input relevant information according to page configuration and instantiate the data source; S103, according to the HTTP request sent by the user, the fields in the data table are concatenated into the request URL through the & symbol; S104, the server creates the basic image of Xugu connection pool API, NoSql database connection pool API, and interface API according to the preset, and stores it in the image warehouse; S105. After the server responds to the instructions of S102 and S103, it extracts the basic image generated by S104 and generates a container of scalable Xugu connection pool API, NoSql connection pool API, and interface connection pool API; S106, automatically connect to the Xugu database or NoSql database according to the parameters of the Http request in the interface connection pool API container; S107. Register the split API algorithm in the business platform.

3. A meteorological grid algorithm access assessment method according to claim 1, characterized in that: The S2 further comprises the following sub-steps: S201, connect all registered API algorithms in series according to business logic to form an algorithm flow; S202, determining an initial algorithm for each algorithm flow; S203, determining input parameters of upstream and downstream algorithms in each algorithm flow; S204. Save all algorithm flows to form tasks.

4. A meteorological grid algorithm access assessment method according to claim 1, characterized in that: The S3 further comprises the following sub-steps: S301, obtaining information related to the API algorithm; S302, inputting the information into the evaluation model to calculate the algorithm evaluation index; S303. Determine the API algorithm evaluation result according to the algorithm evaluation index.

5. A meteorological grid algorithm access assessment method according to claim 4, characterized in that: The information includes: the amount of data input to the algorithm, the amount of data output from the algorithm in theory, the amount of data output from the algorithm, the total time consumed by the algorithm, the time consumed by the server to obtain data, the time consumed from the end of the algorithm calculation to the end of the user, the number of CPU cores requested by the algorithm, the amount of memory requested by the algorithm, the number of users who call the algorithm, the number of visits per day, the number of successful algorithm transfers per day, the number of algorithms, and the number of units to which the algorithm-calling users belong; The algorithm evaluation indicators include algorithm single-second data volume indicator, accuracy indicator, algorithm call situation indicator, algorithm resource utilization indicator, and algorithm decoupling degree indicator; The evaluation model includes an algorithm single-second data volume model, an algorithm accuracy model, an algorithm call index model, an algorithm resource utilization index model, and a decoupling index model.

6. A meteorological grid algorithm access assessment method according to claim 5, characterized in that: The algorithm single-second data volume model calculates the algorithm single-second data volume index based on the algorithm input data volume, algorithm output data volume, algorithm total time consumption, server end data acquisition time consumption, algorithm calculation end to user end time consumption, algorithm application CPU core number, algorithm application memory volume; The algorithm accuracy model calculates the accuracy index based on the algorithm theoretical output data volume and the algorithm output data volume; The algorithm call index model calculates the algorithm call status index based on the number of users who call the algorithm, the number of daily visits, and the number of successful daily algorithm dispatches; The algorithm resource utilization index model counts the highest CPU and memory utilization within a certain period of time based on the number of daily visits; The decoupling index model calculates the decoupling degree according to the amount of data output by the algorithm and the number of units to which the algorithm calls the user.

7. A meteorological grid algorithm access assessment method according to claim 6, characterized in that: The expression of the single-second data volume model of the algorithm is: ； Among them, S1 represents the amount of data input to the algorithm, S2 represents the amount of data output by the algorithm, T represents the total time taken by the algorithm, T1 represents the time taken by the server to obtain data, T2 represents the time taken from the end of the algorithm calculation to the end of the user, C represents the number of CPU cores requested by the algorithm, and G represents the amount of memory requested by the algorithm; The expression of the algorithm accuracy model is: ； Among them, S2 represents the amount of algorithm output data recorded by the gateway, and S3 represents the theoretical output data of the algorithm for a single successful call; The expression of the algorithm calling indicator model is: ； ； ； ； ； Where E represents the number of successful algorithm calls, n represents the number of users who call the algorithm, and E i represents the number of successful calls of each algorithm, excluding the maximum and minimum access times within k days, σ represents the standard deviation of the algorithm call times, represents the average number of successful calls to the algorithm, C j represents the number of successful algorithm visits per day within k-2 days; D j Indicates the number of suspicious visits to the algorithm every day. The statistical rule is the total number of successful calls with the same IP and the same input within the sliding time period. In each sliding time period, the first successful call is not counted, and the rest are counted as suspicious calls. a Indicates the success rate of algorithm call, C a Indicates the number of daily algorithm calls within k-2 days; The expression of the algorithm resource utilization index model is: ； ； Among them, U_memory represents memory utilization, G i represents the actual maximum memory usage of algorithm i, U_cpu represents the CPU utilization, C i Indicates the actual maximum CPU usage of algorithm i; The expression of the decoupling index model is: ； Among them, γ represents the Pearson correlation coefficient, V β Represents the standard deviation of the algorithm output data volume, represents the average amount of data output by the algorithm, m represents the number of units to which the algorithm calls the user. represents the average number of units, and g represents the number of algorithms.

8. A meteorological grid algorithm access assessment system, characterized in that: include: Algorithm registration module, used to complete the registration of API algorithms according to user instructions; The task scheduling module is used to connect the registered API algorithms in series according to the business logic and form tasks; The algorithm evaluation module is used to evaluate the registered API algorithms according to the evaluation model.

9. A meteorological grid algorithm access assessment system according to claim 8, characterized in that: Also includes: The algorithm management module is used to edit the API algorithm according to user instructions and generate algorithm logs; The algorithm calling module is used for permission management, providing task load, request forwarding, and accessing algorithm logs.

10. A meteorological grid algorithm access assessment device, characterized in that: The method comprises a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to execute the computer program to implement the steps of the method according to claim 1.

Images