Model interface field checking method and device, computer device, and storage medium

Abstract

The application discloses a model interface field verification method and device, computer equipment and a storage medium. The method comprises the following steps: synchronizing the interface called field information in the interface call history data record table to a big data warehouse by using a data synchronization tool, and writing the running data into the corresponding online landing table by using a scheduling system to run the synchronization data; storing the offline data taking SQL script corresponding to the model interface in the scheduling system, calling the scheduling interface of the scheduling system to run the offline data taking SQL script, and writing the running data into the corresponding offline landing table; associating the online landing table and the offline landing table according to the interface basic information table and the interface parameter table, splicing the model interface field information, generating splicing data verification SQL, running the splicing data verification SQL by using the scheduling interface of the scheduling system, and obtaining the verification result of the model interface field. The application reduces the inconvenience of manually obtaining online and offline data, and greatly saves the labor cost.

Description

Technical Field

[0001] This application relates to the field of model verification technology, and in particular to a method, apparatus, computer device and storage medium for verifying model interface fields. Background Technology

[0002] The development of emerging technologies such as cloud computing, big data, artificial intelligence, and blockchain has brought opportunities for the reshaping and restructuring of value chains in various sectors of the financial industry. From the perspective of practical application of machine learning methods, the commonly used algorithm model application architecture typically involves packaging the algorithm model as an API (Application Programming Interface), and obtaining the model's prediction results in real time by calling the API. The drawbacks of this application architecture are: it requires significant computing resources to ensure the timeliness of predictions; and because API interface parameters come from different systems and involve field calculations, current technologies cannot monitor these parameters, thus compromising the accuracy of the interface fields. Furthermore, in related technologies, testing the accuracy of interface fields mainly relies on traditional manual verification, which places a heavy workload on verifiers, is inefficient, and is highly repetitive and mechanical. Summary of the Invention

[0003] This application provides a method, apparatus, computer device, and storage medium for verifying model interface fields, aiming to solve the technical problems that existing interface field accuracy testing mainly relies on manual verification, which is labor-intensive and inefficient.

[0004] To solve the above-mentioned technical problems, the technical solution adopted in this application is as follows:

[0005] A method for validating model interface fields, comprising:

[0006] On a preset platform, a basic information table for the interface, a historical data record table for interface calls, and an interface parameter table are set up respectively. The basic information table is used to record the basic field information of the interface on the preset platform, the historical data record table for interface calls is used to record the field information of the model interface being called, and the interface parameter table is used to record the parameter field information of the model interface.

[0007] The interface call history data record table is synchronized to the big data warehouse using a data synchronization tool, and the running data is written to the corresponding online landing table using a scheduling system.

[0008] The offline data retrieval SQL script corresponding to the model interface is stored in the scheduling system. The scheduling interface of the scheduling system is called to run the offline data retrieval SQL script, and the running data is written to the corresponding offline landing table.

[0009] The online and offline landing tables are associated based on the interface basic information table and the interface parameter table, and the model interface field information that needs to be verified is obtained. The model interface field information is concatenated to generate concatenated data verification SQL. The concatenated data verification SQL is run using the scheduling interface of the scheduling system to obtain the verification result of the model interface field.

[0010] The technical solution adopted in this application embodiment further includes: before setting the interface basic information table, interface call history data record table, and interface parameter table of the model interface on the preset platform, it also includes:

[0011] The model interface fields are divided into two types: time-series fields (time_series) and non-time-series fields (no_time_series). The time-series fields (time_series) are non-fixed fields that change over time, while the non-time-series fields (no_time_series) are fixed fields that do not change over time.

[0012] The technical solution adopted in this application embodiment further includes: the interface basic information table records interface basic field information including interface ID, interface name, N value, online landing table name tableA corresponding to the interface, offline data retrieval script name, and offline landing table name tableB, where N value is the number of days in advance for verification data; the interface call history data record table records interface call field information including interface ID, caller ID, input parameter data, and interface call time; the interface parameter table records interface parameter field information including interface ID, parameter field, field name (col1, col2...), field type (time_serials / no_time_serials), verification flag (True, false), field error range (range1, range2...), and primary key field list col_ids = [col1, col2...]; wherein, the primary key field list includes the association conditions between the online landing table and the offline landing table; the field with the verification flag set to True is the field that needs to be verified; the non-time-series field no_time_serial... The field error range for es is 0.

[0013] The technical solution adopted in this application embodiment further includes: specifically, calling the scheduling interface of the scheduling system to run the offline data retrieval SQL script and writing the running data into the corresponding offline landing table.

[0014] A first interface and a second interface for running SQL are added to the scheduling system.

[0015] The second interface of the scheduling system is called, and the corresponding offline data retrieval SQL script is run according to the offline data retrieval script name, and the running data is written to the corresponding offline landing table.

[0016] The technical solution adopted in this application embodiment further includes: associating the online landing table and the offline landing table according to the interface basic information table and the interface parameter table, obtaining the model interface field information that needs to be verified, and concatenating the model interface field information to generate concatenated data verification SQL, specifically as follows:

[0017] The data verification algorithm of the preset platform is used to parse the interface basic information table and interface parameter table to obtain the online landing table name tab l eA, the offline landing table name tab l eB and the N value corresponding to the model interface.

[0018] The online landing table (tabletA) and the offline landing table (tabletB) are associated, and the fields with the validation flag set to True, as well as the field name, field type, and field error range of the fields, are obtained.

[0019] The field name, field type, and field error range of the fields whose verification flag is True are concatenated to generate concatenated data verification SQL.

[0020] The technical solution adopted in this application embodiment further includes: associating the online landing table and the offline landing table according to the interface basic information table and the interface parameter table, obtaining the model interface field information that needs to be verified, and concatenating the model interface field information to generate concatenated data verification SQL, which further includes:

[0021] The interface basic information table is parsed to obtain the online landing table name tableA, the offline landing table name tableB, and the N value corresponding to the model interface.

[0022] The interface parameter table is parsed to obtain the fields in the model interface whose validation flag is True, and the field name (col1, col2...), field type (time_series / no_time_series), and field error range (range1, range2...) of the field are obtained. The field name, field type, and field error range of the field are combined into cols = [{'col_name':'col1','range':range1,'type':'time_series'}....];

[0023] Obtain the primary key field list col_ids = [col1, col2, ...] from the interface parameter table;

[0024] The code 'col_name':'col1','range':range1,'type':'time_series'}....' is parsed, and the accuracy of each field is verified according to preset conditions. The result is then translated into an SQL field query statement. The preset conditions are: whether the absolute value of the field in the online and offline landing tables is less than the error value.

[0025] The primary key field list coll_ids is parsed to obtain the association conditions between the online landing table and the offline landing table. The online landing table and the offline landing table are associated according to the association conditions. The online landing table name tab_eA, the offline landing table name tab_eB, and the N value are replaced in the corresponding positions in the SQL field query statement to generate the final concatenated data verification SQL.

[0026] The technical solution adopted in this application embodiment further includes: the step of running the concatenated data verification SQL using the scheduling interface of the scheduling system specifically involves: running the concatenated data verification SQL using the first interface; after obtaining the verification result of the model interface field, the solution further includes:

[0027] Create a validation result record table in the big data warehouse, and insert the validation results of the model interface fields into the validation result record table;

[0028] The verification result record table is displayed through BI reports.

[0029] Another technical solution adopted in this application embodiment is: a model interface field validation device, comprising:

[0030] Interface setting module: used to set the interface basic information table, interface call history data record table and interface parameter table of the model interface on the preset platform respectively. The interface basic information table is used to record the interface basic field information on the preset platform, the interface call history data record table is used to record the field information of the model interface being called, and the interface parameter table is used to record the parameter field information of the model interface.

[0031] The first data execution module is used to use a data synchronization tool to synchronize the interface call history data record table to the big data warehouse, and to use a scheduling system to run the synchronized data and write the execution data to the corresponding online landing table.

[0032] The second data execution module stores the offline data retrieval SQL script corresponding to the model interface in the scheduling system, calls the scheduling interface of the scheduling system to run the offline data retrieval SQL script, and writes the running data into the corresponding offline landing table.

[0033] Field validation module: It is used to associate the online landing table and the offline landing table according to the interface basic information table and the interface parameter table, obtain the model interface field information that needs to be validated, concatenate the model interface field information to generate concatenated data validation SQL, run the concatenated data validation SQL using the scheduling interface of the scheduling system, and obtain the validation result of the model interface field.

[0034] Another technical solution adopted in this application embodiment is: a computer device, the computer device comprising:

[0035] A memory that stores executable program instructions;

[0036] The processor connected to the memory;

[0037] The processor calls the executable program instructions stored in the memory to execute the model interface field verification method as described above.

[0038] Another technical solution adopted in this application embodiment is: a storage medium storing processor-executable program instructions, the program instructions being used to execute the above-mentioned model interface field verification method.

[0039] The model interface field verification method, apparatus, computer equipment, and storage medium of this application break down data barriers between different systems by storing online and offline data in a scheduling system, reducing the inconvenience of manually acquiring online and offline data, and greatly saving labor costs. This invention adds a model interface field verification function to a preset platform and runs tasks using a scheduling interface, transforming the repetitive and mechanical data verification process that relies on traditional manual testing into automated data verification, which can quickly verify the accuracy of data in the interface and improve the testing efficiency of model interface fields. Attached Figure Description

[0040] Figure 1 This is a flowchart illustrating the model interface field validation method of the first embodiment of this application;

[0041] Figure 2 This is a flowchart illustrating the model interface field validation method of the second embodiment of this application;

[0042] Figure 3 This is a schematic diagram of the structure of the model interface field validation device in an embodiment of this application;

[0043] Figure 4 This is a schematic diagram of the computer device structure according to an embodiment of this application;

[0044] Figure 5 This is a schematic diagram of the storage medium structure according to an embodiment of this application. Detailed Implementation

[0045] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0046] The terms "first," "second," and "third" in this application are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first," "second," or "third" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified. All directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of this application are only used to explain the relative positional relationships and movements between components in a specific orientation (as shown in the figures). If the specific orientation changes, the directional indications also change accordingly. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0047] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0048] Please see Figure 1 This is a flowchart illustrating the model interface field validation method of the first embodiment of this application. The model interface field validation method of the first embodiment of this application includes the following steps:

[0049] S100: Set up a basic information table, a historical data record table, and an interface parameter table for the model interface on a preset platform. The basic information table is used to record basic field information of the interface on the preset platform. The historical data record table is used to record field information of the model interface being called. The interface parameter table is used to record parameter field information of the model interface.

[0050] S110: Use a data synchronization tool to synchronize the interface call field information in the interface call history data record table to the big data warehouse, and use the scheduling system to run the synchronized data and write the running data into the corresponding online landing table;

[0051] S120: Store the offline data retrieval SQL script corresponding to the model interface in the scheduling system, call the scheduling interface of the scheduling system to run the offline data retrieval SQL script, and write the running data into the corresponding offline landing table;

[0052] S130: Based on the interface basic information table and the interface parameter table, associate the online landing table and the offline landing table, obtain the model interface field information that needs to be verified, concatenate the model interface field information to generate concatenated data verification SQL, run the concatenated data verification SQL using the scheduling interface of the scheduling system, and obtain the verification result of the model interface field.

[0053] Please see Figure 2 This is a flowchart illustrating the model interface field validation method according to the second embodiment of this application. The model interface field validation method according to the second embodiment of this application includes the following steps:

[0054] S200: Divide the model interface fields into two types: time-series fields and non-time-series fields;

[0055] In this step, the time-series field `time_serial` refers to a non-fixed field that changes over time, such as the API call time. The non-time-series field `no_time_serial` refers to a fixed field that does not change over time, such as API ID and other API parameter fields. For non-time-series fields, the error range is set to 0. For time-series fields, a certain error range can be set according to the actual application scenario to make the automated validation results of non-time-series fields more consistent with reality.

[0056] S210: Set up an interface basic information table, an interface call history data record table, and an interface parameter table on the preset platform respectively. Use the interface basic information table to record the interface basic field information on the preset platform, use the interface call history data record table to record the field information of the model interface being called, and use the interface parameter table to record the parameter field information of the model interface.

[0057] In this step, the interface basic information table records basic interface fields including, but not limited to, interface ID, interface name, parameter validation status, scheduled validation, N value, online landing table name (tab_eA) corresponding to the model interface, offline data retrieval script name, and offline landing table name (tab_eB). The N value is the number of days in advance for data validation. For example, validating data T-1-N, where T represents the current date, means validating data N+1 days prior to the current date. The specific N value can be set for different model interfaces. The interface call history data record table records interface call fields including, but not limited to, interface ID, caller ID, input parameter data (JSON), prediction result, and interface call time. The interface parameter table records interface parameter field information including, but not limited to, interface ID, parameter field, field name (col1, col2...), field type (time_serials / no_time_serials), validation flag (True, false), field error range (range1, range2...), and primary key field list col_ids = [col1, col2...]. The primary key field list col_ids = [col1, col2...] includes the association conditions between the online and offline landing tables. Fields with a validation flag of True are the fields that need to be validated. The field error range for non-time-series fields is 0, while the field error range for time-series fields can be set according to the actual application scenario of each field.

[0058] S220: Use a data synchronization tool to periodically synchronize the newly added interface call field information in the interface call history data record table to the big data warehouse, and record the synchronized data as Task1;

[0059] In this step, the data synchronization tool is DataX, and the synchronized interface is called with fields including runtime log data.

[0060] S230: Create the first partition table in the big data warehouse, start Task1 on a schedule, write the synchronized data of day T-1 (i.e. the previous day) into the first partition table in an incremental manner, and record the partition as T-1. Then use the scheduling system to run Task1 on a schedule and write the running data into the corresponding online landing table.

[0061] S240: Set up a first interface and a second interface for running SQL in the scheduling system of the preset platform, and store the offline data retrieval SQL script corresponding to the model in the scheduling system;

[0062] In this step, the first interface supports running SQL scripts, and the second interface supports running SQL files under the scheduling system through parameters. Since different models have different parameters and different parameter acquisition logic, the corresponding offline SQL data retrieval scripts also have certain differences. This embodiment of the application pre-stores both online and offline data (i.e., the interface call field information in the interface call history record table and the offline SQL data retrieval script) in the scheduling system, breaking down the data barriers between different systems. This eliminates the need to manually retrieve offline SQL data retrieval scripts, greatly saving manpower costs.

[0063] Based on the above, in traditional model interface field validation, it is necessary to manually obtain online and offline data from big data warehouses and preset platforms for validation, which consumes a lot of manpower and has low validation efficiency. The embodiments of this application break down the data barriers between different platforms by storing online and offline data in the scheduling system, which is conducive to improving the validation efficiency of model interface fields.

[0064] S250: Call the second interface of the scheduling system, use the second interface to run the corresponding offline data retrieval SQL script according to the offline data retrieval script name, and write the running data into the corresponding offline landing table, and record the partition as T-1;

[0065] In this step, the offline data retrieval SQL script is run using a scheduling interface, thereby transforming the repetitive and mechanical data verification process that relies on manual testing into automated data verification. This improves testing efficiency and enables rapid verification of the accuracy of model interface fields.

[0066] S260: Use the preset platform's data verification algorithm to parse the interface basic information table and interface parameter table, obtain the online landing table name tableA, the offline landing table name tableB, and the N value corresponding to the model interface, associate the online landing table and the offline landing table based on the online landing table name tableA and the offline landing table name tableB, and obtain the fields with the verification flag True, as well as the field name, field type, and field error range. Concatenate the field name, field type, and field error range of the fields with the verification flag True to generate concatenated data verification SQL.

[0067] In this step, the process of generating the concatenated data validation SQL is as follows:

[0068] S261: Parse the interface basic information table to obtain the online landing table name tab_leA, the offline landing table name tab_leB, and the N value corresponding to the model interface;

[0069] S262: Parse the interface parameter table, obtain all fields in the model interface whose validation flag is True, obtain the field name (col1, col2...), field type (time_series / no_time_series), and field error range (range1, range2...) of all fields, and combine the field name, field type, and field error range of each field into cols = [{'col_name':'col1','range':range1,'type':'time_series'}....];

[0070] S263: Retrieve the primary key field list from the interface parameter table: col_ids = [col1, col2, ...].

[0071] S264: Parse the column = [{'col_name':'col1','range':range1,'type':'time_series'}....], verify the accuracy of each field according to preset conditions, and translate it into an SQL field query statement. The preset conditions are: whether the absolute value of each field in the online and offline landing tables is less than the error value. Then, parse the primary key field list column_ids to obtain the association conditions between the online and offline landing tables. Finally, associate the online and offline landing tables according to the association conditions, and replace the online landing table name tab_eA, the offline landing table name tab_eB, and the value of N in the corresponding positions in the SQL field query statement to generate the final concatenated data verification SQL. The specific implementation process is as follows:

[0072]

[0073] As mentioned above, due to the latency of offline data, real-time verification of model interface fields is difficult to achieve. Existing technologies for model interface field verification all verify the correctness of data from N days ago. However, since the latency of different business data varies, a uniform N value cannot meet flexible business needs. Therefore, this application embodiment analyzes from the perspective of actual business data and sets different N values ​​for different model interfaces, thereby enabling more flexible verification of model interface fields from TN days ago. Furthermore, this application embodiment divides model interface fields into two types: time-series fields and non-time-series fields. A certain error range is set for non-time-series fields, making the verification results more consistent with the actual situation.

[0074] S270: And use the first interface to run the concatenated data verification SQL to obtain the model interface field verification results, create a verification result record table in the big data warehouse, and insert the model interface field verification results into the verification result record table, and record the partition as T-1;

[0075] S280: Display the verification result record table through BI reports;

[0076] In this step, the BI report displays content including but not limited to the number of errors and error rate for each model interface field, as well as detailed data. This detailed data can be filtered according to set filtering conditions. Additionally, this embodiment can also set an alert function in the BI report. When the error rate of a model interface field exceeds a set percentage (this embodiment sets this percentage to 20%, but the specific percentage can be set according to the actual application scenario), an alert notification will be sent via telephone or SMS.

[0077] Based on the above, the model interface field verification method of the second embodiment of this application breaks down data barriers between different systems by storing online and offline data in a scheduling system, reducing the inconvenience of manually acquiring online and offline data and greatly saving manpower costs. This invention adds a model interface field verification algorithm to a preset platform and runs tasks using a scheduling interface, transforming the repetitive and mechanical data verification process that relies on traditional manual testing into automated data verification. This enables rapid verification of the accuracy of data in the interface and improves the testing efficiency of model interface fields. This embodiment of the application can perform model interface field verification on different model interfaces in a scheduled or unscheduled manner. For newly launched model interfaces, the latest progress of data repair can be quickly understood; for stable model interfaces, data inspection can be performed, reducing the abuse of computing resources, ensuring the correctness and stability of data operation, and improving the predictive accuracy of the model interface.

[0078] In an optional implementation, the result of the model interface field validation method can also be uploaded to the blockchain.

[0079] Specifically, the corresponding digest information is obtained based on the result of the model interface field verification method. More specifically, the digest information is obtained by hashing the result of the model interface field verification method, for example, using the SHA256s algorithm. Uploading the digest information to the blockchain ensures its security and fairness and transparency to users. Users can download the digest information from the blockchain to verify whether the result of the model interface field verification method has been tampered with. The blockchain referred to in this example is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. A blockchain is essentially a decentralized database, a chain of data blocks linked using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include a blockchain underlying platform, a platform product service layer, and an application service layer.

[0080] Please see Figure 3 This is a schematic diagram of the structure of the model interface field verification device according to an embodiment of this application. The model interface field verification device 40 according to an embodiment of this application includes:

[0081] Interface setting module 41: used to set the interface basic information table, interface call history data record table and interface parameter table of the model interface on the preset platform respectively. The interface basic information table is used to record the interface basic field information on the preset platform, the interface call history data record table is used to record the field information of the model interface being called, and the interface parameter table is used to record the parameter field information of the model interface.

[0082] First data execution module 42: Used to use a data synchronization tool to synchronize the interface call field information in the interface call history data record table to the big data warehouse, and use a scheduling system to run the synchronized data and write the running data into the corresponding online landing table;

[0083] Second data execution module 43: Stores the offline data retrieval SQL script corresponding to the model interface in the scheduling system, calls the scheduling interface of the scheduling system to run the offline data retrieval SQL script, and writes the running data into the corresponding offline landing table;

[0084] Field validation module 44: It is used to associate the online landing table and the offline landing table according to the interface basic information table and the interface parameter table, obtain the model interface field information to be validated, concatenate the model interface field information to generate concatenated data validation SQL, run the concatenated data validation SQL using the scheduling interface of the scheduling system, and obtain the validation result of the model interface field.

[0085] The model interface field verification device of this application breaks down the data barriers between different systems by storing online and offline data in the scheduling system, reducing the inconvenience of manually acquiring online and offline data and greatly saving manpower costs. The present invention adds a model interface field verification function to the preset platform and runs tasks by using the scheduling interface, transforming the repetitive and mechanical data verification process that relies on traditional manual testing into automated data verification, which can quickly verify the accuracy of data in the interface and improve the testing efficiency of model interface fields.

[0086] Please see Figure 4 This is a schematic diagram of a computer device structure according to an embodiment of this application. The computer device 50 includes:

[0087] Memory 51 storing executable program instructions;

[0088] Processor 52 connected to memory 51;

[0089] The processor 52 is used to call the executable program instructions stored in the memory 51 and perform the following steps: On a preset platform, an interface basic information table, an interface call history data record table, and an interface parameter table are set up respectively. The interface basic information table is used to record basic interface field information on the preset platform. The interface call history data record table is used to record the called field information of the model interface. The interface parameter table is used to record the model interface parameter field information. A data synchronization tool is used to synchronize the called field information of the interface in the interface call history data record table to a big data warehouse, and a scheduling system is used to run the synchronized data, writing the running data to the corresponding online landing table. The offline data retrieval SQL script corresponding to the model interface is stored in the scheduling system. The scheduling interface of the scheduling system is called to run the offline data retrieval SQL script, and the running data is written to the corresponding offline landing table. The online landing table and the offline landing table are associated according to the interface basic information table and the interface parameter table, and the model interface field information that needs to be verified is obtained. The model interface field information is concatenated to generate a concatenated data verification SQL. The scheduling interface of the scheduling system is used to run the concatenated data verification SQL to obtain the verification result of the model interface field.

[0090] The processor 52 can also be referred to as a CPU (Central Processing Unit). The processor 52 may be an integrated circuit chip with signal processing capabilities. The processor 52 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor.

[0091] The computer device in this application embodiment breaks down the data barriers between different systems by storing online and offline data in the scheduling system, reducing the inconvenience of manually acquiring online and offline data and greatly saving manpower costs. The present invention adds a model interface field verification function to the preset platform and runs tasks by using the scheduling interface, transforming the repetitive and mechanical data verification process that relies on traditional manual testing into automated data verification, which can quickly verify the accuracy of data in the interface and improve the testing efficiency of model interface fields.

[0092] Please see Figure 5 , Figure 5This is a schematic diagram of the storage medium structure according to an embodiment of this application. The storage medium of this embodiment stores program instructions 61 capable of implementing the following steps: Setting up an interface basic information table, an interface call history data record table, and an interface parameter table on a preset platform. The interface basic information table records basic interface field information on the preset platform; the interface call history data record table records called field information of the model interface; and the interface parameter table records model interface parameter field information. Using a data synchronization tool, the called field information of the interface in the interface call history data record table is synchronized to a big data warehouse, and the synchronized data is run using a scheduling system, writing the running data to the corresponding online landing table. The offline data retrieval SQL script corresponding to the model interface is stored in the scheduling system; the scheduling interface of the scheduling system is called to run the offline data retrieval SQL script, and the running data is written to the corresponding offline landing table. The online landing table and the offline landing table are associated according to the interface basic information table and the interface parameter table, and the model interface field information to be verified is obtained. The model interface field information is concatenated to generate a concatenated data verification SQL, and the concatenated data verification SQL is run using the scheduling interface of the scheduling system to obtain the verification result of the model interface field. The program instructions 61 can be stored in the aforementioned storage medium in the form of a software product. These instructions include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program instructions, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, or computer devices such as computers, servers, mobile phones, and tablets. The server can be a standalone server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0093] The storage medium in this application embodiment breaks down the data barriers between different systems by storing online and offline data in the scheduling system, reducing the inconvenience of manually acquiring online and offline data and greatly saving manpower costs. The present invention adds a model interface field verification function to the preset platform and runs tasks by using the scheduling interface, transforming the repetitive and mechanical data verification process that relies on traditional manual testing into automated data verification, which can quickly verify the accuracy of data in the interface and improve the testing efficiency of model interface fields.

[0094] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, apparatuses, or units, and may be electrical, mechanical, or other forms.

[0095] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units described above can be implemented in hardware or as software functional units. The above are merely embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made based on the description and drawings of this application, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for validating model interface fields, characterized in that, include: On a preset platform, a basic information table for the interface, a historical data record table for interface calls, and an interface parameter table are set up respectively. The basic information table for the interface is used to record the basic field information of the interface on the preset platform, the historical data record table for the interface calls is used to record the field information of the model interface calls, and the interface parameter table is used to record the parameter field information of the model interface. The interface call history data record table is synchronized to the big data warehouse using a data synchronization tool, and the running data is written to the corresponding online landing table using a scheduling system. The offline data retrieval SQL script corresponding to the model interface is stored in the scheduling system. The scheduling interface of the scheduling system is called to run the offline data retrieval SQL script, and the running data is written to the corresponding offline landing table. The online and offline landing tables are associated based on the interface basic information table and the interface parameter table, and the model interface field information that needs to be verified is obtained. The model interface field information is concatenated to generate concatenated data verification SQL. The concatenated data verification SQL is run using the scheduling interface of the scheduling system to obtain the verification result of the model interface field. The interface basic information table records the interface basic field information including interface ID, interface name, N value, online landing table name tableA corresponding to the interface, offline data retrieval script name, and offline landing table name tableB, where N value is the number of days in advance for data verification; the interface call history data record table records the interface call field information including interface ID, caller ID, input parameter data, and interface call time; The interface parameter table records interface parameter field information including interface ID, parameter field, field name, field type, verification flag, field error range, and primary key field list col_ids=[col1,col2..]; wherein, the primary key field list includes the association conditions between the online landing table and the offline landing table; fields with the verification flag set to True are fields that need to be verified; the error range of the non-time-series field no_time_series is 0.

2. The model interface field validation method according to claim 1, characterized in that, Before setting the interface basic information table, interface call history data record table, and interface parameter table for the model interface on the preset platform, the following steps are also included: The model interface fields are divided into two types: time-series fields and no-time-series fields. The time-series fields are non-fixed fields that change over time, while the no-time-series fields are fixed fields that do not change over time.

3. The model interface field validation method according to claim 1, characterized in that, The specific steps of calling the scheduling interface of the scheduling system to run the offline data retrieval SQL script and writing the running data to the corresponding offline landing table are as follows: A first interface and a second interface for running SQL are added to the scheduling system. The second interface of the scheduling system is called, and the corresponding offline data retrieval SQL script is run according to the offline data retrieval script name, and the running data is written to the corresponding offline landing table.

4. The model interface field validation method according to claim 3, characterized in that, The step of associating the online and offline landing tables based on the interface basic information table and the interface parameter table, obtaining the model interface field information that needs to be verified, and concatenating the model interface field information to generate concatenated data verification SQL is as follows: The data verification algorithm of the preset platform is used to parse the interface basic information table and interface parameter table to obtain the online landing table name tableA, the offline landing table name tableB, and the N value corresponding to the model interface; The online landing table (tableA) and the offline landing table (tableB) are associated, and the fields with the validation flag set to True, as well as the field name, field type, and field error range of the fields, are obtained. The field name, field type, and field error range of the fields whose verification flag is True are concatenated to generate concatenated data verification SQL.

5. The model interface field validation method according to claim 4, characterized in that, The step of associating the online and offline landing tables based on the interface basic information table and the interface parameter table, obtaining the model interface field information to be verified, and concatenating the model interface field information to generate concatenated data verification SQL also includes: The interface basic information table is parsed to obtain the online landing table name tableA, the offline landing table name tableB, and the N value corresponding to the model interface; The interface parameter table is parsed to obtain the fields in the model interface whose validation flag is True, and the field name, field type, and field error range of the field are obtained. The field name, field type, and field error range of the field are combined into cols=[{'col_name':'col1', 'range':range1, 'type':'time_series'}....]; Obtain the primary key field list col_ids=[col1,col2..] from the interface parameter table; The function `cols=[{'col_name':'col1', 'range':range1, 'type':'time_series'}....]` is parsed, and the accuracy of each field is verified according to preset conditions. The result is then translated into an SQL field query statement. The preset conditions are: whether the absolute value of the field in the online and offline landing tables is less than the error value. The primary key field list col_ids is parsed to obtain the association conditions between the online landing table and the offline landing table. The online landing table and the offline landing table are associated according to the association conditions. The online landing table name tableA, the offline landing table name tableB, and the value N are replaced in the corresponding positions in the SQL field query statement to generate the final concatenated data verification SQL.

6. The model interface field validation method according to any one of claims 3 to 5, characterized in that, The specific steps of running the concatenated data verification SQL using the scheduling interface of the scheduling system are: running the concatenated data verification SQL using the first interface; After obtaining the validation results of the model interface fields, the process also includes: Create a validation result record table in the big data warehouse, and insert the validation results of the model interface fields into the validation result record table; The verification result record table is displayed through BI reports.

7. A model interface field validation device, characterized in that, include: Interface setting module: used to set the interface basic information table, interface call history data record table and interface parameter table of the model interface on the preset platform respectively. The interface basic information table is used to record the interface basic field information on the preset platform, the interface call history data record table is used to record the field information of the model interface being called, and the interface parameter table is used to record the parameter field information of the model interface. The first data execution module is used to use a data synchronization tool to synchronize the interface call history data record table to the big data warehouse, and to use a scheduling system to run the synchronized data and write the execution data to the corresponding online landing table. The second data execution module stores the offline data retrieval SQL script corresponding to the model interface in the scheduling system, calls the scheduling interface of the scheduling system to run the offline data retrieval SQL script, and writes the running data into the corresponding offline landing table. Field validation module: It is used to associate the online landing table and the offline landing table according to the interface basic information table and the interface parameter table, obtain the model interface field information that needs to be validated, concatenate the model interface field information to generate concatenated data validation SQL, run the concatenated data validation SQL using the scheduling interface of the scheduling system, and obtain the validation result of the model interface field. The interface basic information table records the interface basic field information including interface ID, interface name, N value, online landing table name tableA corresponding to the interface, offline data retrieval script name, and offline landing table name tableB, where N value is the number of days in advance for data verification; the interface call history data record table records the interface call field information including interface ID, caller ID, input parameter data, and interface call time; The interface parameter table records interface parameter field information including interface ID, parameter field, field name, field type, verification flag, field error range, and primary key field list col_ids=[col1,col2..]; wherein, the primary key field list includes the association conditions between the online landing table and the offline landing table; fields with the verification flag set to True are fields that need to be verified; the error range of the non-time-series field no_time_series is 0.

8. A computer device, characterized in that, The computer device includes: A memory that stores executable program instructions; The processor connected to the memory; The processor calls the executable program instructions stored in the memory to execute the model interface field verification method as described in any one of claims 1-6.

9. A storage medium storing processor-executable program instructions, characterized in that, The program instructions are used to execute the model interface field verification method according to any one of claims 1 to 6.

Images