Method, device, equipment and medium for monitoring stability of online financial system models

Abstract

The present application is applicable to the field of financial services technology, and in particular relates to a method, device, equipment and medium for monitoring the stability of an online model of a financial system. The method extracts data type features from offline tag data to obtain target data characterized as a target floating-point number type, and based on at least one preset field type parameter, sequentially calls the same packaged sharding logic to shard the target data to obtain sharding fields of the corresponding field types, calculates the first data volume corresponding to each field type in the current cycle, and combines the second data volume corresponding to each field type in the cycle before the current cycle to calculate the first stability index between the two cycles. When the first stability index is less than the stability threshold, an abnormal alarm is generated. Based on the same packaged sharding logic, data sharding can be performed for different field types for subsequent data statistics, thereby helping to improve the efficiency of PSI calculation.

Description

Technical Field

[0001] The present application is applicable to the field of financial services technology, and in particular relates to a method, apparatus, device, and medium for monitoring the stability of an online model of a financial system. Background Art

[0002] In order to better serve customers, the financial industry, especially the insurance and banking industries, provides services to customers through corresponding service systems, and configures some intelligent models in the corresponding service systems to realize functions such as user recommendation and data calculation. Existing service systems use online models to provide customers with real-time model functions. The stability of online models is the most important indicator for model consideration, and the stability of indicators is an important basis for the importance of model stability. The Population Stability Index (PSI) is used to determine the stability of the model. It can measure the distribution difference between the test sample and the modeling sample scores. It is an important indicator for measuring feature stability. Therefore, real-time monitoring of PSI is an important measure to ensure the stable operation of the online model, ensuring that data anomalies can be discovered as soon as possible and repaired as promptly as possible.

[0003] Currently, the PSI monitoring method for online models relies on Python's random sampling method. However, Python cannot be deployed in a distributed manner and tends to use the resources of a single machine in a cluster. Random sampling is also prone to omissions, making it less effective when monitoring PSI for the entire data set in actual production environments. Using a computing engine (for example, the Spark computing engine) to monitor PSI for the entire data set can effectively solve the problem of full data monitoring. However, this computing engine requires the development of corresponding function packages, which must be constantly changed during use according to actual needs. This process is relatively cumbersome and results in low monitoring efficiency. Therefore, how to use a unified processing logic to implement PSI calculation and monitoring to improve monitoring efficiency has become an urgent problem to be solved. Summary of the Invention

[0004] In view of this, the embodiments of the present application provide a method, apparatus, device and medium for monitoring the stability of an online model of a financial system to solve the problem of how to use a unified processing logic to realize the calculation and monitoring of PSI to improve the monitoring efficiency.

[0005] In a first aspect, an embodiment of the present application provides a method for monitoring the stability of an online model of a financial system, the method comprising:

[0006] Before the database of the online model imports the offline tag data acquired in the current cycle, data type feature extraction is performed on the offline tag data to obtain N target data characterized as a target floating point number type, where N is an integer greater than zero;

[0007] For any target data, based on at least one preset field type parameter, the same encapsulated sharding logic is called in sequence to shard the target data respectively to obtain sharding fields of the corresponding field type;

[0008] Traversing all target data, obtaining shard fields corresponding to all field types in the current cycle, and calculating a first data volume corresponding to each field type in the current cycle based on the shard fields corresponding to all field types in the current cycle;

[0009] Obtaining a second data volume corresponding to each field type in a period previous to the current period, and for any field type, calculating a first stability index between the two periods based on the first data volume of the current period and the second data volume of the previous period;

[0010] It is detected whether the first stability index is less than a stability threshold. If it is detected that the first stability index is less than the stability threshold, an abnormality alarm prompt is generated to prompt the corresponding user to check the abnormality.

[0011] In a second aspect, an embodiment of the present application provides a stability monitoring device for an online model of a financial system, the stability monitoring device comprising:

[0012] a data acquisition module, configured to extract data type features of the offline tag data acquired in the current cycle before the database of the online model is imported, to obtain N target data characterized as target floating-point number types, where N is an integer greater than zero;

[0013] The data sharding module is used to call the same encapsulated sharding logic in sequence for any target data based on at least one preset field type parameter, shard the target data respectively, and obtain sharding fields of corresponding field types;

[0014] a data statistics module, configured to traverse all target data, obtain shard fields corresponding to all field types in the current period, and calculate a first data volume corresponding to each field type in the current period based on the shard fields corresponding to all field types in the current period;

[0015] a first indicator calculation module, configured to obtain a second data volume corresponding to each field type in a period preceding the current period, and for any field type, calculate a first stability indicator between the two periods based on the first data volume of the current period and the second data volume of the previous period;

[0016] The first stability monitoring module is configured to detect whether the first stability index is less than a stability threshold, and if it is detected that the first stability index is less than the stability threshold, generate an abnormality alarm prompt to prompt the corresponding user to check the abnormality.

[0017] In a third aspect, an embodiment of the present application provides a computer device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the stability monitoring method as described in the first aspect when executing the computer program.

[0018] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the stability monitoring method as described in the first aspect is implemented.

[0019] Compared with the prior art, the embodiments of the present application have the following beneficial effects: before the offline label data obtained in the current cycle is imported into the database of the online model, the present application extracts the data type feature of the offline label data to obtain N target data characterized as the target floating-point type. For any target data, based on at least one preset field type parameter, the same encapsulated sharding logic is called in sequence to shard the target data respectively to obtain sharding fields of corresponding field types, traverse all target data to obtain sharding fields corresponding to all field types in the current cycle, and calculate each sharding field in the current cycle based on the sharding fields corresponding to all field types in the current cycle. The first data volume corresponding to the field type is obtained, and the second data volume corresponding to each field type in the period before the current period is obtained. For any field type, the first stability index between the two periods is calculated based on the first data volume of the current period and the second data volume of the previous period. It is detected whether the first stability index is less than the stability threshold. If it is detected that the first stability index is less than the stability threshold, an abnormal alarm prompt is generated to prompt the corresponding user to check the abnormality. Based on the same encapsulated sharding logic, data sharding can be performed for different field types for subsequent data statistics, which helps to improve the efficiency of PSI calculation and make monitoring more efficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0021] Figure 1This is a schematic diagram of an application environment for a method for monitoring the stability of an online model of a financial system provided in Example 1 of the present application;

[0022] Figure 2 This is a flow chart of a method for monitoring the stability of an online model of a financial system provided in Example 2 of the present application;

[0023] Figure 3 This is a flow chart of a method for monitoring the stability of an online model of a financial system provided in Example 3 of the present application;

[0024] Figure 4 This is a flow chart of a method for monitoring the stability of an online model of a financial system provided in Example 4 of the present application;

[0025] Figure 5 This is a flowchart of a method for monitoring the stability of an online model of a financial system provided in Example 5 of the present application;

[0026] Figure 6 This is a flowchart of a method for monitoring the stability of an online model of a financial system provided in Example 6 of the present application;

[0027] Figure 7 This is a flow chart of a method for monitoring the stability of an online model of a financial system provided in Example 7 of the present application;

[0028] Figure 8 This is a schematic structural diagram of a device for monitoring the stability of an online model of a financial system provided in Example 8 of the present application;

[0029] Figure 9 This is a structural diagram of a computer device provided in Example 9 of the present application. DETAILED DESCRIPTION

[0030] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0031] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0032] It will also be understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0033] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0034] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0035] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0036] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to achieve optimal results.

[0037] Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0038] It should be understood that the size of the serial numbers of the steps in the following embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0039] In order to illustrate the technical solution of the present application, specific embodiments are provided below.

[0040] The first embodiment of the present application provides a method for monitoring the stability of an online model of a financial system, which can be applied in the following situations: Figure 1 In an application environment, a client communicates with a server. Clients include, but are not limited to, PDAs, desktop computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, cloud computing devices, and personal digital assistants (PDAs). The server can be implemented as a standalone server or a server cluster consisting of multiple servers.

[0041] See also Figure 2 , is a flow chart of a method for monitoring the stability of an online model of a financial system provided in Example 2 of the present application. The method for monitoring the stability of an online model of a financial system is applied to Figure 1 The server in the financial system carries the financial system and the corresponding online model. In addition, the data basis for the operation of the online model is a stored database, which is also carried by the server. The client (i.e., the front end) is used by the user. The client connects to the server to call the above-mentioned online model to the server and obtain the results of the model feedback. Of course, the server can obtain data generated by non-server operations, that is, data collected by other devices after they are out of the supervision of the server (i.e., offline data), and will add offline data to the above-mentioned database in a timely manner to realize the support of offline data for the operation of the online model. Figure 2 As shown, the stability monitoring method of the online financial system model may include the following steps:

[0042] Step S201 : before the online model database imports the offline tag data acquired in the current cycle, data type feature extraction is performed on the offline tag data to obtain N target data characterized as target floating point number types.

[0043] In this application, the current period is a preset time period. This time period can be set based on the frequency of use of the online model. If the frequency of use is high, the time period can be set to a shorter time period to enable timely updates to the online model database. For example, the time period can be one day, one week, or even one month. The corresponding current period is the time period before the current time point. The current time point is a certain moment today. If the period is one day, the current period can refer to the previous day, or between the corresponding moment of the previous day and the corresponding moment of today.

[0044] Offline label data can refer to data represented as labels in offline data. Labels can refer to indicators represented as fixed meanings or names. The offline label data can be specifically represented by data values. Since the data contains multiple data types, including 4 integer types (byte, short, int, long), 2 floating-point types (float, double), 1 character type "char", and 1 Boolean type "boolean", and the online model based on this application needs to process data of floating-point type, it is necessary to identify the data type of the offline label data in order to find floating-point type data.

[0045] Depending on the accuracy requirements of the online model, the target floating-point number type can be a single-precision floating-point number or a double-precision floating-point number. Ultimately, at least one target data is obtained from the offline tag data, that is, N is an integer greater than zero. If there is no target data, there is no need to perform PSI monitoring subsequently.

[0046] Step S202: for any target data, based on at least one preset field type parameter, call the same encapsulated sharding logic in sequence to shard the target data respectively to obtain sharding fields of corresponding field types.

[0047] In this application, since a target data may be composed of sub-data of multiple fields, and the field types of each sub-data corresponding to the field may be different, for example, the field types may include time fields, numeric fields, character fields, and binary fields, etc., a target data may be composed of sub-data corresponding to two fields: time fields and numeric fields.

[0048] Each field type parameter corresponds to a field type. For any field type, a packaged sharding logic is called based on the corresponding field type parameter to implement sharding of the target data and obtain the corresponding sub-data.

[0049] Sub-data representing the same field type are grouped into a sub-data set. This sub-data set is also the shard field corresponding to that field type. As can be seen, a shard field includes at least one sub-data. The purpose of sharding is to separate data of different field types for subsequent data volume calculation and statistics.

[0050] The encapsulated sharding logic is a universal sharding logic. A field type parameter can be written into the sharding logic, so that the sharding logic can slice out the sub-data of the corresponding field type in the target data. This eliminates the need to write corresponding sharding logic for each field type, reduces code redundancy, makes the implementation process more concise, and helps improve efficiency.

[0051] Step S203: traverse all target data to obtain shard fields corresponding to all field types in the current cycle, and calculate the first data volume corresponding to each field type in the current cycle based on the shard fields corresponding to all field types in the current cycle.

[0052] In this application, all target data are processed through the above-mentioned step S202, and finally the shard fields corresponding to each field type in each target data are obtained. For any field type, all shard fields under the field type are grouped together as all shard fields under the field type in the current period, and all shard fields under all field types in the current period can be obtained.

[0053] The data volume of the shard field is calculated to obtain the data volume size of the corresponding shard field. Subsequently, the data volumes of all shard fields under a field type are added together to obtain the sum of the data volumes and record it as the first data volume.

[0054] Step S204: Obtain the second data volume corresponding to each field type in the previous cycle of the current cycle. For any field type, calculate the first stability index between the two cycles based on the first data volume of the current cycle and the second data volume of the previous cycle.

[0055] The previous cycle is the cycle before the current cycle. The corresponding second data volume and other information for the data from the previous week have already been recorded during the last monitoring. For any field type, the corresponding first data volume and second data volume are compared to obtain a stability assessment value corresponding to that field type. The stability assessment values ​​for all field types are then added together to obtain the sum, which is the first stability index.

[0056] The first stability index is the PIS value, which can measure the distribution difference between the test sample and the modeling sample scores and is used to determine the stability of the model.

[0057] Step S205 , detecting whether the first stability index is less than a stability threshold. If it is detected that the first stability index is less than the stability threshold, generating an abnormality alarm prompt to prompt the corresponding user to check the abnormality.

[0058] In this application, if the stability index is high, it can be said that the data has high stability for the online model and will not affect the operation of the online model. If the stability index is low, it can be said that the data has a certain impact on the operation of the online model.

[0059] The stability threshold may be a value obtained based on demand or long-term experimental verification. If the value is higher than the stability threshold, it indicates that the data does not affect the operation of the online model. If the value is lower than the stability threshold, it indicates that the data affects the operation of the online model.

[0060] When it is detected that the first stability index is less than the stability threshold, that is, the first stability index is low, an abnormal alarm prompt is generated to prompt the user to check the corresponding abnormal situation and solve the abnormal problem, thereby ensuring the stable operation of the online model.

[0061] In an embodiment of the present application, before the offline tag data acquired in the current cycle is imported into the online model database, data type features are extracted from the offline tag data to obtain N target data characterized as target floating-point number types. For any target data, based on at least one preset field type parameter, the same encapsulated sharding logic is sequentially called to shard the target data to obtain sharding fields of the corresponding field type. All target data are traversed to obtain sharding fields corresponding to all field types in the current cycle. Based on the sharding fields corresponding to all field types in the current cycle, a first data volume corresponding to each field type in the current cycle is calculated. A second data volume corresponding to each field type in the cycle before the current cycle is obtained. For any field type, a first stability index between the two cycles is calculated based on the first data volume of the current cycle and the second data volume of the previous cycle. The first stability index is then detected to determine whether it is less than a stability threshold. If the first stability index is less than the stability threshold, an abnormality alarm is generated to prompt the corresponding user to check for the abnormality. Based on the same encapsulated sharding logic, data sharding can be performed for different field types for subsequent data statistics, thereby improving the efficiency of PSI calculation and enhancing monitoring efficiency.

[0062] See also Figure 3 , is a flow chart of a method for monitoring the stability of an online model of a financial system provided in Example 3 of this application. Figure 3 As shown, the stability monitoring method may include the following steps based on the above step S203:

[0063] Step S301: construct a first temporary Hive table, and write the shard fields corresponding to all field types in the current cycle into the first temporary Hive table.

[0064] Hive is a Hadoop-based data warehouse tool used for data extraction, transformation, and loading. It provides a mechanism for storing, querying, and analyzing large amounts of data stored in Hadoop. The Hive data warehouse tool maps structured data files into a database table and provides SQL query functionality, converting SQL statements into MapReduce tasks for execution. Hive tables can store each of the aforementioned shard fields and enable fast MapReduce statistics using SQL-like statements, making MapReduce much simpler without the need to develop dedicated MapReduce applications.

[0065] Step S302 : In the first temporary Hive table, based on the sum command of the Hive table, statistics are collected on the data volume of the shard fields of each field type to obtain a first data volume of the corresponding field type.

[0066] Among them, after the shard fields are stored in the table, the data volume of each shard field can be obtained through the corresponding command, and the data volume of all shard fields can be summed using the corresponding sum command, so as to quickly and accurately count the first data volume.

[0067] During this process, data volume statistics can be performed on all field types in parallel without the need for other complex statistical methods. This can reduce code complexity to a certain extent, improve statistical efficiency, and thus increase the timeliness of monitoring.

[0068] See also Figure 4 , is a flow chart of a method for monitoring the stability of an online model of a financial system provided in Example 4 of this application. Figure 4 As shown, the stability monitoring method may include the following steps based on the above step S202:

[0069] Step S401: construct a second temporary Hive table, and write the first data volume of each field type in the current cycle and the second data volume of each field type in the previous cycle into the second temporary Hive table respectively.

[0070] Step S402 : In the second temporary Hive table, based on the PSI calculation command of the Hive table, a first stability index between two periods is calculated.

[0071] Among them, the above-mentioned steps S401 and S402 have the same contents as steps S301 and S302, and the descriptions of steps S301 and S302 may be referred to, and will not be repeated here.

[0072] Optionally, the calculation formula corresponding to the PSI calculation command is as follows:

[0073] PSI1=SUM((X1-X2)*ln(X1 / X2))

[0074] Among them, PSI1 represents the first stability index, for any field type, X1 represents the first data volume of the field type, X2 represents the second data volume of the field type, and SUM represents the sum operation of all field types.

[0075] Among them, for the combination of the above-mentioned embodiment three and embodiment four, the PSI calculation command is then used in the second temporary Hive table. The PSI calculation command is a value calculated by comparing the data volume and the change distribution of the compared data volume, and this value is the stability index.

[0076] See also Figure 5 , is a flowchart of a method for monitoring the stability of an online model of a financial system provided in Example 5 of the present application, such as Figure 5 As shown, the stability monitoring method of the online financial system model may include the following steps:

[0077] Step S501 : before the online model database imports the offline tag data acquired in the current cycle, data type feature extraction is performed on the offline tag data to obtain N target data characterized as target floating point number types.

[0078] Step S502: for any target data, based on at least one preset field type parameter, call the same encapsulated sharding logic in sequence to shard the target data respectively to obtain sharding fields of corresponding field types.

[0079] Step S503: traverse all target data to obtain shard fields corresponding to all field types in the current cycle, and calculate the first data volume corresponding to each field type in the current cycle based on the shard fields corresponding to all field types in the current cycle.

[0080] Step S504: Obtain the second data volume corresponding to each field type in the previous cycle of the current cycle. For any field type, calculate the first stability index between the two cycles based on the first data volume of the current cycle and the second data volume of the previous cycle.

[0081] The contents of steps S501 to S504 are partially identical to the contents of steps S201 to S204 described above, and reference may be made to the description of steps S201 to S204 , which will not be repeated here.

[0082] Step S505 , detecting whether the first stability index is less than a stability threshold; if it is detected that the first stability index is not less than the stability threshold, obtaining a third data volume corresponding to each field type within a historical target period.

[0083] If the first stability index is not less than the stability threshold, it indicates that the first stability index is high. However, the first stability index is based on the data volume between the current cycle and the previous cycle. Considering that the data in the previous cycle may contain anomalies, it is necessary to additionally monitor the data volume between the current cycle and a target cycle. This target cycle is generally a period without anomalies. For example, the target cycle can be the data at the end of the previous month to reduce the impact of errors.

[0084] Step S506 , calculating a second stability index between the two cycles based on the first data volume of the current cycle and the third data volume of the target cycle.

[0085] Step S507 , detecting whether the second stability index is less than a stability threshold. If it is detected that the second stability index is less than the stability threshold, an abnormality alarm is generated.

[0086] The calculation method of the stability index is the same as that of the above-mentioned third and fourth embodiments, and will not be repeated here.

[0087] Before the offline tag data acquired in the current cycle is imported into the database of the online model, this embodiment performs data type feature extraction on the offline tag data to obtain N target data characterized as target floating-point number types. For any target data, based on at least one preset field type parameter, the same packaged sharding logic is called in sequence to shard the target data respectively to obtain sharding fields of corresponding field types. All target data are traversed to obtain sharding fields corresponding to all field types in the current cycle. Based on the sharding fields corresponding to all field types in the current cycle, the first data volume corresponding to each field type in the current cycle is calculated. The second data volume corresponding to each field type in the cycle before the current cycle is obtained. For any field type, the first data volume of the current cycle and the second data volume of the previous cycle are calculated. The first stability indicator between the two periods is used to detect whether the first stability indicator is less than the stability threshold. If it is detected that the first stability indicator is not less than the stability threshold, the third data volume corresponding to each field type in a historical target period is obtained. According to the first data volume of the current period and the third data volume of the target period, the second stability indicator between the two periods is calculated to detect whether the second stability indicator is less than the stability threshold. If it is detected that the second stability indicator is less than the stability threshold, an abnormal alarm is generated. Based on the same encapsulated sharding logic, data sharding can be performed for different field types for subsequent data statistics, which helps to improve the efficiency of PSI calculation, makes monitoring more efficient, and can effectively reduce the impact of errors, thereby realizing more accurate monitoring to ensure the stability of the online model.

[0088] See also Figure 6, is a flow chart of a method for monitoring the stability of an online model of a financial system provided in Example 6 of this application. Figure 6 As shown, the stability monitoring method may include the following steps based on the above step S505:

[0089] Step S601: construct a third temporary Hive table, and write the first data volume of each field type in the current cycle and the second data volume of each field type in the target cycle into the third temporary Hive table respectively.

[0090] Step S602: In the third temporary Hive table, based on the PSI calculation command of the Hive table, a second stability index between two periods is calculated.

[0091] The contents of the above steps S601 and S602 may refer to the descriptions of the above steps S301 and S302 and may be processed in a similar manner.

[0092] See also Figure 7 , is a flow chart of a method for monitoring the stability of an online model of a financial system provided in Example 7 of this application. Figure 7 As shown, the stability monitoring method may include the following steps based on the above step S205:

[0093] Step S701: If it is detected that the first stability index is less than the stability threshold, then the import of offline tag data into the database of the online model is stopped.

[0094] Among them, if the stability index is small, it is necessary to stop importing offline label data into the online model database to prevent the online model from using the data and avoid abnormal operation of the online model.

[0095] Step S702: construct an abnormality warning file based on the offline tag data, and send the abnormality warning file to the target address in the form of an email to form an abnormality warning prompt.

[0096] Based on the offline tag data, an abnormality alert file is constructed, specifically including the tag name, data content, and generation time. The abnormality alert file is then emailed to the target address to notify the user, thereby providing an abnormality alert.

[0097] Corresponding to the stability monitoring method of the online model of the financial system in the above embodiment, Figure 8 The structural block diagram of the stability monitoring device of the financial system online model provided by the eighth embodiment of the present application is shown. The stability monitoring device is applied to Figure 1The server in the financial system carries the financial system and the corresponding online model. In addition, the data basis for the operation of the online model is a stored database, which is also carried by the server. The client (i.e., the front end) is used by the user. The client connects to the server to call the above-mentioned online model to the server and obtain the results of the model feedback. Of course, the server can obtain data generated by non-server operations, that is, data collected by other devices after they are out of the supervision of the server (i.e., offline data), and will add offline data to the above-mentioned database in a timely manner to realize the support of offline data for the operation of the online model. For ease of explanation, only the parts related to the embodiments of the present application are shown.

[0098] See also Figure 8 , the stability monitoring device comprises:

[0099] The data acquisition module 81 is used to extract data type features of the offline tag data acquired in the current cycle before importing the offline tag data into the online model database to obtain N target data represented by the target floating point type, where N is an integer greater than zero;

[0100] The data sharding module 82 is used to sequentially call the same encapsulated sharding logic for any target data based on at least one preset field type parameter to shard the target data and obtain sharding fields of the corresponding field type;

[0101] The data statistics module 83 is configured to traverse all target data, obtain shard fields corresponding to all field types in the current period, and calculate the first data volume corresponding to each field type in the current period based on the shard fields corresponding to all field types in the current period;

[0102] A first indicator calculation module 84 is configured to obtain the second data volume corresponding to each field type in the period preceding the current period, and for any field type, calculate a first stability indicator between the two periods based on the first data volume of the current period and the second data volume of the previous period;

[0103] The first stability monitoring module 85 is configured to detect whether the first stability index is less than a stability threshold. If it is detected that the first stability index is less than the stability threshold, an abnormality alarm is generated to prompt the corresponding user to check for abnormalities.

[0104] Optionally, the data statistics module 83 includes:

[0105] A first table construction unit is configured to construct a first temporary Hive table and write shard fields corresponding to all field types in the current cycle into the first temporary Hive table;

[0106] The first data volume calculation unit is configured to count the data volume of the shard field of each field type in the first temporary Hive table based on the sum command of the Hive table to obtain a first data volume of the corresponding field type.

[0107] Optionally, the first indicator calculation module 84 includes:

[0108] A second table construction unit is used to construct a second temporary Hive table, and write the first data volume of each field type in the current cycle and the second data volume of each field type in the previous cycle into the second temporary Hive table respectively;

[0109] The first indicator calculation unit is configured to calculate, in the second temporary Hive table, a first stability indicator between two periods based on a PSI calculation command in the Hive table.

[0110] Optionally, the calculation formula corresponding to the PSI calculation command is as follows:

[0111] PSI1=SUM((X1-X2)*ln(X1 / X2))

[0112] Among them, PSI1 represents the first stability index, for any field type, X1 represents the first data volume of the field type, X2 represents the second data volume of the field type, and SUM represents the sum operation of all field types.

[0113] Optionally, the stability monitoring device further includes:

[0114] a historical data volume acquisition module, configured to, after detecting whether the first stability index is less than a stability threshold, acquire a third data volume corresponding to each field type within a historical target period if it is detected that the first stability index is not less than the stability threshold;

[0115] A second indicator calculation module is used to calculate a second stability indicator between the two periods based on the first data volume of the current period and the third data volume of the target period;

[0116] The second stability monitoring module is used to detect whether the second stability index is less than the stability threshold, and generate an abnormal alarm prompt if it is detected that the second stability index is less than the stability threshold.

[0117] Optionally, the second indicator calculation module includes:

[0118] A third table construction unit is configured to construct a third temporary Hive table, and write the first data volume of each field type in the current cycle and the second data volume of each field type in the target cycle into the third temporary Hive table respectively;

[0119] The second indicator calculation unit is configured to calculate, in a third temporary Hive table, a second stability indicator between two periods based on a PSI calculation command of the Hive table.

[0120] Optionally, the first stability monitoring module 85 includes:

[0121] a stopping import unit, configured to stop importing the offline label data into the online model database if it is detected that the first stability index is less than a stability threshold;

[0122] The alarm generation unit is used to construct an abnormal alarm file based on the offline tag data, and send the abnormal alarm file to the target address in the form of an email to form an abnormal alarm prompt.

[0123] It should be noted that the information interaction, execution process and other contents between the above modules are based on the same concept as the method embodiment of this application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0124] Figure 9 This is a schematic diagram of the structure of a computer device provided in Example 9 of this application. Figure 9 As shown, the computer device of this embodiment includes: at least one processor ( Figure 9 Only one is shown), a memory, and a computer program stored in the memory and executable on at least one processor, which implements the steps of any of the above-mentioned embodiments of the method for monitoring the stability of an online model of a financial system when executing the computer program.

[0125] The computer device may include, but is not limited to, a processor and a memory. It will be understood by those skilled in the art that Figure 9 The above is merely an example of a computer device and does not constitute a limitation on the computer device. The computer device may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include a network interface, a display screen, and an input device.

[0126] The processor may be a CPU, or other general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. A general-purpose processor may be a microprocessor, or any conventional processor.

[0127] The memory includes a readable storage medium, an internal memory, etc., wherein the internal memory can be the memory of a computer device, and the internal memory provides an environment for the operation of the operating system and computer-readable instructions in the readable storage medium. The readable storage medium can be the hard disk of the computer device, and in other embodiments, it can also be an external storage device of the computer device, for example, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device. Furthermore, the memory can also include both the internal storage unit of the computer device and the external storage device. The memory is used to store the operating system, application programs, boot loaders (BootLoader), data, and other programs, such as the program code of the computer program. The memory can also be used to temporarily store data that has been output or is about to be output.

[0128] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned device can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, it can implement the steps of the above-mentioned method embodiment. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include at least: any entity or device capable of carrying computer program code, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0129] The present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed through a computer program product. When the computer program product runs on a computer device, the computer device can implement the steps in the above-mentioned method embodiment when executing it.

[0130] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0131] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0132] In the embodiments provided in this application, it should be understood that the disclosed apparatus / computer equipment and methods can be implemented in other ways. For example, the apparatus / computer equipment embodiments described above are merely schematic. For example, the division of modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of the apparatus or unit, which can be electrical, mechanical or other forms.

[0133] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0134] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A method for monitoring the stability of an online model of a financial system, characterized in that: The stability monitoring method comprises: Before the database of the online model imports the offline tag data acquired in the current cycle, data type feature extraction is performed on the offline tag data to obtain N target data characterized as a target floating point number type, where N is an integer greater than zero; For any target data, based on at least one preset field type parameter, the same encapsulated sharding logic is called in sequence to shard the target data respectively to obtain sharding fields of the corresponding field type; Traversing all target data, obtaining shard fields corresponding to all field types in the current cycle, and calculating a first data volume corresponding to each field type in the current cycle based on the shard fields corresponding to all field types in the current cycle; Obtaining a second data volume corresponding to each field type in a period previous to the current period, and for any field type, calculating a first stability index between the two periods based on the first data volume of the current period and the second data volume of the previous period; It is detected whether the first stability index is less than a stability threshold. If it is detected that the first stability index is less than the stability threshold, an abnormality alarm prompt is generated to prompt the corresponding user to check the abnormality.

2. The stability monitoring method according to claim 1, characterized in that: Calculating a first data volume corresponding to each field type in the current period according to the shard fields corresponding to all field types in the current period includes: Construct a first temporary Hive table, and write the shard fields corresponding to all field types in the current cycle into the first temporary Hive table; In the first temporary Hive table, based on a sum command of the Hive table, statistics are collected on the data volume of the shard fields of each field type to obtain a first data volume of the corresponding field type.

3. The stability monitoring method according to claim 2, characterized in that: For any field type, calculating a first stability index between two cycles based on the first data volume of the current cycle and the second data volume of the previous cycle includes: Constructing a second temporary Hive table, and writing the first data volume of each field type in the current cycle and the second data volume of each field type in the previous cycle into the second temporary Hive table respectively; In the second temporary Hive table, a first stability index between two periods is calculated based on a PSI calculation command of the Hive table.

4. The stability monitoring method according to claim 3, characterized in that: The calculation formula corresponding to the PSI calculation command is as follows: PSI1=SUM((X1-X2)*ln(X1 / X2)) PSI1 represents a first stability index, for any field type, X1 represents a first data volume of the field type, X2 represents a second data volume of the field type, and SUM represents a summation operation on all field types.

5. The stability monitoring method according to claim 1, wherein: After detecting whether the first stability indicator is less than a stability threshold, the method further includes: If it is detected that the first stability index is not less than the stability threshold, obtaining a third data volume corresponding to each field type within a historical target period; Calculating a second stability index between the two cycles based on the first data volume of the current cycle and the third data volume of the target cycle; Detecting whether the second stability index is less than the stability threshold; if it is detected that the second stability index is less than the stability threshold, generating the abnormal alarm prompt.

6. The stability monitoring method according to claim 5, characterized in that: Calculating a second stability index between the two cycles based on the first data volume of the current cycle and the third data volume of the target cycle includes: Constructing a third temporary Hive table, and writing the first data volume of each field type in the current cycle and the second data volume of each field type in the target cycle into the third temporary Hive table respectively; In the third temporary Hive table, based on the PSI calculation command of the Hive table, a second stability index between two periods is calculated.

7. The stability monitoring method according to any one of claims 1 to 6, characterized in that: If it is detected that the first stability index is less than the stability threshold, an abnormality alarm is generated, including: If it is detected that the first stability index is less than the stability threshold, stopping importing the offline tag data into the database of the online model; An abnormality warning file is constructed based on the offline tag data, and the abnormality warning file is sent to a target address in the form of an email to form an abnormality warning prompt.

8. A device for monitoring the stability of an online model of a financial system, characterized in that: The stability monitoring device comprises: a data acquisition module, configured to extract data type features of the offline tag data acquired in the current cycle before the database of the online model is imported, to obtain N target data characterized as target floating-point number types, where N is an integer greater than zero; The data sharding module is used to call the same encapsulated sharding logic in sequence for any target data based on at least one preset field type parameter, shard the target data respectively, and obtain sharding fields of corresponding field types; a data statistics module, configured to traverse all target data, obtain shard fields corresponding to all field types in the current period, and calculate a first data volume corresponding to each field type in the current period based on the shard fields corresponding to all field types in the current period; a first indicator calculation module, configured to obtain a second data volume corresponding to each field type in a period preceding the current period, and for any field type, calculate a first stability indicator between the two periods based on the first data volume of the current period and the second data volume of the previous period; The first stability monitoring module is configured to detect whether the first stability index is less than a stability threshold, and if it is detected that the first stability index is less than the stability threshold, generate an abnormality alarm prompt to prompt the corresponding user to check the abnormality.

9. A computer device, characterized in that: The computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the stability monitoring method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the stability monitoring method according to any one of claims 1 to 7 is implemented.

Images