An intelligent acquisition and reporting system based on regulatory data

Abstract

The application discloses a kind of based on supervision data intelligent acquisition and reporting system, the system includes supervision module, positioning module and reporting module;Its technical key points are: obtaining first business requirement, first business requirement is abstractly modeled as first business network, and each layer of first business network contains several nodes, configure acquisition strategy for each node, if the first node of first business network is abnormal, change acquisition strategy on first business network to update original acquisition configuration, obtain update field set, simultaneously, there is linkage in the second node of first business network tracking;Determine the linkage range of the first business requirement update based on second node, obtain second business requirement;Based on second business requirement, reconfigure as second business network, and generate compliance reporting dataset;The application is tracked by each level network linkage, identifies abnormal node, accurately responds to exception, realizes intelligent dynamic acquisition and reporting to supervision data.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data supervision collection and reporting, and particularly relates to a supervision data intelligent collection and reporting system. BACKGROUND

[0002] In the strong supervision field of finance, medical treatment, government affairs and the like, a data collection and reporting system is a core infrastructure for ensuring compliance, and in the financial field, in particular, a supervision data collection method is a support for related personnel (for example, banks) of an enterprise to perform data analysis and report processes, and provides necessary data support to promote system optimization and business adjustment.

[0003] In a traditional supervision data intelligent collection and reporting system, a business behavior or a business activity drives supervision data generation, that is, a running state of a business system has an influence on supervision data generation, and this is particularly embodied in supervision data collection and data quality guarantee. On the one hand, for various business nodes, a fixed strategy or manual batch adjustment is often used, such as transaction flow, customer information and the like, and whether data changes or not, full-amount collection is performed at a fixed frequency (for example, daily), and even if data does not change, repeated storage is still performed, which causes great waste of storage and computing resources. On the other hand, data quality depends on manual review, and once the data volume is large, manual review is difficult to cover all data, potential data quality risks are difficult to effectively control, and update lag is caused, so that when an error occurs in a business at a certain link, a timely response cannot be performed, in the case of batch reporting, reporting is not timely, and finally, reporting is not up to standard, and the supervision requirements cannot be met. SUMMARY

[0004] (I) Technical problems solved

[0005] In view of the deficiencies in the prior art, the present application provides a supervision data intelligent collection and reporting system, which includes a supervision module, a positioning module and a reporting module, abstractly models a first business requirement as a first business network, and the first business network includes a task layer, a processing layer and a reporting layer, realizes full-link tracking of a hierarchical network, if an abnormal node occurs, a first node of the first business network is identified, based on a semantic relationship between the hierarchical networks, a second node existing linkage is accurately tracked, intelligent dynamic collection and reporting of supervision data are realized, and the problems proposed in the background technology are solved.

[0006] (II) Technical solutions

[0007] To achieve the above object, the present application is implemented by the following technical solutions:

[0008] In a first aspect, the present application provides a supervision data intelligent collection and reporting system, and the system includes:

[0009] The supervision module: obtains the first business requirement, abstractly models the first business requirement as a first business network, each layer of the first business network contains a plurality of nodes, configures a collection strategy for each node, including a collection switch, a sampling rate, and a collection probe;

[0010] The positioning module: if an abnormality occurs in a first node of the first business network, changes the collection strategy on the first business network to update the original collection configuration, obtains an updated field set, and simultaneously tracks a second node associated with the first business network; determines a linkage range of the first business requirement update based on the second node, and obtains a second business requirement;

[0011] The reporting module: based on the second business requirement, reconstructs a second business network, and generates a compliance reporting data set.

[0012] Further, the first business requirement is obtained, including:

[0013] Receiving a supervision request;

[0014] The supervision request includes a first business requirement ID, a first business requirement content, and a first business requirement description, and the first business requirement description includes a requirement business address and a requirement business attribute, and the requirement business address has a plurality of requirement business attributes.

[0015] Based on TF-IDF, the key elements of the first business requirement content are analyzed, including tasks, processing, and reporting; wherein the tasks include task types and data sources, the processing includes calculation rules and judgment conditions, and the judgment conditions include first judgment conditions, second judgment conditions, and third judgment conditions, the reporting includes template formats, transmission protocols, and receiving institutions.

[0016] Further, the calculation rule includes:

[0017] The first information and the second information are input into a preset multi-channel training model, the first information is extracted after passing through the first channel, the second information is extracted after passing through the second channel, and the first feature and the second feature are identified after passing through the third channel. Abnormal nodes in the first business network.

[0018] Further, the abnormal node in the first business network is identified, including:

[0019] The first judgment condition is set in the first channel;

[0020] The second judgment condition is set in the second channel;

[0021] The third channel is provided with a third judgment condition, including: based on the first feature or the second feature, a plurality of time windows in which the first feature or the second feature appears are acquired and marked as evaluation intervals; the minimum value of the interval length of all evaluation intervals is taken as the upper limit of the time delay of the autocorrelation function, and the time delay is less than or equal to half of the interval length; an evaluation parameter set of the autocorrelation function is acquired, including: the position, width and amplitude of the peak; wherein the position of the peak represents the time delay corresponding to the peak.

[0022] The evaluation parameter set is compared and analyzed with a preset evaluation standard set to determine an abnormal result and obtain an abnormal score; wherein the preset evaluation standard set includes an offset threshold, a width threshold and an amplitude threshold.

[0023] Further, the first business network includes a task layer, a processing layer and a reporting layer, and the first business requirement is abstractly modeled as the first business network, including:

[0024] Key elements are extracted, and corresponding hierarchical networks are created with each key element as a node, and each hierarchical network works independently;

[0025] The task layer nodes and the processing layer nodes are connected through a first association relationship, and the processing layer nodes and the reporting layer nodes are connected through a second association relationship; the first association relationship includes a trigger condition, and the trigger condition binds the start event of the corresponding node of the task layer, including: based on the task type, automatically matching the task layer node and the processing layer node; the second association relationship includes a transmission protocol and a verification mechanism;

[0026] Based on the first business requirement ID, a corresponding requirement description set is set: {requirement business address description, requirement business attribute description}, and according to a preset matching rule, each description element is compared and matched one by one, and if any description element meets the matching rule, it is determined that this description element exists in the hierarchical network.

[0027] Further, the collection strategy is changed on the first business network to update the original collection configuration, including:

[0028] Based on the first node, a corresponding abnormal score is extracted, and the abnormal score is mapped to a sampling mode, including a normal mode, an enhanced mode and a diagnosis mode;

[0029] Based on the sampling mode, a sampling ratio and a collection probe are determined;

[0030] At the same time, the changed collection configuration is loaded, and the supervision data is collected; based on the first node, if the collection switch is off, all collection probes are unloaded; if the collection switch is on, probe injection is performed, and the sampling rate is updated and adjusted; after completion, the probe is unloaded; wherein the collection probe includes a basic probe, an enhanced probe and a diagnostic probe.

[0031] Further, the linkage comprises: a linkage type, a linkage content set, and a linkage relationship set; wherein the linkage type is a linkage type of the first business requirement; the linkage content set is a first business requirement content; and the linkage relationship set is a relationship between nodes.

[0032] Further, the second node existing linkage on the first business network is tracked, comprising:

[0033] Based on the first business network, direct upstream and downstream nodes of the first node are acquired and marked as candidate nodes;

[0034] The demand business attribute of the first node is extracted, and nodes matching the demand business attribute of the first node are screened from the candidate nodes and marked as second nodes.

[0035] Further, the linkage range comprises: the linked node.

[0036] In a second aspect, the application provides an intelligent collection and reporting method based on supervision data, comprising:

[0037] A first business requirement is acquired, and the first business requirement is abstractly modeled as a first business network, wherein each layer of the first business network comprises a plurality of nodes, and each node is configured with a collection strategy, including a collection switch, a sampling rate, and a collection probe;

[0038] If an abnormality occurs in a first node of the first business network, the collection strategy is changed on the first business network to update the original collection configuration, an updated field set is obtained, and a second node existing linkage on the first business network is tracked; based on the second node, a linkage range of the first business requirement update is determined, and a second business requirement is obtained;

[0039] Based on the second business requirement, a second business network is reconstructed, and a compliance reporting data set is generated.

[0040] (Three) beneficial effects

[0041] The application provides an intelligent collection and reporting system based on supervision data, having the following beneficial effects:

[0042] 1. The application abstractly models the first business requirement as the first business network, and the first business network comprises a task layer, a processing layer, and a reporting layer; by establishing hierarchical matching, the task layer, the processing layer, and the reporting layer are matched one by one in terms of demand address and demand attribute, full-link tracking of the hierarchical network is realized, if an abnormal node occurs, subsequent dynamic adjustment of sampling driven by the abnormality is performed, unnecessary data collection is reduced on the premise of ensuring full-precision collection, and intelligent optimization of supervision data collection and reporting is realized.

[0043] 2、The application accurately tracks the second node existing linkage based on the semantic relationship between each hierarchical network by identifying the first node of the first business network; in this process, by structurally defining the linkage type, linkage content set and linkage relationship set, accurate mapping from the abnormal node to the linkage range can be realized, and the comprehensiveness and efficiency of business demand update are ensured;

[0044] 3、The application sets a self-correlation function, performs abnormal analysis on a plurality of nodes, takes the self-correlation function law of normal operation of each node of the first business network as a basic reference, and once abnormal change occurs, the system can quickly perceive and perform adaptive adjustment, optimizes the data processing flow based on the self-correlation function law, can reduce redundant operations, improves the data processing efficiency, ensures efficient operation of the business network, and better meets the regulatory requirements and business demands. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 It is a module schematic diagram of an intelligent collection and reporting system according to an exemplary embodiment. DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the application.

[0047] Embodiment 1:

[0048] The embodiment of the application provides an intelligent collection and reporting system based on supervision data; Figure 1 It is a module schematic diagram of an intelligent collection and reporting system according to an exemplary embodiment. Please refer to Figure 1 The system is applied to different source business platforms, and the system comprises a supervision module, a positioning module and a reporting module, and the supervision module, the positioning module and the reporting module are in communication connection;

[0049] Before introducing the embodiments of the application, first introduce several terms related to the application:

[0050] Supervision data: refers to a structured data set reflecting business compliance, risk status and business results collected, processed and reported by a financial institution according to the requirements of a regulatory agency;

[0051] Regulatory request: The source business platform issues instructions for collecting and reporting regulatory data, and the collected regulatory data is stored in the source structure. That is, the original structure and organization of the data in the source system are directly retained without format conversion. In the first business network, when the regulatory data collected by each node is transmitted to the storage system, the storage system will automatically identify the source structure according to the data source and store it according to the structure. For example, if the source data is table structure data in a relational database, the storage system will completely retain the table structure, field definition, index, etc. If it is unstructured data in JSON format, the original JSON document is directly stored without disassembly or reorganization.

[0052] Traffic data: In / Out traffic information, that is, the traffic information generated during the transmission of data between nodes in the business network. For example: payment transaction throughput cross-site data transmission.

[0053] Resource occupation data: Reflects the use of various system resources during the operation of the business network, directly affecting the performance and stability of the network.

[0054] Collection switch: A binary switch 0 / 1 used to control whether to collect regulatory data from the node.

[0055] Sampling rate: Used to define the probability distribution of data sampling, that is, the frequency of data collection during the operation of the business network.

[0056] Basic probe: A bottom-layer monitoring tool deployed on each node of the business network, such as performance counters, log collectors, and default components.

[0057] Enhanced probe: Extends the capabilities of the basic probe, focusing on deep data mining and complex business scenario requirements, such as dynamically loaded bytecode injection components according to business needs.

[0058] Diagnosis probe: The core tool for fault diagnosis and repair of the business network. For example, when the business network fails to meet the data reporting standards, it quickly locates the abnormal node and, with the help of the original data stored in the source structure, restores the complete data scenario when the anomaly occurs, improving the accuracy and efficiency of fault location.

[0059] Sampling ratio: The proportion of the amount of collected data in the total amount of data.

[0060] The following is an explanation of each module:

[0061] Regulatory module: Obtain the first business requirement, abstract the first business requirement as a first business network, including a task layer, a processing layer, and a reporting layer, and each layer of the first business network contains a plurality of nodes. Configure collection strategies for each node, including collection switches, sampling rates, and collection probes.

[0062] Obtaining a first service requirement, comprising:

[0063] Receiving a regulatory request;

[0064] The regulatory request includes a first service requirement ID, a first service requirement content, and a first service requirement description, and the first service requirement description includes a requirement service address and a requirement service attribute, and the requirement service address has several, and the requirement service attribute has several;

[0065] Based on TF-IDF, the key elements of the first service requirement content are parsed, including tasks, processing, and reporting; wherein the task includes task type and data source, the processing includes calculation rule and judgment condition, and the reporting includes template format, transmission protocol and receiving institution;

[0066] The task type includes: real-time analysis service and batch analysis service; the data source includes: calling service and state service;

[0067] The calling service: in the first service authorization process, the corresponding first information is called, including but not limited to identity data and behavior data; wherein the identity data includes: customer identity information and customer account information, and the behavior data includes: electronic signature, face parameter, pupil parameter, voiceprint parameter and fingerprint parameter;

[0068] The state service: in the first service transmission process, the corresponding second information is identified, including but not limited to traffic data and resource occupation data; the state service reflects the system performance monitoring;

[0069] The calculation rule includes:

[0070] The first information and the second information are input into the preset multi-channel training model, the first information is extracted after the first channel, the second information is extracted after the second channel, and the first feature and the second feature are identified after the third channel. Abnormal nodes appear in the first business network;

[0071] Identifying abnormal nodes in the first business network includes:

[0072] The first judgment condition is set in the first channel, including:

[0073] The first information is matched with the preset authentication database for similarity, if the similarity is greater than or equal to the preset similarity threshold, it is determined that the authorization is successful; if the similarity is less than the preset similarity threshold, it is determined that the authorization is abnormal;

[0074] At the same time, the similarity matching result of the authorization exception is taken as the first feature;

[0075] When the identity data (customer identity, account information) and behavior data (face, pupil, voiceprint, fingerprint parameters) are abnormal, it indicates that the authorization process is wrong, and the collection will be delayed;

[0076] The second channel is provided with a second judgment condition, including:

[0077] The flow data is compared with the preset flow threshold value: if the flow data is greater than or equal to the preset flow threshold value, it is determined that the flow is abnormal; if the flow data is less than the preset flow threshold value, it is determined that the flow is normal;

[0078] The resource occupation data is compared with the preset occupation threshold value: if the resource occupation data is greater than or equal to the preset occupation threshold value, it is determined that the resource occupation is abnormal; if the resource occupation data is less than the preset occupation threshold value, it is determined that the resource occupation is normal;

[0079] At the same time, the comparison result of the abnormal result is taken as the second feature;

[0080] The third channel is provided with a third judgment condition, including:

[0081] Based on the first feature or the second feature, a plurality of time windows in which the first feature or the second feature appears are obtained, and are marked as evaluation intervals; the minimum value of the interval length of all evaluation intervals is taken as the upper limit of the time delay for calculating the autocorrelation function, and the time delay is less than or equal to half of the interval length; based on the upper limit of the time delay, an evaluation parameter set of the autocorrelation function is obtained, including: the position, width and amplitude of the peak value; wherein the position of the peak value represents the time delay corresponding to the peak value;

[0082] The evaluation parameter set is compared and analyzed with the preset evaluation standard set to determine the abnormal result; wherein the preset evaluation standard set includes an offset threshold, a width threshold and an amplitude threshold;

[0083] If the position of the peak value changes, and the position change is greater than or equal to the preset offset threshold, it is determined that an abnormality occurs;

[0084] If the width of the peak value changes, and the width change is greater than or equal to the preset width threshold, it is determined that an abnormality occurs;

[0085] If the amplitude of the peak value changes, and the amplitude change is greater than or equal to the preset amplitude threshold, it is determined that an abnormality occurs;

[0086] The absolute difference value between the position of the peak value and the offset threshold is calculated, and then divided by the upper limit of the time delay to obtain the position abnormal score: ; In the formula, represents the position of the peak value, represents the offset threshold, represents the upper limit of the time delay;

[0087] The width abnormality score is obtained by calculating the absolute value of the ratio of the width of the peak to the width threshold value on a logarithmic scale: ; wherein, represents the width of the peak, represents the width threshold value;

[0088] The amplitude abnormality score is obtained by measuring the difference amplitude of the amplitude of the peak and the amplitude threshold value based on the standard deviation: ; wherein, b represents the amplitude of the peak, b ref represents the amplitude threshold value, and b represents the standard deviation of the amplitude;

[0089] The final abnormality score is obtained by weighted fusion of the position abnormality score, the width abnormality score, and the amplitude abnormality score;

[0090] The weight coefficients in the weighted fusion are determined by the coefficient of variation method, which is a method of weighting each index according to the variation degree of the current value and the target value of each evaluation index. If the numerical difference of an index is large and can clearly distinguish each evaluated object, it means that the index has rich discrimination information, and therefore the index should be given a larger weight. Conversely, if the numerical difference of each evaluated object on an index is small, the index has weak ability to distinguish each evaluation object, and therefore the index should be given a smaller weight. This method directly uses the information contained in each index and obtains the weight of the index by calculation, so it is objective;

[0091] It should be noted that in the first business network, if each node is normally running, the autocorrelation function of its time series will usually show a certain regularity, for example: periodic task processing will cause the autocorrelation function to have a peak at a certain lag time; by judging the shift of the peak position of the autocorrelation function, it is indicated that the period has changed; by judging the narrowing of the peak width, it is indicated that the stability of the periodic signal has decreased; by judging the decrease of the peak amplitude, it is indicated that the periodicity has weakened; that is, by the position, width, and amplitude of the peak, it can be determined that an abnormality has occurred at a node in the first business network; at the same time, the regularity analysis of the autocorrelation function of the time series of each node helps to clarify the data correlation and business collaboration relationship between nodes, promote more efficient collaboration between nodes, optimize the business network architecture, and improve the overall collaboration capability, laying a solid foundation for subsequent implementation of second business requirement update and business network reconstruction, and ensuring the continuous and efficient operation of the business network in a complex and changing environment;

[0092] The first business requirement is abstractly modeled as a first business network, comprising:

[0093] Creating a hierarchical relationship: extracting key elements and creating corresponding hierarchical networks with each key element as a node, and each hierarchical network works independently;

[0094] For example: task layer: taking task as node, target business attribute including task type, execution subject, data source and completion time limit; execution subject is the department or system responsible for executing the task, and completion time limit is the time requirement for completing the task; processing layer: taking processing as node, target business attribute including processing logic, data input source and output result; processing logic is the description of business logic executed, such as calculation rules and judgment conditions in processing process, data input source is the data source received by the task, and output result is the output after processing; reporting layer: taking receiving institution as node, target business attribute including institution code, reporting format and encryption standard; institution code is the unique identification of regulatory institution (such as PBC_110 for central bank code), reporting format is the required data format (such as XML, JSON, ISO20022), and encryption standard is the encryption requirement for data transmission;

[0095] Establish cross-layer connection: connect task layer node and processing layer node through first association relationship, and connect processing layer node and reporting layer node through second association relationship;

[0096] Among them, the first association relationship includes: trigger condition, and the trigger condition binds the start event of the task layer corresponding node, including: based on task type, automatically matching task layer node and processing layer node: for example: when the task starts, trigger the processing node to execute; when the task type is real-time analysis business, the strong binding mode is adopted; when the task type is batch analysis business, the weak binding mode is adopted;

[0097] Among them, the second association relationship includes: transmission protocol, verification mechanism; for example: transmission protocol includes but is not limited to HTTP, HTTPS protocol; for example: real-time reporting uses HTTPS protocol, batch reporting uses SFTP protocol; verification mechanism includes but is not limited to hash verification, digital signature;

[0098] Establish hierarchical matching: set the corresponding demand description set A based on the first business demand ID: A={demand business address description a1, demand business attribute description a2}, and compare and match one by one according to the preset matching rule, if any description element meets the matching rule, it is determined that there is this description element in the hierarchical network; among them, demand business address includes network endpoint, API path, and target business attribute includes key-value pair;

[0099] According to the preset matching rule, compare and match one by one, including:

[0100] First, judge whether the demand business address exists in the demand business address description of the demand description set corresponding to a hierarchical network:

[0101] If not, it is determined that there is no matching demand description in the hierarchical network;

[0102] If the demand service address description exists, it is determined whether the demand service attribute exists in the demand service attribute description of the demand description set corresponding to the certain level network; if not, it is determined that there is no matching demand description in the level network; if yes, it is determined that there is a matching demand description in the level network;

[0103] Specifically, the node sets of the task layer, the processing layer and the reporting layer are scanned in sequence, and the demand description set A and the demand description under the node set of the task layer are matched first;

[0104] If the demand service address description exists, it is determined whether the demand service attribute exists in the demand service attribute description of the demand description corresponding to the task layer;

[0105] If there is no matching demand description in the task layer, the demand description element is matched with the demand description under the processing layer;

[0106] If there is no matching demand description in the processing layer, the demand description element is matched with the demand description under the reporting layer;

[0107] If there is a matching demand description in the task layer, the processing layer and the reporting layer, the matching demand description is obtained to call the corresponding level network information, so as to realize full-link tracking of the level network, and if an abnormal node appears in a certain level network, it can be quickly located and positioned to the associated abnormal node;

[0108] In addition, the matching principles include a demand service address matching rule and a demand service attribute matching rule;

[0109] For the demand service address:

[0110] Load the demand service addresses of each node of each level network and collect them: a network endpoint set E={e1, e2, …, em0} and an API path set F={f1, f2, …, fm1}; wherein m0 is the number of endpoints, m1 is the number of API paths, and m0 and m1 are both greater than or equal to 1;

[0111] The Levenshtein distance algorithm is used to compare the URL similarity, wherein the URL is the minimum additive combination of the network endpoint and the API path, i.e. the complete address = network endpoint + API path;

[0112] For example:

[0113] Network endpoint: usually a service address: (such as: 192.168.1.1:8080, api.example.com);

[0114] API path: usually a resource path (such as: / api / v1 / payment);

[0115] URL: full address = network endpoint + API path (e.g., https: / / api.example.com / api / v1 / payment);

[0116] The similarity calculation formula is: ;

[0117] In the formula, is the minimum edit distance of two URL strings, μ i and μ j represent the URLs to be compared; when If the calculated similarity is greater than the preset similarity threshold, it is determined that the two demand business addresses match; otherwise, it is determined that the two demand business addresses do not match.

[0118] For demand business attributes: business attribute key-value pairs are the basic units of structured representation of business demand elements, key: the name of the demand attribute (such as field, frequency, protocol); value: the specific content or parameter of the attribute; by loading the demand business attributes of each node of each level network and collecting: G = {Kp, Lp | p = 1, 2, …, m3}, where Kp is the attribute key, Lp is the attribute value, and m3 is greater than 0; in the strong binding mode, precise matching is implemented, and in the weak binding mode, fuzzy logic matching is implemented, and the preset matching threshold of precise matching is greater than the preset matching threshold of fuzzy logic matching.

[0119] Through the above steps, the first business demand can be abstractly modeled as a first business network, including a task layer, a processing layer, and a reporting layer, and a plurality of nodes of the task layer are used for a plurality of task assignments, a plurality of nodes of the processing layer are used for a plurality of processing analyses, and a plurality of nodes of the reporting layer are used for a plurality of reporting reports.

[0120] In addition, by analyzing the first business demand change, the supervision data collection strategy is changed, because the business behavior or business activity drives the generation of supervision data, for example: in the authorization process, the face verification authorization fails, and the customer identity verification field under the supervision field is missing; for example: in the business processing process, the business delay suddenly increases, and the corresponding supervision field calculation timeout.

[0121] Positioning module: if the first node of the first business network is abnormal, change the collection strategy on the first business network to update the original collection configuration, obtain an updated field set, and at the same time, track the second node linked with the first business network; based on the second node, determine the linkage range of the first business demand update, and obtain the second business demand;

[0122] Change the collection strategy on the first business network to update the original collection configuration, including:

[0123] Based on the first node, the corresponding anomaly score is extracted, and the anomaly score is mapped to a sampling mode:

[0124] If the anomaly score is less than the first threshold, it is mapped to a normal mode;

[0125] If the first threshold is less than or equal to the anomaly score and less than the second threshold, it is mapped to an enhanced mode;

[0126] If the anomaly score is greater than the second threshold, it is mapped to a diagnostic mode; wherein 0 < first threshold < second threshold < 1;

[0127] Based on the sampling mode, the sampling ratio and the collection probe are determined;

[0128] At the same time, the changed collection configuration is loaded, and the supervision data is collected; based on the first node, if the collection switch is closed, all collection probes are unloaded; if the collection switch is opened, probe injection is performed, and the sampling rate is updated and adjusted; after completion, probe unloading is performed; wherein the collection probe includes: basic probe, enhanced probe and diagnostic probe;

[0129] Based on the sampling mode, the sampling ratio and the collection probe are determined, comprising:

[0130] In the normal mode, only the basic probe is included, and the sampling ratio R is: R ∈ [R1_min, R1_max];

[0131] In the enhanced mode, the basic probe and the enhanced probe are included, and the sampling ratio R is: R ∈ [R2_min, R2_max];

[0132] In the diagnostic mode, the basic probe, the enhanced probe and the diagnostic probe are included, and the sampling ratio R is: R ∈ [R3_min, R3_max]; wherein 0 < R1_max < R2_min < R3_max ≤ 1; R1, R2 and R3 are the data volume in the normal mode, the enhanced mode and the diagnostic mode, respectively;

[0133] Wherein, the generation of the updated field set comprises:

[0134] By comparing the current collection configuration of the first node with the changed collection configuration, the difference fields are identified and classified as field addition, field modification and field deletion; for example: if the changed configuration contains a field that is not included in the current configuration, it is determined as field addition; if the field name is the same but the attribute (type, length, etc.) is different, it is determined as field modification; if the field existing in the current configuration is removed in the changed configuration, it is determined as field deletion;

[0135] The operation instruction for each difference field is generated, including a field path, an operation type, and a new value; wherein the operation type includes one-to-one correspondence with the difference type, including addition, modification, and deletion; the field path is represented in the format: {hierarchical network, configuration category, field name}; the new value at least contains the field name, the data type, and the field length;

[0136] Through this structured update field set, the system can automatically adjust the collection strategy of the supervision data when the first business network detects an exception, realize intelligent collection, ensure the compliance and integrity of data collection and reporting, and realize closed-loop management from exception discovery to collection configuration update in the whole process, effectively reducing the cost of manual intervention and compliance risk;

[0137] The linkage includes: a linkage type, a linkage content set, and a linkage relationship set;

[0138] The linkage type is the linkage type of the first business requirement, and the linkage type is upstream and downstream linkage; the linkage content set is the first business requirement content; the linkage relationship set is the relationship between nodes, including task-processing mapping relationship, task-processing mapping relationship, and reporting-processing dependency relationship;

[0139] The first business network tracks the second node with linkage, including:

[0140] Based on the first business network, the direct upstream and downstream nodes of the first node are obtained and marked as candidate nodes;

[0141] If the first node belongs to the task layer, scan downward, the scanning level is the processing layer, and the linkage relationship is the task-processing mapping relationship;

[0142] If the first node belongs to the processing layer, scan bidirectionally, scan the task layer upward and the reporting layer downward, and the linkage relationship is the task-processing mapping relationship and the reporting-processing dependency relationship;

[0143] If the first node belongs to the reporting layer, scan upward, the scanning level is the processing layer, and the linkage relationship is the reporting-dependency relationship;

[0144] The demand business attribute of the first node is extracted, and within the determined scanning direction and target level range, nodes matching the demand business attribute of the first node are selected from the candidate nodes, including: calculating the matching degree of the demand business attribute of the candidate node and the demand business attribute of the first node, and when the matching degree is greater than the corresponding preset matching threshold, the candidate node is marked as the second node;

[0145] In the case that the first business requirement is abstracted as a first business network, the linkage type, linkage content set and linkage relationship set are structured and defined, including analyzing the node relationship and linkage type, such as: reporting-dependent relationship; This way facilitates the computer to perform logical operations of linkage, and can realize accurate mapping from abnormal nodes to linkage range, ensuring the comprehensiveness and efficiency of business requirement update; Through dynamic adjustment sampling driven by abnormalities, intelligent optimization of supervision data collection is realized under the premise of ensuring full-precision collection;

[0146] Determine the linkage range of the first business requirement update based on the second node, including:

[0147] Update the first business network based on the second node to obtain a second business network;

[0148] Compare the first business network with the second business network to obtain the linkage range generated by the first business requirement update;

[0149] The linkage range includes: the linked node;

[0150] Based on the identified second node (an abnormal influence node having data dependency or business association with the first node), the collection configuration of the first business network is adaptively updated to generate a second business network containing the second node; When updating the collection configuration of the first business network, the collection configuration of the second node is adjusted, and the association relationship between the first node and the second node is established or modified, including but not limited to transmission protocol and verification mechanism;

[0151] For example: a data transmission link is added between two nodes to transmit the associated information in the financial data and customer identity data, and the encrypted transmission of data between nodes is realized by building a secure HTTPS channel; For example: the first node originally uses JSON format to transmit financial data, and the second node uses XML format to transmit customer identity data; In order to realize smooth data interaction, the output interface protocol of the second node is adjusted to JSON format, and the data transmission standard is unified to avoid data transmission errors caused by format incompatibility.

[0152] Reporting module: based on the second business requirement, reconstructing the second business network and generating a compliance reporting data set.

[0153] Embodiment 2:

[0154] The embodiment of the application provides a supervision data intelligent collection and reporting method; the method comprises the following steps:

[0155] Obtaining a first service requirement, abstractly modeling the first service requirement as a first service network, each layer of the first service network comprising a plurality of nodes, configuring a collection strategy for each node, including a collection switch, a sampling rate, and a collection probe;

[0156] If an abnormality occurs in a first node of the first service network, changing the collection strategy on the first service network to update the original collection configuration, obtaining an updated field set, and simultaneously tracking a second node associated with the first node; determining a linkage range of the first service requirement update based on the second node, and obtaining a second service requirement;

[0157] Based on the second service requirement, reconstructing a second service network, and generating a compliance reporting data set.

[0158] In the application, the several formulas involved are dimensionless values, and the formulas are obtained by software simulation of a large amount of data to obtain a formula of the most recent real situation, and the formula is set by a person skilled in the art according to the actual situation.

[0159] The above embodiments can be realized wholly or partially by software, hardware, firmware, or any other combination. When realized by software, the above embodiments can be realized wholly or partially in the form of a computer program product. Those skilled in the art can realize that the units and algorithm steps of the examples described in connection with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized by hardware or software depends on the specific application and design constraints of the technical solutions.

[0160] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, and can be located in one place or distributed on multiple network units. Part or all of the units can be selected to achieve the purpose of the embodiments according to actual needs.

[0161] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application.

Claims

1. An intelligent collection and reporting system based on regulatory data, characterized in that, The method comprises the following steps: a supervision module: obtaining a first business requirement, which is abstractly modeled as a first business network, and each layer of the first business network comprises a plurality of nodes, and a collection strategy is configured for each node, including a collection switch, a sampling rate, and a collection probe; a positioning module: if an abnormality occurs in a first node of the first business network, the collection strategy is changed on the first business network to update the original collection configuration, and an updated field set is obtained, and at the same time, a second node linked to the first business network is tracked; determining a linkage range of the first business requirement update based on the second node, and obtaining a second business requirement; a reporting module: based on the second business requirement, reconstructing a second business network, and generating a compliance reporting data set; wherein the first business requirement comprises a supervision request, wherein the supervision request comprises a first business requirement ID, a first business requirement content, and a first business requirement description, and the first business requirement description comprises a requirement business address and a requirement business attribute, and the requirement business address has a plurality of requirement business attributes; based on TF-IDF, the key elements of the first business requirement content are analyzed, including task, processing, and reporting; wherein the task includes task type and data source, the processing includes calculation rule and judgment condition, and the judgment condition includes first judgment condition, second judgment condition and third judgment condition, the reporting includes template format, transmission protocol and receiving institution; the calculation rule comprises: inputting the first information and the second information into a preset multi-channel training model, extracting the first feature after the first information passes through the first channel, extracting the second feature after the second information passes through the second channel, and identifying the abnormal node of the first business network after the first feature and the second feature pass through the third channel; the identification of the abnormal node of the first business network comprises: the first judgment condition is set in the first channel; the second judgment condition is set in the second channel; the third judgment condition is set in the third channel, which comprises: based on the first feature or the second feature, a plurality of time windows in which the first feature or the second feature appears are obtained, and are marked as evaluation intervals; the minimum value of the interval length of all evaluation intervals is taken as the upper limit of the time delay of the autocorrelation function, and the time delay is less than or equal to half of the interval length, wherein the autocorrelation function is the autocorrelation function of the time series; obtaining an evaluation parameter set of the autocorrelation function, including the position, width and amplitude of the peak; wherein the position of the peak represents the time delay corresponding to the peak; comparing and analyzing the evaluation parameter set with the preset evaluation standard set to determine the abnormal result and obtain the abnormal score; wherein the preset evaluation standard set comprises a shift threshold, a width threshold and an amplitude threshold.

2. The intelligent collection and reporting system based on regulatory data according to claim 1, characterized in that, The first business requirement is abstractly modeled as a first business network, which comprises: extracting key elements and creating corresponding hierarchical networks with each key element as a node, and each hierarchical network works independently; The task layer node and the processing layer node are connected through a first association relationship, and the processing layer node and the reporting layer node are connected through a second association relationship; the first association relationship comprises a trigger condition, and the trigger condition binds a start event of the task layer corresponding node, and comprises: based on the task type, automatically matching the task layer node and the processing layer node; the second association relationship comprises a transmission protocol and a verification mechanism; Based on the first business requirement ID, set the corresponding requirement description set: {requirement business address description, requirement business attribute description}, and according to the preset matching rule, compare and match one by one, if any description element meets the matching rule, it is judged that there is this description element in the hierarchical network.

3. The system according to claim 1, wherein, The first business network is changed to update the original collection configuration, comprising: Based on the first node, the corresponding abnormal score is extracted, and the abnormal score is mapped to the sampling mode, including the normal mode, the enhanced mode and the diagnosis mode; Based on the sampling mode, the sampling ratio and the collection probe are determined; At the same time, load the changed collection configuration, collect the supervision data; based on the first node, if the collection switch is closed, unload all collection probes; if the collection switch is opened, execute probe injection, and update and adjust the sampling rate; after completion, execute probe unloading; wherein the collection probe includes: basic probe, enhanced probe and diagnostic probe.

4. The intelligent collection and reporting system based on regulatory data according to claim 1, wherein, The linkage includes: linkage type, linkage content set and linkage relationship set; wherein, the linkage type is the linkage type of the first business requirement; the linkage content set is the first business requirement content; the linkage relationship set is the relationship between nodes.

5. The system according to claim 4, wherein, The first business network is tracked to exist the second node of linkage, comprising: Based on the first business network, the direct upstream and downstream nodes of the first node are obtained and marked as candidate nodes; Extract the requirement business attribute of the first node, and filter the nodes matched with the requirement business attribute of the first node from the candidate nodes, and mark the nodes as second nodes.

6. The intelligent collection and reporting system based on regulatory data according to claim 5, wherein, The linkage range includes: the linked node.

7. An intelligent collection and reporting method based on regulatory data, characterized in that, Comprising: Obtain the first business requirement, the first business requirement is abstractly modeled as a first business network, and each layer of the first business network contains a plurality of nodes, configure collection strategy for each node, including collection switch, sampling rate and collection probe; If the first node of the first business network is abnormal, change the collection strategy on the first business network to update the original collection configuration, obtain an update field set, and at the same time, track the second node of linkage on the first business network; Based on the second node, determine the linkage range of the first business requirement update, obtain the second business requirement; Based on the second business requirement, reconstruct as a second business network, and generate a compliance reporting data set; The first business demand is obtained, including: receiving a supervision request; wherein the supervision request includes a first business demand ID, a first business demand content, and a first business demand description, and the first business demand description includes a demand business address and demand business attributes, and the demand business address has a plurality of demand business attributes; the key elements of the first business demand content are analyzed based on TF-IDF, including: task, processing, and reporting; wherein the task includes task type and data source, the processing includes calculation rule and judgment condition, and the judgment condition includes first judgment condition, second judgment condition, and third judgment condition, and the reporting includes template format, transmission protocol, and receiving institution; The calculation rule includes: inputting the first information and the second information into a preset multi-channel training model, extracting the first feature after the first information passes through the first channel, extracting the second feature after the second information passes through the second channel, and identifying the abnormal nodes of the first business network through the third channel after the first feature and the second feature pass through the third channel; The abnormal nodes of the first business network are identified, including: the first judgment condition is set in the first channel; the second judgment condition is set in the second channel; the third judgment condition is set in the third channel, including: based on the first feature or the second feature, a plurality of time windows in which the first feature or the second feature appears are obtained, and are marked as evaluation intervals; the minimum value of the interval length of all evaluation intervals is taken as the upper limit of the time delay of the autocorrelation function, and the time delay is less than or equal to half of the interval length, wherein the autocorrelation function is the autocorrelation function of the time series; the evaluation parameter set of the autocorrelation function is obtained, including: the position, width, and amplitude of the peak value; wherein the position of the peak value represents the time delay corresponding to the peak value; The evaluation parameter set is compared and analyzed with the preset evaluation standard set to determine the abnormal result and obtain the abnormal score; wherein the preset evaluation standard set includes offset threshold, width threshold, and amplitude threshold.

Images