Insurance service data processing method and device, storage medium and computer equipment

Abstract

The invention discloses an insurance service data processing method and device, a storage medium and computer equipment, relates to the technical field of data processing, and mainly aims at improving the data processing efficiency and processing accuracy. The method comprises the following steps: acquiring to-be-processed business data; judging whether the data volume of the to-be-processed service data is greater than a preset threshold value or not; if the data volume is greater than a preset threshold value, classifying the to-be-processed service data to obtain service data in different categories; determining service processing types respectively corresponding to the service data under the different categories, and determining service processing strategies corresponding to the service processing types; and performing parallel processing on the business data under the corresponding categories by using the business processing strategy to obtain processing results corresponding to the business data under the different categories. The method and the device are suitable for processing the insurance service data.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method, device, storage medium and computer equipment for processing insurance business data. Background Art

[0002] In the insurance industry, with the rapid growth of data volume of insurance companies, data processing has become a core technical challenge.

[0003] Currently, data is usually processed in a centralized manner. However, this processing method may cause processing resources to become stuck or damaged when the amount of data is large, resulting in low data processing efficiency and low processing accuracy. Summary of the invention

[0004] The present invention provides a method, device, storage medium and computer equipment for processing insurance business data, which are mainly capable of improving data processing efficiency and processing accuracy.

[0005] According to a first aspect of the present invention, there is provided a method for processing insurance business data, comprising:

[0006] Obtain business data to be processed;

[0007] Determine whether the amount of the service data to be processed is greater than a preset threshold;

[0008] If the data volume is greater than a preset threshold, classifying the service data to be processed to obtain service data under different categories;

[0009] Determine the business processing types corresponding to the business data under the different categories, and determine the business processing strategies corresponding to the business processing types;

[0010] The business processing strategy is used to process the business data under the corresponding categories in parallel to obtain the processing results corresponding to the business data under the different categories.

[0011] Optionally, before using the business processing strategy to process the business data of corresponding categories in parallel to obtain the processing results respectively corresponding to the business data of different categories, the method further includes:

[0012] The business data under the different categories are stored in a preset distributed database using a preset distributed storage method, wherein the storage process is: determining the category data volume corresponding to the business data under the different categories, and based on the category data volume, determining the number of storage nodes in the preset distributed database, and storing the business data under the different categories in storage nodes with the corresponding number of nodes;

[0013] The using the business processing strategy to process the business data of corresponding categories in parallel to obtain the processing results corresponding to the business data of different categories respectively includes:

[0014] In response to processing signals corresponding to business data under different categories, the business data under corresponding categories in the preset distributed database are parallel scheduled and processed using the business processing strategy to obtain processing results corresponding to the business data to be processed.

[0015] Optionally, before storing the business data of different categories in a preset distributed database using a preset distributed storage method, the method further includes:

[0016] Call the encryption key generated by the encryption service of the preset key management platform;

[0017] Using a preset block encryption algorithm to symmetrically encrypt the business data under the different categories to obtain encrypted business data;

[0018] The storing of the business data of different categories into a preset distributed database by using a preset distributed storage method includes:

[0019] Storing the encrypted business data in a preset distributed database using a preset distributed storage method;

[0020] The using the business processing strategy to perform parallel scheduling processing on the business data of the corresponding category in the preset distributed database to obtain the processing result corresponding to the business data to be processed includes:

[0021] Call the decryption key generated by the preset key management platform decryption service;

[0022] Decrypting the business data of the corresponding category in the preset distributed database by using a preset group decryption algorithm to obtain the decrypted business data;

[0023] The business processing strategy is used to perform parallel scheduling processing on the decrypted business data under the corresponding category in the preset distributed database to obtain the processing results corresponding to the business data to be processed.

[0024] Optionally, after obtaining the service data to be processed, the method further includes:

[0025] Determine a compressed package corresponding to the original business data to be processed, and determine an original hash value corresponding to the original business data to be processed;

[0026] Decompressing the compressed package to obtain the service data to be verified;

[0027] Calculate the actual hash value corresponding to the business data to be verified;

[0028] If the original hash value is equal to the actual hash value, determining the service data to be verified as the service data to be processed;

[0029] If the original hash value is not equal to the actual hash value, the business data to be verified is deleted and the compressed package is decompressed again.

[0030] Optionally, after the business data of corresponding categories are processed in parallel by using the business processing strategy to obtain processing results corresponding to the business data of different categories, the method further includes:

[0031] The preset dimensional modeling algorithm is used to perform data integration on different processing results to obtain the integrated processing results corresponding to the business data to be processed.

[0032] Optionally, the using the business processing strategy to process the business data of corresponding categories in parallel to obtain processing results corresponding to the business data of different categories respectively includes:

[0033] If the business data under any category of the business data under the different categories belong to different batches of data, determining the multi-batch processing types corresponding to the different batches of data respectively;

[0034] determining a multi-batch processing strategy corresponding to the multi-batch processing type;

[0035] Using the multi-batch processing strategy to process the corresponding batches of data in parallel, to obtain processing results corresponding to the different batches of data respectively;

[0036] If the business data under any category belongs to single batch data, then determining multiple single batch processing types corresponding to the single batch data, and determining the processing order corresponding to the multiple single batch processing types;

[0037] Determining single batch processing strategies corresponding to the multiple single batch processing types;

[0038] Based on the processing sequence, the single batch data is serially processed using the single batch processing strategy to obtain processing results corresponding to the single batch data.

[0039] Optionally, the method further comprises:

[0040] In the process of parallel processing of the business data under the different categories, recording parallel processing information, and storing the parallel processing information in a preset information table;

[0041] Checking the preset information table at preset intervals, and judging whether any data in the business data of different categories has a processing abnormality based on the parallel processing information recorded in the preset information table;

[0042] If the processing exception exists, determining the exception level corresponding to the arbitrary data;

[0043] If the abnormal level is greater than a preset level threshold, a preset communication tool interface is called, and the abnormal alarm information is sent to an operation and maintenance personnel terminal by using the preset communication tool interface, so that the operation and maintenance personnel of the operation and maintenance personnel terminal can adjust the processing process of the arbitrary data based on the abnormal alarm information;

[0044] If the abnormality level is less than or equal to a preset level threshold, determining an abnormality handling strategy corresponding to the business processing type of the arbitrary data;

[0045] The exception handling strategy is used to automatically adjust the processing process of the arbitrary data.

[0046] According to a second aspect of the present invention, there is provided a device for processing insurance business data, comprising:

[0047] An acquisition unit, used for acquiring business data to be processed;

[0048] A judging unit, used to judge whether the amount of the service data to be processed is greater than a preset threshold;

[0049] A classification unit, configured to classify the service data to be processed to obtain service data of different categories if the data volume is greater than a preset threshold;

[0050] A determination unit, configured to determine the business processing types corresponding to the business data under the different categories, and determine the business processing strategies corresponding to the business processing types;

[0051] The processing unit is used to use the business processing strategy to perform parallel processing on the business data under the corresponding categories to obtain processing results corresponding to the business data under the different categories.

[0052] According to a third aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, which implements the above insurance business data processing method when executed by a processor.

[0053] According to a fourth aspect of the present invention, there is provided a computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the above insurance business data processing method when executing the program.

[0054] According to a processing method, device, storage medium and computer equipment of insurance business data provided by the present invention, compared with the current method of processing data by centralized processing, the present invention obtains the business data to be processed; and determines whether the data volume of the business data to be processed is greater than a preset threshold; if the data volume is greater than the preset threshold, the business data to be processed is classified to obtain business data under different categories; then the business processing types corresponding to the business data under the different categories are determined, and the business processing strategies corresponding to the business processing types are determined; finally, the business data under the corresponding categories are processed in parallel using the business processing strategies to obtain the processing results corresponding to the business data under the different categories. Thus, by classifying business data greater than a certain data volume, business data under different categories are obtained, and the processing strategies corresponding to the business data under different categories are determined, and finally the processing strategies are used to process the business data under the corresponding categories in parallel, that is, a large amount of business data is divided into multiple groups of small business data, and multiple processing threads are used to process multiple groups of business data in parallel, avoiding the jamming and damage of processing resources, thereby improving the processing efficiency and accuracy of business data. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0056] Figure 1 A flow chart of a method for processing insurance business data provided by an embodiment of the present invention is shown;

[0057] Figure 2 A flow chart of another method for processing insurance business data provided by an embodiment of the present invention is shown;

[0058] Figure 3 A schematic diagram showing the structure of a device for processing insurance business data provided by an embodiment of the present invention;

[0059] Figure 4 A schematic diagram showing the structure of another device for processing insurance business data provided by an embodiment of the present invention;

[0060] Figure 5 A schematic diagram of the physical structure of a computer device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0061] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

[0062] Currently, the centralized processing method for data processing may cause jamming or damage to processing resources, resulting in low data processing efficiency and low processing accuracy.

[0063] In order to solve the above problems, the embodiment of the present invention provides a method for processing insurance business data, such as Figure 1 As shown, the method includes:

[0064] 101. Obtain business data to be processed.

[0065] The business data to be processed may be business data in the insurance field. The present invention does not set a limit on the amount of the business data to be processed. For example, it may be PB-level business data.

[0066] For the embodiment of the present invention, when the business data to be processed is PB (petabyte, a high-level storage unit) level business data, in order to reduce the congestion of processing resources and improve the processing efficiency and accuracy of business data, it is necessary to classify the business data to be processed according to the company category or business category to obtain business data under different categories. For example, if the business data to be processed is a full data table containing a large amount of data, then in order to improve the processing efficiency of the data, the full data table can be divided into data tables of different categories according to the company category and business category, and finally different processing processes are assigned to the data tables under different categories, and finally different processing processes are used to process the data in the corresponding data tables in parallel, so that the processing efficiency and accuracy of the business data to be processed can be improved by processing the data in different data tables in parallel. The embodiment of the present invention is mainly applicable to the scenario of processing business data. The execution subject of the embodiment of the present invention is a device or equipment capable of processing business data, which can be specifically set on the server side.

[0067] 102. Determine whether the amount of the business data to be processed is greater than a preset threshold.

[0068] 103. If the data volume is greater than a preset threshold, the business data to be processed is classified to obtain business data under different categories.

[0069] Among them, the preset threshold is set according to actual needs. For the embodiment of the present invention, since the computer system may experience resource jamming or damage when processing large amounts of data, after obtaining the business data to be processed, it is first necessary to determine whether the data volume of the business data to be processed is greater than the preset threshold. If the data volume of the business data to be processed is greater than the preset threshold, it can be determined that the data volume of the business data to be processed is large. At this time, in order to avoid jamming and damage to processing resources, it is necessary to classify the business data to be processed according to at least one of the company category, business category, etc., to obtain business data under different categories, and finally use multiple threads to process business data under different categories in parallel, thereby improving the processing efficiency and accuracy of business data.

[0070] 104. Determine the business processing types corresponding to the business data under different categories, and determine the business processing strategies corresponding to the business processing types.

[0071] Among them, business processing types include: data preprocessing, data loading, sensitive data identification, data standardization, data calculation, data sorting, etc.; data preprocessing corresponds to data preprocessing strategy, data loading corresponds to data loading strategy, sensitive data identification corresponds to sensitive data identification strategy, data standardization corresponds to standardization strategy, data calculation corresponds to calculation strategy, and data sorting corresponds to sorting strategy.

[0072] 105. Use the business processing strategy to process the business data under the corresponding categories in parallel to obtain the processing results corresponding to the business data under different categories.

[0073] For the embodiment of the present invention, after classifying the business data to be processed, it is also necessary to determine the business processing types corresponding to the business data under different categories. Different business processing types correspond to different business processing strategies. The preset strategy configuration table stores the business processing strategies corresponding to the different business processing types. Based on this, the business processing strategies corresponding to the business data under different categories can be determined in the preset strategy configuration table, and finally the business data under the corresponding categories are processed in parallel using different business processing strategies to obtain the processing results corresponding to the business data under different categories. For example, if the business data under categories 1, 2, and 3 all correspond to data loading strategies, then the data loading strategies are finally used to load and process the three categories of business data in parallel. If the business data under category 1 corresponds to the data loading strategy, the business data under category 2 corresponds to the standardization strategy, and the business data under category 3 corresponds to the sensitive data identification strategy, then the above three business processing strategies are used to process the business data under the corresponding categories in parallel. Therefore, by classifying business data greater than a certain amount of data, business data under different categories are obtained, and the processing strategies corresponding to the business data under different categories are determined. Finally, the processing strategies are used to perform parallel processing on the business data under the corresponding categories, that is, a large amount of business data is divided into multiple groups of small amounts of business data, and multiple processing threads are used to process multiple groups of business data in parallel, avoiding jams and damage to processing resources, thereby improving the processing efficiency and accuracy of business data.

[0074] According to a method for processing insurance business data provided by the present invention, compared with the current method of processing data by centralized processing, the present invention obtains the business data to be processed; and determines whether the data volume of the business data to be processed is greater than a preset threshold; if the data volume is greater than the preset threshold, the business data to be processed is classified to obtain business data under different categories; then the business processing types corresponding to the business data under the different categories are determined, and the business processing strategies corresponding to the business processing types are determined; finally, the business data under the corresponding categories are processed in parallel using the business processing strategies to obtain the processing results corresponding to the business data under the different categories. Thus, by classifying business data greater than a certain data volume, business data under different categories are obtained, and the processing strategies corresponding to the business data under different categories are determined, and finally the processing strategies are used to process the business data under the corresponding categories in parallel, that is, a large amount of business data is divided into multiple groups of small business data, and multiple processing threads are used to process multiple groups of business data in parallel, avoiding the jamming and damage of processing resources, thereby improving the processing efficiency and accuracy of business data.

[0075] Further, in order to better illustrate the above process of processing insurance business data, as a refinement and extension of the above embodiment, the embodiment of the present invention provides another method for processing insurance business data, such as Figure 2 As shown, the method includes:

[0076] 201. Obtain business data to be processed.

[0077] For the embodiments of the present invention, after obtaining the business data to be processed, in order to ensure the accuracy of the business data to be processed, it is first necessary to perform accuracy verification on the business data to be processed. Based on this, the method includes: determining the compressed package corresponding to the original business data to be processed, and determining the original hash value corresponding to the original business data to be processed; decompressing the compressed package to obtain the business data to be verified; calculating the actual hash value corresponding to the business data to be verified; if the original hash value is equal to the actual hash value, determining the business data to be verified as the business data to be processed; if the original hash value is not equal to the actual hash value, deleting the business data to be verified, and re-decompressing the compressed package.

[0078] Specifically, the business data to be processed is transmitted to the business data processing system in the form of a compressed package. The original business data to be processed is the business data before compression. In order to verify the accuracy of the business data to be processed, it is first necessary to calculate the hash value corresponding to the original business data to be processed before compression. The hash value can specifically be an MD5 value. The original business data to be processed is then compressed to obtain a compressed package, which is then transmitted to the business data processing system. After receiving the compressed package, the business data processing system decompresses the compressed package to obtain the decompressed business data to be verified, and then calculates the hash value corresponding to the business data to be verified. If the hash value of the data before decompression is the same as the hash value of the data after decompression, it is confirmed. The data to be verified is determined to be the business data to be processed that needs to be obtained, and then the business data to be processed is processed in parallel. If the hash value of the data before decompression is different from the hash value of the data after decompression, there may be a problem in the decompression process. At this time, the decompressed data to be verified is discarded, and the compressed package is decompressed again. If the hash value of the data to be verified after decompression for a preset number of times is still different from the hash value of the data before decompression, the compressed package is discarded, and the compressed package corresponding to the original business data is obtained again until the hash value of the data before decompression is the same as the hash value of the data after decompression. In this way, the accuracy of the business data to be processed can be verified by verifying the hash value, thereby avoiding wasted time and cost caused by processing erroneous data.

[0079] 202. Determine whether the amount of the service data to be processed is greater than a preset threshold.

[0080] 203. If the data volume is greater than a preset threshold, the business data to be processed is classified to obtain business data in different categories.

[0081] Specifically, if the amount of business data to be processed is greater than a preset threshold, the business data to be processed is classified according to at least one of the company category and business category to obtain business data under different categories. In the process of classifying the business data to be processed, classification can be performed based on the data model. The data model is designed to be three layers, namely: the table layer, the write area integration layer, and the read area integration layer. Among them, the table layer user decompresses and verifies the data to ensure the accuracy of the data. The write area integration layer and the read area integration layer are above the table layer, and the business data is verified and classified according to the idea of ​​dimensional modeling, ultimately achieving the goal of improving processing efficiency and accuracy.

[0082] 204. Determine the business processing types corresponding to the business data under different categories, and determine the business processing strategies corresponding to the business processing types.

[0083] For the embodiment of the present invention, the business processing types corresponding to business data under different categories are pre-set, and then the business processing strategies corresponding to the different business processing types are determined in the preset strategy configuration table. Finally, different business processing strategies process the business data under the corresponding categories in parallel.

[0084] 205. Using a preset distributed storage method, business data of different categories are stored in a preset distributed database.

[0085] Among them, the storage process of storing business data under different categories using a preset distributed storage method is: determining the category data volume corresponding to the business data under the different categories, and based on the category data volume, determining the number of storage nodes in the preset distributed database, and storing the business data under the different categories in the storage nodes with the corresponding number of nodes.

[0086] For the embodiments of the present invention, in order to ensure the security and privacy of data, the transmission process and storage process of data in the business data processing system should be in encrypted form. Based on this, the method includes: calling the encryption key generated by the preset key management platform encryption service; using the preset group encryption algorithm to symmetrically encrypt the business data under the different categories to obtain encrypted business data; using the preset distributed storage method to store the encrypted business data in a preset distributed database; calling the decryption key generated by the preset key management platform decryption service; using the preset group decryption algorithm to decrypt the business data under the corresponding category in the preset distributed database to obtain the decrypted business data; using the business processing strategy to perform parallel scheduling processing on the decrypted business data under the corresponding category in the preset distributed database to obtain the processing results corresponding to the business data to be processed.

[0087] Specifically, during the transmission of the business data to be processed or the business data of different categories after classification, the existing SFTP server can be replaced by an SSL security gateway and FTP server based on a domestic cryptographic algorithm to realize the encrypted transmission of the business data based on the domestic cryptographic algorithm. At the same time, the business data of different business categories need to be stored in a preset distributed database, wherein the preset distributed database can adopt an MPP (Massively Parallel Processing) architecture database. The MPP architecture database can vertically integrate the various business data of a single machine, and then horizontally connect and integrate the business data of multiple single machines. It can be said that it is vertical first and then horizontal. The MPP architecture database has a huge advantage over the traditional database in core technology, can solve the data processing performance problems of many industry users, can provide a cost-effective general computing platform for ultra-large-scale data management, and can be used to support various data warehouse systems, such as BI (Business Intelligence) systems and decision support systems, and provide services for decision analysis of upper-level applications. The MPP architecture database can save storage resources and storage costs by deploying multiple nodes for data storage. Furthermore, when storing business data under different categories, it is first necessary to call the encryption service of the key management platform to generate an encryption key, and use algorithms such as SM4 (symmetric encryption block cipher algorithm) to symmetrically encrypt the stored business data, and then return the encrypted ciphertext to the server, which stores the ciphertext in a preset distributed database. Further, when reading data, read the storage content in the preset distributed database and call the decryption service of the key management platform to decrypt the ciphertext data using algorithms such as SM4 to obtain the original text of the stored business data, and then process the decrypted original text, that is, the decrypted business data in parallel. Therefore, the business data is transmitted and stored in an encrypted form, which can avoid the leakage of business data and improve the processing security of business data.

[0088] 206. In response to processing signals corresponding to business data of different categories, the business data of corresponding categories in the preset distributed database are processed in parallel by using the business processing strategy to obtain processing results corresponding to the business data to be processed.

[0089] For the embodiments of the present invention, before performing business data processing, it is necessary to establish tasks, that is, to periodically scan the records of tasks to be generated, establish corresponding task sets according to the types, such as data verification tasks, loading tasks, preprocessing tasks, etc., and set the expected execution time of the tasks and the dependencies between tasks according to the business processing strategy. At the same time, the identifier of the record of the task to be generated is updated and changed to a generated task, etc. Furthermore, after receiving the processing signal corresponding to the business data under different categories, it is necessary to execute the task, and the specific process of executing the task is: if the business data under any category among the business data under different categories belongs to different batches of data, then determine the multi-batch processing types corresponding to the different batches of data respectively; determine the multi-batch processing strategy corresponding to the multi-batch processing type; use the multi-batch processing strategy to perform parallel processing on the corresponding batch data to obtain the processing results corresponding to the different batches of data respectively; if the business data under any category belongs to single batch data, then determine the multiple single batch processing types corresponding to the single batch data, and determine the processing order corresponding to the multiple single batch processing types; determine the single batch processing strategy corresponding to the multiple single batch processing types; based on the processing order, use the single batch processing strategy to perform serial processing on the single batch data to obtain the processing results corresponding to the single batch data.

[0090] Specifically, the data processing tasks are scanned and started at regular intervals. When each data processing task is started, it is determined whether the preceding task has been completed. If it has not been completed, the execution time of the task needs to be postponed. After the task is started, the task start time and status are changed, and the task is added to the monitoring list. After the task is executed, the task status and completion time are changed. For example, when a data preprocessing task of incremental reporting is executed at a certain point, it is determined whether the loading task of the batch incremental reporting has been completed. If it has not been completed, it is set to pending processing. If it has been completed, the data loading task is executed. Furthermore, the data processing scheduling can process different batches (multiple batches) of data under the same category in parallel according to the business processing strategy, such as parallel loading, parallel preprocessing, parallel standardization, etc., and a single batch of data is configured for serial processing. If a single batch of data contains multiple processing types, it is necessary to process the single batch of data in serial order according to the processing order of the multiple processing types to ensure the orderly processing of the single batch of data. At the same time, all daily data integration tasks and new tasks of the company can also be processed in parallel, thereby shortening the overall processing time. Among them, the process of configuring business data scheduling and processing strategies can configure the execution strategy of each task through the page, including trigger strategy, scheduling expiration strategy, blocking processing strategy, task timeout control, task failure retry, task failure alarm, routing strategy, shard broadcast task, dynamic sharding, failover, task dependency, data encryption, email alarm and other configuration contents.

[0091] Furthermore, in the process of processing the business data to be processed, in order to record the processing process in real time and prompt and correct abnormal situations, the method also includes: in the process of parallel processing of the business data under the different categories, recording parallel processing information, and storing the parallel processing information in a preset information table; checking the preset information table at preset time intervals, and judging whether any data in the business data under the different categories has a processing abnormality based on the parallel processing information recorded in the preset information table; if the processing abnormality exists, determining the abnormality level corresponding to the arbitrary data; if the abnormality level is greater than the preset level threshold, calling a preset communication tool interface, and using the preset communication tool interface to send the abnormal alarm information to the operation and maintenance personnel terminal, so that the operation and maintenance personnel of the operation and maintenance personnel terminal can adjust the processing process of the arbitrary data based on the abnormal alarm information; if the abnormality level is less than or equal to the preset level threshold, determining the abnormality handling strategy corresponding to the business processing type of the arbitrary data; and using the abnormality handling strategy to automatically adjust the processing process of the arbitrary data.

[0092] Among them, the preset level threshold is set according to actual needs, the preset time is set according to actual needs, such as 2 hours, and the preset communication tools include: WeChat, QQ, email, etc. Specifically, in the process of parallel processing of business data under different categories, the parallel processing information is recorded in real time, and the parallel processing information is stored in real time in the preset information table. The preset information table is checked at regular intervals to keep an eye on the execution status of each business data at any time. If there is a processing exception in a certain business data, an alarm message is generated, and according to the abnormality level, it is determined whether manual intervention is required or the system automatically intervenes in the abnormal situation. If the abnormality level is greater than the preset level threshold, the alarm information can be sent to the corresponding operation and maintenance personnel using communication tools such as email, and the operation and maintenance personnel will intervene in the abnormal situation according to the alarm information. Pre-processing, such as postponing execution, canceling processing, shutting down processing and other operations, if the exception level is less than or equal to the preset level threshold, the exception handling strategy corresponding to the business processing of the exception is determined, and the business processing system will automatically handle the exception according to the exception handling strategy. For example, for data preprocessing tasks, the exception handling strategy for the exception situation can be preset in advance, and the exception handling strategy can be automatically used to handle the exception situation. At the same time, in the process of exception handling, it is necessary to try to start the exception handling strategy again at a certain interval, such as 10 minutes. When the exception handling strategy does not work, an alarm is issued and manual intervention is required.

[0093] Furthermore, after the processing of business data under different categories is completed, in order to facilitate the staff to view the processing results, it is also necessary to integrate the processing results under different categories. Based on this, the method includes: using a preset dimensional modeling algorithm to integrate data of different processing results to obtain the integrated processing results corresponding to the business data to be processed.

[0094] Specifically, the preset dimensional modeling algorithm is a data modeling method in data warehouse construction. It is a logical design method for structuring data. It divides the processing results into metrics and contexts, and integrates the metrics and contexts into a unified structure to obtain the integrated processing results corresponding to the business data to be processed, which is convenient for users to query the processing results.

[0095] According to a method for processing insurance business data provided by the present invention, compared with the current method of processing data through centralized processing, the present invention obtains business data to be processed; and determines whether the data volume of the business data to be processed is greater than a preset threshold; if the data volume is greater than the preset threshold, the business data to be processed is classified to obtain business data under different categories; at the same time, the business processing types corresponding to the business data under the different categories are determined, and the business processing strategies corresponding to the business processing types are determined; then the business data under the different categories are stored in a preset distributed database using a preset distributed storage method, and finally in response to processing signals corresponding to the business data under the different categories, the business data under the corresponding categories in the preset distributed database are parallelly scheduled and processed using the business processing strategy to obtain processing results corresponding to the business data to be processed. Therefore, by classifying business data larger than a certain amount of data, business data under different categories are obtained, and the processing strategies corresponding to the business data under different categories are determined. Finally, the processing strategies are used to process the business data under the corresponding categories in parallel, that is, a large amount of business data is divided into multiple groups of small amounts of business data, and multiple processing threads are used to process multiple groups of business data in parallel, avoiding jams and damage to processing resources, thereby improving the processing efficiency and accuracy of business data. At the same time, by distributing and storing business data in a database, the cost of large-scale data storage can be effectively reduced while maintaining efficient data access and processing capabilities.

[0096] Further, as Figure 1 The specific implementation of the present invention provides a device for processing insurance business data, such as Figure 3 As shown, the device includes: an acquisition unit 31, a judgment unit 32, a classification unit 33, a determination unit 34 and a processing unit 35.

[0097] The acquisition unit 31 may be used to acquire the service data to be processed.

[0098] The determining unit 32 may be configured to determine whether the amount of the service data to be processed is greater than a preset threshold.

[0099] The classification unit 33 may be configured to classify the service data to be processed to obtain service data in different categories if the data volume is greater than a preset threshold.

[0100] The determining unit 34 may be configured to determine the service processing types corresponding to the service data of different categories, and determine the service processing strategies corresponding to the service processing types.

[0101] The processing unit 35 may be used to perform parallel processing on the business data of corresponding categories using the business processing strategy to obtain processing results corresponding to the business data of different categories.

[0102] In specific application scenarios, in order to store business data under different categories, such as Figure 4 As shown, the device further comprises a storage unit 36 ​​.

[0103] The storage unit 36 ​​can be used to store the business data of different categories in a preset distributed database using a preset distributed storage method, wherein the storage process is: determining the category data volume corresponding to the business data of different categories, and based on the category data volume, determining the number of storage nodes in the preset distributed database, and storing the business data of different categories in storage nodes with the corresponding number of nodes,

[0104] The processing unit 35 can be specifically used to respond to processing signals corresponding to business data under different categories, use the business processing strategy to parallel schedule the business data under the corresponding category in the preset distributed database, and obtain processing results corresponding to the business data to be processed.

[0105] In a specific application scenario, in order to encrypt the service data, the device further includes an encryption unit 37 .

[0106] The encryption unit 37 can be used to call the encryption key generated by the encryption service of the preset key management platform; use the preset group encryption algorithm to symmetrically encrypt the business data under the different categories to obtain encrypted business data.

[0107] The storage unit 36 ​​may be specifically configured to store the encrypted business data in a preset distributed database using a preset distributed storage method.

[0108] The processing unit 35 can be specifically used to call the decryption key generated by the preset key management platform decryption service; use the preset group decryption algorithm to decrypt the business data under the corresponding category in the preset distributed database to obtain the decrypted business data; use the business processing strategy to perform parallel scheduling processing on the decrypted business data under the corresponding category in the preset distributed database to obtain the processing results corresponding to the business data to be processed.

[0109] In a specific application scenario, in order to perform data verification on the service data to be processed, the device further includes: a verification unit 38 .

[0110] The verification unit 38 can be used to determine the compressed package corresponding to the original business data to be processed, and determine the original hash value corresponding to the original business data to be processed; decompress the compressed package to obtain the business data to be verified; calculate the actual hash value corresponding to the business data to be verified; if the original hash value is equal to the actual hash value, the business data to be verified is determined as the business data to be processed; if the original hash value is not equal to the actual hash value, the business data to be verified is deleted, and the compressed package is decompressed again.

[0111] In a specific application scenario, in order to integrate the processed data, the device further includes an integration unit 39 .

[0112] The integration unit 39 may be used to perform data integration on different processing results using a preset dimensional modeling algorithm to obtain an integrated processing result corresponding to the business data to be processed.

[0113] In a specific application scenario, in order to perform parallel processing on business data of different categories, the processing unit 35 includes a determination module 351 and a processing module 352 .

[0114] The determination module 351 may be configured to determine the multiple batch processing types corresponding to the different batches of data if the business data of any category among the business data of different categories belongs to different batches of data.

[0115] The determination module 351 may also be used to determine a multi-batch processing strategy corresponding to the multi-batch processing type.

[0116] The processing module 352 may be used to perform parallel processing on corresponding batches of data using the multi-batch processing strategy to obtain processing results corresponding to different batches of data.

[0117] The determination module 351 may also be used to determine multiple single batch processing types corresponding to the single batch data if the business data under any category belongs to single batch data, and determine the processing order corresponding to the multiple single batch processing types.

[0118] The determination module 351 may also be used to determine single batch processing strategies corresponding to the multiple single batch processing types.

[0119] The processing module 352 may also be used to perform serial processing on the single batch of data based on the processing sequence and using the single batch processing strategy to obtain processing results corresponding to the single batch of data.

[0120] In a specific application scenario, in order to adjust the exception processing situation, the storage unit 36 ​​can also be used to check the preset information table at preset time intervals, and based on the parallel processing information recorded in the preset information table, determine whether any data in the business data under the different categories has any processing exception.

[0121] The determining unit 34 may also be configured to determine an abnormality level corresponding to the arbitrary data if the processing abnormality exists.

[0122] The processing unit 35 may also be used to call a preset communication tool interface if the abnormal level is greater than a preset level threshold, and use the preset communication tool interface to send the abnormal alarm information to the operation and maintenance personnel terminal, so that the operation and maintenance personnel of the operation and maintenance personnel terminal can adjust the processing process of the arbitrary data based on the abnormal alarm information.

[0123] The determining unit 34 may also be configured to determine an exception handling strategy corresponding to the business processing type of the arbitrary data if the exception level is less than or equal to a preset level threshold.

[0124] The processing unit 35 may also be used to automatically modify the processing process of the arbitrary data using the exception handling strategy.

[0125] It should be noted that for other corresponding descriptions of the functional modules involved in the insurance business data processing device provided in the embodiment of the present invention, reference can be made to Figure 1 The corresponding description of the method shown will not be repeated here.

[0126] Based on the above Figure 1The method shown, accordingly, an embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the following steps are implemented: obtaining business data to be processed; determining whether the data volume of the business data to be processed is greater than a preset threshold; if the data volume is greater than the preset threshold, classifying the business data to be processed to obtain business data under different categories; determining the business processing types corresponding to the business data under the different categories, and determining the business processing strategies corresponding to the business processing types; using the business processing strategies to perform parallel processing on the business data under the corresponding categories, to obtain processing results corresponding to the business data under the different categories.

[0127] Based on the above Figure 1 The method shown and Figure 3 The embodiment of the device shown in the figure, the embodiment of the present invention also provides a physical structure diagram of a computer device, such as Figure 5 As shown, the computer device includes: a processor 41, a memory 42, and a computer program stored in the memory 42 and executable on the processor, wherein the memory 42 and the processor 41 are both arranged on a bus 43, and the processor 41 implements the following steps when executing the program: obtaining the business data to be processed; determining whether the data volume of the business data to be processed is greater than a preset threshold; if the data volume is greater than the preset threshold, classifying the business data to be processed to obtain business data under different categories; determining the business processing types corresponding to the business data under the different categories, and determining the business processing strategies corresponding to the business processing types; using the business processing strategies to process the business data under the corresponding categories in parallel, and obtaining the processing results corresponding to the business data under the different categories.

[0128] Through the technical solution of the present invention, the present invention obtains the business data to be processed; and determines whether the data volume of the business data to be processed is greater than a preset threshold; if the data volume is greater than the preset threshold, the business data to be processed is classified to obtain business data under different categories; then the business processing types corresponding to the business data under the different categories are determined, and the business processing strategies corresponding to the business processing types are determined; finally, the business data under the corresponding categories are processed in parallel using the business processing strategies to obtain the processing results corresponding to the business data under the different categories. Thus, by classifying business data greater than a certain data volume, business data under different categories are obtained, and the processing strategies corresponding to the business data under different categories are determined, and finally, the processing strategies are used to process the business data under the corresponding categories in parallel, that is, a large amount of business data is divided into multiple groups of small amounts of business data, and multiple processing threads are used to process multiple groups of business data in parallel, avoiding the jamming and damage of processing resources, thereby improving the processing efficiency and accuracy of business data.

[0129] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0130] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for processing insurance business data, characterized in that: include: Obtain business data to be processed; Determine whether the amount of the service data to be processed is greater than a preset threshold; If the data volume is greater than a preset threshold, classifying the service data to be processed to obtain service data under different categories; Determine the business processing types corresponding to the business data under the different categories, and determine the business processing strategies corresponding to the business processing types; The business processing strategy is used to process the business data under the corresponding categories in parallel to obtain the processing results corresponding to the business data under the different categories.

2. The method according to claim 1, characterized in that Before the business data of corresponding categories are processed in parallel by using the business processing strategy to obtain the processing results corresponding to the business data of different categories, the method further includes: The business data under the different categories are stored in a preset distributed database using a preset distributed storage method, wherein the storage process is: determining the category data volume corresponding to the business data under the different categories, and based on the category data volume, determining the number of storage nodes in the preset distributed database, and storing the business data under the different categories in storage nodes with the corresponding number of nodes; The using the business processing strategy to process the business data of corresponding categories in parallel to obtain the processing results corresponding to the business data of different categories respectively includes: In response to processing signals corresponding to business data under different categories, the business data under corresponding categories in the preset distributed database are parallel scheduled and processed using the business processing strategy to obtain processing results corresponding to the business data to be processed.

3. The method according to claim 2, characterized in that Before storing the business data of different categories in a preset distributed database using a preset distributed storage method, the method further includes: Call the encryption key generated by the encryption service of the preset key management platform; Using a preset block encryption algorithm to symmetrically encrypt the business data under the different categories to obtain encrypted business data; The storing of the business data of different categories into a preset distributed database by using a preset distributed storage method includes: Storing the encrypted business data in a preset distributed database using a preset distributed storage method; The using the business processing strategy to perform parallel scheduling processing on the business data of the corresponding category in the preset distributed database to obtain the processing result corresponding to the business data to be processed includes: Call the decryption key generated by the preset key management platform decryption service; Decrypting the business data of the corresponding category in the preset distributed database by using a preset group decryption algorithm to obtain the decrypted business data; The business processing strategy is used to perform parallel scheduling processing on the decrypted business data under the corresponding category in the preset distributed database to obtain the processing results corresponding to the business data to be processed.

4. The method according to claim 1, characterized in that: After obtaining the service data to be processed, the method further includes: Determine a compressed package corresponding to the original business data to be processed, and determine an original hash value corresponding to the original business data to be processed; Decompressing the compressed package to obtain the service data to be verified; Calculate the actual hash value corresponding to the business data to be verified; If the original hash value is equal to the actual hash value, determining the service data to be verified as the service data to be processed; If the original hash value is not equal to the actual hash value, the business data to be verified is deleted and the compressed package is decompressed again.

5. The method according to claim 1, characterized in that After the business data of the corresponding categories are processed in parallel by using the business processing strategy to obtain the processing results corresponding to the business data of different categories, the method further includes: The preset dimensional modeling algorithm is used to perform data integration on different processing results to obtain the integrated processing results corresponding to the business data to be processed.

6. The method according to claim 1, characterized in that The using the business processing strategy to process the business data of corresponding categories in parallel to obtain the processing results corresponding to the business data of different categories respectively includes: If the business data under any category of the business data under the different categories belong to different batches of data, determining the multi-batch processing types corresponding to the different batches of data respectively; determining a multi-batch processing strategy corresponding to the multi-batch processing type; Using the multi-batch processing strategy to process the corresponding batches of data in parallel, to obtain processing results corresponding to the different batches of data respectively; If the business data under any category belongs to single batch data, then determining multiple single batch processing types corresponding to the single batch data, and determining the processing order corresponding to the multiple single batch processing types; Determining single batch processing strategies corresponding to the multiple single batch processing types; Based on the processing sequence, the single batch data is serially processed using the single batch processing strategy to obtain processing results corresponding to the single batch data.

7. The method according to claim 1, characterized in that The method further comprises: In the process of parallel processing of the business data under the different categories, recording parallel processing information, and storing the parallel processing information in a preset information table; Checking the preset information table at preset intervals, and judging whether any data in the business data of different categories has a processing abnormality based on the parallel processing information recorded in the preset information table; If the processing exception exists, determining the exception level corresponding to the arbitrary data; If the abnormal level is greater than a preset level threshold, a preset communication tool interface is called, and the abnormal alarm information is sent to an operation and maintenance personnel terminal by using the preset communication tool interface, so that the operation and maintenance personnel of the operation and maintenance personnel terminal can adjust the processing process of the arbitrary data based on the abnormal alarm information; If the abnormality level is less than or equal to a preset level threshold, determining an abnormality handling strategy corresponding to the business processing type of the arbitrary data; The exception handling strategy is used to automatically adjust the processing process of the arbitrary data.

8. A device for processing insurance business data, characterized in that: include: An acquisition unit, used for acquiring business data to be processed; A judging unit, used to judge whether the amount of the service data to be processed is greater than a preset threshold; A classification unit, configured to classify the service data to be processed to obtain service data of different categories if the data volume is greater than a preset threshold; A determination unit, configured to determine the business processing types corresponding to the business data under the different categories, and determine the business processing strategies corresponding to the business processing types; The processing unit is used to use the business processing strategy to perform parallel processing on the business data under the corresponding categories to obtain processing results corresponding to the business data under the different categories.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images