Data updating method and device, electronic equipment and computer readable medium

Abstract

The invention discloses a data updating method and device, electronic equipment and a computer readable medium, and relates to the technical field of computers.The specific implementation mode comprises the steps that a data updating request is received, and submitted data and a corresponding application system interface identifier are obtained; matching the submitted data with a field in the target data table to obtain a matching type; performing batch division on the submitted data in response to the fact that the matching type corresponds to non-complete matching so as to obtain each batch of data; generating an asynchronous updating task based on each batch of data; and calling an application system interface corresponding to the application system interface identifier to call the thread pool to execute the asynchronous updating task so as to obtain updating data. The data updating efficiency and accuracy are improved, and the requirement for large-scale data processing can be met.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data updating method, device, electronic device and computer-readable medium. Background Art

[0002] At present, with the continuous improvement of enterprise informatization, the management and update of various data are becoming more and more important. Especially in the financial industry, enterprises need to frequently compare and update with industrial and commercial data to ensure the accuracy and timeliness of customer information. The existing data matching and updating methods are inefficient and cannot meet the needs of large-scale data processing. Summary of the invention

[0003] In view of this, the embodiments of the present application provide a data updating method, device, electronic device and computer-readable medium, which can solve the problem that the existing data matching and updating methods are inefficient and cannot meet the needs of large-scale data processing.

[0004] To achieve the above object, according to one aspect of an embodiment of the present application, a data updating method is provided, comprising:

[0005] Receive data update requests, obtain reported data and corresponding application system interface identifiers;

[0006] Match the submitted data with the fields in the target data table to obtain the matching type;

[0007] In response to the match type corresponding to a non-complete match, the submitted data is divided into batches to obtain batches of data;

[0008] Generate asynchronous update tasks based on each batch of data;

[0009] The application system interface corresponding to the application system interface identifier is called to call the thread pool to execute the asynchronous update task to obtain the updated data.

[0010] Optionally, match the reported data with fields in the target data table to obtain matching types, including:

[0011] Similarity matching is performed between the reported data and the fields in the target data table to obtain the respective similarities;

[0012] In response to a similarity among the similarities being less than a preset threshold, the match type is determined to be an incomplete match.

[0013] Optionally, the submitted data is divided into batches to obtain data for each batch, including:

[0014] Paginate the submitted data according to the preset number of paging entries to obtain paging data;

[0015] According to the number of entries in each batch set for each page, the corresponding paging data is processed in batches to obtain each batch of data.

[0016] Optionally, based on each batch of data, an asynchronous update task is generated, including:

[0017] Determine the preset update period of each batch of data, and generate a corresponding asynchronous update task based on each batch of data and the corresponding preset update period.

[0018] Optionally, perform asynchronous update tasks, including:

[0019] In response to the difference between the current time when executing the asynchronous update task and the previous update time being greater than the preset update cycle, the unified social credit code of the batch data corresponding to the asynchronous update task is used to call the external data transaction, and then the batch data corresponding to the asynchronous update task is updated according to the returned result.

[0020] Optionally, before calling the thread pool, the method further includes:

[0021] In response to the system corresponding to the thread pool not setting system parameters, obtaining the number of central processing unit cores of the system;

[0022] Set the number of core threads in the thread pool to twice the number of CPU cores, and set the maximum number of threads in the thread pool to four times the number of CPU cores.

[0023] In addition, the present application also provides a data updating device, including:

[0024] A receiving unit is configured to receive a data update request and obtain the reported data and a corresponding application system interface identifier;

[0025] A matching unit is configured to match the reported data with fields in a target data table to obtain a matching type;

[0026] A batch division unit is configured to divide the submitted data into batches in response to the matching type corresponding to a non-complete match, so as to obtain batches of data;

[0027] The task generation unit is configured to generate asynchronous update tasks based on each batch of data;

[0028] The execution unit is configured to call the application system interface corresponding to the application system interface identifier to call the thread pool to execute the asynchronous update task to obtain the update data.

[0029] Optionally, the matching unit is further configured to:

[0030] Similarity matching is performed between the reported data and the fields in the target data table to obtain the respective similarities;

[0031] In response to a similarity among the similarities being less than a preset threshold, the match type is determined to be an incomplete match.

[0032] Optionally, the batch division unit is further configured to:

[0033] Paginate the submitted data according to the preset number of paging entries to obtain paging data;

[0034] According to the number of entries in each batch set for each page, the corresponding paging data is processed in batches to obtain each batch of data.

[0035] Optionally, the task generation unit is further configured to:

[0036] Determine the preset update period of each batch of data, and generate a corresponding asynchronous update task based on each batch of data and the corresponding preset update period.

[0037] Optionally, the execution unit is further configured to:

[0038] In response to the difference between the current time when executing the asynchronous update task and the previous update time being greater than the preset update cycle, the unified social credit code of the batch data corresponding to the asynchronous update task is used to call the external data transaction, and then the batch data corresponding to the asynchronous update task is updated according to the returned result.

[0039] Optionally, the device further comprises a setting unit configured to:

[0040] In response to the system corresponding to the thread pool not setting system parameters, obtaining the number of central processing unit cores of the system;

[0041] Set the number of core threads in the thread pool to twice the number of CPU cores, and set the maximum number of threads in the thread pool to four times the number of CPU cores.

[0042] In addition, the present application also provides a data updating electronic device, including: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by one or more processors, the one or more processors implement the data updating method as described above.

[0043] In addition, the present application also provides a computer-readable medium on which a computer program is stored, and when the program is executed by a processor, the data updating method as described above is implemented.

[0044] To achieve the above objective, according to another aspect of the embodiments of the present application, a computer program product is provided.

[0045] A computer program product according to an embodiment of the present application includes a computer program, and when the program is executed by a processor, the data updating method provided by the embodiment of the present application is implemented.

[0046] One embodiment of the above invention has the following advantages or beneficial effects: the application receives a data update request to obtain the submitted data and the corresponding application system interface identifier; matches the submitted data with the fields in the target data table to obtain a matching type; in response to the matching type corresponding to a non-complete match, the submitted data is divided into batches to obtain each batch of data; based on each batch of data, an asynchronous update task is generated; and the application system interface corresponding to the application system interface identifier is called to call the thread pool to execute the asynchronous update task to obtain the updated data. The data update efficiency and accuracy are improved to meet the needs of large-scale data processing.

[0047] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings are used to better understand the present application and do not constitute an improper limitation on the present application.

[0049] Figure 1 is a schematic diagram of the main process of a data updating method according to an embodiment of the present application;

[0050] Figure 2 is a schematic diagram of the main process of a data updating method according to an embodiment of the present application;

[0051] Figure 3 It is a main flow chart of a data updating method according to an embodiment of the present application;

[0052] Figure 4 is a schematic diagram of main units of a data updating device according to an embodiment of the present application;

[0053] Figure 5 is an exemplary system architecture diagram to which the embodiments of the present application can be applied;

[0054] Figure 6 It is a structural diagram of a computer system of a terminal device or server suitable for implementing an embodiment of the present application. DETAILED DESCRIPTION

[0055] The following is a description of the exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, and they should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of known functions and structures is omitted in the following description. It should be noted that the acquisition, transmission, storage, use, and processing of data in the technical solution of the present application are in compliance with the relevant provisions of national laws and regulations. It should be noted that in the embodiments of the present application, certain software, components, models, and other existing solutions in the industry may be mentioned, which should be considered as exemplary, and their purpose is only to illustrate the feasibility of the implementation of the technical solution of the present application, but it does not mean that the applicant has or must have used the solution. In the technical solution of the present application, the collection, analysis, use, transmission, storage, and other aspects of the user's personal information involved are in compliance with the provisions of relevant laws and regulations, and are used for legal and reasonable purposes, and are not shared, leaked, or sold outside of these legal uses, and are subject to supervision and management by regulatory authorities. Take necessary measures to prevent illegal access to such personal information, ensure that persons with access to personal information comply with relevant laws and regulations, and ensure the security of personal information. Once such personal information is no longer needed, risks should be minimized by limiting or even prohibiting data collection and / or deleting data.

[0056] When used, including in certain related applications, user privacy is protected by de-identifying data, such as by removing specific identifiers when used, controlling the amount or specificity of data stored, controlling how data is stored, and / or other methods of de-identification.

[0057] Figure 1 is a schematic diagram of the main process of a data updating method according to an embodiment of the present application, such as Figure 1 As shown, the data updating method mainly includes the following steps S101 to S105.

[0058] Step S101, receiving a data update request, obtaining reporting data and a corresponding application system interface identifier.

[0059] In this embodiment, the execution subject of the data update method (for example, a server) can receive a data update request through a wired connection or a wireless connection. The data to be updated in the data update request can be industrial and commercial data. After receiving the data update request, the execution subject can obtain the reported data and the application system interface identifier. The reported data can be the reported industrial and commercial data. The application system interface identifier is used to characterize the application system used to process asynchronous tasks and call external data interfaces.

[0060] Step S102, matching the submitted data with the fields in the target data table to obtain a matching type.

[0061] The target data table may be, for example, an industrial and commercial data table. After obtaining the submitted data, the execution entity may match the submitted data with each field in the target data table (for example, an industrial and commercial data table) to obtain a matching type, wherein the matching type may include a complete match and an incomplete match.

[0062] If the match type is a complete match, the matching status of the reported data will be updated directly.

[0063] Step S103, in response to the matching type corresponding to an incomplete match, the submitted data is divided into batches to obtain batches of data.

[0064] If the match type is not a complete match, the reported data will be inserted into the update table corresponding to the industrial and commercial data to achieve accurate and efficient data updates.

[0065] Specifically, the submitted data is divided into batches to obtain each batch of data, including: paging the submitted data according to a preset number of paging items to obtain paging data; batch processing the corresponding paging data according to the number of items in each batch set per page to obtain each batch of data.

[0066] If the match type is non-complete match, the reported data will be divided into batches. Specifically, it can be based on the preset number of paging items, for example, 1,000 data items per page, and paging can be performed first to obtain each page of data. Then, each page of data can be processed in batches. Specifically, it can be based on the number of items in each batch set per page, for example, 100 data items per batch, and then batch processing can be performed to obtain each batch of data, so as to update the data based on each batch of data to speed up the data update.

[0067] Step S104: Generate an asynchronous update task based on each batch of data.

[0068] Specifically, based on each batch of data, an asynchronous update task is generated, including: determining a preset update period of each batch of data, and generating a corresponding asynchronous update task based on each batch of data and the corresponding preset update period.

[0069] The execution entity can determine the preset update cycle based on the preset update frequency, and then combine the preset update cycle with each batch of data in the form of parameters to obtain each combined data, and generate corresponding asynchronous update tasks based on each combined data, thereby improving the accuracy of asynchronous update task execution.

[0070] Step S105 , calling the application system interface corresponding to the application system interface identifier to call the thread pool to execute the asynchronous update task to obtain update data.

[0071] Specifically, the asynchronous update task is performed, including:

[0072] In response to the difference between the current time when executing the asynchronous update task and the previous update time being greater than the preset update cycle, the unified social credit code of the batch data corresponding to the asynchronous update task is used to call the external data transaction, and then the batch data corresponding to the asynchronous update task is updated according to the returned result.

[0073] By using external data transactions to obtain the latest industrial and commercial data to update the batch data when the difference between the current time and the previous update time when executing the asynchronous update task is greater than the preset update cycle, the update of the batch data can be made more accurate.

[0074] This embodiment receives a data update request, obtains the submitted data and the corresponding application system interface identifier; matches the submitted data with the fields in the target data table to obtain a match type; in response to the match type corresponding to a non-complete match, the submitted data is divided into batches to obtain each batch of data; based on each batch of data, an asynchronous update task is generated; and the application system interface corresponding to the application system interface identifier is called to call the thread pool to execute the asynchronous update task to obtain updated data. This improves data update efficiency and accuracy, and can meet the needs of large-scale data processing.

[0075] Figure 2 is a schematic diagram of the main flow of a data updating method according to an embodiment of the present application, such as Figure 2 As shown, the data updating method mainly includes the following steps S201 to S206.

[0076] Step S201, receiving a data update request, obtaining reporting data and a corresponding application system interface identifier.

[0077] Step S202, performing similarity matching between the submitted data and the fields in the target data table to obtain respective similarities.

[0078] Specifically, each similarity obtained may be a cosine similarity. By calculating the cosine similarity between the submitted data and the fields in the target data table, the matching between the submitted data and the fields in the target data table may be made more accurate.

[0079] In some embodiments, the execution entity may split the reported data based on preset fields to obtain individual split data, wherein the preset fields may be summarized based on the field portraits in the historical data table, and the individual split data may be matched with the fields in the target data table by cosine similarity, thereby obtaining individual cosine similarities. The preset fields summarized based on the field portraits in the historical data table are used to split the reported data, which may make the reported data match the fields in the target data table more accurately.

[0080] Step S203: In response to a similarity among the similarities being less than a preset threshold, determining the matching type as an incomplete match.

[0081] If all the obtained cosine similarities are greater than the preset threshold, the match type is determined to be a complete match; if one or more of the similarities are less than the preset threshold, the match type is determined to be an incomplete match.

[0082] Step S204, in response to the matching type corresponding to an incomplete match, the submitted data is divided into batches to obtain batches of data.

[0083] When the match type is non-complete match, the execution entity can divide the submitted data into batches according to the configured preset data size to obtain each batch of data, thereby achieving flexibility in batch division.

[0084] Step S205: Generate an asynchronous update task based on each batch of data.

[0085] The execution subject can set the update time and outbound call address identifier for each batch of data, and generate an asynchronous update task by combining the update time, outbound call address identifier and the corresponding batch data, thereby ensuring the accuracy of the asynchronous update task execution.

[0086] By using the corresponding outbound call address identifier to obtain the corresponding external data transaction when the difference between the current time and the previous update time when executing the asynchronous update task is greater than the preset update period to obtain the latest industrial and commercial data for updating the batch data, the update of the batch data can be made more accurate.

[0087] Step S206, calling the application system interface corresponding to the application system interface identifier to call the thread pool to execute the asynchronous update task to obtain the update data.

[0088] Specifically, before calling the thread pool, the method also includes: in response to the system corresponding to the thread pool not setting system parameters, obtaining the number of central processing unit cores of the system; setting the number of core threads of the thread pool to twice the number of central processing unit cores, and setting the maximum number of threads of the thread pool to four times the number of central processing unit cores.

[0089] For example, create a processing thread pool, set the core thread number and maximum thread number of the thread pool according to the system parameters and the number of CPU cores of the system. If the system parameters set these two parameters, they are set according to the system parameters. If the system parameters are not set, the core thread number is twice the number of cores, and the maximum thread number is four times the number of cores.

[0090] After generating the asynchronous update task, the execution subject can call the application system interface corresponding to the application system interface identifier to call the created processing thread pool to execute the asynchronous update task, so as to accurately obtain the updated data.

[0091] Figure 3 FIG. 1 is a schematic diagram of the main flow of a data updating method according to an embodiment of the present application. Figure 3 In the embodiment of the present application, ETL system: data extraction, transformation and loading system, used to extract data from the source data system and load it into the target data system. AP system: application system, used to process asynchronous tasks and call external data interfaces. The embodiment of the present application realizes the matching and efficient update of batch industrial and commercial data through the following steps: The ETL system receives the reported data, including fields such as the unified social credit code and customer name. The reported data is matched with the fields in the industrial and commercial data table, and the matching results are divided into complete match and incomplete match (including customer name mismatch, unified social credit code mismatch and complete mismatch). The complete matching data directly updates the matching status of the reported data. The result data of incomplete matching is inserted into the update table. Specifically, if the customer name does not match, the customer name is saved to the update table; if the unified social credit code does not match, the unified social credit code is saved to the update table; if there is no complete match, two data are generated according to the above logic. If it is not a complete match, an asynchronous task is generated and the asynchronous task number is updated to the update table. After the AP system monitors the asynchronous task, it calls out to request a transaction, that is, calls the external data transaction according to the customer name or unified social credit code to obtain the latest industrial and commercial data. Determine whether the corresponding data is found based on the returned results. If found, update the industrial and commercial data, that is, update it to the corresponding update table of the industrial and commercial data, and modify the update time of the industrial and commercial data. After the data update is completed, update the asynchronous task status and notify the ETL system. After the ETL system receives the update notification of the asynchronous task status, it re-matches according to the latest industrial and commercial data, obtains the final matching status, and updates it to the matching status of the reported data. The asynchronous project checks the difference between the current time when the asynchronous update task is executed and the previous update time at a fixed frequency. If the time limit is exceeded (the time limit can be a preset update cycle), an outbound call request is sent, that is, the unified social credit code of the data is used to call the external data transaction, and then the data is updated according to the returned results. If the time limit is not exceeded, the process is terminated.

[0092] The update strategy of the embodiment of the present application is as follows: create a processing thread pool, and set the core thread number and the maximum thread number of the thread pool according to the system parameters and the number of CPU cores of the system. If the system parameters set these two parameters, they are set according to the system parameters. If the system parameters are not set, the core thread number is twice the number of cores, and the maximum thread number is four times the number of cores. In order to cope with large-scale processing, a paging query method is used to gradually obtain data, and the data of each page is processed in batches to ensure that the amount of data queried and processed each time is moderate, avoiding memory overflow and processing bottlenecks. Paging query: Perform paging query according to the set paging size (such as 1,000 data per page), and obtain data page by page for processing. Batch processing: The batch size set for each page of data (such as 100 data per batch) is processed in batches, submitted to the thread pool for parallel processing, and external transactions are requested to obtain the latest industrial and commercial data. Batch update: Poll and listen to the batches that have been processed, obtain the industrial and commercial data of the batch for batch update.

[0093] The embodiments of the present application can improve data processing efficiency: through the automated matching and updating process, the efficiency of data processing is significantly improved, and it is suitable for large-scale data processing scenarios. Ensure data accuracy: the automatic matching and updating process reduces errors caused by human operations and ensures data accuracy. Improve data timeliness: the regular inspection and real-time update mechanism ensures the timeliness of industrial and commercial data and improves the reliability of business decisions and operations.

[0094] The embodiment of the present application realizes the automatic matching and efficient updating of industrial and commercial data through the combination of ETL system and AP system. ETL system: data extraction, conversion and loading system, used to extract data from the source data system and load it into the target data system. AP system: application system, used to process asynchronous tasks and call external data interfaces. The ETL system is used to automatically match the reported data and industrial and commercial data, and classify the matching results. The asynchronous task mechanism of the AP system is used to realize efficient data update and status notification. Regularly check the update time of industrial and commercial data, automatically generate update tasks, and ensure the timeliness of data.

[0095] Figure 4 Schematic diagram of the main units of the data updating device according to the embodiment of the present application. Figure 4 As shown, the data updating device 400 includes a receiving unit 401 , a matching unit 402 , a batch dividing unit 403 , a task generating unit 404 and an executing unit 405 .

[0096] The receiving unit 401 is configured to receive a data update request and obtain the reported data and the corresponding application system interface identifier.

[0097] The matching unit 402 is configured to match the submitted data with the fields in the target data table to obtain a matching type.

[0098] The batch division unit 403 is configured to divide the submitted data into batches in response to the matching type corresponding to an incomplete match, so as to obtain batches of data.

[0099] The task generating unit 404 is configured to generate an asynchronous update task based on each batch of data.

[0100] The execution unit 405 is configured to call the application system interface corresponding to the application system interface identifier to call the thread pool to execute the asynchronous update task to obtain the update data.

[0101] In some embodiments, the matching unit 402 is further configured to: perform similarity matching between the reported data and the fields in the target data table to obtain various similarities; in response to a similarity among the various similarities being less than a preset threshold, determine the matching type as an incomplete match.

[0102] In some embodiments, the batch division unit 403 is further configured to: paginate the reported data according to a preset number of paging entries to obtain paging data; and batch process the corresponding paging data according to the number of entries in each batch set per page to obtain batches of data.

[0103] In some embodiments, the task generation unit 404 is further configured to: determine a preset update period for each batch of data, and generate a corresponding asynchronous update task based on each batch of data and the corresponding preset update period.

[0104] In some embodiments, the execution unit 405 is further configured to: in response to the difference between the current time when executing the asynchronous update task and the previous update time being greater than a preset update cycle, use the unified social credit code of the batch data corresponding to the asynchronous update task to call an external data transaction, and then update the batch data corresponding to the asynchronous update task according to the returned result.

[0105] In some embodiments, the data updating device further includes Figure 4 The setting unit not shown in the figure is configured to: in response to the system corresponding to the thread pool not setting the system parameters, obtain the number of central processing unit cores of the system; set the number of core threads of the thread pool to twice the number of central processing unit cores, and set the maximum number of threads of the thread pool to four times the number of central processing unit cores.

[0106] It should be noted that the data updating method and the data updating device of the present application have a corresponding relationship in terms of specific implementation contents, so the repeated contents will not be described again.

[0107] Figure 5An exemplary system architecture 500 is shown to which the data updating method or data updating device according to the embodiment of the present application can be applied.

[0108] like Figure 5 As shown, system architecture 500 may include terminal devices 501, 502, 503, a network 504 and a server 505. Network 504 is used to provide a medium for communication links between terminal devices 501, 502, 503 and server 505. Network 504 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0109] Users can use terminal devices 501, 502, 503 to interact with server 505 through network 504 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 501, 502, 503, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).

[0110] The terminal devices 501 , 502 , and 503 may be various electronic devices having a data update processing screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0111] Server 505 can be a server that provides various services, such as a background management server that provides support for data update requests submitted by users using terminal devices 501, 502, and 503 (only as an example). The background management server can receive data update requests, obtain reported data and corresponding application system interface identifiers; match the reported data with fields in the target data table to obtain a match type; in response to the match type corresponding to a non-complete match, divide the reported data into batches to obtain each batch of data; generate an asynchronous update task based on each batch of data; call the application system interface corresponding to the application system interface identifier to call the thread pool to execute the asynchronous update task to obtain updated data. Improving data update efficiency and accuracy can meet the needs of large-scale data processing.

[0112] It should be noted that the data updating method provided in the embodiment of the present application is generally executed by the server 505 , and accordingly, the data updating device is generally arranged in the server 505 .

[0113] It should be understood that Figure 5 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0114] Reference below Figure 6, which shows a schematic diagram of the structure of a computer system 600 of a terminal device suitable for implementing an embodiment of the present application. Figure 6 The terminal device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0115] like Figure 6 As shown, the computer system 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the computer system 600 are also stored. The CPU 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0116] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage section 608 as needed.

[0117] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the central processing unit (CPU) 601, the above-mentioned functions defined in the system of the present application are executed.

[0118] It should be noted that the computer-readable medium shown in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. Computer-readable storage media may include, for example, but are not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0119] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0120] The units involved in the embodiments described in the present application may be implemented by software or hardware. The units described may also be set in a processor, for example, it may be described as: a processor includes a receiving unit, a matching unit, a batch division unit, a task generation unit, and an execution unit. The names of these units do not constitute limitations on the units themselves in some cases.

[0121] As another aspect, the present application also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device receives a data update request, obtains the reported data and the corresponding application system interface identifier; matches the reported data with the fields in the target data table to obtain a matching type; in response to the matching type corresponding to a non-complete match, the reported data is divided into batches to obtain each batch of data; based on each batch of data, an asynchronous update task is generated; and the application system interface corresponding to the application system interface identifier is called to call the thread pool to execute the asynchronous update task to obtain the updated data.

[0122] The computer program product of the present application includes a computer program, and when the computer program is executed by a processor, the data updating method in the embodiment of the present application is implemented.

[0123] According to the technical solution of the embodiment of the present application, the data updating efficiency and accuracy can be improved, and the needs of large-scale data processing can be met.

[0124] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. A data updating method, characterized in that: include: Receive data update requests, obtain reported data and corresponding application system interface identifiers; Matching the submitted data with fields in the target data table to obtain a matching type; In response to the match type corresponding to an incomplete match, dividing the submitted data into batches to obtain batches of data; Based on the respective batches of data, an asynchronous update task is generated; The application system interface corresponding to the application system interface identifier is called to call the thread pool to execute the asynchronous update task to obtain update data.

2. The method according to claim 1, characterized in that The matching of the submitted data with the fields in the target data table to obtain a matching type includes: Performing similarity matching between the reported data and the fields in the target data table to obtain respective similarities; In response to a similarity among the various similarities being less than a preset threshold, the match type is determined to be an incomplete match.

3. The method according to claim 1, characterized in that The reported data is divided into batches to obtain data of each batch, including: Paging the reported data according to a preset number of paging entries to obtain paging data; According to the number of entries in each batch set for each page, the corresponding paging data is processed in batches to obtain each batch of data.

4. The method according to claim 1, characterized in that: The step of generating an asynchronous update task based on each batch of data includes: The preset update period of each batch of data is determined, and the corresponding asynchronous update task is generated based on each batch of data and the corresponding preset update period.

5. The method according to claim 1, characterized in that The executing the asynchronous update task includes: In response to the difference between the current time when executing the asynchronous update task and the previous update time being greater than the preset update cycle, the unified social credit code of the batch data corresponding to the asynchronous update task is used to call an external data transaction, and then the batch data corresponding to the asynchronous update task is updated according to the returned result.

6. The method according to claim 1, characterized in that Before calling the thread pool, the method further includes: In response to the system corresponding to the thread pool not setting system parameters, obtaining the number of central processing unit cores of the system; The number of core threads in the thread pool is set to be twice the number of cores in the central processing unit, and the maximum number of threads in the thread pool is set to be four times the number of cores in the central processing unit.

7. A data updating device, characterized in that: include: A receiving unit is configured to receive a data update request and obtain the reported data and a corresponding application system interface identifier; A matching unit, configured to match the reported data with fields in a target data table to obtain a matching type; a batch division unit, configured to divide the submitted data into batches in response to the matching type corresponding to an incomplete match, so as to obtain batches of data; A task generation unit is configured to generate an asynchronous update task based on each batch of data; The execution unit is configured to call the application system interface corresponding to the application system interface identifier to call the thread pool to execute the asynchronous update task to obtain the update data.

8. The device according to claim 7, characterized in that The matching unit is further configured to: Performing similarity matching between the reported data and the fields in the target data table to obtain respective similarities; In response to a similarity among the various similarities being less than a preset threshold, the match type is determined to be an incomplete match.

9. The device according to claim 7, characterized in that The batch division unit is further configured to: Paging the reported data according to a preset number of paging entries to obtain paging data; According to the number of entries in each batch set for each page, the corresponding paging data is processed in batches to obtain each batch of data.

10. The device according to claim 7, characterized in that The task generation unit is further configured to: The preset update period of each batch of data is determined, and the corresponding asynchronous update task is generated based on each batch of data and the corresponding preset update period.

11. The device according to claim 7, characterized in that The execution unit is further configured to: In response to the difference between the current time when executing the asynchronous update task and the previous update time being greater than the preset update cycle, the unified social credit code of the batch data corresponding to the asynchronous update task is used to call an external data transaction, and then the batch data corresponding to the asynchronous update task is updated according to the returned result.

12. The device according to claim 7, characterized in that The device also includes a setting unit configured to: In response to the system corresponding to the thread pool not setting system parameters, obtaining the number of central processing unit cores of the system; The number of core threads in the thread pool is set to be twice the number of cores in the central processing unit, and the maximum number of threads in the thread pool is set to be four times the number of cores in the central processing unit.

13. A data updating electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

14. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images