Data processing method, electronic device, readable medium and program product

Abstract

The present disclosure relates to a data processing method, an electronic device, a readable medium and a program product. Before executing the ownership transfer operation, an update rollback operation can be performed on the first data update requirement to obtain the first data update requirement at the generation time, and after executing the ownership transfer operation according to the first data update requirement, the second data update requirement can be updated according to the first update record. Thus, while ensuring the accuracy of the resource data, operations such as resource return can be performed on the unreturned resources within the time period from the generation time to the first query time. On the one hand, the flexibility of processing resource data can be improved, and on the other hand, the business scenarios of the ownership transfer operation can be expanded, that is, in the business scenario where a single financial institution is used as a resource provider to provide resources to the target object applying for resources, the data processing method can also be applied, further improving the flexibility of data processing.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular, to a data processing method, an electronic device, a readable medium, and a program product. Background Art

[0002] In the related art, a resource provider can provide the held resources to the target object, and after providing the held resources to the target object, in order to improve the utilization efficiency of the resources, the provided held resources can be transferred. However, in the related art, when performing the ownership transfer operation on the provided held resources, there are problems such as inflexible and inconvenient data processing. Summary of the invention

[0003] This summary is provided to introduce concepts in a brief form that will be described in detail in the detailed description below. This summary is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0004] In a first aspect, the present disclosure provides a data processing method, the data processing method comprising: Sending the state data of the resources in the database to the first system, the state data is used by the first system to generate data to be transferred for the resources whose ownership information is to be changed, the data to be transferred includes the resource identifier, the transfer condition and the generation time of the data to be transferred; Acquire the data to be transferred from the first system, and query the first data update requirement included in the corresponding status data at the first query time according to the resource identifier; According to the first update record that the first data update requirement has occurred during the time period from the generation time to the first query time, an update rollback operation is performed on the first data update requirement to obtain the first data update requirement included in the state data corresponding to the resource identifier at the generation time; The ownership transfer operation is performed according to the first data update requirement included in the generation time and the transfer condition to change the first ownership information included in the status data corresponding to the resource identifier to the second ownership information and generate a second data update requirement, and the second data update requirement is updated according to the first update record.

[0005] In a second aspect, the present disclosure provides an electronic device, including: a storage device having a computer program stored thereon; A processing device is used to execute the computer program in the storage device to implement the steps of the method described in the first aspect.

[0006] In a third aspect, the present disclosure provides a computer-readable medium having a computer program stored thereon, which implements the steps of the method described in the first aspect when the computer program is executed by a processing device.

[0007] In a fourth aspect, the present disclosure provides a computer program product, comprising a computer program, which implements the steps of the method described in the first aspect when executed by a processor.

[0008] Through the above technical solution, before executing the ownership transfer operation, the first data update requirement can be updated and rolled back to obtain the first data update requirement at the generation time, and after executing the ownership transfer operation according to the first data update requirement, the second data update requirement can be updated according to the first update record. Thus, while ensuring the accuracy of resource data, operations such as resource return can be performed on unreturned resources within the time period from the generation time to the first query time. On the one hand, the flexibility of resource data processing can be improved, and on the other hand, the business scenarios of ownership transfer operations can be expanded, that is, in business scenarios where a single financial institution is used as a resource provider to provide resources to the target object applying for resources, the data processing method can also be applied, further improving the flexibility of data processing.

[0009] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale. In the drawings: Figure 1 is a schematic diagram of an application environment of a data processing method according to an exemplary embodiment of the present disclosure; Figure 2 is a flow chart of a data processing method according to an exemplary embodiment of the present disclosure; Figure 3 is a flow chart showing a method of obtaining data to be transferred according to an exemplary embodiment of the present disclosure; Figure 4 is a flow chart showing a method of processing data to be transferred according to an exemplary embodiment of the present disclosure; Figure 5 is a flow chart showing a method of processing data to be transferred according to an exemplary embodiment of the present disclosure; Figure 6 is a flow chart showing a method of processing data to be transferred according to an exemplary embodiment of the present disclosure; Figure 7 It is a structural block diagram of a data processing device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0011] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0012] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0013] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0014] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0015] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0016] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0017] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0018] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0019] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0020] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0021] At the same time, it is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0022] To facilitate understanding of the data processing method provided by the present disclosure, the following explains the terms involved in this embodiment: Distributed scheduled task scheduling platform: A distributed scheduled task scheduling platform is a system that can uniformly manage and execute scheduled tasks in a cluster composed of multiple servers. The distributed task scheduling platform includes a variety of scheduling strategies, and different scheduling strategies can be adopted for different business scenarios. For example, it can include a round-robin strategy: evenly distribute tasks to each executor, and each executor receives tasks in sequence to achieve load balancing. This strategy is suitable for scenarios where executor resources are similar and task processing time is similar, so as to ensure that the workload of each executor is roughly equal. Random strategy: randomly select executors to execute tasks to avoid the situation where some executors may bear too many tasks for a long time. This strategy is suitable for scenarios where the execution time of tasks is not much different and there is no strict requirement for the execution order, and it increases the randomness of task allocation. Consistent Hashing strategy: hash operations are performed based on certain attributes of the task (such as task ID) to determine which executor the task is assigned to, ensuring that tasks with the same attributes are always assigned to the same executor. This strategy is suitable for stateful tasks or tasks that require idempotence. Sharding & Broadcast strategy: Split a large task data set into multiple small data fragments (shards) and broadcast the information to all executors. Each executor claims and processes different shards in parallel. This strategy can greatly improve processing efficiency and is particularly suitable for scenarios that require rapid response and processing of large amounts of data. Failover measurement: When the preferred executor is unavailable, the task is automatically transferred to other healthy executors to ensure uninterrupted task execution. This strategy is used in systems that emphasize task execution continuity and high availability to prevent single point failures from affecting task execution. Priority scheduling strategy: Allocate executor resources according to task priority to ensure that high-priority tasks are executed first. This strategy is used to ensure the timely completion of critical tasks in systems with diverse task requirements.

[0023] Idempotence check: used to ensure that when an operation or function is executed multiple times, its result is the same as the result of executing it once, that is, multiple executions will not change the state of the system or produce additional impacts.

[0024] As mentioned in the background technology, in the related technology, when performing the ownership transfer operation on the provided holding resources, there are problems such as insufficient flexibility and convenience in data processing.

[0025] For example, in the financial field, a single financial institution can be used as a resource provider to provide resources to a target object that applies for resources. Alternatively, multiple financial institutions can be used as resource providers together, so that multiple financial institutions can jointly provide resources to the target object.

[0026] However, in the related art, when executing the ownership transfer operation, it is generally necessary to first determine the target object's unreturned resources, the data update requirements for the unreturned resources, and the transfer conditions at a first preset time (e.g., 0:00 a.m.), and then execute the ownership transfer operation based on the data update requirements and transfer conditions of the unreturned resources at a second preset time (e.g., 20:00 p.m.), to obtain the ownership information of the unreturned resources and the data update requirements for the ownership information. At the same time, in order to ensure the accuracy of resource data after the execution of the ownership transfer operation as much as possible, the unreturned resources, the data update requirements for the unreturned resources, and the transfer conditions should be kept unchanged between the first preset time and the second preset time. That is to say, between the first preset time and the second preset time, operations such as resource return for unreturned resources are generally not supported, and there are problems such as insufficient flexibility in data processing.

[0027] In addition, in a business scenario where a single financial institution acts as a resource provider and provides resources to a target object that applies for resources, since the target object can perform resource return operations at any time point, this business scenario generally does not support the execution of ownership transfer operations and has certain limitations.

[0028] In view of this, the present disclosure provides a data processing method, an electronic device, a readable medium and a program product, which obtains a first data update requirement for executing the ownership transfer operation by performing an update rollback operation on the first data update requirement before executing the ownership transfer operation, and updates the second data update requirement according to the first update record after executing the ownership transfer operation according to the first data update requirement. Thus, while ensuring the accuracy of resource data, operations such as resource return can be performed on unreturned resources within the time period from the generation time to the first query time. On the one hand, the flexibility of processing resource data can be improved, and on the other hand, the business scenarios of ownership transfer operations can be expanded, that is, in the business scenario where a single financial institution is used as a resource provider to provide resources to the target object applying for resources, the data processing method can also be applied, thereby further improving the flexibility of data processing.

[0029] In order to better understand a data processing method, a readable medium, an electronic device, and a program product provided in an embodiment of the present application, an application environment applicable to an embodiment of the present application is first described below.

[0030] See also Figure 1 , Figure 1 1 shows a schematic diagram of an application environment of a data processing method provided by an embodiment of the present application. As an implementation mode, the data processing method provided by an embodiment of the present application can be applied to an electronic device. The electronic device can be as follows: Figure 1The server 110 shown in the figure can be connected to the terminal device 120 through a network. The network is used to provide a medium for a communication link between the server 110 and the terminal device 120. The network can include various connection types, such as a wired communication link, a wireless communication link, etc., which is not limited in the embodiment of the present application. Optionally, in other embodiments, the electronic device can also be a smart phone, a laptop computer, etc.

[0031] It should be understood that Figure 1 The server 110, network and terminal device 120 are merely illustrative. Depending on the implementation requirements, there may be any number of servers, networks and terminal devices. For example, the server 110 may be a physical server or a server cluster composed of multiple servers, and the terminal device 120 may be a mobile phone, a tablet, a desktop computer, a laptop computer, and the like. It is understood that the embodiments of the present application may also allow multiple terminal devices 120 to access the server 110 at the same time.

[0032] The embodiments of the present disclosure are further explained below with reference to the accompanying drawings.

[0033] Figure 2 is a flowchart of a data processing method according to an exemplary embodiment of the present disclosure, referring to Figure 2 , the data processing method may include the following steps: S201: Sending status data of resources in the database to the first system. The status data is used by the first system to generate data to be transferred for resources whose ownership information is to be changed. The data to be transferred includes resource identification, transfer conditions, and generation time of the data to be transferred.

[0034] In this embodiment, the execution subject of the data processing method, the first system, resources, resource identifiers, data to be transferred, and transfer conditions can be set according to actual conditions, and the embodiment of the present disclosure does not impose any restrictions on this.

[0035] For example, in a business scenario where multiple financial institutions jointly serve as resource providers to provide resources to a target object, the execution subject may be a cooperative system of one of the financial institutions for cooperating with other financial institutions to provide resources to the target object. The first system may include a system for changing ownership and a server, wherein the system for changing ownership may be used by the financial institution to generate data to be transferred, and the server may be used by the financial institution to store data to be transferred. The resource may be a promissory note, the resource identifier may be a promissory note number, the data to be transferred may be a file to be transferred, and the transfer conditions may include the purchase of shares, etc. Thus, the cooperative system may periodically send the status data of the promissory note number in the database to the first system, for example, the status data of the resources stored in the database on the previous day may be sent to the first system at 0:00 am every day. After receiving the status data sent by the cooperative system, the first system may generate a file to be transferred for the resource whose ownership information is to be changed according to preset screening rules and status data through the system for changing ownership, and store the file to be transferred in the server, for example, in an sftp server.

[0036] S202: Acquire the data to be transferred from the first system, and query the first data update requirement included in the corresponding status data at the first query time according to the resource identifier.

[0037] In this embodiment, the first data update requirement can be used to guide the resource borrower to perform a resource return operation. For example, the first data update requirement can be a first resource return plan, whereby the resource borrower can perform a resource return operation according to the first resource return plan.

[0038] For example, continuing to refer to the above example, after the file to be transferred is stored in the sftp server, the cooperative system can obtain the file to be transferred from the sftp server at a regular interval. For example, the file to be transferred can be obtained from the sftp server at 20:00 in the evening, and the first resource return plan included in the corresponding status data at the current time (i.e. 20:00 in the evening) can be queried based on the loan number in the file to be transferred.

[0039] S203: According to the first update record of the first data update demand that has occurred within the time period from the generation time to the first query time, perform an update rollback operation on the first data update demand to obtain the first data update demand included in the state data corresponding to the resource identifier at the generation time.

[0040] In this embodiment, the first update record can be used to represent the first resource return record of the resource borrower in the time period from the generation time to the first query time. Therefore, according to the first resource return record, the first resource return plan included in the first query time can be updated and rolled back to obtain the first resource return plan included in the state data corresponding to the resource identifier at the generation time.

[0041] For example, continuing to refer to the above example, after obtaining the file to be transferred from the sftp server, the cooperation system can query the first resource return record between 0:00 in the morning and 20:00 in the evening according to the loan number, and update and roll back the first resource return plan included in 20:00 in the evening according to the first resource return record to obtain the first resource return plan included in 0:00 in the morning.

[0042] S204: Execute the ownership transfer operation according to the first data update requirement and transfer conditions included at the generation time, so as to change the first ownership information included in the status data corresponding to the resource identifier to the second ownership information and generate a second data update requirement, and update the second data update requirement according to the first update record.

[0043] In this embodiment, the second data update requirement can be used to guide the resource borrower to perform a resource return operation to the new owner after the ownership is changed. For example, the second data update requirement can be a second resource return plan, so that the resource borrower can perform a resource return operation according to the second resource return plan.

[0044] For example, continuing to refer to the above example, after obtaining the first resource return plan included at 0:00 in the morning, the ownership transfer operation can be performed according to the transfer conditions in the file to be transferred and the first resource return plan included at 0:00 in the morning, so that the ownership information of the status data corresponding to the loan number is changed from the first ownership information to the second ownership information, and a second resource return plan is generated to guide the resource borrower to perform the resource return operation after the ownership information is changed. Since the ownership transfer operation is performed based on the first resource return plan at 0:00 in the morning, and the resource borrower performs the resource return operation between 0:00 in the morning and 20:00 in the evening, the second resource return plan generated at this time cannot accurately reflect the resource data of the resource borrower. Therefore, after the ownership transfer operation is performed, the second resource return plan needs to be updated according to the first resource return record to obtain an accurate second resource return plan.

[0045] Through the above technical solution, before executing the ownership transfer operation, the first data update requirement can be updated and rolled back to obtain the first data update requirement at the generation time, and after executing the ownership transfer operation according to the first data update requirement, the second data update requirement can be updated according to the first update record. Thus, while ensuring the accuracy of resource data, operations such as resource return can be performed on unreturned resources within the time period from the generation time to the first query time. On the one hand, the flexibility of resource data processing can be improved, and on the other hand, the business scenarios of ownership transfer operations can be expanded, that is, in business scenarios where a single financial institution is used as a resource provider to provide resources to the target object applying for resources, the data processing method can also be applied, further improving the flexibility of data processing.

[0046] To facilitate understanding of the data processing method provided by the present disclosure, possible implementation methods in the present disclosure are described below.

[0047] In a possible manner, the first data update requirement may include multiple update times, requirement data corresponding to each update time, and a completion status corresponding to each update time; wherein the completion status is used to indicate whether the resource borrower completes the resource return operation that satisfies the requirement data at the corresponding update time; accordingly, before updating the second data update requirement according to the first update record, the data processing method may further include: The first update record is determined to be a record in which the completion status of any update time is changed to completed.

[0048] As mentioned above, the first data update requirement can be used to guide the resource borrower to perform a resource return operation. Therefore, the update time in this embodiment can be the time used to guide the resource borrower to perform a resource return operation, and the demand data corresponding to the update time can be the number of resources that the resource borrower needs to return when performing a resource return operation. Therefore, before updating the second data update requirement according to the first update record, the resource update status in the time period from the generation time to the first query time, i.e., the first update record, can be quickly determined based on the completion status corresponding to each update time and the corresponding number of resources, so that the second data update requirement can be quickly updated according to the first update record, thereby improving the processing efficiency of resource data.

[0049] In a possible manner, obtaining the data to be transferred from the first system may include: A data packet to be transferred is obtained from the first system, the data packet to be transferred includes a plurality of data to be transferred; the plurality of data to be transferred are processed in shards to store different data to be transferred in different shard storage units of the database; the data to be transferred stored in the plurality of shard storage units are processed in parallel according to a plurality of ownership change service instances to perform an update rollback operation and an ownership transfer operation, wherein the plurality of ownership change service instances correspond one to one to the plurality of shard storage units.

[0050] It should be understood that after the relevant technology has screened out the status data of the resources whose ownership information is to be changed, it will generally be packaged into a data packet and uploaded to the server for storage. Then, when executing the ownership transfer operation, the data packet of the day will be pulled from the server and parsed line by line to obtain the data to be transferred, and then the ownership transfer operation will be performed according to the data to be transferred. Since the data packet includes multiple data to be transferred, different data to be transferred may exist in different databases or database instances. Therefore, when executing the ownership transfer operation, it is necessary to frequently switch database instances or databases, and each time a connection is established with a new database or database instance, it takes a certain amount of time and resources. Therefore, frequent switching of databases or database instances will reduce the processing performance of the system, thereby reducing the data processing efficiency. In addition, since the relevant technology is based on a single thread to read the data to be transferred in the data packet, when the data packet is too large, the processing time is too long, and there is a problem that all the data to be transferred cannot be processed on the same day, which in turn affects the downstream business.

[0051] In this embodiment, after obtaining the transfer data packet, multiple data to be transferred in the transfer data packet can be processed in slices, so that different data to be transferred can be stored in different slice storage units of the database. At the same time, multiple ownership change service instances can be set to process the data to be transferred stored in multiple slice storage units in parallel, and multiple ownership change service instances correspond to multiple slice storage units one by one. Therefore, when executing the ownership transfer operation, one ownership change service instance can only perform the ownership transfer operation on the data to be transferred in one slice storage unit during one startup process, thereby reducing the frequency of switching database instances or databases, and thus improving data processing efficiency.

[0052] For example, continuing to refer to the above example, a distributed timed task scheduling platform can be set up in the cooperation system in advance, so that after obtaining the transfer data packet from the sftp server, the files to be transferred can be read from the transfer data packet based on the streaming reading technology, and the files to be transferred can be divided into different business systems according to the pre-set business identifiers (i.e., identifiers of different business types). After all the files to be transferred are divided, for each file to be transferred in different business systems, the UUID (Universally Unique Identifier) ​​of the file to be transferred can be obtained, and then the last four characters of the UUID can be extracted, and the extracted 4 characters can be converted into an integer. Finally, the integer is modulo-operated according to the number of shard storage units to obtain the shard storage unit corresponding to the file to be transferred, and the file to be transferred is stored in the shard storage unit. After all the files to be transferred are stored in the shard storage units, the distributed scheduled task scheduling platform can be started every half an hour during the period from 20:00 to 23:00 in the evening to call the ownership change service instance through the distributed scheduled task scheduling platform to trigger the ownership change service instance to perform ownership transfer operations on the data to be transferred in multiple shard storage units in parallel, such as Figure 3 shown.

[0053] For example, for each ownership change service instance, when the ownership change service instance is started, the number of the shard storage unit to be processed can be obtained first. Among them, the number of the shard storage unit can be determined according to the actual situation, and the embodiment of the present disclosure does not impose any restrictions on this. For example, the number of the shard storage unit can be represented as partnerdb01, partnerdb02, and partnerdb03, etc. Afterwards, the file path of the file to be transferred stored in the number can be obtained from the sftp server, and the target shard file and target shard control file required to perform the ownership transfer operation on the file to be transferred can be obtained from the stfp server according to the file path. Then, it is determined whether the target shard file and the target shard control file exist in the sftp server. If not, it is determined whether there is a file in the sftp server that is used to represent that the target shard control file is being processed (i.e. Figure 4 xxdbxx.ctrl.ding file in the target shard control file), and whether there is a file that indicates that the target shard control file has been processed (i.e. Figure 4), if not, download the target shard file and the target shard control file to the local computer, and try to obtain the distributed lock. After obtaining the distributed lock, in order to avoid other ownership change service instances processing the target shard control file or other ownership change service instances processing the target shard control file before obtaining the distributed lock, it is possible to determine again whether there is a file in the sftp server that represents the target shard control file being processed, and whether there is a file that represents the target shard control file that has been processed. If not, generate a file representing the target shard control file being processed to the sftp server to avoid other ownership change service instances processing the data to be transferred in the shard storage unit corresponding to the number. After the file representing the target shard control file being processed is sent to the sftp server, update rollback operations and ownership transfer operations can be performed based on the aforementioned method, such as Figure 4 shown.

[0054] In a possible manner, one shard storage unit stores multiple pieces of data to be transferred, and one ownership change service instance has multiple threads; accordingly, processing the data to be transferred stored in multiple shard storage units in parallel according to the multiple ownership change service instances may include: For multiple pieces of data to be transferred stored in any shard storage unit, the multiple pieces of data to be transferred are processed in parallel through multiple threads of the corresponding ownership change service instance.

[0055] It should be understood that in the related art, when executing the ownership transfer operation, the ownership transfer operation is generally performed based on a single thread, so there are problems such as low efficiency. In this embodiment, since an ownership change service instance has multiple threads, multiple pieces of data to be transferred stored in a shard storage unit can be processed in parallel by multiple threads in the ownership change service instance, thereby further improving data processing efficiency.

[0056] For example, Figure 5 As shown, for each ownership change service instance, after the ownership change service instance is started, multiple pieces of data to be transferred in the shard storage unit can be read, and the multiple pieces of data to be transferred can be divided into multiple groups, and a thread can be assigned to each group of data to be transferred, so as to perform ownership transfer operations on the assigned data to be transferred through the thread. For example, for each thread, each piece of data to be transferred in each group of data to be transferred can be traversed, and for each piece of data to be transferred that has been traversed, it is verified based on the idempotent verification method whether the data to be transferred has been processed. If it has been processed, no processing is performed; if it has not been processed, the ownership transfer operation is performed on the data to be transferred based on the above-mentioned data processing method.

[0057] Through the above method, multiple threads in the ownership change service instance can be used to process multiple pieces of data to be transferred stored in a shard storage unit in parallel. On the one hand, this can improve data processing efficiency, and on the other hand, it can reduce the risk of not being able to process all the data to be transferred on the same day, thereby reducing the risk of affecting downstream businesses.

[0058] In a possible manner, the data to be transferred may include a generation time period indicating that the second data update requirement needs to be generated. Accordingly, obtaining the data to be transferred from the first system may include: Periodically obtain the data to be transferred that is currently in the generation time period from the first system; and / or, based on the abnormal repair time period configured by the hot loading tool, obtain the data to be transferred that is currently in the abnormal repair time period from the first system, wherein the abnormal repair time period is a historical time period of the generation time period.

[0059] In this embodiment, the period of obtaining the data to be transferred that is currently in the generation time period from the first system can be determined according to actual conditions, and the present embodiment does not impose any restrictions on this. For example, the data to be transferred can be obtained from the first system every half an hour between 20:00 and 23:00 every night.

[0060] In this embodiment, configuring the abnormal repair time period based on the hot loading tool can be: configuring the hot loading tool Nacos component in the first system, so that the abnormal repair time period can be configured based on the Nacos component. For example, if there is data to be transferred that fails to perform the ownership transfer operation on June 2, the user can configure the abnormal repair time period from 0:00 am on June 2 to 24:00 pm on June 2 through the Nacos component. After the abnormal repair time period is configured, the configured abnormal repair time period can be loaded through the hot loading mechanism of the Nacos component, so that the data to be transferred in the abnormal repair time period can be obtained from the first system without restarting the system, and then the data to be transferred is processed based on the above method to obtain the second data update requirement.

[0061] For example, referring to the above example, if the current time is 23:00 on June 3, after the user configures the abnormal repair time period through the Nacos component, for example, the abnormal repair time period is configured from 0:00 am on June 1 to 24:00 pm on June 1. The user can trigger the distributed scheduled task scheduling platform by manual triggering, so as to enable the ownership change service instance through the distributed scheduled task scheduling platform to perform the ownership transfer operation on the data to be transferred in the abnormal repair time period.

[0062] After the ownership change service instance is started, the to-be-transferred file that is currently in the abnormal repair time period is obtained from the stfp server. If the to-be-transferred file that is currently in the abnormal repair time period is to-be-transferred file a, including loan number a, transfer condition a, and generation time a (0:00 am on June 1st), then the first resource return plan a included in the corresponding status data at the current time and the resource return record a between the generation time a and the current time can be queried according to the loan number a in the to-be-transferred file a, and the first resource return plan a is updated and rolled back according to the resource return record a to obtain the first resource return plan b included in the generation time a. Finally, the ownership transfer operation can be performed according to the first resource return record b and the transfer condition a, so that the ownership information of the status data corresponding to the loan number a is changed from the first ownership information to the second ownership information, and a second resource return plan a is generated to guide the resource borrower to perform the resource return operation after the ownership information is changed, and the second resource return plan a is updated based on the resource return record a, so that the file a to be transferred can be generated within the generation time period indicating the need to generate the second data update requirement.

[0063] It should be understood that in the related technology, it is generally necessary to complete the ownership transfer operation within a specified time period. If it is not completed within the specified time period, the resource borrower may perform a resource return operation later, which will cause the resource borrower's unreturned resources and the data update requirements for the unreturned resources to change. If the ownership transfer operation is performed based on the previous data update requirements, the accuracy of the resource data will be affected. In this embodiment, since the data update requirements can be updated and rolled back before the ownership transfer operation is performed, the data update requirements of the generation time can be obtained, and after the ownership transfer operation is performed according to the data update requirements of the generation time, the second data update requirements can be updated according to the first update record. Therefore, when the ownership transfer operation is not completed within the specified time period, the ownership transfer operation can be performed again based on the abnormal repair time period configured by the hot loading tool, thereby overcoming the above-mentioned technical problems and further improving the flexibility of resource data processing.

[0064] In a possible manner, the data processing method may further include: In the case where the first update record includes a record of merging multiple update times in the first data update requirement, the state data of the corresponding resource is resent to the first system so that the first system regenerates the to-be-transferred data corresponding to the resource.

[0065] As mentioned above, the first data update requirement can be used to guide the resource borrower to perform a resource return operation. Therefore, the update time in this embodiment can be the time used to guide the resource borrower to perform a resource return operation. The demand data corresponding to the update time can be the amount of resources that the resource borrower needs to return when performing a resource return operation. Merging multiple update times in the first data update requirement can be merging the demand data corresponding to multiple update times into demand data corresponding to one update time. For example, the first data update requirement can be shown in Table 1: Table 1 First data update requirements

[0066] After the third update time and the fourth update time in the first data update requirement are combined, the combined first data update requirement may be as shown in Table 2: Table 2 First data update requirements after merger

[0067] It should be understood that when multiple update times are merged, the number of resources to be returned by the resource borrower will be affected, such as a reduction in the number of interest resources. Therefore, after executing the ownership transfer operation based on the previous data to be transferred, the accurate second data update requirement cannot be obtained. Therefore, in order to ensure the accuracy of the resource borrower's resource data, the data to be transferred needs to be regenerated.

[0068] For example, continuing with the above example, the distributed scheduled task scheduling platform can be triggered every half an hour between 23:00 and 24:00 every night to enable the ownership change service instance through the distributed scheduled task scheduling platform. After the ownership change service instance example is started, the target file to be transferred that has executed the ownership transfer operation in the corresponding shard storage unit can be determined, and then based on the loan number in the target file to be transferred, it is determined whether the first data update requirement in the corresponding status data merges multiple update times during the ownership transfer operation. If the first data update requirement in the corresponding status data merges multiple update times during the ownership transfer operation, the corresponding second data update requirement is deleted, and the status data of the corresponding resource is resent to the first system so that the first system can regenerate the data to be transferred corresponding to the resource, such as Figure 6 shown.

[0069] In a possible manner, the data processing method may further include: In response to obtaining the second data update requirement, querying the corresponding state data from the database according to the resource identifier for the first data update requirement included in the second query time; determining a second update record of the first data update requirement occurring within a time period from the first query time to the second query time; Accordingly, updating the second data update requirement according to the first update record may include: The second data update requirement is updated according to the first update record and the second update record.

[0070] It should be understood that the execution of the ownership transfer operation generally takes some time, and the resource borrower can perform a resource return operation to the resource provider at any time. Therefore, in order to avoid the resource borrower performing a resource return operation during the execution of the ownership transfer operation, thereby affecting the accuracy of the resource data. Before updating the second data update requirement according to the first update record, this embodiment can also obtain a second update record of the first data update requirement occurring during the time period from the first query time to the second query time, and then update the second data update requirement according to the first update record and the second update record, thereby improving the accuracy of the resource data.

[0071] For example, continuing to refer to the above example, if the ownership transfer operation is started at 20:00 in the evening and ends at 20:10 in the evening, a second resource return plan is generated. And between 20:00 in the evening and 20:10 in the evening, the resource borrower performs a resource return operation and generates a second resource return plan. Because when the ownership transfer operation is performed, it is based on the first resource return plan at 0:00 in the morning, and between 0:00 in the morning and 20:10 in the evening, the resource borrower performs a resource return operation, therefore, the second resource return plan generated at this time cannot accurately reflect the resource data of the resource borrower. Therefore, after the ownership transfer operation is executed, the second data update requirement needs to be updated according to the first resource return plan and the second resource return plan, so as to obtain an accurate second resource return plan.

[0072] It should be understood that in this embodiment, the updating of the second data update requirement according to the first update record and the second update record is only for illustrative purposes and is not a limitation of the paired solution. In a possible manner, when the second data update requirement is generated based on the file to be transferred and the transfer conditions, the second data update requirement can be updated based on the first update record. Then, it is determined at a fixed time whether the second update record is generated during the ownership transfer operation. If the second update record is generated, the updated second data update requirement is updated based on the second update record.

[0073] For example, continue to refer to Figure 6As shown, if the first data update requirement in the corresponding state data does not merge multiple update times during the ownership transfer operation, it is determined whether the first data update requirement in the corresponding state data generates a second update record during the ownership transfer operation. If the second update record is generated, the second data update requirement is updated based on the second update record.

[0074] To facilitate understanding of the present application, the data processing method provided by the embodiment of the present disclosure is introduced below by taking the loan business in the financial field as an example.

[0075] In the loan business, the resource borrower may be the debtor; the resource may be a loan note; the resource identifier may be a loan note number; the ownership information may be used to characterize the creditor information; the data to be transferred may be a debt transfer file; the first data update requirement may be the debtor's repayment plan for the financial institution; the second data update requirement may be the debtor's repayment plan for the new creditor after the debt transfer, or it may be called a debt transfer repayment plan; the update record may be a repayment flow; the ownership transfer operation may be a debt transfer operation, or it may be called a debt transfer operation. For example, the repayment plan of the debt transfer file at the time of generation may be split into a trust side and an ABS side repayment plan according to the purchase amount, and accordingly, the transfer conditions may include a trust plan and an ABS share; the executor of the data processing method may be a loan business system of a financial institution, and the first system may include a debt change system and a server, wherein the debt change system may be used by the financial institution to generate the debt transfer file, and the server may be used by the financial institution to store the debt transfer file. Thus, the data processing method may include: The loan business system can send the status data of the loan note number stored in the database on the previous day to the first system at 0:00 am every day. After receiving the status data sent by the business system, the first system can generate the debt transfer file for the loan note number of the debt information to be changed according to the preset screening rules and status data through the debt change system, and package the debt transfer file into a debt transfer data package and store it on the sftp server.

[0076] After that, the loan business system can obtain the data package to be transferred from the sftp server at 20:00 in the evening, and then read the files to be transferred from the data package to be transferred based on the streaming reading technology, and divide the multiple files to be transferred in the data package to be transferred into different business systems according to the pre-set business identifier, for example, according to the self-operated business (i.e., only providing loans to the debtor through the current financial institution) and the cooperative business (i.e., providing loans to the debtor together with other financial institutions). After all the files to be transferred are divided, the UUID (Universally Unique Identifier) ​​of the file to be transferred can be obtained for each file to be transferred in different business systems, and then the last four characters of the UUID are extracted, and the extracted 4 characters are converted into integers. Finally, the integer is modulo-operated according to the number of shard storage units to obtain the shard storage unit corresponding to the file to be transferred, and the file to be transferred is stored in the shard storage unit. After all the files to be transferred are stored in the shard storage units, the distributed scheduled task scheduling platform can be started every half an hour during the period from 20:00 to 23:00 in the evening to call the debt change service instance through the distributed scheduled task scheduling platform to trigger the debt change service instance to perform debt transfer operations on the files to be transferred in multiple shard storage units in parallel.

[0077] After the debt change service instance is started, it can read multiple debt transfer files in the shard storage unit, and split the multiple debt transfer files into multiple groups. A thread is assigned to each group of debt transfer files to perform debt transfer operations on the assigned debt transfer files through the thread.

[0078] For a thread, after obtaining the debt transfer file, the thread can query the repayment plan included in the corresponding status data at the current time (i.e. 20:00 p.m.) and the first repayment flow between 0:00 a.m. and 20:00 p.m. according to the loan number in the debt transfer file, and update and roll back the repayment plan included at 20:00 p.m. according to the first repayment flow, and obtain the repayment plan included at 0:00 a.m. After obtaining the repayment plan included at 0:00 a.m., the debt transfer operation can be performed according to the transfer conditions in the debt transfer file and the repayment plan included at 0:00 a.m., so as to change the debt information of the status data corresponding to the loan number from the first debt information to the second debt information, and generate a debt transfer repayment plan for guiding the debtor to perform the repayment operation after the debt information is changed. Since the debt transfer operation is executed based on the repayment plan included at 0:00 in the morning, and the debtor has performed the repayment operation between 0:00 in the morning and 20:00 in the evening, the debt transfer repayment plan generated at this time cannot accurately reflect the debtor's debt status. Therefore, after the debt transfer operation is completed, it is necessary to deduct the debt transfer repayment plan according to the first repayment flow to obtain an accurate debt transfer repayment plan. If during the debt transfer operation, the debtor performs a repayment operation and generates a second repayment flow, then after the debt transfer repayment plan is generated, it is necessary to update the debt transfer repayment plan based on the first repayment flow and the second repayment flow.

[0079] If it is detected that a debt transfer file has not completed the debt transfer within the specified time period, the debt transfer time period of the debt transfer file can be configured through the hot loading tool, so that the debt transfer can be re-performed through the above method to obtain the debt transfer repayment plan.

[0080] Based on the same concept, the embodiment of the present disclosure also provides a data processing device, such as Figure 7 As shown, the data processing device 700 may include: The first sending module 701 is used to send the state data of the resources in the database to the first system. The state data is used by the first system to generate the data to be transferred for the resources whose ownership information is to be changed. The data to be transferred includes the resource identifier, the transfer condition and the generation time of the data to be transferred; A first query module 702 is used to obtain the data to be transferred from the first system, and query the first data update requirement included in the corresponding state data at the first query time according to the resource identifier; The first processing module 703 is used to perform an update rollback operation on the first data update demand according to the first update record that the first data update demand has occurred during the time period from the generation time to the first query time, and obtain the first data update demand included in the state data corresponding to the resource identifier at the generation time; The second processing module 704 is used to perform the ownership transfer operation according to the first data update requirement and transfer conditions included at the generation time, so as to change the first ownership information included in the status data corresponding to the resource identifier to the second ownership information and generate a second data update requirement, and update the second data update requirement according to the first update record.

[0081] Through the above-mentioned data processing device 700, before executing the ownership transfer operation, the first data update demand can be updated and rolled back to obtain the first data update demand at the generation time, and after executing the ownership transfer operation according to the first data update demand, the second data update demand can be updated according to the first update record. Thus, while ensuring the accuracy of resource data, operations such as resource return can be performed on unreturned resources within the time period from the generation time to the first query time. On the one hand, the flexibility of resource data processing can be improved, and on the other hand, the business scenarios of ownership transfer operations can be expanded, that is, in the business scenario where a single financial institution is used as a resource provider to provide resources to the target object applying for resources, the data processing method can also be applied, further improving the flexibility of data processing.

[0082] In a possible manner, the first data update requirement may include multiple update times, requirement data corresponding to each update time, and a completion status corresponding to each update time; the completion status is used to indicate whether the resource borrower completes the resource return operation that satisfies the requirement data at the corresponding update time. Accordingly, the data processing device 700 may further include: The first determination module is used to determine that the first update record is a record of changing the completion status of any update time to completed before the second processing module 704 updates the second data update requirement according to the first update record.

[0083] In a possible manner, the data processing device 700 may further include: The second sending module is used to resend the status data of the corresponding resource to the first system when the first update record includes a record of merging multiple update times in the first data update requirement, so that the first system regenerates the to-be-transferred data corresponding to the resource.

[0084] In a possible manner, the data processing device 700 may further include: A second query module is used for querying the corresponding state data from the database according to the resource identifier in response to obtaining the second data update requirement, and including the first data update requirement at the second query time; A second determination module is used to determine a second update record of the first data update requirement occurring within a time period from the first query time to the second query time; Accordingly, the second processing module 704 may be configured to update the second data update requirement according to the first update record and the second update record.

[0085] In a possible manner, the data to be transferred includes a generation time period indicating that a second data update requirement needs to be generated. Accordingly, the first query module 702 can be used to periodically obtain the data to be transferred that is currently within the generation time period from the first system; and / or, based on the exception repair time period configured by the hot loading tool, obtain the data to be transferred that is currently within the exception repair time period from the first system, wherein the exception repair time period is a historical time period of the generation time period.

[0086] In a possible manner, the first query module 702 may include: An acquisition unit, configured to acquire a data packet to be transferred from the first system, wherein the data packet to be transferred includes a plurality of data to be transferred; A first processing unit is used to perform sharding processing on the plurality of data to be transferred, so as to store different data to be transferred in different sharding storage units of the database; The second processing unit is used to process the data to be transferred stored in multiple shard storage units in parallel according to multiple ownership change service instances to perform update rollback operations and ownership transfer operations, wherein the multiple ownership change service instances correspond one-to-one to the multiple shard storage units.

[0087] In a possible manner, a shard storage unit stores multiple pieces of data to be transferred, and a change of ownership service instance has multiple threads. Accordingly, the second processing unit can be used to process the multiple pieces of data to be transferred stored in any shard storage unit in parallel through the multiple threads of the corresponding change of ownership service instance.

[0088] Based on the same concept, the present disclosure also provides an electronic device, which may include: a storage device having a computer program stored thereon; The processing device is used to execute the computer program in the storage device to implement the steps of any of the above data processing methods.

[0089] Based on the same concept, an embodiment of the present disclosure further provides a computer-readable medium on which a computer program is stored, and when the program is executed by a processing device, the steps of any of the above-mentioned data processing methods are implemented.

[0090] Based on the same concept, an embodiment of the present disclosure further provides a computer program product, including a computer program, which implements the steps of any of the above data processing methods when executed by a processor.

[0091] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0092] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0093] Although the subject matter has been described in language specific to structural features and / or method logic actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims. Regarding the device in the above embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

Claims

1. A data processing method, characterized in that: The data processing method comprises: Sending the state data of the resources in the database to the first system, the state data is used by the first system to generate data to be transferred for the resources whose ownership information is to be changed, the data to be transferred includes the resource identifier, the transfer condition and the generation time of the data to be transferred; Acquire the data to be transferred from the first system, and query the first data update requirement included in the corresponding status data at the first query time according to the resource identifier; According to the first update record that the first data update requirement has occurred during the time period from the generation time to the first query time, an update rollback operation is performed on the first data update requirement to obtain the first data update requirement included in the state data corresponding to the resource identifier at the generation time; The ownership transfer operation is performed according to the first data update requirement included in the generation time and the transfer condition to change the first ownership information included in the status data corresponding to the resource identifier to the second ownership information and generate a second data update requirement, and the second data update requirement is updated according to the first update record.

2. The data processing method according to claim 1, characterized in that: The first data update requirement includes multiple update times, requirement data corresponding to each update time, and a completion status corresponding to each update time; The completion status is used to indicate whether the resource borrower has completed the resource return operation that meets the demand data at the corresponding update time; Before updating the second data update requirement according to the first update record, the method further includes: The first update record is determined to be a record of changing the completion status of any of the update times to completed.

3. The data processing method according to claim 2, characterized in that: The data processing method further includes: In the case where the first update record includes a record of merging the multiple update times in the first data update requirement, the status data of the corresponding resource is resent to the first system so that the first system regenerates the to-be-transferred data corresponding to the resource.

4. The data processing method according to any one of claims 1 to 3, characterized in that: The data processing method further includes: In response to obtaining the second data update requirement, querying the corresponding state data from the database according to the resource identifier for the first data update requirement included in the second query time; Determine a second update record of the first data update requirement occurring within a time period from the first query time to the second query time; The updating of the second data update requirement according to the first update record includes: The second data update requirement is updated according to the first update record and the second update record.

5. The data processing method according to claim 3, characterized in that: The data to be transferred includes a generation time period indicating a generation time period for generating the second data update requirement, and the acquiring the data to be transferred from the first system includes: Periodically acquiring the data to be transferred that is currently within the generation time period from the first system; and / or, Based on the abnormal repair time period configured by the hot loading tool, the data to be transferred that is currently within the abnormal repair time period is obtained from the first system, wherein the abnormal repair time period is a historical time period of the generated time period.

6. The data processing method according to any one of claims 1 to 3, characterized in that: The step of obtaining the data to be transferred from the first system includes: Acquire a data package to be transferred from the first system, wherein the data package to be transferred includes a plurality of pieces of data to be transferred; Slice the plurality of pieces of data to be transferred so as to store different pieces of data to be transferred in different slicing storage units of the database; The data to be transferred stored in the multiple shard storage units are processed in parallel according to multiple ownership change service instances to execute the update rollback operation and the ownership transfer operation, wherein the multiple ownership change service instances correspond one-to-one to the multiple shard storage units.

7. The data processing method according to claim 6, characterized in that: One of the shard storage units stores a plurality of pieces of data to be transferred, and one of the ownership change service instances has a plurality of threads; The processing of the to-be-transferred data stored in the plurality of shard storage units in parallel according to the plurality of ownership change service instances includes: For the multiple pieces of data to be transferred stored in any of the shard storage units, the multiple pieces of data to be transferred are processed in parallel through the corresponding multiple threads of the ownership change service instance.

8. An electronic device, characterized in that: include: a storage device having a computer program stored thereon; A processing device, configured to execute the computer program in the storage device to implement the steps of the method according to any one of claims 1 to 7.

9. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processing device, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images