Data cleaning method, device, electronic device, and computer-readable storage medium

Abstract

The present disclosure provides a data cleaning method, device, electronic device, and computer-readable medium, the method comprising: for each service instance, triggering the service instance to configure the cleaning task parameters of the service instance according to target configuration information, the cleaning task parameters of each service instance including the target number of threads for data cleaning of the service instance; calling the service instance to obtain the data to be cleaned according to the preconfigured data retrieval qualification identifier; if it is determined that the multiple service instances have all obtained the data to be cleaned, triggering the multiple service instances to perform data cleaning according to the cleaning task parameters of the service instance, wherein, corresponding to each service instance, the service instance is triggered to call the threads of the target number of threads for data cleaning of the service instance to clean the data to be cleaned corresponding to the service instance. According to the embodiments of the present disclosure, multi-instance and multi-threaded data cleaning can be achieved, thereby improving cleaning efficiency.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a data cleaning method, device, electronic device, and computer-readable storage medium. Background Art

[0002] Data cleansing is the process of removing invalid data generated during software operation to free up storage space, improve operation speed, and eliminate invalid data. Conventional data cleansing mechanisms include: performing data cleansing on a service instance-by-service basis.

[0003] However, with the development of technology, the amount of data generated by software operation is increasing, and users' requirements for running speed are getting higher and higher. In comparison, the conventional data cleaning mechanism has low data cleaning efficiency and cannot adapt to the needs of business development. Summary of the Invention

[0004] The present disclosure provides a data cleaning method, device, electronic device, and computer-readable storage medium.

[0005] In a first aspect, the present disclosure provides a data cleaning method, the method comprising:

[0006] In response to the target cleanup instruction, for each of the multiple service instances, triggering the service instance to configure cleanup task parameters of the service instance according to target configuration information, the target configuration information being pre-set according to the cleanup task indicated by the target cleanup instruction, the cleanup task parameters of the service instance including: a target number of threads for the service instance to perform data cleanup;

[0007] By matching the multiple service instances with pre-configured data retrieval qualification identifiers, calling the service instance to retrieve the data to be cleaned from the database to be cleaned according to the data to be cleaned identifier, the data retrieval qualification identifier is used to indicate that the identified service instance is qualified to obtain the data to be cleaned, and the data to be cleaned identifier is used to indicate the order of the data to be cleaned in the database to be cleaned;

[0008] If it is determined that the multiple service instances have all obtained the data to be cleaned, triggering the multiple service instances to clean the data according to the cleaning task parameters of the service instances, wherein triggering the multiple service instances to clean the data according to the cleaning task parameters of the service instances includes:

[0009] For each of the plurality of service instances, the following processing steps are performed:

[0010] The service instance is triggered to call threads of the target number of threads for data cleaning of the service instance, and clean the data to be cleaned corresponding to the service instance.

[0011] In a second aspect, the present disclosure provides a data cleaning device, the device comprising:

[0012] a configuration module, configured to, in response to a target cleanup instruction, trigger, for each of a plurality of service instances, the service instance to configure a cleanup task parameter of the service instance according to target configuration information, wherein the target configuration information is pre-set according to the cleanup task indicated by the target cleanup instruction, and the cleanup task parameter of the service instance includes at least one of the following: a target number of threads for the service instance to perform data cleanup;

[0013] a retrieval module, configured to match the multiple service instances with preconfigured data retrieval qualification identifiers, and call the service instances to retrieve the data to be cleaned from the database to be cleaned according to the data to be cleaned identifiers, wherein the data retrieval qualification identifiers are used to indicate that the identified service instances are qualified to retrieve the data to be cleaned, and the data to be cleaned identifiers are used to indicate the order of the data to be cleaned in the database to be cleaned;

[0014] A cleaning module is used to trigger the multiple service instances to perform data cleaning according to the cleaning task parameters of the service instances if it is determined that the multiple service instances have all obtained the data to be cleaned, wherein the triggering of the multiple service instances to perform data cleaning according to the cleaning task parameters of the service instances includes: for each of the multiple service instances, executing the following processing steps: triggering the service instance to call the target number of threads for the service instance to perform data cleaning, and cleaning the data to be cleaned corresponding to the service instance.

[0015] In a third aspect, the present disclosure provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores one or more computer programs executable by the at least one processor, and the one or more computer programs are executed by the at least one processor to enable the at least one processor to perform the above-mentioned data cleaning method.

[0016] In a fourth aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the above-mentioned data cleaning method when executed by a processor.

[0017] In a fifth aspect, the present disclosure provides a computer program or a computer program product, which includes a computer program stored in a computer-readable storage medium, and the computer program implements the above-mentioned data cleaning method when executed by a processor.

[0018] When the embodiment of the present disclosure is implemented, in response to a target cleanup instruction, for each of the multiple service instances, the service instance is triggered to configure the cleanup task parameters of the service instance according to the target configuration information. Afterwards, by matching the multiple service instances with the preconfigured data retrieval qualification identifiers, the service instance is called to obtain the data to be cleaned from the database to be cleaned according to the data to be cleaned identifier. If it is determined that the multiple service instances have all obtained the data to be cleaned, the multiple service instances are triggered to perform data cleanup according to the cleanup task parameters of the service instance, wherein the triggering of the multiple service instances to perform data cleanup according to the cleanup task parameters of the service instance includes: for each of the multiple service instances, triggering the service instance to call the target number of threads for the service instance to perform data cleanup, and cleaning the data to be cleaned corresponding to the service instance. The target configuration information is pre-set according to the cleanup task indicated by the target cleanup instruction, and the cleanup task parameters of each service instance include: the target number of threads for the service instance to perform data cleanup, the data retrieval qualification identifier is used to indicate that the identified service instance is qualified to obtain the data to be cleaned, and the data to be cleaned identifier is used to indicate the order of the data to be cleaned in the database to be cleaned. As can be seen, in order to ensure that multiple service instances do not obtain the same data to be cleaned when retrieving it from the database to be cleaned, the disclosed embodiment uses a method in which multiple service instances compete for the right to retrieve the data to be cleaned, and triggers each service instance to retrieve the data to be cleaned from the database to be cleaned in order according to the data identifier to be cleaned, so as to prevent each service instance from retrieving the same data to be cleaned. In other words, compared to conventional techniques in which multiple service instances are all directed to the database to be cleaned, if multiple service instances clean the same data to be cleaned, resulting in a cleaning error, the disclosed embodiment triggers multiple service instances to configure cleaning task parameters, and each service instance deploys multiple threads to perform the cleaning task. Furthermore, when each service instance retrieves data to be cleaned from the database to be cleaned, it triggers the service instance to retrieve the data according to the order of the data identifier corresponding to the service instance in the database to be cleaned, so as to avoid multiple service instances retrieving the same data to be cleaned when retrieving data from the same database to be cleaned, thereby triggering multiple service instances to perform data cleaning in parallel, and each service instance uses multiple threads to perform data cleaning. It can be seen that compared with the inefficient data cleaning of traditional single-threaded service instances one by one, the embodiment of the present disclosure uses multiple service instances and multiple threads to perform data cleaning, which not only improves the efficiency of data cleaning and meets the needs of business growth, but also each service instance can configure the cleaning task parameters by itself, can flexibly adapt to various business scenarios, and has good scalability.

[0019] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. The above and other features and advantages will become more apparent to those skilled in the art by describing detailed example embodiments with reference to the accompanying drawings. In the accompanying drawings:

[0021] Figure 1 A flowchart of a data cleaning method provided in an embodiment of the present disclosure;

[0022] Figure 2A A flowchart of an exemplary data cleaning method provided in an embodiment of the present disclosure;

[0023] Figure 2B A flow chart of a method for retrieving data based on Redis locks provided in an embodiment of the present disclosure;

[0024] Figure 3 A structural diagram of a data cleaning device provided in an embodiment of the present disclosure;

[0025] Figure 4 A structural diagram of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0026] To enable those skilled in the art to better understand the technical solutions of the present disclosure, exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0027] In the absence of conflict, the various embodiments of the present disclosure and the various features therein may be combined with each other.

[0028] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0029] The terms used herein are only used to describe specific embodiments and are not intended to limit the present disclosure. As used herein, the singular forms "a" and "the" are also intended to include the plural forms, unless the context clearly indicates otherwise. It will also be understood that when the terms "comprising" and / or "made of" are used in this specification, the presence of the features, wholes, steps, operations, elements and / or components is specified, but the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof is not excluded. Similar words such as "connected" or "connected" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect.

[0030] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted as having an idealized or overly formal meaning unless expressly defined as such herein.

[0031] The embodiments of the present disclosure relate to the field of data processing technology, and more specifically, to data cleaning technology. Data cleaning technology can be widely used to clean up invalid data in software systems in various scenarios, such as telephone sales systems and after-sales consulting systems. Invalid data may include, for example, historical data, expired data, and erroneous data generated during system operation. Taking the telephone sales system as an example, each sales cycle corresponds to a series of lists. After a sales cycle ends, the corresponding lists in the corresponding sales cycle become expired data and need to be cleaned up.

[0032] To accommodate business growth, a software system can deploy multiple servers, each capable of handling a certain volume of business. For example, in a telesales system, when the business volume is 1,000 seats, one server can be deployed to support business processing. When the business volume grows to 5,000 seats, five servers can be deployed, each supporting 1,000 seats, for example.

[0033] It should be pointed out that the above-mentioned server can refer to a "server" in the logical function dimension. Based on this, this specification will refer to a server as a "service instance" below.

[0034] Usually, invalid data is stored in a database to be cleaned, and all service instances in the software system have the authority to obtain invalid data from the database to be cleaned. However, multiple service instances are independent of each other and do not exchange information. If multiple service instances obtain data to be cleaned from the database to be cleaned, and then the multiple service instances perform data cleaning in parallel, it may cause different service instances to obtain the same data to be cleaned, which makes the data cleaning process difficult to carry out. Based on this, a conventional data cleaning method is implemented as follows: data cleaning is performed on a single thread for each service instance. However, as the business develops, more and more service instances are deployed in the system, and the amount of data that needs to be cleaned for each data cleaning task is also increasing, resulting in the processing efficiency of conventional data cleaning methods being unable to meet the needs of business development.

[0035] In view of this, the embodiments of the present disclosure provide a data cleaning method. In a scenario facing multiple service instances, the multiple service instances can be triggered to configure the number of threads for cleaning data. Then, the service instance is called to obtain the data to be cleaned from the database to be cleaned according to the data to be cleaned identifier. After that, the multiple service instances are triggered to perform data cleaning in parallel, and each service instance performs data cleaning in a multi-threaded manner. Among them, the data retrieval qualification identifier is used to indicate that the identified service instance is qualified to obtain the data to be cleaned, and the data to be cleaned identifier is used to indicate the order of the data to be cleaned in the database to be cleaned. In this way, multiple service instances are used to compete for the retrieval qualification to retrieve the data to be cleaned one by one, and each service instance is triggered to retrieve the data to be cleaned from the database to be cleaned in order according to the data to be cleaned identifier. This can prevent each service instance from retrieving the same data to be cleaned, thereby realizing a data cleaning method using multiple service instances and multiple threads. This not only improves the efficiency of data cleaning and meets the needs of business growth, but also allows each service instance to configure the cleaning task parameters independently, can flexibly adapt to various business scenarios, and has good scalability.

[0036] The data cleaning method illustrated in the embodiments of the present disclosure may be performed by an electronic device, which may be an in-vehicle device, a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, an in-vehicle device, a wearable device, etc. In some implementations, the data cleaning method may be implemented by a processor in the electronic device invoking computer-readable program instructions stored in a memory.

[0037] The data cleaning method of the embodiment of the present disclosure is introduced below in conjunction with an exemplary implementation.

[0038] Figure 1 A flow chart of a data cleaning method provided by an embodiment of the present disclosure. Figure 1 The method can be applied to multiple service instances, each of which is used to clean up data. The method includes:

[0039] In step S11 , in response to the target cleanup instruction, for each of the multiple service instances, the service instance is triggered to configure the cleanup task parameters of the service instance according to the target configuration information.

[0040] The target cleaning instruction may be a pre-set timed task instruction, which refers to a data cleaning task that is periodically executed at a certain time interval. The target cleaning instruction may include target configuration information of the data to be cleaned.

[0041] In some implementations, the target configuration information may include at least one of the following: a time range for data to be cleaned, the amount of data to be cleaned by each thread, a data extraction qualification identifier type, and cleanup task parameters for each service instance. The cleanup task parameters for each service instance may include at least one of the following: a target number of threads for data cleanup by the service instance, and a target amount of data to be cleaned by the service instance.

[0042] Exemplarily, the interval range corresponding to the expiration time of the data to be cleaned is, for example, the number of days with the date corresponding to this cleaning task as the end date. For example, the date corresponding to this cleaning task is October 4, 2022, and the expiration time range of the data to be cleaned is 3 days, which indicates that expired data within 3 days from October 2, 2022 to October 4, 2022 needs to be cleaned. The data retrieval qualification identifier is used to characterize that the identified service instance is qualified to obtain the data to be cleaned, while the unidentified service instance is not qualified to obtain the data to be cleaned. In the embodiment of the present disclosure, the data retrieval qualification identifier can be a distributed coordination lock, and the type of the data retrieval qualification identifier can be, for example, at least one of a ZooKeeper distributed type lock and a Remote Dictionary Server (Redis) distributed type lock. Each service instance can configure the target number of threads for data cleaning of the service instance and the target number of data to be cleaned of the service instance based on the amount of data cleaned by each thread and its own processing capacity. For example, the target configuration information indicates that the amount of data to be cleaned by each thread is 500. A service instance may be configured with 8 threads. Accordingly, the target amount of data to be cleaned by the service instance is 500*8=4000.

[0043] In some implementations, the electronic device can send the scheduled task instruction to multiple service instances by broadcasting, and can determine by polling that each service instance is capable of executing the scheduled task, thereby triggering multiple service instances to perform data cleaning tasks in parallel.

[0044] It should be noted that the amount of data to be cleaned in each data cleansing task can differ from the amount of data cleaned in previous data cleansing tasks. Based on this, the configuration information corresponding to the cleansing instructions at different stages can be flexibly set according to the actual implementation scenario. Accordingly, the content of the target configuration information corresponding to the target cleansing instruction can be different from the content of the configuration information corresponding to other cleansing instructions. Furthermore, because the amount of data to be cleaned varies between different data cleansing tasks, the number of service instances invoked by each data cleansing task can also vary.

[0045] It can be seen that by adopting this implementation method, the electronic device can dynamically adjust the number of called service instances and the number of threads of each service instance according to the situation of the data to be cleaned, so that the method of the embodiment of the present disclosure is not only highly flexible, but also has good scalability, and can adapt well to the development needs of the business.

[0046] In step S12, by matching multiple service instances with pre-configured data retrieval qualification identifiers, the service instance is called to obtain the data to be cleaned from the database to be cleaned according to the data to be cleaned identifier.

[0047] The data retrieval qualification identifier is used to indicate that the identified service instance is qualified to obtain the data to be cleaned, and the data to be cleaned identifier is used to indicate the order of the data to be cleaned in the database to be cleaned.

[0048] In an embodiment of the present disclosure, the electronic device can control the order in which multiple service instances retrieve the data to be cleaned through a data retrieval qualification identifier. Exemplarily, the electronic device can call the service instances among the multiple service instances that have not obtained the data to be cleaned to jointly obtain the data retrieval qualification identifier. If it is determined that any service instance has obtained the data retrieval qualification identifier, for example, after determining that the first service instance has obtained the data retrieval qualification identifier, the electronic device can call the first service instance to obtain the target number of data to be cleaned corresponding to the first service instance from the database to be cleaned according to the starting data to be cleaned identifier corresponding to the first service instance. After determining that the first service instance has completed the process of obtaining the data to be cleaned, the electronic device can call the first service instance to release the data retrieval qualification identifier, and execute again the step of calling all the service instances among the multiple service instances that have not obtained the data to be cleaned to jointly obtain the data retrieval qualification identifier, until all the multiple service instances have retrieved the data to be cleaned.

[0049] For example, when the data retrieval qualification identifier is implemented as a distributed coordination lock, the electronic device can trigger the multiple service instances to obtain the distributed coordination lock. Among them, in each round of obtaining the distributed coordination lock, only one service instance can obtain the corresponding distributed coordination lock, and the service instance that obtains the corresponding distributed coordination lock obtains the permission to retrieve the data to be cleaned. Accordingly, the service instance that obtains the corresponding distributed coordination lock can retrieve the data to be cleaned from the database of the data to be cleaned according to the starting data to be cleaned identifier corresponding to the service instance, and then release the distributed coordination lock after the retrieval is completed, and notify other service instances to obtain the distributed coordination lock again.

[0050] It should be noted that, in the embodiment of the present disclosure, the processing of retrieving data to be cleaned between multiple service instances is independent of each other, and the data to be cleaned are all stored in the same database to be cleaned. Therefore, in order to prevent any two service instances from retrieving the same data to be cleaned, in the process of calling any service instance to obtain the target number of data to be cleaned corresponding to any service instance from the database to be cleaned according to the starting data to be cleaned identifier corresponding to any service instance, the starting data to be cleaned identifier corresponding to any service instance can be determined. The starting data to be cleaned identifier can include the number of the starting data to be cleaned and the target expiration time. The target expiration time is the expiration time of the starting data to be cleaned, and the number of each data to be cleaned is used to identify the order of the corresponding data to be cleaned in the database to be cleaned. The number of each data to be cleaned can be, for example, the identity document (ID) of the data to be cleaned in the database to be cleaned. Afterwards, among the data to be cleaned determined according to the target expiration time, the number of the starting data to be cleaned is used as the starting number, and the target number of data to be cleaned is obtained from the database to be cleaned in the order of the numbers of the data to be cleaned.

[0051] It should be noted that the number of the starting data to be cleared and the target expiration time corresponding to any service instance can be determined based on the maximum number and expiration time of the data to be cleared obtained by the previous service instance of any service instance. The number of the starting data to be cleared corresponding to any service instance is, for example, the maximum number of the data to be cleared obtained by the previous service instance plus 1, and the expiration time of the starting data to be cleared corresponding to any service instance is the same as the expiration time of the data to be cleared obtained by the previous service instance.

[0052] Exemplarily, the electronic device can determine whether the first data to be cleaned is greater than or equal to the target number corresponding to any of the service instances, wherein the first data to be cleaned is the data to be cleaned that has not been retrieved and whose expiration period meets the target expiration period. If the number of the first data to be cleaned is greater than or equal to the target number corresponding to any of the service instances, the target number of data to be cleaned is acquired in sequence starting from the starting data to be cleaned. If the number of the first data to be cleaned is less than the target number corresponding to any of the service instances, the target number of data to be cleaned is acquired starting from the starting data to be cleaned and from the data to be cleaned corresponding to the target expiration period and the day before the target expiration period in the order of the numbers of the data to be cleaned.

[0053] For example, after service instance x obtains the distributed coordination lock, the target number of data to be cleaned corresponding to service instance x is 400, and the ID of the starting data to be cleaned corresponding to service instance x is, for example, 301, and the expiration period is, for example, 1 day, then the target expiration period is 1 day. Further, the electronic device can detect the amount of remaining data to be cleaned with an expiration period of 1 day in the database to be cleaned. The remaining data to be cleaned with an expiration period of 1 day, for example, includes data to be cleaned with IDs from 300 to 600. Accordingly, the amount of remaining data to be cleaned with an expiration period of 1 day is 300. Since the amount of remaining data to be cleaned with an expiration period of 1 day is less than 400, the electronic device can obtain 100 data from the data to be cleaned with an expiration period of 2 days as the data to be cleaned for service instance x. Exemplarily, the electronic device can obtain data to be cleaned with an expiration period of 2 days and IDs from 1 to 100 as the data to be cleaned for service instance x.

[0054] Furthermore, the next service instance of service instance x that obtains data to be cleared is, for example, service instance y. The electronic device can use the data to be cleared with an expiration time of 2 days and an ID of 101 as the starting data to be cleared of service instance y.

[0055] By adopting this implementation, when multiple service instances respectively obtain data to be cleaned from the database to be cleaned, it is possible to prevent any two service instances from obtaining the same data to be cleaned, thereby improving the reliability of data cleaning.

[0056] In step S13 , if it is determined that multiple service instances have acquired the data to be cleaned, the multiple service instances are triggered to perform data cleansing according to the cleansing task parameters of the service instances.

[0057] When multiple service instances have acquired data to be cleaned, the electronic device can trigger the multiple service instances to clean the data according to the cleaning task parameters of the service instances. For example, for each of the multiple service instances, the electronic device can trigger the service instance to call a target number of threads for data cleaning of the service instance to clean the data to be cleaned corresponding to the service instance.

[0058] In some implementations, corresponding to each service instance, the service instance can be called to group the data to be cleaned according to the target number of threads and the data to be cleaned, and the data to be cleaned corresponding to each thread corresponding to the service instance can be obtained. Then, each thread of the target number of threads can be called to clean the data to be cleaned corresponding to each thread in parallel.

[0059] For example, if the target number of data items to be cleaned obtained by service instance x is 400 and the pre-configured target number of threads is 8, service instance x can evenly divide the 400 data items to be cleaned into 8 data groups, where each data group contains 50 data items to be cleaned. 8 threads correspond to the 8 data groups one by one, and service instance x can call each of the 8 threads to clean the 50 data items to be cleaned in the data group corresponding to each thread.

[0060] In some implementations, when executing step S13, the electronic device may also store the data to be cleared by the multiple service instances in a cold table, which is a table that is not needed for business operations. In this way, the data to be cleared can be cleared from the system memory to improve the running speed of the software system, and the data to be cleared can be backed up so that it can be called when needed.

[0061] It should be pointed out that, in the process of cleaning data, if an exception occurs in the conventional data processing mechanism, the data cleaning task will be terminated directly, which will result in the data to be cleaned not being cleaned in time, resulting in poor data cleaning performance.

[0062] In view of this, during the execution of step S13, the electronic device can also monitor whether each thread of each service instance executes abnormally. If any thread executes abnormally, the service instance is triggered to call the abnormal thread to re-execute within a preset time range. Exemplarily, the preset time range can be a few hours from the start of the data cleaning task, such as 8 hours. Specifically, it can be flexibly set according to the usage time of the software system.

[0063] In some implementations, an alarm may also be issued for re-execution of a failed data cleanup thread within a preset time.

[0064] By adopting this implementation method, the electronic device can monitor the anomalies in the data cleaning task and automatically retry or alarm when an anomaly is captured, thereby being able to clean the data to be cleaned efficiently and timely and optimize the performance of data cleaning.

[0065] The data cleaning method of the embodiment of the present disclosure is described below with reference to examples.

[0066] Taking a telephone sales system as an example, the telephone sales system includes multiple service instances (hereinafter referred to as "instances"), and the data to be cleaned in the telephone sales system is, for example, a marketing list (hereinafter referred to as "list"). Before cleaning the list, for example, a scheduled task and configuration information for list cleaning are pre-configured.

[0067] Figure 2A A flowchart of an exemplary data cleaning method provided in an embodiment of the present disclosure is provided. Figure 2A The illustrative method may include the following steps:

[0068] Step S21: broadcast the scheduled task of list clearing to multiple instances.

[0069] Among them, after reaching the time corresponding to the scheduled task of the day, each instance in the multiple instances executes the list cleaning task of the day. In this example, the scheduled task that triggers the list cleaning task of the day is used as the target cleaning instruction.

[0070] Step S22: Start the abnormal monitoring and alarm mechanism.

[0071] Among them, the abnormal monitoring and alarm mechanism is aimed at the list cleaning tasks of multiple instances.

[0072] Step S23: trigger multiple instances to configure cleaning task parameters.

[0073] The aforementioned configuration information is, for example, target configuration information corresponding to a target cleaning instruction. The telephone sales system can trigger each of the multiple instances and configure cleaning task parameters according to the aforementioned configuration information.

[0074] Exemplarily, the aforementioned configuration information includes, for example, the following configuration items and configuration parameters:

[0075] The default number of items to be retrieved at a time is 400. The verification range is 100–2000.

[0076] Default number of threads: 8 Verification range: 1-16;

[0077] The default number of lists to be cleaned up per thread is 50. The verification range is 1–200.

[0078] The default range of the expiration time is: 1 verification range 1-30;

[0079] The default maximum execution time is 14400 minutes. The verification range is: 3600-28800;

[0080] Lock type (1: ZooKeeper 0: Redis) default 1 check range 0-1;

[0081] The maximum waiting time for acquiring a lock, in milliseconds, the default value is 180000, and the verification range is: [100, 1200000];

[0082] Among them, the number of lists retrieved at a time is used to indicate that the target number of data to be cleaned for each instance is 400 lists; the number of threads is used to indicate that the target number of threads that can be configured for each instance is 8; the amount of lists cleaned each time by a single thread is used to indicate that the amount of data cleaned by each thread of each instance is 50 lists; the expiration time range is used to indicate that the data to be cleaned for this data cleaning task is the lists within the day before the current day (i.e. 24 hours); the maximum execution time is used to indicate that the preset execution time of this data cleaning task is 14400 minutes; the lock types corresponding to this data cleaning task include ZooKeeper and Redis locks, and ZooKeeper is used by default; the maximum waiting time for obtaining the lock is used to indicate the length of the lock grabbing cycle.

[0083] The validation range corresponding to each configuration item indicates the range within which the instance can dynamically adjust. For any configuration item, if the instance does not support dynamic adjustment within the corresponding validation range, the instance sets the parameter of the configuration item to the default value. If the instance supports dynamic adjustment within the corresponding validation range, it can dynamically set the parameter of the configuration item within the validation range based on its own processing capabilities.

[0084] For example, regarding the number of threads, an instance can verify whether it supports dynamic thread adjustment between 1 and 16 threads. If the instance determines that it can support 10 threads based on its processing capabilities, then it can be set to 10 threads. If the instance does not support dynamic thread adjustment, it will be configured with 8 threads by default. It should be understood that the verification scope of other configuration items is similar and will not be detailed here.

[0085] Step S24: Multiple instances grab the lock, and the target instance that grabs the lock follows Figure 2B The target instance completes the list retrieval process and releases the lock after completing the list retrieval. Execute step S24 in a loop until multiple instances have completed the list retrieval operation, and then execute step S25:

[0086] In step S25 , each instance groups the retrieved list.

[0087] For example, if an instance is configured with 8 threads and retrieves 400 lists, and the number of lists to be cleaned by a single thread in the configuration information is 50, the instance will divide the 400 lists into 8 groups, with 50 lists in each group, and assign the lists in each group to one of the 8 threads.

[0088] Step S26: Multiple instances perform list cleaning in parallel, and each instance uses multiple threads to clean the list.

[0089] Step S27: insert the lists corresponding to the multiple instances into the cold table.

[0090] Step S28: If any thread is monitored to have an execution exception, the corresponding instance calls the thread with the execution exception to repeat the list clearing operation within the maximum execution time.

[0091] It should be noted that since both ZooKeeper and Redis locks are set in this example, when executing step S24, multiple instances first retrieve the list based on the ZooKeeper lock mechanism. If the ZooKeeper lock fails, the list retrieval can be switched to the Redis lock. This implementation method can improve the reliability of data cleaning.

[0092] The following uses the Redis lock mechanism as an example to illustrate the process of retrieving the list of the target instance that has obtained the Redis lock.

[0093] See also Figure 2B , Figure 2B It illustrates a method process for data retrieval based on Redis lock.

[0094] Step S201: The target instance obtains the ID of the starting list and the target expiration time from the Redis cache.

[0095] The Redis cache refers to the cached data stored in the Redis lock. The data in the Redis cache is written after the last instance that grabbed the Redis lock retrieved the list.

[0096] For example, the previous instance that obtained the Redis lock can write the maximum ID and expiration time of the retrieved list to the Redis cache. After the target instance obtains the Redis lock, it can use the next ID of the list with the maximum ID in the Redis cache as the starting ID of the target instance's list, and use the expiration time in the Redis cache as the target expiration time.

[0097] It should be noted that if the target instance is the first instance among multiple instances to grab the Redis lock, then the list ID in the Redis cache can be 0, and the expiration time can be the day before the current day.

[0098] Step S202 , determining whether the number of retrievable lists that meet the target expiration time is greater than or equal to the number of single-time retrievable lists of the target instance; if so, executing step S203 ; if not, executing step S204 .

[0099] Step S203: using the ID of the starting list as the starting ID, sequentially retrieve lists that meet the number of lists to be retrieved at a single time from the available lists with the target expiration time.

[0100] Step S204: add 1 to the target expiration time, and retrieve the list that meets the number of lists retrieved at a single time from the lists corresponding to the target expiration time and the target expiration time plus 1.

[0101] Among them, the target instance retrieves the list in order from the smallest to the largest ID of the list.

[0102] In step S205, the target instance stores the maximum ID and expiration time of the retrieved list in the redis cache.

[0103] It should be noted that if the target expiration time is the maximum expiration time of this list cleaning task, then the target instance can update the data in the redis cache to finish to indicate that the expired list query is completed.

[0104] It can be seen that, by adopting the embodiment provided by the present disclosure, in response to the target cleanup instruction, for each of the multiple service instances, the service instance is triggered to configure the cleanup task parameters of the service instance according to the target configuration information. Afterwards, by matching the multiple service instances with the preconfigured data retrieval qualification identifier, the service instances are called one by one to obtain the data to be cleaned. The data retrieval qualification identifier is used to characterize that the identified service instance is qualified to obtain the data to be cleaned. When it is determined that the multiple service instances have all obtained the data to be cleaned, the service instance is triggered to call the target number of threads of the service instance for each service instance to clean the target number of data to be cleaned for the service instance. Wherein, the target configuration information is pre-set according to the cleanup task indicated by the target cleanup instruction, and the cleanup task parameters of each service instance include at least one of the following: the target number of threads for data cleaning of the service instance, and the target number of data to be cleaned of the service instance. That is to say, compared to the conventional solution of performing data cleaning on each service instance in a single thread, the embodiment of the present disclosure triggers multiple service instances to configure cleaning task parameters, and enables each service instance to deploy multiple threads to perform cleaning tasks, thereby triggering multiple service instances to obtain the data to be cleaned respectively, so as to trigger multiple service instances to perform data cleaning in parallel, and each service instance uses multiple threads to perform data cleaning. It can be seen that compared to the traditional low efficiency of performing data cleaning on each service instance in a single thread, the embodiment of the present disclosure uses multiple service instances and multiple threads to perform data cleaning, which not only improves the efficiency of data cleaning and meets the needs of business growth, but also each service instance can configure the cleaning task parameters by itself, can flexibly adapt to various business scenarios, and has good scalability.

[0105] It is understood that the above-mentioned various method embodiments mentioned in this disclosure can be combined with each other to form combined embodiments without violating the principle logic. Due to space limitations, this disclosure will not go into details. It is understood by those skilled in the art that in the above-mentioned methods of specific implementation, the specific execution order of each step should be determined by its function and possible internal logic.

[0106] In addition, the present disclosure also provides a data cleaning device, an electronic device, and a computer-readable storage medium, all of which can be used to implement any data cleaning method provided by the present disclosure. The corresponding technical solutions and descriptions are referred to the corresponding records in the method section and will not be repeated here.

[0107] Reference Figure 3 The present disclosure provides a data cleaning device that is applied to multiple service instances, each of which is used to clean data. The device includes a configuration module 31, a retrieval module 32, and a cleaning module 33. Each module, when running, can implement some or all of the functions described in the aforementioned method implementation.

[0108] For example, a configuration module 31 is used to respond to a target cleaning instruction and, for each service instance in a plurality of service instances, trigger the service instance to configure the cleaning task parameters of the service instance according to the target configuration information, wherein the target configuration information is pre-set according to the cleaning task indicated by the target cleaning instruction, and the cleaning task parameters of the service instance include at least one of the following: the target number of threads for data cleaning by the service instance; a retrieval module 32 is used to call the service instance to obtain the data to be cleaned from the to-be-cleaned database according to the to-be-cleaned data identification by matching the plurality of service instances with the pre-configured data retrieval qualification identification, and the data retrieval qualification identification is used to characterize the identified service instance. The service instance is qualified to obtain the data to be cleaned, and the data to be cleaned identifier is used to indicate the order of the data to be cleaned in the database to be cleaned; the cleaning module 33 is used to trigger the multiple service instances to perform data cleaning according to the cleaning task parameters of the service instance if it is determined that the multiple service instances have all obtained the data to be cleaned, wherein the triggering of the multiple service instances to perform data cleaning according to the cleaning task parameters of the service instance includes: for each of the multiple service instances, executing the following processing steps: triggering the service instance to call the target number of threads for the service instance to perform data cleaning, and cleaning the data to be cleaned corresponding to the service instance.

[0109] For details on the implementation, see the above Figures 1 to 2B The method has been described in detail in the embodiment and will not be described again here.

[0110] It is understandable that the division of the above modules / units is only a division of logical functions. In actual implementation, the above modules / units can be integrated into hardware. Figure 4 As shown, Figure 4 A block diagram of an electronic device provided in an embodiment of the present disclosure. Figure 4 An embodiment of the present disclosure provides an electronic device, which includes: at least one processor 401; at least one memory 402, and one or more I / O interfaces 403, connected between the processor 401 and the memory 402; wherein the memory 402 stores one or more computer programs that can be executed by the at least one processor 401, and the one or more computer programs are executed by the at least one processor 401 to enable the at least one processor 401 to perform the above-mentioned data cleaning method.

[0111] The present disclosure also provides a computer-readable storage medium, which may be a volatile or non-volatile computer-readable storage medium and stores a computer program. When the computer program is executed by the processor 401:

[0112] In response to a target cleanup instruction, for each of a plurality of service instances, triggering the service instance to configure a cleanup task parameter of the service instance according to target configuration information, the target configuration information being pre-set according to the cleanup task indicated by the target cleanup instruction, the cleanup task parameters of the service instance including: a target number of threads for the service instance to perform data cleanup; calling the service instance to obtain the data to be cleaned from the database to be cleaned according to the data to be cleaned identifier by matching the plurality of service instances with pre-configured data retrieval qualification identifiers, the data retrieval qualification identifier being used to characterize that the identified service instance is qualified to obtain the data to be cleaned, and the data to be cleaned identifier being used to indicate the order of the data to be cleaned in the database to be cleaned; if it is determined that all of the plurality of service instances have obtained the data to be cleaned, triggering the plurality of service instances to perform data cleanup according to the cleanup task parameters of the service instance, wherein triggering the plurality of service instances to perform data cleanup according to the cleanup task parameters of the service instance includes: for each of the plurality of service instances, executing the following processing step: triggering the service instance to call threads corresponding to the target number of threads for data cleanup of the service instance to clean the data to be cleaned corresponding to the service instance.

[0113] In some embodiments, the target configuration information further includes at least one of the following: an interval range corresponding to the expiration time of the data to be cleaned, an amount of data cleaned by each thread, and a data retrieval qualification identification type.

[0114] In some embodiments, the processor 401 is further used to: corresponding to each service instance, monitor whether each thread of the service instance executes abnormally; if any thread executes abnormally, trigger the service instance to call the thread with the execution abnormality to re-execute within a preset time range.

[0115] In some embodiments, the processor 401 is also used to: call all the service instances among the multiple service instances that have not obtained the data to be cleaned to jointly obtain the data retrieval qualification identifier; if it is determined that any service instance has obtained the data retrieval qualification identifier, then call any service instance to obtain the target number of data to be cleaned corresponding to any service instance from the database to be cleaned according to the starting data to be cleaned identifier corresponding to any service instance; after determining that any service instance has completed the processing of obtaining the data to be cleaned, call any service instance to release the data retrieval qualification identifier, and execute again the step of calling all the service instances among the multiple service instances that have not obtained the data to be cleaned to jointly obtain the data retrieval qualification identifier.

[0116] In some embodiments, the processor 401 is also used to: determine the starting data to be cleaned identifier corresponding to any service instance, the starting data to be cleaned identifier includes the number of the starting data to be cleaned and the target expiration time, the target expiration time is the expiration time of the starting data to be cleaned, and the number of each data to be cleaned is used to identify the order of the corresponding data to be cleaned in the database to be cleaned; among the data to be cleaned determined according to the target expiration time, the number of the starting data to be cleaned is used as the starting number, and the target number of data to be cleaned is obtained from the database to be cleaned in the order of the number of the data to be cleaned.

[0117] In some embodiments, the processor 401 is further configured to: if the amount of first data to be cleared is greater than or equal to the target amount corresponding to any one of the service instances, sequentially retrieve the target amount of data to be cleared starting from the starting data to be cleared; if the amount of first data to be cleared is less than the target amount corresponding to any one of the service instances, sequentially retrieve the target amount of data to be cleared starting from the starting data to be cleared from the data to be cleared corresponding to the target expiration time and the data to be cleared the day before the target expiration time, in order of the order of the to-be-cleared data serial numbers; wherein the first data to be cleared is data to be cleared whose expiration time meets the target expiration time but has not been retrieved.

[0118] In some embodiments, the processor 401 is also used to: call the service instance according to the target number of threads and the data to be cleaned, group the data to be cleaned, and obtain the data to be cleaned corresponding to each thread corresponding to the service instance; call each thread of the target number of threads to clean the data to be cleaned corresponding to each thread in parallel.

[0119] In some embodiments, the processor 401 is further configured to: store the data to be cleaned in a cold table, where the cold table refers to a table that does not need to be called during business operation.

[0120] In some embodiments, the data retrieval qualification identifier is a distributed coordination lock.

[0121] An embodiment of the present disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above-mentioned data cleaning method.

[0122] It will be understood by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable storage medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium).

[0123] As is well known to those skilled in the art, the term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information (such as computer-readable program instructions, data structures, program modules or other data). Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technology, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, as is well known to those skilled in the art, communication media typically contains computer-readable program instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0124] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in the computer-readable storage medium in each computing / processing device.

[0125] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, and conventional procedural programming languages ​​such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., utilizing an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions. The electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0126] The computer program product described herein may be implemented in hardware, software, or a combination thereof. In one embodiment, the computer program product is implemented as a computer storage medium. In another embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0127] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0128] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0129] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0130] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and the part of the module, program segment or instruction contains one or more executable instructions for realizing the prescribed logical function. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the prescribed function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0131] Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should be interpreted only in a general illustrative sense and not for purposes of limitation. In some instances, it will be apparent to those skilled in the art that, unless otherwise expressly indicated, features, characteristics, and / or elements described in conjunction with a particular embodiment may be used alone or in combination with features, characteristics, and / or elements described in conjunction with other embodiments. Therefore, it will be understood by those skilled in the art that various changes in form and detail may be made without departing from the scope of the present disclosure as set forth in the appended claims.

Claims

1. A data cleaning method, characterized in that: The method comprises: In response to the target cleanup instruction, for each of the multiple service instances, triggering the service instance to configure cleanup task parameters of the service instance according to target configuration information, the target configuration information being pre-set according to the cleanup task indicated by the target cleanup instruction, the cleanup task parameters of the service instance including: a target number of threads for the service instance to perform data cleanup; By matching the multiple service instances with pre-configured data retrieval qualification identifiers, calling the service instance to retrieve the data to be cleaned from the database to be cleaned according to the data to be cleaned identifier, the data retrieval qualification identifier is used to indicate that the identified service instance is qualified to obtain the data to be cleaned, and the data to be cleaned identifier is used to indicate the order of the data to be cleaned in the database to be cleaned; If it is determined that the multiple service instances have all obtained the data to be cleaned, triggering the multiple service instances to clean the data according to the cleaning task parameters of the service instances, wherein triggering the multiple service instances to clean the data according to the cleaning task parameters of the service instances includes: For each of the plurality of service instances, the following processing steps are performed: The service instance is triggered to call threads of the target number of threads for data cleaning of the service instance, and clean the data to be cleaned corresponding to the service instance.

2. The data cleaning method according to claim 1, wherein: The target configuration information also includes at least one of the following: the interval range corresponding to the expiration time of the data to be cleaned, the amount of data cleaned by each thread, and the data retrieval qualification identification type.

3. The data cleaning method according to claim 1, wherein: Corresponding to each service instance, in the process of triggering the service instance to call the thread of the target number of threads of the service instance, cleaning the target number of to-be-cleaned data corresponding to the service instance, the process further includes: Monitor whether each thread of the service instance executes abnormally; If any thread executes abnormally, the service instance is triggered to call the thread that executes abnormally to re-execute within a preset time range.

4. The data cleaning method according to claim 1, wherein: The method of matching the plurality of service instances with preconfigured data retrieval qualification identifiers and calling the service instance to obtain the data to be cleaned from the database to be cleaned according to the data to be cleaned identifiers includes: Calling all service instances among the multiple service instances that have not obtained the data to be cleaned to jointly obtain the data retrieval qualification identifier; If it is determined that any service instance has obtained the data retrieval qualification identifier, calling any service instance to obtain the target number of to-be-cleaned data corresponding to any service instance from the to-be-cleaned database according to the starting to-be-cleaned data identifier corresponding to any service instance; After determining that any one of the service instances has completed the process of obtaining the data to be cleaned, call any one of the service instances to release the data retrieval qualification identifier, and again execute the step of calling all the service instances among the multiple service instances that have not obtained the data to be cleaned to jointly obtain the data retrieval qualification identifier.

5. The data cleaning method according to claim 4, characterized in that: The calling of any service instance to obtain a target amount of to-be-cleaned data corresponding to any service instance from the to-be-cleaned database according to a starting to-be-cleaned data identifier corresponding to any service instance includes: Determine a starting identifier of data to be cleaned corresponding to any one of the service instances, where the starting identifier of data to be cleaned includes a number of the starting data to be cleaned and a target expiration time, where the target expiration time is the expiration time of the starting data to be cleaned, and the number of each data to be cleaned is used to identify the order of the corresponding data to be cleaned in the database to be cleaned; Among the data to be cleared determined according to the target expiration time, the target number of data to be cleared is obtained from the database to be cleared in order of the size of the data to be cleared numbers, starting with the number of the starting data to be cleared.

6. The data cleaning method according to claim 5, characterized in that: The method of obtaining the target number of data to be cleared from the database to be cleared in the order of the numbers of the data to be cleared, using the number of the starting data to be cleared as the starting number among the data to be cleared determined according to the target expiration time, includes: If the amount of the first data to be cleaned is greater than or equal to the target amount corresponding to any one of the service instances, then sequentially acquiring the target amount of data to be cleaned starting from the starting data to be cleaned; If the number of the first data to be cleaned is less than the target number corresponding to any of the service instances, then starting from the starting data to be cleaned, the target number of data to be cleaned is obtained from the data to be cleaned corresponding to the target expiration time and the data to be cleaned one day before the target expiration time in the order of the size of the numbers of the data to be cleaned; The first data to be cleaned is data to be cleaned that has not been retrieved and whose expiration time meets the target expiration time.

7. The data cleaning method according to claim 1, wherein: The triggering service instance calls the target number of threads of the service instance for data cleaning, and cleans the data to be cleaned corresponding to the service instance, including: Calling the service instance to group the data to be cleaned according to the target number of threads and the data to be cleaned, and obtaining the data to be cleaned corresponding to each thread corresponding to the service instance; Each thread of the target number of threads is called to clean up the to-be-cleaned data corresponding to each thread in parallel.

8. The data cleaning method according to claim 1, wherein: Triggering a service instance to call a target number of threads for data cleaning by the service instance to clean up the data to be cleaned corresponding to the service instance also includes: The data to be cleaned is stored in a cold table, where the cold table refers to a table that does not need to be called during business operation.

9. The data cleaning method according to claim 1, wherein: The data retrieval qualification identifier is a distributed coordination lock.

10. A data cleaning device, characterized in that: The device comprises: a configuration module, configured to, in response to a target cleanup instruction, trigger, for each of a plurality of service instances, the service instance to configure a cleanup task parameter of the service instance according to target configuration information, wherein the target configuration information is pre-set according to the cleanup task indicated by the target cleanup instruction, and the cleanup task parameter of the service instance includes at least one of the following: a target number of threads for the service instance to perform data cleanup; a retrieval module, configured to match the multiple service instances with preconfigured data retrieval qualification identifiers, and call the service instances to retrieve the data to be cleaned from the database to be cleaned according to the data to be cleaned identifiers, wherein the data retrieval qualification identifiers are used to indicate that the identified service instances are qualified to retrieve the data to be cleaned, and the data to be cleaned identifiers are used to indicate the order of the data to be cleaned in the database to be cleaned; A cleaning module is used to trigger the multiple service instances to perform data cleaning according to the cleaning task parameters of the service instances if it is determined that the multiple service instances have all obtained the data to be cleaned, wherein the triggering of the multiple service instances to perform data cleaning according to the cleaning task parameters of the service instances includes: for each of the multiple service instances, executing the following processing steps: triggering the service instance to call the target number of threads for the service instance to perform data cleaning, and cleaning the data to be cleaned corresponding to the service instance.

11. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs executable by the at least one processor. The one or more computer programs are executed by the at least one processor to enable the at least one processor to perform the data cleaning method according to any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the computer program implements the data cleaning method according to any one of claims 1 to 9.

Images