Data processing method and device

Abstract

The embodiment of the present application discloses a data processing method and device, the method comprising: obtaining a data task to be processed, determining a data source corresponding to the data task to be processed; judging whether the data associated with the data task to be processed in the data source is in a ready state; if it is in an unready state, caching the data task to be processed, asynchronously performing a periodic state query on the data source in an unready state; if it is found that the data associated with the data task to be processed in the data source is switched from an unready state to a ready state, performing data judgment processing on the data associated with the data task to be processed. In this way, the accuracy of the judgment result can be guaranteed, and the problem of invalid analysis caused by the situation that the data is not ready can be avoided, thereby saving the resource overhead of the system, and the problem of inaccurate judgment result caused by the data not being ready can also be avoided.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of Internet technology, and specifically to a data processing method and device. Background Art

[0002] With the advent of the big data era, more and more companies are beginning to pay attention to the value of data, and many companies hope to use the data they own to conduct corresponding analysis or judgment.

[0003] However, it is very likely that the data is not ready when analysis or judgment is needed, which will lead to inaccurate results of data judgment or data analysis, which not only increases the resource overhead of the system, but also makes the analysis or judgment meaningless. Summary of the invention

[0004] In view of the above problems, the present application is proposed to provide a data processing method, apparatus, computing device, computer storage medium and computer program product that overcome the above problems or at least partially solve the above problems.

[0005] According to one aspect of an embodiment of the present application, a data processing method is provided, which includes:

[0006] Obtain the data task to be processed and determine the data source corresponding to the data task to be processed;

[0007] Determine whether the data associated with the data task to be processed in the data source is in a ready state;

[0008] If it is in the unready state, cache the pending data tasks and asynchronously perform periodic status queries on the data source in the unready state;

[0009] If it is found that the data associated with the data task to be processed in the data source is switched from the unready state to the ready state, data judgment processing is performed on the data associated with the data task to be processed.

[0010] Furthermore, the method further includes: during the asynchronous query process, determining whether the current time exceeds a preset time;

[0011] If the current time exceeds the preset time and the data associated with the data task to be processed in the data source is not ready, an alarm message is generated and returned, or historical data is obtained as the data associated with the data task to be processed, and data judgment processing is performed on the historical data.

[0012] Furthermore, determining whether the data associated with the data task to be processed in the data source is in a ready state further includes:

[0013] Obtain the data validation rules corresponding to the data source, and determine whether the data associated with the data task to be processed in the data source is in a ready state according to the data validation rules; or

[0014] The data status interface corresponding to the data source is called to read the data status, and the data associated with the data task to be processed in the data source is judged according to the read data status whether it is in a ready state.

[0015] Furthermore, the task of caching the data to be processed further includes:

[0016] The data source identifier in the unready state is stored in a key-value pair database, wherein the data key is the data source and the data value is the data source identifier in the unready state;

[0017] The data source identifier and the data task to be processed are stored in a key-value pair database, wherein the data key is the data source identifier in an unready state, and the data value is the data task to be processed.

[0018] Furthermore, the method further includes: obtaining a registration request sent by the service end, wherein the registration request carries a data task judgment condition and a data source identifier;

[0019] The data task judgment conditions and data source identifiers are associated and stored.

[0020] Furthermore, the task of obtaining data to be processed further includes:

[0021] Obtain the information of the data task object to be processed and the data task judgment conditions;

[0022] Determining the data source corresponding to the data task to be processed further includes: obtaining a data source identifier that is associated with the data task judgment condition.

[0023] According to another aspect of an embodiment of the present application, there is provided a data processing device, comprising:

[0024] A first acquisition module, adapted to acquire data tasks to be processed;

[0025] A determination module, adapted to determine a data source corresponding to a data task to be processed;

[0026] A first judgment module is adapted to judge whether the data associated with the data task to be processed in the data source is in a ready state;

[0027] The asynchronous query module is suitable for caching pending data tasks if the data source is in an unready state, and asynchronously performing periodic status queries on the data source in an unready state;

[0028] The data judgment processing module is adapted to perform data judgment processing on the data associated with the data task to be processed if it is found that the data associated with the data task to be processed in the data source is switched from a not-ready state to a ready state.

[0029] According to another aspect of the embodiment of the present application, there is provided a computing device, including: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus;

[0030] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute the operation corresponding to the above data processing method.

[0031] According to another aspect of the embodiments of the present application, a computer storage medium is provided, in which at least one executable instruction is stored, and the executable instruction enables a processor to perform operations corresponding to the above-mentioned data processing method.

[0032] According to another aspect of the embodiments of the present application, a computer program product is provided, comprising at least one executable instruction, wherein the executable instruction enables a processor to perform operations corresponding to the above-mentioned data processing method.

[0033] According to the data processing method and device provided by the embodiment of the present application, after receiving the data task to be processed, it is possible to first determine the data source corresponding to the data task, and then determine whether the data in the data source is in a ready state. If so, data judgment processing is directly performed on the data; otherwise, the data task is first cached, and periodic status queries are asynchronously performed on the data source in an unready state until the state of the data associated with the data task to be processed in the data source is queried to switch from an unready state to a ready state, and then data judgment processing is performed on the data associated with the data task. This can avoid the problem of invalid analysis caused by the fact that the data is not yet ready, thereby saving the system's resource overhead, and can also avoid the problem of inaccurate judgment results caused by the fact that the data is not yet ready.

[0034] The above description is only an overview of the technical solution of the embodiment of the present application. In order to more clearly understand the technical means of the embodiment of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiment of the present application more obvious and easy to understand, the specific implementation method of the embodiment of the present application is listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only used for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the embodiments of the present application. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0036] Figure 1 A schematic diagram showing a flow chart of a data processing method according to an embodiment of the present application is shown;

[0037] Figure 2 A schematic diagram showing a flow chart of a data processing method according to another embodiment of the present application is shown;

[0038] Figure 3 A schematic diagram showing a data processing process according to an example of the present application is shown;

[0039] Figure 4 A structural block diagram of a data processing device according to an embodiment of the present application is shown;

[0040] Figure 5 A schematic diagram of the structure of a computing device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0041] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0042] Figure 1 A flow chart of a data processing method according to an embodiment of the present application is shown. Figure 1 As shown, the method comprises the following steps:

[0043] Step S101, obtaining a data task to be processed and determining a data source corresponding to the data task to be processed.

[0044] Specifically, the data tasks to be processed are data tasks that require data judgment and processing. The data tasks to be processed can be obtained in the following ways: receiving the data tasks to be processed uploaded from the business end, or actively obtaining the data tasks to be processed from the data task list. Of course, it is also possible to simultaneously receive the data tasks to be processed from the business end and obtain them from the data task list.

[0045] After obtaining the data task to be processed, you need to determine the data source corresponding to the data task to be processed. The data source refers to the source of some basic data that the data task to be processed needs to rely on. The data source can be physical (such as database, file system) or logical (such as API, Web service).

[0046] Step S102, determining whether the data associated with the data task to be processed in the data source is in a ready state, if not, executing step S103.

[0047] Specifically, after determining the data source corresponding to the data task to be processed, it is necessary to further determine whether the data associated with the data task to be processed in the data source is in a ready state, wherein the ready state means that the data associated with the data task to be processed is ready and can be used for further data judgment and processing, and the not ready state means that the data associated with the data task to be processed is not ready and data judgment and processing cannot be performed directly.

[0048] For different data sources, different methods can be used to determine whether the data associated with the pending data tasks in the data source is ready. For example, if the data volume of a data source is very large, the data status interface (API interface) corresponding to the data source can be directly called to obtain the latest data status information, and then determine whether the data associated with the pending data tasks in the data source is in a ready state. If the data volume of a data source is relatively small, polling can be used to check whether the data associated with the pending data tasks in the data source is in a ready state.

[0049] Step S103: cache the data tasks to be processed, and asynchronously perform periodic status queries on data sources that are not ready.

[0050] Specifically, if it is found through the judgment in the previous step that the data required by the data task to be processed is not yet ready in a certain data source, it can be determined that the data source is in an unready state. In this case, the data task to be processed cannot be processed immediately. Instead, the data task to be processed must be cached and wait for subsequent processing. Caching refers to temporarily storing the data task to be processed, such as temporarily storing it in the memory, and then taking out the data task to be processed and continuing to process it after the data associated with the data task to be processed is ready in the future.

[0051] After caching the pending data tasks, in order not to affect the processing of other pending data tasks, and to be able to promptly perform data judgment processing on the cached pending data tasks after the data associated with the cached pending data tasks are ready, it is also necessary to asynchronously perform periodic status queries on the data sources that are not ready, so that the preparation status of the data associated with the cached pending data tasks in the data sources that are not ready can be known at any time. For example, a new thread can be re-enabled to periodically query the data preparation status of the data sources that are not ready. Among them, periodic status query refers to checking whether the status of the relevant data in a data source has changed at a certain time interval, for example, checking the status of the data associated with the pending data tasks in a data source that is not ready every 5 seconds.

[0052] Step S104: if it is found that the data associated with the data task to be processed in the data source is switched from the unready state to the ready state, data judgment processing is performed on the data associated with the data task to be processed.

[0053] Specifically, if it is found through periodic status query that the status of the data associated with the pending data task in a data source that is in an unready state has changed, for example, from "unready state" to "ready state", then this means that the data source has prepared the data associated with the pending data task. In this case, the previously cached pending data task can be immediately retrieved and data judgment processing of its associated data can be started.

[0054] According to the data processing method provided in the embodiment of the present application, after receiving the data task to be processed, it is possible to first determine the data source corresponding to the data task, and then determine whether the data in the data source is in a ready state. If so, the data judgment processing is directly performed on the data; otherwise, the data task is first cached, and the data source in an unready state is asynchronously and periodically queried until the state of the data associated with the data task to be processed in the data source is queried to switch from an unready state to a ready state, and then the data judgment processing is performed on the data associated with the data task. This can avoid the problem of invalid analysis caused by the fact that the data is not yet ready, thereby saving the system's resource overhead, and can also avoid the problem of inaccurate judgment results caused by the fact that the data is not yet ready.

[0055] Figure 2 A flow chart of a data processing method according to another embodiment of the present application is shown. Figure 2 As shown, the method comprises the following steps:

[0056] Step S201, obtain the data task to be processed and determine the data source corresponding to the data task to be processed. Specifically, the data task to be processed is a data task that needs to be processed by data judgment. The data task to be processed can be obtained in the following ways: receiving the data task to be processed uploaded from the business end, or actively obtaining the data task to be processed from the data task list. Of course, it is also possible to simultaneously receive the data task from the business end and obtain the data task to be processed from the data task list.

[0057] After obtaining the data task to be processed, you need to determine the data source corresponding to the data task to be processed. The data source refers to the source of some basic data that the data task to be processed needs to rely on. The data source can be physical (such as database, file system) or logical (such as API, Web service).

[0058] Step S202, obtaining the data verification rule corresponding to the data source, and judging whether the data associated with the data task to be processed in the data source is in a ready state according to the data verification rule, if not, executing step S203.

[0059] Specifically, after determining the data source corresponding to the data task to be processed, it is necessary to further determine whether the data associated with the data task to be processed in the data source is in a ready state. For example, one or more sets of data verification rules can be pre-defined to check whether the data associated with the data task to be processed in the data source is in a ready state. The data verification rules can be provided by the business end, for example, the business end configures the data verification rules through the DSL language, or they can be preset general rules for the business end to choose, such as judging whether it is ready by checking a certain field in the data, or judging whether it is ready from a certain field in the return value. Using data verification rules to determine whether the data associated with the data task to be processed in the data source is in a ready state is relatively simple, intuitive, and easy to understand and modify.

[0060] In an optional implementation, the data status can also be read by calling the data status interface corresponding to the data source, and then judging whether the data associated with the data task to be processed in the data source is in a ready state according to the read data status. This verification method requires the business end to register the data status interface corresponding to the data source in advance, so that it is convenient to read the data status through the data status interface later. This method is more flexible and efficient and suitable for large-scale concurrent scenarios.

[0061] If it is in the ready state, execute step S207.

[0062] Step S203: Store the data source identifier in the not-ready state into a key-value pair database, wherein the data key is the data source and the data value is the data source identifier in the not-ready state.

[0063] Step S204: store the data source identifier and the data task to be processed into a key-value pair database, wherein the data key is the data source identifier in an unready state, and the data value is the data task to be processed.

[0064] If it is determined according to step S202 that the data source has not yet prepared the data required by the data task to be processed, it can be determined that the data source is in an unready state. In this case, the data task to be processed cannot be processed immediately, but the data task to be processed must be cached and wait for subsequent processing. Caching means temporarily storing the data task to be processed and taking out the data task to be processed for further processing after the data associated with the data task to be processed is ready in the future.

[0065] Specifically, the data source identifier in the unready state and the data task to be processed associated therewith can be stored in a key-value database. For example, the data source identifier in the unready state can be stored in a key-value database (KV database), wherein the data key is the data source, and the data value is the data source identifier in the unready state. In addition, the data source identifier and the data task to be processed can also be stored in the KV database, wherein the data key is the data source identifier in the unready state, and the data value is the data task to be processed, so as to facilitate subsequent search and recovery operations. As for which type of database to use, it depends on factors such as actual needs and performance requirements. For example, a Redis database is used.

[0066] Step S205, asynchronously perform periodic status query on the data source in the unready state; if it is found that the data associated with the data task to be processed in the data source is switched from the unready state to the ready state, execute step S207; if it is found that the data associated with the data task to be processed in the data source is in the unready state, execute step S206.

[0067] In order not to affect the processing of other pending data tasks, and at the same time to be able to timely perform data judgment processing on the cached pending data tasks after the data associated with the cached pending data tasks are ready, it is also necessary to asynchronously perform periodic status queries on the data source in the unready state, so as to grasp the state changes of the data in the data source at any time. By performing periodic status queries, the query results mainly include: the data associated with the pending data tasks in the data source is switched from the unready state to the ready state and the data associated with the pending data tasks in the data source is still in the unready state. Different processing will be performed for the above two results. If it is queried that the data associated with the pending data tasks in the data source is switched from the unready state to the ready state, step S207 is executed; if it is queried that the data associated with the pending data tasks in the data source is in the unready state, step S206 is executed.

[0068] Step S206, determine whether the current time exceeds the preset time; if not, execute step S205, if so, execute step S208.

[0069] Specifically, for the case where the data associated with the pending data task in the data source is still in an unready state, in order to avoid certain extreme situations (such as a data source being in an unready state for some reason), some additional safety measures can be added to the solution provided in the embodiment of the present application. For example, a timeout mechanism can be added during the asynchronous query process. Specifically, after each asynchronous periodic status query, it can be determined whether the current time has exceeded a preset time (for example, 12 o'clock, or 1 hour, which can be specified by the business end). If the current time exceeds the preset time, it means that there may be a problem. At this time, you can choose to immediately trigger the alarm mechanism to remind the relevant business end to pay attention to the specific situation. For example, you can generate an alarm message and report it to the relevant business end for processing; in addition, you can also consider using some other data as a substitute. In other words, jump to step S208.

[0070] If the current time does not exceed the preset time, the process may jump to step S205.

[0071] Step S207: performing data judgment processing on the data associated with the data task to be processed.

[0072] If the data associated with the pending data task in the data source is queried and switched from the unready state to the ready state, it means that the data associated with the pending data task is ready, so the data associated with the pending data task can be processed. For example, the data task judgment condition corresponding to the pending data task can be obtained, and then the data judgment process is performed according to the data task judgment condition. For example, the data task judgment condition is: whether the revenue flow yesterday reaches 100 yuan. The data task judgment condition can be pre-configured or pre-registered, or defined in the pending data task.

[0073] Step S208, generate and return warning information.

[0074] If the data associated with the pending data task in the data source has been in an unready state and the current time exceeds the preset time, an alarm message can be generated and returned to the business end. The alarm message prompts the business end that an abnormality has occurred in the data update in the data source, so that the business end can process it.

[0075] In an optional implementation, in the case where the data associated with the data task to be processed in the data source has been in an unready state and the current time has exceeded a preset time, historical data can also be obtained as the data associated with the data task to be processed, and then data judgment processing is performed on the historical data, thereby overcoming the problem that the data in the data source has not been ready and data judgment cannot be performed.

[0076] In an optional implementation, the business end may pre-register the data task judgment condition and the data source identifier, so the method further includes: obtaining a registration request sent by the business end, wherein the registration request carries the data task judgment condition and the data source identifier; and storing the data task judgment condition and the data source identifier in association. Thus, the acquired data task to be processed may include: the object information of the data task to be processed and the data task judgment condition, so that the data source corresponding to the data task to be processed can be determined by obtaining the data source identifier associated with the data task judgment condition.

[0077] In an optional implementation, object information may be defined in the data task to be processed, that is, the object targeted by the data task, for example, may be a user, in which case the object information may be a UID; it may also be a data source, in which case the object information may be a data source identifier. In addition, the object may also be an APP, etc., which are not specifically limited here. For the case where the object is a data source, the data task to be processed may further limit which users or APPs are to be processed for data judgment. Thus, the object information may also be stored in a key-value pair database, in which the data key is an object list and the data value is the object information.

[0078] According to the data processing method provided in the embodiment of the present application, after receiving the data task to be processed, it is possible to first determine the data source corresponding to the data task, and then determine whether the data in the data source is in a ready state. If it is in an unready state, the data task to be processed is cached and a periodic status query is asynchronously performed on the data source in an unready state. At the same time, it is also necessary to determine whether the current time exceeds the preset time. If it exceeds, an alarm message is generated and returned. If it is queried that the data associated with the data task to be processed in the data source is switched from an unready state to a ready state, data judgment processing is performed on the data associated with the data task to be processed, thereby avoiding the problem of invalid analysis caused by the fact that the data is not yet ready, thereby saving the system's resource overhead, and avoiding the problem of inaccurate judgment results due to the fact that the data is not ready, and avoiding the problem of infinite waiting due to the fact that the data is not ready, thereby ensuring the accuracy of the data judgment results.

[0079] Crowd selection refers to judging whether a broadcaster meets certain conditions, such as whether revenue flow, bullet comment data, new fans, number of viewers, etc. have reached the corresponding values. However, in the process of judging the broadcaster's crowd conditions, some indicators, such as yesterday's revenue data, bullet comment data, new fans, number of viewers, etc., may still be calculated upstream and not ready, resulting in abnormal crowd selection or inaccurate data condition judgment results. In order to overcome the above problems, Figure 3 A schematic diagram of a data processing process according to an example of the present application is shown, Figure 3 The specific processing process is as follows:

[0080] 1. First, you need to register the data judgment condition, assign a data judgment condition identifier to the data judgment condition, register the data source, and assign a data source identifier to the data source.

[0081] 2. First, determine the content to be judged, such as starting with crowd selection, that is, determine the UID of the anchor for this judgment.

[0082] 3. Determine the data judgment conditions, such as whether yesterday’s revenue flow reached 100 yuan.

[0083] 4. Use the data judgment condition identifier to find the corresponding data source and determine whether the upstream data of yesterday's revenue flow corresponding to the anchor UID in the data source is ready. If it is ready, make the judgment directly and the process ends.

[0084] 5. If the upstream data is not ready, the anchor UID and the data judgment condition will be temporarily stored, and the unblocked data source will be stored in redis, the key is the unblocked data source list, and the value is the unready data source identifier; the crowd condition will be stored in redis, the key is the unready data source, and the value is the data judgment condition identifier; the uid will be stored in redis to mark the pending judgment status, the key is the uid list to be judged, and the value is the content to be judged.

[0085] 6. The asynchronous service continuously pulls the upstream data that needs to be judged, and periodically queries whether the data of the unready data source is ready.

[0086] 7 If the upstream data is ready, the data conditions and content to be judged corresponding to the data source are taken out and the normal judgment logic is performed.

[0087] 8. If the waiting time is configured, if it is still not ready after 1 hour, the data judgment conditions and the content to be judged that have run out of time will be taken out, and the normal judgment logic will be performed based on the existing data.

[0088] Figure 4 A structural block diagram of a data processing device according to an embodiment of the present application is shown. Figure 4 As shown, the device comprises:

[0089] A first acquisition module 401, adapted to acquire a data task to be processed;

[0090] A determination module 402 is adapted to determine a data source corresponding to a data task to be processed;

[0091] The first judgment module 403 is adapted to judge whether the data associated with the data task to be processed in the data source is in a ready state;

[0092] The asynchronous query module 404 is adapted to cache the pending data tasks if the data source is in the unready state, and asynchronously perform periodic status queries on the data source in the unready state;

[0093] The data judgment processing module 405 is adapted to perform data judgment processing on the data associated with the data task to be processed if it is found that the data associated with the data task to be processed in the data source is switched from a not-ready state to a ready state.

[0094] Further, the device also includes: a second judgment module, adapted to judge whether the current time exceeds the preset time during the asynchronous query process;

[0095] A generating module adapted to generate and return an alarm message if the current time exceeds a preset time and the data associated with the data task to be processed in the data source is in an unready state; or

[0096] The data judgment processing module is also suitable for: acquiring historical data as data associated with the data task to be processed, and performing data judgment processing on the historical data.

[0097] Furthermore, the first judgment module is further adapted to: obtain a data verification rule corresponding to the data source, and judge whether the data associated with the data task to be processed in the data source is in a ready state according to the data verification rule; or

[0098] The data status interface corresponding to the data source is called to read the data status, and the data associated with the data task to be processed in the data source is judged according to the read data status whether it is in a ready state.

[0099] Furthermore, the asynchronous query module is further adapted to: store the data source identifier in the unready state into a key-value pair database, wherein the data key is the data source and the data value is the data source identifier in the unready state;

[0100] The data source identifier and the data task to be processed are stored in a key-value pair database, wherein the data key is the data source identifier in an unready state, and the data value is the data task to be processed.

[0101] Furthermore, the device further comprises: a second acquisition module adapted to acquire a registration request sent by the service end, wherein the registration request carries a data task judgment condition and a data source identifier;

[0102] The storage module is suitable for associating and storing the data task judgment conditions and the data source identifiers.

[0103] Furthermore, the first acquisition module is further adapted to: acquire information of the data task object to be processed and data task judgment conditions;

[0104] The determination module is further adapted to: obtain a data source identifier associated with the data task judgment condition.

[0105] The description of each module above refers to the corresponding description in the method embodiment and will not be repeated here.

[0106] According to the data processing device provided by the embodiment of the present application, after receiving the data task to be processed, it can first determine the data source corresponding to the data task, and then determine whether the data in the data source is in a ready state. If it is in an unready state, the data task to be processed is cached and the data source in the unready state is asynchronously periodically queried. At the same time, it is also necessary to determine whether the current time exceeds the preset time. If it exceeds, an alarm message is generated and returned. If it is queried that the data associated with the data task to be processed in the data source is switched from an unready state to a ready state, data judgment processing is performed on the data associated with the data task to be processed. This can avoid the problem of invalid analysis caused by the fact that the data is not yet ready, thereby saving the system's resource overhead, and can also avoid the problem of inaccurate judgment results due to the fact that the data is not ready, and can also avoid the problem of infinite waiting due to the fact that the data is not ready, thereby ensuring the accuracy of the data judgment results.

[0107] An embodiment of the present application provides a non-volatile computer storage medium, which stores at least one executable instruction or computer program, and the executable instruction or computer program can enable a processor to perform operations corresponding to the data processing method in any of the above method embodiments.

[0108] An embodiment of the present application provides a computer program product, which includes at least one executable instruction or computer program, and the executable instruction or computer program can enable a processor to execute operations corresponding to the data processing method in any of the above method embodiments.

[0109] Figure 5 A schematic diagram of the structure of an embodiment of a computing device of the present application is shown. The specific embodiment of the present application does not limit the specific implementation of the computing device.

[0110] like Figure 5 As shown, the computing device may include: a processor (processor) 502 , a communication interface (Communications Interface) 504 , a memory (memory) 506 , and a communication bus 508 .

[0111] The processor 502, the communication interface 504, and the memory 506 communicate with each other via a communication bus 508. The communication interface 504 is used to communicate with other devices such as a client or other server network elements. The processor 502 is used to execute a program 510, which can specifically execute the relevant steps in the above-mentioned data processing method embodiment for a computing device.

[0112] Specifically, the program 510 may include program codes, which include computer operation instructions.

[0113] The processor 502 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the computing device may be processors of the same type, such as one or more CPUs; or may be processors of different types, such as one or more CPUs and one or more ASICs.

[0114] The memory 506 is used to store the program 510. The memory 506 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0115] The program 510 can be specifically used to enable the processor 502 to execute the data processing method in any of the above method embodiments. The specific implementation of each step in the program 510 can refer to the corresponding description of the corresponding steps and units in the above data processing embodiment, which will not be repeated here. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described devices and modules can refer to the corresponding process description in the above method embodiments, which will not be repeated here.

[0116] The algorithm and display provided herein are not inherently related to any specific computer, virtual system or other equipment. Various general purpose systems can also be used together with the teachings based on this. According to the above description, it is obvious to construct the structure required for this type of system. In addition, the present application embodiment is not directed to any specific programming language yet. It should be understood that various programming languages ​​can be utilized to realize the content of the present application embodiment described herein, and the above description of specific languages ​​is to disclose the best mode of implementation of the present application embodiment.

[0117] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0118] Similarly, it should be understood that in order to streamline the present disclosure and help understand one or more of the various inventive aspects, in the above description of the exemplary embodiments of the present application, the various features of the embodiments of the present application are sometimes grouped together into a single embodiment, figure, or description thereof. However, the disclosed method should not be interpreted as reflecting the following intention: the claimed embodiments of the present application require more features than the features clearly stated in each claim. More specifically, as reflected in the claims below, the inventive aspects are less than all the features of the single embodiment disclosed above. Therefore, the claims following the specific embodiment are hereby explicitly incorporated into the specific embodiment, wherein each claim itself serves as a separate embodiment of the embodiment of the present application.

[0119] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0120] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the embodiments of the present application and to form different embodiments. For example, in the claims below, any one of the claimed embodiments may be used in any combination.

[0121] The various component embodiments of the embodiments of the present application can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all functions of some or all components according to the embodiments of the present application. The embodiments of the present application can also be implemented as a device or apparatus program (e.g., computer program and computer program product) for executing part or all of the methods described herein. Such a program implementing the embodiments of the present application can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0122] It should be noted that the above embodiments illustrate rather than limit the embodiments of the present application, and that those skilled in the art may design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets should not be constructed as a limitation to the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of multiple such elements. The embodiments of the present application may be implemented by means of hardware including several different elements and by means of appropriately programmed computers. In a unit claim that lists several devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names.

Claims

1. A data processing method, comprising: Obtaining a data task to be processed, and determining a data source corresponding to the data task to be processed; Determine whether the data associated with the to-be-processed data task in the data source is in a ready state; If it is in the unready state, the to-be-processed data task is cached, and a periodic status query is asynchronously performed on the data source in the unready state; If it is found in the query data source that the data associated with the data task to be processed is switched from a not-ready state to a ready state, data judgment processing is performed on the data associated with the data task to be processed.

2. The method according to claim 1, wherein: The method further comprises: during the asynchronous query process, determining whether the current time exceeds a preset time; If the current time exceeds the preset time and the data associated with the data task to be processed in the data source is not ready, an alarm message is generated and returned, or historical data is obtained as the data associated with the data task to be processed, and data judgment processing is performed on the historical data.

3. The method according to claim 1 or 2, wherein: The step of determining whether the data associated with the to-be-processed data task in the data source is in a ready state further includes: Obtaining a data verification rule corresponding to the data source, and determining whether the data associated with the to-be-processed data task in the data source is in a ready state according to the data verification rule; or The data status interface corresponding to the data source is called to read the data status, and whether the data associated with the to-be-processed data task in the data source is in a ready state is determined according to the read data status.

4. The method according to any one of claims 1 to 3, wherein: The task of caching the data to be processed further includes: The data source identifier in the unready state is stored in a key-value pair database, wherein the data key is the data source and the data value is the data source identifier in the unready state; The data source identifier and the data task to be processed are stored in a key-value pair database, wherein the data key is the data source identifier in an unready state, and the data value is the data task to be processed.

5. The method according to any one of claims 1 to 4, wherein: The method further includes: obtaining a registration request sent by the service end, wherein the registration request carries a data task judgment condition and a data source identifier; The data task judgment condition and the data source identifier are associated and stored.

6. The method according to claim 5, wherein: The task of obtaining data to be processed further includes: Obtain the information of the data task object to be processed and the data task judgment conditions; The determining of the data source corresponding to the to-be-processed data task further includes: obtaining a data source identifier associated with the data task judgment condition.

7. A data processing device, comprising: A first acquisition module, adapted to acquire data tasks to be processed; A determination module, adapted to determine a data source corresponding to the data task to be processed; A first judgment module, adapted to judge whether the data associated with the to-be-processed data task in the data source is in a ready state; An asynchronous query module, adapted to cache the pending data task if it is in a not-ready state, and asynchronously perform periodic status query on the data source in the not-ready state; The data judgment processing module is adapted to perform data judgment processing on the data associated with the data task to be processed if it is found in the data source that the data associated with the data task to be processed is switched from a not-ready state to a ready state.

8. A computing device comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform an operation corresponding to the data processing method according to any one of claims 1 to 6.

9. A computer storage medium, wherein at least one executable instruction is stored in the storage medium, and the executable instruction enables a processor to perform an operation corresponding to the data processing method according to any one of claims 1 to 6.

10. A computer program product, comprising at least one executable instruction, wherein the executable instruction enables a processor to execute operations corresponding to the data processing method according to any one of claims 1 to 6.

Images