Data processing method, data processing device, computer system and storage medium

Abstract

The present disclosure provides a data processing method, a data processing device, a computer system and a storage medium. The method includes: obtaining abnormal data to be processed; determining a target data channel for processing abnormal data from at least one data channel according to the reporting scenario of the abnormal data, wherein different data channels are used to process data reported through different reporting scenarios, each data channel includes at least one matching mode, and each matching mode includes a matching rule and business benchmark data that needs to be referenced when using the matching mode; inputting the abnormal data into the target data channel so that the abnormal data is compared with the business benchmark data according to the matching rule based on the matching mode in the target data channel; when it is determined that the abnormal data matches the business benchmark data according to the matching rule, the matching mode to which the matching rule belongs is used as the target matching mode; and determining the target processing party for processing the abnormal data according to the target matching mode.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and more specifically, to a data processing method, a data processing device, a computer system, and a computer-readable storage medium. Background Art

[0002] At present, in the logistics field, during the reporting or processing of abnormal orders, users need to repeatedly query the routing information or full-process data of the package to determine the processing department to which the current abnormal order data needs to be reported, so as to determine the final responsible unit for the abnormal package.

[0003] In the process of implementing the concept of the present disclosure, the inventors found that there are at least the following problems in the related art: the manual selection of abnormal data processing has problems of low accuracy and efficiency. Summary of the invention

[0004] In view of this, the present disclosure provides a data processing method, a data processing device, a computer system, and a computer-readable storage medium.

[0005] One aspect of the present disclosure provides a data processing method, comprising:

[0006] Get the exception data to be processed;

[0007] According to the reporting scenario of the abnormal data, determine a target data channel for processing the abnormal data from at least one data channel, wherein different data channels are used to process data reported through different reporting scenarios, each data channel includes at least one matching mode, and each matching mode includes matching rules and business benchmark data that needs to be referenced when using the matching mode;

[0008] Inputting the abnormal data into the target data channel, so as to compare the abnormal data with the business benchmark data according to the matching rules based on the matching pattern in the target data channel;

[0009] In the case where it is determined according to the matching rule that the abnormal data matches the business benchmark data, the matching pattern to which the matching rule belongs is used as the target matching pattern;

[0010] A target processor for processing the abnormal data is determined according to the target matching pattern.

[0011] According to an embodiment of the present disclosure, after determining the target processor for processing the abnormal data according to the target matching pattern, the method further includes:

[0012] The abnormal data and the target processing party corresponding to the abnormal data are stored in a database.

[0013] According to an embodiment of the present disclosure, the data processing method further includes:

[0014] According to the reporting scenario of the abnormal data, the abnormal data in the database is input into the target data channel again by using a time scheduler at a preset time interval corresponding to the reporting scenario, wherein different preset time intervals are applicable to different reporting scenarios;

[0015] Based on the matching pattern in the target data channel, the abnormal data is re-compared with the new business benchmark data in the matching pattern according to the matching rules to determine a new target matching pattern;

[0016] A new target processor for processing the abnormal data is determined according to the new target matching pattern.

[0017] According to an embodiment of the present disclosure, according to the reporting scenario of the abnormal data, using a time scheduler to input the abnormal data in the database into the target data channel again at a preset time interval corresponding to the reporting scenario includes:

[0018] Using the time scheduler, the preset time interval corresponding to the reporting scenario and the abnormal data are sent to the time wheel queue;

[0019] Using the time wheel queue to generate a scheduled task for the preset time interval corresponding to the reporting scenario and the abnormal data; and

[0020] The timing task is executed by using the time wheel queue so as to input the abnormal data into the target data channel again.

[0021] According to an embodiment of the present disclosure, the data processing method further includes:

[0022] Before processing the above-mentioned exception data to be processed, storing the above-mentioned exception data to be processed in a service framework for persistently storing log entry streams;

[0023] The abnormal data, the target matching pattern, and the target processing party are stored in association with each other in the service framework;

[0024] After the above-mentioned exception data to be processed is processed by the above-mentioned target processing party, the obtained processed exception data is stored in the above-mentioned service framework.

[0025] According to an embodiment of the present disclosure, the data channel includes a plurality of matching modes, wherein the plurality of matching modes are connected in series to form the data channel.

[0026] According to an embodiment of the present disclosure, the above matching mode includes at least one of the following: full process data matching mode, sealed vehicle data matching mode, unsealed vehicle data matching mode, sorting and delivery data matching mode;

[0027] The above-mentioned business benchmark data includes at least one of the following: full-process benchmark data, sealed vehicle benchmark data, unsealed vehicle benchmark data, and sorting and delivery benchmark data;

[0028] The above-mentioned reporting scenario includes at least one of the following: terminal reporting scenario and program capture scenario.

[0029] According to an embodiment of the present disclosure, the above-mentioned abnormal data includes abnormal orders; the above-mentioned matching rules include that the length of time that the above-mentioned abnormal order stays in the site included in the above-mentioned business benchmark data exceeds a preset length of time;

[0030] Comparing the above abnormal data with the above business benchmark data according to the above matching rules includes:

[0031] Determine the actual length of time that the abnormal order stays at the site included in the business benchmark data;

[0032] The actual duration is compared with the preset duration; when the actual duration exceeds the preset duration, it is determined that the abnormal data matches the business benchmark data; when the actual duration is less than or equal to the preset duration, it is determined that the abnormal data does not match the business benchmark data.

[0033] According to an embodiment of the present disclosure, the data processing method further includes:

[0034] After comparing the above abnormal data with the above business benchmark data according to the above matching rules, the comparison result is recorded using an event collector.

[0035] Another aspect of the present disclosure provides a data processing device, comprising:

[0036] An acquisition module is used to obtain the abnormal data to be processed;

[0037] A first determination module is used to determine a target data channel for processing the abnormal data from at least one data channel according to a reporting scenario of the abnormal data, wherein different data channels are used to process data reported through different reporting scenarios, each of the data channels includes at least one matching mode, and each of the matching modes includes matching rules and business benchmark data that needs to be referenced when using the matching mode;

[0038] An input module, used for inputting the abnormal data into the target data channel, so as to compare the abnormal data with the business benchmark data according to the matching rule based on the matching pattern in the target data channel;

[0039] A second determination module is used for, when it is determined according to the matching rule that the abnormal data matches the business benchmark data, taking the matching mode to which the matching rule belongs as the target matching mode; and

[0040] The third determination module is used to determine a target processing party for processing the above-mentioned abnormal data according to the above-mentioned target matching pattern.

[0041] Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the method described in the embodiments of the present disclosure when executed.

[0042] Another aspect of the present disclosure provides a computer program, which includes computer executable instructions. When the instructions are executed, they are used to implement the method described in the embodiments of the present disclosure.

[0043] According to an embodiment of the present disclosure, a target data channel for processing abnormal data is determined from at least one data channel according to a reporting scenario of abnormal data, and within the target data channel, reference business benchmark data in at least one matching pattern is compared with the abnormal data. When it is determined through a matching rule in the matching pattern that the reference business benchmark data matches the abnormal data, a matching pattern corresponding to the matching rule is determined as a target matching pattern, and the matching pattern is determined as a technical means for processing the target processor of the abnormal data. Therefore, the technical problem of low accuracy and timeliness of manually matching the processor corresponding to the abnormal data is at least partially overcome, thereby improving the accuracy and timeliness of assigning target processors to the abnormal data. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0045] Figure 1 A flowchart schematically shows a data processing method according to an embodiment of the present disclosure;

[0046] Figure 2 A system principle diagram of a data processing method according to an embodiment of the present disclosure is schematically shown;

[0047] Figure 3 A system principle diagram of a data processing method according to another embodiment of the present disclosure is schematically shown;

[0048] Figure 4 A system principle diagram of a data processing method according to another embodiment of the present disclosure is schematically shown;

[0049] Figure 5 A block diagram schematically shows a data processing device according to an embodiment of the present disclosure;

[0050] Figure 6 A block diagram schematically illustrates a comparison module according to an embodiment of the present disclosure; and

[0051] Figure 7 A block diagram of an electronic device for implementing a data processing method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0052] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0053] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0054] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0055] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0056] In the logistics industry, when abnormal data in the system is reported or processed through the quality control system, users (such as business parties providing logistics services) are required to repeatedly query the routing information or full-process data of the logistics to determine the current processor of the abnormal data, thereby determining the final target processor of the package of the abnormal data, providing an important basis for the synchronization of subsequent downstream system data.

[0057] At the same time, when abnormal data circulates in the quality control system, different processing parties will often change, and the circulation of abnormal data between different processing parties will produce more invalid circulation, thereby extending the closed loop of abnormal data and making the quality control system more complex in logical processing, data flow, and data analysis statistics.

[0058] However, during the reporting process of abnormal data, manual selection of the processor to be reported is required, such as determining the processor of abnormal data by querying routing information or full-process information. The logistics information of the package is usually planned in advance. When an abnormality occurs when the package arrives at a certain station, the processor corresponding to the abnormal data of the package is the previous station. Manual reporting of abnormal data not only affects the reporting efficiency, but also may result in errors, such as reporting errors to the processor, resulting in a longer circulation time for the abnormal data of the abnormal package and a slower processing time, which in turn makes it impossible for the downstream of the quality control system to complete the claims or accountability of the abnormal package in a timely manner.

[0059] The inventors found that after obtaining the abnormal data to be processed, the target processing channel for processing the abnormal data is determined according to the reporting scenario of the abnormal data, and the matching pattern in the data channel is used to match the abnormal data. When the matching pattern matches the abnormal data, the target processing party for processing the abnormal data is determined according to the matching pattern. The above method can avoid the problems of poor accuracy and timeliness caused by the manual selection of reporting.

[0060] In view of this, the embodiments of the present disclosure provide a data processing method, a data processing device, a computer system and a computer-readable storage medium. The method includes obtaining abnormal data to be processed; determining a target data channel for processing the abnormal data from at least one data channel according to the reporting scenario of the abnormal data; inputting the abnormal data into the target data channel so as to compare the abnormal data with the business benchmark data according to the matching rules based on the matching pattern in the target data channel; when it is determined that the abnormal data matches the business benchmark data according to the matching rules, the matching pattern to which the matching rules belong is used as the target matching pattern; and determining the target processing party for processing the abnormal data according to the target matching pattern. .

[0061] Figure 1 The flowchart of the data processing method according to the embodiment of the present disclosure is schematically shown.

[0062] Figure 2 The system principle diagram of the data processing method according to the embodiment of the present disclosure is schematically shown.

[0063] like Figure 1 As shown, the method may include operations S110 to S150.

[0064] In operation S110 , abnormal data to be processed is acquired.

[0065] In operation S120, according to the reporting scenario of the abnormal data, a target data channel for processing the abnormal data is determined from at least one data channel, wherein different data channels are used to process data reported through different reporting scenarios, each data channel includes at least one matching mode, and each matching mode includes matching rules and business benchmark data that needs to be referenced when using the matching mode.

[0066] In operation S130 , the abnormal data is input into the target data channel so as to compare the abnormal data with the reference data according to a matching rule based on a matching pattern in the target data channel.

[0067] In operation S140, when it is determined according to the matching rule that the abnormal data matches the business benchmark data, the matching pattern to which the matching rule belongs is used as a target matching pattern.

[0068] In operation S150, a target processor for processing abnormal data is determined according to the target matching pattern.

[0069] According to an embodiment of the present disclosure, a data channel is a transmission channel that allows two SCTP (Stream Control Transmission Protocol) endpoints to use a data protocol to transmit data over the data channel, wherein the data channel has a high processing efficiency for data and can ensure the integrity and accuracy of the data.

[0070] According to an embodiment of the present disclosure, the reporting scenario may include at least one of the following: a terminal reporting scenario, a program capture scenario, and other scenarios.

[0071] Among them, the terminal reporting scenario can refer to the scenario where sites such as business halls report abnormal data. The abnormal data in this scenario may include data on packages that are damaged, lost, or have incorrect delivery information. Among them, the site may include but is not limited to an express station or a business hall.

[0072] The program crawling scenario may refer to the scenario of using program crawling tools to crawl abnormal data. The abnormal data in this scenario may include data of packages that are lost within the station, lost during handover, or lost during package collection. Among them, handover loss may refer to the phenomenon that a package is lost when the station delivers the package to the delivery person, and package collection loss refers to the situation that when multiple small packages are packaged into a larger package, there are no small packages in the larger package.

[0073] According to an embodiment of the present disclosure, the target processor may include a department responsible for processing the package corresponding to the abnormal data. For example, when the actual recipient is different from the preset recipient corresponding to the package during package delivery, the target processor may be the delivery department.

[0074] like Figure 2 As shown, the number of data channels may include but is not limited to 3, namely data channel 1, data channel 2 and data channel 3, and they correspond to different reporting scenarios respectively, wherein each data channel includes at least one matching mode, wherein the sources of the business benchmark data of different matching modes in the same data channel are different, but the sources of the business benchmark data of different matching modes in different data channels may be the same, for example Figure 2 The sources of the business benchmark data for matching mode 1 and matching mode 7 in data channel 1 are different, while in the three data channels, the sources of the business benchmark data in matching mode 1 and matching mode 2 are the same, the sources of the business benchmark data in matching mode 3 and matching mode 4 are the same, and the sources of the business benchmark data in matching mode 5, matching mode 6 and matching mode 7 are the same.

[0075] It should be noted that the number of data channels and the number of matching patterns in each data channel are not limited to Figure 2 The quantity shown can be specifically set according to the needs of the actual usage scenario.

[0076] According to an embodiment of the present disclosure, the reporting scenario of abnormal data is first identified to determine the target data channel for processing the abnormal data. For example, data channel 2 can be determined as the target data channel, and the abnormal data is input into data channel 2.

[0077] According to an embodiment of the present disclosure, the abnormal data in data channel 2 is matched with matching pattern 2, matching pattern 3 and matching pattern 5 in data channel 2 in sequence. For example, in the process of matching with matching pattern 2, when the matching rule corresponding to matching pattern 2 determines that the abnormal data does not match matching pattern 2, matching is performed using matching pattern 3. When the matching rule corresponding to matching pattern 3 determines that the abnormal data matches matching pattern 3, matching pattern 3 is used as a target matching pattern. At the same time, matching is continued using matching pattern 5. Assuming that the matching rule of matching pattern 5 determines that the abnormal data also matches the business benchmark data of matching pattern 5, matching pattern 5 is also used as a target matching pattern.

[0078] According to the embodiments of the present disclosure, the target processor for processing abnormal data is finally determined according to matching mode 3 and matching mode 5. For example, in the process of comparing the business benchmark data in matching mode 3 or matching mode 5 with the abnormal data, it is found that the abnormal data is different from the business benchmark data. In other words, the abnormal data matches the matching mode corresponding to the business benchmark data. For example, the actual number of items in the package corresponding to the abnormal data is 2, while the preset number of items in the business benchmark data corresponding to the package is 3. At this time, it means that the collection department has an abnormality in the process of distribution or acceptance. Therefore, the processor corresponding to the abnormal data can be the collection department, and then the collection department can be determined as the target processor for processing the abnormal data.

[0079] According to an embodiment of the present disclosure, a target data channel for processing abnormal data is determined from at least one data channel according to a reporting scenario of abnormal data, and within the target data channel, reference business benchmark data in at least one matching pattern is compared with the abnormal data. When it is determined through a matching rule in the matching pattern that the reference business benchmark data matches the abnormal data, a matching pattern corresponding to the matching rule is determined as a target matching pattern, and the matching pattern is determined as a technical means for processing the target processor of the abnormal data. Therefore, the technical problem of low accuracy and timeliness of manually matching the processor corresponding to the abnormal data is at least partially overcome, thereby improving the accuracy and timeliness of assigning target processors to the abnormal data.

[0080] According to an embodiment of the present disclosure, the data processing method may further include the following operations.

[0081] The abnormal data to be processed is marked with a reporting scenario, so as to determine a target data channel for processing the abnormal data from at least one data channel according to the reporting scenario of the abnormal data.

[0082] According to an embodiment of the present disclosure, a data channel includes multiple matching modes, wherein the multiple matching modes are connected in series to form the data channel.

[0083] According to the embodiments of the present disclosure, Figure 2 As shown, the abnormal data in data channel 2 can be matched with matching pattern 2, matching pattern 3 and matching pattern 5 in data channel 2 in sequence.

[0084] For example, in the process of matching abnormal data with matching pattern 2, when the matching rule of matching pattern 2 determines that the abnormal data does not match the business benchmark data of matching pattern 2, matching pattern 3 is then used to match the abnormal data. Assuming that the matching rule of matching pattern 3 determines that the abnormal data matches the business benchmark data of matching pattern 3, matching pattern 3 is used as a target matching pattern, and matching pattern 5 is continued to be used to match the abnormal data. When the matching rule of matching pattern 5 determines that the abnormal data matches the business benchmark data of matching pattern 5, matching pattern 5 is also used as a target matching pattern, and the target processing party for processing the abnormal data is determined based on matching pattern 3 and matching pattern 5.

[0085] Assuming that the matching rules of matching pattern 3 also determine that the abnormal data does not match the business benchmark data of matching pattern 3, continue to use matching pattern 5 to match the abnormal data. Assuming that the matching rules of matching pattern 5 determine that the abnormal data matches the business benchmark data of matching pattern 5, use matching pattern 5 as the target matching pattern and determine the target processing party for processing the abnormal data based on matching pattern 5.

[0086] According to an embodiment of the present disclosure, since multiple matching patterns are connected in series, when a data channel processes abnormal data, it is necessary to compare the multiple matching patterns connected in series with the abnormal data one by one, so as to determine the target matching pattern from the multiple matching patterns.

[0087] According to an embodiment of the present disclosure, after determining a target processor for processing abnormal data according to a target matching pattern, the method may further include the following operations.

[0088] The abnormal data and the target processing party corresponding to the abnormal data are stored in a database.

[0089] Figure 3 A system principle diagram of a data processing method according to another embodiment of the present disclosure is schematically shown.

[0090] like Figure 3 As shown, storing the abnormal data and the target processing party corresponding to the abnormal data in the database may include the following operations.

[0091] The trigger is used to transmit the abnormal data and the target processing party corresponding to the abnormal data to the message selector, and the message selector is used to store the abnormal data in the database.

[0092] According to an embodiment of the present disclosure, a trigger can monitor the matching status of matching rules within a data channel. When it is detected that the matching rules of the matching pattern within the target data channel determine that the abnormal data matches the business benchmark data, the abnormal data and the target processor corresponding to the abnormal data are transmitted to the message selector, so that the message selector stores the abnormal data in a database.

[0093] According to an embodiment of the present disclosure, the matching mode includes at least one of the following: full process data matching mode, sealed vehicle data matching mode, unsealed vehicle data matching mode, and sorting and shipping data matching mode. The business benchmark data includes at least one of the following: full process benchmark data, sealed vehicle benchmark data, unsealed vehicle benchmark data, and sorting and shipping benchmark data.

[0094] According to an embodiment of the present disclosure, for example, Figure 3 In the illustrated embodiment, the business benchmark data for matching mode 1 and matching mode 2 can be full-process benchmark data, the business benchmark data for matching mode 3 and matching mode 4 can be vehicle sealing benchmark data, the business benchmark data for matching mode 5, matching mode 6, and matching mode 7 can be vehicle unsealing benchmark data, and the business benchmark data for matching mode 8 can be sorting and shipping benchmark data.

[0095] In the process of implementing the present disclosure, it is also found that after the abnormal data enters the data channel for the last time to process the matching pattern, the target processing party is obtained and stored in the database. However, since the package data corresponding to the abnormal data is still circulating, with the circulation of the package data, the target processing party obtained by the last matching may no longer be accurate. At this time, it is necessary to re-match the matching pattern in the data channel. However, when the abnormal data of different scenes enter the data channel again for matching, the time when the abnormal data of different scenes enter the data channel is different. Therefore, the time scheduler can be used to store the preset time interval and the abnormal data through the time wheel queue according to the configured preset time interval, and the abnormal data of different scenes can be transmitted to the corresponding data channel at regular intervals.

[0096] In view of this, the above data processing method may also include: Figure 3 The following operations are shown.

[0097] According to the reporting scenario of abnormal data, the abnormal data in the database is input into the target data channel again according to the preset time interval corresponding to the reporting scenario using the time scheduler, wherein different preset time intervals are applicable to different reporting scenarios.

[0098] Based on the matching pattern in the target data channel, the abnormal data is re-compared with the new business benchmark data in the matching pattern according to the matching rules to determine a new target matching pattern. A new target processing method for processing the abnormal data is determined according to the new target matching pattern.

[0099] According to an embodiment of the present disclosure, the preset time interval can be specifically set according to different reporting scenarios. For example, the preset time interval for reporting abnormal data of the terminal in the reporting scenario can be set to 8 hours, and the preset time interval for reporting abnormal data of data capture in the reporting scenario can be set to 24 hours.

[0100] According to an embodiment of the present disclosure, in the matching pattern in the target data channel, new business benchmark data is re-compared with the abnormal data according to the matching rules to determine a new target matching pattern, and then a new target processor for processing the abnormal data is determined according to the new target matching pattern.

[0101] like Figure 3 As shown, according to the reporting scenario of abnormal data, the abnormal data in the database is re-input into the target data channel according to the preset time interval corresponding to the reporting scenario using the time scheduler, which may include the following operations.

[0102] The preset time interval and abnormal data corresponding to the reporting scenario are sent to the time wheel queue by using the time scheduler. The preset time interval and abnormal data corresponding to the reporting scenario are generated into a timed task by using the time wheel queue. The time wheel queue is used to execute the timed task so that the abnormal data can be input into the target data channel again. The abnormal data and the preset time interval are obtained by the scheduler and stored in the time wheel queue, which can improve the circulation efficiency of the abnormal data and thus ensure the high throughput writing of the data.

[0103] According to an embodiment of the present disclosure, the time wheel queue may include a circular queue for storing timed tasks, the bottom layer of which uses an array, and each element in the array can store a timed task list (TimerTaskList). The timed task list is a circular bidirectional linked list, and each item in the linked list represents a timed task item (TimerTaskEntry), and the timed task item encapsulates the real timed task, such as the abnormal data and preset time interval described in the above method.

[0104] According to an embodiment of the present disclosure, the time round queue can execute multiple timing tasks to input abnormal data in the timing tasks into the target data channel again according to corresponding preset time intervals.

[0105] Figure 4 A system principle diagram of a data processing method according to another embodiment of the present disclosure is schematically shown.

[0106] like Figure 4 As shown, the above data processing method may also include the following operations.

[0107] Before the exception data to be processed is processed, the exception data to be processed is stored in a service framework for persistently storing log entry streams.

[0108] The exception data, target matching pattern, and target processing party are associated and stored in the service framework.

[0109] After the exception data to be processed is processed by the target processing party, the processed exception data is stored in the service framework.

[0110] According to an embodiment of the present disclosure, a service framework may include but is not limited to Bookkeeper. By using a service framework such as Bookkeeper to store the above-mentioned exception data, target matching patterns and target processors, the fault tolerance of the exception data can be improved. Among them, Bookkeeper can be used to store log streams. For example, it can be used as a persistent framework for the Apache Pulsar message queue. Bookkeeper can be used for data writing, such as high-fault-tolerance and high-throughput data writing based on replication. At the same time, high-throughput data writing also provides a guarantee for efficient data processing.

[0111] According to the embodiments of the present disclosure, when abnormal data enters the target data channel, the data to be circulated can be stored in the service framework, and the processed abnormal data, the target matching pattern, and the target processing party can be associated and stored in the service framework. By using the service framework, the backup of abnormal data can be achieved, and the backup abnormal data can be quickly obtained from the service framework when the data in the database is lost or damaged.

[0112] like Figure 4 As shown, the above data processing method also includes the following operations.

[0113] After comparing the abnormal data with the business benchmark data according to the matching rules, the event collector is used to record the comparison results.

[0114] like Figure 4 As shown, the above data processing method also includes the following operations.

[0115] A context manager is used to store the matching status of the exception data and the plurality of matching patterns, wherein the matching status includes whether the exception data matches or does not match the matching pattern.

[0116] According to an embodiment of the present disclosure, the abnormal data includes an abnormal ticket; the matching rule includes that the duration of the abnormal ticket staying in a site included in the business benchmark data exceeds a preset duration.

[0117] Comparing the abnormal data with the business benchmark data according to the matching rules may include the following operations.

[0118] Determine the actual length of time that the exception ticket stays at the sites included in the business benchmark data.

[0119] The actual duration is compared with the preset duration; when the actual duration exceeds the preset duration, it is determined that the abnormal data matches the business benchmark data; when the actual duration is less than or equal to the preset duration, it is determined that the abnormal data does not match the business benchmark data.

[0120] According to an embodiment of the present disclosure, an abnormal order may include, but is not limited to, an express delivery order with data abnormality caused by an abnormality in a package in the field of logistics technology.

[0121] According to an embodiment of the present disclosure, the preset duration may be set according to actual needs, for example, may include 24 hours.

[0122] According to an embodiment of the present disclosure, for example, after a parcel arrives at a site, the site does not process the parcel within a preset period of time, such as arranging a courier for delivery. In this case, it can be determined that the exception order of the parcel matches the business benchmark data, and the target processor of the exception data can be determined based on the matching pattern corresponding to the business benchmark data, such as the site.

[0123] According to an embodiment of the present disclosure, for example, if the site has processed the package within a preset time period, in other words, the actual time for processing the package is less than or equal to the preset time period, it can be determined that the exception list of the package does not match the actual business benchmark data, so that the business benchmark data of other matching modes can be compared with the exception data according to the matching rules.

[0124] According to an embodiment of the present disclosure, for example, the preset duration may be 8 hours. After the vehicle sealing department performs a vehicle sealing operation on a package, if the actual duration exceeds the preset duration and the abnormal data of the package is not updated in time, it can be determined that the abnormal data of the package matches the business benchmark data, so that the target processing party of the abnormal data can be determined, for example, the vehicle sealing department.

[0125] According to an embodiment of the present disclosure, for example, within a preset time period, if the next site updates the abnormal data of the package, it can be determined that the abnormal data of the package does not match the business benchmark data, so that the business benchmark data of other matching modes can be compared with the abnormal data according to the matching rules.

[0126] According to an embodiment of the present disclosure, the matching rule may further include that there is an abnormal order in the site included in the business benchmark data.

[0127] Comparing the abnormal data with the business benchmark data according to the matching rules may also include the following operations.

[0128] Determine whether there are exception orders in the sites included in the business benchmark data.

[0129] In the case where the site does not have an exception order, it is determined that the exception data matches the business benchmark data; in the case where the site has an exception order, it is determined that the exception data does not match the business benchmark data.

[0130] According to an embodiment of the present disclosure, for example, multiple small parcels are packaged into a large package at a collection site. When the large package is transported to a transit site, the multiple small parcels in the large package are identified. When it is identified that the small package corresponding to the exception order is missing among the multiple small parcels, the exception order corresponding to the missing small parcel is determined to match the business benchmark data, thereby determining the matching pattern corresponding to the business benchmark data as the target matching pattern, and determining the target processor based on the target matching pattern, for example, the target processor may be the collection site.

[0131] According to an embodiment of the present disclosure, for example, when a transit station identifies that a small package corresponding to an abnormal order is included in multiple small packages, the abnormal order corresponding to the missing small package is determined to be inconsistent with the business benchmark data, thereby utilizing the business benchmark data of other matching modes to compare with the abnormal data according to the matching rules.

[0132] According to an embodiment of the present disclosure, the matching rule may further include that the actual item information contained in the abnormal order in the business benchmark data is different from the preset item information.

[0133] Comparing the abnormal data with the business benchmark data according to the matching rules may also include the following operations.

[0134] Determine the preset item information included in the exception order in the business benchmark data.

[0135] The actual item information is compared with the preset item information; wherein, when the actual item information is different from the preset item information, it is determined that the abnormal data matches the business benchmark data; when the actual item information is the same as the preset item information, it is determined that the abnormal data does not match the business benchmark data.

[0136] According to an embodiment of the present disclosure, for example, when the collection department completes the packaging of a parcel according to the waybill information, the parcel includes preset items. When the parcel is delivered, if the actual item information in the parcel is different from the preset item information, it means that the collection department made an operation error when packaging the parcel, and it can be determined that the abnormal data matches the business benchmark data, so as to determine the target matching mode according to the matching mode corresponding to the business benchmark data, and determine the target processing party according to the target matching mode, for example, the target processing party can be the collection site.

[0137] According to an embodiment of the present disclosure, when the package is delivered, if the actual item information in the package is the same as the preset item information, it can be determined that the abnormal data does not match the business benchmark data, and then the business benchmark data of other matching modes can be compared with the abnormal data according to the matching rules.

[0138] According to the embodiments of the present disclosure, the above matching rules are only illustrative examples, and the present disclosure does not limit the matching rules. Those skilled in the art can set appropriate rules based on different reporting scenarios and different business requirements to achieve the effect of more accurately determining the processor of the abnormal order.

[0139] Figure 5 The block diagram schematically shows a data processing device according to an embodiment of the present disclosure.

[0140] like Figure 5 As shown, the data processing device 500 may include an acquisition module 510 , a first determination module 520 , a comparison module 530 , a second determination module 540 and a third determination module 550 .

[0141] The acquisition module 510 is used to acquire the abnormal data to be processed.

[0142] The first determination module 520 is used to determine a target data channel for processing abnormal data from at least one data channel according to the reporting scenario of the abnormal data, wherein different data channels are used to process data reported through different reporting scenarios, each data channel includes at least one matching mode, and each matching mode includes matching rules and business benchmark data that needs to be referenced when using the matching mode.

[0143] The comparison module 530 is used to input the abnormal data into the target data channel so as to compare the abnormal data with the business benchmark data according to the matching rules based on the matching pattern in the target data channel.

[0144] The second determination module 540 is configured to, when it is determined according to the matching rule that the abnormal data matches the business benchmark data, use the matching pattern to which the matching rule belongs as the target matching pattern.

[0145] The third determination module 550 is used to determine the target processing party for processing abnormal data according to the target matching pattern. It should be noted that the embodiments of the device part of the present disclosure are the same or similar to the embodiments of the method part of the present disclosure, and the present disclosure will not repeat them here.

[0146] According to an embodiment of the present disclosure, a target data channel for processing abnormal data is determined from at least one data channel according to a reporting scenario of abnormal data, and within the target data channel, reference business benchmark data in at least one matching pattern is compared with the abnormal data. When it is determined through a matching rule in the matching pattern that the reference business benchmark data matches the abnormal data, a matching pattern corresponding to the matching rule is determined as a target matching pattern, and the matching pattern is determined as a technical means for processing the target processor of the abnormal data. Therefore, the technical problem of low accuracy and timeliness of manually matching the processor corresponding to the abnormal data is at least partially overcome, thereby improving the accuracy and timeliness of assigning target processors to the abnormal data.

[0147] According to an embodiment of the present disclosure, the data processing device 500 may further include a first storage module.

[0148] The first storage module is used to store the abnormal data and the target processing party corresponding to the abnormal data in a database.

[0149] According to an embodiment of the present disclosure, the data processing device 500 may further include an input module, a fourth determination module, and a processing module.

[0150] The input module is used to input the abnormal data in the database into the target data channel again according to the reporting scenario of the abnormal data using the time scheduler at the preset time interval corresponding to the reporting scenario, wherein different reporting scenarios are applicable to different preset time intervals.

[0151] The fourth determination module is used to re-compare the abnormal data with the new business benchmark data in the matching pattern according to the matching rules based on the matching pattern in the target data channel, so as to determine a new target matching pattern.

[0152] The processing module is used to determine a new target processing party for processing abnormal data according to the new target matching pattern.

[0153] According to an embodiment of the present disclosure, the input module may include a transmitting unit, a generating unit and an input unit.

[0154] The sending unit is used to use the time scheduler to send the preset time interval and abnormal data corresponding to the reporting scenario to the time wheel queue.

[0155] The generating unit is used to generate a scheduled task using a time wheel queue to generate preset time intervals and abnormal data corresponding to the reporting scenario.

[0156] The input unit is used to execute the timing task by using the time wheel queue so as to input the abnormal data into the target data channel again.

[0157] According to an embodiment of the present disclosure, the data processing device 500 may further include a second storage module, a third storage module, and a fourth storage module.

[0158] The second storage module is used to store the exception data to be processed in a service framework for persistently storing log entry streams before processing the exception data to be processed.

[0159] The third storage module is used to associate and store the abnormal data, the target matching pattern, and the target processing party in the service framework.

[0160] The fourth storage module is used to store the processed exception data in the service framework after the target processing party has processed the exception data to be processed.

[0161] According to an embodiment of the present disclosure, a data channel includes multiple matching modes, wherein the multiple matching modes are connected in series to form the data channel.

[0162] According to an embodiment of the present disclosure, the matching mode includes at least one of the following: a full-process data matching mode, a sealed vehicle data matching mode, an unsealed vehicle data matching mode, and a sorting and shipping data matching mode.

[0163] The business benchmark data includes at least one of the following: full-process benchmark data, sealed vehicle benchmark data, unsealed vehicle benchmark data, and sorting and shipping benchmark data.

[0164] The reporting scenario includes at least one of the following: terminal reporting scenario and program capture scenario.

[0165] According to an embodiment of the present disclosure, the abnormal data includes an abnormal ticket, and the matching rule includes that the duration of the abnormal ticket staying in a site included in the business benchmark data exceeds a preset duration.

[0166] Figure 6 The block diagram of the comparison module according to the embodiment of the present disclosure is schematically shown.

[0167] like Figure 6 As shown, the comparison module 530 may include a first determination unit 531 , a comparison unit 532 , and a second determination unit 533 .

[0168] The first determining unit 531 is used to determine the actual duration of the abnormal order staying in the site included in the business benchmark data.

[0169] The comparison unit 532 is used to compare the actual duration with the preset duration; wherein, when the actual duration exceeds the preset duration, it is determined that the abnormal data matches the business benchmark data.

[0170] The second determining unit 533 is configured to determine that the abnormal data does not match the service benchmark data when the actual duration is less than or equal to the preset duration.

[0171] According to an embodiment of the present disclosure, the data processing device 500 may include a recording module.

[0172] The recording module is used to record the comparison results using the event collector after comparing the abnormal data with the business benchmark data according to the matching rules.

[0173] According to the modules and units of the embodiments of the present disclosure, any multiple or at least part of the functions of any multiple thereof can be implemented in one module. According to the modules and units of the embodiments of the present disclosure, any one or more can be split into multiple modules for implementation. According to the modules and units of the embodiments of the present disclosure, any one or more can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware of any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in any appropriate combination of any of them. Alternatively, according to the modules and units of the embodiments of the present disclosure, one or more can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0174] For example, any multiple of the acquisition module 510, the first determination module 520, the comparison module 530, the second determination module 540, and the third determination module 550 can be combined in one module / unit / subunit for implementation, or any one of the modules / units / subunits can be split into multiple modules / units / subunits. Alternatively, at least part of the functions of one or more of these modules / units / subunits can be combined with at least part of the functions of other modules / units / subunits and implemented in one module / unit / subunit. According to an embodiment of the present disclosure, at least one of the acquisition module 510, the first determination module 520, the comparison module 530, the second determination module 540, and the third determination module 550 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or by any one of the three implementation methods of software, hardware, and firmware, or by a suitable combination of any of them. Alternatively, at least one of the acquisition module 510, the first determination module 520, the comparison module 530, the second determination module 540 and the third determination module 550 may be at least partially implemented as a computer program module, which may perform a corresponding function when executed.

[0175] Figure 7 A block diagram of an electronic device suitable for implementing the method described above according to an embodiment of the present disclosure is schematically shown. Figure 7 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0176] like Figure 7 As shown, the electronic device 700 according to an embodiment of the present disclosure includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage part 708 to a random access memory (RAM) 703. The processor 701 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include an onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0177] In RAM 703, various programs and data required for the operation of electronic device 700 are stored. Processor 701, ROM 702 and RAM 703 are connected to each other via bus 704. Processor 701 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 702 and / or RAM 703. It should be noted that the program can also be stored in one or more memories other than ROM 702 and RAM 703. Processor 701 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0178] According to an embodiment of the present disclosure, the electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to the bus 704. The system 700 may further include one or more of the following components connected to the I / O interface 705: an input portion 706 including a keyboard, a mouse, etc.; an output portion 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 708 including a hard disk, etc.; and a communication portion 709 including a network interface card such as a LAN card, a modem, etc. The communication portion 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed, so that a computer program read therefrom is installed into the storage portion 708 as needed.

[0179] According to an embodiment of the present disclosure, the method flow according to an embodiment of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the processor 701, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0180] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0181] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include, but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device.

[0182] For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 702 and / or the RAM 703 described above and / or one or more memories other than the ROM 702 and the RAM 703 .

[0183] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0184] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations and / or combinations are not explicitly described in the present disclosure. In particular, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways without departing from the spirit and teachings of the present disclosure. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0185] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A data processing method, comprising: Get the exception data to be processed; According to the reporting scenario of the abnormal data, determine a target data channel for processing the abnormal data from at least one data channel, wherein different data channels are used to process data reported through different reporting scenarios, each of the data channels includes at least one matching mode, each of the matching modes includes matching rules and business benchmark data that needs to be referenced when using the matching mode, wherein the reporting scenario represents a scenario for uploading or capturing the abnormal data; Inputting the abnormal data into the target data channel so as to compare the abnormal data with the business benchmark data according to the matching rule based on the matching pattern in the target data channel; In the case where it is determined according to the matching rule that the abnormal data matches the business benchmark data, taking the matching mode to which the matching rule belongs as the target matching mode; A target processor for processing the abnormal data is determined according to the target matching pattern.

2. The data processing method according to claim 1, wherein: After determining a target processor for processing the abnormal data according to the target matching pattern, the method further includes: The abnormal data and the target processing party corresponding to the abnormal data are stored in a database.

3. The data processing method according to claim 2, further comprising: According to the reporting scenario of the abnormal data, the abnormal data in the database is input into the target data channel again by using a time scheduler at a preset time interval corresponding to the reporting scenario, wherein different preset time intervals are applicable to different reporting scenarios; Based on the matching pattern in the target data channel, re-comparing the abnormal data with the new business benchmark data in the matching pattern according to the matching rule to determine a new target matching pattern; A new target processor for processing the abnormal data is determined according to the new target matching pattern.

4. The data processing method according to claim 3, wherein: According to the reporting scenario of the abnormal data, using a time scheduler at a preset time interval corresponding to the reporting scenario, inputting the abnormal data in the database into the target data channel again includes: Using the time scheduler to send the preset time interval corresponding to the reporting scenario and the abnormal data to the time wheel queue; Using the time wheel queue to generate a scheduled task for the preset time interval corresponding to the reporting scenario and the abnormal data; and The timing task is executed by utilizing the time round queue so as to input the abnormal data into the target data channel again.

5. The data processing method according to claim 1, further comprising: Before processing the exception data to be processed, storing the exception data to be processed in a service framework for persistently storing a log entry stream; The abnormal data, the target matching pattern, and the target processing party are stored in association in the service framework; After the target processing party has processed the to-be-processed exception data, the processed exception data is stored in the service framework.

6. The data processing method according to claim 1, wherein: The data channel includes a plurality of matching modes, wherein the plurality of matching modes are connected in series to form the data channel.

7. The data processing method according to claim 1, wherein: The matching mode includes at least one of the following: full process data matching mode, sealed vehicle data matching mode, unsealed vehicle data matching mode, sorting and delivery data matching mode; The business benchmark data includes at least one of the following: full-process benchmark data, vehicle blocking benchmark data, vehicle unblocking benchmark data, and sorting and shipping benchmark data; The reporting scenario includes at least one of the following: a terminal reporting scenario and a program capture scenario.

8. The data processing method according to claim 7, wherein: The abnormal data includes an abnormal ticket; the matching rule includes that the abnormal ticket stays at the site included in the business benchmark data for a period exceeding a preset period; Comparing the abnormal data with the business benchmark data according to the matching rule includes: Determine the actual length of time that the abnormal order stays at the site included in the business benchmark data; The actual duration is compared with the preset duration; wherein, when the actual duration exceeds the preset duration, it is determined that the abnormal data matches the business benchmark data; and when the actual duration is less than or equal to the preset duration, it is determined that the abnormal data does not match the business benchmark data.

9. The data processing method according to claim 1, further comprising: After comparing the abnormal data with the business benchmark data according to the matching rule, the comparison result is recorded using an event collector.

10. A data processing device, comprising: An acquisition module is used to obtain the abnormal data to be processed; A first determination module is used to determine a target data channel for processing the abnormal data from at least one data channel according to a reporting scenario of the abnormal data, wherein different data channels are used to process data reported through different reporting scenarios, each of the data channels includes at least one matching mode, each of the matching modes includes a matching rule and business benchmark data that needs to be referenced when using the matching mode, wherein the reporting scenario represents a scenario for uploading or capturing the abnormal data; An input module, configured to input the abnormal data into the target data channel, so as to compare the abnormal data with the business benchmark data according to the matching rule based on the matching pattern in the target data channel; A second determination module is used to use the matching mode to which the matching rule belongs as a target matching mode when it is determined according to the matching rule that the abnormal data matches the business benchmark data; and The third determination module is used to determine a target processor for processing the abnormal data according to the target matching pattern.

11. A computer system comprising: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 9.

12. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enables the processor to implement the method according to any one of claims 1 to 9.

Images