Method and device for detecting repeated waybill data

Abstract

The invention discloses a detection method and device for repeated waybill data, and relates to the technical field of computers. A specific embodiment of the method comprises the following steps: in response to a detection instruction of repeated waybill data, obtaining operation log records of a first database and a second database in real time; for each acquired operation log record, extracting a user identifier and a user document number corresponding to the waybill data from the operation log record; and comparing the user identifier and the user document number with a data set in a specified time period, and detecting whether repeated waybill data exists in the first database and the second database by judging whether the user identifier and the user document number exist in the data set. According to the embodiment, the waybill data created by the new system and the old system can be monitored in real time on the premise of not invading the service system, and repeated waybill data detection is accurately performed, so that the service loss caused by system reconstruction is reduced, and no pressure is caused to the database.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method and device for detecting duplicate waybill data. Background Art

[0002] With the continuous development of business and market adjustments, the existing logistics platform system can no longer match the new business system organizational structure, and a new logistics platform system needs to be developed. However, in the process of switching between the old and new systems, a logistics waybill for the same customer will create a waybill data in both the new and old systems. At present, in the process of switching between the old and new systems, the problem of duplicate waybill data in the old and new systems is mostly avoided by periodically querying whether the same waybill data exists in the database of the old and new systems, or by making the old and new systems share cache resources.

[0003] In the process of implementing the present invention, the inventors found that there are at least the following problems in the prior art:

[0004] The method of periodically querying whether there is the same waybill data in the database of the new and old systems will put pressure on the database, and cannot discover problems in real time, and has a lag; the method of allowing the new and old systems to share cache resources will intrude into the system business logic, increase the complexity of the system business logic processing, and reduce the system business processing performance. Summary of the invention

[0005] In view of this, an embodiment of the present invention provides a method and device for detecting duplicate waybill data, which can achieve user-unawareness during the switching process between the new and old systems, monitor the waybill data created by the new and old systems in real time without affecting or intruding the business system, and accurately detect duplicate waybill data, thereby reducing business losses caused by system reconstruction without causing pressure on the database, thereby improving the accuracy of system order acceptance and enhancing the system order acceptance performance.

[0006] To achieve the above-mentioned purpose, according to one aspect of an embodiment of the present invention, a method for detecting duplicate waybill data is provided, comprising: in response to a detection instruction for duplicate waybill data, obtaining operation log records of a first database and a second database in real time, the operation log records including a user identifier and a user document number corresponding to the waybill data; for each operation log record obtained, extracting the user identifier and the user document number corresponding to the waybill data from the operation log record; comparing the user identifier and the user document number with a data set of a specified time period, and detecting whether duplicate waybill data exists in the first database and the second database by judging whether the user identifier and the user document number already exist in the data set, the data set storing the user identifier and the user document number corresponding to the waybill data created within the specified time period.

[0007] Optionally, the method further comprises: if the user identifier and the user document number do not exist in the data set, saving the user identifier and the user document number into the data set.

[0008] Optionally, whether the user identifier and the user document number already exist in the data set is determined to detect whether there is duplicate waybill data in the first database and the second database, including: based on the time window characteristics of the real-time data processing platform, attempting to save the user identifier and the user document number into the data set; if the saving is successful, it is determined that the user identifier and the user document number did not exist in the data set before, and there is no duplicate waybill data in the first database and the second database; if the saving is unsuccessful, it is determined that the waybill data corresponding to the user identifier and the user document number already exist in the data set, and there is duplicate waybill data in the first database and the second database.

[0009] Optionally, after obtaining the operation log records of the first database and the second database in real time, it also includes: saving the operation log records to the same message queue of the message platform; and monitoring the messages sent by the message platform through the real-time data processing platform to obtain each operation log record.

[0010] Optionally, the method further includes: generating duplicate waybill warning information when it is detected that duplicate waybill data exists in the first database and the second database.

[0011] According to another aspect of an embodiment of the present invention, a device for detecting duplicate waybill data is provided, comprising: a log record acquisition module, for responding to a duplicate waybill data detection instruction, and acquiring operation log records of a first database and a second database in real time, the operation log records including a user identifier and a user document number corresponding to the waybill data; a data extraction module, for extracting the user identifier and the user document number corresponding to the waybill data from the operation log record for each operation log record acquired; and a data comparison detection module, for comparing the user identifier and the user document number with a data set of a specified time period, and detecting whether duplicate waybill data exists in the first database and the second database by judging whether the user identifier and the user document number already exist in the data set, the data set storing the user identifier and the user document number corresponding to the waybill data created within the specified time period.

[0012] Optionally, the device further comprises a data saving module, configured to save the user identifier and the user document number into the data set if the user identifier and the user document number do not exist in the data set.

[0013] Optionally, the data comparison detection module is also used to: based on the time window characteristics of the real-time data processing platform, attempt to save the user identifier and the user document number into the data set; if the saving is successful, it is determined that the user identifier and the user document number did not exist in the data set before, and there is no duplicate waybill data in the first database and the second database; if the saving is unsuccessful, it is determined that the waybill data corresponding to the user identifier and the user document number already exists in the data set, and there is duplicate waybill data in the first database and the second database.

[0014] According to another aspect of an embodiment of the present invention, there is provided an electronic device, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method for detecting duplicate waybill data provided in an embodiment of the present invention.

[0015] According to another aspect of an embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored. When the program is executed by a processor, the method for detecting duplicate waybill data provided in an embodiment of the present invention is implemented.

[0016] An embodiment of the above invention has the following advantages or beneficial effects: by responding to the detection instruction of duplicate waybill data, the operation log records of the first database and the second database are obtained in real time, and the operation log records include the user identifier and the user document number corresponding to the waybill data; for each operation log record obtained, the user identifier and the user document number corresponding to the waybill data are extracted from the operation log record; the user identifier and the user document number are compared with the data set of the specified time period, and it is detected whether there is duplicate waybill data in the first database and the second database by judging whether the user identifier and the user document number already exist in the data set. The technical solution of storing the user identifier and the user document number corresponding to the waybill data created within the specified time period in the data set can achieve user-unawareness during the switching process between the new and old systems, and monitor the waybill data created by the new and old systems in real time without affecting or intruding the business system, and accurately detect duplicate waybill data, thereby reducing business losses caused by system reconstruction, and will not cause pressure on the database, thereby improving the accuracy of system order acceptance and improving the system order acceptance performance.

[0017] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0019] Figure 1 is a schematic diagram of main steps of a method for detecting duplicate waybill data according to an embodiment of the present invention;

[0020] Figure 2 is a schematic diagram of a detection process of duplicate waybill data according to an embodiment of the present invention;

[0021] Figure 3 is a schematic diagram of main modules of a device for detecting duplicate waybill data according to an embodiment of the present invention;

[0022] Figure 4 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0023] Figure 5 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0024] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0025] It should be noted that the collection, collection, updating, analysis, processing, use, transmission, storage and other aspects of user personal information involved in the technical solution disclosed in the present invention are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information, network security and national security.

[0026] During the switchover process between the old and new systems of the logistics platform, the old and new systems need to run in parallel, and gradually switch the customer traffic in the old system to the new system. This switching process is also called switching. During the switching process between the old and new systems, the external software program interface of the logistics platform is imperceptible to the customer. After receiving the customer's request to generate a waybill, the adaptation layer first determines whether the waybill data of the customer (for example, the merchant of the e-commerce platform) needs to be switched. If the customer needs to switch to the new system, the waybill data is created in the new system and stored in the warehouse; otherwise, the waybill data is created in the original old system and stored in the warehouse.

[0027] In order to monitor whether the new and old systems repeatedly create data for the same waybill number of the customer, the existing implementation schemes are generally divided into two types: one is to query according to the scheduled task at a fixed time period, and the other is to share the same resource to determine the unique value. Specifically, the first scheme is to query the old system database in a fixed time period through the scheduled task to see whether the order in the time period also exists in the database of the new system. If so, it means that the new and old systems have repeatedly created waybill data, and an early warning will be issued. However, the time interval of this scheme is not easy to define, and the business volume in different time periods is different, which will cause pressure on the database when the query data volume is too large; and this method can only perceive the problem when the scheduled task is running, resulting in a lag in problem perception. The second scheme is to use a shared cache resource. Before creating waybill data, the new and old systems use business logic to check whether a waybill data has been processed from the shared cache to avoid creating waybill data repeatedly. However, this scheme couples the two applications (new system and old system) that should have been independent of each other again, and is too invasive to the system architecture. The monitoring behavior of waybill data invades the business logic, which increases the complexity of order acceptance and reduces the performance of system order acceptance.

[0028] In order to solve the above technical problems, the present invention provides a method and device for detecting duplicate waybill data. During the switching process between the new and old systems, the duplicate ordering problem of waybill data created by both the new and old systems can be monitored in real time without affecting or intruding the business system, thereby reducing business losses caused by system reconstruction, improving the accuracy of system order acceptance, enhancing the system order acceptance performance, and increasing customers' trust in the platform.

[0029] In an embodiment of the present invention, in order to reduce the intrusion into the business system and not affect the order-taking performance of the original system, all waybill data information received by the logistics platform is collected by monitoring the database log file Binlog (Binlog is used to record the write operation information executed by the database and is saved on the disk in binary form), and BinLog is sent to the message platform to avoid abnormalities in the subsequent monitoring logic processing due to excessive business volume. Then, the real-time data processing platform monitors the message platform to process real-time data, and uses the time window of the real-time data processing platform to put the customer's unique order number corresponding to the customer's waybill data within the most recent specified time period (for example, the last 30 days or other appropriate time) into the cache. Each time the real-time data processing platform consumes a message in the message platform, it determines whether the customer's unique order number corresponding to the customer's waybill data exists in the cache to perform abnormal alarms and processing operations.

[0030] Figure 1 Schematic diagram of the main steps of the method for detecting duplicate waybill data according to an embodiment of the present invention. Figure 1As shown, the method for detecting duplicate waybill data according to an embodiment of the present invention mainly includes the following steps S101 to S103.

[0031] Step S101: In response to the detection instruction of duplicate waybill data, the operation log records of the first database and the second database are obtained in real time, and the operation log records include the user identifier and the user document number corresponding to the waybill data. According to an embodiment of the present invention, the first database and the second database are, for example, databases corresponding to the new and old systems, respectively. The operation log records of the database are, for example, the BinLog (binary log records SQL statements that have or potentially changed the data and are saved in binary form on the disk) data of the database, so that the monitoring system can monitor all waybill data.

[0032] In an embodiment of the present invention, BinLog data includes, for example: order time, user identification (such as the merchant number of the e-commerce platform), the user's own user document number when placing an order (it can also be other information used to identify the unique attribute value of the merchant document), etc.

[0033] Step S102: For each operation log record obtained, extract the user ID and user document number corresponding to the waybill data from the operation log record. In an embodiment of the present invention, it is not necessary to perform duplicate waybill data detection based on the waybill data generated in the database, but it is necessary to perform duplicate waybill detection based on the user ID and user document number corresponding to the waybill data. Since each user's own user document number may be the same as the user document number of other users, the present invention needs to consider the two parameters of user ID and user document number to perform duplicate waybill data detection. In this way, when a merchant on the e-commerce platform needs to place an order on the logistics platform for his shopping order, even if the merchant repeats the operation on a shopping order when the system is switched, the logistics platform will only create one logistics waybill for the shopping order, thereby avoiding the situation of duplicate waybills. Here, the user ID is the merchant ID, and the user document number is the order number of the shopping order.

[0034] Step S103: Compare the user ID and the user document number with the data set of the specified time period, and detect whether there is duplicate waybill data in the first database and the second database by judging whether the user ID and the user document number already exist in the data set. The data set stores the user ID and the user document number corresponding to the waybill data created within the specified time period. In an embodiment of the present invention, the data set is, for example, a cache, and its data storage and query efficiency are relatively high. In general, when judging whether there is duplicate waybill data, it is only necessary to judge the data within the most recent period of time. In order to facilitate rapid data comparison, the specified time period can be set to the most recent month or the most recent week, etc. according to business needs.

[0035] When the user ID and the user document number already exist in the data set, it means that the waybill data for the user ID and the user document number have been created and saved in a certain database (which may be the first database or the second database) before this processing. At this time, new waybill data is created and saved in another database. It can be judged that there are duplicate waybill data in the first database and the second database, and the duplicate waybill data can be obtained from the new and old systems respectively according to the user ID and the user document number. Then, accordingly, when the user ID and the user document number do not exist in the data set, it means that the waybill data for the user ID and the user document number have not been created before this processing, and it can be judged that there are no duplicate waybill data corresponding to the user ID and the user document number in the first database and the second database.

[0036] According to one embodiment of the present invention, when the user ID and the user document number do not exist in the data set, the user ID and the user document number are saved in the data set. When the user ID and the user document number do not exist in the data set, it means that the user ID and the user document number have not generated waybill data at this time, but in order to avoid the user ID and the user document number generating duplicate waybill data in the future, it is necessary to save the user ID and the user document number in the data set for further determining whether duplicate waybill data will be generated according to the user ID and the user document number in the future, so as to ensure the uniqueness of the waybill data.

[0037] According to another embodiment of the present invention, by determining whether the user identifier and the user document number already exist in the data set to detect whether there is duplicate waybill data in the first database and the second database, it can specifically include: based on the time window characteristics of the real-time data processing platform, trying to save the user identifier and the user document number into the data set; if the saving is successful, it is determined that the user identifier and the user document number did not exist in the data set before, and there is no duplicate waybill data in the first database and the second database; if the saving is unsuccessful, it is determined that the waybill data corresponding to the user identifier and the user document number already exists in the data set, and there is duplicate waybill data in the first database and the second database. According to an embodiment of the present invention, data processing is performed based on the time window characteristics of the real-time data processing platform, which can ensure that the data stored in the data set is within a specified time period. When determining whether the user identifier and the user document number exist in the data set, it can be achieved by trying to save the user identifier and the user document number into the data set. If the saving is successful, it means that the user identifier and the user document number did not exist in the data set before, and if the saving cannot be done, it means that there was already a record corresponding to the user identifier and the user document number in the data set.

[0038] According to another embodiment of the present invention, when duplicate waybill data is detected in the first database and the second database, duplicate waybill warning information is generated. When duplicate waybill data is detected in the two databases corresponding to the new and old systems, duplicate waybill warning information is generated to remind the processing personnel to handle it, not to repeat the shipment, and stop the loss in time.

[0039] According to another embodiment of the present invention, after obtaining the operation log records of the first database and the second database in real time, the operation log records can also be saved in the same message queue of the message platform; and the message sent by the message platform is monitored by the real-time data processing platform to obtain each operation log record. After the operation logs of the two databases corresponding to the new and old systems are monitored in real time, the monitored operation logs will be saved in the same message queue of the message platform, and after the real-time data processing platform monitors the message sent by the message platform, each operation log record is obtained from the message queue. By saving the operation log records to the message platform, it is convenient to cut the peak of the business peak period and avoid the blocking of business processing caused by the processing of log records. Among them, the same message queue of the message platform is, for example, the same message queue of the message middleware or the same topic topic of the Kafka message platform or other message platforms, etc., so that it is convenient to make repeated judgments.

[0040] Figure 2FIG. 1 is a schematic diagram of a detection process of duplicate waybill data according to an embodiment of the present invention. Figure 2 As shown, in one embodiment of the present invention, first, the Binlog data of the database of the new and old systems are monitored in real time, so that the monitoring system can monitor all the document data. Then, the monitored BinLog data is sent to the message platform, where the message platform is, for example, KafKa or other message platforms. After that, a real-time data processing platform (such as Flink or other real-time data processing tools) is used to monitor the messages sent by the message platform, so as to take out the messages from the message platform for consumption, so as to parse the messages to obtain the user identifier and the user document number. Finally, the user identifier and the user document number are saved in the cache using the Flink time window. Each time the user identifier and the user document number are processed, it is checked whether the user identifier and the user document number exist in the cache. If the user identifier and the user document number exist in the cache, a duplicate order alarm is issued so that the loss can be stopped in time after receiving the alarm; if the user identifier and the user document number do not exist in the cache, the user identifier and the user document number are saved in the cache.

[0041] Figure 3 Schematic diagram of the main modules of the device for detecting duplicate waybill data according to an embodiment of the present invention. Figure 3 As shown, the duplicate waybill data detection device 300 of the embodiment of the present invention mainly includes a log record acquisition module 301, a data extraction module 302 and a data comparison detection module 303.

[0042] A log record acquisition module 301, configured to acquire operation log records of the first database and the second database in real time in response to a detection instruction for duplicate waybill data, wherein the operation log records include a user identifier and a user document number corresponding to the waybill data;

[0043] The data extraction module 302 is used to extract the user identifier and the user document number corresponding to the waybill data from each operation log record obtained;

[0044] The data comparison detection module 303 is used to compare the user identifier and the user document number with a data set of a specified time period, and to detect whether there is duplicate waybill data in the first database and the second database by judging whether the user identifier and the user document number already exist in the data set. The data set stores the user identifier and the user document number corresponding to the waybill data created within the specified time period.

[0045] According to one embodiment of the present invention, the device 300 also includes a data saving module (not shown in the figure), which is used to: save the user identifier and the user document number into the data set when the user identifier and the user document number do not exist in the data set.

[0046] According to another embodiment of the present invention, the data comparison detection module 303 can also be used for: based on the time window characteristics of the real-time data processing platform, trying to save the user identifier and the user document number into the data set; if the saving is successful, it is determined that the user identifier and the user document number did not exist in the data set before, and there is no duplicate waybill data in the first database and the second database; if the saving is unsuccessful, it is determined that the waybill data corresponding to the user identifier and the user document number already exists in the data set, and there is duplicate waybill data in the first database and the second database.

[0047] According to another embodiment of the present invention, the device 300 also includes a log saving module (not shown in the figure), which is used to: after obtaining the operation log records of the first database and the second database in real time, save the operation log records to the same message queue of the message platform; and monitor the messages sent by the message platform through the real-time data processing platform to obtain each operation log record.

[0048] According to another embodiment of the present invention, the device 300 further includes an alarm information generation module (not shown in the figure), which is used to generate duplicate waybill alarm information when duplicate waybill data is detected in the first database and the second database.

[0049] According to the technical solution of the embodiment of the present invention, by responding to the detection instruction of duplicate waybill data, the operation log records of the first database and the second database are obtained in real time, and the operation log records include the user identifier and the user document number corresponding to the waybill data; for each operation log record obtained, the user identifier and the user document number corresponding to the waybill data are extracted from the operation log record; the user identifier and the user document number are compared with the data set of the specified time period, and it is determined whether the user identifier and the user document number already exist in the data set to detect whether there is duplicate waybill data in the first database and the second database. The data set stores the user identifier and the user document number corresponding to the waybill data created within the specified time period. This technical solution can achieve user-unawareness during the switching process between the new and old systems, monitor the waybill data created by the new and old systems in real time without affecting or intruding the business system, and accurately detect duplicate waybill data, thereby reducing business losses caused by system reconstruction without causing pressure on the database, thereby improving the accuracy of system order acceptance and improving the system order acceptance performance.

[0050] Figure 4 An exemplary system architecture 400 is shown to which a method for detecting duplicate waybill data or an apparatus for detecting duplicate waybill data according to an embodiment of the present invention may be applied.

[0051] like Figure 4 As shown, system architecture 400 may include terminal devices 401, 402, 403, network 404 and server 405. Network 404 is used to provide a medium for communication links between terminal devices 401, 402, 403 and server 405. Network 404 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0052] Users can use terminal devices 401, 402, 403 to interact with server 405 through network 404 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 401, 402, 403, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0053] The terminal devices 401 , 402 , and 403 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0054] The server 405 may be a server that provides various services, such as a backend management server (for example only) that provides support for shopping websites browsed by users using terminal devices 401, 402, and 403. The backend management server may respond to the detection instruction of duplicate waybill data for the received detection request of duplicate waybill data and other data, and obtain the operation log records of the first database and the second database in real time, wherein the operation log records include the user identifier and the user document number corresponding to the waybill data; for each operation log record obtained, the user identifier and the user document number corresponding to the waybill data are extracted from the operation log record; the user identifier and the user document number are compared with the data set of the specified time period, and the first database and the second database are detected to determine whether the user identifier and the user document number already exist in the data set to detect whether there is duplicate waybill data, wherein the data set stores the user identifier and the user document number corresponding to the waybill data created within the specified time period, and feeds back the processing result (for example, the detection result - for example only) to the terminal device.

[0055] It should be noted that the method for detecting duplicate waybill data provided in the embodiment of the present invention is generally executed by the server 405 , and accordingly, the device for detecting duplicate waybill data is generally disposed in the server 405 .

[0056] It should be understood that Figure 4 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0057] Reference below Figure 5 , which shows a schematic diagram of the structure of a computer system 500 of a terminal device or a server suitable for implementing an embodiment of the present invention. Figure 5 The terminal device or server shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0058] like Figure 5 As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage part 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the system 500 are also stored. The CPU 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0059] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, etc.; an output section 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed, so that a computer program read therefrom is installed into the storage section 508 as needed.

[0060] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the removable medium 511. When the computer program is executed by the central processing unit (CPU) 501, the above-mentioned functions defined in the system of the present invention are executed.

[0061] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0062] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0063] The units or modules involved in the embodiments of the present invention may be implemented in software or hardware. The units or modules described may also be provided in a processor. For example, they may be described as follows: a processor includes a log record acquisition module, a data extraction module, and a data comparison detection module. The names of these units or modules do not, in certain cases, constitute limitations on the units or modules themselves. For example, the log record acquisition module may also be described as a "module for acquiring operation log records of the first database and the second database in real time in response to detection instructions for duplicate waybill data."

[0064] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes: in response to a detection instruction for duplicate waybill data, real-time acquisition of operation log records of the first database and the second database, the operation log record including the user identifier and user document number corresponding to the waybill data; for each operation log record acquired, extracting the user identifier and user document number corresponding to the waybill data from the operation log record; comparing the user identifier and the user document number with the data set of the specified time period, and detecting whether the user identifier and the user document number already exist in the data set to determine whether the first database and the second database have duplicate waybill data, the data set stores the user identifier and user document number corresponding to the waybill data created within the specified time period.

[0065] According to the technical solution of the embodiment of the present invention, by responding to the detection instruction of duplicate waybill data, the operation log records of the first database and the second database are obtained in real time, and the operation log records include the user identifier and the user document number corresponding to the waybill data; for each operation log record obtained, the user identifier and the user document number corresponding to the waybill data are extracted from the operation log record; the user identifier and the user document number are compared with the data set of the specified time period, and it is determined whether the user identifier and the user document number already exist in the data set to detect whether there is duplicate waybill data in the first database and the second database. The data set stores the user identifier and the user document number corresponding to the waybill data created within the specified time period. This technical solution can achieve user-unawareness during the switching process between the new and old systems, monitor the waybill data created by the new and old systems in real time without affecting or intruding the business system, and accurately detect duplicate waybill data, thereby reducing business losses caused by system reconstruction without causing pressure on the database, thereby improving the accuracy of system order acceptance and improving the system order acceptance performance.

[0066] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for detecting duplicate waybill data, characterized in that: include: In response to a detection instruction for duplicate waybill data, obtaining operation log records of the first database and the second database in real time, the operation log records including a user identifier and a user document number corresponding to the waybill data; For each operation log record obtained, extract the user identifier and user document number corresponding to the waybill data from the operation log record; The user ID and the user document number are compared with a data set of a specified time period, and whether the user ID and the user document number already exist in the data set is determined to detect whether there is duplicate waybill data in the first database and the second database. The data set stores user IDs and user document numbers corresponding to waybill data created within the specified time period.

2. The method according to claim 1, characterized in that The method further comprises: In the case that the user identifier and the user document number do not exist in the data set, the user identifier and the user document number are saved in the data set.

3. The method according to claim 1, characterized in that: Detecting whether there is duplicate waybill data in the first database and the second database by judging whether the user identifier and the user document number already exist in the data set includes: Based on the time window characteristics of the real-time data processing platform, try to save the user identifier and the user document number into the data set; If the saving is successful, it is determined that the user identifier and the user document number did not exist in the data set before, and there is no duplicate waybill data in the first database and the second database; If the saving fails, it is determined that waybill data corresponding to the user identifier and the user document number already exist in the data set, and duplicate waybill data exist in the first database and the second database.

4. The method according to claim 1, characterized in that After obtaining the operation log records of the first database and the second database in real time, the method further includes: The operation log record is saved in the same message queue of the message platform; And, the real-time data processing platform monitors the messages sent by the message platform to obtain each operation log record.

5. The method according to claim 1, characterized in that The method further comprises: When it is detected that duplicate waybill data exists in the first database and the second database, duplicate waybill warning information is generated.

6. A device for detecting duplicate waybill data, characterized in that: include: A log record acquisition module, configured to acquire operation log records of the first database and the second database in real time in response to a detection instruction of duplicate waybill data, wherein the operation log records include a user identifier and a user document number corresponding to the waybill data; A data extraction module, for extracting, for each acquired operation log record, a user identifier and a user document number corresponding to the waybill data from the operation log record; A data comparison detection module is used to compare the user identifier and the user document number with a data set of a specified time period, and to detect whether there is duplicate waybill data in the first database and the second database by judging whether the user identifier and the user document number already exist in the data set, wherein the data set stores the user identifier and the user document number corresponding to the waybill data created within the specified time period.

7. The device according to claim 6, characterized in that The device also includes a data storage module, which is used to: In the case that the user identifier and the user document number do not exist in the data set, the user identifier and the user document number are saved in the data set.

8. The device according to claim 6, characterized in that The data comparison detection module is also used for: Based on the time window characteristics of the real-time data processing platform, try to save the user identifier and the user document number into the data set; If the saving is successful, it is determined that the user identifier and the user document number did not exist in the data set before, and there is no duplicate waybill data in the first database and the second database; If the saving fails, it is determined that waybill data corresponding to the user identifier and the user document number already exist in the data set, and duplicate waybill data exist in the first database and the second database.

9. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 5.

10. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images