Order data processing method, system, device and storage medium based on Kafka technology

Abstract

The present invention discloses an order data processing method, system, device and storage medium based on Kafka technology, including: pre-creating an order server as a message producer through a Kafka cluster, and creating a first topic title for storing direct interaction order records, and a processing server that directly processes data interaction is a message subscriber of the first topic title; the order server grabs logistics order information with a direct interaction identifier from current logistics order data through a Kafka message queue, and publishes the information to the first topic title; the processing server consumes the logistics order information published to the first topic title, and after the logistics order information is processed by a second business, a terminal matching the logistics order is found to complete data transmission. The present invention grabs the marked logistics order through a Kafka queue, and the processing server independently processes the second business, and the information push omits the intermediate link, saves processing time, improves processing efficiency, and has high security.

Description

Technical Field

[0001] The present invention belongs to the technical field of logistics order processing, and in particular relates to an order data processing method, system, device and storage medium based on Kafka technology. Background Art

[0002] The full name of OMS system is "Order Management System" in Chinese and "OMS" in English. OMS order management system is a part of logistics management system. It manages and tracks the orders placed by customers, dynamically grasps the progress and completion of orders, improves the operation efficiency in the logistics process, thereby saving operation time and operation costs, and improving the market competitiveness of logistics companies. OMS system is positioned to provide more complete logistics outsourcing services for both trading parties, and provides users with integrated one-stop supply chain services through unified orders. Order management and order tracking management can satisfy users' logistics services throughout the process.

[0003] The OMS order management system is positioned to provide a system for business communication based on orders, logistics services and control management for suppliers in the upstream of the supply chain and buyers in the downstream, as well as various logistics service providers that provide various logistics services. Through the management and control of orders, the delivery plan can be arranged reasonably and the inventory can be effectively reduced. The logistics industry has also developed rapidly, and a lot of business data will be circulated between logistics outlets and logistics headquarters or between logistics outlets.

[0004] by Figure 1 For example, the logistics headquarters platform 1 at least sets up an order server 11 where the OMS order management system is located. The order server 11 communicates with each logistics outlet 2 through the network 4, and then interacts with the corresponding terminal 3 (such as the communication terminal where the courier is located) through the corresponding logistics outlet 2. The link for this kind of data transmission is long, and the security of data transmission is not high. In particular, when the logistics outlet 2 does not interact with the terminal 3, the order server 11 needs to continuously track the interaction record of the order data until the logistics outlet 2 and the terminal 3 complete the interaction, and then the message of the completion of the interaction can be fed back to the order server 11, which is slow to respond and occupies a large amount of network interaction resources.

[0005] Taking the cold chain attribute flash delivery order as an example, the order server 11 first batches the cold chain attribute flash delivery order data regularly, and learns whether the corresponding terminal 3 in the logistics network 2 sends the order to the corresponding client, and then calculates the logistics business fees required for the logistics network 2 to batch process the flash delivery order data according to the settlement model. The logistics business fee settlement is settled through the logistics headquarters, and the logistics headquarters transfers the settlement list to the logistics network 2, and the logistics network 2 transfers the money to the corresponding terminal 3. The entire data processing link is too long, the security is not high, and the accuracy is not high. Summary of the invention

[0006] The purpose of the present invention is to provide an order data processing method, system and storage medium based on Kafka technology, which can directly complete data transmission between the processor and the terminal after data processing based on Kafka technology.

[0007] An order data processing method based on Kafka technology, comprising:

[0008] An order server is created in advance through the Kafka cluster as a message producer, and a first topic title is created for storing direct interaction order records. The processing server that directly processes data interaction is the message subscriber of the first topic title;

[0009] The order server captures the logistics order information with the direct interaction identifier from the current logistics order data through the Kafka message queue, and publishes the information to the first topic title;

[0010] The processing server consumes the logistics order information published to the first topic title, performs second business processing on the logistics order information, finds a terminal matching the logistics order, and completes data transmission.

[0011] Furthermore, the processing server consumes the logistics order information published to the first topic title further including:

[0012] The processing server pre-sets and maintains offset information, where the offset information indicates location information of a partition where the current logistics order information consumed by the first topic title is located;

[0013] When the processing server is in an idle state, it uses the offset information to access the logistics order information corresponding to the adapted partition, and puts the logistics order information into the preset database in sequence, and then updates the offset information.

[0014] Furthermore, before performing the second business processing on the logistics order information, the following steps are also included:

[0015] Read the logistics order information, and parse the order identification information from the logistics order information;

[0016] The matching order logistics track information is obtained through the order identification, which at least includes the receipt information of the logistics order.

[0017] Furthermore, obtaining the matching order logistics track information through the order identifier further includes:

[0018] Pre-create a logistics track server as a message producer through the Kafka cluster, and create a second topic title for storing logistics track records, and the processing server is a message subscriber of the second topic;

[0019] The logistics track server captures the logistics track information that meets the preset requirements from the current logistics track data through the kafka message queue, and publishes the logistics track information to the second topic title;

[0020] The processing server consumes the logistics track information published to the second topic title, and saves the logistics track information into the record corresponding to the order identifier in the preset database.

[0021] Further, the second business processing of the logistics order information specifically includes:

[0022] The processing server calculates the order dispatch billing data in the preset database to generate payment information according to the preset order dispatch billing model;

[0023] A payment bill is generated based on the payment information so that the fee is paid to the terminal matched with the logistics order.

[0024] Furthermore, calculating the dispatch billing data in the preset database further includes:

[0025] The preset database is further configured to configure a first database, a second database and a dispatch and billing database;

[0026] The processing server accesses the logistics order information corresponding to the adapted partition through the offset information, and puts the logistics order information into the first database; and the processing server consumes the logistics track information published to the second topic title, and saves the logistics track information into the second database;

[0027] Regularly or in a triggered manner, updating the same order identification data of the first database and the second database to the dispatch and billing database;

[0028] The express delivery receipt time under the order identification is obtained from the dispatch and billing database, and the delivery time of the express delivery order is calculated to calculate the delivery fee information of the order.

[0029] Further, finding a terminal matching the logistics order to complete data transmission further includes:

[0030] Find the terminal that matches the logistics order;

[0031] The processing server is used to start the callback in the main thread, open the swoole_websocket_server service, and use swoole+Redis to push the data after the second business processing in real time, which is specifically:

[0032] The manager process and the worker process are created in advance;

[0033] Monitor timer;

[0034] When the data processed by the second business is monitored to be calculated, the worker process uses the swoole_server_task function to deliver the task sent to the corresponding terminal;

[0035] When the onTask callback function is called, the current Task process will switch the process state to busy, and will no longer receive new Tasks. When the onTask function returns, the process state will switch to idle and continue to receive new Tasks.

[0036] When the task delivered by the worker process is completed, the Task process will send the result of the task processing to the worker process through the $ws->finish() method, thereby completing the second business processed data and transmitting it to the corresponding terminal in a timely manner based on asynchronous event-driven and coroutine parallel network communication.

[0037] The present invention also provides an order data processing system based on Kafka technology, comprising:

[0038] Kafka cluster: used to create an order server as a message producer, and create a first topic title for storing direct interaction order records. The processing server that directly processes data interaction is the message subscriber of the first topic title;

[0039] Order server: used to capture logistics order information with direct interaction identifier from current logistics order data through Kafka message queue, and publish the information to the first topic title;

[0040] Processing server: used to consume the logistics order information published to the first topic title, perform second business processing on the logistics order information, and then find the terminal matching the logistics order to complete data transmission.

[0041] The present invention also provides an order data processing device based on Kafka technology, characterized in that:

[0042] Kafka cluster interaction unit: used to create an order server as a message producer in advance through the Kafka cluster, and create a first topic title for storing direct interaction order records. The processing server that directly processes data interaction is the message subscriber of the first topic title;

[0043] Publishing unit: used for capturing logistics order information with a direct interaction identifier from the current logistics order data through the Kafka message queue, and publishing the information to the first topic title;

[0044] The second business processing unit is used to consume the logistics order information published to the first topic title, perform the second business processing on the logistics order information, and then find the terminal matching the logistics order to complete the data transmission.

[0045] A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements any one of the above-mentioned order data processing methods based on Kafka technology.

[0046] Due to the adoption of the above technical solution, the present invention has the following advantages and positive effects compared with the prior art: the marked logistics orders are captured through the Kafka queue, the order server acts as the message producer, the processing server independently processes the second business, and the information processed by the second business is pushed to eliminate the intermediate links, thereby saving processing time, improving processing efficiency, and more importantly, increasing security. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 This is the schematic diagram of the existing OMS order management system;

[0048] Figure 2 This is the first implementation diagram of an order data processing system based on Kafka technology;

[0049] Figure 3This is the order data processing flow chart of the first example based on Kafka technology;

[0050] Figure 4 This is the second implementation diagram of an order data processing system based on Kafka technology;

[0051] Figure 5 This is a schematic diagram of an order data processing device based on Kafka technology. DETAILED DESCRIPTION

[0052] The following is a further detailed description of an order data processing method, system and storage medium based on Kafka technology proposed by the present invention in conjunction with the accompanying drawings and specific embodiments. The advantages and features of the present invention will become more apparent from the following description and claims.

[0053] The applicant found that: taking the cold chain attribute flash delivery order as an example, the order server first batches the cold chain attribute flash delivery order data regularly, and learns whether the corresponding terminal in the logistics network has sent the order to the corresponding client, and then calculates the logistics business expenses required for these batches of flash delivery order data of the logistics network according to the settlement model. The logistics business fee settlement is settled by the logistics headquarters, and the logistics headquarters transfers the settlement list obtained from the settlement to the logistics network. The flash delivery orders with cold chain attributes are processed at the level of tens of millions of orders every day. The order server can only calculate the logistics business expenses of the relevant terminal 3 of the logistics network after updating whether the terminal of the logistics network has sent the order to the corresponding client at night or at a regular time. The processing efficiency is slow and not timely. And the entire data transmission link is long. After analysis, the applicant knows that the amount of data processed is large, and the feedback of the terminal sending the order (whether the order is signed, the time of signing, etc.) is tracked by a single process.

[0054] To this end, the applicant introduced Kafka technology. Kafka was originally developed by Linkedin. It is a distributed, partitioned, multi-copy, multi-subscriber distributed log system (also known as MQ system) coordinated by Zookeeper (distributed application coordination service software). It can be commonly used for access logs, message services, etc. Linkedin contributed it to the Apache Software Foundation (ASF, Apache Software Foundation) in 2010 and became a top open source project. The main application scenarios are: log collection system and message system. A message system is responsible for transferring data from one application to another. The application only needs to focus on the data, not how the data is transferred between two or more applications. Distributed messaging is based on reliable message queues, asynchronously transferring messages between client applications and message systems. There are two main messaging modes: point-to-point messaging mode and publish-subscribe mode. Most messaging systems use the publish-subscribe mode. Kafka is a publish-subscribe mode.

[0055] See also Figure 2 , which is an implementation diagram of an order data processing system based on Kafka technology. It includes: an order server 21, a Kafka cluster 22, a first topic title 24, a processing server 23, a network 25, a logistics website 26 and each courier terminal 27 (hereinafter referred to as terminal 27).

[0056] Order server 21: an OMS order management system is set up therein to complete order data processing.

[0057] The processing server 23 is used to complete the processing service of direct interaction between the logistics platform and the terminal.

[0058] The processing server 23 can be a simple physical server or a logical server. In this example, the processing server 23 and the order server 21 can be in a server cluster or can be set in the same server. In addition to processing the conventional OMS order management, the order server 21 is also required to process the processed data as real-time and quickly as possible and send it directly to the corresponding terminal 27. The first innovation of the present invention is to divide the regular OMS order management business of the order server 21 and the timely processing business such as calculating the cost and sending it to the terminal 27 into two separate processing processes.

[0059] Kafka cluster 22: Kafka cluster 22 contains one or more servers, and the server nodes are called brokers. Brokers store topic (title) data. If a topic has N partitions and the cluster has N brokers (server nodes), then each broker stores one partition of the topic. If a topic has N partitions and the cluster has (N+M) brokers, then N of them store one partition of the topic. If a topic has N partitions and the number of brokers in the cluster is less than N, then one broker stores one or more partitions of the topic. Each message published to the Kafka cluster has a title, which is called a Topic. (Physically, messages from different Topics are stored separately. Logically, although messages from a Topic are stored on one or more brokers, users only need to specify the Topic of the message to produce or consume data without having to worry about where the data is stored). The data in a topic is divided into one or more partitions. Each topic has at least one partition. The data in each partition is stored using multiple segment files. The data in a partition is ordered, but the data between different partitions loses its order.

[0060] The order server 21 acts as a message producer, and the Kafka cluster 22 creates a first topic title 24 for storing direct interaction order records. The processing server 23 that directly processes data interaction is a message subscriber of the first topic title. The direct interaction order record is a relatively general description, indicating that the second business has been completed. When the second business is the express delivery fee settlement of the flash delivery, the first topic title 24 is used to save the logistics order information with the flash delivery logo. The first topic title 24 that stores the direct interaction order record is the logistics order information that stores the unique logo of the second business.

[0061] The logistics platform including the order server 21 and the processing server 23 can communicate with various logistics websites 26 and terminals 27 through the network 25.

[0062] First embodiment

[0063] See also Figure 3 , which is a flow chart of order data processing based on Kafka technology. It includes:

[0064] S110: Create an order server as a message producer in advance through the Kafka cluster, and create a first topic title for storing direct interaction order records. The processing server that directly processes data interaction is the message subscriber of the first topic title.

[0065] S120: The order server captures the logistics order information with the direct interaction identifier from the current logistics order data through the Kafka message queue, and publishes the information to the first topic title;

[0066] S130: The processing server consumes the logistics order information published to the first topic title, processes the logistics order information, finds a terminal that matches the logistics order, and completes the data transmission.

[0067] The core of the present invention is to divide the conventional order business and the second business (process the processed data as real-time and quickly as possible and then directly send it to the corresponding terminal 27) into two independent processes, which can be processed in parallel. The order business is carried out independently. In the order business, the logistics order information that meets the conditions is captured in real time or regularly from the current order processed in real time, and stored in the first topic title using the kafka technology. The processing server for the second business consumes the logistics order information in the first topic title when it is idle to complete the second business processing. The two independent processes mentioned above can be processed in parallel, and can be implemented by the same physical server or service cluster.

[0068] Considering the massive amount of order data processed on a logistics platform at the same time, the OMS order management system of a logistics platform may have to process tens of millions of order records within a day. Taking the "Quick Delivery" service as an example, the customer information marked with the "Quick Delivery" logo is first transmitted to the OMS order management system. In this way, during the real-time order processing, the order with the customer information can be marked with the "Quick Delivery" logo. The logistics order information with the "Quick Delivery" logo can be captured from the current logistics order data through the Kafka message queue and published to the first topic title. The "real-time" here is also relative. Generally, the logos that meet the "Quick Delivery" condition are captured in batches.

[0069] The processing server consumes the logistics order information published to the first topic title. In this example, we process it considering the massive amount of data. The first topic title can exist in multiple partitions, and the data in each partition is stored using multiple segment files. Each segment contains .log, .index, and .timeindex files, and each order information stored contains offset, message size, message body... Use the offset information to access the logistics order information corresponding to the adapted partition. The order server continuously places the logistics orders that meet the conditions (such as the second business-specific identifier) ​​in the current logistics order data in the first topic title. The first topic title has many partitions and multiple segments. According to the logic or storage method of the first topic title, the logistics orders with the second business-specific identifier are stored. The storage process of storing the first topic title is a separate process. And retrieving the logistics information is an additional separate process. In the present invention, the offset information is used to save the location information of the last logistics order information retrieved from the first topic title. When the processing server consumes the logistics order information published to the first topic title, it can find the location of the last consumed logistics order information through the offset information and then process the message. After the message is consumed, the offset information needs to be updated to complete the maintenance of the offset information.

[0070] In addition, the present invention usually presets an offset amplitude (such as N, relative offset) for preprocessing, and then adjusts the offset value according to a certain relative offset and performs preprocessing again until the processing effect reaches a preset range (the time when the second service is sent), and then saves the offset value.

[0071] For example, when searching for order data, segment+offset is used. What is the process if you need to search for a message with an offset of 378801?

[0072] First find the segment file where the offset 378801message is located (using binary search). The file found here is the second segment file.

[0073] Open the .index file in the found segment (that is, the 378796.index file, the starting offset of which is 378796+1, the offset of the message with offset 378801 that we are looking for in the index is 368796+5=378801, so the relative offset to be found here is 5). Since the file uses a sparse index to store the relationship between the relative offset and the corresponding message physical offset, the index with a relative offset of 5 cannot be found directly. Here, the binary search is also used to find the largest relative offset among the index entries with a relative offset less than or equal to the specified relative offset, so the index with a relative offset of 4 is found.

[0074] According to the index with relative offset 4, the physical offset of the message storage is determined to be 256. Open the data file and scan sequentially from the position 256 until the message with offset 378801 is found.

[0075] Use segment + ordered offset + sparse index + binary search + sequential search and other methods to efficiently find data! At this point, the processing server can get the data to be processed. So how does the processing server record the location of its consumption? The offset consumed by the processing server has been directly maintained in the __consumer_offsets topic of the kafk cluster!

[0076] That is, "processing the logistics order information published by the server to the first topic title" further includes:

[0077] The processing server pre-sets and maintains offset information, which indicates the location information of the currently processed order in the current partition partition consumed by the first topic title;

[0078] When the processing server is in idle state, it accesses the order information corresponding to the adapted partition through the offset information, puts the logistics order information into the preset database in sequence, and then updates the offset information.

[0079] Through the above method, the logistics order information that meets the conditions can be placed in a preset database. The preset database can be set up separately on the processing server, or it can be a space for storing data on the logistics platform, or even a cache space, that is, the preset database can be a physical database or a logical storage space. The processing server can directly process the data in the preset database for the second business. If the order contains the customer's confirmation of receiving the order, the time of receipt, and the terminal information, the billing information of the flash delivery in the second business can be directly calculated and transmitted to the corresponding terminal. This processing is efficient, does not occupy the existing order processing process, and does not have the problem of long transmission links for the second business and the inability to respond and feedback in a timely manner.

[0080] Second embodiment

[0081] When the order information processed by the OMS order management system of the logistics platform does not include information such as whether the customer has signed for the order and the time of receipt after signing, it is necessary to obtain such information in a timely manner before processing. The logistics track server of the logistics platform saves this information in real time. Of course, in principle, the logistics track server and the order server can also be a server or a server cluster. Considering the processing efficiency, the logistics real-time track process is logically virtualized to a logistics track server to achieve it. The logistics real-time track process independently obtains information such as whether the customer has signed for the order and the time of receipt after signing, without affecting the conventional logistics track server processing business.

[0082] See also Figure 4 , which is a second implementation diagram of an order data processing system based on Kafka technology. In addition to: an order server 21, a Kafka cluster 22, a first topic title 24, a processing server 23, a network 25, a logistics website 26 and each courier terminal 27 (hereinafter referred to as terminal 27), it also includes a logistics track server 28 and a second topic title 29.

[0083] Compared with the first example, this method adds the following process:

[0084] A logistics track server is created in advance through the Kafka cluster as a message producer, and a second topic title is created for storing logistics track records. The logistics track server for real-time processing of logistics tracks is the message subscriber of the second topic;

[0085] The logistics track server captures the logistics track information that meets the preset requirements from the current logistics track data through the Kafka message queue, and publishes the information to the second topic title that matches it;

[0086] The server consumes the logistics track information published to the second topic title, and saves the logistics track information into the records corresponding to the order identifiers in the preset database.

[0087] For example, the logistics track server stores the qualified records in the second topic title. If the qualified condition is that the goods are delivered by flash and signed by the customer, the qualified records are directly placed in the second topic title. The processing server consumes the logistics track information published to the second topic title and saves the logistics track information to the records corresponding to the order IDs in the preset database. Each order has a unique ID, which can be saved in the records with the same ID.

[0088] In this example, (1) the order server grabs the logistics order information with the direct interaction identifier from the current logistics order data through the Kafka message queue and publishes the information to the first topic title;

[0089] (2) Process the logistics order information published by the server to the first topic title and place it in the preset database;

[0090] (3) The logistics track server captures the logistics track information that meets the preset requirements from the current logistics track data through the Kafka message queue, and publishes the information to the matching second topic title;

[0091] (4) The processing server consumes the logistics track information published to the second topic title, and saves the logistics track information to the records corresponding to the order identifiers in the preset database;

[0092] The above four processes are independent processes and can be processed in parallel to improve processing efficiency.

[0093] In addition, a general table is set in the preset database. The information of the general table will be modified in process (2), and the information of the general table will also be modified in process (4). This scheme is a feasible scheme. Considering the security of data processing and storage, the preset database further includes a first database, a second database and a second business database. The logistics order information of process (2) is directly stored in the first database, and the logistics track information of process (4) is directly stored in the second database. The above two databases are only stored without modification. This ensures that the obtained data cannot be modified and also ensures the security of the data. The data of the first database and the second database are regularly updated to the second business database and saved according to the order identifier. When the second business is a dispatch billing business, the second business database is called a dispatch billing database. In this example, the logistics platform stores the information of the logistics website, terminal, etc. in a basic database. Therefore, in the actual operation of this example, the information of the logistics website, terminal, etc. must be imported into the dispatch billing database to complete the second business processing of dispatch billing.

[0094] Therefore, before "processing the logistics order information", it also includes: reading the current logistics order information, parsing the order identification information from the current logistics order information; obtaining the matching order logistics track information through the order identification, which at least includes the receipt information of the order.

[0095] “Calculating dispatch and billing data in the preset database” further includes:

[0096] Setting the preset database further sets the first database, the second database and the dispatch and billing database;

[0097] The processing server accesses the order information corresponding to the adapted partition through the offset information thread, and puts the logistics order information into the first database of the preset database; and the processing server thread consumes the logistics track information published to the second topic title, and saves the logistics track information into the second database of the preset database respectively;

[0098] Regularly or in a triggered manner, updating the same order identification data of the first database and the second database to the dispatch and billing database;

[0099] The express delivery receipt time under the order identification is obtained from the dispatch and billing database, and the delivery time of the express delivery order is calculated to calculate the delivery fee information of the order.

[0100] “Processing logistics order information” specifically includes:

[0101] The processing server calculates the order dispatch billing data in the preset database according to the pre-set order dispatch billing model and pushes the payment information. For example, the signing time - the collection time > the signing time. The rate information of the relevant fees is pre-configured. The rate information includes the type, name, basic rate, renewal rate, rate cap, etc. The rate unit is RMB / ticket or RMB / day / ticket. For example, if the type is payment item or receipt, and the name is single order dispatch fee, delivery point delay penalty, etc., the order dispatch billing model can calculate the payment information of the order according to the pre-set rate information.

[0102] Generate a payment bill based on the payment information so that the fee can be paid to the courier's corresponding terminal. Push the payment information to the fee settlement terminal, and use swoole+Redis to achieve real-time data push.

[0103] Generate a payment bill based on the payment information and make the payment. The fee settlement terminal will settle the fee, and the OMS order management system will confirm the refund. If the deduction status is failed, a push button will be displayed. Click it to push the deduction information. If the deduction status is successful, a refund button will be displayed. Click the pop-up box to remind you whether to refund. After confirmation, the refund operation is completed.

[0104] Finding the terminal that matches the logistics order to complete the data transmission further includes:

[0105] Find the terminal that matches the logistics order;

[0106] Use the processing server to start the callback in the main thread, open the swoole_websocket_server service, and use swoole+Redis to push the data processed by the second business in real time, which is specifically:

[0107] The manager process and the worker process are created in advance;

[0108] Monitor timer;

[0109] When the data after the second business processing is calculated, the worker process uses the swoole_server_task function to deliver the task sent to the corresponding terminal;

[0110] When the onTask callback function is called, the current Task process will switch the process state to busy, and will no longer receive new Tasks. When the onTask function returns, the process state will switch to idle and continue to receive new Tasks.

[0111] When the task delivered by the worker process is completed, the Task process will send the result of the task processing to the worker process through the $ws->finish() method, thereby completing the second business processing data and transmitting it to the corresponding terminal in a timely manner based on the parallel network communication of asynchronous event-driven and coroutine.

[0112] Third embodiment

[0113] This embodiment provides an order data processing device based on Kafka technology. Figure 5 The order data processing device 500 based on the Kafka technology may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 510 (for example, one or more processors) and a memory 520, and one or more storage media 530 (for example, one or more mass storage devices) storing application programs 533 or data 532. Among them, the memory 520 and the storage medium 530 can be short-term storage or persistent storage. The program stored in the storage medium 530 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the order data processing device 500 based on the Kafka technology.

[0114] Furthermore, the processor 510 may be configured to communicate with the storage medium 530 and execute a series of instruction operations in the storage medium 530 on the order data processing device 500 based on the Kafka technology.

[0115] The terminal code-based express delivery equipment may also include one or more power supplies 540, one or more wired or wireless network interfaces 550, one or more input and output interfaces 560, and / or one or more operating systems 531, such as Windows Serve, Vista, etc.

[0116] The order data processing device based on Kafka technology, in addition to the OMS order management system processing unit, further includes:

[0117] Kafka cluster interaction unit: used to create an order server as a message producer in advance through the Kafka cluster, and create a first topic title for storing direct interaction order records. The processing server that directly processes data interaction is the message subscriber of the first topic title;

[0118] Publishing unit: used for capturing logistics order information with a direct interaction identifier from the current logistics order data through the Kafka message queue, and publishing the information to the first topic title;

[0119] The second business processing unit is used to consume the logistics order information published to the first topic title, perform the second business processing on the logistics order information, and then find the terminal matching the logistics order to complete the data transmission.

[0120] The second business processing unit can independently perform concurrent processing with the OMS order management system processing unit, thereby improving the efficiency of the entire processing.

[0121] Those skilled in the art will understand that Figure 5 The order data processing device based on Kafka technology shown does not constitute a limitation of the order data processing device based on Kafka technology, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0122] Another embodiment of the present invention further provides a computer-readable storage medium.

[0123] The computer-readable storage medium may be a non-volatile computer-readable storage medium, or a volatile computer-readable storage medium. The computer-readable storage medium stores instructions, which, when executed on a computer, cause the computer to execute the steps of the order data processing method based on the Kafka technology in the first embodiment.

[0124] If the order data processing method based on Kafka technology is implemented in the form of program instructions and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of software, and the computer software is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0125] Those skilled in the art can clearly understand that, for the sake of convenience and brevity in description, the identification content specifically executed by the above-described system and device can refer to the corresponding process in the aforementioned method embodiment.

[0126] The embodiments of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above embodiments. Even if various changes are made to the present invention, if these changes fall within the scope of the claims of the present invention and their equivalents, they still fall within the protection scope of the present invention.

Claims

1. A method for processing order data based on Kafka technology, characterized in that: include: An order server is created in advance through the Kafka cluster as a message producer, and a first topic title is created for storing direct interaction order records. The processing server that directly processes data interaction is the message subscriber of the first topic title; The order server captures the logistics order information with the direct interaction identifier from the current logistics order data through the Kafka message queue, and publishes the information to the first topic title; The processing server consumes the logistics order information published to the first topic title, performs second business processing on the logistics order information, finds a terminal matching the logistics order, and completes data transmission; Wherein, finding the terminal matching the logistics order to complete data transmission further includes: Find the terminal that matches the logistics order; The processing server is used to start the callback in the main thread, open the swoole_websocket_server service, and use swoole+Redis to push the data after the second business processing in real time, which is specifically: The manager process and the worker process are created in advance; Monitor timer; When the data processed by the second business is monitored to be calculated, the worker process uses the swoole_server_task function to deliver the task sent to the corresponding terminal; When the onTask callback function is called, the current Task process will switch the process state to busy, and will no longer receive new Tasks. When the onTask function returns, the process state will switch to idle and continue to receive new Tasks. When the task delivered by the worker process is completed, the Task process will send the result of the task processing to the worker process through the $ws->finish() method, thereby completing the second business processed data and transmitting it to the corresponding terminal in a timely manner based on asynchronous event-driven and coroutine parallel network communication.

2. The order data processing method based on Kafka technology according to claim 1, characterized in that: The processing server consumes the logistics order information published to the first topic title further comprising: The processing server pre-sets and maintains offset information, where the offset information indicates location information of a partition where the current logistics order information consumed by the first topic title is located; When the processing server is in an idle state, it uses the offset information to access the logistics order information corresponding to the adapted partition, and puts the logistics order information into the preset database in sequence, and then updates the offset information.

3. The order data processing method based on Kafka technology as claimed in claim 2, characterized in that: Before the logistics order information is subjected to the second business processing, the following steps are also included: Read the logistics order information, and parse the order identification information from the logistics order information; The matching order logistics track information is obtained through the order identification, which at least includes the receipt information of the logistics order.

4. The order data processing method based on Kafka technology as claimed in claim 3 is characterized in that: Obtaining the matching order logistics track information through the order identifier further includes: Pre-create a logistics track server as a message producer through the Kafka cluster, and create a second topic title for storing logistics track records, and the processing server is a message subscriber of the second topic; The logistics track server captures the logistics track information that meets the preset requirements from the current logistics track data through the kafka message queue, and publishes the logistics track information to the second topic title; The processing server consumes the logistics track information published to the second topic title, and saves the logistics track information into the record corresponding to the order identifier in the preset database.

5. The order data processing method based on Kafka technology as claimed in claim 4, characterized in that: The second business processing of the logistics order information specifically includes: The processing server calculates the order dispatch billing data in the preset database to generate payment information according to the preset order dispatch billing model; A payment bill is generated based on the payment information so that the fee is paid to the terminal matched with the logistics order.

6. The order data processing method based on Kafka technology as claimed in claim 5, characterized in that: Calculating the dispatch billing data in the preset database further includes: The preset database is further configured to configure a first database, a second database and a dispatch and billing database; The processing server accesses the logistics order information corresponding to the adapted partition through the offset information, and puts the logistics order information into the first database; and the processing server consumes the logistics track information published to the second topic title, and saves the logistics track information into the second database; Regularly or in a triggered manner, updating the same order identification data of the first database and the second database to the dispatch and billing database; The express delivery receipt time under the order identification is obtained from the dispatch and billing database, and the delivery time of the express delivery order is calculated to calculate the delivery fee information of the order.

7. An order data processing system based on Kafka technology, characterized in that: include: Kafka cluster: used to create an order server as a message producer, and create a first topic title for storing direct interaction order records. The processing server that directly processes data interaction is the message subscriber of the first topic title; Order server: used to capture logistics order information with direct interaction identifier from current logistics order data through Kafka message queue, and publish the information to the first topic title; Processing server: used to consume the logistics order information published to the first topic title, perform second business processing on the logistics order information, and then find a terminal matching the logistics order to complete data transmission; Wherein, finding the terminal matching the logistics order to complete data transmission further includes: Find the terminal that matches the logistics order; The processing server is used to start the callback in the main thread, open the swoole_websocket_server service, and use swoole+Redis to push the data after the second business processing in real time, which is specifically: The manager process and the worker process are created in advance; Monitor timer; When the data processed by the second business is monitored to be calculated, the worker process uses the swoole_server_task function to deliver the task sent to the corresponding terminal; When the onTask callback function is called, the current Task process will switch the process state to busy, and will no longer receive new Tasks. When the onTask function returns, the process state will switch to idle and continue to receive new Tasks. When the task delivered by the worker process is completed, the Task process will send the result of the task processing to the worker process through the $ws->finish() method, thereby completing the second business processed data and transmitting it to the corresponding terminal in a timely manner based on asynchronous event-driven and coroutine parallel network communication.

8. An order data processing device based on Kafka technology, characterized in that: Kafka cluster interaction unit: used to create an order server as a message producer in advance through the Kafka cluster, and create a first topic title for storing direct interaction order records. The processing server that directly processes data interaction is the message subscriber of the first topic title; Publishing unit: used for capturing logistics order information with a direct interaction identifier from the current logistics order data through the Kafka message queue, and publishing the information to the first topic title; The second business processing unit is used to consume the logistics order information published to the first topic title, perform the second business processing on the logistics order information, and find the terminal matching the logistics order to complete the data transmission; Wherein, finding the terminal matching the logistics order to complete data transmission further includes: Find the terminal that matches the logistics order; The processing server is used to start the callback in the main thread, open the swoole_websocket_server service, and use swoole+Redis to push the data after the second business processing in real time, which is specifically: The manager process and the worker process are created in advance; Monitor timer; When the data processed by the second business is monitored to be calculated, the worker process uses the swoole_server_task function to deliver the task sent to the corresponding terminal; When the onTask callback function is called, the current Task process will switch the process state to busy, and will no longer receive new Tasks. When the onTask function returns, the process state will switch to idle and continue to receive new Tasks. When the task delivered by the worker process is completed, the Task process will send the result of the task processing to the worker process through the $ws->finish() method, thereby completing the second business processed data and transmitting it to the corresponding terminal in a timely manner based on asynchronous event-driven and coroutine parallel network communication.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the order data processing method based on Kafka technology as described in any one of claims 1 to 6 is implemented.

Images