Logistics transportation information processing method and device, query method, equipment and medium

Abstract

Embodiments of the present application provide a logistics transportation information processing method and device, a query method, equipment and a medium, belonging to the field of computer technology. The method comprises: obtaining to-be-processed logistics data in a logistics scene, the logistics scene comprising a plurality of waybill scenes; determining a preset key control association result table matched with the waybill scene corresponding to the to-be-processed logistics data, and performing key control state detection on the to-be-processed logistics data according to the preset key control association result table to obtain a logistics key control state; performing data update on the preset key control association result table according to the logistics key control state to obtain a target key control association result table; performing data extraction on the target key control association result table to obtain target association data corresponding to the to-be-processed logistics data; and sending the target association data to a distributed stream data platform for storage. The present application can improve the throughput of real-time association, reduce the delay performance, and thus improve the computing efficiency of the downstream scene.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a logistics transportation information processing method and apparatus, query method, equipment and medium. Background Technology

[0002] Logistics transportation information processing methods refer to the process of collecting, organizing, analyzing, and utilizing transportation-related information through various technologies and means during the logistics process. This information includes waybill data, routing data, and more. By correlating and processing transportation-related information, information support can be provided for downstream business processes such as information querying, scheduling decisions, and contract fulfillment. For example, when conducting real-time waybill monitoring in international express delivery, the system's real-time calculation engine is relied upon to correlate the international waybill flow (i.e., waybill data) with the routing trajectories belonging to that international waybill in the global routing flow (i.e., global routing data). This ensures that the waybill system operates stably under high load, providing stable real-time query capabilities for customer service, apps, and other applications.

[0003] Currently, related technologies rely on HBase as the data storage structure, storing both waybill data and routing events in HBase regardless of whether they arrive first. When the other related event arrives (e.g., the route arrives first, then the waybill), the association is completed by querying HBase to retrieve the stored information. However, because HBase stores a large amount of complete, undifferentiated routing trajectory data, this approach limits the throughput and low-latency performance of real-time association when processing transportation information, thus impacting the computational efficiency for downstream scenarios such as transportation efficiency, on-time performance, and anomaly warnings. Summary of the Invention

[0004] The main objective of this application is to propose a logistics transportation information processing method and apparatus, query method, equipment and medium, which can improve the throughput of real-time correlation during transportation information processing, reduce latency, and thus improve the computational efficiency for downstream scenarios.

[0005] To achieve the above objectives, a first aspect of this application proposes a logistics transportation information processing method, the method comprising: Acquire unprocessed logistics data in a logistics scenario, wherein the unprocessed logistics data is waybill data or routing data, and the logistics scenario includes multiple waybill scenarios; A preset keying association result table matching the waybill scenario corresponding to the logistics data to be processed is determined, and the keying status of the logistics data to be processed is detected according to the preset keying association result table to obtain the logistics keying status. The preset keying association result table is used to store the field data that associates waybill data and routing data in the waybill scenario. The preset keying association result table is updated according to the logistics keying status to obtain the target keying association result table. Data is extracted from the target keying association result table to obtain the target association data corresponding to the logistics data to be processed; The target associated data is sent to a distributed streaming data platform for storage.

[0006] To achieve the above objectives, a second aspect of this application proposes a method for querying logistics transportation information, the method comprising: Receive logistics query requests generated based on target waybill data; The distributed flow data platform described in the first aspect of the present application embodiment is invoked according to the logistics query request, and target associated data matching the target waybill data is extracted from the distributed flow data platform. Obtain the waybill routing information of the target waybill data based on the target associated data.

[0007] To achieve the above objectives, a third aspect of this application provides a logistics transportation information processing device, the device comprising: The acquisition module is used to acquire logistics data to be processed in the logistics scenario. The logistics data to be processed is waybill data or routing data. The logistics scenario includes multiple waybill scenarios. The status detection module is used to determine the preset keying association result table that matches the waybill scenario corresponding to the logistics data to be processed, and to perform keying status detection on the logistics data to be processed according to the preset keying association result table to obtain the logistics keying status. The preset keying association result table is used to store the field data that associates waybill data and routing data in the waybill scenario. The data update module is used to update the preset key association result table according to the logistics keying status to obtain the target keying association result table; The data extraction module is used to extract data from the target key association result table to obtain the target association data corresponding to the logistics data to be processed; The data sending module is used to send the target-related data to a distributed streaming data platform for storage.

[0008] To achieve the above objectives, a fourth aspect of the present application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described in any one of the embodiments of the first and second aspects described above.

[0009] To achieve the above objectives, a fifth aspect of the present application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in any one of the embodiments of the first and second aspects described above.

[0010] The logistics transportation information processing method, apparatus, query method, equipment, and medium proposed in this application, when processing logistics data, can first determine a preset keying association result table that matches the waybill scenario corresponding to the logistics data to be processed in the logistics scenario. Then, based on the preset keying association result table, the logistics data to be processed is keyed to obtain the logistics keying status. This preset keying association result table stores the field data that associates waybill data and routing data in the waybill scenario. Furthermore, this preset keying association result table records field data related to a specific waybill scenario, which can improve the data processing speed. Further, the preset keying association result table is updated according to the logistics keying status to obtain a target keying association result table, and the target association data corresponding to the logistics data to be processed is extracted from the target keying association result table. Then, the target association data is sent to a distributed streaming data platform for storage. This enables accurate dynamic updating and reliable storage of the association data between waybill data and routing data, providing complete and real-time business data support for downstream processes. Thus, the embodiments of this application can improve the throughput of real-time correlation during transportation information processing, reduce latency, and thereby improve the computational efficiency for downstream scenarios. Attached Figure Description

[0011] Figure 1 This is a flowchart of a logistics transportation information processing method provided in an embodiment of this application; Figure 2 yes Figure 1 A flowchart of step S120 in the process; Figure 3 yes Figure 2 A flowchart of step S210 in the process; Figure 4 yes Figure 2 A flowchart of step S220 in the process; Figure 5 yes Figure 1 A flowchart of step S130 in the process; Figure 6 yes Figure 5 A flowchart of step S510 in the process; Figure 7 This is a specific flowchart of the logistics transportation information processing method provided in the embodiments of this application; Figure 8 This is a flowchart of a logistics transportation information query method provided in an embodiment of this application; Figure 9 This is a schematic diagram of the structure of the logistics transportation information processing device provided in the embodiments of this application; Figure 10 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0013] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., used in the specification, claims, and the foregoing drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0014] First, let's analyze some of the terms used in this application: Flink is a powerful distributed stream processing framework. Its key features include the ability to process unbounded data streams and achieve high throughput, low latency, and exactly-once computation.

[0015] Redis is an open-source, in-memory key-value store used as a database, cache, and message queue. Redis supports various data structures, including strings, hashes, lists, sets, and sorted sets (zsets), making its applications far beyond simple caching. For example, it can be used to implement message queues, session storage, and leaderboards. Redis provides data persistence, allowing data to be saved from memory to disk via snapshots (RedisDataBase, RDB) and append-only file (AOF) mechanisms, enabling data recovery after a restart.

[0016] Memcached is a high-performance distributed memory object caching system primarily used in dynamic web applications to reduce database load. It improves website speed by caching data and objects in memory, reducing the number of database accesses. Memcached has a relatively simple design, supporting only basic key-value pair storage. Its internal memory management uses a SlabAllocation mechanism, which is efficient and avoids memory fragmentation, but is more memory-efficient for storing small amounts of data. Memcached does not support data persistence; data is lost upon restart, and it does not provide built-in distributed or data consistency mechanisms, which must be implemented at the application layer.

[0017] HBase is a distributed, column-oriented open-source database, a subproject of the Apache Hadoop project, designed to store and process massive amounts of structured data on inexpensive commercial server clusters.

[0018] Logistics transportation information processing methods refer to the process of collecting, organizing, analyzing, and utilizing transportation-related information through various technologies and means during the logistics process. This information includes waybill data, routing data, and so on. By correlating and processing transportation-related information, information support can be provided for downstream business processes such as information retrieval, scheduling decisions, and contract fulfillment.

[0019] In ensuring waybill fulfillment, end-to-end real-time monitoring is crucial, which involves real-time correlation processing of waybills and their routing trajectories. For example, in international express delivery, real-time waybill monitoring relies on the waybill system's real-time computing engine to correlate the international waybill stream (i.e., waybill data) with the routing trajectories belonging to that international waybill within the global routing stream (i.e., global routing data). This ensures the waybill system remains stable under high load, providing stable real-time query capabilities for customer service and apps. However, the order of waybill creation events and routing trajectory events is not guaranteed in the business logic (e.g., route scanning may arrive at the system before waybill information). Furthermore, message queue (such as Kafka) transmission mechanisms may introduce additional out-of-order issues.

[0020] Currently, to address the aforementioned out-of-order association problem, related technologies rely on HBase as the data storage structure. Regardless of whether waybill data or routing events arrive first, they are all stored in HBase. Thus, when the other event arrives (e.g., the route arrives first, then the waybill), the association is completed by querying HBase to retrieve the stored information. However, because HBase stores a massive amount of full, undifferentiated routing trajectory data, its required storage capacity is enormous, resulting in extremely high costs. Furthermore, this approach requires handling massive amounts of global routing data, and HBase requires frequent read / write operations (especially for writing large-scale routing data and random queries), becoming a severe performance bottleneck. This limits the throughput and low-latency performance of real-time association during transportation information processing, thus impacting the computational efficiency for downstream scenarios such as transportation efficiency, on-time performance, and anomaly warnings. Additionally, Based on this, embodiments of this application provide a logistics transportation information processing method and apparatus, query method, equipment and medium, which can improve the throughput of real-time correlation during transportation information processing, reduce latency, and thus improve the computational efficiency for downstream scenarios.

[0021] The logistics transportation information processing method and logistics transportation information query method provided in this application belong to the field of computer technology. The logistics transportation information processing method and logistics transportation information query method provided in this application can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet computer, laptop computer, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms; the software can be an application implementing the logistics transportation information processing method and the logistics transportation information query method, but is not limited to the above forms.

[0022] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, network personal computers (PCs), minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0023] Please see Figure 1 , Figure 1 This is a flowchart of a logistics transportation information processing method provided in an embodiment of this application. In some embodiments of this application, Figure 1 The method described below may include, but is not limited to, steps S110 to S150. Figure 1 These five steps will be explained in detail.

[0024] Step S110: Obtain the logistics data to be processed in the logistics scenario. The logistics data to be processed is waybill data or routing data. Step S120: Determine the preset keying association result table that matches the waybill scenario corresponding to the logistics data to be processed, and perform keying status detection on the logistics data to be processed according to the preset keying association result table to obtain the logistics keying status. Step S130: Update the preset keying association result table according to the logistics keying status to obtain the target keying association result table; Step S140: Extract data from the target key association result table to obtain the target association data corresponding to the logistics data to be processed; Step S150: Send the target associated data to the distributed streaming data platform for storage.

[0025] In step S110 of some embodiments, the logistics scenario may include multiple core product lines, such as domestic standard express network, international express, cold chain transportation, and same-city express delivery. Each product line is equivalent to a waybill scenario, meaning the logistics scenario can include multiple waybill scenarios. This application embodiment can process the real-time acquired waybill data or logistics data. Specifically, it can acquire the logistics data to be processed through data acquisition technologies, such as logistics information system interface integration, warehouse terminal uploading, and GPS data reception from transport vehicles. This logistics data to be processed can be waybill data recording order information, consignment details, etc., and can be stored independently in their respective real-time data streams according to product lines (such as in a distributed streaming data platform like Kafka), without overlap. Alternatively, the logistics data to be processed can record routing data such as transportation routes and node transfers. Routing data can also be represented as routing trajectory data. The routing trajectory data of all product lines can be aggregated in a single, undifferentiated global real-time data stream (such as Kafka), thereby achieving comprehensive coverage of core business data under different waybill scenarios.

[0026] Understandably, within the same Flink task, this embodiment of the application can process data from waybill data streams and routing data streams in parallel to update the pre-set association result table in real time, thereby sending more accurate association data to downstream Kafka.

[0027] In step S120 of some embodiments, this application embodiment can use Apache Flink's keyed state to store waybill data and routing data in real time. That is, the received waybill data and routing data can be persisted to the Flink keyed state, and this state can be represented as a keyed association result table. Here, Flink keyed state refers to a mechanism that stores data in memory and can be persisted to disk, mainly used to store intermediate results, and can be replaced by Redis, Memcached, etc. The information stored in the Flink keyed state is the relevant fields in the "keyed association result table," with one data entry per waybill number.

[0028] In other words, this application embodiment can consume international business data streams (such as waybill data streams or routing data streams) in real time through Flink to perform full processing of waybill records. The preset keyed association result table refers to a table constructed based on the characteristics of the waybill scenario to which the logistics data to be processed belongs (such as the association data between international waybills and customs clearance routes stored for international express delivery), selecting association fields matching the scenario from a pre-built set of keyed result tables. The preset keyed association result table uses key fields (such as waybill number and routing node code) as its core, storing the association rules and historical association information between waybill data and routing data.

[0029] For example, the fields stored in the current Flink keying state are the related fields stored in the "Keying Association Result Table". Table 1 below shows the keying result table that stores key field information related to waybills (field data generated based on waybill data), and Table 2 shows the keying result table that stores key field information related to routing trajectories (field data generated based on routing data).

[0030] Table 1

[0031] Table 2

[0032] Furthermore, embodiments of this application can configure different keyed association result tables based on different waybill scenarios to store the field data that associates waybill data and routing data in the waybill scenario. For example, when the waybill scenario corresponding to the logistics data to be processed is an international express delivery business scenario, the corresponding preset keyed association result table can be seen in Table 3 below, which stores the key field information that associates waybill data and routing data.

[0033] Table 3

[0034] It should be noted that the preset keying association result table constructed in this application embodiment does not contain specific routing trajectory data (i.e., routing data), but converts the routing trajectory data into corresponding fields. For example, operation 80 in the routing trajectory will be converted into delivery time, and operation 50 will be converted into receipt time. In this way, the data stored in the Flink keying state will be much less than the original routing data, so as to improve the processing efficiency of logistics data.

[0035] It should be noted that Tables 1 to 3 above are only examples of table fields, and can be flexibly added or deleted according to actual applications.

[0036] Please refer to Figure 2 , Figure 2 This is a flowchart of step S120 provided in an embodiment of this application. In some embodiments, the logistics keying status includes the waybill keying status corresponding to the target waybill data to be processed and the routing keying status corresponding to the target route data to be processed. Then, step S120 may specifically include, but is not limited to, the following steps: Step S210: If the logistics data to be processed is the target waybill data, perform keying status detection on the target waybill data according to the preset keying association result table to obtain the waybill keying status; Step S220: If the logistics data to be processed is target routing data, perform keying status detection on the target routing data according to the preset keying association result table to obtain the routing keying status.

[0037] In step S210 of some embodiments, if the logistics data to be processed is target waybill data, the subsequent process is a sub-process for real-time processing of waybill flows. When the logistics data to be processed is identified as target waybill data (i.e., one or more waybill records in the form of documents that carry core order information in logistics business, such as express pickup slips, less-than-truckload (LTL) consignment slips, etc., containing key information such as waybill number, consignment details, and shipping / receiving address), the keying status detection process is initiated. Specifically, the core association fields that the target waybill data needs to match (such as waybill number, origin post office, consignment risk level code, etc.) are first extracted from a pre-built preset keying association result table. Then, the keying status of the target waybill data is detected based on the extracted field data. The obtained waybill keying status can reflect whether the target waybill data has a valid waybill flow, or whether the waybill status of the target waybill data has expired due to the corresponding time to live (TTL, which is the data packet lifespan defined in the IP protocol), thereby determining whether accurate association data can be extracted in the subsequent processing.

[0038] Please refer to Figure 3 , Figure 3 This is a flowchart of step S210 provided in an embodiment of this application. In some embodiments, the specific process of step S210 may include, but is not limited to, steps S310 to S340, as described below. Figure 3 These four steps will be explained in detail.

[0039] Step S310: If the waybill indicated by the target waybill data is a query waybill, extract the data lifetime corresponding to the target waybill data from the preset key association result table. Step S320: Perform time validity verification on the data lifetime to obtain the time verification result; Step S330: Perform data security verification on the target waybill data and obtain the data verification result; Step S340: Determine the waybill key control status based on the time verification result and the data verification result.

[0040] In step S310 of some embodiments, the target waybill data can be newly added waybill data or specific waybill information for which a query operation is initiated (such as waybill queries for customers to check logistics progress, waybill queries for customer service to check abnormal orders, etc., including core content such as waybill number and consignment details). The preset keyed association result table can use the waybill number as the core index to store the association fields between the waybill and the routing data, as well as the data lifespan (i.e., the time range within which the data is considered valid in the system, generally determined by "creation time + validity period," for example, based on the core characteristic of waybill lifespan ≤ 3 months in logistics business, a precise TTL parameter can be configured). When it is determined that the target waybill data belongs to the "query waybill" type, the field value of the data lifespan corresponding to the waybill can be accurately found and obtained from the preset keyed association result table through extraction operations (such as hash index queries based on waybill numbers, SQL statement filtering, and other database interaction methods), providing basic information for subsequent time validity verification.

[0041] In step S320 of some embodiments, data lifetime is a key indicator for measuring the timeliness of waybill-related data. Time validity verification can be used to determine whether the data lifetime corresponding to the target waybill data is within a valid range. If the currently determined data lifetime is within the lifetime range (or has not exceeded the deadline), the time verification result is determined to be "time valid"; if the current time exceeds the lifetime range, the time verification result is marked as "expired". This process can rely on a timestamp standardization protocol (such as the NTP network time protocol for time synchronization) to ensure time consistency, and utilize a configurable timeliness rule engine (which can set different validity thresholds for different waybill types) to achieve automatic verification, outputting verification results that reflect data timeliness, and providing a time-related decision basis for the final determination of the waybill key control status.

[0042] In step S330 of some embodiments, data security verification can focus on the confidentiality, integrity, and access legitimacy of the target waybill data during transmission and storage. Regarding confidentiality, embodiments of this application can use asymmetric encryption algorithms (such as RSA) to verify the digital signature of the waybill data (in encrypted transmission scenarios, after decryption, compare the signature hash value with the source data to ensure the data has not been maliciously tampered with). Regarding integrity, embodiments of this application can use a hash function (such as SHA-256) to calculate the digest value of the waybill field data corresponding to the target waybill data, and compare it with the digest baseline value stored in the preset keying association result table; if they match, it is determined that the data has not been illegally modified. Regarding access legitimacy, embodiments of this application can call the access control system API to check whether the entity initiating the query operation (such as a customer service account or customer ID) has query permissions for the waybill (e.g., only the shipper, consignee, or authorized customer service can query); if the permissions match, the access is determined to be legitimate. By integrating the results of these encryption verifications, hash comparisons, and permission checks, and outputting data verification results (such as security compliance, risk status indicators, etc.), a security-level decision-making basis is provided for the classification of waybill key control status.

[0043] It should be noted that in various specific embodiments of this application, when processing data related to the object's identity or characteristics, such as object permission information or object account information, is required, the object's permission or consent will be obtained first. Furthermore, the collection, use, and processing of this data will comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require obtaining sensitive personal information of an object, separate permission or consent from the object will be obtained through pop-ups or redirection to a confirmation page. Only after obtaining the object's separate permission or consent will the necessary object-related data for the normal operation of the embodiments of this application be obtained.

[0044] In step S340 of some embodiments, the embodiments of this application can determine the waybill key control status based on the time verification result and the data verification result, using logical judgment rules such as a configurable state machine model. For example, if the time verification is "valid" and the data verification is "secure and compliant", then the corresponding waybill key control status can be determined as "association valid" (corresponding to the subsequent valid status), meaning that the associated data of this queried waybill meets the business requirements in terms of timeliness and security; if the time verification is "expired and invalid" but the data verification is "secure and compliant", then a data update process can be triggered, such as marking the current target waybill data as "pending association update", and then calling the routing application programming interface (API) to complete the latest routing information and re-verify. At this time, the waybill key control status is temporarily set to "association pending update" (corresponding to the subsequent invalid status); if the data verification is "tampering risk" or "insufficient permissions", then the waybill key control status can be determined as "association failure" (corresponding to the subsequent invalid status), which is used to intercept illegal access or prompt data abnormality. In this way, the judgment process can be embedded into the state management logic of the Flink stream computing framework, and combined with the characteristics of real-time computing, it can quickly output state results and provide state-driven signals for subsequent business processes such as data updates and route completion.

[0045] In step S220 of some embodiments, within the same Flink task, this application embodiment can process the routing data stream from the Kafka Topic in parallel, that is, perform keying state detection and field data update on each routing record in the routing data stream. Target routing data refers to the logistics data to be processed. If the logistics data to be processed is target routing data, the subsequent process is a sub-process for real-time processing of the routing stream. When the logistics data to be processed is identified as target routing data (i.e., trajectory data in the logistics scenario that records transportation routes, node flow, and carrier information, such as trunk transportation route sheets, distribution center transit route records, last-mile delivery route trajectories, etc., including key information such as route number, associated waybill number, start and end node codes, and transportation time), the keying state detection process is initiated. Specifically, the associated fields that need to be matched for the route (such as route number, associated waybill number, origin and destination node area codes, carrier authorization codes, etc.) are first extracted from the preset keying association result table. Then, the keying status of the target route data is checked based on the extracted field data. The obtained route keying status can reflect whether there is a matching valid waybill for the target route data. A valid waybill refers to a waybill that has not expired or become invalid, and the waybill data is legal, providing a status-driven basis for business processes such as route information completion and waybill trajectory reconstruction.

[0046] It should be noted that a Flink task can be understood as a process that is constantly running on the server, responsible for consuming data in Kafka to perform real-time calculations. In this embodiment, this task is to continuously consume waybill and route trajectory data in real time, generate the final result data, and write it to Kafka.

[0047] In the above embodiments, this application embodiment can design Flink keying status detection logic based on a preset keying association result table for target waybill data and target route data respectively. On the one hand, by leveraging field-level precise verification and Flink state caching reuse mechanism, the proportion of invalid calculations is significantly reduced (such as avoiding repeated verification of already confirmed associated data), significantly improving the detection efficiency and accuracy of waybill-route association relationships. On the other hand, the clear classification and output of waybill keying status and route keying status provide differentiated execution guidance for subsequent data updates, route completion, and associated data extraction, ensuring the full-link automation and intelligence of data association management in multiple logistics scenarios and avoiding out-of-order events.

[0048] It should be noted that when the waybill data indicating a new waybill is determined based on the waybill keying status, the target waybill data can be persisted to Flink Keyed State, and a TTL countdown can be started to generate the data lifespan corresponding to the target waybill data. Simultaneously, the routing query API is called to obtain the complete routing trajectory data for the target waybill data. After completing the waybill-route association, the association result data is written to the downstream Kafka and simultaneously stored in the Flink state (waybill keying state), with a TTL countdown started to manage state timeliness and automatic cleanup upon timeout.

[0049] Please refer to Figure 4 , Figure 4 This is a flowchart of step S220 provided in an embodiment of this application. In some embodiments, the specific process of step S220 may include, but is not limited to, steps S410 to S420, as described below. Figure 4 These two steps will be explained in detail.

[0050] Step S410: Extract the target waybill number from the target routing data; Step S420: Based on the target waybill number and the preset keying association result table, perform keying status detection to obtain the routing keying status.

[0051] In steps S410 and S420 of some embodiments, when performing keying status detection on the target routing data, the target waybill number can be accurately obtained from the target routing data first. This target waybill number is a unique identifier code of the waybill that is associated with the current target routing data, and it is a core field that links the core information of the waybill with the routing trajectory data. In this way, it can ensure that the waybill number is efficiently and accurately extracted from the complex routing data, providing an accurate index for subsequent keying status detection based on the waybill number, and avoiding misjudgment of the associated status due to incorrect extraction of the waybill number. Furthermore, in this embodiment, keying status detection can be performed based on the target waybill number and a preset keying association result table. The preset table is traversed using the target waybill number as the query condition to locate all route association records corresponding to the waybill number. Then, the core fields in the association records (such as the consistency between the route number and the "waybillNo" field in the target route data, whether the route node code conforms to the preset flow rules of the current waybill scenario, and whether the association effective time is within the valid period) are verified and compared. If the fields are completely matched and the preset integrity rules are met (such as no empty values ​​in the required association fields and the encoding format conforming to industry / enterprise standards), the route keying status is determined to be "valid status", indicating that the association between the current route data and the target waybill meets the business validity requirements. If there are field mismatches (such as no corresponding relationship between the route number and the waybill number, or the node code violating the scenario flow rules) or key fields are missing (such as the route effective start time not being recorded), the status is marked as "invalid status" according to the exception type.

[0052] This application embodiment constructs an association index by selectively extracting the target waybill number from the target routing data, and then performs multi-dimensional keying status verification on the routing association relationship corresponding to the waybill number based on a preset keying association result table. On the one hand, by leveraging structured data parsing technology and index query mechanism, the efficiency of waybill number extraction and routing association detection is greatly improved (such as avoiding invalid verification of routing data without associated waybill numbers), reducing redundant calculations for cross-data domain association. On the other hand, the accurate index and rule-based status detection centered on the waybill number effectively ensure the accuracy and consistency of the association relationship between routing data and waybill data, providing reliable status identifiers for subsequent data updates, association result distribution, and other processes.

[0053] In step S130 of some embodiments, the embodiments of this application may perform iterative updates on the preset keying association result table according to the output logistics keying status to generate a target keying association result table, so as to extract more accurate association data between waybills and routes in the future.

[0054] Please refer to Figure 5 , Figure 5This is a flowchart of step S130 provided in an embodiment of this application. In some embodiments, the specific process of step S130 may include, but is not limited to, steps S510 to S520, as described below. Figure 5 These two steps will be explained in detail.

[0055] Step S510: Update the preset key association result table according to the key status of the waybill to obtain the candidate key association result table; Step S520: Update the candidate keying association result table according to the routing keying status to obtain the target keying association result table.

[0056] In steps S510 and S520 of some embodiments, the present application embodiments can update the preset keying association result table based on the waybill keying status and the route keying status respectively, so as to obtain the target keying association result table that meets the real-time data requirements.

[0057] Please refer to Figure 6 , Figure 6 This is a flowchart of step S510 provided in an embodiment of this application. In some embodiments, the specific process of step S510 may include, but is not limited to, steps S610 to S620, as described below. Figure 6 These two steps will be explained in detail.

[0058] Step S610: If the waybill keying status is valid, update the waybill-related fields in the preset keying association result table according to the target waybill data to obtain the candidate keying association result table. Step S620: If the waybill keying status is invalid, call the routing application interface to obtain the routing trajectory data corresponding to the target waybill data; perform data association between the target waybill data and the routing trajectory data to obtain waybill routing association field data; update the preset keying association result table according to the waybill routing association field data to obtain the candidate keying association result table.

[0059] In step S610 of some embodiments, if the waybill keying status is valid, and the specific method for determining the valid status has been detailed in the above embodiments (i.e., there is a matching valid waybill in the preset keying association result table, and the waybill has not expired), then the waybill-related fields in the preset keying association result table can be updated based on the real-time data in the target waybill data to obtain a candidate keying association result table. This process can rely on transaction management mechanisms (such as database transactions) to ensure the atomicity and consistency of field updates, avoiding data inconsistency problems caused by the failure of updating some fields.

[0060] In step S620 of some embodiments, if the waybill keying status is invalid, i.e., there is no matching valid waybill in the preset keying association result table or the waybill has expired, the routing application programming interface (API) can be called to obtain all routing trajectory data related to the target waybill data. Then, the target waybill data and the routing trajectory data are associated to obtain the waybill routing association field data, and the preset keying association result table is updated according to the waybill routing association field data to obtain the candidate keying association result table. That is, the waybill routing association field data is written to the Flink status (i.e., waybill keying status) corresponding to the target waybill data. This process relies on data association algorithms (such as nearest neighbor association, global nearest neighbor association) to ensure accurate matching between waybill and routing data, and at the same time, the reliability of obtaining routing trajectory data is ensured through the error handling mechanism of API calls (such as retry mechanism, exception capture).

[0061] It should be noted that, for business characteristics where the number of routing entries per waybill is ≤200, the routing system provides a query API with millisecond-level response through distributed caching and index optimization, ensuring that a single request can return all routing records for the waybill.

[0062] In the above embodiments, during the dual-stream processing of waybill stream and routing stream, this application can achieve cross-stream state sharing through FlinkKeyed State. The waybill stream "actively pulls routes" and the routing stream "reversely queries waybills", forming a closed-loop association. Combined with the TTL mechanism, the timeliness of the state is guaranteed, and invalid states are avoided from occupying resources for a long time.

[0063] It should be noted that, in addition to using Flink states to represent waybill keying status and route keying status, Redis, Memcached, etc. can also be used, without limitation.

[0064] In step S140 of some embodiments, extraction conditions can be set based on the identifiers of the logistics data to be processed (such as waybill number, route batch number). From the structured storage area of ​​the updated target keyed association result table, the set of fields and records that are related to the data to be processed can be accurately retrieved and extracted to form target association data. This target association data can include both direct association information between waybills and routes, and association rules derived after status updates (such as the binding relationship between newly added route nodes and waybills).

[0065] In step S150 of some embodiments, the extracted target-related data can be encapsulated into streaming messages and sent to a designated topic on the platform for storage via the client API of a distributed streaming data platform (taking Apache Kafka as an example, which has high throughput and scalability). The distributed architecture ensures reliable persistence of data in high-concurrency and real-time scenarios, and also provides efficient data consumption capabilities for subsequent business modules such as logistics tracking and resource scheduling.

[0066] For example, such as Figure 7 The diagram shown is a specific flowchart of a logistics transportation information processing method provided in an embodiment of this application. In a specific embodiment, this logistics transportation information processing method may include the following steps: Step S701: Obtain logistics data to be processed, which may be waybill data or routing data.

[0067] Step S702: Determine whether there is associated result data corresponding to the logistics data to be processed based on the Flink status stored in the preset keying association result table. If it exists, execute step S703 for waybill data and step S706 for routing data. If it does not exist, execute step S704 for waybill data and step S707 for routing data.

[0068] Step S703: Update the waybill-related fields in the status association result data.

[0069] If there is associated result data corresponding to the logistics data to be processed, the waybill-related fields in the preset keyed association result table can be updated.

[0070] Step S704: Call the routing API interface to obtain all routes for this waybill.

[0071] Among them, all routes of a waybill refer to all routing trajectory data related to the target waybill data.

[0072] Step S705: Associate the waybill and routing data, and write the association result data to the Flink state. Then, proceed to step S708.

[0073] Writing the associated result data to the Flink state means writing the obtained waybill routing association field data to the preset keyed association result table to update the Flink state corresponding to the target waybill data.

[0074] Step S706: Update the routing-related fields in the state association result data. Then, proceed to step S708.

[0075] If there is associated result data corresponding to the logistics data to be processed, the routing-related fields in the preset keyed association result table can be updated, that is, the waybill and route are associated in real time.

[0076] Step S707: Discard the routing data.

[0077] If there is no associated result data corresponding to the logistics data to be processed, the routing record is actively discarded to avoid invalid calculations.

[0078] Step S708: Send the new association result data to Kafka.

[0079] In this embodiment of the application, the updated target association data can be sent to a distributed streaming data platform for storage.

[0080] The logistics transportation information processing method provided in this application is essentially a waybill lifecycle processing method based on stream computing status. The waybill data can be configured with precise Time-to-Live (TTL) parameters according to the waybill lifecycle in logistics operations, enabling automatic cleanup of status data. For routing data, an on-demand query interface can be used to build a low-latency, high-concurrency routing query service. The entire link routing trajectory can be obtained in batches by waybill number, solving the problem of performance bottlenecks caused by the lack of product line partitioning for global routing data. In practical applications, a significant reduction in storage costs can be observed. When performing real-time monitoring of downstream product lines, the method provided in this application eliminates the dependence on HBase global routing storage, requiring only the maintenance of waybill status, reducing storage resource consumption by over 90%. Furthermore, routing filtering is implemented through Flink memory-level state, replacing HBase. This reduces association latency from hundreds of milliseconds to sub-milliseconds, increases Flink task processing throughput by 8-10 times, and reduces CPU / memory resource usage by approximately 60%, effectively improving computing performance. Therefore, this application can improve the throughput of real-time correlation during transportation information processing, reduce latency, and thus improve the computational efficiency for downstream scenarios.

[0081] Please see Figure 8 , Figure 8 This is a flowchart of a logistics transportation information query method provided in an embodiment of this application. This logistics transportation information query method is a flowchart for a specific application scenario corresponding to the aforementioned logistics transportation information processing method. In some embodiments of this application, Figure 8 The method described below may include, but is not limited to, steps S810 to S830. Figure 8 These three steps will be explained in detail.

[0082] Step S810: Receive a logistics query request generated based on the target waybill data; Step S820: Call the distributed flow data platform according to the logistics query request, and extract the target associated data that matches the target waybill data from the distributed flow data platform; Step S830: Obtain the waybill routing information of the target waybill data based on the target associated data.

[0083] In the context of logistics information service interaction, logistics query requests are initiated by external systems (such as e-commerce platform order pages, logistics company customer service work order systems, and consumer-end apps) and use target waybill data (document records carrying core order information, including waybill numbers, consignment types, and other key identifiers) as query conditions. Upon receiving a logistics query request, the system can invoke the distributed streaming data platform based on the query logic identifier carried in the request (e.g., specifying to pull related data from a distributed streaming data platform). Data extraction conditions (such as key filtering rules for Kafka messages or WHERE clauses in stream processing SQL) are constructed based on the unique identifier of the target waybill data (e.g., waybill number) to extract the target related data stored on the platform. Furthermore, this embodiment can parse and map the structured fields following the target related data to extract waybill routing information. Finally, the parsed waybill routing information is encapsulated into a format that can be displayed on the front end or directly consumed by downstream businesses (e.g., HTML page data, JSON response body) for use in business scenarios such as logistics query result display and route visualization.

[0084] In other words, this application embodiment can use Flink tasks to write data from Kafka to Hive and Doris in real time, and then generate a report for business colleagues to use.

[0085] It should be noted that the software tools or components not belonging to our company that appear in the embodiments of this application are merely examples and do not represent actual use.

[0086] Please see Figure 9 This application also provides a logistics transportation information processing device, the logistics transportation information processing device 900 including: The acquisition module 910 is used to acquire logistics data to be processed in the logistics scenario. The logistics data to be processed is waybill data or logistics data. The logistics scenario includes multiple waybill scenarios. The status detection module 920 is used to determine the preset keying association result table that matches the waybill scenario corresponding to the logistics data to be processed, and to perform keying status detection on the logistics data to be processed according to the preset keying association result table to obtain the logistics keying status. The preset keying association result table is used to store the field data that associates waybill data and routing data in the waybill scenario. The data update module 930 is used to update the preset key association result table according to the logistics keying status to obtain the target keying association result table. The data extraction module 940 is used to extract data from the target key association result table to obtain the target association data corresponding to the logistics data to be processed; The data sending module 950 is used to send the target-related data to the distributed streaming data platform for storage.

[0087] It should be noted that the logistics transportation information processing device provided in this application embodiment is used to implement the logistics transportation information processing method provided in the above embodiment, and the specific implementation process corresponds to the logistics transportation information processing method in the above embodiment. It can be referred to the aforementioned logistics transportation information processing method, and will not be repeated here.

[0088] This application also provides an electronic device (i.e., a computer device), which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement any of the logistics transportation information processing methods described in the above embodiments. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0089] Please see Figure 10 , Figure 10 This illustration shows the hardware structure of an electronic device according to another embodiment, the electronic device comprising: The processor 1010 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 1020 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010 to execute the logistics transportation information processing method and logistics transportation information query method of the embodiments of this application. The input / output interface 1030 is used to implement information input and output; The communication interface 1040 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 1050 transmits information between various components of the device (e.g., processor 1010, memory 1020, input / output interface 1030, and communication interface 1040); The processor 1010, memory 1020, input / output interface 1030 and communication interface 1040 are connected to each other within the device via bus 1050.

[0090] This application also provides a computer-readable storage medium storing a computer program for causing a computer to execute the logistics transportation information processing method and the logistics transportation information query method described in the above embodiments.

[0091] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0092] This invention also provides a computer program product that stores program instructions. When executed by a computer, the program instructions cause the computer to implement the logistics transportation information processing method and the logistics transportation information query method described in any of the above embodiments.

[0093] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0094] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0095] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0096] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0097] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0098] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0099] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0100] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0101] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0102] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0103] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A logistics transportation information processing method, characterized in that, The method includes: Acquire unprocessed logistics data in a logistics scenario, wherein the unprocessed logistics data is waybill data or routing data, and the logistics scenario includes multiple waybill scenarios; A preset keying association result table matching the waybill scenario corresponding to the logistics data to be processed is determined, and the keying status of the logistics data to be processed is detected according to the preset keying association result table to obtain the logistics keying status. The preset keying association result table is used to store the field data that associates waybill data and routing data in the waybill scenario. The preset keying association result table is updated according to the logistics keying status to obtain the target keying association result table. Data is extracted from the target keying association result table to obtain the target association data corresponding to the logistics data to be processed; The target associated data is sent to a distributed streaming data platform for storage.

2. The method according to claim 1, characterized in that, The logistics keying status includes the waybill keying status corresponding to the target waybill data to be processed and the route keying status corresponding to the target route data to be processed. The step of performing keying status detection on the logistics data to be processed according to the preset keying association result table to obtain the logistics keying status includes: If the logistics data to be processed is target waybill data, the keying status of the target waybill data is detected according to the preset keying association result table to obtain the keying status of the waybill; If the logistics data to be processed is target routing data, the target routing data is keyed according to the preset keying association result table to obtain the routing keying status.

3. The method according to claim 2, characterized in that, The step of updating the preset keying association result table according to the logistics keying status to obtain the target keying association result table includes: The preset keying association result table is updated according to the keying status of the waybill to obtain a candidate keying association result table. The candidate keying association result table is updated based on the routing keying status to obtain the target keying association result table.

4. The method according to claim 3, characterized in that, The waybill key control status is either valid or invalid. The valid status indicates that there is associated data corresponding to the waybill in the preset key control association result table, and the invalid status indicates that there is no associated data corresponding to the waybill in the preset key control association result table. The step of updating the preset keying association result table based on the keying status of the waybill to obtain a candidate keying association result table includes: If the waybill keying status is valid, the waybill-related fields in the preset keying association result table are updated according to the target waybill data to obtain the candidate keying association result table. If the waybill keying status is invalid, the routing application interface is called to obtain the routing trajectory data corresponding to the target waybill data; the target waybill data and the routing trajectory data are associated to obtain waybill routing association field data; the preset keying association result table is updated according to the waybill routing association field data to obtain the candidate keying association result table.

5. The method according to claim 2, characterized in that, The step of performing keying status detection on the target routing data according to the preset keying association result table to obtain the routing keying status includes: Extract the target waybill number from the target routing data; Based on the target waybill number and the preset keying association result table, the keying status is detected to obtain the routing keying status.

6. The method according to claim 2, characterized in that, The step of performing keying status detection on the target waybill data according to the preset keying association result table to obtain the keying status of the waybill includes: If the waybill indicated by the target waybill data is a query waybill, extract the data lifetime corresponding to the target waybill data from the preset key association result table; The data lifetime is validated to obtain the time validation result. The target waybill data is subjected to data security verification to obtain the data verification result; The waybill key control status is determined based on the time verification result and the data verification result.

7. A method for querying logistics transportation information, characterized in that, The method includes: Receive logistics query requests generated based on target waybill data; The system invokes the distributed flow data platform according to any one of claims 1 to 6 based on the logistics query request, and extracts target associated data that matches the target waybill data from the distributed flow data platform. Obtain the waybill routing information of the target waybill data based on the target associated data.

8. A logistics transportation information processing device, characterized in that, The device includes: The acquisition module is used to acquire logistics data to be processed in the logistics scenario. The logistics data to be processed is waybill data or routing data. The logistics scenario includes multiple waybill scenarios. The status detection module is used to determine the preset keying association result table that matches the waybill scenario corresponding to the logistics data to be processed, and to perform keying status detection on the logistics data to be processed according to the preset keying association result table to obtain the logistics keying status. The preset keying association result table is used to store the field data that associates waybill data and routing data in the waybill scenario. The data update module is used to update the preset key association result table according to the logistics keying status to obtain the target keying association result table; The data extraction module is used to extract data from the target key association result table to obtain the target association data corresponding to the logistics data to be processed; The data sending module is used to send the target-related data to a distributed streaming data platform for storage.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images