Goods inspection and goods tracking method and device and goods inspection and goods system

Abstract

The invention relates to the technical field of goods inspection and goods tracking, and provides a goods inspection and goods tracking method, a goods inspection and goods tracking device and a goods inspection and goods system. The visual sensors are arranged at a plurality of preset positions of target inspection goods in the conveying and running process, and the target inspection goods are provided with unique identification codes; and based on the unique identification code image corresponding to the unique identification code displayed in the visual image and the acquisition position of the visual image, the target goods to be inspected are positioned and tracked in real time, so that the goods to be inspected can be accurately positioned in real time, and the over-inspection efficiency of the goods to be inspected is improved.

Description

Technical Field

[0001] This invention relates to the field of cargo inspection and tracking technology, and in particular to a cargo inspection and tracking method, device and system. Background Technology

[0002] In the field of air cargo security inspection, the accuracy of cargo sorting systems directly affects inspection efficiency and logistics timeliness. In existing technologies, cargo location determination mainly relies on the linear conveyor belt sequence and manually preset spacing control. Specifically, cargo must enter the security inspection channel sequentially at fixed intervals. The sorting system records the initial position of the cargo using sensor arrays or visual recognition devices and maintains this sequence for automated sorting in subsequent processes.

[0003] However, the aforementioned technical solutions have significant drawbacks: when the machine needs to be stopped for recoding during security checks, the recoding operation interrupts the conveyor belt, causing subsequent goods to pile up or shift, disrupting the original order; when goods enter or exit the security scanner, the physical obstruction of the lead curtain creates blind spots, and existing technology cannot correct the deviation between the actual position of the goods and the position recorded by the system in real time. Both of these situations cause a disconnect between the location information obtained by the sorting system and the actual state of the goods, leading to sorting errors or system downtime. Especially in the high-time-sensitivity scenario of air cargo inspection, these problems reduce overall inspection efficiency.

[0004] Therefore, finding a cargo tracking method that can locate and inspect cargo in real time and accurately has become a current research hotspot. Summary of the Invention

[0005] This invention provides a cargo inspection tracking method, device, and cargo inspection system, which enables real-time and accurate location of cargo for inspection, thereby improving the inspection efficiency of cargo.

[0006] This invention provides a method for tracking inspected goods, the method comprising: real-time acquisition of visual images of target inspected goods during transport using a visual sensor, wherein the visual sensor is disposed at multiple preset positions of the target inspected goods during transport, and the target inspected goods are assigned a unique identifier; and real-time location tracking of the target inspected goods based on the unique identifier image displayed in the visual image corresponding to the unique identifier, and the acquisition position of the visual image.

[0007] According to a cargo inspection and tracking method provided by the present invention, after the visual images of the target cargo inspection and tracking are acquired in real time based on a visual sensor during the transport process, the method further includes: when the unique identifier image corresponding to the unique identifier displayed in the visual image is occluded, extracting visual image features of the visual images at different times based on the visual images at different times; acquiring target cargo image features of the target cargo inspection and tracking; and when the visual image features match the target cargo image features of the target cargo inspection and tracking, locating and tracking the target cargo inspection and tracking in real time based on the acquisition position of the visual images.

[0008] According to a cargo inspection and tracking method provided by the present invention, the visual image features of the visual image are obtained by: calling a pre-trained feature extraction model, wherein the feature extraction model is used to obtain visual image features corresponding to the visual image based on the visual image; inputting the visual image into the feature extraction model to obtain the visual image features output by the feature extraction model corresponding to the visual image.

[0009] According to a cargo inspection and tracking method provided by the present invention, the method further includes: calling a pre-trained cargo detection model, wherein the cargo detection model is used to identify cargo inspection items in a visual image based on a visual image; inputting the visual image into the cargo detection model to obtain cargo inspection items corresponding to the visual image output by the cargo detection model; and counting the cargo inspection items monitored by the visual image based on the quantity of the cargo inspection items.

[0010] According to a cargo inspection tracking method provided by the present invention, after obtaining the cargo inspection goods corresponding to the visual image output by the cargo detection model, the method further includes: extracting cargo inspection goods features in the visual image at different times based on the visual image at different times; clustering the cargo inspection goods in the visual image at each time based on the cargo inspection goods features to obtain different types of cargo inspection goods and the quantity of cargo inspection goods of different types; and monitoring the count changes of different types of cargo inspection goods based on the quantity of cargo inspection goods of different types at different times.

[0011] According to a cargo inspection tracking method provided by the present invention, after the real-time positioning and tracking of the target cargo inspection goods, the method further includes: when the target cargo inspection goods arrive at the delivery destination, constructing a three-dimensional position model of a cargo pallet containing the target cargo inspection goods; based on the three-dimensional position model, determining a twin grasping position sequence of the target cargo inspection goods relative to the three-dimensional position model according to the minimum grasping distance; based on the twin grasping position sequence, determining an actual grasping position sequence of the target cargo inspection goods; and based on the actual grasping position sequence, grasping the target cargo inspection goods on the cargo pallet.

[0012] According to a cargo inspection tracking method provided by the present invention, the visual sensor is set at multiple preset positions of the target cargo during the conveying operation, including at least the following positions: the entrance position of the security inspection machine during the conveying operation, the exit position of the security inspection machine during the conveying operation, the straight section of the conveyor belt during the conveying operation, the bend of the conveyor belt during the conveying operation, and the cargo inspection sorting and diversion point.

[0013] The present invention also provides a cargo inspection and tracking device, the device comprising: a data acquisition module, used to acquire visual images of target cargo during transport in real time based on a visual sensor, wherein the visual sensor is disposed at multiple preset positions of the target cargo during transport, and the target cargo is assigned a unique identifier; and a positioning module, used to locate and track the target cargo in real time based on the unique identifier image displayed in the visual image corresponding to the unique identifier, and the acquisition position of the visual image.

[0014] The present invention also provides a cargo inspection system, the system comprising: a plurality of vision sensors disposed at a plurality of preset positions during the transport and operation of the target cargo; a processor for executing any of the cargo inspection tracking methods described herein; and a display device for displaying real-time positioning and tracking information of the target cargo.

[0015] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the cargo inspection and tracking method as described above.

[0016] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the cargo inspection and tracking method as described above.

[0017] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the cargo inspection and tracking method as described above.

[0018] This invention provides a cargo inspection tracking method, device, and system. Based on a vision sensor, it acquires real-time visual images of the target cargo during its transport process. The vision sensor is positioned at multiple preset locations along with a unique identifier. The target cargo is assigned a unique identifier. Based on the unique identifier image displayed in the visual image and the image acquisition location, the system can locate and track the target cargo in real-time, achieving accurate and real-time positioning and improving inspection efficiency. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0020] Figure 1 This is one of the flowcharts of the cargo inspection and tracking method provided by the present invention.

[0021] Figure 2 This is the second flowchart of the cargo inspection and tracking method provided by the present invention.

[0022] Figure 3 This is the third flowchart of the cargo inspection and tracking method provided by the present invention.

[0023] Figure 4 This is the fourth flowchart of the cargo inspection and tracking method provided by the present invention.

[0024] Figure 5 This is a schematic diagram of the cargo inspection and tracking device provided by the present invention.

[0025] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0027] The cargo inspection and tracking method provided by this invention solves the shortcomings of traditional methods that rely on transmission order and spacing. Even in cases of downtime for code replacement or changes in cargo order / position caused by lead curtain obstruction, it can still achieve accurate cargo tracking through multi-source data fusion and feature matching.

[0028] Figure 1 This is one of the flowcharts of the cargo inspection and tracking method provided by the present invention.

[0029] The following will combine Figure 1 The process of the cargo inspection and tracking method provided by the present invention will be described.

[0030] In an exemplary embodiment of the present invention, combined with Figure 1 As can be seen, the cargo inspection and tracking method may include steps 110 and 120, which will be described in detail below.

[0031] In step 110, visual images of the target cargo during the transport process are acquired in real time using a visual sensor. The visual sensor is set at multiple preset positions of the target cargo during the transport process, and the target cargo is assigned a unique identification code.

[0032] In one embodiment, industrial cameras (i.e., vision sensors) can be installed at multiple key preset locations on the cargo conveyor belt, i.e., multiple preset locations of the target cargo during the conveyor operation. For example, a high-resolution industrial camera can be deployed at the conveyor belt entrance (location A), multiple key inspection areas (such as side inspection location B, top inspection location C), and the conveyor belt exit (location D). These cameras are all connected to a central processing server. The vision sensors can also be other types of sensors; this embodiment does not limit the type of sensor.

[0033] In an exemplary embodiment of the present invention, the vision sensor is set at multiple preset positions of the target cargo during the conveying operation, which may include at least the following positions: the entrance position of the security inspection machine during the conveying operation, the exit position of the security inspection machine during the conveying operation, the straight section of the conveyor belt during the conveying operation, the bend of the conveyor belt during the conveying operation, and the sorting and diversion point of the cargo.

[0034] Vision sensors should be strategically installed at key points in the air cargo inspection system, such as the entrances and exits of security scanners, straight sections and curves of conveyor belts, and cargo sorting points (e.g., return line casters). These locations are chosen based on critical areas where cargo may experience changes in condition during transport, enabling comprehensive and timely capture of cargo location information.

[0035] In another embodiment, an RFID tag may be affixed to the outer packaging of the goods to be tracked (i.e., the target goods for inspection), and the tag stores a unique EPC code (i.e., a unique identifier). In addition, a one-dimensional barcode (i.e., a readable form of the unique identifier image) corresponding to the EPC code may be printed next to the RFID tag as a backup and redundancy for visual identification.

[0036] During the application, when goods bearing RFID tags and barcodes enter the conveyor belt, they sequentially pass through positions A, B, C, and D. Industrial cameras positioned at positions A, B, C, and D are triggered or continuously capture images of the goods passing beneath them at a set frame rate, thus obtaining a series of continuous visual images. Each captured image data is accompanied by a timestamp and capture location information (e.g., mapped to its physical installation location such as "entrance" or "X-ray inspection area" via camera ID).

[0037] In step 120, the target cargo is located and tracked in real time based on the unique identifier image displayed in the visual image that corresponds to the unique identifier, and the acquisition location of the visual image.

[0038] In another embodiment, each visual image can be analyzed in real time to locate and identify the barcode region (i.e., the unique identifier image) in the image. The identified barcode image can be decoded to extract the EPC encoding information. For example, in an image transmitted from a camera at location B, the code "EPC123456" can be successfully decoded. In application, the decoded unique identifier ("EPC123456") can be bound to the image acquisition location information ("Location B - Side Inspection Station"). Based on the unique identifier images displayed in the visual images at different times, corresponding to the unique identifier, a cargo tracking log can be constructed based on the corresponding visual image acquisition location, for example: At time T1, the target cargo "EPC123456" appears at [Location A - Entrance]. At time T2, the target cargo for inspection, "EPC123456", appears at [Location B - Side Inspection Station]. Accordingly, monitoring personnel or downstream automated systems can know the exact location of any target cargo (precisely specified by its unique identifier) ​​and its movement trajectory in real time, thereby achieving full-process location tracking.

[0039] In this embodiment, by deploying visual sensors at multiple key nodes along the transport path and processing image data in real time, this method breaks down the information silos of traditional tracking methods. It can continuously and automatically report the location of goods, providing a complete and continuous real-time trajectory from entry to exit, greatly improving the visibility of the goods' status. Furthermore, based on unique identification images (such as barcodes), each item has a unique identity, fundamentally avoiding errors caused by identification based solely on easily confused features such as appearance and size. Even if the goods rotate or tip over during transport, they can still be identified as long as the identification code is visible, ensuring high reliability.

[0040] This invention provides a cargo inspection and tracking method that uses visual sensors to collect real-time visual images of target cargo during its transport process. The visual sensors are positioned at multiple preset locations along with each target cargo, which is assigned a unique identifier. Based on the unique identifier image displayed in the visual image and the location where the visual image was collected, the method enables real-time and accurate location tracking of the target cargo, thereby improving the inspection efficiency.

[0041] Figure 2 This is the second flowchart of the cargo inspection and tracking method provided by the present invention.

[0042] The following will combine Figure 2 The process of the cargo inspection and tracking method provided by the present invention will be described.

[0043] In an exemplary embodiment of the present invention, combined with Figure 2 The process of another cargo inspection and cargo tracking method provided by the present invention will be described.

[0044] In an exemplary embodiment of the present invention, combined with Figure 2 As can be seen, after real-time acquisition of visual images of the target cargo during its transport process based on visual sensors, the cargo tracking method may further include steps 210 to 230, which will be described in detail below: In step 210, when the unique identifier image corresponding to the unique identifier displayed in the visual image is occluded, visual image features of the visual image at different times are extracted based on the visual images at different times.

[0045] In one embodiment, if the barcode on the surface of a package is obscured by the corner of another package due to tilting or improper stacking on the conveyor belt, making it impossible to locate or decode the barcode image, a backup tracking process can be triggered upon detecting the barcode decoding failure.

[0046] In application, visual image features at different times can be extracted from visual images. In one example, the currently acquired visual image (i.e., the image containing the obscured goods) can be analyzed to extract the overall visual image features of the goods. These features can include: appearance features: the overall color distribution (color histogram) and texture features of the goods; shape features: the outer contour shape and size ratio of the goods; and local features: special patterns, text, tape application methods, wear marks, etc. on the packaging.

[0047] In another embodiment, visual image features from multiple frames of images at different times (e.g., 1 second before the occlusion occurs, during the occlusion, and 1 second after the occlusion occurs) can be continuously extracted to form a more stable and comprehensive feature set.

[0048] In step 220, the target cargo image features of the target cargo are obtained.

[0049] In step 230, if the visual image features match the target cargo image features of the target cargo to be inspected, the target cargo to be inspected is located and tracked in real time based on the acquisition location of the visual image.

[0050] In another embodiment, the target cargo image features can be acquired. Further, visual image features extracted from the current occluded image can be compared with the acquired target cargo image features for similarity calculation. Based on a predefined similarity threshold (e.g., 98%), it is determined whether the two match. If a match is successful, the cargo with an unreadable barcode currently within the camera's field of view at the current location can be identified as the target cargo.

[0051] In another embodiment, the target cargo can be bound to the current image acquisition location, and the tracking log can be updated, for example, "Time T2, target cargo confirmed to be located at location X." In this embodiment, despite the unique identifier being obscured, real-time positioning and tracking of the target cargo was still successfully achieved, ensuring seamless continuity of the tracking link.

[0052] Cargo identification codes are frequently obscured, damaged, or detached. This embodiment provides an effective redundant backup identification mechanism. When the primary identification method (code reading) fails, it can seamlessly switch to the backup identification method (cargo image feature matching), thereby overcoming the technical limitations of relying solely on identification codes, ensuring that the tracking process is never interrupted, and greatly improving the reliability and robustness of the entire system in complex real-world environments.

[0053] In yet another exemplary embodiment of the present invention, the visual image features of the visual image can be obtained using the following: The pre-trained feature extraction model is invoked, whereby the feature extraction model is used to obtain visual image features corresponding to the visual image based on the visual image; The visual image is input into the feature extraction model, and the visual image features corresponding to the visual image are obtained from the output of the feature extraction model.

[0054] In one embodiment, a pre-trained feature extraction model can be invoked, wherein the feature extraction model is used to obtain visual image features corresponding to the visual image based on the visual image. The feature extraction model can be a deep convolutional neural network (CNN) for computer vision tasks, such as adopting the architecture of ResNet-50 or Vision Transformer (ViT) and modifying it.

[0055] It's important to note that the feature extraction model is not randomly initialized, but rather pre-trained using a massive dataset of cargo images. The training objective is not classification (e.g., identifying whether it's a "box" or a "bag"), but rather metric learning (e.g., Triplet Loss training), enabling the model to learn to map any input cargo image to a high-dimensional feature vector space. In this space, feature vectors from different images of the same item are close together, while feature vectors from different items are far apart. The sole function of the feature extraction model is to obtain visual image features corresponding to the visual image.

[0056] In application, the visual image to be identified (e.g., scaled to the model's required 224×224 pixels) is input into the feature extraction model. The model's forward propagation network performs a series of complex convolutions and nonlinear transformations on the image, ultimately producing a fixed-length, high-dimensional numerical vector at its output layer. This numerical vector is the visual image feature output by the model, corresponding to the input visual image. This feature vector is essentially a highly abstract and condensed digital representation of the cargo image, containing its most essential, distinguishing information from other cargo.

[0057] In the aforementioned embodiments, by invoking a pre-trained feature extraction model, this solution can automatically learn and extract visual image features from visual images. These features can capture extremely subtle differences between goods, thereby greatly improving the discriminativeness and accuracy of feature representation, laying a solid foundation for subsequent high-precision matching.

[0058] In yet another exemplary embodiment of the present invention, continuing with the previously described embodiments, the cargo inspection and tracking method may further include the following steps: The pre-trained cargo detection model is invoked, which is used to identify cargo in a visual image based on the visual image. The visual image is input into the cargo detection model to obtain the cargo to be inspected, which corresponds to the visual image, output by the cargo detection model. Based on the quantity of goods inspected, the goods inspected by visual image monitoring are counted.

[0059] In one embodiment, the cargo detection model can be a deep learning-based object detection neural network, such as YOLO (You Only Look Once) or Faster R-CNN architecture.

[0060] During training, a large dataset of conveyor belt images labeled with cargo bounding boxes can be used for pre-training, enabling the system to learn to recognize and locate various types of cargo in the images without relying on the identification codes on the cargo.

[0061] In application, visual images can be input into the cargo detection model to obtain the cargo to be inspected corresponding to the visual images, as output by the model. The model can perform real-time inference on the images, identify each cargo in the image, and output information for each identified cargo. This information typically includes: the bounding box coordinates of each cargo, used to locate the cargo in the image, as well as the cargo's category label (such as "cardboard box", "sack", "wooden box") and confidence score.

[0062] In another embodiment, the number of all identified goods to be inspected in the current frame of the visual image can be counted (i.e., the number of bounding boxes). In application, cameras can be deployed at the start (entrance) or end (exit) of the conveyor belt to count each new item entering or leaving the field of view, thereby realizing the statistics of the throughput of the entire production line.

[0063] Figure 3 This is the third flowchart of the cargo inspection and tracking method provided by the present invention.

[0064] The following will combine Figure 3 The process of another cargo inspection and tracking method provided by the present invention will be described.

[0065] In an exemplary embodiment of the present invention, combined with Figure 3 As can be seen, after obtaining the cargo inspection goods corresponding to the visual image output by the cargo inspection model, the cargo inspection tracking method may further include the following steps 310 to 330, which will be described in detail below: In step 310, based on the visual images at different times, the features of the inspected goods in the visual images at different times are extracted.

[0066] In one embodiment, deeper image analysis can be performed on each piece of cargo identified by the detection model (defined by its bounding box) in each frame of the visual image. During application, cargo features can be extracted from visual images at different times.

[0067] In step 320, at each time point, the goods inspected in the visual image are clustered based on the characteristics of the goods inspected to obtain different types of goods inspected and the quantity of goods inspected for each type.

[0068] In another embodiment, an unsupervised clustering algorithm (such as K-Means or DBSCAN) can be used to analyze the cargo characteristics of all inspected goods at the current time. The clustering algorithm will automatically group goods with similar features into the same cluster and goods with large feature differences into different clusters based on the similarity between features. Each cluster represents a different type of cargo to be inspected.

[0069] Furthermore, by counting the number of goods contained in each cluster, the quantity of goods of different types can be obtained. That is, at each time point, the goods in the visual image are clustered based on the characteristics of the goods to obtain different types of goods and the quantity of goods of different types.

[0070] In step 330, the count changes of different types of inspected goods are monitored based on the quantity of different types of inspected goods at different times.

[0071] During application, steps 310 and 320 can be continuously executed, and the classification and counting results at different times (e.g., every minute) can be recorded to monitor the count changes of different types of goods. In this embodiment, without the need for manually predefined rules, the clustering algorithm can automatically discover and distinguish goods categories, achieving refined and automated management of goods flow.

[0072] Figure 4 This is the fourth flowchart of the cargo inspection and tracking method provided by the present invention.

[0073] The following will combine Figure 4 The process of the cargo inspection and tracking method provided by the present invention will be described.

[0074] In an exemplary embodiment of the present invention, combined with Figure 4 As can be seen, after real-time location tracking of the target cargo, the cargo tracking method may further include steps 410 to 440, which will be described in detail below: In step 410, when the target cargo arrives at the destination of the transport operation, a three-dimensional position model of the cargo pallet containing the target cargo is constructed.

[0075] In one embodiment, when the target goods arrive at the destination of the transport operation, the entire pallet can be scanned to collect high-precision point cloud data. Based on this point cloud data, a precise three-dimensional position model reflecting the actual placement of each item on the pallet (including the target goods) can be constructed. This model includes the three-dimensional dimensions, center coordinates, orientation angle, and stacking relationship between each item.

[0076] In step 420, based on the three-dimensional position model, the twin grasping position sequence of the target cargo relative to the three-dimensional position model is determined according to the minimum grasping distance.

[0077] In one embodiment, a 3D position model can be loaded based on a path planning algorithm. The path planning algorithm needs to calculate the optimal path for grasping the target cargo. Since the cargo may be partially obscured by other cargo or located in the middle of a stack, directly grasping it may collide with other cargo. Therefore, it is necessary to remove the cargo on top of it first. The algorithm uses the minimum grasping distance (i.e., the shortest total path of the robotic arm movement and the least time) as the optimization objective, and performs analysis and calculation in the digital twin model to determine an optimal grasping sequence. This sequence can be a twin grasping position sequence, which specifies a series of grasping points (coordinates in the 3D model) and the grasping order, for example: "Step 1: Grab cargo A at the top of the model; Step 2: Grab cargo B on the side of the model; Step 3: The target cargo is now exposed, grasp it."

[0078] In step 430, the actual grab position sequence of the target cargo is determined based on the twin grab position sequence.

[0079] In step 440, the target goods to be inspected on the goods pallet are grasped based on the actual grasping position sequence.

[0080] In another embodiment, the twin grasping position sequence previously calculated in digital space can be used to determine the actual grasping position sequence in the real world corresponding to the twin grasping position sequence through coordinate mapping. This sequence contains the precise three-dimensional coordinates and orientation of each grasping point that the robotic arm needs to move to sequentially in the real physical space.

[0081] Furthermore, the target goods for inspection on the pallet can be grasped based on the actual grasping position sequence. For example, moving to the first grasping point in the sequence, the gripper is controlled to grasp and remove obstacle goods A, placing them aside. Moving to the second grasping point in the sequence, obstacle goods B are grasped and removed. Finally, the gripper moves to the grasping point of the exposed target goods, successfully grasps them, and places them on the designated shipping conveyor belt. This completes the grasping of the target goods for inspection on the pallet. In this embodiment, by constructing a three-dimensional position model and performing pre-simulation and optimization in a digital twin space, the sequence of minimum grasping distances is calculated, ensuring that the robotic arm's grasping path is globally optimal, avoiding invalid movements and "trial and error" operations, thereby significantly improving the efficiency and intelligence of the grasping operation.

[0082] As described above, the cargo inspection and tracking method provided by this invention breaks through the traditional tracking logic that relies on the delivery order. By using video AI technology to establish an independent visual feature identifier for each cargo, it achieves multi-point dynamic tracking of individual cargoes. Even if the cargo spacing is insufficient or the order changes, it can still accurately locate individual cargoes. It integrates real-time position data from sensors with visual feature data from video AI. When sensor data deviates due to occlusion or other issues, it corrects the deviation through visual feature matching. Conversely, when visual recognition is affected by light or occlusion, it uses sensor data as a reference benchmark to improve the stability of position judgment.

[0083] The cargo inspection and tracking device provided by the present invention is described below. The cargo inspection and tracking device described below can be referred to in correspondence with the cargo inspection and tracking method described above.

[0084] Figure 5 This is a schematic diagram of the cargo inspection and tracking device provided by the present invention.

[0085] In an exemplary embodiment of the present invention, combined with Figure 5 As can be seen, the cargo inspection and tracking device may include a data acquisition module 510 and a positioning module 520. Each module will be described in detail below.

[0086] The acquisition module 510 can be configured to acquire visual images of the target cargo during the conveying process in real time based on a visual sensor, wherein the visual sensor is set at multiple preset positions of the target cargo during the conveying process, and the target cargo is set with a unique identification code. The positioning module 520 can be configured to locate and track the target cargo in real time based on the unique identifier image displayed in the visual image corresponding to the unique identifier, and the acquisition location of the visual image.

[0087] In an exemplary embodiment of the present invention, the positioning module 520 may further be configured to: When the unique identifier image corresponding to the unique identifier displayed in the visual image is occluded, visual image features of the visual image at different times are extracted based on the visual image at different times. Obtain the target cargo image features of the target cargo to be inspected; If the visual image features match the target cargo image features of the target cargo to be inspected, the target cargo to be inspected is located and tracked in real time based on the acquisition location of the visual image.

[0088] In an exemplary embodiment of the present invention, the positioning module 520 may obtain the visual image features of the visual image in the following manner: The pre-trained feature extraction model is invoked, wherein the feature extraction model is used to obtain visual image features corresponding to the visual image based on the visual image; The visual image is input into the feature extraction model to obtain the visual image features output by the feature extraction model that correspond to the visual image.

[0089] In an exemplary embodiment of the present invention, the positioning module 520 may further be configured to: A pre-trained cargo detection model is invoked, wherein the cargo detection model is used to identify cargo in the visual image based on the visual image; The visual image is input into the cargo detection model to obtain the cargo to be inspected, which corresponds to the visual image, output by the cargo detection model. Based on the quantity of the inspected goods, the inspected goods monitored by the visual image are counted.

[0090] In an exemplary embodiment of the present invention, the positioning module 520 may further be configured to: Based on the visual images at different times, the features of the inspected goods in the visual images at different times are extracted; At each time point, the inspected goods in the visual image are clustered based on the characteristics of the inspected goods to obtain different types of inspected goods and the quantity of each type of inspected goods. Based on the quantity of different types of goods to be inspected at different times, monitor the changes in the count of different types of goods to be inspected.

[0091] In an exemplary embodiment of the present invention, the positioning module 520 may further be configured to: When the target cargo arrives at the destination of the transport operation, a three-dimensional position model of the cargo pallet containing the target cargo is constructed. Based on the three-dimensional position model, the twin grasping position sequence of the target cargo relative to the three-dimensional position model is determined according to the minimum grasping distance; Based on the twin grasp position sequence, the actual grasp position sequence of the target cargo to be inspected is determined; Based on the actual grabbing position sequence, the target goods for inspection are grabbed from the goods pallet.

[0092] In an exemplary embodiment of the present invention, the visual sensor is disposed at multiple preset positions during the transport of the target cargo, including at least the following positions: The locations of the security inspection machine's entrance and exit points during the conveyor operation, the straight sections and bends of the conveyor belt, and the sorting and distribution points for inspected goods.

[0093] Based on the same inventive concept, the present invention also provides a cargo inspection system, the structure of which will be described below with reference to the following embodiments.

[0094] In an exemplary embodiment of the present invention, the cargo inspection system may include multiple vision sensors, a processor, and a display device, each of which will be described below.

[0095] Visual sensors are set at multiple preset locations during the transport of the target cargo. The processor is used to execute the cargo inspection and tracking method described in any of the preceding embodiments; The display device is used to display the real-time location tracking information of the target cargo for inspection.

[0096] In this embodiment, the real-time visual monitoring of the status, location, and trajectory of goods through the display device facilitates timely detection of anomalies (such as goods stagnation or trajectory deviation) by staff, thereby improving the controllability and management efficiency of the goods inspection process.

[0097] Figure 6 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6As shown, the electronic device may include a processor 610, a communications interface 620, a memory 630, and a communication bus 640. The processor 610, communications interface 620, and memory 630 communicate with each other via the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute a cargo tracking method. This method includes: real-time acquisition of visual images of the target cargo during its transport process using visual sensors, wherein the visual sensors are positioned at multiple preset locations during the transport process, and the target cargo is assigned a unique identifier; and real-time location and tracking of the target cargo based on the unique identifier image displayed in the visual images corresponding to the unique identifier, and the acquisition location of the visual images.

[0098] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0099] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the cargo tracking method provided by the above methods. The method includes: acquiring visual images of target cargo during transport in real time based on a visual sensor, wherein the visual sensor is set at multiple preset positions of the target cargo during transport, and the target cargo is assigned a unique identifier; and locating and tracking the target cargo in real time based on the unique identifier image displayed in the visual image corresponding to the unique identifier and the acquisition position of the visual image.

[0100] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the cargo tracking method provided by the above methods. The method includes: acquiring visual images of target cargo during transport in real time based on a visual sensor, wherein the visual sensor is disposed at multiple preset positions of the target cargo during transport, and the target cargo is assigned a unique identifier; and locating and tracking the target cargo in real time based on the unique identifier image displayed in the visual image corresponding to the unique identifier, and the acquisition position of the visual image.

[0101] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0102] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A cargo inspection and tracking method, characterized in that, The method includes: The visual images of the target cargo during the conveying process are collected in real time using a visual sensor. The visual sensor is set at multiple preset positions during the conveying process of the target cargo, and the target cargo is set with a unique identification code. Based on the unique identifier image displayed in the visual image that corresponds to the unique identifier, and the acquisition location of the visual image, the target cargo is located and tracked in real time.

2. The cargo inspection and tracking method according to claim 1, characterized in that, After acquiring visual images of the target cargo during its transport process in real time using a visual sensor, the method further includes: When the unique identifier image corresponding to the unique identifier displayed in the visual image is occluded, visual image features of the visual image at different times are extracted based on the visual image at different times. Obtain the target cargo image features of the target cargo to be inspected; If the visual image features match the target cargo image features of the target cargo to be inspected, the target cargo to be inspected is located and tracked in real time based on the acquisition location of the visual image.

3. The cargo inspection and tracking method according to claim 2, characterized in that, The visual image features of the visual image are obtained using the following methods: The pre-trained feature extraction model is invoked, wherein the feature extraction model is used to obtain visual image features corresponding to the visual image based on the visual image; The visual image is input into the feature extraction model to obtain the visual image features output by the feature extraction model that correspond to the visual image.

4. The cargo inspection and tracking method according to any one of claims 1 to 3, characterized in that, The method further includes: A pre-trained cargo detection model is invoked, wherein the cargo detection model is used to identify cargo in the visual image based on the visual image; The visual image is input into the cargo detection model to obtain the cargo to be inspected, which corresponds to the visual image, output by the cargo detection model. Based on the quantity of the inspected goods, the inspected goods monitored by the visual image are counted.

5. The cargo inspection and tracking method according to claim 4, characterized in that, After obtaining the cargo to be inspected corresponding to the visual image output by the cargo detection model, the method further includes: Based on the visual images at different times, the features of the inspected goods in the visual images at different times are extracted; At each time point, the inspected goods in the visual image are clustered based on the characteristics of the inspected goods to obtain different types of inspected goods and the quantity of each type of inspected goods. Based on the quantity of different types of goods to be inspected at different times, monitor the changes in the count of different types of goods to be inspected.

6. The cargo inspection and tracking method according to claim 1, characterized in that, After real-time location tracking of the target cargo, the method further includes: When the target cargo arrives at the destination of the transport operation, a three-dimensional position model of the cargo pallet containing the target cargo is constructed. Based on the three-dimensional position model, the twin grasping position sequence of the target cargo relative to the three-dimensional position model is determined according to the minimum grasping distance; Based on the twin grasp position sequence, the actual grasp position sequence of the target cargo to be inspected is determined; Based on the actual grabbing position sequence, the target goods for inspection are grabbed from the goods pallet.

7. The cargo inspection and tracking method according to claim 1, characterized in that, The vision sensors are installed at multiple preset locations during the transport of the target cargo, including at least the following locations: The locations of the security inspection machine's entrance and exit points during the conveyor operation, the straight sections and bends of the conveyor belt, and the sorting and distribution points for inspected goods.

8. A cargo inspection and tracking device, characterized in that, The device includes: The acquisition module is used to acquire visual images of the target cargo during the conveying process in real time based on a visual sensor. The visual sensor is set at multiple preset positions of the target cargo during the conveying process, and the target cargo is set with a unique identification code. The positioning module is used to locate and track the target cargo in real time based on the unique identifier image displayed in the visual image that corresponds to the unique identifier, and the acquisition location of the visual image.

9. A cargo inspection system, characterized in that, The system includes: Multiple vision sensors are installed at multiple preset positions during the transport and operation of the target cargo. A processor, the processor being configured to execute the cargo inspection and tracking method according to any one of claims 1 to 7; The display device is used to display the real-time location tracking information of the target cargo.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the cargo inspection and tracking method as described in any one of claims 1 to 7.

Images