Empty container inspection method and apparatus, storage medium, and program product
By using an empty container detection method to quickly and accurately inspect vehicle cargo compartments and employing machine learning models to identify items inside the cargo compartments, the problem of low efficiency and high false alarm rate of traditional security inspection methods is solved, achieving efficient and accurate security inspection results.
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- NUCTECH CO LTD
- Filing Date
- 2025-06-25
- Publication Date
- 2026-07-02
AI Technical Summary
Traditional security inspection methods are difficult to quickly and accurately process the cargo compartments of complex-structured freight vehicles, resulting in low inspection efficiency and a high false alarm rate.
The empty container detection method is adopted. By acquiring perspective images of the vehicle and segmenting the container images, a machine learning model is used to identify the items to be detected, determine whether they are suspected items or whitelisted items, and output risk indication information.
It enables rapid and accurate inspection of vehicle cargo compartments, improves security inspection efficiency, reduces manual intervention, and lowers the false alarm rate.
Smart Images
Figure CN2025103496_02072026_PF_FP_ABST
Abstract
Description
Empty box testing methods and apparatus, storage media and program products
[0001] Cross-reference to related applications
[0002] This disclosure is based on and claims priority to CN application No. 202411927290.4, filed on December 25, 2024, the contents of which are incorporated herein by reference in their entirety. Technical Field
[0003] This disclosure relates to the field of safety testing, and in particular to a method and apparatus for testing empty boxes, a storage medium, and a program product. Background Technology
[0004] In the current context of globalization, the international security situation is becoming increasingly severe, particularly in the areas of customs and port supervision of import and export activities. With the rise of international terrorism and the increasing sophistication of smuggling and drug trafficking, ensuring the safety and legality of imported and exported goods has become a significant challenge for governments and customs authorities worldwide.
[0005] Large container / vehicle inspection systems, as a non-invasive inspection method, have been widely used globally due to their efficiency and accuracy. These systems not only effectively prevent terrorists from using cargo transportation channels for illegal activities, but also ensure the full collection of taxes due, prevent smuggling, drug trafficking, and other illegal activities, thereby safeguarding national economic security and social stability. Summary of the Invention
[0006] In a first aspect of this disclosure, an empty container detection method is provided, comprising: acquiring a perspective image of a vehicle declared as having an empty container; segmenting the perspective image to obtain a container image of the vehicle; if the container image is found to contain an item to be detected, detecting whether the item to be detected is a suspect item; if the item to be detected is not a suspect item, using a machine learning model to identify whether the item to be detected is a whitelisted item; and outputting risk indication information based on the identification result of the machine learning model.
[0007] In some embodiments, identifying whether the item to be detected is a whitelisted item using a machine learning model includes: detecting whether the whitelist master switch is in the on state; if the whitelist master switch is in the on state, taking the whitelist corresponding to the whitelist category switch in the on state as the current whitelist; using the machine learning model to identify whether the item to be detected is included in the current whitelist; if the item to be detected is included in the current whitelist, determining that the item to be detected is a whitelisted item.
[0008] In some embodiments, the machine learning model includes a first sub-model, a second sub-model, and a third sub-model. The step of using the machine learning model to identify whether the item to be detected is included in the current whitelist includes: using the first sub-model to obtain a local image of the area where the item to be detected is located from the vehicle image; using the second sub-model to extract image features from the local image; and using the third sub-model to detect whether the image features are included in the feature library of the current whitelist, wherein if the image features are included in the feature library of the current whitelist, then it is determined that the item to be detected is included in the current whitelist.
[0009] In some embodiments, outputting risk indication information based on the recognition results of the machine learning model includes: when the machine learning model identifies that the item to be detected is not a whitelisted item, outputting indication information to indicate that the vehicle has a high risk.
[0010] In some embodiments, outputting risk indication information based on the recognition results of the machine learning model includes: when the machine learning model identifies the item to be detected as a whitelisted item, outputting indication information to indicate that the vehicle has a low risk.
[0011] In some embodiments, when the whitelist master switch is in the off state, an indication message is output to indicate that the vehicle is at high risk.
[0012] In some embodiments, if the item to be detected is the suspected item, an indication message is output to indicate that the vehicle is at high risk.
[0013] In some embodiments, if the object to be detected is not included in the vehicle compartment image, an indication message is output to indicate that the vehicle has a low risk.
[0014] In some embodiments, acquiring the perspective image of a vehicle declared as empty includes: obtaining the vehicle's declaration information; determining whether the vehicle is empty based on the declaration information; and, if the vehicle is determined to be empty based on the declaration information, scanning the vehicle with X-rays to obtain the perspective image.
[0015] In some embodiments, the area where the object to be detected is located is marked in the perspective image.
[0016] In some embodiments, the machine learning model is trained, wherein training the machine learning model includes: generating multiple train car sample images; adding annotation information to items included in each train car sample image; processing each train car sample image using the machine learning model to obtain the recognition result of the items in each train car sample image; determining a loss function based on the recognition result and the annotation information of the items; and training the machine learning model based on the loss function.
[0017] In some embodiments, the machine learning model includes one of the Faster R-CNN model, the YOLO model, the Single-Shot Multi-Box Detector (SSD) model, or the EfficientDet model.
[0018] In some embodiments, generating multiple vehicle compartment sample images includes: randomly selecting multiple empty vehicle compartment images from an empty vehicle compartment image library, wherein the empty vehicle compartment image library includes multiple empty vehicle compartment images of different vehicle models; in each of the multiple empty vehicle compartment images, a randomly selected area is used as a target area; at least one item image is added to the target area, wherein the at least one item image includes at least one whitelisted item image and at least one prohibited item image; and the size of the at least one item image is randomly adjusted to generate the multiple vehicle compartment sample images.
[0019] In some embodiments, adding annotation information to the items included in each carriage sample image includes: dividing the plurality of carriage sample images into a training sample set and a test sample set; manually adding annotation information to the items included in each training sample in the training sample set; training a machine learning model using the training sample set; processing each test sample in the test sample set using the trained machine learning model to obtain the item recognition result of each test sample; using the item recognition result of each test sample as the annotation information of the items included in each test sample; if the annotation information of the item included in the i-th test sample matches the item included in the i-th test sample, then storing the annotation information of the item included in the i-th test sample, 1≤i≤N, where N is the total number of test samples in the test sample set.
[0020] In some embodiments, if the labeling information of the items included in the i-th test sample does not match the items included in the i-th test sample, the labeling information of the items included in the i-th test sample is manually modified and added; the machine learning model is repeatedly trained using the training sample set and the i-th test sample; the trained machine learning model is used to process the test samples in the test sample set that have not been labeled, until the items included in each test sample in the test sample set are labeled.
[0021] According to a second aspect of the present disclosure, an empty box detection apparatus is provided, comprising: a memory; and a processor coupled to the memory, the processor being configured to execute instructions stored in the memory to implement the empty box detection method as described in any of the above embodiments.
[0022] According to a third aspect of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions that, when executed by a processor, implement the method as described in any of the above embodiments.
[0023] According to a fourth aspect of the present disclosure, a computer program product is provided, including computer instructions, wherein the computer instructions, when executed by a processor, implement the method as described in any of the above embodiments.
[0024] According to a fifth aspect of the present disclosure, a computer program is provided, including computer instructions, wherein the computer instructions, when executed by a processor, implement the method as described in any of the above embodiments.
[0025] Other features and advantages of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description
[0026] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0027] Figure 1 is a flowchart illustrating an empty box detection method according to an embodiment of this disclosure;
[0028] Figure 2 is a flowchart illustrating a machine learning model training method according to an embodiment of this disclosure;
[0029] Figure 3 is a flowchart illustrating a method for generating a vehicle compartment sample image according to an embodiment of this disclosure;
[0030] Figure 4 is a flowchart illustrating an annotation method according to an embodiment of this disclosure;
[0031] Figure 5 is a flowchart illustrating an empty box detection method according to another embodiment of this disclosure;
[0032] Figure 6 is a schematic diagram of a vehicle perspective image according to an embodiment of the present disclosure;
[0033] Figure 7 is a schematic diagram of a vehicle perspective image according to another embodiment of this disclosure;
[0034] Figure 8 is a schematic diagram of a vehicle perspective image according to yet another embodiment of this disclosure;
[0035] Figure 9 is a schematic diagram of the structure of an empty box detection device according to an embodiment of the present disclosure. Detailed Implementation
[0036] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.
[0037] Unless otherwise specifically stated, the relative arrangement, numerical expressions, and values of the components and steps set forth in these embodiments do not limit the scope of this disclosure.
[0038] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.
[0039] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.
[0040] In all examples shown and discussed herein, any specific values should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.
[0041] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.
[0042] The inventors noted that with the continuous development of global trade, the volume of customs clearance at ports and customs is increasing year by year, the types of transported goods are becoming more and more diverse, and the structure of cargo vehicles is becoming more and more complex. These factors make traditional security inspection methods difficult to adapt to the current situation.
[0043] Accordingly, this disclosure provides an empty container detection method that can quickly and accurately inspect the vehicle's cargo box, thereby effectively improving security inspection efficiency, reducing manual intervention, and effectively lowering the false alarm rate.
[0044] Figure 1 is a schematic flowchart of an empty box detection method according to an embodiment of the present disclosure. In some embodiments, the following empty box detection method is performed by an empty box detection device, including steps 11-15.
[0045] In step 11, a perspective image of the vehicle declared as having an empty container is acquired.
[0046] In some embodiments, the step of acquiring perspective images of vehicles declared as empty includes the following 1)-3).
[0047] 1) Obtain vehicle declaration information.
[0048] 2) Determine whether the vehicle is empty based on the declared information.
[0049] 3) If the vehicle is determined to be empty based on the declared information, use X-rays to scan the vehicle to obtain a transparent image.
[0050] In step 12, the perspective image is segmented to obtain the vehicle's cargo box image.
[0051] It should be noted that, considering that prohibited items are usually placed in the cargo compartment of a vehicle, in order to improve processing efficiency, the cargo compartment image is obtained from the perspective image so that the cargo compartment image can be processed.
[0052] In step 13, if the image of the vehicle compartment contains the item to be detected, it is determined whether the item to be detected is a suspect.
[0053] In some embodiments, a trained recognition model is used to detect whether the item to be detected is a suspect.
[0054] For example, a local image of the area where the object to be detected is located is obtained from an image of the vehicle compartment, and image features of that local image are extracted. If the image features are included in a predetermined suspect feature library, the object to be detected is determined to be a suspect.
[0055] For example, based on the texture and shape features of a local image, it can be used to detect whether an object is a suspect.
[0056] For example, recognition models include the Faster R-CNN model.
[0057] In step 14, if the item to be detected is not a suspect, a machine learning model is used to identify whether the item to be detected is a whitelisted item.
[0058] In some embodiments, the machine learning model includes one of the following: Faster R-CNN, YOLO, SSD (Single Shot MultiBox Detector), or EfficientDet.
[0059] In some embodiments, a machine learning model is trained so that the trained machine learning model can identify whether an item to be detected is a whitelisted item.
[0060] Figure 2 is a flowchart illustrating a machine learning model training method according to an embodiment of the present disclosure, including steps 21-25.
[0061] In step 21, multiple vehicle compartment sample images are generated, wherein each vehicle compartment sample image includes at least one item image, which includes at least one of at least one whitelisted item image and at least one prohibited item image.
[0062] It should be noted that the limited number of items on the whitelist results in a limited number of sample images for the vehicle compartments. To address this issue, image synthesis is used to generate sample images of the vehicle compartments.
[0063] Figure 3 is a flowchart illustrating a method for generating a vehicle compartment sample image according to an embodiment of this disclosure, including steps 211-214.
[0064] In step 211, multiple empty vehicle images are randomly selected from the empty vehicle image library, which includes images of empty vehicles of different models.
[0065] In step 212, in each of the multiple empty car body images, a randomly selected region is taken as the target region.
[0066] In step 213, at least one item image is added to the target area.
[0067] It should be noted that at least one item image includes at least one whitelisted item image and at least one prohibited item image.
[0068] In step 214, the size of the at least one item image is randomly adjusted to generate multiple sample images of the vehicle compartment.
[0069] Through the above processing, at least one of the following can be added to a randomly selected area in the empty cargo box images of different vehicle models: at least one whitelisted item image and at least one prohibited item image. The size of the added item images can be randomly adjusted, thereby effectively increasing the number of cargo box sample images to facilitate effective training of machine learning models.
[0070] In step 22, add annotation information to the items included in each vehicle sample image.
[0071] It should be noted that when dealing with a large volume of sample images of train carriages, manually adding annotations to the items included in each image would be extremely time-consuming. To address this issue, a combination of manual and automatic annotation is employed to effectively reduce annotation time.
[0072] Figure 4 is a flowchart illustrating an annotation method according to an embodiment of the present disclosure, including steps 221-228.
[0073] In step 221, the multiple car body sample images are divided into a training sample set and a test sample set.
[0074] In step 222, annotation information is added manually to the items included in each training sample in the training sample set.
[0075] In step 223, a machine learning model is trained using the training sample set.
[0076] In step 224, the trained machine learning model is used to process each test sample in the test sample set to obtain the item recognition result for each test sample.
[0077] In step 225, the item identification result of each test sample is used as the annotation information of the items included in each test sample.
[0078] In step 226, check whether the labeling information of the items in each test sample matches the items included in the test sample.
[0079] If the labeling information of the item in the i-th test sample matches the items included in the i-th test sample, then proceed to step 227, where 1≤i≤N, and N is the total number of test samples in the test sample set; otherwise, proceed to step 228.
[0080] In step 227, the labeling information of the items included in the i-th test sample is stored.
[0081] In step 228, the labeling information of the items included in the i-th test sample is modified manually.
[0082] Next, the machine learning model is repeatedly trained using the training sample set and the i-th test sample. The trained machine learning model is then used to process the test samples in the test sample set that have not been labeled, until the items included in each test sample in the test sample set are labeled.
[0083] The above processing can effectively reduce the sample labeling time.
[0084] In step 23, a machine learning model is used to process each car body sample image to obtain the identification results of the items in each car body sample image.
[0085] In step 24, the loss function is determined based on the recognition results and the item labeling information.
[0086] In step 25, the machine learning model is trained based on the loss function.
[0087] In some embodiments, the step of using a machine learning model to identify whether an item to be detected is a whitelisted item includes the following 1)-4):
[0088] 1) Check if the whitelist master switch is in the on state.
[0089] 2) When the whitelist master switch is on, the whitelist corresponding to the whitelist category switch that is on will be used as the current whitelist.
[0090] It's important to note that whitelists may differ depending on the scenario. Therefore, multiple whitelists can be set up as needed, along with multiple whitelist category switches corresponding to each whitelist. If a whitelist category switch is enabled, it indicates that the whitelist corresponding to that category switch is currently in use.
[0091] In some embodiments, when the whitelist master switch is off, an indication message is output to indicate that a vehicle is at high risk.
[0092] 3) Use machine learning models to identify whether the item to be detected is included in the current whitelist.
[0093] 4) If the item to be detected is included in the current whitelist, determine that the item to be detected is a whitelisted item.
[0094] In some embodiments, the machine learning model includes a first sub-model, a second sub-model, and a third sub-model. The first sub-model is used to obtain a local image of the region containing the item to be detected from the vehicle image. The second sub-model is used to extract image features from the local image. The third sub-model is used to detect whether the image features are included in the current whitelist feature library, wherein if the image features are included in the current whitelist feature library, it is determined that the item to be detected is included in the current whitelist.
[0095] For example, machine learning models include ResNet (Residual Network) and RPN (Region Proposal Network). ResNet extracts feature maps at different scales, while RPN fuses these feature maps from the ResNet structure to generate a multi-scale feature pyramid. A series of convolutional layers then generate candidate regions. RoI pooling (Region of Interest pooling) layers extract the features of the candidate regions. Finally, classification and regression layers are used to obtain the class labels and bounding box coordinates of the candidate regions.
[0096] In step 15, risk indication information is output based on the recognition results of the machine learning model.
[0097] In some embodiments, if the machine learning model identifies the item to be detected as not being a whitelisted item, it outputs an indication that the vehicle is at high risk. If the machine learning model identifies the item to be detected as being a whitelisted item, it outputs an indication that the vehicle is at low risk.
[0098] In some embodiments, when the item to be detected is a suspect, an indication message is output to indicate that the vehicle poses a high risk.
[0099] In some embodiments, if the image of the vehicle compartment does not include the item to be detected, an indication message is output to indicate that the vehicle has a low risk.
[0100] In some embodiments, when outputting indication information indicating that a vehicle is at high risk, the area where the item to be detected is located is marked in the perspective image so that staff can promptly understand the location of the risky item.
[0101] Figure 5 is a flowchart illustrating an empty box detection method according to another embodiment of this disclosure, including steps 51-55.
[0102] In step 51, a perspective image of the vehicle declared as having an empty container is acquired.
[0103] In step 52, the perspective image is segmented to obtain the vehicle's cargo box image.
[0104] In step 53, it is determined whether the image of the vehicle compartment contains the item to be detected.
[0105] If the item to be detected is not included in the image of the vehicle compartment, proceed to step 54. Otherwise, proceed to step 55.
[0106] In step 54, an indication message is output to indicate that the vehicle has a low risk.
[0107] In step 55, the item to be tested is checked to determine whether it is a suspect.
[0108] If the item to be tested is a suspect, proceed to step 56. Otherwise, proceed to step 57.
[0109] In step 56, an indication message is output to indicate that the vehicle is at high risk.
[0110] In some embodiments, the area where the object to be detected is located is also marked in the perspective image.
[0111] In step 57, when the whitelist master switch is in the on state, the whitelist corresponding to the whitelist category switch that is in the on state is taken as the current whitelist, and the machine learning model is used to identify whether the item to be detected is included in the current whitelist.
[0112] If the item to be detected is included in the current whitelist, proceed to step 54. Otherwise, proceed to step 56.
[0113] In some embodiments, when the whitelist master switch is off, an indication message is output to indicate that a vehicle is at high risk.
[0114] In the empty container detection method provided in the above embodiments of this disclosure, in addition to detecting whether the items to be detected in the vehicle compartment are suspicious, a machine learning model is further used to identify whether the items to be detected are whitelisted items. Based on the identification results of the machine learning model, risk indication information is output. This enables rapid and accurate inspection of the vehicle compartment, thereby effectively improving security inspection efficiency, reducing manual intervention, and effectively lowering the false alarm rate.
[0115] For example, as shown in Figure 6, if there are no items to be detected in the vehicle compartment image, the empty compartment detection device outputs an indication that the vehicle has a low risk.
[0116] For example, as shown in Figure 7, the vehicle compartment image contains an item 71 to be detected, and this item 71 is a whitelisted item. In this case, the empty compartment detection device outputs an indication that the vehicle has a low risk.
[0117] For example, as shown in Figure 8, the vehicle compartment image contains an item 81 to be detected, and this item 81 is not a whitelisted item. In this case, the empty compartment detection device outputs an indication that the vehicle is of high risk. Furthermore, the empty compartment detection device marks the area where the item 81 is located in the perspective image for subsequent security inspection processing.
[0118] Figure 9 is a schematic diagram of the structure of an empty box detection device according to an embodiment of the present disclosure.
[0119] As shown in Figure 9, the empty box detection device 90 is presented in the form of a general-purpose computing device. The empty box detection device 90 includes a memory 91, a processor 92, and a bus 93 connecting different system components.
[0120] The memory 91 may include, for example, system memory, non-volatile storage media, etc. The system memory may store, for example, an operating system, application programs, a boot loader, and other programs. The system memory may include volatile storage media, such as random access memory (RAM) and / or cache memory. The non-volatile storage media may store, for example, instructions for a corresponding embodiment of at least one empty box detection method being executed. Non-volatile storage media include, but are not limited to, disk storage, optical storage, flash memory, etc.
[0121] The processor 92 can be implemented using a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete hardware components such as discrete gates or transistors. Accordingly, each module, such as the acquisition module, the calculation module, and the adjustment module, can be implemented by the central processing unit (CPU) running instructions in the memory to execute the corresponding steps, or by dedicated circuitry to execute the corresponding steps.
[0122] For example, processor 92 is configured to implement the method involved in any of the embodiments shown in Figures 1 to 5 for memory-based instruction execution.
[0123] Bus 93 can use any of the various bus architectures. For example, bus architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MCA) bus, and the Peripheral Component Interconnect (PCI) bus.
[0124] The interfaces 94, 95, and 96 of the empty box detection device 90, as well as the memory 91 and processor 92, can be connected via bus 93. Input / output interface 94 provides a connection interface for input / output devices such as monitors, mice, and keyboards. Network interface 95 provides a connection interface for various networked devices. Storage interface 96 provides a connection interface for external storage devices such as floppy disks, USB flash drives, and SD cards.
[0125] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus, and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations thereof, can be implemented by computer-readable program instructions.
[0126] These computer-readable program instructions are provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable device to produce a machine, such that execution of the instructions by the processor produces means for implementing the functions specified in one or more boxes of the flowchart and / or block diagram.
[0127] These computer-readable program instructions may also be stored in a computer-readable storage medium. These instructions cause a computer to work in a particular manner to produce an article of manufacture, including instructions that implement the functions specified in one or more boxes in a flowchart and / or block diagram.
[0128] This disclosure may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects.
[0129] This disclosure also provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the method involved in any of the embodiments shown in Figures 1 to 5.
[0130] This disclosure also provides a computer program product, including computer instructions, wherein when executed by a processor, the computer instructions implement the method involved in any of the embodiments shown in Figures 1 to 5.
[0131] In some embodiments, the functional units described above may be implemented as general-purpose processors, programmable logic controllers (PLCs), digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or any suitable combination thereof for performing the functions described herein.
[0132] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.
[0133] The description in this disclosure is provided for illustrative and descriptive purposes only and is not intended to be exhaustive or to limit the disclosure to its forms. Many modifications and variations will be apparent to those skilled in the art. The embodiments were chosen and described in order to better illustrate the principles and practical application of this disclosure and to enable those skilled in the art to understand this disclosure and to design various embodiments with various modifications suitable for a particular purpose.
Claims
1. A method for detecting empty boxes, comprising: Collect perspective images of vehicles declared as having empty containers; The perspective image is segmented to obtain an image of the vehicle's cargo box; If the image of the vehicle compartment is found to contain an item to be detected, then the item to be detected is determined to be a suspect. If the item to be detected is not the suspected item, a machine learning model is used to identify whether the item to be detected is a whitelisted item. Based on the recognition results of the machine learning model, risk indication information is output.
2. The empty box detection method according to claim 1, wherein, The step of using a machine learning model to identify whether the item to be detected is a whitelisted item includes: Check if the whitelist master switch is turned on; When the whitelist master switch is in the on state, the whitelist corresponding to the whitelist category switch that is in the on state will be used as the current whitelist. The machine learning model is used to identify whether the item to be detected is included in the current whitelist; If the item to be detected is included in the current whitelist, then the item to be detected is determined to be a whitelisted item.
3. The empty box detection method according to claim 2, wherein, The machine learning model includes a first sub-model, a second sub-model, and a third sub-model. The step of using the machine learning model to identify whether the item to be detected is included in the current whitelist includes: Using the first sub-model, a local image of the area where the object to be detected is located is obtained from the vehicle image; Using the second sub-model, image features of the local image are extracted; Using the third sub-model, it is determined whether the image feature is included in the feature library of the current whitelist. If the image feature is included in the feature library of the current whitelist, then it is determined that the item to be detected is included in the current whitelist.
4. The empty box detection method according to claim 2, wherein, The step of outputting risk indication information based on the identification results of the machine learning model includes: If the machine learning model identifies that the item to be detected is not a whitelisted item, it outputs an indication message to indicate that the vehicle is at high risk.
5. The empty box detection method according to claim 2, wherein, The step of outputting risk indication information based on the identification results of the machine learning model includes: If the machine learning model identifies the item to be detected as a whitelisted item, it outputs an indication that the vehicle has a low risk.
6. The empty box detection method according to claim 2 further includes: When the whitelist master switch is in the off state, an indication message is output to indicate that the vehicle is of high risk.
7. The empty box detection method according to claim 1 further includes: If the item to be detected is the suspected item, an indication message is output to indicate that the vehicle is at high risk.
8. The empty box detection method according to claim 1 further includes: If the detected item is not included in the vehicle compartment image, an indication message is output to indicate that the vehicle has a low risk.
9. The empty box detection method according to claim 1, wherein, The perspective images of vehicles declared as having empty containers include: Obtain the vehicle's declaration information; Determine whether the vehicle is empty based on the declared information; If the vehicle is determined to be empty based on the declared information, the vehicle is scanned with X-rays to obtain the perspective image.
10. The empty box detection method according to claim 4, 6 or 7, further comprising: The area where the object to be detected is located is marked in the perspective image.
11. The empty box detection method according to any one of claims 1-9, further comprising: Training the machine learning model, wherein training the machine learning model includes: Generate multiple sample images of the vehicle compartment; Add annotation information to the items included in each vehicle compartment sample image; The machine learning model is used to process each of the train car sample images to obtain the identification results of the items in each of the train car sample images; The loss function is determined based on the recognition results and the labeling information of the items; The machine learning model is trained based on the loss function.
12. The empty box detection method according to claim 11, wherein, The machine learning model includes one of the following: Faster R-CNN model, YOLO model, Single-Shot Multi-Box Detector (SSD) model, or EfficientDet model.
13. The empty box detection method according to claim 11, wherein, The generation of multiple vehicle compartment sample images includes: Randomly select multiple empty vehicle images from the empty vehicle image library, wherein the empty vehicle image library includes multiple empty vehicle images of different vehicle models; In each of the multiple empty vehicle images, a randomly selected area is used as the target area; Add at least one item image to the target area, wherein the at least one item image includes at least one whitelisted item image and at least one prohibited item image; The size of the at least one item image is randomly adjusted to generate the plurality of vehicle compartment sample images.
14. The empty box detection method according to claim 11, wherein, The step of adding annotation information to the items included in each vehicle compartment sample image includes: The multiple vehicle compartment sample images are divided into a training sample set and a test sample set; Labeling information is added manually to the items included in each training sample in the training sample set; The machine learning model is trained using the aforementioned training sample set; Each test sample in the test sample set is processed using a trained machine learning model to obtain the item recognition result for each test sample. The item identification result of each test sample is used as the labeling information of the items included in each test sample; If the labeling information of the item included in the i-th test sample matches the item included in the i-th test sample, then the labeling information of the item included in the i-th test sample is stored, 1≤i≤N, where N is the total number of test samples in the test sample set.
15. The empty box detection method according to claim 14, wherein, The step of adding annotation information to the items included in each vehicle compartment sample image includes: If the labeling information of the items included in the i-th test sample does not match the items included in the i-th test sample, the labeling information of the items included in the i-th test sample will be manually modified and added. The machine learning model is repeatedly trained using the training sample set and the i-th test sample; The trained machine learning model is used to process the test samples in the test sample set that have not been labeled until the items included in each test sample in the test sample set are labeled.
16. An empty box detection device, comprising: Memory; A processor, coupled to a memory, is configured to execute instructions stored in the memory to implement the empty box detection method as described in any one of claims 1-15.
17. A computer-readable storage medium, wherein, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the empty box detection method as described in any one of claims 1-15.
18. A computer program product comprising computer instructions, wherein the computer instructions, when executed by a processor, implement the empty box detection method as described in any one of claims 1-15.
19. A computer program comprising computer instructions, wherein the computer instructions, when executed by a processor, implement the empty box detection method as described in any one of claims 1-15.