Empty box detection method and device, storage medium and program product

Abstract

The invention provides an empty box detection method and device, a storage medium and a program product. The empty box detection method comprises the following steps: acquiring a perspective image of a vehicle declared as an empty box; segmenting the perspective image to obtain a carriage image of the vehicle; detecting whether the to-be-detected article is a suspected article or not under the condition that the compartment image is identified to comprise the to-be-detected article; under the condition that the to-be-detected article is not the suspected article, identifying whether the to-be-detected article is a white list article or not by utilizing a machine learning model; and outputting risk indication information according to an identification result of the machine learning model.

Description

Technical Field

[0001] The present disclosure relates to the field of safety detection, and in particular to an empty box detection method and device, a storage medium and a program product. Background Art

[0002] In the current globalized context, the security situation of the international community has become increasingly severe, especially in the supervision of import and export activities such as customs and ports. How to ensure the safety and legality of imported and exported goods has become an important challenge facing governments and customs departments of various countries.

[0003] As a non-invasive inspection method, large container / vehicle inspection system has been widely used around the world due to its high efficiency and accuracy. Summary of the invention

[0004] The inventors have noticed that with the continuous development of global trade, the volume of customs clearance at ports and customs has increased year by year, the types of transported goods have become increasingly diverse, and the structure of cargo vehicles has become increasingly complex. These factors make it difficult for traditional security inspection methods to adapt to the current situation.

[0005] Based on this, the present disclosure provides an empty box detection method, which can quickly and accurately check the box of a vehicle, thereby effectively improving security inspection efficiency, reducing manual intervention, and effectively reducing false alarm rates.

[0006] In a first aspect of the present disclosure, there is provided an empty box detection method, comprising: acquiring a perspective image of a vehicle declared as an empty box; segmenting the perspective image to obtain a box image of the vehicle; upon identifying that the box image includes an object to be detected, detecting whether the object to be detected is a suspect; upon identifying that the object to be detected is not a suspect, using a machine learning model to identify whether the object to be detected is a whitelisted object; and outputting risk indication information based on the recognition result of the machine learning model.

[0007] In some embodiments, the use of a machine learning model to identify whether the item to be detected is a whitelist item includes: detecting whether a whitelist master switch is in an on state; when the whitelist master switch is in the on state, using a whitelist corresponding to a whitelist classification switch in the on state as a current whitelist; using the machine learning model to identify whether the item to be detected is included in the current whitelist; and when the item to be detected is included in the current whitelist, determining that the item to be detected is a whitelist item.

[0008] In some embodiments, the machine learning model includes a first sub-model, a second sub-model and a third sub-model, and the use of the machine learning model to identify whether the item to be detected is included in the current whitelist includes: using the first sub-model to obtain a local image of the area where the item to be detected is located from the vehicle compartment image; using the second sub-model to extract image features of the local image; using the third sub-model to detect whether the image features are included in the feature library of the current whitelist, wherein if the image features are included in the feature library of the current whitelist, it is determined that the item to be detected is included in the current whitelist.

[0009] In some embodiments, outputting risk indication information based on the recognition result of the machine learning model includes: when the machine learning model recognizes that the object to be detected is not the whitelist item, outputting indication information for indicating that the vehicle has a high risk.

[0010] In some embodiments, outputting risk indication information based on the recognition result of the machine learning model includes: when the machine learning model recognizes that the object to be detected is the white-list item, outputting indication information for indicating that the vehicle has a low risk.

[0011] In some embodiments, when the whitelist master switch is in an off state, indication information indicating that the vehicle has a high risk is output.

[0012] In some embodiments, when the object to be detected is the suspect, indication information indicating that the vehicle has a high risk is output.

[0013] In some embodiments, when it is identified that the vehicle compartment image does not include the object to be detected, indication information indicating that the vehicle has a low risk is output.

[0014] In some embodiments, the collecting of the perspective image of a vehicle declared as empty includes: obtaining the declaration information of the vehicle; determining whether the vehicle is empty based on the declaration information; and in the case where the vehicle is determined to be empty based on the declaration information, scanning the vehicle using X-rays to obtain the perspective image.

[0015] In some embodiments, the area where the object to be detected is located is marked in the perspective image.

[0016] In some embodiments, the machine learning model is trained, wherein the training of the machine learning model includes: generating multiple vehicle compartment sample images; adding annotation information for items included in each vehicle compartment sample image; processing each vehicle compartment sample image using the machine learning model to obtain recognition results of the items in each vehicle compartment sample image; determining a loss function based on the recognition results and the annotation information of the items; and training the machine learning model based on the loss function.

[0017] In some embodiments, the machine learning model includes one of a Faster R-CNN model, a YOLO model, a Single Shot Multi-Box Detector SSD model, or an EfficientDet model.

[0018] In some embodiments, generating a plurality of vehicle trunk sample images includes: randomly selecting a plurality of empty vehicle trunk images from an empty vehicle trunk library, wherein the empty vehicle trunk library includes empty vehicle trunk images of a plurality of different vehicle models; in each of the plurality of empty vehicle trunk images, taking a randomly selected area as a target area; adding at least one item image to the target area, wherein the at least one item image includes at least one of at least one whitelist item image and at least one illegal item image; and randomly adjusting the size of the at least one item image, thereby generating the plurality of vehicle trunk sample images.

[0019] In some embodiments, adding labeling information to the items included in each of the vehicle compartment sample images includes: dividing the multiple vehicle compartment sample images into a training sample set and a test sample set; manually adding labeling information to the items included in each training sample in the training sample set; using the training sample set to train a machine learning model; using the trained machine learning model to process each test sample in the test sample set to obtain an item recognition result of each test sample; using the item recognition result of each test sample as the labeling information of the items included in each test sample; if the labeling information of the items included in the i-th test sample matches the items included in the i-th test sample, storing the labeling information of the items included in the i-th test sample, , N is the total number of test samples in the test sample set.

[0020] In some embodiments, if the labeling information of the items included in the i-th test sample does not match the items included in the i-th test sample, the items included in the i-th test sample are manually modified to add labeling information; the machine learning model is repeatedly trained using the training sample set and the i-th test sample; and the test samples in the test sample set that do not have labeling information added are processed using the trained machine learning model until labeling information is added to the items included in each test sample in the test sample set.

[0021] According to a second aspect of an embodiment of the present disclosure, there is provided an empty box detection device, comprising: a memory; and a processor, coupled to the memory, the processor being configured to execute an empty box detection method as described in any of the above embodiments based on instructions stored in the memory.

[0022] According to a third aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the method described in any of the above embodiments is implemented.

[0023] According to a fourth aspect of an embodiment of the present disclosure, a computer program product is provided, comprising computer instructions, wherein when the computer instructions are executed by a processor, a method as described in any of the above embodiments is implemented.

[0024] Other features and advantages of the present disclosure will become apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0026] Figure 1 This is a flow chart of an empty box detection method according to an embodiment of the present disclosure;

[0027] Figure 2 A flowchart of a machine learning model training method according to an embodiment of the present disclosure;

[0028] Figure 3 A schematic diagram of a process of generating a vehicle compartment sample image according to an embodiment of the present disclosure;

[0029] Figure 4 A flowchart of a marking method according to an embodiment of the present disclosure;

[0030] Figure 5 This is a flow chart of an empty box detection method according to another embodiment of the present disclosure;

[0031] Figure 6 A schematic diagram of a perspective image of a vehicle according to an embodiment of the present disclosure;

[0032] Figure 7 A schematic diagram of a vehicle perspective image according to another embodiment of the present disclosure;

[0033] Figure 8 A schematic diagram of a vehicle perspective image according to another embodiment of the present disclosure;

[0034] Fig. 9 It is a schematic structural diagram of an empty box detection device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0035] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present disclosure and its application or use. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0036] Unless specifically stated otherwise, the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure.

[0037] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0038] Technologies, methods, and apparatus known to ordinary technicians in the relevant field may not be discussed in detail, but where appropriate, such technologies, methods, and apparatus should be considered part of the authorization specification.

[0039] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0040] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0041] Figure 1The following is a flow chart of an empty box detection method according to an embodiment of the present disclosure. In some embodiments, the following empty box detection method is performed by an empty box detection device, including steps 11-15.

[0042] At step 11, a perspective image of the vehicle declared empty is acquired.

[0043] In some embodiments, the step of acquiring a perspective image of a vehicle declared empty includes the following.

[0044] 1) Obtain the vehicle’s declaration information.

[0045] 2) Determine whether the vehicle is empty based on the declared information.

[0046] 3) When the vehicle is judged to be empty based on the declared information, the vehicle is scanned by X-ray to obtain a perspective image.

[0047] In step 12, the perspective image is segmented to obtain a vehicle compartment image.

[0048] It should be noted here that, considering that prohibited items are usually placed in the trunk of a vehicle, in order to improve processing efficiency, the trunk image of the vehicle is obtained from the perspective image so as to process the trunk image.

[0049] In step 13, when it is identified that the vehicle compartment image includes the object to be detected, it is detected whether the object to be detected is a suspect.

[0050] In some embodiments, a trained recognition model is used to detect whether the object to be detected is a suspect.

[0051] For example, a local image of the area where the object to be detected is located is obtained from the vehicle compartment image, and image features of the local image are extracted. If the image features are included in a predetermined suspicious object feature library, the object to be detected is determined to be a suspicious object.

[0052] For example, based on the texture features and shape features of the local image, it is detected whether the object to be detected is a suspect.

[0053] For example, the recognition models include the Faster R-CNN model.

[0054] In step 14, when the object to be detected is not a suspect, a machine learning model is used to identify whether the object to be detected is a whitelisted object.

[0055] In some embodiments, the machine learning model includes one of a Faster R-CNN model, a YOLO model, an SSD (SingleShot MultiBox Detector) model, or an EfficientDet model.

[0056] In some embodiments, the machine learning model is trained so that the trained machine learning model can identify whether the object to be detected is a whitelist item.

[0057] Figure 2 This is a flowchart of a machine learning model training method according to an embodiment of the present disclosure, including steps 21-25.

[0058] In step 21 , a plurality of vehicle box sample images are generated, wherein each vehicle box sample image includes at least one object image, and the at least one object image includes at least one of at least one whitelist object image and at least one illegal object image.

[0059] It should be noted that the number of whitelist items is small, resulting in a small number of sample images of the carriage. To solve this problem, image synthesis is used here to generate sample images of the carriage.

[0060] Figure 3 This is a flowchart of a method for generating a vehicle compartment sample image according to an embodiment of the present disclosure, including steps 211-214.

[0061] In step 211, a plurality of empty vehicle box images are randomly selected from an empty vehicle box image library, wherein the empty vehicle box image library includes a plurality of empty vehicle box images of different vehicle models.

[0062] In step 212, in each of the plurality of empty box images, a randomly selected area is used as a target area.

[0063] At step 213, at least one item image is added to the target area.

[0064] It should be noted here that the at least one item image includes at least one of at least one whitelist item image and at least one illegal item image.

[0065] In step 214, the size of the at least one object image is randomly adjusted to generate a plurality of vehicle box sample images.

[0066] Through the above processing, at least one whitelist item image and at least one illegal item image can be added to randomly selected areas in the empty trunk images of different vehicle models, and the size of the added item image can be randomly adjusted, thereby effectively increasing the number of trunk sample images to facilitate effective training of the machine learning model.

[0067] In step 22, annotation information is added to the items included in each vehicle box sample image.

[0068] It should be noted that when the amount of sample images of the carriage is large, manually adding annotation information for the items included in each sample image of the carriage will result in a large time cost. In order to solve this problem, a combination of manual annotation and automatic annotation is adopted here to effectively reduce the annotation time.

[0069] Figure 4 It is a flowchart of a marking method according to an embodiment of the present disclosure, including steps 221-228.

[0070] In step 221, a plurality of vehicle box sample images are divided into a training sample set and a test sample set.

[0071] In step 222, labeling information is manually added to the items included in each training sample in the training sample set.

[0072] In step 223, the machine learning model is trained using the training sample set.

[0073] In step 224, each test sample in the test sample set is processed using the trained machine learning model to obtain an object recognition result for each test sample.

[0074] In step 225, the object recognition result of each test sample is used as the labeling information of the object included in each test sample.

[0075] In step 226, it is checked whether the labeling information of the objects in each test sample matches the objects included in the test sample.

[0076] If the labeling information of the object in the i-th test sample matches the object included in the i-th test sample, then step 227 is executed. , N is the total number of test samples in the test sample set; otherwise, execute step 228.

[0077] In step 227, the labeling information of the items included in the i-th test sample is stored.

[0078] In step 228, the labeling information of the items included in the i-th test sample is modified manually.

[0079] Next, the machine learning model is repeatedly trained using the training sample set and the i-th test sample, and the trained machine learning model is used to process the test samples in the test sample set that have no labeled information added, until the items included in each test sample in the test sample set are labeled.

[0080] Through the above processing, the sample labeling time can be effectively reduced.

[0081] In step 23, each vehicle box sample image is processed using a machine learning model to obtain a recognition result of the object in each vehicle box sample image.

[0082] In step 24, a loss function is determined based on the recognition result and the labeling information of the object.

[0083] In step 25, the machine learning model is trained according to the loss function.

[0084] In some embodiments, the step of using a machine learning model to identify whether the object to be detected is a whitelisted object includes the following:

[0085] 1) Check whether the whitelist master switch is on.

[0086] 2) When the whitelist master switch is in the on state, the whitelist corresponding to the whitelist classification switch in the on state is used as the current whitelist.

[0087] It should be noted here that the whitelist will be different in different scenarios. Therefore, multiple whitelists and multiple whitelist classification switches corresponding to the multiple whitelists can be set as needed. If a whitelist classification switch is turned on, it means that the whitelist corresponding to the whitelist classification switch is currently in use.

[0088] In some embodiments, when the whitelist master switch is in the off state, indication information indicating that the vehicle has a high risk is output.

[0089] 3) Use machine learning models to identify whether the item to be detected is included in the current whitelist.

[0090] 4) When the object to be detected is included in the current whitelist, determining that the object to be detected is a whitelist item.

[0091] In some embodiments, the machine learning model includes a first sub-model, a second sub-model, and a third sub-model. By using the first sub-model, a local image of the area where the object to be detected is located is obtained from the vehicle compartment image. By using the second sub-model, image features of the local image are extracted. By using the third sub-model, it is detected whether the image feature is included in the feature library of the current whitelist, wherein if the image feature is included in the feature library of the current whitelist, it is determined that the object to be detected is included in the current whitelist.

[0092] For example, machine learning models include ResNet (Residual Network) and RPN (Region Proposal Network). The ResNet network is used to extract feature maps of different scales. The RPN network fuses the feature maps of different scales output by the ResNet structure to generate a multi-scale feature pyramid, and then generates candidate regions through a series of convolutional layers. The RoI pooling (Region of interest pooling) layer extracts the features of the candidate regions. Finally, the category labels and bounding box coordinates of the candidate regions are obtained by using the classification layer and regression layer.

[0093] In step 15, risk indication information is output based on the recognition result of the machine learning model.

[0094] In some embodiments, when the machine learning model identifies that the object to be detected is not a whitelisted object, an indication information indicating that the vehicle has a high risk is output. When the machine learning model identifies that the object to be detected is a whitelisted object, an indication information indicating that the vehicle has a low risk is output.

[0095] In some embodiments, when the object to be detected is a suspect, indication information indicating that the vehicle has a high risk is output.

[0096] In some embodiments, when it is identified that the vehicle compartment image does not include any object to be detected, indication information indicating that the vehicle has a low risk is output.

[0097] In some embodiments, when the indication information indicating that the vehicle has a high risk is output, the area where the object to be detected is located is marked in the perspective image so that the staff can know the area where the risky objects are located in time.

[0098] Figure 5 This is a flowchart of an empty box detection method according to another embodiment of the present disclosure, including steps 51-55.

[0099] At step 51 , a perspective image of a vehicle declared empty is acquired.

[0100] In step 52, the perspective image is segmented to obtain a vehicle compartment image.

[0101] In step 53, it is identified whether the vehicle box image includes the object to be detected.

[0102] If the vehicle box image does not include the object to be detected, step 54 is executed. Otherwise, step 55 is executed.

[0103] At step 54 , indication information indicating that the vehicle has a low risk is output.

[0104] In step 55, it is detected whether the object to be detected is a suspect object.

[0105] If the object to be detected is a suspect, execute step 56. Otherwise, execute step 57.

[0106] At step 56 , indication information indicating that the vehicle has a high risk is output.

[0107] In some embodiments, the area where the object to be detected is located is also marked in the perspective image.

[0108] In step 57, when the whitelist master switch is in the on state, the whitelist corresponding to the whitelist classification switch in the on state is used as the current whitelist, and the machine learning model is used to identify whether the object to be detected is included in the current whitelist.

[0109] If the object to be detected is included in the current white list, step 54 is executed. Otherwise, step 56 is executed.

[0110] In some embodiments, when the whitelist master switch is in the off state, indication information indicating that the vehicle has a high risk is output.

[0111] In the empty box detection method provided in the above embodiment of the present disclosure, in addition to detecting whether the item to be detected in the box is a suspect, the machine learning model is further used to identify whether the item to be detected is a whitelist item, and then the risk indication information is output according to the identification result of the machine learning model. In this way, the box of the vehicle can be inspected quickly and accurately, thereby effectively improving the security inspection efficiency, reducing manual intervention, and effectively reducing the false alarm rate.

[0112] For example, Figure 6 As shown, there is no object to be detected in the vehicle box image. In this case, the empty box detection device outputs indication information indicating that the vehicle has a low risk.

[0113] For example, Figure 7 As shown, there is an object 71 to be detected in the vehicle box image, and the object 71 to be detected is a whitelist object. In this case, the empty box detection device outputs indication information for indicating that the vehicle has a low risk.

[0114] For example, Figure 8 As shown, there is an object 81 to be detected in the vehicle box image, and the object 81 to be detected is not a whitelisted item. In this case, the empty box detection device outputs indication information indicating that the vehicle has a high risk. In addition, the empty box detection device marks the area where the object 81 to be detected is located in the perspective image for subsequent security inspection.

[0115] Fig. 9 It is a schematic structural diagram of an empty box detection device according to an embodiment of the present disclosure.

[0116] like Fig. 9 As shown, the empty box detection device 90 is in the form of a general-purpose computing device. The empty box detection device 90 includes a memory 91, a processor 92, and a bus 93 connecting different system components.

[0117] The memory 91 may include, for example, a system memory, a non-volatile storage medium, etc. The system memory may store, for example, an operating system, an application program, a boot loader, and other programs. The system memory may include a volatile storage medium, such as a random access memory (RAM) and / or a cache memory. The non-volatile storage medium may store, for example, instructions of a corresponding embodiment of at least one empty box detection method being executed. The non-volatile storage medium includes, but is not limited to, a disk memory, an optical memory, a flash memory, etc.

[0118] The processor 92 may be implemented by a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistors, and other discrete hardware components. Accordingly, each module such as an acquisition module, a calculation module, and an adjustment module may be implemented by a central processing unit (CPU) running instructions in a memory that execute corresponding steps, or may be implemented by a dedicated circuit that executes corresponding steps.

[0119] For example, the processor 92 is configured to execute instructions stored in the memory to implement the following Figures 1 to 5 The method of any one of the embodiments.

[0120] The bus 93 may use any of a variety of bus structures, including, but not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, and a Peripheral Component Interconnect (PCI) bus.

[0121] The interfaces 94, 95, 96 of the empty box detection device 90, the memory 91 and the processor 92 can be connected via a bus 93. The input / output interface 94 can provide a connection interface for input / output devices such as a display, a mouse, and a keyboard. The network interface 95 provides a connection interface for various networked devices. The storage interface 96 provides a connection interface for external storage devices such as a floppy disk, a USB flash drive, and an SD card.

[0122] Here, various aspects of the present disclosure are described with reference to flowcharts and / or block diagrams of methods, devices, and computer program products according to embodiments of the present disclosure. It should be understood that each frame of the flowchart and / or block diagram and the combination of frames can be implemented by computer-readable program instructions.

[0123] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable device to produce a machine, so that the processor executes the instructions to produce means for implementing the functions specified in one or more blocks in the flowchart and / or block diagram.

[0124] These computer-readable program instructions may also be stored in a computer-readable memory, which cause the computer to work in a specific manner to produce an article of manufacture, including instructions for implementing the functions specified in one or more blocks in the flowchart and / or block diagram.

[0125] The present disclosure can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects.

[0126] The present disclosure also provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the following is implemented: Figures 1 to 5 The method of any one of the embodiments.

[0127] The present disclosure also provides a computer program product, including computer instructions, wherein when the computer instructions are executed by a processor, the following is achieved: Figures 1 to 5 The method of any one of the embodiments.

[0128] In some embodiments, the functional unit described above may be implemented as a general-purpose processor, a programmable logic controller (PLC), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or any appropriate combination thereof, for performing the functions described in the present disclosure.

[0129] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0130] The description of the present disclosure is given for the purpose of illustration and description, and is not intended to be exhaustive or to limit the present disclosure to the disclosed form. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiments are selected and described in order to better illustrate the principles and practical applications of the present disclosure, and to enable those of ordinary skill in the art to understand the present disclosure and thereby design various embodiments with various modifications suitable for specific uses.

Claims

1. An empty box detection method, comprising: Capture perspective images of vehicles declared empty; Segmenting the perspective image to obtain a compartment image of the vehicle; In the case where it is identified that the vehicle compartment image includes an object to be detected, detecting whether the object to be detected is a suspect; In the case where the object to be detected is not the suspected object, using a machine learning model to identify whether the object to be detected is a whitelisted object; Based on the recognition results of the machine learning model, risk indication information is output.

2. The method according to claim 1, wherein: The method of using a machine learning model to identify whether the object to be detected is a whitelisted object includes: Check whether the whitelist master switch is turned on; When the whitelist master switch is in the on state, the whitelist corresponding to the whitelist classification switch in the on state is used as the current whitelist; Using the machine learning model to identify whether the object to be detected is included in the current whitelist; In the case that the object to be detected is included in the current whitelist, it is determined that the object to be detected is an object in the whitelist.

3. The method according to claim 2, wherein: The machine learning model includes a first sub-model, a second sub-model and a third sub-model, and using the machine learning model to identify whether the object to be detected is included in the current whitelist includes: Using the first sub-model, a local image of the area where the object to be detected is located is obtained from the vehicle compartment image; extracting image features of the local image using the second sub-model; The third sub-model is used to detect whether the image feature is included in the feature library of the current whitelist, wherein if the image feature is included in the feature library of the current whitelist, it is determined that the object to be detected is included in the current whitelist.

4. The method according to claim 2, wherein: Outputting risk indication information according to the recognition result of the machine learning model includes: When the machine learning model identifies that the object to be detected is not an object in the whitelist, output indication information indicating that the vehicle has a high risk.

5. The method according to claim 2, wherein: Outputting risk indication information according to the recognition result of the machine learning model includes: In the case where the machine learning model recognizes that the object to be detected is the whitelist object, output indication information indicating that the vehicle has a low risk.

6. The method according to claim 2, further comprising: When the whitelist master switch is in the off state, indication information for indicating that the vehicle has a high risk is output.

7. The method according to claim 1, further comprising: In the case where the object to be detected is the suspected object, indication information for indicating that the vehicle has a high risk is output.

8. The method according to claim 1, further comprising: In the case where it is identified that the vehicle compartment image does not include the object to be detected, indication information for indicating that the vehicle has a low risk is output.

9. The method according to claim 1, wherein: The collecting of the perspective image of the vehicle declared as empty includes: Obtaining declaration information of the vehicle; Determining whether the vehicle is an empty box according to the declared information; When it is determined that the vehicle is empty according to the declared information, the vehicle is scanned by X-ray to obtain the perspective image.

10. The method according to claim 4, 6 or 7, further comprising: The area where the object to be detected is located is marked in the perspective image.

11. The method according to any one of claims 1 to 9, further comprising: Training the machine learning model, wherein the training the machine learning model comprises: Generate multiple vehicle box sample images; Add annotation information to the items included in each car box sample image; Processing each of the vehicle box sample images using the machine learning model to obtain a recognition result of the object in each of the vehicle box sample images; Determining a loss function according to the recognition result and the labeling information of the object; The machine learning model is trained according to the loss function.

12. The method according to claim 11, wherein: The machine learning model includes one of a Faster R-CNN model, a YOLO model, a single shot multi-box detector SSD model, or an EfficientDet model.

13. The method according to claim 11, wherein: Generating a plurality of vehicle box sample images comprises: Randomly selecting a plurality of empty car box images from an empty car box image library, wherein the empty car box image library includes a plurality of empty car box images of different car models; In each of the plurality of empty box images, a randomly selected area is used as a target area; adding at least one item image to the target area, the at least one item image comprising at least one of at least one whitelist item image and at least one violation item image; The size of the at least one object image is randomly adjusted to generate the plurality of vehicle box sample images.

14. The method according to claim 11, wherein: The adding of annotation information for the items included in each of the vehicle box sample images comprises: Dividing the plurality of vehicle box sample images into a training sample set and a test sample set; manually adding annotation information to the items included in each training sample in the training sample set; Using the training sample set to train a machine learning model; Using the trained machine learning model to process each test sample in the test sample set to obtain an object recognition result of each test sample; Using the object recognition result of each test sample as the labeling information of the object included in each test sample; If the labeling information of the item included in the i-th test sample matches the item included in the i-th test sample, the labeling information of the item included in the i-th test sample is stored, , N is the total number of test samples in the test sample set.

15. The method according to claim 14, wherein: The adding of annotation information for the items included in each of the vehicle box sample images comprises: If the labeling information of the items included in the i-th test sample does not match the items included in the i-th test sample, manually modifying the items included in the i-th test sample to add labeling information; Repeatedly training the machine learning model using the training sample set and the i-th test sample; The trained machine learning model is used to process the test samples in the test sample set that have not been labeled until the items included in each test sample in the test sample set are labeled.

16. An empty box detection device, comprising: Memory; A processor is coupled to the memory, and the processor is configured to execute the method according to any one of claims 1 to 15 based on instructions stored in the memory.

17. A computer-readable storage medium, wherein: The computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the method according to any one of claims 1 to 15 is implemented.

18. A computer program product, comprising computer instructions, wherein when the computer instructions are executed by a processor, the method according to any one of claims 1 to 15 is implemented.

Images