Contraband carrying detection method, device, system and storage medium

Abstract

The application provides a contraband carrying detection method, device and system and a storage medium, wherein the contraband carrying detection method comprises the following steps: inputting an initial image of a person to be inspected into a target detection model to obtain a 3D doll image and a standing posture information of the person to be inspected; the target detection model is used for 3D reorganization and human key point detection on a person image in the initial image; and according to the 3D doll image and the standing posture information of the person to be inspected, predicted contraband information is drawn on the 3D doll image to obtain a contraband detection image. Through a double-branch network of instance segmentation and key point detection, not only the algorithm time consumption is reduced, the detection efficiency is provided, but also suspicious contraband is mapped on the doll image in the same proportion, the missed detection and false alarm rate are reduced, and the detection accuracy is improved. Meanwhile, there is no requirement for the standing posture of the person to be inspected, that is, any standing posture can be mapped on the doll image, and the comfort of the person to be inspected is improved.

Description

Technical Field

[0001] This application relates to the field of security inspection technology, specifically to a method, device, system, and storage medium for detecting prohibited items. Background Technology

[0002] Millimeter waves have strong penetrating power, capable of penetrating ordinary clothing, textiles, and packaging paper. They are characterized by high resolution, good directivity, and strong anti-interference ability, making them suitable for security inspection processes.

[0003] Currently, millimeter-wave body imaging security scanners use active millimeter-wave synthetic aperture radar imaging technology to scan individuals and detect various prohibited items concealed under clothing or on the body surface. Existing security technologies require specific postures from individuals during the inspection process; they also require them to turn around and cooperate. Summary of the Invention

[0004] In view of this, embodiments of this specification provide a method, apparatus, system, and storage medium for detecting prohibited items, which are applied to the security inspection process of prohibited items.

[0005] The embodiments in this specification provide the following technical solutions:

[0006] This specification provides an embodiment of a method for detecting the carrying of prohibited items, the method comprising:

[0007] The initial image of the person to be inspected is input into the target detection model to obtain a 3D doll image and the person's posture information; the target detection model is used to perform 3D reconstruction and human key point detection on the person image in the initial image;

[0008] Based on the 3D mascot image and the standing posture information of the person being inspected, the predicted contraband information is plotted on the mascot image to obtain a contraband detection image.

[0009] This specification also provides an embodiment of a prohibited item carrying detection device, the prohibited item carrying detection device comprising:

[0010] The acquisition module is used to input the initial image of the person to be inspected into the target detection model to obtain a 3D doll image and the standing posture information of the person to be inspected; the target detection model is used to perform 3D reconstruction and human key point detection on the human image in the initial image;

[0011] The acquisition module is used to draw the predicted contraband information onto the 3D mascot image and the standing posture information of the person being inspected to obtain a contraband detection image.

[0012] This specification also provides a system for detecting the carrying of prohibited items, including a memory, a processor, and a computer program. The computer program is stored in the memory, and the processor runs the computer program as described in any of the technical solutions of this specification for detecting the carrying of prohibited items.

[0013] This specification also provides a readable storage medium storing a computer program, which, when executed by a processor, is used to implement the method for detecting the carrying of contraband as described in any of the technical solutions in this specification.

[0014] Compared with the prior art, the beneficial effects that at least one technical solution adopted in the embodiments of this specification can achieve include at least:

[0015] By employing a dual-branch network of instance segmentation and keypoint detection, suspicious contraband information can be mapped onto mannequin images proportionally, achieving 100% mapping accuracy. This reduces false positives and false negatives, lowers algorithm latency, and improves detection performance and efficiency. Furthermore, the algorithm is easy to maintain, reducing maintenance costs. It eliminates requirements on the subject's posture, allowing mannequin mapping in any position, avoiding mismatches between the human body and the mannequin, thus improving equipment efficiency and subject comfort. It also more effectively protects subject privacy and enhances their comfort. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart of a method for detecting the carrying of prohibited items as described in this application;

[0018] Figure 2 This is a flowchart of another method for detecting prohibited items in this application;

[0019] Figure 3 This is a schematic diagram of a prohibited item carrying detection device according to this application;

[0020] Figure 4 This is a schematic diagram of a prohibited item carrying detection system according to this application. Detailed Implementation

[0021] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0022] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. This application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this application, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number and aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.

[0024] It should also be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. The drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0025] Additionally, specific details are provided in the following description to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that practice can be carried out without these specific details.

[0026] Current millimeter-wave human body imaging security checks have specific requirements regarding the posture of the person being inspected; a particular posture must be maintained for the detection of contraband. Additionally, a pre-defined mannequin image needs to be prepared, key points manually annotated, and coordinate transformation performed between the human and mannequin key points to identify suspicious contraband on the mannequin image. If the mannequin image has scaling or translation errors, the detection accuracy decreases. Furthermore, modifications such as adding or deleting key points on the mannequin or human make algorithm maintenance cumbersome and complex, preventing the achievement of more efficient contraband detection.

[0027] Based on this, the embodiments of this specification propose a novel scheme for detecting contraband: A new detection model is established, employing a dual-branch network of instance segmentation and keypoint detection. On one hand, the human torso is segmented at the pixel level, and the torso contour and human silhouette background are filled. Combined with facial features, a 3D dummy image is generated. On the other hand, keypoint detection of the human skeleton is performed. During the 3D dummy image reconstruction process, based on the human posture information obtained from the human skeleton keypoints, the predicted contraband can be proportionally mapped onto the reconstructed 3D dummy image to obtain a contraband detection image. This not only reduces algorithm time and improves detection efficiency but also proportionally maps suspicious contraband onto the dummy image, reducing false positives and false negatives, and improving detection accuracy. Furthermore, it does not require specific posture from the person being inspected; any posture can be mapped onto the dummy image, improving the comfort of the person being inspected.

[0028] The technical solutions provided by the various embodiments of this application are described below with reference to the accompanying drawings.

[0029] like Figure 1 As shown in the embodiment of this specification, a method for detecting contraband is provided, including step S110: inputting an initial image of the person to be inspected into a target detection model to obtain a 3D dummy image and the person's posture information; the target detection model is used to perform 3D reconstruction and human key point detection on the person image in the initial image. Step S120: based on the 3D dummy image and the person's posture information, plotting the predicted contraband information on the dummy image to obtain a contraband detection image.

[0030] Specifically, in step S110, an initial image of the person being inspected is acquired. This initial image is then input into a trained object detection model. The model performs 3D reconstruction of the person image based on the initial image to obtain a 3D dummy image. It also detects key human body points based on the initial image, including facial key points and skeletal key points. These key points are used to simulate facial features to obtain the 3D dummy image and to obtain posture information based on the skeletal key points. Finally, based on the 3D dummy image and posture information, the predicted contraband information is plotted on the dummy image to obtain a contraband detection image. The posture information may include the current posture information and predicted posture information obtained from the current posture information. This predicted posture information is used to accurately determine the specific location of the contraband information on the 3D dummy image, thereby obtaining a more accurate contraband detection image. The embodiments in this specification achieve contraband detection by setting a 3D human body model and a human body key point detection model in the target detection model. Contraband can be mapped onto the 3D dummy image in proportion, eliminating the need for the inspected person to constantly change their standing posture and eliminating the need for complex security inspection algorithms, thereby improving the accuracy of security inspection and reducing the rate of missed detections and false alarms.

[0031] Step S120: Based on the 3D mascot image and the standing posture information of the person being inspected, the predicted contraband information is drawn on the 3D mascot image to obtain the contraband detection image.

[0032] In conjunction with the above embodiments, based on the 3D mannequin image and the posture information of the person being inspected, information on suspected contraband is mapped proportionally onto the 3D mannequin image to obtain a contraband detection image. In some embodiments, if the similarity between the contraband detection image and the initial image is as high as a preset threshold, the location of the contraband on the person being inspected is determined.

[0033] In some embodiments, the target detection model includes: a 3D puppet composition branch and a human key point detection branch; the 3D puppet composition branch is used to perform pixel-level segmentation on the initial image to obtain a segmented image; and to fill the segmented image according to the human torso contour and the human contour background to obtain a composition image; the human key point detection branch is used to obtain the facial key points and human skeletal key points of the person being inspected according to the initial image, and is also used to integrate the facial features formed by the facial key points with the composition image to generate a 3D puppet image, and to obtain human posture information according to the human skeletal key points.

[0034] Specifically, the target detection model in the embodiments of this specification mainly uses a 3D puppet composition branch and a human key point detection branch to obtain 3D puppet images and human posture information.

[0035] Specifically, during the initial image instance segmentation process, the 3D doll component branch uses an AI recognition model to detect whether the person being inspected in the initial image is carrying suspicious contraband through the instance segmentation branch network. Simultaneously, pixel-level segmentation of the human torso is performed to obtain a segmented image. Some embodiments can also extract the closed-loop contour information of the human torso, fill the segmented image with the background and the human contour's base color, and generate a new image with the same aspect ratio. This new image serves as the base image for the doll image corresponding to the current person being inspected, thus generating the component image.

[0036] Simultaneously, the human keypoint detection branch obtains facial and skeletal keypoints of the person being inspected from the initial image. It accurately locates the person's position on the base image (i.e., their position on the constituent images) using these facial keypoints. It then reconstructs the facial information of the base image using a virtual dummy and performs background 3D simulation processing on the base image based on the 3D human body information acquired by the millimeter-wave human body imaging security scanner, generating a virtual 3D dummy image. The human keypoint detection branch also detects skeletal keypoint information during the image segmentation process, obtaining posture information such as whether the current posture meets a preset posture. If not, it determines whether the body will sway; if it meets the swaying criteria, it predicts the body's posture after a period of time, thus achieving more accurate detection.

[0037] In some embodiments, obtaining human posture information based on human skeletal key point information includes: combining facial features and obtaining several predicted human posture information on a 3D dummy image based on human skeletal key points.

[0038] Specifically, in obtaining human standing posture information, it is necessary to combine facial features and obtain several human standing posture information on a 3D dummy image based on key points of the human skeleton. In some embodiments, combining facial features, it is determined whether the current standing posture meets a preset standing posture on the 3D dummy image based on key points of the human skeleton. If it does, multiple current standing postures can be obtained. If the current standing posture does not meet the preset standing posture, it is combined with facial features and predicted on the 3D dummy image based on key points of the human skeleton and human swaying standards for a period of time (e.g., within a few milliseconds). This enables more accurate and efficient detection of contraband, etc.

[0039] In some embodiments, drawing the predicted contraband information onto a 3D dummy image to obtain a contraband detection image includes: drawing the predicted contraband information based on the predicted human posture information and the 3D dummy image to obtain a contraband detection image.

[0040] In some embodiments, the predicted contraband information can be accurately plotted to obtain a contraband detection image based on the current standing posture of the person being inspected and a 3D mannequin image. In other embodiments, if the person being inspected meets the criteria for body swaying, the predicted contraband information is plotted to obtain a contraband detection image based on the predicted human standing posture information after a period of time combined with the 3D mannequin image. Alternatively, if the person being inspected reports an error, a 3D mannequin image is further generated, and the predicted contraband information is plotted to obtain a contraband detection image. Compared to existing technologies that simply use the current standing posture to obtain a contraband detection image, this embodiment further ensures the accuracy of the plotted contraband detection image by proportionally mapping the predicted human standing posture information to the mannequin image, thus reducing the false alarm rate of missed detections. Furthermore, it achieves accurate contraband detection images without any requirements on the person being inspected's standing posture, improving work efficiency and the comfort of the person being inspected.

[0041] In some embodiments, the target detection model further includes a contraband instance branch and a human torso instance branch. The contraband instance branch is used to plot the predicted contraband information onto a 3D mannequin image. The human torso instance branch is used to fill the segmented image with the human torso outline and the human outline background to obtain a composite image.

[0042] Combination Figure 2In this embodiment, the 3D doll composition branch in the target detection model is used to perform pixel-level segmentation on the initial image to obtain a segmented image; the human torso instance branch fills the segmented image based on the human torso contour and the human contour background to obtain a composition image. The human key point detection branch is used to obtain the facial key points and human skeletal key points of the person being inspected based on the initial image, and is also used to integrate the facial features formed by the facial key points with the composition image to generate a 3D doll image. Finally, the contraband instance branch is used to obtain the human posture information based on the human skeletal key points.

[0043] This embodiment can reduce algorithm processing time, improve detection performance and work efficiency; and the subsequent algorithm maintenance is quite simple, reducing maintenance costs.

[0044] In some embodiments, the method for detecting contraband carrying further includes: constructing a 3D mannequin-based branch and a human keypoint detection branch in the initial detection model; and training the initial detection model using historical contraband carrying data to obtain a target detection model. In other embodiments, the method for detecting contraband carrying further includes: constructing a 3D mannequin-based branch, a human keypoint detection branch, a contraband instance branch, and a human torso instance branch in the initial detection model; and training the initial detection model using historical contraband carrying data to obtain a target detection model.

[0045] In some embodiments, multiple contraband detection images are compared with an initial image to obtain corresponding similarity scores. If each similarity score meets a preset threshold, the contraband information is determined.

[0046] Specifically, each image of a contraband item is obtained and compared with an initial image to determine its similarity. If each similarity score meets a preset threshold (e.g., 99.99%), the contraband information is confirmed. This process can then assist staff in obtaining contraband information.

[0047] like Figure 3 As shown, an embodiment of this specification includes a prohibited item carrying detection device 30 comprising:

[0048] The acquisition module 31 is used to input the initial image of the person to be inspected into the target detection model to obtain a 3D doll image and the standing posture information of the person to be inspected; the target detection model is used to perform 3D reconstruction and human key point detection on the human image in the initial image;

[0049] The acquisition module 32 is used to draw the predicted contraband information onto the 3D mascot image based on the 3D mascot image and the standing posture information of the person being inspected, so as to obtain a contraband detection image.

[0050] Figure 3 The apparatus of the illustrated embodiment can be used to perform corresponding actions. Figure 1The steps in the method embodiments shown are implemented in a similar manner and have similar technical effects, and will not be repeated here.

[0051] Figure 4 This is a schematic diagram of the structure of a prohibited item carrying detection system provided in the embodiments of this specification, such as... Figure 4 As shown, the system 40 includes: a processor 41, a memory 42, and a computer program; wherein

[0052] The memory 42 is used to store the computer program, and the memory may also be flash memory. The computer program is, for example, an application program or functional module that implements the above method.

[0053] The processor 41 is configured to execute the computer program stored in the memory to implement the various steps performed by the device in the above method. For details, please refer to the relevant descriptions in the preceding method embodiments.

[0054] Alternatively, the memory 42 can be either standalone or integrated with the processor 41.

[0055] When the memory 42 is a device independent of the processor 41, the device may further include:

[0056] Bus 43 is used to connect the memory 42 and the processor 41.

[0057] The present invention also provides a readable storage medium storing a computer program, which, when executed by a processor, is used to implement the methods provided in the various embodiments described above.

[0058] The readable storage medium can be a computer storage medium or a communication medium. A communication medium includes any medium that facilitates the transfer of computer programs from one location to another. A computer storage medium can be any available medium accessible to a general-purpose or special-purpose computer. For example, a readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application-Specific Integrated Circuit (ASIC). Alternatively, the ASIC can be located in a user equipment. Of course, the processor and the readable storage medium can also exist as discrete components in a communication device. The readable storage medium can be a read-only memory (ROM), random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0059] The present invention also provides a program product including executable instructions stored in a readable storage medium. At least one processor of the device can read the executable instructions from the readable storage medium, and the at least one processor executes the executable instructions to cause the device to implement the methods provided in the various embodiments described above.

[0060] In the embodiments of the above-described device, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly manifested as execution by a hardware processor, or execution by a combination of hardware and software modules within the processor.

[0061] In this specification, similar or identical parts among the various embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the product embodiments described later are relatively simple in description since they correspond to the methods, and relevant parts can be referred to the descriptions in the system embodiments.

[0062] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for detecting the carrying of prohibited items, characterized in that, The methods for detecting prohibited items include: The initial image of the person to be inspected is input into the target detection model to obtain a 3D doll image and the person's posture information; the target detection model is used to perform 3D reconstruction and human key point detection on the person image in the initial image; The target detection model includes: a 3D human figure composition branch and a human key point detection branch; The 3D dummy component branch is used to perform pixel-level segmentation on the initial image to obtain a segmented image; and to fill the segmented image according to the human torso outline and the human outline background to obtain a component image; The human body key point detection branch is used to obtain the facial key points and human skeletal key points of the person being inspected based on the initial image. It is also used to integrate the facial features formed by the facial key points with the constituent image to generate a 3D dummy image, and to obtain human body posture information based on the human skeletal key points. Based on the 3D mascot image and the standing posture information of the person being inspected, the predicted contraband information is plotted on the 3D mascot image to obtain a contraband detection image.

2. The method for detecting prohibited items according to claim 1, characterized in that, The process of obtaining human standing posture information based on key points of the human skeleton includes: By combining facial features and based on key points of the human skeleton, several predicted human posture information are obtained from the 3D dummy image.

3. The method for detecting prohibited items according to claim 2, characterized in that, The contraband detection image is obtained by plotting the predicted contraband information onto a 3D dummy image, including: Based on the predicted human posture information and 3D dummy image, the predicted contraband information is drawn to obtain the contraband detection image.

4. The method for detecting prohibited items according to claim 1, characterized in that, The target detection model further includes: a contraband instance branch and a human torso instance branch. The contraband instance branch is used to draw the predicted contraband information onto a 3D human figure image. The human torso instance branch is used to fill the segmented image with the human torso outline and the human outline background to obtain the composite image.

5. The method for detecting prohibited items according to claim 1, characterized in that, The method for detecting prohibited items also includes: constructing a 3D puppet composition branch and a human key point detection branch in the initial detection model; The initial detection model is trained using historical data on contraband carrying to obtain a target detection model.

6. The method for detecting prohibited items according to claim 1, characterized in that, The method further includes: Multiple images of contraband detected are compared with an initial image to obtain the corresponding similarity. If each similarity meets a preset threshold, the contraband information is determined.

7. A device for detecting prohibited items, characterized in that, The prohibited item detection device includes: The acquisition module is used to input the initial image of the person to be inspected into the target detection model to obtain a 3D doll image and the standing posture information of the person to be inspected; the target detection model is used to perform 3D reconstruction and human key point detection on the human image in the initial image; The acquisition module is used to draw the predicted contraband information onto the 3D mascot image and the standing posture information of the person being inspected to obtain a contraband detection image.

8. A system for detecting prohibited items, characterized in that, It includes a memory, a processor, and a computer program, the computer program being stored in the memory, and the processor running the computer program to perform the method for detecting the carrying of contraband as described in any one of claims 1-6.

9. A readable storage medium, characterized in that, The readable storage medium stores a computer program, which, when executed by a processor, is used to implement the method for detecting the carrying of contraband as described in any one of claims 1-6.

Images