Unattended millimeter wave human body imaging safety inspection equipment and machine vision algorithm
Through unmanned millimeter-wave human body imaging safety inspection equipment, combined with machine vision technology and Ka frequency band line array, automated detection and automated guidance are realized, solving the problems of large space, high cost and poor resolution of existing equipment. , reduces operating costs and improves efficiency.
Patent Information
- Application Number
- CN202411322031.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-23
- Publication Date
- 2024-11-26
AI Technical Summary
The existing human security equipment has problems such as large space, high cost and poor resolution. At the same time, it requires the operator to be on duty, which reduces the extensive deployment and efficiency of the equipment.
Adopt unattended millimeter-wave human imaging safety inspection equipment, combined with machine vision technology and a variety of strong reminder methods, and achieves one-time coverage of inspected personnel through Ka frequency band line array and machine vision algorithm, automatic detection and guidance, and reduce equipment costs. and take up space.
It realizes unattended full-process human safety inspection, reduces operating costs, avoids the dependence of on-site operators, and balances the inspection coverage and equipment size, and distinguishes performance and equipment costs.
Smart Images

Figure CN119024451A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of security inspection equipment, and in particular to an unattended millimeter wave human body imaging security inspection equipment, a machine vision algorithm and a detection method. Background Art
[0002] Among various human body security inspection methods, millimeter wave imaging technology provides a new path that is non-ionizing, can detect both metals and non-metals, has higher resolution and faster inspection speed.
[0003] Chinese patent CN 113009588 B "Human body security inspection equipment based on millimeter wave imaging" discloses a human body security inspection equipment that can detect two people at a time. A pair of "back-to-back" millimeter wave imaging devices are fixed on both sides of the frame and driven by a common transmission device, and two detection positions are placed in front of the millimeter wave imaging device. The millimeter wave imaging device only needs to be run once to simultaneously inspect the two detection positions. The beneficial effect of reducing the space occupied by the equipment and the cost has been achieved. In this device, although two inspected persons can be inspected at the same time each time, each inspection can only cover half of the inspection area of each inspected person. This requires the inspected person to turn around and then conduct a second inspection for a complete inspection. This reduces the inspection efficiency, makes "two people inspecting together" meaningless, and increases the discomfort during the security inspection process. Chinese patent CN 115951421 B "A channel-type terahertz active imaging human body security inspection instrument" discloses a channel-type human body security inspection equipment based on a 220GHz focal plane camera with a sensitivity of the picowatt level. By properly placing the terahertz double-headed waveguide antenna, focusing lens and reflector, a detection module with a minimum side length of the photosensitive surface greater than or equal to the maximum side length of the imaging area of the imaging objective lens is realized. The inspected person can be fully covered by one active radiation. The beneficial effect of achieving real-time imaging while meeting high-precision imaging is achieved, which is suitable for large-flow security inspection scenes. Although this device can complete the security inspection process at one time, its channel-type placement and terahertz working frequency band cause this device to have the problems of large space occupation and high cost. Chinese patent CN 117930375 B "Channel-type terahertz human body security inspection system integrating multi-dimensional detection technology" discloses a channel-type human body security inspection device composed of two terahertz imaging hosts, two sets of reflectors and optical cameras. By properly designing the host and the reflector, full coverage of the inspected person is also achieved, and the beneficial effect of real-time imaging is achieved. However, this device adopts passive terahertz technology, which significantly reduces its resolution performance. Similarly, its channel-type placement and terahertz working frequency band cause this device to still have the problems of large space occupation and high cost.
[0004] The academic paper "Fully Electronic E-Band Personnel Imager of 2m2ApertureBased on a Multistatic Architecture." IEEE Transactions on Microwave Theory and Techniques, 2012, 61(1): 651-657. Author: Ahmed.SS, et al. reported a flat-panel human body security inspection device that uses all-electronic and all-solid-state technology. The transceiver unit composed of electronic switches can completely cover the person being inspected, so that a single detection takes only 64 milliseconds. The device operates in the E-band, which gives it better resolution performance than Ka-band devices. Although the device has achieved industry-leading performance advantages, it also has the problem of high cost. The academic paper "Improved threat detection in walk-through security scanning using an optimized polarimetric target decomposition method". IEEE Journal of Microwaves, 2022, 3(1): 16-28. Author: Root.K, et al. also reported a flat-panel human body security inspection device that uses all-electronic and all-solid-state technology. Four sets of RF front ends are installed diagonally above the head of the inspected person, and combined with the fast scanning time, the inspected person can walk through the device in a natural way, which greatly simplifies the security inspection process. However, its low operating frequency and limited aperture, although it can significantly reduce the cost of the equipment, result in low imaging quality of the equipment, which cannot meet the actual use needs.
[0005] In addition to the above problems, all current body inspection equipment still requires operators to be on duty next to the equipment. Compared with the traditional security inspection method where security inspectors scan with handheld metal detectors and pat the inspected person with their hands, the deployment of body inspection equipment has increased the equipment procurement cost while failing to reduce the demand for security inspectors to use it. It can be said that the failure to effectively reduce the security inspection operation cost has limited the widespread deployment of body inspection equipment. Summary of the invention
[0006] The technical problem to be solved by the present invention is to provide an unmanned millimeter wave human body imaging security inspection device and the machine vision algorithm used therein. Unmanned operation is achieved through the combined use of machine vision technology and a variety of strong reminder means. Through the reasonable structural design of the detection equipment, one-time coverage of the inspected person is achieved, and the coverage range is larger than that of the single-sided inspection equipment. Although the channel-type equipment can achieve full coverage of one-time detection, its channel-type setting form, terahertz working frequency band and passive imaging system result in large equipment size, high cost and poor resolution. By selecting the Ka-band linear array, while meeting the resolution requirements, the equipment cost can be further reduced, thereby achieving a balance between the detection coverage range and the equipment size, and between the resolution performance and the equipment cost.
[0007] In order to solve the above technical problems, the present invention provides an unattended millimeter wave human body imaging security inspection device, which includes: an active side of the device, a driven side of the device, an inspection channel and a gate, wherein the active side of the device is placed parallel to and opposite to the driven side of the device, and the active side of the device and the driven side of the device work in linkage, the inspection channel is located between the active side of the device and the driven side of the device, and the gate is located at the exit of the inspection channel, the inspection device uses machine vision technology to analyze the collected inspection channel and the optical images of the inspected person, and plays prompt sounds and prompt animations in a targeted manner according to the analysis results, so that the inspected person can make a correct standing posture and trigger the device to automatically start the inspection, and the device automatically opens or does not open the gate according to the inspection results, and prompts the inspected person to leave or turn to the manual inspection channel;
[0008] The machine vision technology adopts the PAFs algorithm, which generates a feature map and a PAFs vector field by sending the input image into a feedforward convolutional neural network model, gradually refines the prediction results, and establishes the connection relationship between the key points of the human body by generating a confidence map and a vector field.
[0009] Wherein, the active side of the device and the passive side of the device are respectively provided with a transceiver array, and the transceiver array operates in the Ka frequency band.
[0010] The transceiver array is composed of an antenna, a switch network, a transceiver and a data acquisition device, and the antenna openings are all oriented toward the inspection channel.
[0011] The active side of the device is provided with a display screen, an indicator light, a prompt doll logo, a camera, a speaker and a projection area;
[0012] The display screen is arranged on the side of the active side of the device, at the entrance of the inspection channel, and displays the working status of the device, prompt information and other contents;
[0013] The indicator light is arranged on the top of the active side of the device, above the display screen;
[0014] The reminder figure mark is set on the panel of the active side of the device to remind the examinee of the orientation and initial standing posture after entering the device;
[0015] The camera is arranged at the top of the active side of the device to collect optical images of the person under examination, and the collection range is the entire area of the inspection channel;
[0016] The speaker is arranged on the top of the active side of the device and faces the inspection channel, and is used to play a prompt sound;
[0017] The projection area is located on the panel on the active side of the device, has a certain height so that the examinee can watch it at eye level and does not interfere with the prompt puppet logo, and is used to play the prompt animation, which together with the played prompt sound prompts the examinee to maintain or adjust the standing posture, enter or leave the device.
[0018] Wherein, a projector is arranged at the driven side of the device, and the projector is arranged at the top of the driven side of the device and is used to project the prompt animation onto the projection area, and the host computer is located inside the driven side of the device.
[0019] Among them, the gates on both sides are respectively arranged on the active side and the driven side of the equipment, and are connected to the equipment as a whole. The gates are driven by motors and controlled by the host computer. The gates are in a closed state by default, and the exit sensing device is arranged on the outside of the gate to detect whether the inspected person passes through.
[0020] The algorithm used in the machine vision technology further specifically includes the following steps:
[0021] The first step is image input and preprocessing;
[0022] The second step is preliminary preparation;
[0023] The third step is loss function optimization;
[0024] Step 4: Build limb connections.
[0025] The first step is further specified as receiving an input optical image, which contains one or more subjects to be detected. The input image is sent to a feedforward convolutional neural network model for processing to generate a high-level feature map F of the image.
[0026] The second step is further specified as follows: the image is processed by a feedforward convolutional neural network model, and the system simultaneously generates a set of PAFs 2D vector fields L=f1(F), where f1 is the reasoning process of the first stage. The PAFs vector field is used to represent the connection strength and direction between different parts of the human body. Each PAFs vector has a specific direction and size, which is used to represent the possible connection between two key points.
[0027] At each stage, the PAFs prediction results L generated in the previous stage t-1 It is concatenated with the original feature map F as a new input to generate a more accurate prediction L t , this iterative process is expressed by the following formula: L t =f t (F,L t-1 ),2≤t≤Tp, where f t is the reasoning process of the tth stage, Tp is the total number of stages predicted by PAFs, and in each iteration, the network uses the current prediction results to further improve the output;
[0028] At each stage, the system generates a series of confidence maps S to indicate the possible locations of various body parts. Each confidence map S corresponds to a specific body part, and each pixel in S represents the probability that the pixel belongs to that body part. The initial confidence map STp is based on the final updated PAFs prediction vector field L t To calculate, the expression is: S Tp =ρ t (F,L t ), where ρ t is the reasoning process of the tth stage. Each subsequent stage will generate a more refined confidence map based on the confidence map and feature map F of the previous stage: S t =ρ t (F,L Tp ,S t-1 ),Tp≤t≤Tp+Tc, where Tc is the total number of stages predicted by the confidence map S.
[0029] The third step is further specified as follows: in the network training process, each stage has a corresponding loss function for optimizing the prediction results. For the prediction of PAFs, each stage uses a loss function f L ti , and for the prediction of the confidence map S, another loss function f is used S tk , the overall objective function is composed of the sum of the loss functions of all stages. The loss function of PAFs is from the 1st stage to the Tpth stage, and the loss function of the confidence map S is from the Tp+1th stage to the Tp+Tcth stage.
[0030] Among them, the fourth step is further specified as follows: in the process of constructing limb connections, first find the local maximum on the confidence map S as the candidate position of each part of the body, and the PAFs vector field L provides a 2D vector for each possible limb to encode the direction and intensity of each part of the body, and then divide the parts of the body into corresponding subjects.
[0031] The present invention also provides a method for implementing security inspection using the unattended millimeter wave human body imaging security inspection device, the method comprising:
[0032] The first step is to determine the initial state of the device;
[0033] The second step is to prompt the examinee to enter;
[0034] The third step is to prompt the examinee to make adjustments;
[0035] The fourth step is to determine the status of the person being examined;
[0036] Step 5: Start checking;
[0037] Step 6: Interpret the imaging results;
[0038] Step 7, open / not open the gate;
[0039] Step 8: Prompt the person being examined to leave.
[0040] Beneficial effects of the present invention
[0041] 1. By using machine vision technology to analyze the inspection channel and the optical images of the inspected person collected by the camera, combined with a variety of prompting methods, the inspected person can walk into the inspection equipment independently, take the correct standing position and trigger the equipment to automatically start the inspection. The equipment automatically opens or does not open the gate according to the inspection results and guides the inspected person to leave from the exit or entrance, realizing human safety inspection without manual intervention throughout the process, effectively avoiding the inspection process's dependence on on-site operators and reducing the operating cost of human safety inspection;
[0042] 2. The active side and the driven side of the equipment are placed opposite to each other, which can complete the entire inspection of the person being inspected at one time without turning around. This also avoids the excessive space occupied by channel-type human body security inspection equipment, and achieves a balance between the detection coverage and the equipment size;
[0043] 3. The millimeter-wave transceiver array operates in the relatively low frequency Ka band and uses a one-dimensional electronic switch network to scan, which reduces the equipment cost while achieving a relatively satisfactory resolution. Compared with the E-band all-electric scanning, it achieves a balance between resolution performance and equipment cost. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] Figure 1 An axonometric diagram of the structure of an unattended millimeter-wave human body imaging security inspection device according to an exemplary embodiment of the present application;
[0045] Figure 2 A perspective view of a driven side of an unattended millimeter wave human body imaging security inspection device according to an exemplary embodiment of the present application;
[0046] Figure 3The present invention is a flowchart of a human body security inspection method according to an exemplary embodiment of the unattended millimeter wave human body imaging security inspection device of the present application.
[0047] The markings in the figure are as follows: 1. Active side of the device; 101. Display screen; 102. Indicator light; 103. Prompt puppet mark; 104. Camera; 105. Speaker; 106. Projection area; 2. Slave side of the device; 201. Millimeter wave transceiver array; 202. Motion mechanism; 203. Host computer and other auxiliary circuits and equipment; 204. Projector; 3. Inspection channel; 301. Prompt footprint mark; 4. Gate; 401. Exit sensing device. DETAILED DESCRIPTION
[0048] The present invention provides an unattended millimeter wave human body imaging security inspection device, which comprises: an active side of the device, a driven side of the device, an inspection channel and a gate. The active side of the device is placed parallel to the driven side of the device, and the active side of the device and the driven side of the device work in linkage. The inspection channel is located between the active side of the device and the driven side of the device. The inspected person enters the inspection channel from the entrance and leaves from the exit. The gate is located at the exit of the inspection channel.
[0049] The front sides of the active side and the passive side of the device are panels that can be penetrated by millimeter waves but not by visible light, and the remaining areas are shells made of materials with sufficiently high strength. The front sides of the devices face the inspection channel.
[0050] The active side and the passive side of the device are respectively provided with a transceiver array, which operates in the Ka frequency band, actively transmits electromagnetic waves and receives the echo of the target. The transceiver array is composed of an antenna, a switch network, a transceiver and a data acquisition device, and the antenna openings are all facing the inspection channel.
[0051] The transceiver array completely covers the scanning area by traversing the switching switch network and the motion mechanism to drive its movement, so that a single-side device can completely cover the area on one side of the subject in one detection. The active side of the device and the driven side of the device are placed relative to each other and work at the same time, so that one detection can cover the entire area of the subject.
[0052] In some embodiments, the active side and the slave side of the device are electrically connected and controlled by a host computer, and the millimeter wave transceiver array, motion mechanism and other auxiliary circuits and devices are all located inside the active side and / or the slave side of the device. The transceiver arrays inside the active side and the slave side of the device start to move at the same time under the drive of the motion mechanism, and the directions are opposite to each other.
[0053] In some embodiments, the active side of the device is provided with a display screen, an indicator light, a prompt puppet mark, a camera, a speaker and a projection area. The display screen is provided on the side of the active side of the device, at the entrance of the inspection channel, and displays the working status of the device, prompt information and the like. The indicator light is provided on the top of the side of the active side of the device, above the display screen, and guides the examinee waiting for inspection to enter the device or continue waiting by changing color. The prompt puppet mark is provided on the panel of the active side of the device, and prompts the examinee of the orientation and initial standing posture after entering the device. The camera is provided on the top of the active side of the device, and collects the optical image of the examinee, and the collection range is the entire area of the inspection channel. The speaker is provided on the top of the active side of the device and faces the inspection channel, and is used to play a prompt sound to prompt the examinee to maintain or adjust his standing position and posture, and enter or leave the device. The projection area is located on the panel of the active side of the device, has a certain height so that the examinee can watch it with eyes level and does not interfere with the prompt puppet mark, and is used to play a prompt animation to prompt the examinee to maintain or adjust his standing posture, and enter or leave the device.
[0054] In some embodiments, a projector is disposed on the slave side of the device. The projector is disposed on the top of the slave side of the device and is used to project the prompt animation onto the projection area. The host computer is located inside the slave side of the device.
[0055] In some embodiments, the inspection channel is provided with a prompt footprint mark, and the prompt footprint mark is located in the middle of the inspection channel floor, which is used to prompt the direction and position of the examinee after entering the equipment.
[0056] In some embodiments, the gates on both sides are respectively arranged on the active side and the driven side of the device, and are connected to the device as a whole. The gates are driven by a motor and controlled by a host computer. The gates are in a closed state by default, and the exit sensing device is arranged on the outside of the gate to detect whether a person being examined passes through.
[0057] Furthermore, the device guides the examinee into the inspection channel through the display screen and indicator lights, and preliminarily guides the examinee through the prompt puppet mark and the prompt footprint mark to form a correct orientation and standing posture. The camera collects the optical image of the examinee and sends the optical image to the host computer. The host computer uses machine vision technology to make judgments including but not limited to whether there is someone following in the device, whether the examinee’s orientation is correct, whether the examinee’s standing position meets the requirements, and whether the examinee has left. When the inspection conditions are met or not, the speaker and projection area animation prompts the examinee to remain still or adjust to the standard standing posture.
[0058] There are two main difficulties in analyzing the optical images collected by the camera during the security inspection process. First, each image may contain an unknown number of people who can appear in any position. Second, the interaction between people will cause complex spatial interference due to contact and / or occlusion, making it difficult to associate parts.
[0059] The present disclosure adopts the PAFs (Part Affinity Fields) algorithm to detect key points of the human body and their connection relationships, which better overcomes the above problems. The algorithm specifically includes the following steps:
[0060] Image input and preprocessing, first determine the number of examinees in the inspection channel, and then determine whether the examinee's posture meets the inspection requirements; in the task of determining the number of examinees in the channel, the PAFs algorithm first processes multiple overlapping or adjacent human bodies in the image, and assigns each body part to the correct examinee; the input optical image is first input into a feedforward convolutional neural network model to generate a set of feature maps F containing high-level features of the image, which is an intermediate representation of the input optical image after being processed by the feedforward convolutional neural network model; the feedforward convolutional neural network model can be VGG-19;
[0061] Generation of PAFs 2D vector field,The feed-forward convolutional neural network model simultaneously generates a set of PAFs 2D vector fields L = f1(F) representing the connection strength and direction between different body parts, where f1 is the reasoning process of the first stage. Each PAFs vector has a specific direction and size, which is used to represent the possible connection between two key points;
[0062] Feature fusion and refinement, the system gradually refines the prediction results of PAFs in multiple stages. In each stage, the PAFs prediction results L generated in the previous stage t-1 It is concatenated with the original feature map F to form a new input for generating a more refined prediction L t , this allows the network to use the latest predictions to improve its output in each iteration:
[0063] L t =f t (F,L t-1 ),2≤t≤Tp;
[0064] In the formula, f t is the reasoning process of the tth stage, Tp is the total number of stages predicted by PAFs; in each iteration, the network uses the current prediction results to further improve the output;
[0065] Generation of confidence maps,The feature map F is also used to generate a series of confidence maps S,which are used to indicate the possible locations of various body parts.,Each confidence map S corresponds to a body part, and each pixel in S represents the,probability that the pixel belongs to the body part;
[0066] The initial confidence map S Tp is the predicted vector field L based on the last updated PAFs t To calculate:
[0067] S Tp =ρ t (F,L t );
[0068] In the formula, ρ t is the reasoning process at stage t;
[0069] Subsequently, each stage generates a more refined confidence map S based on the previous predictions and feature map F t :
[0070] S t =ρ t (F,L Tp ,S t-1 ),Tp≤t≤Tp+Tc;
[0071] Where, T C is the total number of stages predicted by the confidence map S;
[0072] Loss function optimization. During the network training process, each stage has a corresponding loss function to optimize the prediction results of that stage. For PAFs prediction, at each stage t i A loss function f is used L ti , and for the confidence map S prediction, it is at each stage t k The loss function f is used S tk ;
[0073] The overall objective function is composed of the sum of the loss functions of all stages, where the loss function of PAFs is from the 1st to the Tpth stage, and the loss function of the confidence map S is from the Tp+1th to the Tp+Tcth stage; the purpose of this design is to improve the final prediction effect by gradually refining;
[0074] Construct limb connections, that is, determine the connection relationship between various body parts in the image; first, find local maxima on the confidence map S. These maximum points represent the possible positions of various body parts, and each maximum point is regarded as a candidate position of a body part;
[0075] Among them, the PAFs vector field L provides a 2D vector field for each possible limb, each vector encodes the direction and strength from one body part to another. These vectors are used to guide how to connect body parts, thereby clarifying which parts belong to the same subject.
[0076] Through the above steps, the algorithm can determine the number of persons under examination in the examination channel in the optical image.
[0077] Furthermore, in the task of judging the subject’s posture, the PAFs algorithm is used to capture the correlation between different body parts, thereby helping to identify the complete body posture;
[0078] Among them, PAFs are used to detect limb connections. Specifically, the system looks for strongly associated paths from one detected body part to another in PAFs. These paths are indicated by vectors with higher scores in PAFs.
[0079] Once strongly associated paths are detected, they can be connected to form a complete body posture. The system will perform the above steps independently for each limb type to obtain connection candidates for each limb type.
[0080] In order to further optimize the matching problem, the entire matching problem is decomposed into a series of two-by-two matching sub-problems, and the matching between adjacent tree nodes is determined independently. In this way, the algorithm can quickly find the global optimal solution under the premise of only adopting a local greedy strategy;
[0081] The algorithm also handles redundant connections, such as ear to shoulder, wrist to shoulder, etc. These redundant connections improve accuracy in crowded images. To handle these redundant connections, the algorithm sorts all possible pairing connections by their PAFs scores. If a connection attempts to connect two body parts that have been assigned to different individuals, then the connection will be ignored.
[0082] Through the above steps, the algorithm can effectively analyze the complete posture of the subject in the image.
[0083] Furthermore, the millimeter wave array and the motion mechanism start scanning imaging under the control of the host computer, and the imaging results are automatically interpreted by artificial intelligence or sent to a remote image interpreter for manual interpretation. When the interpretation result is pass, the gate opens and guides the examinee to leave the equipment from the inspection channel exit through the speaker and the animation in the projection area. When the interpretation result is fail, the gate does not open and guides the examinee to leave the equipment from the inspection channel entrance through the speaker and the animation in the projection area and enter the manual inspection channel.
[0084] The following embodiments and drawings are used to describe the implementation methods of the present invention in detail, so that the implementation process of how the present invention applies technical means to solve technical problems and achieve technical effects can be fully understood and implemented accordingly.
[0085] Figure 1 The isometric diagram of the structure of an unattended millimeter wave human body imaging security inspection device according to an embodiment of the present application is shown. As shown in the figure, the device includes an active side 1 of the device, a passive side 2 of the device, an inspection channel 3 and a gate 4.
[0086] The current mainstream active millimeter wave human body imaging security inspection equipment can be divided into cylindrical linear scanning, planar linear scanning and planar array scanning according to the scanning form. In this embodiment, both the active side 1 of the device and the passive side 2 of the device adopt planar linear scanning. In some other embodiments, imaging can also be based on passive imaging technology, that is, imaging is achieved by collecting weak signals reflected or scattered by the subject.
[0087] In the planar linear scanning form, according to the direction of movement of the transceiver array, there are two types: one is to use the electronic switch network to turn on / off in sequence in the horizontal direction to perform traversal scanning, and use the motion mechanism to drive the transceiver array to move in the vertical direction to perform traversal scanning; the other is the opposite, the electronic switch network is turned on / off in sequence to perform traversal scanning in the vertical direction, and the motion mechanism is used to drive the transceiver array to move in the horizontal direction to perform traversal scanning. In this embodiment, both the active side 1 of the device and the passive side 2 of the device adopt the first method.
[0088] like Figure 1 In the illustrated embodiment, the device active side 1 and the device driven side 2 are respectively located on both sides of the inspection channel 3, placed in parallel and opposite to each other, and work in conjunction under the control of the host computer and other auxiliary circuits and equipment 203. The inspection line is located in the middle of the inspection channel 3 and is parallel to the device active side 1 and the device driven side 2. One end of the inspection line is the entrance of the inspection channel 3, and the other end is the exit of the inspection channel 3. The gate 4 is arranged near the exit of the inspection channel 3 and is in a closed state by default.
[0089] The front sides of the device active side 1 and the device passive side 2 are panels that are well penetrated by millimeter waves but not by visible light, and the remaining areas are shells made of materials with sufficiently high strength, with the front sides facing the inspection channel 3.
[0090] The display screen 101 , the indicator light 102 , the prompt puppet mark 103 , the camera 104 , the speaker 105 and the projection area 106 are arranged on the active side 1 of the device.
[0091] The display screen 101 is arranged on the side of the active side 1 of the device, near the entrance of the inspection channel 3, and is used to prompt the examinee waiting outside the device with relevant information in the form of graphics and text, including but not limited to standard standing posture information, waiting or entry information and current time, etc. The display screen 101 can be an LED display screen, or various displays or screens capable of displaying graphics and text information.
[0092] The indicator light 102 is arranged on the top of the side of the active side 1 and the passive side 2 of the device, and is used to prompt the examinee waiting outside the device with waiting or entry information in the form of light, and guide the examinee to enter the device or wait outside the device by changing the color of the light. The indicator light 102 can be any light source that can change color. In some embodiments, the indicator light 102 is also used to display the device status or alarm.
[0093] The reminder puppet mark 103 is set on the panel of the active side 1 of the device, below the projection area 106, and is used to remind the examinee of the direction and initial standing posture after entering the device. The reminder puppet mark 103 is different from the color of the panel itself, and is attached to the side of the panel facing the inspection channel 3 in the form of stickers or spray paint. The reminder puppet marks 103 are set below the projection area 106 and do not overlap with each other, so as not to affect the examinee's viewing of the content in the projection area 106. In some embodiments, the reminder puppet mark 103 can also be projected and displayed by the projector 204, and can dynamically change its shape to remind the examinee to adjust his standing posture according to the standard state, but no matter what form it is, it should not have an adverse effect on the millimeter wave penetrating the panel.
[0094] The camera 104 is set at the top of the active side 1 of the device, and is used to collect optical images in the inspection channel 3 in real time, and send the optical images to the host computer as the main basis for the machine vision recognition algorithm to judge the inside of the inspection channel 3 and the state of the inspected person. It mainly undertakes the judgment function of the operator whether the inspected person's standing position is standard and whether the inspection channel 3 is clear during the operation of the traditional manned millimeter wave human imaging safety inspection equipment. In some embodiments, there can be more than one optical collection device or camera 104, and multiple cameras 104 in the device can simultaneously collect the inspection channel 3 and the state of the inspected person, so as to make a more accurate judgment.
[0095] The speaker 105 is arranged at the top of the active side 1 of the device, and is used to prompt the examinee to pay attention to the content in the projection area 106, adjust the standing position, or enter and leave the device in an audio manner. The speaker 105 is controlled by the host computer and can also be remotely controlled when the remote reader actively intervenes. In some embodiments, the audio device or speaker 105 can be more than one place, for example, it can be set at the active side 1 of the device and the slave side 2 of the device at the same time. In some embodiments, the speaker 105 is also used to prompt the device status or alarm.
[0096] The projection area 106 is arranged on the panel of the active side 1 of the device, and its height is equivalent to the position of the head of the examinee, so that the examinee can easily see the display content of this area. The projection area 106 is the display area of the projector 204 located at the top of the passive side 2 of the device, and is used to display prompt information for the examinee. It mainly undertakes the prompt function of the operator to the examinee during the operation of traditional manned millimeter-wave human imaging safety inspection equipment.
[0097] The display content of the display screen 101, the color change of the indicator light 102, the prompt puppet logo 103, the audio of the speaker 105 and the display content of the projection area 106 work together to form a strong reminder for the person being inspected, so as to ensure that in an unattended state, the person being inspected can still receive clear prompt information that can be verified by each other, so as to cooperate in completing the safety inspection.
[0098] The prompt footprint mark 301 is set in the center of the inspection channel 3. It is different from the color and material of the floor of the inspection channel 3. It is used to prompt the direction and initial standing posture of the inspected person after entering the equipment. The length of the inspection channel 3 from the entrance to the exit is roughly the same as the equipment itself, and the width is determined by the position of the focal plane determined by factors such as the imaging system and the frequency band used. The prompt footprint mark 301 is consistent with the position of the focal plane.
[0099] The gate of the gate machine 4 is made of a transparent acrylic plate with a certain thickness, fixed on the transmission shaft, and the transmission shaft is further connected to the active side 1 of the device and the driven side 2 of the device. The relevant mechanism for controlling and driving the operation of the gate machine 4 is arranged inside the device. In some embodiments, the gate machine 4 has an anti-pinch function, and the exit sensing device 401 is an infrared radiation switch.
[0100] The exit sensing device 401 is arranged outside the gate to detect whether a person under inspection has passed through. When it is detected that a person under inspection has left, a signal is sent to the host computer. The host computer recognizes the optical image captured by the camera 104, confirms that the inspection channel is clear, and then closes the gate.
[0101] like Figure 2 As shown, the transceiver array 201, the motion mechanism 202, the host computer and other auxiliary circuits and devices 203 are placed inside the device slave side 2, the transceiver array 201 is mounted on the motion mechanism 202 and can reciprocate along a predetermined direction under the drive of the motion mechanism 202, and the antenna surfaces of the transceiver array 201 are all facing the inspection channel 3. The projector 204 is mounted on the top of the device slave side 2, and the projection content is displayed on the projection area 106.
[0102] The transceiver array 201 can generate millimeter wave signals and transmit the millimeter wave signals to the subject through its own antenna array. The signal is blocked and reflected by the subject and then received by the antenna. After being processed into an intermediate frequency signal by the transceiver array 201, it is sent to the data acquisition device. The data acquisition device performs "analog-to-digital" conversion on the intermediate frequency signal and sends the processed information to the host computer for imaging.
[0103] The motion mechanism 202 can drive the transceiver array 201 to reciprocate along a fixed path at a predetermined speed. The transceiver array 201 and the motion mechanism 202 inside the active side 1 of the device and the driven side 2 of the device are configured the same, and the antenna aperture faces the inspection channel 3 and are controlled by the device 203. When working in linkage, the transceiver arrays 201 on both sides start to move at the same time, but the zero point position and the direction of movement are opposite. This avoids the transceiver arrays 201 on both sides from interfering with each other, and achieves the goal of covering the entire inspection area of the examinee in one scan. In some embodiments, the motion mechanism is driven by a motor and driven by a belt or a lead screw. The transceiver arrays 201 on both sides can be driven by one motor at the same time, or by two motors respectively.
[0104] The device 203 is placed at the bottom of the slave side 2 of the device, and plays the role of controlling the operation of the whole device, identifying the optical image collected by the camera 104, processing the intermediate frequency signal sent by the transceiver array 201 and imaging, identifying and marking the imaging results, and communicating with other devices. In some embodiments, the host computer and other auxiliary circuits and devices 203 are separated and placed on both sides of the device, and then connected by wires and network cables, or placed outside or on the top of the security inspection device, or some functions are transferred to the cloud.
[0105] The projector 204 is arranged at the top of the driven side 2 of the device, and is used to project prompt information for the examinee to the designated projection area 106, so as to prompt the examinee to adjust his / her standing posture or enter or leave the device. Using the projector 204 instead of a display device such as a display screen can conveniently display the prompt information in front of the examinee on the one hand, and can effectively avoid the problem of millimeter wave signals being blocked on the other hand.
[0106] The examinee outside the device can judge whether to enter the device according to the display contents of the display screen 101 and the indicator light 102. After entering the device, the examinee can preliminarily determine the standing position under the prompt of the puppet mark 103, the footprint mark 301 and the display contents of the projection area 106. The camera 104 collects the situation in the inspection channel 3 in real time and sends the collected optical image to the host computer. The machine vision recognition algorithm judges the situation in the inspection channel and the examinee's standing position in real time, and prompts the examinee to make corresponding adjustments through the speaker 105 and the content of the projection area 106. When the examinee adjusts to the standard standing position, the host computer triggers the millimeter wave transceiver array 201 and the motion mechanism 203 to work, collect the examinee's body surface feature information and image it, open or not open the gate 4 according to the imaging result, and prompt the examinee to leave from the exit or entrance.
[0107] like Figure 3 As shown, the present disclosure also provides a method for implementing security inspection using unattended millimeter wave human body imaging security inspection equipment, the method comprising:
[0108] Step 1: Determine the initial state of the device. The device self-checks to confirm whether each subsystem is ready. Check whether channel 3 is clear through optical image judgment.
[0109] Step 2: Prompt the examinee to enter. When the subsystems of the device are ready and the inspection channel 3 is clear, the display content of the display screen 101 and the display status of the indicator light 102 are changed to prompt the examinee in the waiting area to enter the device. When the subsystems of the device are not ready or the inspection channel 3 is not clear, the display screen 101 and the indicator light 102 prompt the examinee to continue waiting in the waiting area;
[0110] Step 3: Prompt the examinee to adjust. After the examinee enters the device, the differences between the examinee's position, posture, direction, or whether someone is following him and the standard state are compared in real time through optical images. According to the differences, different prompt animations and prompt sounds are played to the examinee to prompt the examinee to adjust, and the display screen 101 changes the display content and the indicator light 102 changes the state, prompting people outside the device not to enter the device;
[0111] Step 4: Determine the status of the examinee. Determine in real time whether the examinee is in place and adjust the prompt animation and prompt tone content according to the difference;
[0112] Step 5: Start the examination. After the examinee has adjusted himself, the prompt animation and prompt sound prompt the examinee to keep still, and the transceiver array 201 completes the scanning of the examinee under the drive of the motion mechanism 202;
[0113] Step 6: Interpret the imaging results. After the scan is completed and the image is formed, the device will send the examination results to the remote interpreter or automatically identify and interpret them, and prompt the examinee to wait for the interpretation results on the spot through prompt animation and prompt sound;
[0114] Step 7: Open / not open the gate. When the judgment result is passed, the gate is opened, and when the judgment result is not passed, the gate remains closed;
[0115] Step 8: Prompt the examinee to leave. Play prompt animation and prompt sound to the examinee according to the judgment result. When the judgment result is passed, the examinee is prompted to leave from the exit. When the judgment result is not passed, the examinee is prompted to leave from the entrance. When the exit sensing device 401 detects that the examinee has left, the device determines whether the inspection channel 3 is empty through optical images, and closes the gate after confirming that the examinee has left.
[0116] As mentioned above, this inspection method can complete the safety inspection of the human body without human supervision on site, effectively reducing the operating costs of security inspection. In addition, compared with the traditional manned millimeter wave human body imaging security inspection equipment, which requires the operator to judge the standing posture and position of the inspected person, the unmanned millimeter wave human body imaging security inspection equipment proposed in the present disclosure uses machine vision technology to identify the standing posture and position of the inspected person, avoiding the error of the operator's subjective judgment, thereby ensuring that the standing posture and position of the inspected person can always meet the standards, thereby ensuring the quality of millimeter wave imaging.
[0117] All of the above are primary implementations of this intellectual property and do not set limitations on other forms of implementation of this new product and / or new method. Those skilled in the art will use this important information and modify the above content to achieve similar implementations. However, all modifications or transformations based on the new product of this invention belong to the reserved rights.
[0118] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.
Claims
1. An unattended millimeter wave human body imaging safety inspection device, characterized in that: include: The active side of the device, the driven side of the device, the inspection channel and the gate, the active side of the device and the driven side of the device are placed parallel to each other, the active side of the device and the driven side of the device work in linkage, the inspection channel is located between the active side of the device and the driven side of the device, the gate is located at the exit of the inspection channel, the inspection device uses machine vision technology to analyze the collected optical images of the inspection channel and the subject, and plays prompt sounds and prompt animations in a targeted manner according to the analysis results, so that the subject can make a correct standing posture and trigger the device to automatically start the inspection, the device automatically opens or does not open the gate according to the inspection results, and prompts the subject to leave or turn to the manual inspection channel; The machine vision technology adopts the PAFs algorithm, which generates a feature map and a PAFs vector field by sending the input image into a feedforward convolutional neural network model, gradually refines the prediction results, and establishes the connection relationship between the key points of the human body by generating a confidence map and a vector field.
2. The unattended millimeter wave human body imaging safety inspection equipment as claimed in claim 1, characterized in that: The active side of the device and the passive side of the device are respectively provided with a transceiver array, and the transceiver array operates in the Ka frequency band.
3. The unattended millimeter wave human body imaging safety inspection equipment according to claim 1 or 2, characterized in that: The transceiver array is composed of an antenna, a switch network, a transceiver and a data acquisition device, and the antenna openings are all oriented toward the inspection channel.
4. The unattended millimeter wave human body imaging safety inspection equipment according to claim 1 or 2, characterized in that: The active side of the device is provided with a display screen, an indicator light, a prompt puppet mark, a camera, a speaker and a projection area; The display screen is arranged on the side of the active side of the device, at the entrance of the inspection channel, and displays the working status of the device, prompt information and other contents; The indicator light is arranged on the top of the active side of the device, above the display screen; The reminder figure mark is set on the panel of the active side of the device to remind the examinee of the orientation and initial standing posture after entering the device; The camera is arranged at the top of the active side of the device to collect optical images of the person under examination, and the collection range is the entire area of the inspection channel; The speaker is arranged on the top of the active side of the device and faces the inspection channel, and is used to play a prompt sound; The projection area is located on the panel on the active side of the device, has a certain height so that the examinee can watch it at eye level and does not interfere with the prompt puppet logo, and is used to play the prompt animation, which together with the played prompt sound prompts the examinee to maintain or adjust the standing posture, enter or leave the device.
5. The machine vision algorithm used by the unattended millimeter wave human body imaging security inspection equipment according to claim 1 or 2 is characterized in that: Specifically include: The first step is image input and preprocessing; The second step is the generation of PAFs 2D vector field; The third step is feature fusion and refinement; The fourth step is the generation of confidence map; Step 5: Loss function optimization; Step 6: Build limb connections.
6. The machine vision algorithm of claim 5, wherein: The first step is further specifically as follows: receiving an input optical image, which contains one or more subjects to be detected, and the input image is sent to a feedforward convolutional neural network model for processing to generate a high-level feature map F of the image.
7. The machine vision algorithm according to claim 5 or 6, characterized in that: The second step is further specified as follows: the image is processed by a feedforward convolutional neural network model, and the system simultaneously generates a set of PAFs2D vector fields L=f1(F), where f1 is the reasoning process of the first stage, and the PAFs vector field is used to represent the connection strength and direction between different parts of the human body. Each PAFs vector has a specific direction and size, which is used to represent the possible connection between two key points; At each stage, the PAFs prediction results L generated in the previous stage t-1 It is concatenated with the original feature map F as a new input to generate a more accurate prediction L t , this iterative process is expressed by the following formula: L t =f t (F,L t-1 ),2≤t≤Tp, where f t is the reasoning process of the tth stage, Tp is the total number of stages predicted by PAFs, and in each iteration, the network uses the current prediction results to further improve the output; At each stage, the system generates a series of confidence maps S to indicate the possible locations of various body parts. Each confidence map S corresponds to a specific body part, and each pixel in S represents the probability that the pixel belongs to that body part. The initial confidence map STp is based on the final updated PAFs prediction vector field L t To calculate, the expression is: S Tp =ρ t (F,L t ), where ρ t is the reasoning process of the tth stage. Each subsequent stage will generate a more refined confidence map based on the confidence map and feature map F of the previous stage: S t =ρ t (F,L Tp ,S t-1 ),Tp≤t≤Tp+Tc, where Tc is the total number of stages predicted by the confidence map S.
8. The machine vision algorithm according to claim 5 or 6, characterized in that: The third step is further specified as follows: in the network training process, each stage has a corresponding loss function for optimizing the prediction results. For the prediction of PAFs, each stage uses a loss function f L ti , and for the prediction of the confidence map S, another loss function f is used S tk , the overall objective function is composed of the sum of the loss functions of all stages. The loss function of PAFs is from the 1st stage to the Tpth stage, and the loss function of the confidence map S is from the Tp+1th stage to the Tp+Tcth stage.
9. The machine vision algorithm according to claim 5 or 6, characterized in that: The fourth step is further specified as follows: in the process of constructing limb connections, firstly, local maxima are found on the confidence map S as candidate positions of various parts of the body. The PAFs vector field L provides a 2D vector for each possible limb to encode the direction and intensity of each part of the body, thereby dividing the parts of the body into corresponding subjects.
10. A method for implementing security inspection using the unattended millimeter wave human body imaging security inspection equipment according to claim 1 or 2, characterized in that: include: The first step is to determine the initial state of the device; The second step is to prompt the examinee to enter; The third step is to prompt the examinee to make adjustments; The fourth step is to determine the status of the person being examined; Step 5: Start checking; Step 6: Interpret the imaging results; Step 7, open / not open the gate; Step 8: Prompt the person being examined to leave.
