Specified standing posture prompt graph generation method and system and millimeter wave self-service scanning security check method and system

Abstract

The invention provides a specified standing posture prompt graph generation method and system and a millimeter wave self-service scanning security check method and system. The method comprises the following steps: obtaining a standing posture graph of a detected person through a camera device; obtaining a body type standing posture graph of the examined person through the standing posture graph of the examined person; based on a standard standing posture graph, adjusting the body type standing posture graph of the examined person, obtaining a standing posture prompt graph of the examined person, and taking the standing posture prompt graph as a specified standing posture prompt graph of the examined person; obtaining a current standing posture image of the examined person based on the standing posture image and the posture characteristics of the examined person obtained through a camera device; the designated standing posture prompt graph of the examinee and the current standing posture graph are displayed to the examinee at the same time, and the designated standing posture prompt graph can be different according to the posture characteristics of different examinees.

Description

Technical Field

[0001] The present disclosure relates to the field of security inspection, specifically to image generation in a security inspection process, and more specifically to a system and method for generating a designated standing posture diagram, and a millimeter wave self-service scanning security inspection system and method including the system for generating the designated standing posture diagram. Background Art

[0002] The background description provided herein is to generally present the context of the present disclosure. To the extent described in this background section, the work of the presently named inventors and aspects of the description that may not constitute prior art at the time of filing are neither explicitly nor implicitly admitted to be prior art against the present disclosure.

[0003] Millimeter wave human imaging technology is currently an advanced technology in the global security field and has been used for passenger security checks at airports in various countries. The equipment can effectively detect objects hidden in various parts of the human body under clothing without direct contact with the human body, especially non-metallic objects, and can obtain information such as the shape, size and location of the hidden objects from the image. In addition, millimeter wave human imaging equipment is harmless to the human body and has strong penetrating power. Its transmission power is less than one thousandth of the electromagnetic wave radiation of a mobile phone. It can accurately identify objects carried by the human body, effectively improve the objectivity, accuracy and pertinence of the inspection, reduce the labor intensity of security inspectors, and improve security inspection efficiency.

[0004] Due to the limitation of imaging principle, when the inspected person stands in the security inspection channel and is scanned by the millimeter wave security scanner, he must strike a specific posture, and his arms and legs must be opened at a certain angle, otherwise the arms will block the armpits and the legs will block the inside of the legs, or artifacts will be formed between the legs and under the armpits. In addition, the automatic target recognition (ATR) module also has requirements for standing posture when identifying suspects and converting the original image into a cartoon display. When the standing posture is judged to be wrong, the inspected person will be asked to adjust the standing posture and rescan. When the equipment operator or guide is not familiar with the standing posture requirements, the scanning will frequently encounter the problem of incorrect standing posture, so it takes a lot of time to train the equipment operator or guide.

[0005] Therefore, there is an urgent need for a method and system that can generate a designated standing posture prompt diagram that is suitable for different examinees. Summary of the invention

[0006] The present disclosure provides a method for generating a designated standing posture prompt image, the method comprising: obtaining a standing posture image of a subject through a camera device; obtaining a body shape standing posture image of the subject through the standing posture image of the subject; adjusting the body shape standing posture image of the subject based on a standard standing posture image, and obtaining a standing posture prompt image of the subject, and using the standing posture prompt image as the designated standing posture prompt image of the subject; obtaining a current standing posture image of the subject based on the standing posture image and body shape characteristics of the subject obtained through a camera device; and simultaneously displaying the designated standing posture prompt image and the current standing posture image of the subject to the subject, wherein the designated standing posture prompt image can be different according to the body shape characteristics of different subjects.

[0007] In one embodiment of the present disclosure, obtaining the subject's body posture diagram includes: obtaining the subject's key point coordinates based on the subject's posture diagram, and locating the subject's body key points using a key point positioning model, wherein the key point positioning model is based on a convolutional neural network model, and the convolutional neural network model includes multiple residual blocks, convolutional layers, pooling layers and fully connected layers.

[0008] In one embodiment of the present disclosure, the key points of the human body include the head, shoulders, elbows, wrists, hips, knees, and ankles.

[0009] In one embodiment of the present disclosure, obtaining the subject's body posture diagram includes: obtaining the human body parts corresponding to the human body key points based on the located human body key points, and connecting the obtained human body parts to obtain the contour line of the subject's current posture diagram.

[0010] In one embodiment of the present disclosure, obtaining the body posture diagram of the subject also includes: calculating the translation distance and / or scaling factor of the key points based on the standard posture diagram and the key points of the human body, and performing translation and scaling operations on the obtained body posture diagram based on the translation distance and / or scaling factor.

[0011] In one embodiment of the present disclosure, obtaining the current standing posture diagram of the subject includes: obtaining the current standing posture diagram of the subject using a three-dimensional reconstruction model of the human body, and the three-dimensional reconstruction model of the human body includes binocular camera alignment processing, point cloud stitching and denoising processing, point cloud plane mapping processing and GPU acceleration processing.

[0012] In one embodiment of the present disclosure, the method for generating a designated standing posture prompt diagram further includes: comparing the current standing posture diagram of the examinee with the designated standing posture prompt diagram, and displaying the comparison result to the examinee through a display device to guide the examinee to make his current standing posture diagram within the range of the designated standing posture prompt diagram.

[0013] In one embodiment of the present disclosure, the method for generating a designated standing posture prompt diagram further includes: when the front standing posture diagram of the subject is located within the range of the designated standing posture prompt diagram, the designated standing posture prompt diagram is displayed in a first color; when at least a portion of the front standing posture diagram of the subject is located outside the range of the designated standing posture prompt diagram, the designated standing posture prompt diagram is displayed in a second color, and the second color is different from the first color.

[0014] In one embodiment of the present disclosure, the method for generating a specified standing posture prompt diagram further includes: generating a specified standing posture prompt diagram of a fixed size, scaling the current standing posture diagram to make it the same size as the specified standing posture prompt diagram, and Figure 1 The same reality is given to the examinee.

[0015] The present disclosure also provides a millimeter wave security inspection method, which includes the following steps: generating a designated standing posture prompt diagram according to the method described in any of the above embodiments; continuously capturing an image of a person under inspection by using a camera device; determining whether the current standing posture of the person under inspection is consistent with the designated standing posture prompt diagram; when the current standing posture of the person under inspection is inconsistent with the designated standing posture prompt diagram, reminding the person under inspection to adjust his standing posture; and when the current standing posture of the person under inspection is inconsistent with the designated standing posture prompt diagram, Figure 1 When a person is found to be suspicious, a millimeter wave security scanner is used to check the person being inspected, and the location corresponding to the suspicious object is identified and marked.

[0016] The present disclosure also provides a designated standing posture generation system for generating a prompt image of a designated standing posture, comprising: a camera device, configured to obtain image data of a subject's standing posture and body shape; an operation processing module, configured to obtain a body shape standing posture image of the subject through the subject's standing posture image; based on the standard standing posture image, adjusting the subject's body shape standing posture image, and obtaining a standing posture prompt image of the subject, and using the standing posture prompt image as the designated standing posture prompt image of the subject, and obtaining a current standing posture image of the subject based on the subject's standing posture image and body shape characteristics; and simultaneously displaying the subject's designated standing posture prompt image and the current standing posture image to the subject, wherein the designated standing posture prompt image is different according to the body shape characteristics of different subjects.

[0017] In one embodiment of the present disclosure, the designated standing posture generating system further comprises a display panel for displaying a current standing posture diagram and a designated standing posture prompt diagram.

[0018] In one embodiment of the present disclosure, the camera device includes a visible light camera device, an infrared camera device, and a binocular depth camera device.

[0019] In one embodiment of the present disclosure, the operation processing module includes a key point positioning model for determining the key point coordinates of the subject's standing posture diagram, and the key point positioning model is based on a convolutional neural network model, which includes multiple residual blocks, convolutional layers, pooling layers and fully connected layers.

[0020] In one embodiment of the present disclosure, the operation processing module is configured to obtain the coordinates of key points of the subject based on the subject's standing posture diagram, obtain the human body parts corresponding to the human body key points according to the located human body key points, connect the obtained human body parts and obtain the contour lines of the subject's current standing posture diagram, and calculate the translation distance and / or scaling factor of the key points based on the standard standing posture diagram and the human body key points, and perform translation and scaling operations on the obtained body shape standing posture diagram based on the translation distance and / or scaling factor.

[0021] In one embodiment of the present disclosure, the operation processing module is also configured to simultaneously display the current standing posture diagram and the designated standing posture prompt diagram to the examinee to guide the examinee to adjust the current standing posture, and adjusting the standing posture includes adjusting the raising and / or lowering of the arms and the closing and / or separation of the legs.

[0022] The present disclosure also provides a millimeter wave security inspection system, wherein the millimeter wave security inspection system includes a designated standing posture generation system, a millimeter wave security inspection instrument and an inspection channel as described in any of the above systems, the millimeter wave security inspection instrument is configured to perform a millimeter wave scanning inspection on the inspected person to identify and mark suspicious objects, and the inspection channel is configured for the inspected person to undergo a millimeter wave scanning inspection therein.

[0023] These and other aspects of the disclosure will become apparent from the following description of preferred embodiments taken in conjunction with the accompanying drawings and description thereof, but variations and modifications may be made thereto without departing from the spirit and scope of the novel concepts of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The present disclosure will be more fully understood from the detailed description and the accompanying drawings. These drawings illustrate one or more embodiments of the present disclosure and, together with the written description, are used to explain the principles of the present disclosure. Wherever possible, the same reference numerals are used throughout the drawings to represent the same or similar elements of the embodiments, and wherein:

[0025] Figure 1 is a flowchart of a method for generating a designated standing posture diagram according to an exemplary embodiment of the present disclosure.

[0026] Figure 2 is a schematic diagram of a standing posture of a subject according to an exemplary embodiment of the present disclosure.

[0027] Figure 3Detailed description of the drawings is provided for describing a current standing posture diagram of a subject and a designated standing posture indication diagram according to an exemplary embodiment of the present disclosure.

[0028] Figure 4 A three-dimensional reconstruction model of a human body according to an exemplary embodiment of the present disclosure.

[0029] Figure 5 1 is a standard standing posture diagram and a body shape standing posture diagram of a subject according to an exemplary embodiment of the present disclosure.

[0030] Figure 6 The figure is a flow chart of a millimeter wave self-service scanning security inspection method according to an exemplary embodiment of the present disclosure.

[0031] Figure 7 Schematic diagram of artifacts between arms and torsos and between legs of different subjects according to an exemplary embodiment of the present disclosure.

[0032] Figure 8 is a schematic diagram of a millimeter wave self-service scanning security inspection system according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0033] Hereinafter, the present disclosure will be described more fully with reference to the accompanying drawings, in which exemplary embodiments of the present disclosure are shown. However, the present disclosure can be implemented in different embodiments and should not be construed as being limited to the embodiments described herein. These embodiments provided by the present disclosure are to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art. In the accompanying drawings, for clarity, the thickness and region of the layer may be magnified. Throughout the specification, the same reference numerals are used to represent the same elements. For different embodiments, elements may have different relationships and different positions.

[0034] The present disclosure proposes a method for generating a designated standing posture prompt image. The method for generating a designated standing posture prompt image provided by the present disclosure can be applied to places requiring security checks, such as airports and train stations, and can generate specific standing posture prompt images belonging to different inspected persons. In other words, different standing posture prompt images are generated according to the body characteristics of each inspected person, such as height, shortness, fatness, and thinness. Since a designated standing posture prompt image is generated for each inspected person, security check operations can be provided more accurately and conveniently.

[0035] like Figure 1As shown, the method for generating a designated standing posture prompt diagram comprises the following steps: in step S1, a standing posture diagram of a subject is obtained through a camera device; in step S2, a body shape standing posture diagram of the subject is obtained through the standing posture diagram of the subject; in step S3, based on the standard standing posture diagram, the body shape standing posture diagram of the subject is adjusted, and a standing posture prompt diagram of the subject is obtained, and the standing posture prompt diagram is used as the designated standing posture prompt diagram of the subject; in step S4, based on the standing posture diagram and body shape characteristics of the subject obtained through the camera device, a current standing posture diagram of the subject is obtained; in step S5, the designated standing posture prompt diagram and the current standing posture diagram of the subject are simultaneously displayed to the subject, wherein the designated standing posture prompt diagram can be different according to the body shape characteristics of different subjects.

[0036] In one embodiment of the present disclosure, when the subject is located in the examination channel, a camera device may be used to capture an image of the subject, preferably a standing posture image of the subject, such as Figure 2 The operation processing module 103 obtains the key point coordinates of the subject through processing by means of the key point positioning model, and then calculates the graphic standing posture diagram of the subject using the key point coordinates of the standing posture diagram of the subject.

[0037] The key point positioning model is based on a convolutional neural network model of an improved residual network. The convolutional neural network model may include: N residual blocks, a convolutional layer, a pooling layer, and a fully connected layer. In a preferred embodiment, a Resnet convolutional neural network with three residual blocks can be used as the backbone network, but the present disclosure does not limit the number of residual blocks, which can be more or less than three. The structure of each residual block is basically the same. The residual block contains two parts, one of which is a 3x3 convolutional layer and is processed with a batch normalization algorithm and a Relu activation function, and a 3x3 convolutional layer. The other part is a 1x1 convolutional layer, and the two parts are finally summed and processed with a batch normalization algorithm and a Relu activation function. In the residual block, a 3x3 convolutional layer with a step size of 2 is first passed, followed by a 3x3 convolutional layer with a step size of 1, to reduce the number of channels. The structure of the residual block effectively reduces the number of parameters and the amount of calculation of the convolution. Among them, the batch normalization algorithm also speeds up the convergence of the network and improves the training speed.

[0038] The operation processing module 103 in the embodiment of the present disclosure locates the key point positions of the human body of the subject through the key point positioning model. The key point positioning model is based on a convolutional neural network model, which is not limited to the VGG model or the Resnet model, etc. The most suitable model can be selected according to actual needs. At the same time, the key point positioning model also supports an integrated model formed by multiple models such as VGG and Resnet to improve the accuracy and robustness of positioning. The model selection mechanism can automatically select the most suitable key point positioning model according to different application scenarios and requirements to ensure that the best effect can be achieved in various environments. The key points include at least the head, shoulders, elbows, wrists, hips, knees, ankles, etc.

[0039] In the embodiments of the present disclosure, Figure 4 and Figure 5 As shown, the operation processing module 103 uses the key point coordinate information to calculate the body shape and posture diagram of the subject. The operation processing module 103 can detect important key points of the human body according to the posture positioning model; use a suitable difference or fitting algorithm to process the key points corresponding to the human body parts, generate contour curves or line segments of each part, and then splice the contour curves or line segments of these parts according to the structural order of the human body to form a complete human body shape and posture diagram. In a preferred embodiment, the extracted key points are also sorted and arranged according to the logical order of the human body parts, for example, the order of the head first, then the neck, the trunk, and finally the limbs. After the key point set is arranged, possible errors or abnormal key points are removed.

[0040] In the embodiments of the present disclosure, Figure 4 and Figure 5 As shown, according to the standard standing posture diagram and the key points of the human body, the translation distance and / or scaling factor of the key points are calculated, and the obtained body standing posture diagram is translated and scaled based on the translation distance and / or scaling factor. The operation processing module 103 can determine the translation distance based on the key points at the center position of the human body, such as the center point of the shoulder, so that the body standing posture diagram of the subject is aligned with the standard standing posture diagram in the plane position. The size measurement (such as height, shoulder width, etc.) is calculated through the key points, and the corresponding scaling factor is calculated. Then, the body standing posture diagram of the subject is enlarged or reduced according to the scaling factor, so that it matches the standard standing posture in overall size or local details (such as key point position), and the final human standing posture prompt diagram is generated.

[0041] Finally, in the embodiments of the present disclosure, Figure 3 , Figure 6 and Figure 8As shown, the operation processing module 103 can compare the current standing posture diagram of the examinee with the designated standing posture prompt diagram, and display the comparison result to the examinee through the display device, so as to guide the examinee to adjust the current standing posture of the examinee according to the designated standing posture prompt diagram to meet the requirements of the designated standing posture prompt diagram. For example, adjust the raising and / or lowering of the arms, the closing and / or separation of the legs, or the angle between the examinee's stretched arms, the angle between the open legs, etc. The designated standing posture prompt diagram varies according to the examinee's body shape. For example, the designated standing posture prompt diagram can generate different prompt images according to the height, shortness, fatness, thinness, etc. of the examinee.

[0042] In an embodiment of the present disclosure, the operation processing module 103 can also use the human body 3D reconstruction model to generate a projection image of the current standing posture of the subject according to the standing posture and body shape characteristics of the subject captured by the camera device, and the current standing posture prompt projection image is an image generated in real time by continuous calculation. In an exemplary embodiment of the present disclosure, the human body 3D reconstruction model can include binocular camera registration processing, point cloud stitching and denoising processing, point cloud plane mapping processing and GPU acceleration processing.

[0043] In a preferred embodiment, the main function of the binocular camera alignment technology is to align the cameras on both sides to the coordinate axes XYZ of the world coordinate system. For example, the three-dimensional space after alignment uses the direction of the correct standing posture of the human body as the Z axis, and the XY plane as the front of the correct standing posture of the human body.

[0044] In a preferred embodiment, the purpose of the point cloud stitching technology is to stitch the single-sided human body point cloud obtained after the registration of the two side cameras into a complete human body point cloud. The point cloud denoising technology can remove the noise in the point cloud to achieve a better human body display effect. The point cloud stitching algorithm includes a point cloud coarse registration algorithm and a point cloud fusion algorithm. In an exemplary embodiment, the algorithm first extracts key points from the point cloud data on both sides of the human body, and uses the point cloud coarse registration algorithm and key points to calculate the point cloud rigid body transformation matrix. Then the registered point cloud is clustered, screened and weightedly fused to obtain a complete human body point cloud after stitching. Finally, Gaussian filtering and SOR denoising algorithm are used to denoise the point cloud.

[0045] In a preferred embodiment, the main function of the point cloud plane mapping technology is to map the obtained 3D human point cloud model to a 2D plane for display and subsequent algorithm use. The front image of the human body in correct standing posture can be the XY plane in the coordinate system. This technology maps the human point cloud to the XY plane and retains the data on the Z axis, so that the world coordinates on the Z axis can be obtained based on the pixel coordinates while performing subsequent detection tasks on the 2D image.

[0046] In a preferred embodiment, all algorithms used in the human body 3D reconstruction technology are written as GPU kernel operators and processed on the GPU side to ensure the real-time performance of the algorithms.

[0047] In one embodiment of the present disclosure, the method for generating a specified standing posture prompt diagram further includes: generating a specified standing posture prompt diagram of a fixed size, scaling the current standing posture diagram to make it the same size as the specified standing posture prompt diagram, and Figure 1 The same reality is given to the examinee.

[0048] In the inspection channel, the inspection module is run to generate a fixed-size standing posture prompt image. The camera continuously shoots the inspected person. The inspection module is run to generate the current standing posture image and scale it to the same size as the specified standing posture prompt image. Figure 1 The two images are displayed simultaneously (for example, both images are projected onto the inner channel panel using a projector), and it is simultaneously identified whether the subject's standing posture meets the requirements of the specified standing posture prompt diagram. In other words, if the subject's current standing posture diagram does not meet the requirements of the specified standing posture prompt diagram, a color prompt is given, for example, the outline of the specified standing posture prompt diagram is displayed in red ( Figure 3 As shown in the left figure), the examinee is prompted to change his / her standing posture. If the examinee's standing posture meets the requirements of the specified standing posture prompt diagram, the outline of the specified standing posture prompt diagram will be displayed in green ( Figure 3 As shown in the figure on the right). Figure 8 As shown, the millimeter wave security scanner performs millimeter wave scanning on the inspected person and identifies and marks the suspicious object. The inspected person can undergo millimeter wave scanning in the inspection channel.

[0049] like Figure 6 As shown, in another embodiment of the present disclosure, a flow chart of a millimeter wave self-service scanning security inspection method is also disclosed, and the millimeter wave security inspection method includes the following steps:

[0050] S10 : While the inspected person 102 enters the inspection channel 109 defined by the millimeter wave security scanner, image data of the inspected person 102 is acquired by the camera device 104 , such as taking a photo or video of the inspected person 102 , and transmitted to the operation processing module 103 .

[0051] S11: the operation processing module 103 calculates the body characteristics of the examinee based on the image data of the examinee taken by the camera device;

[0052] S12: The operation processing module 103 generates a designated standing posture prompt diagram corresponding to the features of the subject 102 using the method for generating the designated standing posture prompt diagram in the present disclosure.

[0053] S13: The operation processing module 103 displays the designated standing posture prompt diagram and the current standing posture diagram to the examinee 102 through the display device 110 .

[0054] S14 : The camera device 104 continuously takes pictures or videos of the subject 102 , and sends the image data to the operation processing module 103 .

[0055] S15: The processing module 103 is run to determine whether the current standing posture of the examinee 102 is consistent with the specified standing posture prompt. Figure 1 To.

[0056] S16: Prompt the current standing posture and the specified standing posture Figure 1 If the current posture image completely falls within the outline of the specified posture prompt image, it is considered that the current posture of the inspected person meets the requirements (such as Figure 3 ).

[0057] S17: The current stance does not match the specified stance prompt Figure 1 If the position of the examinee 102 is not in the same position as the position of the specified posture prompt image, the examinee 102 is guided to adjust the posture through images and / or voice. For example, if a part of the examinee is beyond the outline of the specified posture prompt image, the examinee is given a corresponding reminder to change the action and make adjustments according to the outline of the specified posture prompt image until the projection of the current posture of the examinee is completely within the outline of the specified posture prompt image. The adjustment of the examinee's posture may include but is not limited to adjusting the raising and / or lowering of the arms and the closing and / or separation of the legs.

[0058] S18: The millimeter wave security scanner performs a millimeter wave scan on the inspected person and marks the location of the suspicious object carried by the inspected person 102 .

[0059] like Figure 8 As shown, the present disclosure also proposes a designated standing posture generation system for generating a prompt diagram of a designated standing posture, the designated standing posture generation system comprising: a camera device, configured to obtain image data of the standing posture and body shape of a subject; an operation processing module, configured to obtain key point coordinates of the standing posture diagram of the subject through the standing posture diagram of the subject, and calculate the body shape standing posture diagram of the subject using the key point coordinates; based on the standard standing posture diagram, adjust the body shape standing posture diagram of the subject, and obtain the standing posture prompt diagram of the subject, and use the standing posture prompt diagram as the designated standing posture prompt diagram of the subject, and based on the standing posture diagram and body shape characteristics of the subject, obtain the current standing posture diagram of the subject using a three-dimensional reconstruction model of the human body; and simultaneously display the designated standing posture prompt diagram and the current standing posture diagram of the subject to the subject, wherein the designated standing posture prompt diagram is different according to the body shape characteristics of different subjects.

[0060] The designated standing posture generation system can be applied in the millimeter wave detector 100 to perform millimeter wave safety detection on the subject 102. The camera device 104 of the designated standing posture generation system can include a visible light camera device, an infrared camera device, and a binocular depth camera device. The designated standing posture generation system can also include a projector 105. The designated standing posture generation system can also include a display board 106 for displaying a current standing posture diagram and a designated standing posture prompt diagram.

[0061] In one embodiment of the present disclosure, the operation processing module can generate key point coordinate information of the examinee in real time, and compare and analyze the detected key point coordinate information with the corresponding standards in the standard standing posture, so as to better guide the examinee to adjust the standing posture.

[0062] In one embodiment of the present disclosure, the operation processing module 103 can calculate the angle α between the line L between the i-th key point and the adjacent i+1-th key point of the subject and the line L' between the i-th key point and the adjacent i-1-th key point. The obtained angle α between the line L and the line L' is compared with the corresponding angle β between the key points of the human body in the standard standing posture. Based on the comparison result, the angle α between the obtained line L and the line L' is adjusted so that the difference between it and the angle β between the corresponding key points of the human body in the standard standing posture is within a range. For example, the angle between α degrees and β degrees is set not to exceed γ degrees, γ is set to 10, and i is a positive integer greater than 1. That is, the angle error of the angle between the line between the key points positioned by the standing posture model and the line between the key points of the standard standing posture can be controlled within 10° as a basis for whether the standard standing posture is met.

[0063] Figure 5 The schematic diagram of the angle of the connection line of the standard key points of the standing posture shown on the left side is as shown in the figure. The left shoulder point is set as the i-th key point. For the sake of clarity, it is called key point b. The i-1-th key point is called key point a, which is the left elbow point. The connection line between the left elbow point and the left shoulder point is line segment ab. The i+1-th key point is called key point c, which is the left chest point. The connection line between the key point b of the left shoulder point and the key point c of the left chest point is line segment bc. For example, the angle β between the key point connection line ab and the key point connection line bc is 45°. Figure 5 The diagram of the key point connection lines of the current standing posture is shown on the right side. Key point a, key point b, and key point c correspond to key point d, key point e, and key point f respectively. That is, key point d is the left elbow point, key point e is the left shoulder point, and f is the left chest point. The angle α between the key point connection line ef in the standard standing posture and the key point connection line de is 15°. The difference between the angle in the current standing posture and the angle in the standard standing posture exceeds the set value of γ of 10°, so it can be judged that Figure 5 The left hand part of the standing posture on the right is unqualified, but other positions are qualified, thus forming the information for adjusting the left hand posture.

[0064] In the method for generating a specified standing posture prompt diagram proposed in the present disclosure, as Figure 7 As shown in the figure, the artifacts between the arms and the torso, as well as the artifacts between the legs, will be reduced when the angle becomes larger, so the larger the angle, the better. However, if the angle of the arms and legs is too large and the person is required to stand for 2 seconds, it will make the examinee uncomfortable. On the other hand, the fixed footprints in the channel will indicate that the angle of the legs will be too small for tall people, and too large for short people. Therefore, it is very important to generate a unique standing posture for each examinee of different body shapes that does not affect the millimeter wave image and makes the examinee more comfortable.

[0065] First, by generating a customized standing posture based on the individual's body shape, it can ensure that the millimeter wave signal covers the whole body and reduce the detection blind spots caused by improper posture. For example, adjust the arm angle to ensure that the signal can effectively cover the waist and thighs. Secondly, an inappropriate standing posture may cause artifacts or false alarms in the scan, and the system may misjudge ordinary clothing wrinkles or body postures as potential threats. Through personalized standing posture prompts, the false alarm rate can be significantly reduced and the overall detection efficiency can be improved. Furthermore, the person being inspected can adjust his standing posture according to his own situation to avoid the discomfort caused by forced uniform posture. For example, short people may feel uncomfortable in a specific posture, and customized standing posture guidance can allow the person being inspected to cooperate with the detection more naturally. Finally, during the peak security inspection period, customized standing posture can speed up the scanning process, reduce adjustment time, and allow the system to adapt to the scanning needs of different groups of people more quickly.

[0066] The terms used herein are only used for exemplary purposes of the present disclosure and should not be interpreted as limiting the meaning or scope of the present disclosure. As used in this specification, unless a specific example is clearly indicated in the context, the singular form may include the plural form. Moreover, the expressions "including" and / or "including" used in this specification neither limit the shapes, numbers, steps, operations, components, elements and / or their groups mentioned, nor exclude the occurrence or addition of one or more other different shapes, numbers, steps, operations, components, elements and / or their groups, or add these.

[0067] As used herein, terms such as "first," "second," and the like are used to describe various components, assemblies, regions, and / or portions. These terms are used only to distinguish one component, component, region, layer, or portion from another component, component, region, or portion. Thus, a description of a first component, component, region, layer, or portion may also refer to a second component, component, region, or portion without departing from the scope of this disclosure.

[0068] The above description of the exemplary embodiments of the present disclosure is presented only for the purpose of illustration and description, and is not intended to be exhaustive or to limit the present disclosure to the precise form disclosed. In accordance with the above teachings, many modifications and variations are possible. The embodiments are selected and described to explain the principles of the present disclosure and its practical application, so that other technical personnel in the field can utilize the present disclosure and various embodiments, and have various modifications suitable for the specific purposes contemplated. Without departing from the spirit and scope of the present disclosure, replacement embodiments will become apparent to those skilled in the art to which the present disclosure belongs. Therefore, the scope of the present disclosure is limited by the appended claims rather than by the foregoing description and the exemplary embodiments described therein.

Claims

1. A method for generating a specified standing posture prompt diagram, the method comprising: Obtaining a standing posture image of the subject through a camera device; Obtaining a body shape and posture diagram of the subject through the posture diagram of the subject; Based on the standard standing posture diagram, adjusting the body shape standing posture diagram of the subject, and obtaining a standing posture prompt diagram of the subject, and using the standing posture prompt diagram as a designated standing posture prompt diagram of the subject; Based on the standing posture image and body characteristics of the subject obtained by the camera device, obtaining a current standing posture image of the subject; and The designated standing posture prompt image of the examinee and the current standing posture image are simultaneously displayed to the examinee, The designated standing posture prompt diagram may be different according to the body posture characteristics of different examinees.

2. The method according to claim 1, wherein: The method of obtaining the body shape and posture diagram of the subject includes: obtaining the coordinates of the key points of the subject based on the subject's posture diagram, and locating the key points of the subject's body using a key point positioning model, wherein the key point positioning model is based on a convolutional neural network model, and the convolutional neural network model includes multiple residual blocks, convolutional layers, pooling layers and fully connected layers.

3. The method according to claim 2, wherein: The key points of the human body include the head, shoulders, elbows, wrists, hips, knees, and ankles.

4. The method according to claim 3, wherein: The method of obtaining the body shape and posture diagram of the subject includes: obtaining the body parts corresponding to the body key points according to the located body key points, and connecting the obtained body parts to obtain the contour line of the current standing posture diagram of the subject.

5. The method according to claim 4, wherein: The method of obtaining the subject's body posture diagram also includes: calculating the translation distance and / or scaling factor of the key points according to the standard posture diagram and the key points of the human body, and performing translation and scaling operations on the obtained body posture diagram based on the translation distance and / or scaling factor.

6. The method according to claim 1, wherein: The obtaining of the current standing posture diagram of the examinee includes: obtaining the current standing posture diagram of the examinee using a three-dimensional human body reconstruction model, wherein the three-dimensional human body reconstruction model includes binocular camera registration processing, point cloud stitching and denoising processing, point cloud plane mapping processing and GPU acceleration processing.

7. The method according to claim 1, wherein: The method for generating a designated standing posture prompt diagram also includes: comparing the current standing posture diagram of the examinee with the designated standing posture prompt diagram, and displaying the comparison result to the examinee through a display device to guide the examinee to make his current standing posture diagram fall within the range of the designated standing posture prompt diagram.

8. The method according to claim 1, wherein: The method for generating a designated standing posture prompt diagram also includes: when the front standing posture diagram of the subject is within the range of the designated standing posture prompt diagram, the designated standing posture prompt diagram is displayed in a first color; when at least a portion of the front standing posture diagram of the subject is outside the range of the designated standing posture prompt diagram, the designated standing posture prompt diagram is displayed in a second color, and the second color is different from the first color.

9. The method according to claim 1, wherein: The method for generating a designated standing posture prompt image also includes: generating a designated standing posture prompt image of a fixed size, scaling the current standing posture image to make it the same size as the designated standing posture prompt image, and displaying it to the examinee together with the designated standing posture prompt image.

10. A millimeter wave self-service scanning security inspection method, characterized in that: The millimeter wave security inspection method comprises the following steps: Generate a specified standing posture prompt diagram according to the method described in any one of claims 1 to 9; Continuously capturing images of the subject using a camera device; Determine whether the current standing posture of the examinee is consistent with the designated standing posture prompt diagram; When the current standing posture of the examinee is inconsistent with the designated standing posture prompt diagram, the examinee is reminded to adjust the standing posture; When the current standing posture of the person being inspected is consistent with the designated standing posture prompt diagram, the millimeter wave security scanner is used to inspect the person being inspected, and the corresponding position of the suspicious object is identified and marked.

11. A designated standing posture generating system for generating a prompt image of a designated standing posture, comprising: A camera device configured to obtain image data of the standing posture and body shape of the subject; The running processing module is configured to obtain a body posture diagram of the examinee through the examinee's standing posture diagram; adjust the examinee's body posture diagram based on the standard standing posture diagram, and obtain the examinee's standing posture prompt diagram, and use the standing posture prompt diagram as the examinee's designated standing posture prompt diagram, and obtain the examinee's current standing posture diagram based on the examinee's standing posture diagram and body shape characteristics; and simultaneously display the examinee's designated standing posture prompt diagram and the current standing posture diagram, wherein the designated standing posture prompt diagram is different according to the body shape characteristics of different examinees.

12. The designated posture generation system according to claim 11, wherein: The designated standing posture generating system further comprises a display panel for displaying a current standing posture diagram and a designated standing posture prompt diagram.

13. The designated posture generation system according to claim 11, wherein: The camera device includes a visible light camera device, an infrared camera device, and a binocular depth camera device.

14. The system for generating a designated standing posture according to any one of claims 11 to 13, wherein: The operation processing module includes a key point positioning model for determining the key point coordinates of the subject's standing posture diagram, and the key point positioning model is based on a convolutional neural network model, which includes multiple residual blocks, convolutional layers, pooling layers and fully connected layers.

15. The system for generating a designated standing posture according to any one of claims 11 to 13, wherein: The operation processing module is configured to obtain the coordinates of key points of the subject based on the subject's standing posture diagram, obtain the human body parts corresponding to the human body key points according to the located human body key points, connect the obtained human body parts and obtain the contour lines of the subject's current standing posture diagram, and calculate the translation distance and / or scaling factor of the key points based on the standard standing posture diagram and the human body key points, and perform translation and scaling operations on the obtained body shape standing posture diagram based on the translation distance and / or scaling factor.

16. The system for generating a designated standing posture according to any one of claims 11 to 13, wherein: The operation processing module is also configured to simultaneously display the current standing posture diagram and the designated standing posture prompt diagram to the examinee to guide the examinee to adjust the current standing posture, wherein adjusting the standing posture includes adjusting the raising and / or lowering of the arms and the closing and / or separation of the legs.

17. A millimeter wave self-service scanning security inspection system, wherein: The millimeter wave security inspection system includes a designated standing posture generation system, a millimeter wave security inspection instrument and an inspection channel as described in any one of claims 11 to 16, wherein the millimeter wave security inspection instrument is configured to perform a millimeter wave scanning inspection on the inspected person to identify and mark suspicious objects, and the inspection channel is configured for the inspected person to undergo a millimeter wave scanning inspection therein.

Images