A dangerous goods coordinate mapping method based on a millimeter wave security inspection image

Abstract

The millimeter wave imaging device has the characteristics of high imaging resolution and penetrating clothes, which makes it have the advantage of finding the hidden items in clothes, but this characteristic will also cause the problem of passenger privacy exposure. In order to avoid this problem, the mannequin image marked with the position of dangerous goods should be displayed on the security officer interface displayed to the public. Since the mannequin image generally adopts a fixed model, it is obviously impossible to accurately map the dangerous goods coordinates from the millimeter wave image to the mannequin image through simple scaling mapping. In order to solve the above problem, the application proposes a dangerous goods coordinate mapping method based on human body key point information. The method finely divides the region to which the dangerous goods coordinates belong through the relative position relationship between the dangerous goods coordinates and the key point coordinates, and then adopts the corresponding mapping method based on the characteristics of the region to carry out coordinate mapping, so as to obtain accurate mapping result.

Description

Technical Field

[0001] This invention belongs to the field of image processing, specifically relating to a method for mapping the coordinates of hazardous materials in millimeter-wave security inspection images. Background Technology

[0002] Millimeter-wave imaging equipment boasts high resolution and the ability to penetrate clothing, giving it an advantage in detecting items hidden within garments. However, this advantage also raises concerns about passenger privacy. To avoid this issue, the security personnel's interface displayed to the public should show a mannequin image with the location of dangerous items marked. However, no publicly published papers or patents have yet described a method for mapping the location information of dangerous items onto the mannequin image. Since mannequin images typically use a fixed model, while passenger body shapes vary greatly, and the posture of passengers undergoing imaging checks inevitably deviates from the posture required by the inspection device, simple scaling and mapping is clearly insufficient to accurately map the coordinates of dangerous items from the millimeter-wave image onto the mannequin image. Summary of the Invention

[0003] To overcome the shortcomings of existing technologies, this invention proposes a method for mapping the coordinates of hazardous materials based on key human body information. This method finely divides the area to which the hazardous material belongs by the relative positional relationship between its coordinates and the coordinates of key points. Then, based on the characteristics of the area, an appropriate mapping method is used to perform coordinate mapping, thereby obtaining accurate mapping results. By mapping the coordinates of hazardous materials from the original millimeter-wave security inspection image onto the mannequin image, passenger privacy is protected, and the automation level and execution efficiency of the security inspection process are improved. Specifically, the method includes the following steps:

[0004] 1) Human body modeling

[0005] The overall flow of the method of the present invention is as follows: Figure 1 As shown, the core idea of ​​the algorithm is to compare the coordinates of hazardous materials with the key point coordinates of millimeter-wave human images. Through a series of criteria, it identifies the specific human body part where the hazardous material is located. Then, based on the characteristics of the human body part, it uses an appropriate mapping method to obtain the reasonable coordinates of the item on the mannequin image. This part-based mapping method effectively overcomes the problem of inaccurate mapping caused by differences in human body shape and posture.

[0006] This invention models the human body using key parts and joints, as shown in Figures 2(a) and 2(b). Figure 2(a) shows the key points marked on a millimeter-wave image, and Figure 2(b) shows the key points marked on a doll image. A total of 13 key points are involved: head, left shoulder (LS), right shoulder (RS), left waist (LW), right waist (RW), left knee (LK), right knee (RK), left foot (LF), right foot (RF), left elbow (LE), right elbow (RE), left hand (LH), and right hand (RH). The key points on the doll correspond one-to-one with the key points on the human body, distinguished by adding an apostrophe ('). Based on these key points, the human body is divided into the following eight regions: 1. Torso; 2. Thigh; 3. Left calf; 4. Right calf; 5. Left upper arm; 6. Left forearm; 7. Right upper arm; 8. Right forearm. Since the head, face, and hands are generally not obstructed during security checks, they do not need to be inspected using millimeter-wave technology. Therefore, even if a dangerous goods alarm is triggered, coordinate mapping is not required.

[0007] 2) Data Preparation

[0008] Collect and input key point information data from millimeter-wave images and puppet images, and complete the data preparation;

[0009] 3) Area Determination Method

[0010] This invention uses seven sets of judgment conditions to determine the area to which the dangerous goods belong. The area can be divided into two categories: one is the human torso and thigh, which are two surface areas with large projected areas, and the other, such as the forearm and lower leg, are line areas. The judgment methods and coordinate mapping methods for these two types of areas are different.

[0011] The method and process are as follows Figure 1 As shown, the first step is to determine whether the hazardous material is located on the human torso (Judgment 1). The human torso area is defined as... Figure 3 The quadrilateral region enclosed by the four points LS2, RS2, LW2, and RW2. This area is wider than the area enclosed by the input human body keypoints LS, RS, LW, and RW. This is because the center coordinates of the hazardous material located on the torso may be situated on the side of the torso. Due to potential errors in the keypoint coordinates, the center coordinates of the hazardous material can easily extend beyond the area enclosed by the keypoints. The coordinates of LS2, RS2, LW2, and RW2 are calculated as follows:

[0012] in This is the horizontal offset. This represents the vertical offset. Algorithms such as the ray method and the angle method can be used to easily determine whether the coordinates of the hazardous material are located inside the quadrilateral.

[0013] If the hazardous material is not located on the torso, then determine whether the hazardous material is located in the thigh area (Judgment 2). The definition of the thigh area is similar to that of the torso area, such as... Figure 3 The quadrilateral region enclosed by LW2, RW2, LK2, and RK2. As shown. The coordinates of LK2 and RK2 are calculated as follows:

[0014] like Figure 4 As shown, suppose a certain dangerous goods D The center coordinates are If its ordinate Less than knee level If the dangerous item is located on the lower leg, then the dangerous item should be located on the lower leg (judgment 3).

[0015] Define the distance function: for A Point and B The distance between points.

[0016] If the hazardous material is closer to the left knee, that is... If the location is not specified, the dangerous item is determined to be located on the left calf; otherwise, it is located on the right calf (judgment 4).

[0017] If the hazardous materials are not present Alternatively, if the hazardous material is located in the lower leg area, it should be placed in the arm area. If the hazardous material is closer to the left elbow joint (LE), then... If the angle is right, the hazardous material is located on the left arm; otherwise, it is located on the right arm (judgment 5). Judgment 6 determines whether the hazardous material belongs to the left forearm or left upper arm, and is achieved by comparing the angle between the two arms. Let... From A Click B Vector of a point ( O (where the origin is) Dangerous goods D If the hazardous material is located on the forearm, then it is located on the upper arm; otherwise, it is located on the upper arm. Judgment 7 is the same as judgment 6, except that the judgment location changes from the left arm to the right arm.

[0018] 4) Coordinate mapping

[0019] As mentioned earlier, the torso and thighs are considered as a planar region. Hazardous items located within this region need to be transformed as a whole along this plane to achieve a better indication effect. Therefore, this invention employs a projection transformation method to map the coordinates of hazardous items. Taking the human torso as an example, the coordinates of the key point group (LS, RS, LW, RW) on the human body are as follows: The coordinates of the key points (LS', RS', LW', RW') on the doll are as follows: The transformation matrix M between these two sets of coordinates is:

[0020] The matrix has 8 unknowns, which can be solved by solving the following linear equations:

[0021] Solving for the M matrix yields eight unknowns. Once the M matrix is ​​obtained, the hazardous materials... Corresponding points on the doll The coordinates are:

[0022] For dangerous items located in the thigh area, the transformation formula is the same as the above formula, except that the transformation matrix is ​​calculated from the key point group (LW, RW, LK, RK) on the human body and the key point group (LW', RW', LK', RK') on the doll.

[0023] For other hazardous items located on the limbs, the coordinate mapping problem is equivalent to mapping the coordinates of the hazardous items in the original diagram onto the figure diagram, so that their spatial geometric relationship with adjacent line segments (i.e., the lower leg or forearm, etc.) can be reflected in the figure diagram (e.g., ...). Figure 4 As shown, U and V Represents the endpoint of a segment of a limb. D It is a dangerous good. W Coordinates of dangerous goods UV The perpendicular point of a line segment O The origin of the original image is (0,0). To solve this problem, a method based on similarity transformations (including translation, rotation, and scaling) is used to map the coordinate points. First, based on the original image... U , V , D The coordinates of the three points can be easily obtained. W Point coordinates, according to the principle of similarity transformation, the corresponding W ' The point is also located in U ' V ' On the line segment, and satisfying the scaling relationship:

[0024] in The magnitude represents the vector, which is also the length of the line segment. Therefore The coordinates can be calculated in the following way:

[0025] By calculating vectors and vector Pointing angle difference The rotation matrix in the similarity transformation can be obtained as follows:

[0026] therefore, The coordinates can be obtained using the following formula:

[0027] in O’ The origin of the coordinate system for the puppet diagram is... This is a distance adjustment factor used to adjust the distance between the mapped hazardous object and its limbs based on the thickness of the doll model, ensuring that the mapped coordinates always lie on the doll's body. In extreme cases, it takes... Then D ' With W ' Overlap means that the hazardous material is always mapped onto the limb segments.

[0028] The beneficial effects of this invention are as follows:

[0029] (1) The dangerous goods are processed in different areas to achieve high-precision coordinate mapping of the dangerous goods coordinates from the original map to the puppet map.

[0030] (2) There are no restrictions on the gender, age, body type, etc. of the passengers being tested, and accurate coordinate mapping can still be achieved even when the posture of the passengers being tested is not standard.

[0031] (3) Complete coverage of the human body except for the necessary areas (above the shoulders).

[0032] (4) When applied to millimeter-wave human body security inspection systems, it can play a role in protecting passenger privacy. Attached Figure Description

[0033] Figure 1 This is a flowchart of the method of the present invention.

[0034] Figures 2(a) and 2(b) are examples of defining key points on the human body.

[0035] Figure 3 A diagram illustrating the area where hazardous materials are located.

[0036] Figure 4 This is a schematic diagram of the limb mapping method.

[0037] Figures 5(a1)-5(d3) This is an example diagram showing the coordinate mapping results for hazardous materials. Detailed Implementation

[0038] The technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0039] The four specific implementation examples are shown in Figures 5(a1)-5(d3), including coordinate mapping results of two sets of frontal human images of different body types and two sets of back images of different body types. The points on the connecting lines in the figures correspond to the markings in Figures 2(a) and 2(b), representing the locations of key human body points manually marked based on the original images (including millimeter-wave images and mannequin images). The points outside the connecting lines in the figures are hazardous material markers, used only for demonstration purposes and do not necessarily represent the actual presence of hazardous materials at the corresponding locations in the original images. Each set of results uses two different mannequin models with different body proportions and significantly different angles of arm and leg openings.

[0040] In this embodiment, the input millimeter-wave image size is 512*1024. Based on the proportion of the human body in the image, and The values ​​are all 25 pixels. Because the doll's limbs are more slender and longer than those of a normal person, to prevent dangerous items located on the limbs from being mapped outside the doll's body, a distance adjustment factor is used. Set it to 0.5.

[0041] As can be seen from the results in the figure, the method of the present invention can obtain good mapping results for individuals of different body types and different doll models, and the dangerous items located on the limbs are also correctly mapped onto the dolls. It is worth noting that the human arm postures in each group of millimeter-wave images are not standardized, but good mapping results are still obtained on the doll images, indicating that the method has strong adaptability to posture errors.

[0042] This invention is not limited to the specific embodiments described above, and various modifications and variations are possible. Any modifications, equivalent substitutions, or improvements made to the above embodiments based on the technical essence of this invention should be included within the scope of protection of this invention.

Claims

1. A method for mapping the coordinates of dangerous goods based on millimeter-wave security inspection images, characterized in that: By comparing the coordinates of hazardous materials with the key point coordinates of millimeter-wave human images, the specific human body part where the hazardous material is located is identified. Then, the coordinates of the item on the mannequin image are obtained by mapping the human body part. The specific steps include the following: 1) Human body modeling The human body is modeled by key parts and joints, the regions are divided, the key parts and joints are set as key points, and the key points are marked on the puppet image accordingly. There are 13 key points in total: Head, Left Shoulder (LS), Right Shoulder (RS), Left Waist (LW), Right Waist (RW), Left Knee (LK), Right Knee (RK), Left Foot (LF), Right Foot (RF), Left Elbow (LE), Right Elbow (RE), Left Hand (LH), and Right Hand (RH). The key points on the puppet correspond one-to-one with the key points on the human body. The area is divided into the following 8 regions: human torso, thigh, left calf, right calf, left upper arm, left forearm, right upper arm, and right forearm. The regions are divided into two categories: one is the torso and thigh, which are surface regions, and the other is the forearm, upper arm, and calf, which are line regions. 2) Data Preparation This includes the input and data preparation of key point information for millimeter-wave images and puppet images; Key point information in millimeter-wave images; Key point information of the doll image; 3) Area Determination Based on the location information of the hazardous materials, and based on the human body region division in step 1), the human body region where the hazardous materials are located is determined. The specific method for determining the region is as follows: First, it is determined whether the hazardous materials are located in the surface region of the torso. A quadrilateral region enclosed by four points is defined in the torso region. This region is wider than the region enclosed by the key points of the human body, left shoulder, right shoulder, left waist, and right waist. The ray method or the corner method is used to determine whether the coordinates of the hazardous materials are located inside the quadrilateral region of the torso. If the hazardous material is not located on the torso, determine if it is located in the thigh region. Similarly, define a quadrilateral region enclosed by four points in the thigh region. This region is wider than the region enclosed by the key points of the body: left waist, right waist, left knee, and right knee. Use the ray method or the corner method to determine if the coordinates of the hazardous material are located inside the quadrilateral region of the thigh. If the ordinate of the center coordinate of the hazardous material is smaller than the ordinates of the left and right knees, determine if the hazardous material is located in the lower leg region. Calculate which distance the hazardous material is closer to, the left or right lower leg. If the hazardous material is not in any of the above regions, it should be located in the arm region. If the hazardous material is closer to the left elbow, it is located in the left arm; otherwise, it is located in the right arm. Use the comparison of the angles to determine if the hazardous material is located in the forearm or upper arm. If the angle between the vectors from the hazardous material to the elbow and the vector between the hazardous material to the hand is smaller than the angle between the vectors from the hazardous material to the elbow and the vector between the hazardous material to the shoulder, the hazardous material is located in the forearm; otherwise, it is located in the upper arm. 4) Coordinate mapping By mapping the key points of the human body to the key points on the corresponding puppet image, the coordinates of the hazardous materials are mapped onto the puppet image. For the surface areas of the torso and thighs, a projection transformation method is used to map the coordinates of the hazardous materials. For the line areas of the forearm, upper arm, and lower leg, a similarity transformation-based method is used to map the coordinate points and map the coordinates of the hazardous materials onto the puppet image, so that the spatial geometric relationship between the hazardous materials and the adjacent line areas can be reflected in the puppet image. For the surface region of the torso, let's define a key point group on the human body. The coordinates are as follows: Key points on the corresponding doll The coordinates are as follows: The transformation matrix M between these two sets of coordinates is: The matrix has 8 unknowns, which can be solved by solving the following linear equations: Solving for the M matrix yields eight unknowns; once the M matrix is ​​obtained, the hazardous materials... Corresponding points on the doll The coordinates are: For hazardous materials located in the thigh area, the transformation formula is the same as the formula above, and the transformation matrix consists of key points on the human body. Key point group on the doll Calculated; For other hazardous materials located on the limbs, the coordinate mapping problem is equivalent to mapping the coordinates of the hazardous materials in the original diagram to the puppet diagram, so that the spatial geometric relationship between the hazardous materials and the adjacent line segments can be reflected in the puppet diagram. The coordinate point mapping is performed using a method based on similarity transformation.

2. The method for mapping the coordinates of dangerous goods based on millimeter-wave security inspection images according to claim 1, characterized in that: The methods for similarity transformation include translation, rotation, and scaling transformations.

Images