Method and equipment for detecting objects bound and hidden in human body

Abstract

The invention discloses a method for detecting objects bound and hidden in a human body. The method comprises the steps that a visible light image and an infrared image which are synchronously collected and pass through pedestrians in a detection area are acquired, and image registration is carried out; on the basis of a segmentation result and a key point detection result on the visible light image, reducing a misrecognition probability when target detection is carried out on the infrared image, and constructing a trained deep learning target detection model; and performing target detection in the registered infrared image by adopting the trained model, respectively acquiring an article exposure proportion of each article area pixel point to a human body area pixel point, respectively judging whether the article exposure proportion of each article area is greater than a preset article exposure proportion threshold value, and if so, determining that the article exposure proportion of each article area is greater than the preset article exposure proportion threshold value. And if yes, judging that articles are bound in the human body area. According to the invention, based on a dual-light image registration technology, inhibition of a human body area which is easy to identify mistakenly is realized through key point detection, and in combination with article exposure proportion discrimination, the adaptive capacity and detection precision of various human body binding scenes can be significantly improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent security inspection technology, and more specifically, to a method and device for detecting objects hidden by a human body. Background Art

[0002] In order to evade customs supervision, some criminals use the method of "hiding with human flesh" to illegally transport or carry items that are prohibited or restricted from entering or leaving the country, in order to evade customs inspection and make huge profits. At present, customs inspection mainly relies on the following two methods.

[0003] One is for customs officers to observe the external clothing, movements and expressions of the person with their naked eyes, and to conduct individual inspections of suspicious persons based on their experience. However, this method requires a high level of experience from customs officers, and relying solely on experience can easily lead to misidentification;

[0004] The second is to use more advanced instruments for inspection, such as terahertz security gates and millimeter wave security gates. However, these more advanced instruments also have their own technical defects, which are discussed in detail below.

[0005] "Terahertz" electromagnetic waves are between the infrared frequency band in the optical domain and the millimeter wave frequency band in the microwave domain. They are a mysterious electromagnetic wave with a spectrum range of 0.1 to 10 THz (terahertz) and a wavelength of 0.03 to 3 mm. They are mainly based on the interaction between terahertz waves and matter for imaging. Although they can penetrate clothing and some non-metallic materials to detect prohibited items hidden under clothing. However, the penetration depth of terahertz waves is still relatively limited, and they are easily affected by environmental factors during transmission, such as absorption and scattering of substances such as moisture and air, as well as electromagnetic interference. The generally low power is also a major problem facing terahertz technology. In the process of imaging a two-dimensional object, the energy of the terahertz source will be dispersed to various points on the two-dimensional plane, resulting in lower terahertz energy that can be received by a single detection point. This directly affects the sensitivity and resolution of the imaging, making the imaging effect less than ideal. The resolution is often less than 256*192PPI, which is not easy to detect, and has great limitations in the application field of human body binding and concealment detection. Moreover, the screening efficiency of "terahertz" detection means is low, and it takes an average of 2-5 minutes to screen a person. In addition, the detection distance of terahertz technology is relatively short, making it difficult to carry out long-distance transmission and imaging. The hardware installation is also relatively bulky and occupies a large area.

[0006] Millimeter wave security gates use millimeter wave technology to scan the human body through harmless electromagnetic waves to form a three-dimensional image, which is then processed by an algorithm to identify abnormal objects. Millimeter waves have a shorter wavelength than terahertz waves and have higher penetration and better imaging effects, but they still face similar limitations, including large equipment size and complex installation.

[0007] These existing technologies face certain challenges in handling complex binding and hiding situations and improving recognition accuracy, and therefore further technological innovation and optimization are urgently needed. Summary of the invention

[0008] In response to at least one defect or improvement need in the prior art, the present application provides a method and device for detecting objects hidden on the human body, which is used to at least overcome some technical defects of terahertz security gates and millimeter wave security gates and realize real-time detection of objects hidden on the human body.

[0009] To achieve the above objectives, in a first aspect, the present application provides a method for detecting objects hidden on a human body, comprising:

[0010] acquiring simultaneously collected visible light images and infrared images of pedestrians passing through a detection area;

[0011] Performing image registration on the infrared image and the visible light image;

[0012] Based on the segmentation results and key point detection results on the visible light image, the probability of misidentification when detecting targets on the infrared image is reduced, and a trained deep learning target detection model is constructed;

[0013] The trained deep learning target detection model is used to perform target detection in the registered infrared image, and the object exposure ratio of each object area pixel point to the corresponding human body area pixel point is obtained respectively, and it is judged whether the object exposure ratio of each object area is greater than a preset object exposure ratio threshold value. If greater, it is determined that the corresponding human body area has hidden an object.

[0014] Furthermore, based on the key point detection result on the visible light image, reducing the probability of misidentification when detecting the target on the infrared image includes:

[0015] Perform key point detection on the visible light image and obtain the key point coordinate set;

[0016] Based on the key point coordinate set, obtaining key point detection results of a human body region that is easily misidentified, including a navel and a crotch;

[0017] A mask is placed on the human body easily misidentified area to shield the content in the area, so as to reduce the probability of misidentification when performing target detection on the infrared image.

[0018] Furthermore, the method of obtaining the trained deep learning target detection model includes:

[0019] Collect videos of people hiding objects in multiple scenes, convert the videos into pictures at preset frame rate intervals, remove duplicates, and obtain image data sets;

[0020] Segment and exclude the non-human area on the image of the image data set, shield the content in the human body easily misidentified area, and then label the object;

[0021] After data enhancement processing is performed on the image dataset, the YOLOv8 model is used for training to obtain the trained deep learning target detection model, which specifically includes:

[0022] The image dataset after data enhancement processing is divided into a training set and a test set according to a preset ratio;

[0023] Input the training set into the YOLOv8 model to generate target detection results, target segmentation results and key point positions;

[0024] In each round, the comprehensive loss value between the predicted image and the annotated image is calculated based on the loss of target detection, the loss of target segmentation, and the loss of key point detection; the loss calculation of target detection includes one or more of position loss, category loss, and confidence loss; the loss calculation of target segmentation includes comparing the difference between the predicted segmentation mask and the true mask; the loss calculation of key point detection includes the difference between the predicted key point position and the true key point position;

[0025] Based on the comprehensive loss value, gradient backpropagation is performed to optimize the parameters of the model, and the iteration is continued until the model parameters converge to obtain a deep learning target detection model;

[0026] The test set is used to evaluate the deep learning target detection model, and relevant hyperparameters are adjusted according to the evaluation results to obtain the trained deep learning target detection model.

[0027] Furthermore, the method for acquiring the easily misidentified area of ​​the human body includes:

[0028] Through the coordinates of the shoulders and hips, the coordinates of the waist and belly button are obtained. The relevant formulas include:

[0029] Y Waist =Y S_center *(1-k)+Y H_center *k;

[0030]

[0031] Among them, Y S_center Indicates the vertical coordinate of the center point of the shoulder; Y H_center represents the ordinate of the hip center point; k represents the proportional factor representing the position of the waist relative to the shoulder center and the hip center; Y Waist Indicates the vertical coordinate of the waist; Y Navel represents the vertical coordinate of the belly button; d represents the vertical distance from the belly button to the waist;

[0032] The area enclosed by the vertical and horizontal lines drawn on the vertical coordinate of the belly button, the horizontal coordinate of the shoulders on both sides, and the vertical coordinate of the center point of the hips is determined as the human body's easily misidentified area.

[0033] Furthermore, the determination method of the object exposure ratio of the object pixel points to the corresponding human body area pixel points includes:

[0034] In the registered infrared image, the total number of pixels in each object area is obtained respectively;

[0035] Obtain the total number of human body area pixels on the corresponding human body area;

[0036] The object exposure ratio of the object area=(the total number of pixels in the object area) / (the total number of pixels in the human body area);

[0037] If the item exposure ratio is greater than the preset item exposure ratio threshold, it is determined that the corresponding human body area hides the item; otherwise, it is determined that the corresponding human body area does not hide the item.

[0038] Furthermore, the performing image registration on the infrared image and the visible light image includes:

[0039] Extracting several groups of corresponding feature points from the infrared image and the visible light image and performing normalization processing, obtaining normalized control points, constructing a matrix representing the normalized control point set, and expanding the normalized control point set into homogeneous coordinates;

[0040] The inverse matrix of the matrix is ​​obtained by using the singular value decomposition method to determine the linear independence of the points. If the linear independence condition is met, the matrix calculation is continued.

[0041] Based on the correspondence between the normalized matrix, the inverse matrix and the original feature point set, matrix operations are performed on the inverse matrix and the matrix to obtain a transformation matrix from the visible light image to the infrared image, and the point coordinates in the visible light image are converted to the corresponding positions in the infrared image to achieve alignment of the visible light image with the infrared image.

[0042] Furthermore, it also includes:

[0043] It is confirmed that the facial features of the pedestrian match a preset database including facial information of persons with a history of kidnapping, and an alarm message is automatically issued.

[0044] Furthermore, it also includes:

[0045] It is determined that the object is hidden in the corresponding human body area and an alarm message is automatically issued.

[0046] Furthermore, the non-human body area includes one or more of a backpack, a shoulder bag, a hat, a skirt and a coat.

[0047] In a second aspect, the present application provides a device comprising at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program, and when the computer program is executed by the processing unit, the processing unit is enabled to perform the steps of any of the aforementioned methods for detecting objects hidden on a human body.

[0048] In general, the above technical solutions conceived by the present application can achieve the following beneficial effects compared with the prior art:

[0049] (1) This application integrates dual-light imaging technology and gives full play to the complementary advantages of dual-light images, thereby effectively overcoming the limitations of traditional single-modality recognition methods and achieving higher detection accuracy. In visible light images, this application applies human body detection, segmentation, and key point detection technology. Key point detection technology is used to accurately locate key parts of the human body and can effectively shield areas that are prone to misidentification, such as the hem (referring to the lower edge of the clothes, usually located near the crotch or belly button), thereby improving the accuracy of segmentation and target detection. Infrared images rely on their imaging principles based on heat distribution and temperature difference to focus on the detection of illegal objects. By applying the segmentation, key point detection and detection results in visible light images to infrared images, and using the advantages of infrared thermal imaging to reduce background interference, combined with the identification of the proportion of exposed objects, the adaptability, detection accuracy and detection efficiency of various illegal objects tied to the human body can be significantly improved.

[0050] (2) This application introduces the hem area suppression technology, which accurately identifies the hem area through key point detection, and then masks the area to a certain extent, thereby reducing the probability of thermal imaging misidentification. The hem area suppression technology improves the accuracy of target detection, especially for the application scenario of illegal items hidden in human bodies, and can significantly reduce the false detection rate of illegal items hidden in human bodies.

[0051] (3) The present application can determine whether illegal items are hidden by the ratio of the total number of pixels in each item area to the total number of pixels in the human body area. If this ratio exceeds a preset threshold, it is determined that illegal items are hidden in the human body area; if this ratio is lower than the threshold, it is considered that no illegal items are hidden. This threshold determination method for the ratio of exposed items effectively improves the accuracy and reliability of illegal item detection.

[0052] (4) This application segments and excludes areas such as backpacks, shoulder bags, hats, skirts and coats on the images in the image data set, and then labels the illegal items, thereby effectively reducing the problem of misidentification and improving the detection accuracy of illegal items.

[0053] (5) This application integrates face recognition technology, and stores the facial information of people with a history of binding and concealing items in a database. When the facial features of the perpetrator are detected to match the records in the database, an alarm will be automatically triggered. This function significantly enhances the ability to identify potential offenders and improves the reliability and efficiency of the detection of binding and concealing illegal items. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0055] Figure 1 A core flow chart of a method for detecting objects hidden in a human body provided in an embodiment of the present application;

[0056] Figure 2 A flow chart of a method for detecting dangerous goods (illegal items) hidden in customs using dual-light images based on face recognition and hem suppression provided in an embodiment of the present application;

[0057] Figure 3 This is one of the thermal imaging effect diagrams provided in the embodiment of the present application;

[0058] Figure 4 The second thermal imaging effect diagram provided in the embodiment of the present application;

[0059] Figure 5 A block diagram of a device suitable for implementing the above-described method for detecting objects hidden on a human body provided in an embodiment of the present application. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as they do not conflict with each other.

[0061] The terms "first", "second" or "nth" in the specification, claims or drawings of the present application are used to distinguish different objects rather than to describe a specific order. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices.

[0062] The existing terahertz security gates and millimeter wave security gates both have some technical defects, which are discussed in detail as follows.

[0063] First, the energy of terahertz waves is low, which makes the detector less sensitive when receiving signals, thus affecting the clarity and details of the image and reducing the accuracy of detection. Secondly, the penetration of terahertz waves is limited. Although it can penetrate some non-metallic materials, the penetration depth is relatively shallow and is easily affected by material absorption and scattering. Furthermore, the resolution of terahertz imaging is usually low, making it difficult to provide high-precision images, and it may not be possible to clearly distinguish small or complex items. In addition, terahertz waves are easily absorbed and scattered by environmental factors such as moisture and air, as well as electromagnetic interference during transmission, resulting in reduced image quality. Finally, the equipment size of terahertz security doors is often large, occupies a large area, and has high production and maintenance costs, which limits their widespread deployment in practical applications.

[0064] The effective detection distance of millimeter-wave security gates is usually short, which limits their application in large-scale or long-distance security inspections. Although millimeter-wave technology has a high image resolution, it may not be able to achieve high accuracy requirements in the detection of complex backgrounds or objects with rich details. The equipment and maintenance costs of millimeter-wave security gates are also high, which may be a limiting factor for users with limited budgets. In addition, the installation and maintenance of millimeter-wave equipment are relatively complex, requiring professional technical support and environmental requirements, which increases the difficulty of use and overall cost. In certain environmental conditions, such as high humidity or dusty environments, millimeter-wave technology may also perform poorly, resulting in reduced image quality.

[0065] In view of this, the present application proposes a method and device for detecting items hidden on the human body, which is used to at least overcome some technical defects of terahertz security gates and millimeter wave security gates, and achieve comprehensive detection of illegal items hidden on the body with high adaptability, high precision and high detection efficiency.

[0066] The overall technical idea of ​​the embodiment of the present application is as follows: First, the visible light image and the infrared image are accurately aligned using image registration technology. Subsequently, a specially designed recognition algorithm is applied to the registered image to give full play to the complementary advantages of the dual light image to improve the overall detection performance. In the visible light image, this embodiment applies human body detection, segmentation and key point detection technology. The key point detection technology is used to accurately locate various parts of the human body, calculate the area of ​​the hem through the coordinates of the key points, and determine whether the area needs to be filtered by the aspect ratio, thereby improving the accuracy of segmentation and target detection. Infrared images rely on their imaging principles based on heat distribution and temperature difference to focus on the detection of dangerous goods. By applying the segmentation, key point detection and detection results in the visible light image to the infrared image, and using the advantages of infrared thermal imaging to reduce background interference, this embodiment can accurately identify the dangerous goods (illegal items) in the infrared image. At the same time, this embodiment integrates advanced face recognition technology to store the facial information of people with a history of binding and hiding in the database. When the facial features of the actor are detected to match the records in the database, an alarm will be automatically triggered. This function significantly enhances the ability to identify potential lawbreakers, and combined with intelligent hem suppression technology, it further reduces misidentification caused by hems, and improves the reliability and efficiency of detection of hidden illegal items.

[0067] refer to Figure 1 and Figure 2 An embodiment of the present application provides a method for detecting objects hidden in a human body, which may specifically include the following steps.

[0068] Step 1: Acquire synchronously collected visible light images and infrared images of pedestrians passing through the detection area.

[0069] In some embodiments, specifically, the dual-light collection device is used to capture infrared images and visible light images of pedestrians passing through the security inspection area in real time. This step is performed using a fixed-focus high-resolution camera and an infrared sensor to ensure that clear image data is captured.

[0070] This application uses dual-light image fusion technology, combining information from visible light and infrared imaging. This technology can improve image quality under different lighting conditions and enhance the accuracy of target detection and segmentation. Through dual-light fusion, the viewing angle and environmental limitations of traditional single light sources are effectively overcome, allowing the model to maintain high performance in various environments.

[0071] Step 2: perform image registration on the infrared image and the visible light image.

[0072] In some embodiments, specifically, several groups of corresponding feature points are extracted from visible light images and infrared images, the several groups of feature points are normalized, normalized control points are calculated, the normalized control point sets are expanded into homogeneous coordinates, a matrix representing the normalized control point sets is constructed, and the normalized control point sets can be stored in the matrix.

[0073] The SVD (Singular Value Decomposition) method is used to calculate the inverse matrix of the normalized control point set and determine the linear independence of the points (linear independence condition: at least 3 non-collinear points are required). If the linear independence condition is met, the matrix calculation continues.

[0074] Through matrix operations, the inverse matrix and the normalized control point set are calculated. The relationship between the normalized matrix, the inverse matrix and the original feature point set is used to obtain the 3x3 transformation matrix from the visible light image to the infrared image. The 3x3 transformation matrix can convert the point coordinates in the visible light image to the corresponding positions in the infrared image, thereby realizing the alignment of the visible light image with the infrared image.

[0075] The embodiments of the present application are based on dual-light image registration calculation to align the visible light image and the infrared image. The segmentation and key point detection results on the visible light image can be applied to the infrared image as filtering conditions, thereby improving the recognition effect in the infrared image.

[0076] Step 3: Based on the segmentation results and key point detection results on the visible light image, reduce the probability of misidentification when performing target detection on the infrared image, and build a trained deep learning target detection model.

[0077] In some embodiments, based on the key point detection result on the visible light image, reducing the probability of misidentification when detecting the target on the infrared image specifically includes:

[0078] By detecting key points on the visible light image, the coordinates of key points such as the shoulder, hips (hips) and so on can be obtained. The key point coordinate set includes the left shoulder coordinate (L_shoulder), the right shoulder coordinate (R_shoulder), the left hip coordinate (L_hip) and the right hip coordinate (R_hip).

[0079] Based on the aforementioned key point coordinate set, the easily misidentified area of ​​the human body including the navel and the crotch is delineated (key point detection result). The relevant formulas include:

[0080] Y Waist =Y S_center *(1-k)+Y H_center *k;

[0081]

[0082] Among them, Y S_center The ordinate of the center point of the shoulder (obtained by taking the midpoint between the left shoulder coordinate and the right shoulder coordinate); H_center represents the ordinate of the hip center point (obtained by taking the midpoint between the left hip coordinate and the right hip coordinate); k represents the scale factor representing the position of the waist relative to the shoulder center and the hip center; Y Waist Indicates the vertical coordinate of the waist; Y Navel represents the vertical coordinate of the belly button; d represents the vertical distance from the belly button to the waist, d is the maximum value, 0 is the minimum value, and the following Represents a value range of 0 to d.

[0083] The area enclosed by the vertical and horizontal lines of the vertical coordinate of the belly button, the horizontal coordinate of the shoulders on both sides, and the vertical coordinate of the center point of the hips (i.e., draw the first horizontal line and the second horizontal line at the belly button point and the center point of the hips respectively, draw the first vertical line and the second vertical line at the points of the left and right shoulders respectively, and the area enclosed by the first horizontal line, the second horizontal line, the first vertical line and the second vertical line) is determined as the area of ​​the human body that is easily misidentified.

[0084] A mask is applied to the area where the human body is easily misidentified to shield the content in the area, so as to reduce the probability of misidentification when detecting targets on infrared images.

[0085] After many simulation tests, we found that setting k to 0.6 and d to 0.1m is suitable for most scenarios. Using such parameters to define the human body's easily misidentified area can effectively filter the hem area, improving the accuracy and reliability of illegal object binding and concealment detection in infrared images.

[0086] refer to Figure 3 and Figure 4After a large number of security inspection tests, it was found that the hem area (the area near the crotch or belly button) often produces shadows that are easy to cause misidentification. The reason is that the hem of the hem area is often not close to the flesh, while the inner trousers are generally closer to the human body and closer to the flesh. This kind of layered clothing (hem area, chest pocket, carry-on bag, etc.)-like area is very likely to produce thermal imaging shadows that are easy to cause misidentification, resulting in misidentification and causing trouble for security personnel. People who engage in illegal smuggling often tie and hide a large amount of smuggled goods close to their flesh. Since the amount of smuggling is generally large (in the detection practice, some people have tied and hidden 70 to 80 mobile phones in the waist, crotch and other areas at one time), they will choose to tie and hide in the area near the crotch (hem area). In view of this, the present application introduces the hem area suppression technology, which accurately identifies the hem area through key point detection, and then performs a certain degree of mask shielding on the area to reduce the probability of thermal imaging misidentification. The hem area suppression technology improves the accuracy of target detection, and is especially suitable for the application scenario of smuggling detection of illegal items hidden in the human body. It can significantly reduce the false detection rate of smuggling detection of illegal items hidden in the human body.

[0087] In some embodiments, more specifically, a method for obtaining a trained deep learning target detection model may include the following parts.

[0088] (1) Dataset Collection

[0089] Based on the company's special dual-light acquisition device K1280, the test camera is placed on a tripod at a height of 1.3m from the ground, and the tested object is 0.5 to 5m away from the camera; electronic products (mobile phones), jewelry, or even guns, drugs and other illegal items that may be suspected of smuggling are simulated, and the binding and hiding places include waist, legs and back, etc.; through video streaming, videos of binding and hiding illegal items are collected taking into account factors such as different heights, genders, and clothing, and then the videos are converted into pictures at a certain frame rate interval, and duplicate pictures are deleted to ensure the uniqueness of each picture, and 8000+ pairs of image data sets containing diverse dual-light images are obtained. This image dataset includes negative samples such as water stains, sweat stains and hems on the clothes of the tested person, ensuring that the model has high robustness and accuracy for actual detection tasks.

[0090] (2) Dataset Annotation

[0091] The infrared imaging data collected on site is analyzed, and the items with high imaging similarity to the illegal items are divided into eight categories: backpacks, shoulder bags, hats, skirts, loose coats, water stains, sweat stains and oil stains. Any non-illegal items that present similar imaging to the suspected illegal items will affect the accuracy of the recognition results. Therefore, it is necessary to segment and exclude the non-human parts of the image (backpacks, shoulder bags, hats, skirts, loose coats, etc.), and use the threshold segmentation method to exclude the water stains, sweat stains and oil stains. Then, the hidden illegal items are marked. When marking, only the part within the human torso is marked, and a maximum of two frames are used to detect one person. This application segments and excludes areas such as backpacks, shoulder bags, hats, skirts, coats, etc. on the image data set, and excludes the water stains, sweat stains and oil stains. The parts are excluded using the threshold segmentation method, and then the labels of the illegal items are marked, thereby effectively reducing the problem of misidentification and improving the detection accuracy of illegal items.

[0092] (3) Data enhancement

[0093] Used to balance the number of category samples and increase the generalization ability of the model.

[0094] Image enhancement: The main methods include horizontal flipping, random translation, random angle change, random cropping, random brightness change, random contrast change, random sharpness change and random scale change.

[0095] Image registration enhancement: affine transformation.

[0096] Enhancements specific to infrared images: temperature range adjustment, thermal noise simulation, motion blur, etc.

[0097] Target scale transformation: target distance adjustment, target size adjustment, dynamic target boundary enhancement, etc.

[0098] Background adjustment: background blur, background noise adjustment, etc.

[0099] Data mixing: image fusion, synthetic data generation, etc.

[0100] (4) Model training and test adjustment

[0101] The image dataset is divided into training and testing sets in a ratio of 8:2, and each image must be paired with its corresponding label (including object bounding box and segmentation mask).

[0102] After applying various data augmentation techniques to the training set data, the YOLOv8 model is used for training. During the model training process, the input data augmented images are processed through three parts: object detection, segmentation, and key point detection. The model simultaneously generates object detection results (bounding box, category, confidence) and segmentation results (mask of each object) as well as key point locations (key point coordinates of each object).

[0103] Then the loss is calculated to iterate the model effect. The loss calculation of target detection includes position loss (bounding box regression loss), category loss (classification loss) and confidence loss (target existence probability loss). The loss calculation of target segmentation includes comparing the difference between the predicted segmentation mask and the true mask. The key point detection loss calculates the difference between the predicted key point position and the true key point position. By analyzing the output results of each round, the comprehensive loss value between the predicted image and the annotated image (true value image) is calculated. Based on the comprehensive loss value, the gradient is back-transferred to optimize the model parameters, and iterates continuously until the model parameters converge to obtain a deep learning target detection model.

[0104] Finally, the performance of the model is evaluated on the test set, including the accuracy of target detection (mAP), segmentation performance (IoU), and key point detection accuracy. According to the evaluation results, hyperparameters such as learning rate and batch size are adjusted to improve the performance of the model and ensure excellent performance in target detection, image segmentation, and key point detection tasks. After such test evaluation and hyperparameter adjustment, a trained deep learning target detection model can be obtained.

[0105] Traditional target detection and image segmentation techniques are usually processed separately. This traditional method often fails to fully utilize information when processing complex scenes, resulting in limited overall performance of the model. The embodiments of the present application integrate target detection, image segmentation, and key point detection into one model by adopting a multi-task learning framework. This integration method can simultaneously optimize multiple tasks in the same network model, thereby improving the overall performance and efficiency of the model in various complex scenes.

[0106] Step 4: Use the trained deep learning target detection model to perform target detection in the registered infrared image, obtain the object exposure ratio of each object area pixel to the corresponding human body area pixel, and judge whether the object exposure ratio of each object area is greater than the preset object exposure ratio threshold. If so, it is determined that the corresponding human body area has hidden objects.

[0107] In some embodiments, specifically, in the infrared image after registration, the determination of whether illegal items are hidden is achieved by calculating the proportion of pixels of suspected illegal items in the human body area. The algorithm first detects the set of pixels of suspected illegal items and calculates the number of pixels of these suspected illegal items respectively. Then, the total number of pixels of the corresponding human body area is calculated. By calculating the ratio of the number of pixels of these suspected illegal items to the total number of pixels of the human body area, that is, the "exposure ratio of the item", it is determined whether there are illegal items hidden. In view of the fact that some items are not dangerous goods, such as mobile phones, but a large number of hidden items for smuggling will be suspected of violations, it is set as follows: if the exposure ratio of this item exceeds the preset threshold, it is determined that there are illegal items hidden in the area; if the exposure ratio of the item is lower than the preset threshold, it is considered that there are no illegal items hidden. This threshold discrimination method of the exposure ratio of items effectively improves the accuracy and reliability of illegal item detection.

[0108] refer to Figure 3 , respectively calculated Figure 3 The ratio of the black area pixels (i.e., the pixels in the object area) in the four red boxes in the image to the pixels in the human body area (the pixels in the entire human body area) is used to obtain the corresponding object exposure ratio in each box. Then, the relationship between the exposure ratio of each object and the preset object exposure ratio threshold is determined one by one, so as to further determine whether illegal objects are hidden in each box. The four red boxes indicate that the pedestrian is carrying objects in the human body area at the four red boxes. Only when the exposure ratio of the object in the box exceeds the preset object exposure ratio threshold, it is determined that there is a high probability that illegal objects are hidden in the human body area at the box. As for whether it is definitely an illegal object, further confirmation is needed.

[0109] In order to improve the accuracy of detection, the results detected on the infrared image can also be fed back to the segmentation results of the aforementioned visible light image for verification to eliminate misidentification.

[0110] Step 5: Preferably, after determining that an illegal item is hidden in the corresponding human body area, an alarm message is automatically issued.

[0111] In some embodiments, when potential illegal items are detected, the intelligent alarm mechanism will be automatically triggered to promptly remind security inspectors to conduct further inspections. Combining the features of infrared images and visible light images, the dual-light image data is analyzed in depth by comprehensively applying segmentation and detection models to determine whether it meets the illegal items binding and concealment standards. This multimodal information fusion method can effectively eliminate the misidentification of non-illegal items and improve the accuracy of detection.

[0112] To ensure the effectiveness and reliability of the method for detecting objects hidden on human bodies in practical applications, a 60-day real-world scenario test was conducted. During the test, the detection threshold was optimized according to different environments and personnel status to minimize misidentification. Ultimately, the intelligent alarm function not only accurately identified illegal objects hidden on human bodies, but also reduced the false alarm rate and improved the overall detection effect.

[0113] This application integrates dual-light image technology and gives full play to the complementary advantages of dual-light images, thereby effectively overcoming the limitations of traditional single-modal recognition methods and achieving higher detection accuracy. In visible light images, this application applies human body detection, segmentation, and key point detection technology. Key point detection technology is used to accurately locate key parts of the human body, and can effectively shield areas that are prone to misidentification, such as the hem (referring to the lower edge of the clothes, usually located near the crotch or belly button), thereby improving the accuracy of segmentation and target detection. Infrared images rely on their imaging principles based on heat distribution and temperature difference, focusing on the detection of illegal items. By applying segmentation, key point detection and detection results in visible light images to infrared images, and using the advantages of infrared thermal imaging to reduce background interference, combined with the identification of the proportion of exposed objects, the adaptability and detection accuracy of various illegal items tied to the human body can be significantly improved.

[0114] Step 6. Preferably, an embodiment of the present application also integrates face recognition technology, and stores the facial information of people with a history of binding and hiding in a database in advance. When the facial features of the person are detected to match the records in the database, an alarm will be automatically triggered. This function significantly enhances the ability to identify potential lawbreakers and improves the reliability and efficiency of the detection of illegal items being bound and hidden.

[0115] The method of detecting items hidden on the human body of the present application can prevent criminals from using the flow of people passing through customs as a cover to mingle among passengers and engage in smuggling activities. It can severely crack down on the smuggling of items by smugglers and has significant practical value in intelligent security inspection applications.

[0116] Figure 5 A block diagram of a device suitable for implementing the above-described method for detecting objects hidden on a human body according to an embodiment of the present application is schematically shown. Figure 5 The device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0117] like Figure 5As shown, the device 1000 described in this embodiment includes: a processor 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage part 1008 to a random access memory (RAM) 1003. The processor 1001 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 1001 may also include an onboard memory for caching purposes. The processor 1001 may include a single processing unit or multiple processing units for executing different actions of the method flow for detecting objects hidden in a human body according to an embodiment of the present application.

[0118] In RAM 1003, various programs and data required for the operation of device 1000 are stored. Processor 1001, ROM 1002 and RAM 1003 are connected to each other via bus 1004. Processor 1001 performs various operations of the method flow for detecting objects hidden on a human body according to an embodiment of the present application by executing the programs in ROM 1002 and / or RAM 1003. It should be noted that the program may also be stored in one or more memories other than ROM 1002 and RAM 1003. Processor 1001 may also perform various operations of the method flow for detecting objects hidden on a human body according to an embodiment of the present application by executing the programs stored in the one or more memories.

[0119] According to an embodiment of the present application, the device 1000 may further include an input / output (I / O) interface 1005, which is also connected to the bus 1004. The device 1000 may further include one or more of the following components connected to the I / O interface 1005: an input portion 1006 including a keyboard, a mouse, etc.; an output portion 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 1008 including a hard disk, etc.; and a communication portion 1009 including a network interface card such as a LAN card, a modem, etc. The communication portion 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed, so that a computer program read therefrom is installed into the storage portion 1008 as needed.

[0120] According to an embodiment of the present application, the device 1000 may also include a dedicated dual-light collection device K1280 for synchronously collecting visible light images and infrared images of pedestrians passing through the detection area. After the collection is completed, the relevant image data can be transmitted to the processor 1001 to generate and execute the process instructions of the aforementioned method for detecting objects hidden in human bodies, thereby realizing accurate detection of illegal objects hidden in human body areas. Preferably, the device of the present application can actively receive infrared radiation from objects to achieve non-sensing channel identification, and supports hoisting without occupying ground space.

[0121] According to the method flow of detecting objects hidden on the human body according to the embodiment of the present application, it can be implemented as a computer software program. For example, the embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains program code for executing the method of detecting objects hidden on the human body shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1009, and / or installed from the removable medium 1011. When the computer program is executed by the processor 1001, the above-mentioned functions defined in the system of the embodiment of the present application are executed. According to the embodiment of the present application, the systems, devices, means, modules and / or units described above can be implemented by computer program modules.

[0122] The embodiments of the present application also provide a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments, or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the steps of the method for detecting objects hidden on a human body according to the embodiments of the present application can be implemented.

[0123] According to an embodiment of the present application, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In an embodiment of the present application, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus, or a device. For example, according to an embodiment of the present application, the computer-readable storage medium may include one or more memories other than the ROM 1002 and / or RAM 1003 described above.

[0124] It should be noted that the functional modules in the various embodiments of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software functional modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product.

[0125] The flowchart and / or block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flowchart and / or block diagram can represent a part of a module, program segment or code, and a part of the above-mentioned module, program segment or code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. It should also be noted that each box in the block diagram or flowchart, and the combination of the boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0126] Those skilled in the art will appreciate that the features described in the various embodiments and / or claims of the present application may be combined and / or combined in a variety of ways, even if such combinations and / or combinations are not explicitly described in the present application. In particular, without departing from the spirit and teachings of the present application, the technical features described in the various embodiments and / or claims of the present application may be combined and / or combined in a variety of ways, and all of these combinations and / or combinations fall within the scope of the present application.

[0127] Although the present application has been shown and described with reference to specific exemplary embodiments of the present application, it should be understood by those skilled in the art that various changes in form and details may be made to the present application without departing from the spirit and scope of the present application as defined by the appended claims and their equivalents. Therefore, the scope of the present application should not be limited to the above-mentioned embodiments, but should be determined not only by the appended claims, but also by the equivalents of the appended claims.

Claims

1. A method for detecting objects hidden in a human body, characterized in that: include: acquiring simultaneously collected visible light images and infrared images of pedestrians passing through a detection area; Performing image registration on the infrared image and the visible light image; Based on the segmentation results and key point detection results on the visible light image, the probability of misidentification when detecting targets on the infrared image is reduced, and a trained deep learning target detection model is constructed; The trained deep learning target detection model is used to perform target detection in the registered infrared image, and the object exposure ratio of each object area pixel point to the corresponding human body area pixel point is obtained respectively, and it is judged whether the object exposure ratio of each object area is greater than a preset object exposure ratio threshold value. If greater, it is determined that the corresponding human body area has hidden an object.

2. The method for detecting objects hidden on a human body as claimed in claim 1, characterized in that: Based on the key point detection results on the visible light image, the probability of misidentification when detecting targets on the infrared image is reduced by: Perform key point detection on the visible light image and obtain the key point coordinate set; Based on the key point coordinate set, obtaining key point detection results of a human body region that is easily misidentified, including a navel and a crotch; A mask is placed on the human body easily misidentified area to shield the content in the area, so as to reduce the probability of misidentification when performing target detection on the infrared image.

3. The method for detecting objects hidden on a human body as claimed in claim 2, characterized in that: The method of obtaining the trained deep learning target detection model includes: Collect videos of people hiding objects in multiple scenes, convert the videos into pictures at preset frame rate intervals, remove duplicates, and obtain image data sets; Segment and exclude the non-human area on the image of the image data set, shield the content in the human body easily misidentified area, and then label the object; After data enhancement processing is performed on the image dataset, the YOLOv8 model is used for training to obtain the trained deep learning target detection model, which specifically includes: The image dataset after data enhancement processing is divided into a training set and a test set according to a preset ratio; Input the training set into the YOLOv8 model to generate target detection results, target segmentation results and key point positions; In each round, the comprehensive loss value between the predicted image and the annotated image is calculated based on the loss of target detection, the loss of target segmentation, and the loss of key point detection; the loss calculation of target detection includes one or more of position loss, category loss, and confidence loss; the loss calculation of target segmentation includes comparing the difference between the predicted segmentation mask and the true mask; the loss calculation of key point detection includes the difference between the predicted key point position and the true key point position; Based on the comprehensive loss value, gradient backpropagation is performed to optimize the parameters of the model, and the iteration is continued until the model parameters converge to obtain a deep learning target detection model; The test set is used to evaluate the deep learning target detection model, and relevant hyperparameters are adjusted according to the evaluation results to obtain the trained deep learning target detection model.

4. The method for detecting objects hidden on a human body according to claim 2 or 3, characterized in that: The method for obtaining the human body misidentification area includes: Through the coordinates of the shoulders and hips, the coordinates of the waist and belly button are obtained. The relevant formulas include: AND Waist =And S_center *(1-k)+Y H_center *k; Among them, Y S_center Indicates the vertical coordinate of the center point of the shoulder; Y H_center represents the ordinate of the hip center point; k represents the proportional factor representing the position of the waist relative to the shoulder center and the hip center; Y Waist Indicates the vertical coordinate of the waist; Y Navel represents the vertical coordinate of the belly button; Y represents the vertical distance from the belly button to the waist; The area enclosed by the vertical and horizontal lines drawn on the vertical coordinate of the belly button, the horizontal coordinate of the shoulders on both sides, and the vertical coordinate of the center point of the hips is determined as the human body's easily misidentified area.

5. The method for detecting objects hidden on a human body as claimed in claim 1, characterized in that: Methods for determining the object exposure ratio of the object area pixels to the corresponding human body area pixels include: In the registered infrared image, the total number of pixels in each object area is obtained respectively; Obtain the total number of human body area pixels on the corresponding human body area; The object exposure ratio of the object area=(the total number of pixels in the object area) / (the total number of pixels in the human body area); If the item exposure ratio is greater than the preset item exposure ratio threshold, it is determined that the corresponding human body area hides the item; otherwise, it is determined that the corresponding human body area does not hide the item.

6. The method for detecting objects hidden on a human body as claimed in claim 1, characterized in that: The performing image registration on the infrared image and the visible light image comprises: Extracting several groups of corresponding feature points from the infrared image and the visible light image and performing normalization processing, obtaining normalized control points, constructing a matrix representing the normalized control point set, and expanding the normalized control point set into homogeneous coordinates; The inverse matrix of the matrix is ​​obtained by using the singular value decomposition method to determine the linear independence of the points. If the linear independence condition is met, the matrix calculation is continued. Based on the correspondence between the normalized matrix, the inverse matrix and the original feature point set, matrix operations are performed on the inverse matrix and the matrix to obtain a transformation matrix from the visible light image to the infrared image, and the point coordinates in the visible light image are converted to the corresponding positions in the infrared image to achieve alignment of the visible light image with the infrared image.

7. The method for detecting objects hidden on a human body as claimed in claim 1, characterized in that: Also includes: It is confirmed that the facial features of the pedestrian match a preset database including facial information of persons with a history of kidnapping, and an alarm message is automatically issued.

8. The method for detecting objects hidden on a human body according to claim 1 or 7, characterized in that: Also includes: It is determined that the object is hidden in the corresponding human body area and an alarm message is automatically issued.

9. The method for detecting objects hidden on a human body as claimed in claim 3, characterized in that: The non-human body area includes one or more of a backpack, a shoulder bag, a hat, a skirt and a coat.

10. A device, characterized in that: The method comprises at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program, and when the computer program is executed by the processing unit, the processing unit is enabled to execute the steps of the method for detecting objects hidden on a human body as described in any one of claims 1 to 9.

Images