Real-time passing security method based on deep learning and intelligent channel

Abstract

The application provides a real-time passing security method based on deep learning and an intelligent channel, comprising the following steps: collecting real-time images in the channel; performing target detection on the real-time images to obtain pedestrian targets, luggage targets and depth information of all targets; and detecting whether the pedestrian carries luggage by calculating the depth relationship between the pedestrian targets and all the luggage targets in the effective area of the channel. The real-time passing security method based on deep learning of the application obtains pedestrian targets, luggage targets and depth information of corresponding targets through a target detection model; calculates the depth difference between the pedestrian targets and the luggage targets, introduces the spatial relationship between the pedestrian and the luggage in a pure visual manner, and improves the detection accuracy of the pedestrian carrying luggage; and the intelligent channel of the application can correspondingly obtain a first real-time image and a second real-time image from two mutually opposite visual angles, thereby improving the detection accuracy.

Description

Technical Field

[0001] This application belongs to the field of security inspection technology, and more specifically, it relates to a real-time access security inspection method and intelligent channel based on deep learning. Background Technology

[0002] In public places such as subway stations, high-speed rail stations, and airports, turnstiles are used to achieve intelligent and automated detection of people and luggage entering and exiting the station.

[0003] In existing technologies, the detection of people and luggage in security checkpoints employs deep learning-based visual detection methods, directly identifying passengers' bags in the surveillance footage to determine if they are carrying luggage. However, this method has significant limitations: since the surveillance field of view typically covers both passengers undergoing security checks inside the checkpoint and those waiting or already checked outside, existing algorithms, relying solely on visual features for target detection, cannot distinguish the ownership of luggage. This leads to frequent misclassifications of luggage carried by individuals outside the checkpoint as belonging to the currently checked passenger, resulting in numerous false alarms. Such misclassifications not only reduce security check efficiency but may also lead to unnecessary duplication of security procedures, severely impacting the passenger experience. Furthermore, when multiple people are passing through densely packed areas or luggage obscures each other, existing methods struggle to accurately associate luggage with corresponding passengers, further increasing the risk of misclassification. Therefore, there is an urgent need for an intelligent detection method capable of accurately distinguishing between passengers being checked within the checkpoint and their luggage to address the technical problem of insufficient detection accuracy in existing technologies. Summary of the Invention

[0004] The purpose of this application is to provide a real-time access security check method and intelligent channel based on deep learning, so as to solve the technical problem of insufficient detection accuracy in the process of detecting whether pedestrians are carrying luggage in the prior art.

[0005] To achieve the above objectives, the technical solution adopted in this application is: to provide a real-time access security check method based on deep learning, including the following steps:

[0006] Acquire real-time images within the acquisition channel;

[0007] Perform object detection on real-time images to obtain depth information of pedestrians, luggage, and all other objects;

[0008] Whether a pedestrian is carrying luggage is detected by calculating the depth relationship between pedestrian targets and all luggage targets within the effective area of ​​the passage.

[0009] In a preferred embodiment, a method for object detection in real-time images includes the steps of:

[0010] The backbone network uses convolutional layers and layer normalization to continuously extract multi-level image features F. base The calculation formula is:

[0011]

[0012] Where Conv represents convolution operation, BN represents layer normalization, Re represents repetition operation, and I represents layer repetition. ori This represents the real-time image input.

[0013] In a preferred embodiment, a method for object detection in real-time images includes the steps of:

[0014] At the end of the backbone network, pooling windows of different sizes are used to process image features F. base Multi-scale feature capture is performed to obtain multi-scale image features F. backbone .

[0015] In a preferred embodiment, a method for object detection in real-time images includes the steps of:

[0016] The image features F after upsampling backbone With image features F base Intermediate feature maps are obtained by concatenating them, and then downsampling through strided convolutions is used to concatenate these intermediate feature maps. Finally, feature fusion is performed at different levels to obtain the image features F. Neck .

[0017] In a preferred embodiment, a method for object detection in real-time images includes the steps of:

[0018] Image features F Neck The input is fed into the detection layer Detect, and the output is the target detection result F. Detect And based on the target detection result F Detect Obj obtains the coordinates of the four points of the target detection box. box The category of the object detection box (Obj) cls Target depth information Obj depth The category of the object detection box is Obj. cls This includes pedestrians and luggage.

[0019] In a preferred embodiment, the acquisition of the target's depth information Obj depth The method includes the following steps:

[0020] Obtain target detection results F at different levels Detect Confidence level;

[0021] Determine the target's depth information based on the level with the highest confidence level. depth .

[0022] In a preferred embodiment, a method for object detection in real-time images includes the steps of:

[0023] A static background image without human presence is pre-acquired as a baseline background model;

[0024] The foreground region is obtained by comparing the real-time image with the reference background model pixel by pixel.

[0025] The foreground region is binarized to generate a foreground mask;

[0026] Target detection is performed on the foreground mask to obtain pedestrian targets and baggage targets.

[0027] In a preferred embodiment, before detecting whether a pedestrian is carrying luggage by calculating the depth relationship between pedestrian targets and all luggage targets within the effective area of ​​the channel, the method further includes the step of:

[0028] The effective area of ​​the channel is preset based on the benchmark background model;

[0029] Detect the intersection-union ratio (IOU) between the foreground mask region and the effective channel region corresponding to the pedestrian target;

[0030] If IOU > T, the pedestrian target is retained; otherwise, the pedestrian target is filtered out, where T is a preset threshold.

[0031] This application also provides an intelligent channel, which includes a pedestrian channel and a baggage inspection belt, which are arranged in parallel. A first camera is installed at the entrance of the pedestrian channel, and a second camera is installed at the exit of the pedestrian channel. The first camera acquires a first real-time image, and the second camera acquires a second real-time image. Based on the first real-time image and the second real-time image, the real-time passage security inspection method based on deep learning described above is used to comprehensively detect whether a pedestrian is carrying baggage.

[0032] In a preferred embodiment, a method for detecting whether a pedestrian is carrying luggage by calculating the depth relationship between pedestrian targets and all luggage targets within the effective area of ​​the channel includes the following steps:

[0033] First depth information of all pedestrian and baggage targets is obtained based on the first real-time image, and second depth information of all pedestrian and baggage targets is obtained based on the second real-time image;

[0034] Generate a depth interval using the first and second depth information of the same target as endpoint values;

[0035] The system detects whether a pedestrian is carrying luggage by calculating whether there is an intersection between the depth ranges of pedestrian targets and all baggage targets within the effective area of ​​the passage.

[0036] The real-time security inspection method based on deep learning provided in this application, compared with the prior art, obtains the depth information of pedestrian targets, baggage targets and corresponding targets through a target detection model; calculates the depth difference between pedestrian targets and baggage targets, and introduces the spatial relationship between pedestrians and baggage in a purely visual way, thereby improving the detection accuracy of pedestrians carrying baggage.

[0037] The beneficial effect of the intelligent channel provided in this application is that, compared with the prior art, it can acquire the first real-time image and the second real-time image from two opposing perspectives, thereby improving the detection accuracy. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 A flowchart illustrating the real-time access security check method based on deep learning provided in Embodiment 1 of this application;

[0040] Figure 2 The inference diagram of the real-time access security inspection method based on deep learning provided in Embodiment 1 of this application;

[0041] Figure 3 This is a reasoning graph for another form of the deep learning-based real-time access security check method provided in Embodiment 1 of this application;

[0042] Figure 4 A comparison diagram of the real-time image and its visualization effect provided in Embodiment 1 of this application;

[0043] Figure 5 This is a comparison diagram of the second real-time image and its visualization effect provided in Embodiment 5 of this application;

[0044] Figure 6 A comparison diagram of the first real-time image and its visualization effect provided in Embodiment 5 of this application;

[0045] Figure 7 This is a schematic diagram of the intelligent channel provided in Embodiment 7 of this application acquiring depth information of a target. Detailed Implementation

[0046] To make the technical problems, technical solutions, and beneficial effects to be solved by this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the scope of this application.

[0047] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0048] Please refer to the following: Figure 1 The real-time access security check method based on deep learning provided in Embodiment 1 of this application will now be described. The real-time access security check method based on deep learning includes the following steps:

[0049] Step S1: Acquire real-time images within the channel;

[0050] Step S2: Perform target detection on the real-time image to obtain the depth information of pedestrian targets, baggage targets, and all targets;

[0051] Step S3: Detect whether a pedestrian is carrying luggage by calculating the depth relationship between pedestrian targets and all luggage targets within the effective area of ​​the channel.

[0052] It is understood that when a pedestrian arrives at a designated area within the passageway, a real-time image containing the pedestrian can be obtained by real-time image acquisition. The object of interest can be obtained from the real-time image through an object detection model. The object of interest may only include pedestrian targets and luggage targets. In another embodiment, it may include other common targets in public places, such as trolleys, wheelchairs, hand-held clothing, pets, etc.

[0053] For further details, please refer to the following: Figure 4 Before calculating the depth difference, a channel area is preset. By judging whether the pedestrian target is within the channel area, interference from passing or queuing pedestrian targets is filtered out. If the pedestrian target is within the channel, it is then judged whether luggage is detected. If luggage is detected, the depth difference between the pedestrian target and the luggage target within the channel area is used to determine whether the pedestrian is carrying luggage.

[0054] Furthermore, if the depth difference between a pedestrian and all baggage targets is greater than a threshold, the pedestrian is determined not to be carrying baggage. If the depth difference between a pedestrian and baggage targets is less than the threshold, the pedestrian is determined to be carrying baggage, and passage is denied. The pedestrian is then reminded to place their baggage into the security screening equipment. This prevents pedestrians carrying baggage from interfering with passengers undergoing security checks.

[0055] The real-time security inspection method based on deep learning provided in this application, compared with the prior art, obtains the depth information of pedestrian targets, baggage targets and corresponding targets through a target detection model; calculates the depth difference between pedestrian targets and baggage targets, and introduces the spatial relationship between pedestrians and baggage in a purely visual way, thereby improving the detection accuracy of pedestrians carrying baggage.

[0056] This application's second embodiment is based on the first embodiment; please refer to [link / reference needed]. Figure 2 and Figure 3 A method for object detection in real-time images, comprising the following steps:

[0057] The backbone network uses convolutional layers and layer normalization to continuously extract multi-level image features F. base The calculation formula is:

[0058]

[0059] Where Conv represents convolution operation, BN represents layer normalization, Re represents repetition operation, and I represents layer repetition. ori This represents the real-time image input.

[0060] Understandably, the operation is repeated at least twice. Preferably, it is repeated four to eight times. The convolutional layer uses two convolutional kernels of different sizes instead of the traditional 3x3 kernel, which increases the network's receptive field while reducing computational cost.

[0061] For further details, please refer to Figure 2 and Figure 3 A method for object detection in real-time images, comprising the following steps:

[0062] At the end of the backbone network, pooling windows of different sizes are used to process image features F. base Multi-scale feature capture is performed to obtain multi-scale image features F. backbone .

[0063] It is understandable that pooling windows of different sizes are used to refine the image features F. base Multi-scale feature capture enables the model to detect targets of different sizes, solving the problem that a single receptive field cannot cover both near and far targets.

[0064] For further details, please refer to Figure 2 and Figure 3 A method for object detection in real-time images, comprising the following steps:

[0065] The image features F after upsampling backbone With image features F baseIntermediate feature maps are obtained by concatenating them, and then downsampling through strided convolutions is used to concatenate these intermediate feature maps. Finally, feature fusion is performed at different levels to obtain the image features F. Neck .

[0066] It is understandable that image features F backbone Upsampling can improve resolution, therefore the image features F after upsampling are... backbone It can be compared with the image features F at the corresponding level. base Maintaining the same scale ensures consistency in the fusion scale. Since the input to this process is image features F... base With image features F backbone In other words, the input consists of shallow features (high resolution) and deep features (high semantics). By concatenating the deep semantic features with the shallow detail features, the problem of missing small targets (such as luggage partially obscured by the human body) caused by the loss of details in deep features can be solved, and the accuracy of target detection at different scales can be further improved.

[0067] It is worth adding that, Figure 2 In this context, Cat represents the concatenation operation, UP represents the upsampling operation, and Conv represents the convolution operation.

[0068] For further details, please refer to Figure 2 and Figure 3 A method for object detection in real-time images, comprising the following steps:

[0069] Image features F Neck The input is fed into the detection layer Detect, and the output is the target detection result F. Detect And based on the target detection result F Detect Obj obtains the coordinates of the four points of the target detection box. box The category of the object detection box (Obj) cls Target depth information Obj depth The category of the object detection box is Obj. cls This includes pedestrians and luggage.

[0070] For further details, please refer to Figure 2 and Figure 3 The coordinates of the four points of the object detection box (Obj) box The object detection head can output the category of the object detection bounding box (Obj). cls The depth information of the target can be output by the classification header. (Obj) depth It can be output by the target depth head.

[0071] Furthermore, obtain the coordinates of the four points of the target detection bounding box (Obj). box The category of the object detection box (Obj) cls Target depth information Objdepth The method is as follows:

[0072]

[0073] Where Reg represents the regression branch and Cls represents the classification branch.

[0074] Understandably, the detection layer (Detect) can be composed of regular convolutional layers and different loss functions. Each loss function generates the four coordinates (Obj) of the target detection box based on the corresponding constraints. box The category of the object detection box (Obj) cls Target depth information Obj depth The process of obtaining it.

[0075] This application's third embodiment is based on embodiment two, but unlike embodiment two, this third embodiment provides another method for acquiring the target's depth information. depth The method differs from Example 2 only in the depth information of the target. depth It is not obtained through a regression branch. The depth information of the target, Obj, is obtained as described. depth The method includes the following steps:

[0076] Obtain target detection results F at different levels Detect Confidence level;

[0077] Determine the target's depth information based on the level with the highest confidence level. depth .

[0078] Understandably, this is because the image features F are normalized during the convolutional layers and layer normalization processes used in the backbone network. base The network is divided into multiple consecutive layers, with a continuous gradient relationship between the resolutions of these layers. Based on this, the multi-scale feature capture step at the end of the backbone network and the feature fusion process in the backbone part enable the model to simultaneously recognize both near and far targets and address the loss of details in deep features. Therefore, the same target may obtain confidence scores in multiple layers, but the highest confidence score is only obtained in the layer that best matches the target's shape and size. Thus, the depth information of the target can be determined based on the layer with the highest confidence score. depth .

[0079] Furthermore, regarding the depth information of the target, Obj depthThe data is categorized by level, with each level corresponding to a category. For example, the category is used to represent the specific depth information. A depth of 1 means that the specific depth is between 1m and 1.5m. On average, the category ID is incremented by 1 for every 0.5m. If the category for pedestrian depth information in the passage is 2, it means that the depth is between 1.5m and 2m.

[0080] This fourth embodiment is based on any one of embodiments one to three, and includes a method for target detection in real-time images, comprising the following steps:

[0081] A static background image without human presence is pre-acquired as a baseline background model;

[0082] The foreground region is obtained by comparing the real-time image with the reference background model pixel by pixel.

[0083] The foreground region is binarized to generate a foreground mask;

[0084] Target detection is performed on the foreground mask to obtain pedestrian targets and baggage targets.

[0085] It is understandable that there are many interfering factors in the background of public places, such as on-site advertisements and ambient lighting, which can cause false detection or missed detection of targets. In this embodiment, the real-time image is compared pixel by pixel with the benchmark background model to obtain the foreground area with no background interference, thus improving the accuracy of target detection.

[0086] Furthermore, since pedestrian and baggage targets are obtained based on the foreground mask, the interference of background pixels is filtered out when obtaining the depth information of all pedestrian and baggage targets, thus improving the accuracy of obtaining depth information.

[0087] Furthermore, before detecting whether a pedestrian is carrying luggage by calculating the depth relationship between pedestrian targets and all luggage targets within the effective area of ​​the channel, the following steps are also included:

[0088] The effective area of ​​the channel is preset based on the benchmark background model;

[0089] Detect the intersection-union ratio (IOU) between the foreground mask region and the effective channel region corresponding to the pedestrian target;

[0090] If IOU > T, the pedestrian target is retained; otherwise, the pedestrian target is filtered out, where T is a preset threshold.

[0091] Understandably, existing technologies typically use Intersection over Union (IOU) to compare the entire target detection bounding box with a specified region to determine the object's position. However, background areas can interfere with the target detection bounding box, leading to inaccurate detection results.

[0092] In this fourth embodiment, the intersection-union ratio (IOU) between the foreground mask region and the effective channel region corresponding to the detected pedestrian target is improved by eliminating the interference from the background region, thus enhancing the detection accuracy compared to existing technologies.

[0093] Please refer to the following: Figures 5 to 7 The intelligent passage provided in Embodiment 5 will now be described. The intelligent passage includes a pedestrian walkway 10 and a baggage inspection belt 20, which are arranged parallel to each other. A first camera 30 is installed at the entrance of the pedestrian walkway 10, and a second camera 40 is installed at the exit of the pedestrian walkway 10. The first camera 30 acquires a first real-time image, and the second camera 40 acquires a second real-time image. Based on the first and second real-time images, any one of the deep learning-based real-time passage security inspection methods from Embodiments 1 to 4 is used to comprehensively detect whether a pedestrian is carrying baggage.

[0094] It is understandable that since the first camera 30 and the second camera 40 are respectively located at the entrance and exit of the pedestrian passage 10, the first real-time image and the second real-time image can be obtained from two opposing perspectives, thereby reducing blind spots and improving detection accuracy.

[0095] This sixth embodiment is based on the fifth embodiment. When determining whether a pedestrian is carrying luggage, the two cameras operate independently. As long as either camera detects that a pedestrian is carrying luggage, it is determined that the pedestrian is carrying luggage.

[0096] This seventh embodiment is based on the fifth embodiment; please refer to both embodiments as well. Figure 7 When determining whether a pedestrian is carrying luggage, parameters from two cameras are combined for judgment. Specifically, this method detects whether a pedestrian is carrying luggage by calculating the depth relationship between pedestrian targets and all luggage targets within the effective area of ​​the passage. The steps include:

[0097] First depth information of all pedestrian and baggage targets is obtained based on the first real-time image, and second depth information of all pedestrian and baggage targets is obtained based on the second real-time image;

[0098] Generate a depth interval using the first and second depth information of the same target as endpoint values;

[0099] The system detects whether a pedestrian is carrying luggage by calculating whether there is an intersection between the depth ranges of pedestrian targets and all baggage targets within the effective area of ​​the passage.

[0100] It is understood that the first depth information and the second depth information should be understood as parameter information that has been pre-calibrated and located in the same coordinate system, and the same target should be understood as a target that exists simultaneously in the first real-time image and the second real-time image and is detected as the same pedestrian or luggage.

[0101] Because human posture and body shape, as well as the shape and orientation of luggage, are constantly changing within the passageway, the depth information of the same target within the passageway is constantly changing. In this embodiment seven, two cameras located at the entrance and exit of the pedestrian passageway 10, facing opposite directions, simultaneously acquire depth information from both ends of the same target and generate a depth range. This reduces the fluctuations in depth information caused by differences in human posture and body shape, as well as the shape and orientation of luggage, thereby improving the accuracy of detecting whether pedestrians are carrying luggage.

[0102] For example, if the first camera 30 detects depth information of a pedestrian and a piece of luggage as 7 and 5 respectively, and the second camera 40 detects depth information of the same pedestrian and the same piece of luggage as 4 and 3 respectively, then the depth range for the pedestrian is [4,7] and the depth range for the luggage is [3,5]. If the depth ranges [4,7] and [3,5] intersect, it is determined that the pedestrian is carrying luggage, and the exit gate will not allow passage until the pedestrian puts all their luggage into the baggage inspection belt for processing. Once it is detected that the pedestrian has no luggage, the exit gate will be opened to allow passage. After passing through the exit gate, the pedestrian walks to the end of the baggage inspection belt to retrieve their checked luggage, thus achieving the separation and inspection of people and goods.

[0103] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An intelligent channel, characterized in that, The intelligent channel includes a pedestrian channel and a baggage inspection belt, which are arranged in parallel. A first camera is installed at the entrance of the pedestrian channel and a second camera is installed at the exit of the pedestrian channel. The first camera acquires a first real-time image, and the second camera acquires a second real-time image. Based on the first real-time image and the second real-time image, a real-time passage security inspection method based on deep learning is used to comprehensively detect whether pedestrians are carrying baggage. The aforementioned real-time access security check method based on deep learning includes the following steps: Acquire real-time images within the acquisition channel; Perform object detection on real-time images to obtain depth information of pedestrians, luggage, and all other objects; Whether a pedestrian is carrying luggage is detected by calculating the depth relationship between pedestrian targets and all luggage targets within the effective area of ​​the channel. Real-time object detection in images includes the following steps: The convolutional layers and layer normalization used by the backbone network are used to successively extract multi-level image features F base The calculation formula is: ， wherein Conv denotes a convolution operation, BN denotes a batch normalization, Re denotes a repetition operation, I ori represents a real-time image input; At the end of the backbone network, the image features F are pooled with different size pooling windows base Multi-scale feature capturing is performed to obtain multi-scale image features F backbone ; The image feature F backbone is obtained by upsampling the image feature F base and splicing the intermediate feature maps, and the intermediate feature maps are spliced by down-sampling of the cross-step convolution, and finally the image feature F Neck is obtained by feature fusion at different levels. Image features F Neck The input is fed into the detection layer Detect, and the output is the target detection result F. Detect And based on the target detection result F Detect Obj obtains the coordinates of the four points of the target detection box. box The category of the object detection bounding box (Obj) cls Target depth information Obj depth The category of the object detection box is Obj. cls Including pedestrians and luggage; Obj obtains the depth information of the target. depth The steps include: Obtain target detection results F at different levels Detect Confidence level; Determine the target's depth information based on the level with the highest confidence level. depth ; Depth information of the target (Obj) depth The categories are classified according to their levels, with each level corresponding to a category with a different depth range; The detection of whether a pedestrian is carrying luggage involves calculating the depth relationship between pedestrian targets and all luggage targets within the effective area of ​​the passage, including the following steps: First depth information of all pedestrian and baggage targets is obtained based on the first real-time image, and second depth information of all pedestrian and baggage targets is obtained based on the second real-time image; Generate a depth interval using the first and second depth information of the same target as endpoint values; The system detects whether a pedestrian is carrying luggage by calculating whether there is an intersection between the depth ranges of pedestrian targets and all baggage targets within the effective area of ​​the passage.

2. The intelligent channel as described in claim 1, characterized in that, Real-time object detection in images includes the following steps: A static background image without human presence is pre-acquired as a baseline background model; The foreground region is obtained by comparing the real-time image with the reference background model pixel by pixel. The foreground region is binarized to generate a foreground mask; Target detection is performed on the foreground mask to obtain pedestrian targets and baggage targets.

3. The intelligent channel as described in claim 2, characterized in that, Before detecting whether a pedestrian is carrying luggage by calculating the depth relationship between pedestrian targets and all luggage targets within the effective area of ​​the channel, the following steps are also included: The effective area of ​​the channel is preset based on the benchmark background model; Detect the intersection-union ratio (IOU) between the foreground mask region and the effective channel region corresponding to the pedestrian target; If IOU > T, the pedestrian target is retained; otherwise, the pedestrian target is filtered out, where T is a preset threshold.

Images