Security inspection image recognition method and device
By using different types of security inspection image recognition models to identify and fuse the results of the security inspection images to be identified, the problem of low accuracy in security inspection image recognition in existing technologies is solved, and higher accuracy and recall are achieved, thereby improving security inspection efficiency.
Patent Information
- Application Number
- CN202211328659.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-10-27
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2042-10-27
AI Technical Summary
Existing security inspection image recognition methods have low accuracy, resulting in problems such as low efficiency, low recall rate, high false negative rate, and high labor costs.
Two different types of security inspection image recognition models are used to identify the security inspection images to be identified. After obtaining the initial results, they are fused, and the final recognition result is determined by predictive indicators and cross-union ratio, etc.
It improved the accuracy and recall rate of security inspection image recognition, enhanced the security inspection effect, and reduced labor costs.
Smart Images

Figure CN117994712B_ABST
Abstract
Description
Technical Field
[0001] This application mainly relates to the field of image processing technology, specifically to a security inspection image recognition method and device. Background Technology
[0002] Security inspection machines, also known as security screening equipment, include X-ray security machines, baggage security machines, walk-in X-ray machines, object inspection X-ray machines, X-ray security inspection instruments, X-ray baggage security machines, X-ray detectors, X-ray foreign object detectors, X-ray security inspection machines, X-ray baggage detectors, prohibited items detectors, prohibited items inspection machines, and dangerous goods inspection devices. Security inspection machines are widely used in airports, train stations, bus stations, government buildings, embassies, conference centers, exhibition centers, hotels, shopping malls, large events, post offices, schools, logistics industries, and industrial inspection. Currently, security inspection systems primarily use single-image imaging and manual interpretation. This method suffers from low efficiency, low recall rate, high false negative rate, and high labor costs, resulting in poor security and inadequate security. Some systems employ conventional deep learning algorithms for identification, but this is limited to the optimization of a single network, leading to low accuracy.
[0003] In other words, the accuracy of existing security inspection image recognition methods is relatively low. Summary of the Invention
[0004] This application provides a security inspection image recognition method and apparatus, aiming to solve the problem of low accuracy in existing security inspection image recognition methods.
[0005] In a first aspect, this application provides a security inspection image recognition method, the security inspection image recognition method comprising:
[0006] Acquire the security inspection image to be identified;
[0007] The security inspection image to be identified is input into two security inspection image recognition models respectively, and two initial image recognition results are obtained from the output of the two security inspection image recognition models. The two security inspection image recognition models are of different types.
[0008] The security check image recognition result is determined based on the two initial image recognition results.
[0009] Optionally, the two initial image recognition results include multiple first target detection boxes, and determining the security check image recognition result based on the two initial image recognition results includes:
[0010] Multiple preset detection box categories are respectively determined as target detection box categories;
[0011] The prediction metrics of the target detection box categories obtained by the two security inspection image recognition models on the preset test set are obtained respectively.
[0012] The second target detection box corresponding to the target detection box category is determined based on the prediction index of the target detection box category of the two security inspection image recognition models;
[0013] The second target detection box corresponding to multiple preset detection box categories is determined as the security inspection image recognition result.
[0014] Optionally, determining the second target detection box corresponding to the target detection box category based on the prediction index of the target detection box categories of the two security inspection image recognition models includes:
[0015] Determine whether the prediction index of the target detection box category in the two security inspection image recognition models meets the preset conditions;
[0016] The security image recognition model whose prediction index of the target detection box category in the two security image recognition models meets the preset conditions is determined as the target image recognition model;
[0017] If there is only one target image recognition model, then the first target detection box of the target detection box category output by the target image recognition model is determined as the second target detection box corresponding to the target detection box category.
[0018] Optionally, determining the second target detection box corresponding to the target detection box category based on the prediction index of the target detection box categories of the two security inspection image recognition models includes:
[0019] If the number of target image recognition models is zero, then calculate the crossover ratio (CUP) between the first target detection boxes of the target detection box categories of the two security inspection image recognition models respectively;
[0020] If the cross-union ratio between the first target detection boxes of the target detection box categories of the two security inspection image recognition models is greater than a preset value, then the first target detection boxes of the target detection box categories of the two security inspection image recognition models will be merged to obtain the second target detection box corresponding to the target detection box category.
[0021] Optionally, the prediction metrics include an accuracy metric and a recall metric, and the preset conditions include an accuracy metric greater than a preset accuracy metric and a recall metric greater than a preset recall metric.
[0022] Optionally, the two security inspection image recognition models are an anchor-based target detection model and an anchor-free target detection model, respectively.
[0023] Optionally, before determining the multiple preset detection box categories as target detection box categories, the following steps are included:
[0024] Obtain the detection box category for each first target detection box;
[0025] The first target detection box belonging to the prohibited detection box category is determined as one of the multiple preset detection box categories.
[0026] Secondly, this application provides a security inspection image recognition device, the security inspection image recognition device comprising:
[0027] The acquisition unit is used to acquire the security inspection image to be identified;
[0028] The prediction unit is used to input the security inspection image to be identified into two security inspection image recognition models respectively, and obtain two initial image recognition results output by the two security inspection image recognition models. The two security inspection image recognition models are of different types.
[0029] The determining unit is used to determine the security inspection image recognition result based on the two initial image recognition results.
[0030] Optionally, the determining unit is configured to:
[0031] Multiple preset detection box categories are respectively determined as target detection box categories;
[0032] The prediction metrics of the target detection box categories obtained by the two security inspection image recognition models on the preset test set are obtained respectively.
[0033] The second target detection box corresponding to the target detection box category is determined based on the prediction index of the target detection box category of the two security inspection image recognition models;
[0034] The second target detection box corresponding to multiple preset detection box categories is determined as the security inspection image recognition result.
[0035] Optionally, the determining unit is configured to:
[0036] Determine whether the prediction index of the target detection box category in the two security inspection image recognition models meets the preset conditions;
[0037] The security image recognition model whose prediction index of the target detection box category in the two security image recognition models meets the preset conditions is determined as the target image recognition model;
[0038] If there is only one target image recognition model, then the first target detection box of the target detection box category output by the target image recognition model is determined as the second target detection box corresponding to the target detection box category.
[0039] Optionally, the determining unit is configured to:
[0040] If the number of target image recognition models is zero, then calculate the crossover ratio (CUP) between the first target detection boxes of the target detection box categories of the two security inspection image recognition models respectively;
[0041] If the cross-union ratio between the first target detection boxes of the target detection box categories of the two security inspection image recognition models is greater than a preset value, then the first target detection boxes of the target detection box categories of the two security inspection image recognition models will be merged to obtain the second target detection box corresponding to the target detection box category.
[0042] Optionally, the prediction metrics include an accuracy metric and a recall metric, and the preset conditions include an accuracy metric greater than a preset accuracy metric and a recall metric greater than a preset recall metric.
[0043] Optionally, the two security inspection image recognition models are an anchor-based target detection model and an anchor-free target detection model, respectively.
[0044] Optionally, the determining unit is configured to:
[0045] Obtain the detection box category for each first target detection box;
[0046] The first target detection box belonging to the prohibited detection box category is determined as one of the multiple preset detection box categories.
[0047] Thirdly, this application provides a computer device, the computer device comprising:
[0048] One or more processors;
[0049] Memory; and
[0050] One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the security inspection image recognition method described in any one of the first aspects.
[0051] Fourthly, this application provides a computer-readable storage medium storing a plurality of instructions adapted for loading by a processor to perform the steps of the security inspection image recognition method described in any one of the first aspects.
[0052] This application provides a security inspection image recognition method and apparatus. The security inspection image recognition method includes: acquiring a security inspection image to be recognized; inputting the security inspection image to be recognized into two security inspection image recognition models respectively, obtaining two initial image recognition results output by the two security inspection image recognition models, wherein the two security inspection image recognition models are of different types; and determining the security inspection image recognition result based on the two initial image recognition results. This application utilizes two security inspection image recognition models of different types to recognize the same security inspection image to be recognized, obtaining two initial image recognition results, and then determining the security inspection image recognition result based on the two initial image recognition results. This can combine the advantages of the two networks, and fusing the two initial image recognition results can achieve feature complementarity between the two networks, thereby improving the accuracy of the security inspection image recognition method. Attached Figure Description
[0053] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0054] Figure 1 This is a schematic diagram of a security inspection image recognition system provided in an embodiment of this application;
[0055] Figure 2 This is a schematic flowchart of an embodiment of the security inspection image recognition method provided in this application.
[0056] Figure 3 This is a flowchart illustrating the process of determining the second target detection box corresponding to the target detection box category based on the prediction index of the target detection box category of two security inspection image recognition models in one embodiment of the security inspection image recognition method provided in this application.
[0057] Figure 4 This is a schematic diagram of an embodiment of the security inspection image recognition device provided in this application.
[0058] Figure 5 This is a schematic diagram of an embodiment of the computer device provided in this application. Detailed Implementation
[0059] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0060] In the description of this application, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are used only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more features. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.
[0061] In this application, the term "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described as "exemplary" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use this application. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that this application can be made without using these specific details. In other instances, well-known structures and processes are not described in detail to avoid obscuring the description of this application with unnecessary detail. Therefore, this application is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.
[0062] This application provides a security inspection image recognition method and apparatus, which will be described in detail below.
[0063] Please see Figure 1 , Figure 1 This is a schematic diagram of a security inspection image recognition system provided in an embodiment of this application. The security inspection image recognition system may include a computer device 100, which integrates a security inspection image recognition device.
[0064] In this embodiment, the computer device 100 can be a standalone server, a server network, or a server cluster. For example, the computer device 100 described in this embodiment includes, but is not limited to, a computer, a network host, a single network server, a set of multiple network servers, or a cloud server composed of multiple servers. The cloud server is composed of a large number of computers or network servers based on cloud computing.
[0065] In this embodiment, the computer device 100 described above can be a general-purpose computer device or a special-purpose computer device. In specific implementations, the computer device 100 can be a desktop computer, a portable computer, a network server, a handheld computer (Personal Digital Assistant, PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, an embedded device, etc. This embodiment does not limit the type of computer device 100.
[0066] Those skilled in the art will understand that Figure 1 The application environment shown is merely one application scenario of the solution in this application and does not constitute a limitation on the application scenario of the solution in this application. Other application environments may include more than one application scenario. Figure 1 The number of computer devices shown is more or less, for example Figure 1 Only one computer device is shown in the image. It is understood that the security inspection image recognition system may also include one or more other computer devices capable of processing data, which are not specifically limited here.
[0067] In addition, such as Figure 1 As shown, the security inspection image recognition system may also include a memory 200 for storing data.
[0068] It should be noted that, Figure 1 The schematic diagram of the security inspection image recognition system shown is merely an example. The security inspection image recognition system and scenario described in this application are for the purpose of more clearly illustrating the technical solutions of this application, and do not constitute a limitation on the technical solutions provided in this application. As those skilled in the art will know, with the evolution of security inspection image recognition systems and the emergence of new business scenarios, the technical solutions provided in this application are also applicable to similar technical problems.
[0069] First, this application provides a security inspection image recognition method, which includes: acquiring a security inspection image to be recognized; inputting the security inspection image to be recognized into two security inspection image recognition models respectively to obtain two initial image recognition results output by the two security inspection image recognition models, wherein the two security inspection image recognition models are of different types; and determining the security inspection image recognition result based on the two initial image recognition results.
[0070] like Figure 2 As shown, Figure 2 This is a schematic flowchart of an embodiment of the security inspection image recognition method provided in this application, which includes the following steps S201 to S203:
[0071] S201. Obtain the security inspection image to be identified.
[0072] In this embodiment, a security inspection machine acquires the monitoring video stream corresponding to the monitored area of the transmitted package, extracts the current frame image from the monitoring video stream, and obtains the security inspection image to be identified. Security inspection machines, also known as security screening devices, include security X-ray machines, baggage inspection machines, tunnel-type X-ray machines, object inspection X-ray machines, X-ray security inspection instruments, X-ray baggage inspection machines, X-ray detectors, X-ray foreign object detectors, X-ray security inspection machines, X-ray baggage detectors, prohibited items detectors, prohibited items inspection machines, and dangerous goods inspection devices. Security inspection machines are widely used in airports, train stations, subway stations, bus stations, government buildings, embassies, conference centers, exhibition centers, hotels, shopping malls, large-scale events, post offices, schools, logistics industries, industrial inspection, etc.
[0073] In one specific embodiment, the raw surveillance video stream of the security inspection machine is acquired through an interface, depending on the model and interface type of the security inspection machine's host or display. The interface can be VGA (Video Graphics Array), DVI (Digital Visual Interface), HDMI (High Definition Multimedia Interface), etc. VGA uses analog signals, while DVI and HDMI use digital signals. HDMI supports audio output compared to DVI. When the security inspection machine supports multiple interface types, the interface type is selected in the order of HDMI, DVI, and VGA to minimize video signal loss.
[0074] In one specific embodiment, a security inspection machine acquires the surveillance video stream corresponding to the monitored area where the package is being transmitted. The current frame image is extracted from the video stream, scaled proportionally to a preset size, and then normalized to obtain the security inspection image to be identified. Specifically, the RGB values of the scaled image are subtracted by 114 and divided by 255 to obtain the security inspection image to be identified.
[0075] S202. Input the security inspection image to be identified into two security inspection image recognition models respectively to obtain two initial image recognition results output by the two security inspection image recognition models. The two security inspection image recognition models are of different types.
[0076] In this embodiment of the application, the two security check recognition images can be classification models such as CNN, or object detection models such as YOLO.
[0077] In one specific embodiment, the two security inspection image recognition models are an anchor-based object detection model and an anchor-free object detection model, respectively. Specifically, the anchor-based object detection model can be Faster R-CNN, SSD, RetinaNet, YOLOv2, YOLOv3, YOLOv5, etc. The anchor-free object detection model can be CornerNet, CenterNet, etc.
[0078] Preferably, the two security inspection image recognition models are YOLOv5 and CenterNet. YOLOv5's detection head is anchor-based and tends to perform detection based on anchor-based features. CenterNet, on the other hand, is anchor-free and uses a heat map to form the center for detection. Each network has its own advantages. Fusing the results of these two networks can achieve feature complementarity between the two networks, which can improve the accuracy and recall of the algorithm.
[0079] Most mainstream object detection algorithms are anchor-based, including both two-stage and one-stage methods. Anchors are pre-defined bounding boxes with different sizes and aspect ratios, set manually or through clustering methods. These boxes cover the entire image to prevent missed detections. During model training, the anchor's dimensions and position are regressed based on the intersection-over-union (IoU) loss between the anchor and the ground truth, making them increasingly closer to the ground truth. Simultaneously, the anchor's category is predicted, and the final output consists of these regressed and classified anchors. Two-stage methods require a significantly larger number of anchors to be selected and optimized compared to one-stage methods, and the selection process is more rigorous, resulting in longer processing time, but higher accuracy. On commonly used detection benchmarks, state-of-the-art (SOTA) methods are generally anchor-based.
[0080] Anchor-free algorithms have emerged due to the effective solutions to insufficient semantic information and imbalanced positive and negative samples in the development of FPN and Focal Loss. Anchor-free detectors detect objects in two different ways: one is to first locate multiple predefined or self-learned keypoints and then constrain the spatial range of the object, called the keypoint-based method; the other is to use the center point or central target region to define positive samples and then predict their distances to the four sides of the target, called the center-based method. Anchor-free methods further simplify the object detection process, reduce related hyperparameters, make network construction and training simpler, and improve generalization ability.
[0081] S203. Determine the security inspection image recognition result based on the two initial image recognition results.
[0082] In this embodiment, the two initial image recognition results include multiple first target detection boxes. Each first target detection box is a detection box obtained by the two security image recognition models performing target detection on the security image to be recognized. The first target detection box is labeled with the detection box category of the item within it and the position of the first target detection box. The detection box category can be firearms, knives, etc. Determining the security image recognition result based on the two initial image recognition results includes:
[0083] (1) Determine the multiple preset detection box categories as target detection box categories respectively.
[0084] In one specific embodiment, the detection box category of each first target detection box is obtained; the first target detection boxes belonging to the prohibited detection box category are determined as multiple preset detection box categories, and the multiple preset detection box categories are respectively determined as target detection box categories. The prohibited detection category can be a firearm category, a knife category, etc., which can be set according to the specific situation.
[0085] In another specific embodiment, all prohibited detection box categories can be determined as preset detection box categories.
[0086] Furthermore, the detection box category of each first target detection box is obtained, and it is determined whether there is a prohibited detection box category in the detection box category of each first target detection box. If there is a prohibited detection box category in the detection box category of each first target detection box, then the multiple preset detection box categories are respectively determined as target detection box categories.
[0087] (2) Obtain the prediction index of the target detection box category obtained by the two security inspection image recognition models on the preset test set.
[0088] The prediction metrics can include precision and recall. Precision indicates how many positive examples in a sample were correctly predicted; it's the proportion of correctly predicted positive examples out of all positive examples, used to evaluate the detector's accuracy on the basis of successful detection. Recall indicates how many of the predicted positive samples are actually positive; it's the proportion of true positive examples in the prediction results.
[0089] In one specific embodiment, a remote connection is established with a deployed security screening machine. Multiple historical security screening images captured and transmitted back from the machine are then acquired remotely. These images are annotated either machine-generated or manually to obtain multiple sample security screening images. For example, the number of historical security screening images is 200,000. Each sample security screening image is labeled with the location and category of a bounding box. These historical security screening images are divided into a pre-set training set and a pre-set test set. The ratio of sample security screening images in the pre-set training set to those in the pre-set test set is 9:1. Two security screening image recognition models are trained using the pre-set training set and saved as ONNX models for TensorRT inference.
[0090] In one specific embodiment, two security image recognition models are trained using different training parameters. For example, the two security image recognition models are YOLOv5 and CenterNet detection networks, respectively. A pre-set training set is prepared in YOLOv5 training data format (cx, cy, w, h, label), and training is initiated. The model is saved using a map, and the optimal model is saved. After 300 epochs of training, the optimal model is output. The model is then converted into an ONNX model for TensorRT inference. Alternatively, a pre-set training set is prepared in COCO dataset format, and training is initiated. The model is saved using a map; if a better map is found in the next iteration, the new model is saved; this process continues until the optimal model is saved. After 300 epochs of training, the optimal model is output. The model is then converted into an ONNX model for TensorRT inference.
[0091] Furthermore, the classification thresholds for each preset detection box category are obtained. These thresholds can be multiple equally spaced values within the 0-1 range. For example, the classification thresholds for each preset detection box category might be 0.2, 0.4, 0.6, and 0.8. The F1-score of each classification threshold for the preset detection box category on a preset test set is calculated. The classification threshold of the preset detection box category with the highest F1-score is determined as the optimal threshold for that preset detection box category. The prediction index obtained by predicting the preset detection box category on the preset test set based on its optimal threshold is determined as the prediction index for that preset detection box category.
[0092] (3) Determine the second target detection box corresponding to the target detection box category based on the prediction index of the target detection box category of the two security inspection image recognition models.
[0093] In one specific embodiment, the first target detection box output by the security inspection image recognition model with the largest sum of prediction metrics for target detection box categories is determined as the second target detection box corresponding to that category. The second target detection box corresponding to the target detection box category is the final output result.
[0094] (4) The second target detection box corresponding to multiple preset detection box categories is determined as the security inspection image recognition result.
[0095] After determining the multiple preset detection box categories as target detection box categories, the second target detection box corresponding to each preset detection box category is obtained.
[0096] Specifically, the category and position of the second target detection box corresponding to multiple preset detection box categories are determined as the security inspection image recognition result.
[0097] Furthermore, the security image recognition results are sent to the cloud, which then forwards them to on-site security personnel for evaluation. On-site security personnel are capable of interpreting the images; the process involves the following steps: on-site security personnel analyze the data to identify prohibited items; if prohibited items are found, a nationwide wanted notice for the package is issued; packages containing prohibited items are intercepted and opened for inspection to determine if the prohibited items are present.
[0098] Furthermore, in order to improve recognition accuracy, such as Figure 3 As shown, in a specific embodiment, the second target detection box corresponding to the target detection box category is determined based on the prediction index of the target detection box category of the two security inspection image recognition models, including: S301-S303:
[0099] S301. Determine whether the prediction index of the target detection box category in the two security inspection image recognition models meets the preset conditions.
[0100] In this embodiment, the prediction metrics include accuracy and recall. Preset conditions include an accuracy greater than a preset accuracy and a recall greater than a preset recall. For example, both the preset accuracy and preset recall are 95%. The preset accuracy and preset recall can be set according to specific circumstances.
[0101] S302. The security image recognition model whose prediction index of the target detection box category in the two security image recognition models meets the preset conditions is determined as the target image recognition model.
[0102] S303. If the number of target image recognition models is 1, then the first target detection box of the target detection box category output by the target image recognition model is determined as the second target detection box corresponding to the target detection box category.
[0103] If there is only one target image recognition model, it means that only one model has high accuracy and recall when identifying the target detection box category. In this case, the first target detection box of the target detection box category output by the target image recognition model is determined as the second target detection box corresponding to that category. That is, the output of the target image recognition model for the target detection box category is taken as the standard. For example, among the two security image recognition models, YOLOv5 and CenterNet, only CenterNet's accuracy and recall meet the preset conditions. Therefore, the first target detection box of the target detection box category output by CenterNet is determined as the second target detection box corresponding to that category. For example, if the target detection box category is firearms, and only CenterNet's accuracy and recall meet the preset conditions, it means that CenterNet has a better prediction effect for firearms. Therefore, the first target detection box of the firearms category predicted by CenterNet is determined as the second target detection box as the final output result.
[0104] Furthermore, based on the prediction metrics of the target detection box categories of the two security inspection image recognition models, the second target detection box corresponding to the target detection box category is determined, including:
[0105] (1) If the number of target image recognition models is zero, calculate the crossover ratio between the first target detection boxes of the target detection box categories of the two security inspection image recognition models respectively.
[0106] The Intersection over Union (LOU) function calculates the ratio of the intersection to the union of two bounding boxes. For example, the YOLOv5 and CenterNet security image recognition models identify two first object detection boxes of each object category, and the LOU of these two first object detection boxes is calculated.
[0107] If the number of target image recognition models is zero, it means that neither of the two target image recognition models meets the two prediction indicators of accuracy and recall. In this case, the crossover ratio between the first target detection boxes of the target detection box categories of the two security inspection image recognition models is calculated.
[0108] (2) If the crossover ratio between the first target detection boxes of the target detection box categories of the two security inspection image recognition models is greater than the preset value, then the first target detection boxes of the target detection box categories of the two security inspection image recognition models are merged to obtain the second target detection box corresponding to the target detection box category.
[0109] The preset value can be 0.7, which can be set according to specific circumstances. If the intersection-union ratio (IUR) between the first target detection boxes of the target detection box categories of the two security inspection image recognition models is greater than the preset value, it indicates that the two first target detection boxes predict the same item. In this case, the first target detection boxes of the target detection box categories of the two security inspection image recognition models are merged to obtain the second target detection box corresponding to the target detection box category. The second target detection box has a higher probability of containing items of the target detection category, thereby improving the recognition accuracy. If the IUR between the first target detection boxes of the target detection box categories of the two security inspection image recognition models is not greater than the preset value, it indicates that they are two different detection boxes. In this case, the first target detection boxes of the target detection box categories of the two security inspection image recognition models are both determined as the second target detection boxes corresponding to the target detection box category.
[0110] If there are two target image recognition models, the first target detection box of the target detection box category of the target image recognition model with the higher recall rate is determined as the second target detection box corresponding to the target detection box category.
[0111] To better implement the security inspection image recognition method in the embodiments of this application, a security inspection image recognition device is also provided in the embodiments of this application, such as... Figure 4 As shown, the security inspection image recognition device 400 includes:
[0112] Acquisition unit 401 is used to acquire the security inspection image to be identified;
[0113] The prediction unit 402 is used to input the security inspection image to be identified into two security inspection image recognition models respectively, and obtain two initial image recognition results output by the two security inspection image recognition models, wherein the two security inspection image recognition models are of different types;
[0114] The determining unit 403 is used to determine the security inspection image recognition result based on the two initial image recognition results.
[0115] Optionally, the determining unit 403 is used for:
[0116] Multiple preset detection box categories are respectively determined as target detection box categories;
[0117] The prediction metrics of the target detection box categories obtained by the two security inspection image recognition models on the preset test set are obtained respectively.
[0118] The second target detection box corresponding to the target detection box category is determined based on the prediction index of the target detection box category of the two security inspection image recognition models;
[0119] The second target detection box corresponding to multiple preset detection box categories is determined as the security inspection image recognition result.
[0120] Optionally, the determining unit 403 is used for:
[0121] Determine whether the prediction index of the target detection box category in the two security inspection image recognition models meets the preset conditions;
[0122] The security image recognition model whose prediction index of the target detection box category in the two security image recognition models meets the preset conditions is determined as the target image recognition model;
[0123] If there is only one target image recognition model, then the first target detection box of the target detection box category output by the target image recognition model is determined as the second target detection box corresponding to the target detection box category.
[0124] Optionally, the determining unit 403 is used for:
[0125] If the number of target image recognition models is zero, then calculate the crossover ratio (CUP) between the first target detection boxes of the target detection box categories of the two security inspection image recognition models respectively;
[0126] If the cross-union ratio between the first target detection boxes of the target detection box categories of the two security inspection image recognition models is greater than a preset value, then the first target detection boxes of the target detection box categories of the two security inspection image recognition models will be merged to obtain the second target detection box corresponding to the target detection box category.
[0127] Optionally, the prediction metrics include accuracy and recall, and the preset conditions include that the accuracy metric is greater than the preset accuracy and the recall metric is greater than the preset recall.
[0128] Optionally, the two security inspection image recognition models are an anchor-based target detection model and an anchor-free target detection model, respectively.
[0129] Optionally, the determining unit 403 is used for:
[0130] Obtain the detection box category for each first target detection box;
[0131] The first target detection box belonging to the prohibited detection box category is determined as multiple preset detection box categories.
[0132] This application also provides a computer device that integrates any of the security inspection image recognition devices provided in this application. The computer device includes:
[0133] One or more processors;
[0134] Memory; and
[0135] One or more applications, wherein the applications are stored in memory and configured to be executed by a processor as steps of the security image recognition method in any of the embodiments described above.
[0136] like Figure 5 As shown, it illustrates a structural schematic diagram of the computer device involved in the embodiments of this application, specifically:
[0137] The computer device may include components such as a processor 601 with one or more processing cores, a memory 602 with one or more computer-readable storage media, a power supply 603, and an input unit 604. Those skilled in the art will understand that the computer device structure shown in the figures does not constitute a limitation on the computer device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:
[0138] Processor 601 is the control center of the computer device. It connects various parts of the computer device via various interfaces and lines, and performs various functions and processes data by running or executing software programs and / or modules stored in memory 602, and by calling data stored in memory 602, thereby providing overall monitoring of the computer device. Optionally, processor 601 may include one or more processing cores; processor 601 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor. Preferably, processor 601 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the aforementioned modem processor may not be integrated into processor 601.
[0139] The memory 602 can be used to store software programs and modules. The processor 601 executes various functional applications and data processing by running the software programs and modules stored in the memory 602. The memory 602 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 602 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 602 may also include a memory controller to provide the processor 601 with access to the memory 602.
[0140] The computer device also includes a power supply 603 that supplies power to the various components. Preferably, the power supply 603 can be logically connected to the processor 601 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 603 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.
[0141] The computer device may also include an input unit 604, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.
[0142] Although not shown, the computer device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 601 in the computer device loads the executable files corresponding to the processes of one or more application programs into the memory 602 according to the following instructions, and the processor 601 runs the application programs stored in the memory 602 to realize various functions, as follows:
[0143] Acquire the security inspection image to be identified; input the security inspection image to be identified into two security inspection image recognition models respectively, and obtain two initial image recognition results output by the two security inspection image recognition models. The two security inspection image recognition models are of different types; determine the security inspection image recognition result based on the two initial image recognition results.
[0144] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.
[0145] Therefore, embodiments of this application provide a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), a disk, or an optical disk, etc. A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in any of the security inspection image recognition methods provided in embodiments of this application. For example, the computer program loaded by the processor can execute the following steps:
[0146] Acquire the security inspection image to be identified; input the security inspection image to be identified into two security inspection image recognition models respectively, and obtain two initial image recognition results output by the two security inspection image recognition models. The two security inspection image recognition models are of different types; determine the security inspection image recognition result based on the two initial image recognition results.
[0147] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the detailed descriptions of other embodiments above, which will not be repeated here.
[0148] In practice, each of the above units or structures can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units or structures, please refer to the previous method embodiments, which will not be repeated here.
[0149] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.
[0150] The above provides a detailed description of a security inspection image recognition method and apparatus provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. A security inspection image recognition method, characterized in that, The security inspection image recognition method includes: Acquire the security inspection image to be identified; The security inspection image to be identified is input into two security inspection image recognition models respectively, and two initial image recognition results are obtained from the output of the two security inspection image recognition models. The two security inspection image recognition models are of different types. The security inspection image recognition result is determined based on the two initial image recognition results; The two initial image recognition results include multiple first target detection boxes. Determining the security check image recognition result based on the two initial image recognition results includes: Multiple preset detection box categories are respectively determined as target detection box categories; The prediction metrics of the target detection box categories obtained by the two security inspection image recognition models on the preset test set are obtained respectively. The second target detection box corresponding to the target detection box category is determined based on the prediction index of the target detection box category of the two security inspection image recognition models; The second target detection box corresponding to multiple preset detection box categories is determined as the security inspection image recognition result.
2. The security inspection image recognition method according to claim 1, characterized in that, The step of determining the second target detection box corresponding to the target detection box category based on the prediction index of the target detection box category of the two security inspection image recognition models includes: Determine whether the prediction index of the target detection box category in the two security inspection image recognition models meets the preset conditions; The security image recognition model whose prediction index of the target detection box category in the two security image recognition models meets the preset conditions is determined as the target image recognition model; If there is only one target image recognition model, then the first target detection box of the target detection box category output by the target image recognition model is determined as the second target detection box corresponding to the target detection box category.
3. The security inspection image recognition method according to claim 2, characterized in that, The step of determining the second target detection box corresponding to the target detection box category based on the prediction index of the target detection box category of the two security inspection image recognition models includes: If the number of target image recognition models is zero, then calculate the crossover ratio (CUP) between the first target detection boxes of the target detection box categories of the two security inspection image recognition models respectively; If the cross-union ratio between the first target detection boxes of the target detection box categories of the two security inspection image recognition models is greater than a preset value, then the first target detection boxes of the target detection box categories of the two security inspection image recognition models will be merged to obtain the second target detection box corresponding to the target detection box category.
4. The security inspection image recognition method according to claim 2, characterized in that, The prediction metrics include accuracy and recall, and the preset conditions include that the accuracy metric is greater than a preset accuracy and the recall metric is greater than a preset recall.
5. The security inspection image recognition method according to claim 1, characterized in that, The two security inspection image recognition models are an anchor-based target detection model and an anchor-free target detection model, respectively.
6. The security inspection image recognition method according to claim 1, characterized in that, Before determining the multiple preset detection box categories as target detection box categories, the following steps are included: Obtain the detection box category for each first target detection box; The first target detection box belonging to the prohibited detection box category is determined as one of the multiple preset detection box categories.
7. A security inspection image recognition device, characterized in that, The security inspection image recognition device includes: The acquisition unit is used to acquire the security inspection image to be identified; The prediction unit is used to input the security inspection image to be identified into two security inspection image recognition models respectively, and obtain two initial image recognition results output by the two security inspection image recognition models. The two security inspection image recognition models are of different types. The determining unit is used to determine the security inspection image recognition result based on the two initial image recognition results; The determining unit is used to: determine multiple preset detection box categories as target detection box categories respectively; The prediction metrics for the target detection box categories obtained by the two security inspection image recognition models on a preset test set are obtained respectively; wherein, the prediction metrics include precision and recall. The second target detection box corresponding to the target detection box category is determined based on the prediction index of the target detection box category of the two security inspection image recognition models; The second target detection box corresponding to multiple preset detection box categories is determined as the security inspection image recognition result.
8. A computer device, characterized in that, The computer device includes: One or more processors; Memory; and One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the security inspection image recognition method according to any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that, It contains a computer program, which is loaded by a processor to perform the steps of the security inspection image recognition method according to any one of claims 1 to 6.
Citation Information
Patent Citations
Security check image recognition method and device, electronic equipment and storage medium
CN112712093A
Door lock recognition method and device and electronic equipment
CN113807146A
