Security check image detection method, device, equipment and product

Abstract

The invention provides a security check image detection method, device, equipment and product, which can be applied to the technical field of target detection. The method comprises the steps that a security check image is obtained, and the security check image is obtained by scanning an object through a ray scanning device; processing the security check image by using a slice convolution network to obtain slice convolution features; processing the slice convolution features by using a multi-residual convolution network to obtain residual convolution features; processing the residual convolution features based on an attention pooling network to obtain pooling attention features; processing the pooling attention features and the residual convolution features based on a feature pyramid network to obtain attention fusion features and convolution attention features; processing the attention fusion feature and the convolution attention feature based on a path aggregation network to obtain a fusion feature; and processing the attention fusion feature and the fusion feature based on the detection head to obtain a detection result of the security check image, the detection result representing whether the security check image contains an abnormal object.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection, and in particular to a security inspection image detection method, device, equipment and product. Background Art

[0002] With the rapid development of tourism and transportation, X-ray security inspection machines play an increasingly important role in modern cities. X-ray security inspection machines are a security inspection method widely used in public places (such as airports, stations, etc.). Security inspectors need to carefully distinguish and observe X-ray images to ensure that there are no abnormal items in passengers' luggage and effectively maintain the safety and order of public places. However, the inventors found that this traditional security inspection method has problems such as low manual operation efficiency, lack of professionalism, and high missed inspection rate, which makes it difficult to meet the modern society's requirements for fast, efficient and accurate security inspections. Summary of the invention

[0003] In view of the above problems, the present invention provides a security inspection image detection method, device, equipment and product.

[0004] According to a first aspect of the present invention, a security inspection image detection method is provided, comprising: acquiring a security inspection image, wherein the security inspection image is obtained by scanning an object using a ray scanning device; processing the security inspection image using a slice convolution network to obtain slice convolution features; processing slice convolution features using a multi-residual convolution network to obtain residual convolution features; processing residual convolution features based on an attention pooling network to obtain pooled attention features; processing pooled attention features and residual convolution features based on a feature pyramid network to obtain attention fusion features and convolution attention features; processing attention fusion features and convolution attention features based on a path aggregation network to obtain fusion features; processing attention fusion features and convolution attention features based on a detection head to obtain a detection result of the security inspection image, the detection result representing whether the security inspection image contains abnormal objects.

[0005] A second aspect of the present invention provides a security inspection image detection device, comprising:

[0006] An acquisition module, used to acquire a security inspection image, wherein the security inspection image is obtained by scanning an object using a ray scanning device;

[0007] The slice convolution feature acquisition module is used to process the security inspection image using the slice convolution network to obtain the slice convolution feature;

[0008] The residual convolution feature acquisition module is used to process the slice convolution features using a multi-residual convolution network to obtain residual convolution features;

[0009] The pooling attention feature acquisition module is used to process the residual convolution features based on the attention pooling network to obtain the pooling attention features;

[0010] The convolutional attention fusion module is used to process the pooled attention features and residual convolutional features based on the feature pyramid network to obtain the attention fusion features and convolutional attention features;

[0011] A fusion feature acquisition module is used to process the attention fusion feature and the convolutional attention feature based on the path aggregation network to obtain the fusion feature;

[0012] The detection result acquisition module is used to process the attention fusion feature and the fusion feature based on the detection head to obtain the detection result of the security inspection image. The detection result represents whether the security inspection image contains abnormal items.

[0013] A third aspect of the present invention provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0014] The fourth aspect of the present invention further provides a computer program product, comprising a computer program or instructions, which implement the steps of the above method when executed by a processor.

[0015] According to an embodiment of the present invention, by using a slice convolution network to process security inspection images, slice convolution features can be obtained, and by using a multi-residual convolution network to process slice convolution features, residual convolution features can be obtained. By processing residual convolution features based on an attention pooling network, features at different scales corresponding to the residual convolution features can be pooled to enrich semantic information, and by processing the pooled features based on an attention mechanism, pooled attention features can be obtained, thereby enhancing the receptive field and improving the detection of small-sized targets. By processing pooled attention features and residual convolution features based on a feature pyramid network, attention fusion features and convolution attention features can be obtained. By processing attention fusion features and convolution attention features based on a path aggregation network, fusion features can be obtained. By processing attention fusion features and fusion features based on a detection head, detection results of security inspection images can be obtained, thereby improving detection accuracy, improving the detection performance of small-sized targets and occluded targets, and improving the generalization ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0017] Figure 1 A diagram showing an application scenario of a security inspection image detection method and device according to an embodiment of the present invention;

[0018] Figure 2A flow chart of a security inspection image detection method according to an embodiment of the present invention is shown;

[0019] Figure 3 A structural diagram of a serial pooling layer according to an embodiment of the present invention is shown;

[0020] Figure 4 A structural diagram of a self-attention layer according to an embodiment of the present invention is shown;

[0021] Figure 5 A network structure diagram of a detection model according to an embodiment of the present invention is shown;

[0022] Figure 6 A flow chart of a detection model training and conversion method according to an embodiment of the present invention is shown;

[0023] Figure 7 The RKNN image reasoning flow chart of the real-time detection system for security inspection images based on the target chip in the present invention is shown;

[0024] Figure 8 A flowchart of the present invention is shown in which the converted RKNN model in the real-time detection system of security inspection images based on the target chip is transmitted to the target chip for reasoning;

[0025] Fig. 9 A schematic diagram of a security inspection image detection result obtained based on the security inspection image detection method of the present invention is shown;

[0026] Fig.10 It shows a structural block diagram of a security inspection image detection device according to an embodiment of the present invention;

[0027] Fig.11 A block diagram of an electronic device suitable for implementing a security inspection image detection method according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0028] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0029] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0030] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0031] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0032] With the rapid development of tourism and transportation, X-ray security inspection machines play an increasingly important role in modern cities. X-ray security inspection machines are a security inspection method widely used in public places (such as airports, stations, etc.). Security inspectors need to carefully distinguish and observe X-ray images to ensure that there are no abnormal items in passengers' luggage and effectively maintain the safety and order of public places. However, the inventors found that this traditional security inspection method has problems such as low manual operation efficiency, lack of professionalism, and high missed inspection rate, which makes it difficult to meet the requirements of modern society for security inspections.

[0033] X-ray image security inspection abnormal object detection technology based on computer vision and artificial intelligence technology has attracted attention and research. This technology aims to automatically identify and mark abnormal objects in X-ray images through computer vision, image processing, machine learning and other technical means, using algorithms and models, and conduct secondary checks manually, thereby improving the accuracy and efficiency of security inspections, while also reducing the impact of human factors on the results, and reducing the skill requirements and training costs for security inspectors. However, the existing target detection network is designed based on natural images. The detection effect of the existing target detection network in the X-ray image scene formed by the security inspection machine is not good, and it is easy to have problems such as missed detection and false detection.

[0034] In view of this, the present invention provides a security inspection image detection method, a security inspection image detection device, equipment and product. The method includes: obtaining a security inspection image, wherein the security inspection image is obtained by scanning an object using a ray scanning device; processing the security inspection image using a slice convolution network to obtain a slice convolution feature; processing the slice convolution feature using a multi-residual convolution network to obtain a residual convolution feature; processing the residual convolution feature based on an attention pooling network to obtain a pooled attention feature; processing the pooled attention feature and the residual convolution feature based on a feature pyramid network to obtain an attention fusion feature and a convolution attention feature; processing the attention fusion feature and the convolution attention feature based on a path aggregation network to obtain a fusion feature; processing the attention fusion feature and the fusion feature based on a detection head to obtain a detection result of the security inspection image, and the detection result represents whether the security inspection image contains abnormal objects.

[0035] It should be noted that the security inspection image detection method and security inspection image detection device provided by the present invention can be used in the field of target detection technology, and can also be used in any field outside the field of target detection technology, such as the public security field. Therefore, the application field of the security inspection image detection method and security inspection image detection device provided by the present invention is not limited.

[0036] In the technical solution of the present invention, the user information (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0037] In the scenario of using personal information for automated decision-making, the methods, devices, and systems provided by the embodiments of the present invention provide users with corresponding operation portals for users to choose to agree or reject the automated decision-making results; if the user chooses to reject, the expert decision-making process will be entered. The expression "automated decision-making" here refers to the activity of automatically analyzing and evaluating an individual's behavioral habits, interests and hobbies, or economic, health, credit status, etc. through computer programs, and making decisions. The expression "expert decision-making" here refers to the activity of making decisions by people who specialize in a certain field, have specialized experience, knowledge and skills, and have reached a certain level of professionalism.

[0038] Figure 1 A diagram showing an application scenario of the security inspection image detection method and device according to an embodiment of the present invention is shown.

[0039] like Figure 1As shown, the application scenario according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0040] The user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).

[0041] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0042] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0043] It should be noted that the security inspection image detection method provided in the embodiment of the present invention can generally be executed by the server 105. Accordingly, the security inspection image detection device provided in the embodiment of the present invention can generally be set in the server 105. The security inspection image detection method provided in the embodiment of the present invention can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Correspondingly, the security inspection image detection device provided in the embodiment of the present invention can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0044] It should be understood that Figure 1The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0045] Figure 2 A flow chart of a security inspection image detection method according to an embodiment of the present invention is shown.

[0046] like Figure 2 As shown, the security inspection image detection method of this embodiment includes operations S210 to S270, and the security inspection image detection method can be executed by an electronic device.

[0047] In operation S210, a security inspection image is acquired.

[0048] In operation S220, the security inspection image is processed using a slice convolutional network to obtain slice convolutional features.

[0049] In operation S230, the slice convolution features are processed using a multi-residual convolutional network to obtain residual convolution features.

[0050] In operation S240, the residual convolutional features are processed based on the attention pooling network to obtain pooled attention features.

[0051] In operation S250, the pooled attention features and the residual convolution features are processed based on the feature pyramid network to obtain attention fusion features and convolution attention features.

[0052] In operation S260, the attention fusion feature and the convolutional attention feature are processed based on the path aggregation network to obtain a fusion feature.

[0053] In operation S270, the detection head processes the attention fusion feature and the fusion feature to obtain a detection result of the security inspection image, and the detection result represents whether the security inspection image contains abnormal items.

[0054] According to an embodiment of the present invention, the security inspection image is obtained by scanning the object using a ray scanning device, and an open source dataset of abnormal objects in X-ray security inspection images can also be used. The dataset may include multiple X-ray security inspection images, and the security inspection images include multiple types of abnormal objects, which may include batons, pliers, hammers, power banks, scissors, wrenches, sprayers, lighters, etc., but are not limited to these, and the embodiments of the present disclosure do not limit this.

[0055] According to an embodiment of the present invention, a slice convolutional network is used to process security inspection images to obtain slice convolutional features. Specifically, the slice convolutional network can be used to slice the security inspection image, and then a convolution operation is performed to obtain the slice convolutional features. The size of the slice convolutional features can be , channel characteristics .

[0056] According to an embodiment of the present invention, a residual convolution feature can be obtained by processing slice convolution features using a multi-residual convolution network. Based on the attention pooling network, the residual convolution features can be processed, and the features at different scales corresponding to the residual convolution features can be pooled and the pooled features can be processed based on the attention mechanism to obtain the pooled attention features.

[0057] According to an embodiment of the present invention, the feature pyramid network is a top-down structure, which fuses the high-level feature map with the low-level feature map after upsampling. Pooled attention features and residual convolution features can be processed based on the feature pyramid network to obtain attention fusion features and convolution attention features.

[0058] According to an embodiment of the present invention, the path aggregation network is a bottom-up structure that can transmit position information to a higher level. Attention fusion features and convolutional attention features can be processed based on the path aggregation network to obtain fusion features.

[0059] According to an embodiment of the present invention, the detection head is responsible for converting features into specific detection results. Based on the detection head processing the attention fusion features and the fusion features, the detection results of the security inspection image can be obtained. The detection results characterize whether the security inspection image contains abnormal objects. The detection results include the category, bounding box information, and width and height of the target object.

[0060] According to an embodiment of the present invention, by using a slice convolution network to process security inspection images, slice convolution features can be obtained, and by using a multi-residual convolution network to process slice convolution features, residual convolution features can be obtained. By processing residual convolution features based on an attention pooling network, features at different scales corresponding to the residual convolution features can be pooled and the pooled features can be processed based on an attention mechanism to obtain pooled attention features, which enriches semantic information, enhances the receptive field, and improves the detection of small-sized targets. By processing pooled attention features and residual convolution features based on a feature pyramid network, attention fusion features and convolution attention features can be obtained. By processing attention fusion features and convolution attention features based on a path aggregation network, fusion features can be obtained. By processing attention fusion features and fusion features based on a detection head, detection results of security inspection images can be obtained, thereby improving detection accuracy, improving the detection performance of small-sized targets and occluded targets, and improving the generalization ability of the model.

[0061] According to an embodiment of the present invention, residual convolution features are processed based on an attention pooling network to obtain pooled attention features, including: using a serial pooling layer to process residual convolution features to obtain multi-scale fusion features; processing multi-scale fusion features based on a self-attention layer to obtain pooled attention features, and the attention pooling network may include a serial pooling layer and a self-attention layer.

[0062] According to an embodiment of the present invention, by processing residual convolution features with a serial pooling layer, multi-scale fusion features can be obtained. By simplifying the pooling operation, the calculation speed is significantly improved. By processing multi-scale fusion features based on the self-attention layer, pooled attention features can be obtained. The global context information can be effectively captured, and the relationship between all features in the image can be considered at the same time. This global information capture capability enables the model to better understand the overall structure in the security inspection image and avoid misjudgment due to the loss of local features.

[0063] According to an embodiment of the present invention, a serial pooling layer is used to process residual convolution features to obtain multi-scale fusion features, including: using a convolution normalization block to process residual convolution features to obtain a first residual normalized feature; performing a multi-scale pooling operation on the first residual normalized feature to obtain a multi-scale pooled feature; fusing the first residual normalized feature and the multi-scale pooled feature to obtain a multi-scale fusion feature, and the serial pooling layer includes a convolution normalization block.

[0064] Figure 3 A structural diagram of a serial pooling layer according to an embodiment of the present invention is shown.

[0065] like Figure 3 As shown, the convolution normalization block 310 can be used to process the residual convolution feature to obtain the first residual normalized feature; based on the first pooling block 320, the first residual normalized feature can be processed to obtain the first pooling feature; based on the second pooling block 330, the first pooling feature can be processed to obtain the second pooling feature; the N-1th pooling feature can be processed based on the Nth pooling block 340 to obtain the Nth pooling feature, and the first residual normalized feature and the multi-scale pooling feature can be fused. The multi-scale pooling feature may include the first pooling feature to the Nth pooling feature, and finally the multi-scale fusion feature can be obtained, wherein N is an integer greater than 1, and the N pooling blocks all use the same pooling kernel, for example, the pooling kernel is Maximum pooling. The embodiment of the present invention does not limit the size of the pooling kernel.

[0066] According to an embodiment of the present invention, a convolution normalization block is used to process residual convolution features to obtain a first residual normalized feature, and a multi-scale pooling operation is performed on the first residual normalized feature to obtain a multi-scale pooled feature, wherein the same size of pooling kernels are used in the multi-scale pooling operation, thereby reducing the complexity of the pooling operation. In addition, the use of the multi-scale pooling operation can more efficiently capture multi-scale information by gradually downsampling the first residual normalized feature, while reducing the amount of calculation. The first residual normalized feature and the multi-scale pooling feature are fused to obtain a multi-scale fusion feature containing multi-scale information. This fusion method retains contextual information of different scales while reducing the computational redundancy caused by traditional pooling operations. By reducing the number of pooling kernels and optimizing the pooling operation, the memory usage and computational complexity are significantly reduced, thereby enhancing the robustness of the model.

[0067] According to an embodiment of the present invention, multi-scale fusion features are processed based on a self-attention layer to obtain pooled attention features, including: processing multi-scale fusion features based on a convolution block to obtain query features, key features and value features; multiplying the query features and the key features to obtain pixel association features; processing the pixel association features using a normalization algorithm to obtain attention weights; obtaining value attention features based on the attention weights and value features; processing the value attention features based on the convolution block to obtain value attention convolution features; performing residual connection and addition operations on the value attention convolution features and the multi-scale fusion features to obtain pooled attention features.

[0068] Figure 4 A structural diagram of a self-attention layer according to an embodiment of the present invention is shown.

[0069] like Figure 4 As shown, based on the first convolution block 420, the multi-scale fusion feature 410 is processed to obtain the query feature Query, based on the second convolution block 430, the multi-scale fusion feature 410 is processed to obtain the key feature Key, and based on the third convolution block 440, the multi-scale fusion feature 410 is processed to obtain the value feature Value. The Query and Key are reshaped, and they are multiplied to obtain the pixel association feature to calculate the relationship between each pixel point on the Query and Key and other pixels. The pixel association feature can be normalized using the softmax function to obtain the attention weight. The attention weight can be weighted to the Value to obtain the value attention feature; the value attention feature is processed based on the fourth convolution block 450 to obtain the value attention convolution feature. Finally, the value attention convolution feature and the multi-scale fusion feature are residually connected and summed to obtain the pooled attention feature.

[0070] According to an embodiment of the present invention, multi-scale fusion features are processed based on convolution blocks to obtain query features, key features and value features; pixel association features are multiplied by the query features and the key features to obtain pixel association features, so as to evaluate the relationship between each element in the feature and all other elements, so that the model can capture the global information in the multi-scale fusion features and better understand the long-distance dependencies. The pixel association features are processed using a normalization algorithm to obtain attention weights. According to the attention weights and value features, value attention features can be obtained, and the focus of attention can be automatically adjusted according to the input of different key information; the value attention features are processed based on the convolution blocks to obtain value attention convolution features; residual connection and addition operations are performed on the value attention convolution features and the multi-scale fusion features to obtain pooled attention features. In this process, features of different scales can be dynamically fused, which can significantly improve the model's detection performance for multi-scale targets. When dealing with complex scenes such as occlusion and distinction between similar objects, the accuracy and robustness of detection can be significantly improved.

[0071] According to an embodiment of the present invention, a multi-residual convolutional network includes I feature extraction layers, where I is an integer greater than 1; a slice convolutional feature is processed using a multi-residual convolutional network to obtain a residual convolutional feature, including: inputting the slice convolutional feature into the first feature extraction layer and outputting the first residual convolutional feature; inputting the i-1th residual convolutional feature into the i-th feature extraction layer and outputting the i-th residual convolutional feature; when i=I, determining I residual convolutional features based on the first residual convolutional feature to the Ith residual convolutional feature, where I≥i>1, and I and i are integers.

[0072] According to an embodiment of the present invention, the multi-residual convolutional network may include i feature extraction layers, where i is an integer greater than 1. The CBS module and the C3 module of the feature extraction layer may be used to extract features and reduce parameters. The CBS module may include a convolution block, a batch normalization block, and a SiLU activation function. The SiLU activation function has the characteristics of no upper bound, lower bound, smoothness, and non-monotonicity. The layer may output Channel data The C3 module can consist of three The module consists of a number of backbone residual blocks. The number of backbone residual blocks is specified by the parameter and is used for feature extraction. Finally, the output size is Channel data .

[0073] According to an embodiment of the present invention, the slice convolution feature is input into the first feature extraction layer, and the first residual convolution feature can be output; the i-1th residual convolution feature is input into the i-th feature extraction layer, and the i-th residual convolution feature can be output; when i=I, I residual convolution features can be determined based on the first residual convolution feature to the Ith residual convolution feature, and I≥i>1.

[0074] According to an embodiment of the present invention, based on the detection head processing attention fusion features and fusion features, the detection results of the security inspection image are obtained, including: based on the first detection head processing the attention fusion features, a first detection result of the security inspection image can be obtained, and the first detection result characterizes whether there are abnormal items with a first size in the security inspection image; based on the second detection head processing the fusion features, a second detection result of the security inspection image can be obtained, and the second detection result characterizes whether there are abnormal items with a second size in the security inspection image, and the size difference value between the first size and the second size is greater than a preset size threshold. The detection head may include a first detection head and a second detection head.

[0075] According to an embodiment of the present invention, for example, the scale of the attention fusion feature is 20×20, and the scale of the fusion feature is 80×80. The image area covered by each pixel in the feature map corresponding to the attention fusion feature is larger, which is suitable for capturing large-sized targets. The image area covered by each pixel in the feature map corresponding to the fusion feature is smaller, which is suitable for capturing small-sized targets.

[0076] According to the embodiments of the present invention, by using different detection heads to detect objects of different sizes, abnormal objects of different sizes can be detected in a targeted manner, thereby significantly improving the efficiency and accuracy of security inspection image detection.

[0077] According to an embodiment of the present invention, before using a slice convolutional network to process a security inspection image and obtaining a slice convolutional feature, the method also includes: using an image local adjustment algorithm to adjust the contrast and brightness of an initial security inspection image to obtain an intermediate image; and performing image enhancement on the intermediate image based on a mosaic algorithm to obtain a security inspection image.

[0078] According to an embodiment of the present invention, the contrast and brightness of the initial security inspection image can be adjusted using an image local adjustment algorithm to obtain an intermediate image, thereby increasing the details and texture information of the image. The specific calculation is shown in formula (1):

[0079] (1);

[0080] Where x and y represent the pixel coordinates of the image. Indicates the brightness of the enhanced image; Represents the brightness value of the corresponding pixel in the input image. Indicates the maximum brightness of the input image. represents the logarithmic mean of the brightness value of the input image, m×n represents the size of the image, Indicates a non-zero and very small constant. This is used to prevent the image from having a brightness of 0 and to avoid value overflow when performing logarithmic calculations on pure black pixels.

[0081] According to an embodiment of the present invention, the intermediate image can be enhanced based on the mosaic algorithm to obtain a security inspection image. The image enhancement may include hue saturation value (HSV) chromaticity, saturation, exposure transformation, random rotation and translation, etc. The image size processing method can also be used to convert the image size Security inspection images The height-width ratio is scaled to image , the edges that do not meet the conditions can be filled with white bars, and the resulting image Input to the slice convolutional network.

[0082] According to the embodiment of the present invention, by adjusting the brightness, contrast, color and other attributes of the initial security inspection image, the details and texture information of the image are increased, so as to facilitate the subsequent accurate target detection of the security inspection image.

[0083] According to an embodiment of the present invention, the image format of the original data set is Portable Network Graphics (png) format, and the images are divided into four folders: training, simple, difficult, and occluded. The labels of the images are stored in four JavaScript Object Notation (json) files according to the folders. The json file contains the category, bounding box information, width and height, file name, etc. of the abnormal objects in each image.

[0084] According to an embodiment of the present invention, the data set can be divided into a training set and a test set in a ratio of 6:4. The training set has a total of 29,457 images, and the test set is divided into three subsets according to the difficulty of abnormal object detection: simple, difficult, and occluded, with the number of images being 9,482, 3,733, and 5,005, respectively.

[0085] According to an embodiment of the present invention, a small target may refer to a pixel smaller than 32×32, a medium target may refer to a pixel between 32×32 and 96×96, and a large target may refer to a pixel larger than 96×96. The specific conditions of small, medium and large targets in the data set are shown in Table 1.

[0086] Table 1. Table of small, medium and large targets in the dataset

[0087] Dataset Small goals Medium target (number) Big goal Simple 85 2278 7119 difficulty 65 1759 7068 Occlusion 130 1392 3486 train 278 11480 27950

[0088] According to an embodiment of the present invention, the dataset is collected from airports, subway stations and train stations, covering various situations of abnormal object detection in reality, especially deliberately hidden objects. The characteristic of this dataset is that the hidden subset in its test set focuses on abnormal objects that are deliberately hidden in cluttered objects, for example, by wrapping wires around abnormal objects to block them, interfering with the detection results; the blockage of abnormal objects is also reflected in the fact that the items in personal luggage are usually randomly placed and overlap each other. These characteristics make the detection of abnormal objects more difficult.

[0089] Figure 5 A network structure diagram of a detection model according to an embodiment of the present invention is shown.

[0090] like Figure 5 As shown, the security inspection image is processed using a slice convolution network 510 to obtain slice convolution features, the slice convolution features are input into the i-1th feature extraction layer 521, and the i-1th residual convolution features are output; the i-1th residual convolution features are input into the i-th feature extraction layer 522, and the i-th residual convolution features are output; the multi-residual convolution network 520 may include I feature extraction layers.

[0091] According to an embodiment of the present invention, the serial pooling layer 531 is used to process the residual convolution features to obtain multi-scale fusion features; the multi-scale fusion features are processed based on the self-attention layer 532 to obtain the pooled attention features. The attention pooling network 530 includes a serial pooling layer 531 and a self-attention layer 532.

[0092] According to an embodiment of the present invention, the pooled attention features and the residual convolution features are processed based on the feature pyramid network 540 to obtain the attention fusion features and the convolution attention features. The attention fusion features and the convolution attention features are processed based on the path aggregation network 550 to obtain the fusion features. The attention fusion features are processed based on the first detection head 561 to obtain the first detection result of the security inspection image, and the fusion features are processed based on the second detection head 562 to obtain the second detection result of the security inspection image. The detection head 560 includes the first detection head 561 and the second detection head 562.

[0093] Figure 6 A flow chart of a detection model training and conversion method according to an embodiment of the present invention is shown.

[0094] like Figure 6 As shown, the detection model training and conversion method includes operations S610 to S619.

[0095] In operation S610, the sample security inspection image and label data are read.

[0096] In operation S611, the sample security inspection image is processed using a slice convolutional network to obtain sample slice convolution features.

[0097] In operation S612, the sample slice convolution features are processed using a multi-residual convolutional network to obtain sample residual convolution features.

[0098] In operation S613, the sample residual convolution features are processed based on the attention pooling network to obtain sample pooling attention features.

[0099] In operation S614, the sample pooled attention features and the sample residual convolution features are processed based on the feature pyramid network to obtain sample attention fusion features and sample convolution attention features.

[0100] In operation S615, the sample attention fusion feature and the sample convolutional attention feature are processed based on the path aggregation network to obtain a sample fusion feature.

[0101] In operation S616, the detection result of the sample security inspection image is obtained based on processing the sample attention fusion feature and the sample fusion feature by the detection head.

[0102] In operation S617, the detection result and label data of the sample security inspection image are processed according to the loss function to obtain a target loss value.

[0103] In operation S618, the detection model is trained according to the target loss value to obtain a trained detection model.

[0104] In operation S619, the trained detection model is converted into a target format and embedded into a target chip for hardware accelerated reasoning.

[0105] According to an embodiment of the present invention, the json annotation file information in the data set can be read, and the annotation information can be converted from the two-point coordinate form Convert to center point coordinates and height and width format Can read sample security inspection images and label data.

[0106] According to an embodiment of the present invention, the sample attention fusion feature and the sample fusion feature are decoded into corresponding position prediction boxes on the image, and then score sorting and non-maximum suppression (NMS) screening are performed to obtain the detection results of the sample security inspection image. The detection results and label data of the sample security inspection image are processed according to the loss function to obtain the target loss value. According to the target loss value, the detection model is trained to obtain the trained detection model. Among them, the comprehensive intersection over union (CloU) and intersection over union (IoU) can be used to evaluate the performance of the detection model. The specific calculation of the comprehensive intersection over union and intersection over union is shown in formula (2).

[0107] (2);

[0108] Among them, IoU represents the predicted box b and the real box The ratio of the intersection area to the union area, is the predicted box b and the real box The square of the center point distance between them, c is the diagonal length of the bounding box (the smallest rectangular box surrounding the predicted box and the real box), represents the weight function, A measure of aspect ratio consistency. , is the width and height of the prediction box b, , is the width and height of the actual frame.

[0109] The final loss function is defined as follows. The loss consists of three parts: category loss, object confidence loss, and bounding box positioning loss. The optimizer is used to minimize the loss function to train the network model. Both category loss and confidence loss use the binary cross entropy loss function, and the calculation formula (3) is as follows:

[0110] (3);

[0111] Where n represents the total number of samples. represents the original value of the model output, represents the true label of the i-th sample, represents the sigmoid function, Represents an exponential function.

[0112] According to an embodiment of the present invention, after obtaining the trained detection model, target-related features of abnormal objects can be obtained, and the trained detection model can be converted into a target format, such as an RKNN format, and embedded into a target chip for hardware accelerated reasoning.

[0113] According to an embodiment of the present invention, the trained detection model is converted into a target format and embedded into a target chip for hardware accelerated reasoning. The security inspection image is processed using a system with a target chip to obtain a detection result of the security inspection image. The following steps may be included: the system obtains an image containing an abnormal security inspection item from an image acquisition unit, and proportionally adjusts the image to a preset size, for example, , sent to the input queue Initialize the Neural Network Processing Unit (NPU) operating environment and continuously read from the input queue The image is acquired, and the NPU with the highest computing performance is responsible for image inference and sends the inference results to the inference queue ; Read the inference results from the output queue and select the detection box with the highest score , remove the output queue, if the remaining candidate boxes are of Value greater than , then the candidate box is deleted; the system draws a frame on the original image, displays the detection results in real time, and sends them to the output queue Save; control specific hardware units or cores on the RKNN board through multiple processes, which are assigned to different processes to handle multiple tasks to achieve parallel processing and improve overall computing efficiency. Through the collaborative work of the Central Processing Unit (CPU) and the NPU, the CPU handles complex logic and control processes, while the NPU focuses on executing deep learning algorithms to maximize hardware performance.

[0114] Figure 7 The RKNN image reasoning flow chart of the real-time detection system for security inspection images based on the target chip in the present invention is shown.

[0115] like Figure 7 As shown, you can first create an RKNN object S710, call the config interface to set the preprocessing parameters of the model S720, call the load_onnx interface to import the ONNX model S730, and call the build interface to build the RKNN model S740. You can initialize the operating environment S750 by calling the init_runtime interface, specify the target platform and NPU core scheduling mode, and use the inference interface for inference to obtain the inference result S760. The input data needs to be preprocessed according to the model requirements. You can also call the export_RKNN interface to export the RKNN model S770. After the reasoning is completed, you can call the release interface to release the RKNN object S780 to release resources.

[0116] According to an embodiment of the present invention, a trained detection model with target weights can be converted into an RKNN model in a virtual machine environment through a model conversion tool, and an X-ray security inspection image detection system can be built on a target chip.

[0117] According to an embodiment of the present invention, the system uses a target chip to construct an X-ray security inspection image detection system. The system obtains an image to be detected through input; the system processes the input X-ray security inspection image. Perform multi-process real-time detection, each image It includes preprocessing, target reasoning and postprocessing; the target chip transmits the detection results back, and the detection system interface displays and saves them. Specifically, it can include preprocessing, target reasoning, post-processing display and saving, and use the system's multi-process CPU and NPU to work together to speed up; the image acquisition unit obtains the X-ray security inspection image and preprocesses it through the image preprocessing unit; the target reasoning unit infers the preprocessed image based on the RKNN model to obtain a reasonable detection frame; the post-processing display unit calls NMS non-maximum suppression to filter repeated frames, and finally marks the original image and displays the reasoning results, realizing real-time detection of X-ray security inspection images on an embedded platform.

[0118] Figure 8 A flow chart of the present invention is shown in which the converted RKNN model in the real-time detection system of security inspection images based on the target chip is transmitted to the target chip for reasoning execution.

[0119] like Figure 8 As shown, you can first create an RKNN object S810, and based on the RKNN object, use the load_rknn interface to import the RKNN model S820: initialize the operating environment S830 by calling the init_runtime interface, specify the target platform and NPU core scheduling mode: use the inference interface to perform inference and obtain the inference result S840. The input data needs to be preprocessed according to the model requirements. After the inference is completed, call the release interface to release the RKNN object S850 to release resources.

[0120] Fig. 9 A schematic diagram of a security inspection image detection result obtained based on the security inspection image detection method in the present invention is shown.

[0121] like Fig. 9 As shown in FIG. 1 , the image post-processing result of the real-time detection system for security inspection images based on the target chip includes abnormal objects such as a hammer, a wrench, and a lighter.

[0122] Fig.10 The structure block diagram of the security inspection image detection device according to an embodiment of the present invention is shown.

[0123] like Fig.10 As shown, the security inspection image detection device 1000 of this embodiment includes an acquisition module 1010, a slice convolution feature acquisition module 1020, a residual convolution feature acquisition module 1030, a pooled attention feature acquisition module 1040, a convolution attention fusion acquisition module 1050, a fusion feature acquisition module 1060 and a detection result acquisition module 1070.

[0124] An acquisition module 1010 is used to acquire a security inspection image, wherein the security inspection image is obtained by scanning an object using a ray scanning device;

[0125] The slice convolution feature obtaining module 1020 is used to process the security inspection image using the slice convolution network to obtain the slice convolution feature;

[0126] A residual convolution feature obtaining module 1030 is used to process the slice convolution features using a multi-residual convolution network to obtain residual convolution features;

[0127] A pooled attention feature obtaining module 1040 is used to process the residual convolution features based on the attention pooling network to obtain a pooled attention feature;

[0128] A convolutional attention fusion obtaining module 1050 is used to process the pooled attention features and the residual convolution features based on the feature pyramid network to obtain attention fusion features and convolutional attention features;

[0129] A fusion feature obtaining module 1060 is used to process the attention fusion feature and the convolution attention feature based on a path aggregation network to obtain a fusion feature;

[0130] The detection result obtaining module 1070 is used to process the attention fusion feature and the fusion feature based on the detection head to obtain the detection result of the security inspection image, and the detection result represents whether the security inspection image contains abnormal items.

[0131] According to an embodiment of the present invention, by using a slice convolution network to process security inspection images, slice convolution features can be obtained, and by using a multi-residual convolution network to process slice convolution features, residual convolution features can be obtained. By processing residual convolution features based on an attention pooling network, features at different scales corresponding to the residual convolution features can be pooled and the pooled features can be processed based on an attention mechanism to obtain pooled attention features, which enriches semantic information, enhances the receptive field, and improves the detection of small-sized targets. By processing pooled attention features and residual convolution features based on a feature pyramid network, attention fusion features and convolution attention features can be obtained. By processing attention fusion features and convolution attention features based on a path aggregation network, fusion features can be obtained. By processing attention fusion features and fusion features based on a detection head, detection results of security inspection images can be obtained, thereby improving detection accuracy, improving the detection performance of small-sized targets and occluded targets, and improving the generalization ability of the model.

[0132] According to an embodiment of the present invention, the pooling attention feature obtaining module 1040 includes: a multi-scale fusion feature obtaining sub-module and a pooling attention feature obtaining sub-module.

[0133] The multi-scale fusion feature acquisition submodule is used to process the residual convolution features using the serial pooling layer to obtain the multi-scale fusion features.

[0134] The pooling attention feature acquisition submodule is used to process multi-scale fusion features based on the self-attention layer to obtain the pooling attention feature. The attention pooling network includes a serial pooling layer and a self-attention layer.

[0135] According to an embodiment of the present invention, the multi-scale fusion feature obtaining submodule includes: a first residual normalization feature obtaining unit, a multi-scale pooling feature obtaining unit and a multi-scale fusion feature obtaining unit.

[0136] The first residual normalized feature obtaining unit is used to process the residual convolution feature using the convolution normalization block to obtain the first residual normalized feature.

[0137] The multi-scale pooling feature obtaining unit is used to perform a multi-scale pooling operation on the first residual normalized feature to obtain a multi-scale pooling feature.

[0138] The multi-scale fusion feature obtaining unit is used to fuse the first residual normalization feature and the multi-scale pooling feature to obtain the multi-scale fusion feature. The serial pooling layer includes a convolution normalization block.

[0139] According to an embodiment of the present invention, the pooling attention feature obtaining submodule includes: a convolution block processing unit, a pixel association feature obtaining unit, an attention weight obtaining unit, a value attention feature obtaining unit, a value attention convolution feature obtaining unit and a pooling attention feature obtaining unit.

[0140] The convolution block processing unit is used to process the multi-scale fusion features based on the convolution block to obtain the query features, key features and value features.

[0141] The pixel association feature obtaining unit is used to multiply the query feature and the key feature to obtain the pixel association feature.

[0142] The attention weight obtaining unit is used to process the pixel association features using a normalization algorithm to obtain the attention weight.

[0143] The value attention feature obtaining unit is used to obtain the value attention feature according to the attention weight and the value feature.

[0144] The value attention convolution feature obtaining unit is used to process the value attention feature based on the convolution block to obtain the value attention convolution feature.

[0145] The pooling attention feature obtaining unit is used to perform residual connection and addition operations on the value attention convolution feature and the multi-scale fusion feature to obtain the pooling attention feature.

[0146] According to an embodiment of the present invention, the multi-residual convolutional network includes i feature extraction layers, where i is an integer greater than 1.

[0147] According to an embodiment of the present invention, the residual convolution feature obtaining module 1030 includes: a first residual convolution feature obtaining submodule, an i-th residual convolution feature obtaining submodule and a determination submodule.

[0148] The first residual convolution feature obtaining submodule is used to input the slice convolution feature into the first feature extraction layer and output the first residual convolution feature.

[0149] The i-th residual convolution feature obtaining submodule is used to input the i-1th residual convolution feature into the i-th feature extraction layer and output the i-th residual convolution feature.

[0150] A determination submodule is used to determine I residual convolution features based on the 1st residual convolution feature to the Ith residual convolution feature when i=I, I≥i>1, and I and i are integers.

[0151] According to an embodiment of the present invention, the detection result obtaining module 1070 includes: a first detection result obtaining submodule and a second detection result obtaining submodule.

[0152] The first detection result obtaining submodule is used to process the attention fusion feature based on the first detection head to obtain a first detection result of the security inspection image, and the first detection result represents whether there is an abnormal object with a first size in the security inspection image.

[0153] The second detection result obtaining submodule is used to process the fusion features based on the second detection head to obtain the second detection result of the security inspection image, the second detection result characterizes whether there are abnormal items with a second size in the security inspection image, and the size difference between the first size and the second size is greater than a preset size threshold. The detection head includes a first detection head and a second detection head.

[0154] According to an embodiment of the present invention, the security inspection image detection device 1000 further includes: an intermediate image obtaining module and a security inspection image obtaining module.

[0155] The intermediate image obtaining module is used to adjust the contrast and brightness of the initial security inspection image by using an image local adjustment algorithm to obtain an intermediate image.

[0156] The security inspection image acquisition module is used to enhance the intermediate image based on the mosaic algorithm to obtain the security inspection image.

[0157] According to an embodiment of the present invention, any multiple modules among the acquisition module 1010, the slice convolution feature acquisition module 1020, the residual convolution feature acquisition module 1030, the pooled attention feature acquisition module 1040, the convolution attention fusion acquisition module 1050, the fusion feature acquisition module 1060 and the detection result acquisition module 1070 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the acquisition module 1010, the slice convolution feature acquisition module 1020, the residual convolution feature acquisition module 1030, the pooled attention feature acquisition module 1040, the convolution attention fusion acquisition module 1050, the fusion feature acquisition module 1060 and the detection result acquisition module 1070 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or can be implemented in any one of the three implementation methods of software, hardware and firmware or in an appropriate combination of any of them. Alternatively, at least one of the acquisition module 1010, the slice convolution feature acquisition module 1020, the residual convolution feature acquisition module 1030, the pooled attention feature acquisition module 1040, the convolution attention fusion acquisition module 1050, the fusion feature acquisition module 1060 and the detection result acquisition module 1070 can be at least partially implemented as a computer program module, which can perform the corresponding function when the computer program module is executed.

[0158] Fig.11 A block diagram of an electronic device suitable for implementing a security inspection image detection method according to an embodiment of the present invention is shown.

[0159] like Fig.11 As shown, the electronic device according to an embodiment of the present invention includes a processor 1101, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1102 or a program loaded from a storage part 1108 to a random access memory (RAM) 1103. The processor 1101 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 1101 may also include an onboard memory for caching purposes. The processor 1101 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0160] In RAM 1103, various programs and data required for the operation of electronic device 1100 are stored. Processor 1101, ROM 1102 and RAM 1103 are connected to each other via bus 1104. Processor 1101 performs various operations of the method flow according to the embodiment of the present invention by executing the program in ROM 1102 and / or RAM 1103. It should be noted that the program can also be stored in one or more memories other than ROM 1102 and RAM 1103. Processor 1101 can also perform various operations of the method flow according to the embodiment of the present invention by executing the program stored in the one or more memories.

[0161] According to an embodiment of the present invention, the electronic device 1100 may further include an input / output (I / O) interface 1105, which is also connected to the bus 1104. The electronic device 1100 may further include one or more of the following components connected to the input / output (I / O) interface 1105: an input portion 1106 including a keyboard, a mouse, etc.; an output portion 1107 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 1108 including a hard disk, etc.; and a communication portion 1109 including a network interface card such as a LAN card, a modem, etc. The communication portion 1109 performs communication processing via a network such as the Internet. A drive 1110 is also connected to the input / output (I / O) interface 1105 as needed. A removable medium 1111, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1110 as needed, so that a computer program read therefrom is installed into the storage portion 1108 as needed.

[0162] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0163] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 1102 and / or RAM 1103 described above and / or one or more memories other than ROM 1102 and RAM 1103.

[0164] The embodiment of the present invention also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the security inspection image detection method provided by the embodiment of the present invention.

[0165] The computer program executes the above functions defined in the system / device of the embodiment of the present invention when it is executed by the processor 1101. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0166] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 1109, and / or installed from a removable medium 1111. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0167] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1109, and / or installed from the removable medium 1111. When the computer program is executed by the processor 1101, the above functions defined in the system of the embodiment of the present invention are performed. According to the embodiment of the present invention, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0168] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0169] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0170] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present invention may be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features described in the various embodiments of the present invention may be combined and / or combined in various ways. All of these combinations and / or combinations fall within the scope of the present invention.

[0171] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination advantageously. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A security inspection image detection method, characterized in that: include: Acquiring a security inspection image, wherein the security inspection image is obtained by scanning the object using a ray scanning device; Processing the security inspection image using a slice convolutional network to obtain slice convolutional features; Processing the slice convolution features using a multi-residual convolutional network to obtain residual convolution features; Processing the residual convolutional features based on an attention pooling network to obtain a pooled attention feature; Processing the pooled attention feature and the residual convolution feature based on a feature pyramid network to obtain an attention fusion feature and a convolution attention feature; Processing the attention fusion feature and the convolutional attention feature based on a path aggregation network to obtain a fusion feature; The detection head processes the attention fusion feature and the fusion feature to obtain a detection result of the security inspection image, wherein the detection result indicates whether the security inspection image contains abnormal items.

2. The method according to claim 1, characterized in that The processing of the residual convolutional features based on the attention pooling network to obtain the pooled attention features includes: Processing the residual convolution features using a serial pooling layer to obtain multi-scale fusion features; The multi-scale fusion features are processed based on the self-attention layer to obtain pooled attention features, and the attention pooling network includes the serial pooling layer and the self-attention layer.

3. The method according to claim 2, characterized in that The method of processing the residual convolution features by using a serial pooling layer to obtain a multi-scale fusion feature includes: Processing the residual convolution feature using a convolution normalization block to obtain a first residual normalized feature; Performing a multi-scale pooling operation on the first residual normalized feature to obtain a multi-scale pooling feature; The first residual normalized feature and the multi-scale pooling feature are fused to obtain the multi-scale fused feature, and the serial pooling layer includes the convolution normalization block.

4. The method according to claim 2, characterized in that: The step of processing the multi-scale fusion features based on the self-attention layer to obtain the pooled attention features includes: Processing the multi-scale fusion features based on a convolution block to obtain query features, key features, and value features; Multiplying the query feature and the key feature to obtain a pixel-related feature; Processing the pixel association features using a normalization algorithm to obtain an attention weight; Obtaining a value attention feature according to the attention weight and the value feature; Processing the value attention feature based on the convolution block to obtain a value attention convolution feature; Residual connection and addition operations are performed on the value attention convolution feature and the multi-scale fusion feature to obtain the pooled attention feature.

5. The method according to claim 1, characterized in that The multi-residual convolutional network includes I feature extraction layers, where I is an integer greater than 1; The using a multi-residual convolutional network to process the slice convolutional features to obtain residual convolutional features includes: Inputting the slice convolution feature into the first feature extraction layer, and outputting the first residual convolution feature; Inputting the i-1th residual convolution feature to the i-th feature extraction layer, and outputting the i-th residual convolution feature; and When i=I, I residual convolution features are determined based on the 1st residual convolution feature to the Ith residual convolution feature, and I≥i>1.

6. The method according to claim 1, characterized in that The detection head-based processing of the attention fusion feature and the fusion feature to obtain the detection result of the security inspection image includes: Processing the attention fusion feature based on the first detection head to obtain a first detection result of the security inspection image, wherein the first detection result indicates whether there is an abnormal object with a first size in the security inspection image; Based on the second detection head processing the fusion feature, a second detection result of the security inspection image is obtained, the second detection result characterizes whether there is an abnormal object of a second size in the security inspection image, and the size difference between the first size and the second size is greater than a preset size threshold. The detection head includes the first detection head and the second detection head.

7. The method according to claim 1, characterized in that Before using the slice convolution network to process the security inspection image to obtain the slice convolution feature, the method further includes: The contrast and brightness of the initial security inspection image are adjusted using the image local adjustment algorithm to obtain an intermediate image; The intermediate image is enhanced based on a mosaic algorithm to obtain the security inspection image.

8. A security inspection image detection device, characterized in that: include: An acquisition module, used for acquiring a security inspection image, wherein the security inspection image is obtained by scanning an object using a ray scanning device; A slice convolution feature acquisition module, used for processing the security inspection image using a slice convolution network to obtain a slice convolution feature; A residual convolution feature acquisition module, used for processing the slice convolution feature using a multi-residual convolution network to obtain a residual convolution feature; A pooled attention feature obtaining module, used for processing the residual convolutional features based on an attention pooling network to obtain a pooled attention feature; A convolutional attention fusion obtaining module, used for processing the pooled attention feature and the residual convolution feature based on a feature pyramid network to obtain an attention fusion feature and a convolutional attention feature; A fusion feature obtaining module, used for processing the attention fusion feature and the convolution attention feature based on a path aggregation network to obtain a fusion feature; The detection result obtaining module is used to process the attention fusion feature and the fusion feature based on the detection head to obtain the detection result of the security inspection image, and the detection result represents whether the security inspection image contains abnormal items.

9. An electronic device, comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images