Contraband detection method and apparatus, electronic device, and storage medium

Abstract

The application discloses a contraband detection method and device, electronic equipment and storage medium, comprising: obtaining a security check image to be detected; obtaining a plurality of initial feature maps corresponding to the security check image; and obtaining a highlight feature map corresponding to each initial feature map to obtain a plurality of highlight feature maps; fusing the plurality of initial feature maps and the plurality of highlight feature maps to obtain a plurality of feature enhancement maps; and obtaining a contraband detection result of the security check image according to the plurality of feature enhancement maps. The application obtains the plurality of initial feature maps for the first time, and then obtains the highlight feature maps corresponding to the plurality of initial feature maps respectively, and highlights the features in the security check image again; and the plurality of initial feature maps and the plurality of highlight feature maps are further fused to highlight the features in the security check image. The application highlights the recognized image features multiple times, effectively improves the recognition accuracy of the image features, that is, improves the accuracy of the contraband features in the recognized image, so as to improve the accuracy of the recognized contraband.

Description

Technical Field

[0001] This application relates to the field of communication technology, specifically to a method, apparatus, electronic device, and storage medium for detecting contraband. Background Technology

[0002] In parcel delivery scenarios, security checks are typically required to prevent the mailing of prohibited items such as flammable and explosive materials. Current technologies usually utilize X-rays for inspection; however, the monochromatic colors of X-ray images result in slight distortion and blurring compared to real images, making it difficult to accurately identify prohibited items. This leads to low accuracy in identifying prohibited items. Summary of the Invention

[0003] This application provides a method, apparatus, electronic device, and storage medium for detecting contraband, aiming to solve the problem of low accuracy in the identification of contraband in the prior art.

[0004] On the one hand, this application provides a method for detecting prohibited substances, the method comprising:

[0005] Acquire the security inspection image to be inspected;

[0006] Feature extraction is performed on the security inspection image to obtain multiple initial feature maps corresponding to the security inspection image;

[0007] Obtain the highlight feature map corresponding to each of the plurality of initial feature maps to obtain a plurality of highlight feature maps, which are used to enhance the image features in the initial feature maps;

[0008] The plurality of initial feature maps and the plurality of highlight feature maps are fused to obtain a plurality of feature enhancement maps, which are used to further enhance the image features in the initial feature maps;

[0009] Contraband detection is performed based on the multiple feature enhancement maps to obtain the contraband detection results of the security inspection image.

[0010] In some embodiments of this application, the step of extracting features from the security inspection image to obtain multiple initial feature maps corresponding to the security inspection image includes:

[0011] The security inspection image is subjected to feature extraction using a preset feature recognition model to obtain multiple initial feature maps corresponding to the security inspection image.

[0012] In some embodiments of this application, the preset feature recognition model includes multiple convolutional layers of different sizes, with each convolutional layer extracting an initial feature map;

[0013] The step of obtaining the highlighted feature map corresponding to each of the plurality of initial feature maps to obtain a plurality of highlighted feature maps includes:

[0014] Using any initial feature map among the plurality of initial feature maps as the target initial feature map, determine the first highlighted feature map corresponding to the target initial feature map;

[0015] Among the plurality of convolutional layers, the target convolutional layer corresponding to the target initial feature map is determined, so as to determine the size of the target convolutional layer;

[0016] Based on the size of the target convolutional layer, determine the second highlighted feature map corresponding to each of the other initial feature maps in the plurality of initial feature maps, excluding the target initial feature map;

[0017] The plurality of highlight feature maps include the first highlight feature map and the second highlight feature map.

[0018] In some embodiments of this application, determining the second highlighted feature map corresponding to each of the other initial feature maps (excluding the target initial feature map) in the plurality of initial feature maps according to the size of the target convolutional layer includes:

[0019] Based on the size of the target convolutional layer, the other initial feature maps are sampled to transform them into the same size as the target convolutional layer, resulting in multiple first initial feature maps.

[0020] Each of the plurality of first initial feature maps is then determined to have its corresponding second highlighted feature map, resulting in a plurality of second highlighted feature maps.

[0021] In some embodiments of this application, the fusion of the plurality of initial feature maps and the plurality of highlighted feature maps to obtain a plurality of enhanced feature maps includes:

[0022] Using any second highlight feature map among the plurality of second highlight feature maps as the target highlight feature map, determine the target first initial feature map corresponding to the target highlight feature map;

[0023] Calculate the first product between the target highlighted feature map and the target first initial feature map;

[0024] After completing multiple second-highlight feature maps as target highlight feature maps, multiple first products are obtained;

[0025] Calculate the second product between the initial feature map of the target and the first highlighted feature map;

[0026] Based on the plurality of first products and second products, the feature enhancement map corresponding to the target initial feature map is obtained;

[0027] Using any initial feature map from the plurality of initial feature maps as the target initial feature map, feature enhancement maps corresponding to each of the plurality of initial feature maps are obtained, resulting in a plurality of feature enhancement maps.

[0028] In some embodiments of this application, obtaining the feature enhancement map corresponding to the initial feature map of the target based on the plurality of first products and the second product includes:

[0029] The sum of the plurality of first products and second products is calculated to obtain the feature enhancement map corresponding to the target initial feature map.

[0030] In some embodiments of this application, the method further includes:

[0031] If contraband is detected, obtain the waybill number corresponding to the contraband, generate and issue a wanted and interception alarm corresponding to the waybill number;

[0032] Packages containing prohibited items are opened for inspection to manually confirm whether they contain prohibited items.

[0033] On the other hand, this application provides a contraband detection device, the contraband detection device comprising:

[0034] The image acquisition module is used to acquire security inspection images to be inspected.

[0035] The feature recognition module is used to extract features from the security inspection image to obtain multiple initial feature maps corresponding to the security inspection image;

[0036] The feature enhancement module is used to obtain a highlight feature map corresponding to each of the multiple initial feature maps, thereby obtaining multiple highlight feature maps, which are used to enhance the image features in the initial feature maps;

[0037] The feature fusion module is used to fuse the multiple initial feature maps and the multiple highlight feature maps to obtain multiple feature enhancement maps, which are used to further enhance the image features in the initial feature maps;

[0038] The detection module is used to detect contraband based on the multiple feature enhancement maps to obtain the contraband detection result of the security inspection image.

[0039] On the other hand, this application also provides an electronic device, the electronic device comprising:

[0040] One or more processors;

[0041] Memory; and

[0042] One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the contraband detection method described in any one of the first aspects.

[0043] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, the computer program being loaded by a processor to perform the steps in the contraband detection method according to any one of the first aspects.

[0044] This application provides a method, apparatus, electronic device, and storage medium for detecting contraband, comprising: acquiring a security inspection image to be detected; extracting features from the security inspection image to obtain multiple initial feature maps corresponding to the security inspection image; acquiring a highlighted feature map corresponding to each of the multiple initial feature maps to obtain multiple highlighted feature maps; fusing the multiple initial feature maps and the multiple highlighted feature maps to obtain multiple feature enhancement maps; and obtaining the contraband detection result of the security inspection image based on the multiple feature enhancement maps. This application, based on the initial acquisition of multiple initial feature maps of the security inspection image, further acquires the highlighted feature maps corresponding to each of the multiple initial feature maps, further highlighting the features in the security inspection image; and further fuses the multiple initial feature maps and the multiple highlighted feature maps to further highlight the features in the security inspection image. This application's repeated highlighting of the identified image features can effectively improve the accuracy of image feature recognition; that is, it improves the accuracy of recognizing contraband features in the image, thereby improving the accuracy of contraband identification. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a schematic diagram of a scenario for the prohibited items detection system provided in an embodiment of this application;

[0047] Figure 2 This is a schematic flowchart of an embodiment of the prohibited items detection method provided in this application.

[0048] Figure 3 A schematic diagram of an embodiment of the feature recognition model provided in this application;

[0049] Figure 4 The acquisition of highlight features provided in the embodiments of this application Figure 1 Flowchart of the embodiment;

[0050] Figure 5Multiple feature enhancements are provided for embodiments of this application. Figure 1 Flowchart of the embodiment;

[0051] Figure 6 A schematic diagram of an embodiment of the fusion module provided in this application;

[0052] Figure 7 This is a schematic diagram of one embodiment of the prohibited items detection device provided in this application;

[0053] Figure 8 This is a schematic diagram of an embodiment of the electronic device provided in this application. Detailed Implementation

[0054] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0055] In the description of this application, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are used only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of the stated features. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0056] In this application, the term "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described as "exemplary" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use this application. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that this application can be made without using these specific details. In other instances, well-known structures and processes are not described in detail to avoid obscuring the description of this application with unnecessary detail. Therefore, this application is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0057] It should be noted that since the method in this application embodiment is executed in an electronic device, the processing objects of each electronic device exist in the form of data or information, such as time, which is essentially time information. It can be understood that if size, quantity, position, etc. are mentioned in subsequent embodiments, they are all corresponding data that exist so that the electronic device can process them. Specific details will not be elaborated here.

[0058] This application provides a method, apparatus, electronic device, and storage medium for detecting contraband, which will be described in detail below.

[0059] Please see Figure 1 , Figure 1 This is a schematic diagram of a prohibited item detection system provided in an embodiment of this application. The prohibited item detection system may include an electronic device 100, which integrates a prohibited item detection device, such as... Figure 1 Electronic devices in the system.

[0060] In this embodiment, the electronic device 100 is mainly used to acquire a security inspection image to be detected; extract features from the security inspection image to obtain multiple initial feature maps corresponding to the security inspection image; acquire a highlight feature map corresponding to each initial feature map in the multiple initial feature maps to obtain multiple highlight feature maps, which are used to enhance the image features in the initial feature maps; fuse the multiple initial feature maps and the multiple highlight feature maps to obtain multiple feature enhancement maps, which are used to further enhance the image features in the initial feature maps; and perform contraband detection based on the multiple feature enhancement maps to obtain the contraband detection result of the security inspection image.

[0061] In this embodiment, the electronic device 100 can be a standalone server, a server network, or a server cluster. For example, the electronic device 100 described in this embodiment includes, but is not limited to, a computer, a network host, a single network server, a set of multiple network servers, or a cloud server composed of multiple servers. The cloud server is composed of a large number of computers or network servers based on cloud computing.

[0062] It is understood that the terminal 100 used in this application embodiment can be a device that includes both receiving and transmitting hardware, that is, a device having receiving and transmitting hardware capable of performing bidirectional communication on a bidirectional communication link. Such a device may include: cellular or other communication devices having a single-line display, a multi-line display, or a cellular or other communication device without a multi-line display. Specifically, the terminal 100 may be a desktop terminal or a mobile terminal, and the terminal 100 may also be one of a mobile phone, tablet computer, laptop computer, etc.

[0063] Those skilled in the art will understand that Figure 1 The application environment shown is merely one application scenario of the solution in this application and does not constitute a limitation on the application scenario of the solution in this application. Other application environments may include those that are more specific to this application. Figure 1 The number of more or fewer electronic devices shown, for example Figure 1 Only one electronic device is shown in the image. It is understood that the contraband detection system may also include one or more other services, which are not specified here.

[0064] In addition, such as Figure 1 As shown, the contraband detection system may also include a storage unit 200 for storing data, such as security inspection images.

[0065] It should be noted that, Figure 1 The schematic diagram of the prohibited items detection system shown is merely an example. The prohibited items detection system and scenario described in this application are for the purpose of more clearly illustrating the technical solutions of this application, and do not constitute a limitation on the technical solutions provided in this application. As those skilled in the art will know, with the evolution of prohibited items detection systems and the emergence of new business scenarios, the technical solutions provided in this application are also applicable to similar technical problems.

[0066] First, this application provides a method for detecting contraband. The execution subject of this method is a contraband detection device applied to electronic devices. The method includes: acquiring a security inspection image to be detected; extracting features from the security inspection image to obtain multiple initial feature maps corresponding to the security inspection image; acquiring a highlight feature map corresponding to each of the multiple initial feature maps to obtain multiple highlight feature maps, which are used to enhance the image features in the initial feature maps; fusing the multiple initial feature maps and the multiple highlight feature maps to obtain multiple feature enhancement maps, which are used to further enhance the image features in the initial feature maps; and performing contraband detection based on the multiple feature enhancement maps to obtain the contraband detection result of the security inspection image.

[0067] like Figure 2 The diagram shown is a flowchart of one embodiment of the prohibited substance detection method in this application. The prohibited substance detection method includes:

[0068] 21. Obtain the security inspection image to be inspected.

[0069] The prohibited item detection method provided in this application is mainly applied to security checks in areas with high traffic, such as subway stations. In these scenarios, X-rays are typically used to obtain security images of the items to be detected. However, using only X-ray images for prohibited item detection yields poor results. Therefore, in this application, after obtaining the security image to be detected, a neural network model is further used to identify prohibited items within the image.

[0070] It should be noted that in the embodiments of this application, the initial security inspection image obtained by using X-rays or the like is the image to be inspected; image preprocessing is also required on the initial security inspection image to obtain a security inspection image suitable for the neural network model.

[0071] Specifically, the initial security inspection image can be scaled proportionally, and then the scaled security inspection image can be placed in the center to fill the surrounding area of ​​the scaled security inspection image; thus obtaining the processed security inspection image, and then performing contraband detection on the security inspection image.

[0072] 22. Extract features from the security inspection images to obtain multiple initial feature maps corresponding to the security inspection images.

[0073] 23. Obtain the highlighted feature map corresponding to each of the multiple initial feature maps to obtain multiple highlighted feature maps.

[0074] 24. Fuse multiple initial feature maps and multiple highlighted feature maps to obtain multiple feature enhancement maps.

[0075] In conventional neural network model recognition processes, neural network models are typically used to extract features from images and then use these features to determine whether certain requirements are met, thus identifying whether an item, such as a contraband, is present. However, existing neural network models do not accurately identify image features, leading to inaccurate identification of contraband. Therefore, in this embodiment, it is necessary to modify the existing neural network model and use the modified model to re-identify security inspection images.

[0076] Specifically, in the embodiments of this application, a feature fusion module can be added to a conventional neural network model to fuse different feature maps identified by the neural network model, thereby enhancing the identified image features.

[0077] It should be noted that, in the embodiments of this application, the first step is still to use a preset feature recognition model to extract features from the security inspection image, obtaining multiple initial feature maps corresponding to the security inspection image; then, the highlight feature maps corresponding to each of the multiple initial feature maps are determined, resulting in multiple highlight feature maps. The highlight feature maps have already enhanced the recognized image features based on the original initial feature maps.

[0078] Simultaneously, the newly added feature fusion module is needed to fuse multiple initial feature maps and multiple highlighted feature maps to further enhance the image features identified by the initial feature maps. The specific fusion process will be described in detail in subsequent embodiments.

[0079] 25. Contraband detection is performed based on multiple feature enhancement maps to obtain the contraband detection results of the security inspection image.

[0080] The prohibited item detection method provided in this application includes: acquiring a security inspection image to be detected; extracting features from the security inspection image to obtain multiple initial feature maps corresponding to the security inspection image; acquiring a highlighted feature map corresponding to each of the multiple initial feature maps to obtain multiple highlighted feature maps; fusing the multiple initial feature maps and the multiple highlighted feature maps to obtain multiple feature enhancement maps; and obtaining the prohibited item detection result of the security inspection image based on the multiple feature enhancement maps. This application, based on the initial acquisition of multiple initial feature maps of the security inspection image, further acquires the highlighted feature maps corresponding to each of the multiple initial feature maps, further highlighting the features in the security inspection image; and further fuses the multiple initial feature maps and the multiple highlighted feature maps to further highlight the features in the security inspection image. This application's repeated highlighting of the identified image features can effectively improve the accuracy of image feature recognition; that is, improve the accuracy of recognizing prohibited item features in the image, thereby improving the accuracy of identifying prohibited items.

[0081] like Figure 3 The diagram shown is a schematic representation of an embodiment of the feature recognition model provided in this application; and in Figure 3 Only a portion of the structure in the feature recognition model is shown. Figure 3 In the illustrated embodiment, the preset feature recognition model includes multiple convolutional layers conv1, conv2, and conv3 of different sizes. Simply put, the convolutional layers can identify image features in the input security inspection image and obtain multiple initial feature maps. Since the sizes of the multiple convolutional layers are different, the resulting initial feature maps are also different. Each convolutional layer can extract one initial feature map, and the size of the initial feature map usually corresponds to the size of the convolutional layer.

[0082] For example, if the size of the convolutional layer is 80*80*128, then the size of the corresponding initial feature map is also 80*80*128. Figure 3 In the illustrated embodiment, the sizes of the three convolutional layers can be 80*80*128, 40*40*256, and 20*20*512, respectively; corresponding to the sizes of the three initial feature maps, which are also 80*80*128, 40*40*256, and 20*20*512, respectively. Of course, in other embodiments, the feature recognition model may include more or fewer convolutional layers of different sizes, and the specific size of the convolutional layers can be set according to actual needs.

[0083] It should be noted that, since this application requires the fusion of multiple initial feature maps, the number of Dorgon's initial feature maps must be greater than or equal to 2, and the number of corresponding convolutional layers must also be greater than or equal to 2; and the sizes of the multiple convolutional layers are different.

[0084] exist Figure 3 In this paper, a feature fusion module is added between the original convolutional layer and the output layer (prediction) to fuse multiple initial feature maps separately, resulting in fused feature maps of different sizes. However, in the embodiments of this application, the fusion actually includes not only multiple initial feature maps, but also the feature enhancement maps corresponding to each of the multiple initial feature maps.

[0085] It should be noted that in the above fusion process, since there are three convolutional layers, there are also three corresponding feature fusion modules and output layers; and since the sizes of the three convolutional layers are different, in actual fusion, the initial feature maps obtained from the same convolutional layer need to be fused three times. Specifically, for convolutional layer 1 (conv1), the initial feature maps obtained from convolutional layer 1, convolutional layer 2 (conv2), and convolutional layer 3 (conv3) need to be fused. Similarly, for convolutional layers 2 and 3, the initial feature maps obtained from convolutional layer 1, convolutional layer 2 (conv2), and convolutional layer 3 (conv3) also need to be fused. Since the sizes of the different convolutional layers are different, the feature enhancement maps obtained by fusion are actually different. It should also be noted that in the actual fusion process, the highlight feature maps corresponding to each initial feature map also need to be fused. The above fusion process is only a brief explanation; for each fusion module, the initial feature maps extracted from all convolutional layers need to be fused simultaneously.

[0086] like Figure 4 As shown, this application provides a method for obtaining highlight features. Figure 1 The flowchart of the embodiment may include:

[0087] 41. Using any initial feature map from multiple initial feature maps as the target initial feature map, determine the first highlighted feature map corresponding to the target initial feature map.

[0088] Using any initial feature map from multiple initial feature maps as the target initial feature map, the following details how to determine multiple highlighted feature maps. Specifically, the initial feature map corresponding to convolutional layer 1 from multiple initial feature maps can be used. Figure 1 The initial feature map is the target; it can be sigmoidized to obtain the first highlighted feature map corresponding to the initial feature map. The specific process of sigmoidization can be found in existing technologies and is not limited here.

[0089] 42. Determine the target convolutional layer corresponding to the initial feature map of the target among multiple convolutional layers, so as to determine the size of the target convolutional layer.

[0090] 43. Based on the size of the target convolutional layer, determine the second highlighted feature map corresponding to each of the other initial feature maps in the multiple initial feature maps, excluding the target initial feature map.

[0091] Since the sizes of the multiple convolutional layers are all different, this application also needs to determine the target convolutional layer corresponding to the initial feature map of the target among the multiple convolutional layers; for example, the initial feature map... Figure 1The corresponding convolutional layer 1 (conv1) is defined; and the size of convolutional layer 1 is determined; so that other initial feature maps are converted to a size corresponding to the size of convolutional layer 1, and then the corresponding highlight feature map is determined.

[0092] Specifically, based on the size of the target convolutional layer, other initial feature maps besides the target initial feature map can be sampled from multiple initial feature maps to transform them into the same size as the target convolutional layer, resulting in multiple transformed first initial feature maps. Then, a second highlighted feature map corresponding to each of the multiple first initial feature maps is determined, resulting in multiple second highlighted feature maps.

[0093] Specifically, sampling other initial feature maps besides the target initial feature map can be performed based on the size of the target convolutional layer. This can be done by upsampling or downsampling other feature maps based on the size of the target convolutional layer. The specific upsampling or downsampling is determined by the size of the target convolutional layer and the size of the convolutional layers corresponding to the other initial feature maps.

[0094] For example: taking convolutional layer 1 as the target convolutional layer, and using the initial features corresponding to convolutional layer 2 based on the size of the target convolutional layer (80*80*128). Figure 2 Sampling may include: Since the target convolutional layer has a size of 80*80*128, and the size of convolutional layer 2 is 40*40*256, then the initial features corresponding to convolutional layer 2 need to be sampled. Figure 2 Downsampling is performed. Similarly, if the target convolutional layer has a size of 40*40*256, then upsampling is performed by sampling the initial feature map of convolutional layer 1 with a size of 80*80*128 according to the size of the target convolutional layer.

[0095] In the above embodiments, after adjusting the size to obtain multiple first initial feature maps, it is also necessary to obtain the highlight feature map corresponding to each first initial feature map. At this time, the multiple first initial feature maps can also be sigmoidized to obtain multiple second highlight feature maps corresponding to the multiple first initial feature maps. The highlight feature maps include the first highlight feature map corresponding to the target initial feature map, and the multiple second highlight feature maps corresponding to the multiple first initial feature maps.

[0096] After identifying multiple initial feature maps and their corresponding highlight feature maps, the initial feature maps and highlight feature maps can be fused. For example... Figure 5 As shown, this application provides an embodiment of multiple feature enhancements. Figure 1 The flowchart of the embodiment may include:

[0097] 51. Using any second highlight feature map among multiple second highlight feature maps as the target highlight feature map, determine the target first initial feature map corresponding to the target highlight feature map.

[0098] 52. Calculate the first product between the target highlighted feature map and the target's first initial feature map;

[0099] 53. After completing multiple second highlight feature maps as target highlight feature maps, multiple first products are obtained.

[0100] 54. Calculate the second product between the initial feature map of the target and the first highlighted feature map.

[0101] 55. Based on multiple first products and second products, obtain the feature enhancement map corresponding to the initial feature map of the target.

[0102] 56. Using any initial feature map from multiple initial feature maps as the target initial feature map, we obtain feature enhancement maps corresponding to each of the multiple initial feature maps, thus obtaining multiple feature enhancement maps.

[0103] like Figure 6 The diagram shown is a schematic representation of an embodiment of the fusion module provided in this application; the following is in conjunction with... Figure 5 and Figure 6 The fusion process provided in this application embodiment is described in detail. Taking the initial feature map corresponding to convolutional layer 1 (conv1) as the target initial feature map as an example, the first highlighted feature map corresponding to the target initial feature map can be determined at this time. Meanwhile, convolutional layer 2 (conv2) and convolutional layer 3 (conv3) each correspond to initial features... Figure 2 and initial features Figure 2 ; and initial features Figure 2 and initial features Figure 3 This refers to the other initial feature maps mentioned above, excluding the target initial feature map.

[0104] At this point, it is necessary to adjust the initial features according to the size of convolutional layer 1. Figure 2 and initial features Figure 3 Perform upsampling or downsampling respectively (i.e.) Figure 6 Upsample (in the context of the initial features) to extract the initial features Figure 2 or initial features Figure 3 The size is transformed to be the same as the target initial feature map; thus, two first initial feature maps with changed sizes are obtained, and the second highlighted feature maps corresponding to the two first initial feature maps need to be determined respectively to obtain two second highlighted feature maps.

[0105] exist Figure 6 In the illustrated embodiment, initial features can be used. Figure 2 The corresponding first initial feature map is the target's first initial feature map, and the initial features Figure 2 The corresponding second highlighted feature map is the target highlighted feature map. At this point, it is necessary to calculate the first product between the target highlighted feature map and the target's first initial feature map, i.e. Figure 6 The target highlighted feature map sigmoid (80*80*125) and the target first initial feature map Upsample (80*80*125) are multiplied (mul) by the following method.

[0106] Similarly, it is also necessary to be able to identify the initial features. Figure 3 The corresponding first initial feature map is used as the target's first initial feature map, and the initial features are... Figure 3 The corresponding second highlighted feature map is the target highlighted feature map. For example... Figure 6 As shown, the initial features are also calculated. Figure 3 The first product between the corresponding first initial feature map and the second highlighted feature map.

[0107] In the above embodiment, in addition to the target initial feature map, i.e. the initial feature map corresponding to convolutional layer 1, it is necessary to determine the first initial feature map after changing the size of other initial feature maps, and then determine the second highlighted feature map corresponding to the first initial feature map; and calculate the first product between the first initial feature map and the second highlighted feature map to obtain multiple first products.

[0108] Simultaneously, it is also necessary to calculate the second product between the target initial feature map (i.e., the initial feature map corresponding to the first convolutional layer) and the first highlighted feature map. The first product has a value of 1, while the number of the second product depends on the number of other initial feature maps besides the target initial feature map among the multiple initial feature maps.

[0109] After obtaining the second product and multiple first products, the feature enhancement map corresponding to the initial target feature map can be determined. Specifically, the sum of the second product and multiple first products can be calculated, i.e. Figure 6 The addition operation is performed to obtain the feature enhancement map corresponding to the initial feature map of the target.

[0110] It should be noted that in the above embodiment, the initial features corresponding to convolutional layer 1 are used. Figure 1 The initial feature map of the target is used, and what is calculated is only the initial feature map. Figure 1 The corresponding feature enhancement map. However, in this application, it is necessary to determine the feature enhancement map corresponding to each initial feature map separately. Therefore, it is necessary to take each initial feature map in the multiple initial feature maps as the target initial feature map, and use the above method to obtain the feature enhancement map corresponding to each of the multiple initial feature maps, thus obtaining multiple feature enhancement maps.

[0111] In the embodiments of this application, not only are multiple initial feature maps identified by multiple convolutional layers of different sizes obtained, but also highlight feature maps corresponding to multiple initial feature maps are further obtained. However, it should be noted that some highlight feature maps in this application are determined after the initial feature maps have undergone size transformation, rather than being determined directly based on the initial feature maps.

[0112] In the embodiments of this application, the existing feature recognition model is mainly modified, primarily by adding a feature fusion module after the convolutional layer to enhance the recognized features, thereby improving the accuracy of contraband identification. In the above embodiments, feature fusion can be performed using the following simple formula:

[0113] conv1*α1+conv2*β1+conv3*γ1=Feature Fusion 1

[0114] Here, conv1, conv2, and conv3 can be understood as the initial features corresponding to three convolutional layers of different sizes. Figure 1-3 α1, β1, and γ1 are the fusion parameters corresponding to each initial feature map. These fusion parameters can actually be determined when adjusting the size of the initial feature maps according to the size of the target convolutional layer. The above formula only describes the fused feature enhancement map corresponding to convolutional layer 1, requiring the simultaneous fusion of all initial feature maps; it does not mean that only initial feature maps are needed during fusion, and other feature maps are not required. In actual fusion, not only are multiple initial feature maps fused, but the corresponding highlight feature maps also need to be determined; and it is not a simple addition or multiplication.

[0115] It should be noted that, in the embodiments of this application, when using the modified feature recognition model with the added fusion module for contraband identification, it is still necessary to train the model using sample data to obtain the optimal feature recognition model including the fusion module. Simultaneously, if contraband is detected, it is necessary to obtain the corresponding waybill number, generate and issue a wanted warning for interception corresponding to the waybill number; furthermore, it is necessary to intercept and open packages containing contraband for manual reconfirmation to avoid false positives.

[0116] This application also provides a contraband detection device, such as... Figure 7 The diagram shown is a schematic representation of an embodiment of the prohibited substance detection device provided in this application, which may include:

[0117] Image acquisition module 701 is used to acquire security inspection images to be inspected.

[0118] The feature recognition module 702 is used to extract features from the security inspection image to obtain multiple initial feature maps corresponding to the security inspection image.

[0119] The feature enhancement module 703 is used to obtain the highlight feature map corresponding to each of the multiple initial feature maps, thereby obtaining multiple highlight feature maps. The highlight feature maps are used to enhance the image features in the initial feature maps.

[0120] The feature fusion module 704 is used to fuse multiple initial feature maps and multiple highlight feature maps to obtain multiple feature enhancement maps. The feature enhancement maps are used to further enhance the image features in the initial feature maps.

[0121] The detection module 705 is used to detect contraband based on multiple feature enhancement maps to obtain the contraband detection results of the security inspection image.

[0122] The prohibited item detection device provided in this application first acquires a security inspection image to be detected; performs feature extraction on the security inspection image to obtain multiple initial feature maps corresponding to the security inspection image; acquires a highlighted feature map corresponding to each of the multiple initial feature maps to obtain multiple highlighted feature maps; fuses the multiple initial feature maps and the multiple highlighted feature maps to obtain multiple feature enhancement maps; and obtains the prohibited item detection result of the security inspection image based on the multiple feature enhancement maps. Based on the initial acquisition of multiple initial feature maps of the security inspection image, this application further acquires the highlighted feature maps corresponding to each of the multiple initial feature maps, further highlighting the features in the security inspection image; and further fuses the multiple initial feature maps and the multiple highlighted feature maps to further highlight the features in the security inspection image. This application's repeated highlighting of the identified image features can effectively improve the accuracy of image feature recognition; that is, it improves the accuracy of recognizing prohibited item features in the image, thereby improving the accuracy of identifying prohibited items.

[0123] In some embodiments, the feature recognition module 702 can be used to: extract features from the security inspection image using a preset feature recognition model to obtain multiple initial feature maps corresponding to the security inspection image.

[0124] In some embodiments, the preset feature recognition model includes multiple convolutional layers of different sizes, with each convolutional layer extracting an initial feature map. The feature enhancement module 703 can specifically be used for:

[0125] Using any initial feature map from multiple initial feature maps as the target initial feature map, determine the first highlighted feature map corresponding to the target initial feature map;

[0126] In multiple convolutional layers, the target convolutional layer corresponding to the initial feature map of the target is determined to determine the size of the target convolutional layer;

[0127] Based on the size of the target convolutional layer, determine the second highlighted feature map corresponding to each of the other initial feature maps in the multiple initial feature maps, excluding the target initial feature map;

[0128] Multiple highlight feature maps include a first highlight feature map and a second highlight feature map.

[0129] In some embodiments, the feature recognition module 702 can be specifically used for:

[0130] Based on the size of the target convolutional layer, other initial feature maps are sampled to transform them into the same size as the target convolutional layer, resulting in multiple first initial feature maps.

[0131] Each of the multiple initial feature maps is used to determine its corresponding second highlighted feature map, resulting in multiple second highlighted feature maps.

[0132] In some embodiments, the feature fusion module 704 may specifically be used for:

[0133] Using any second highlight feature map among multiple second highlight feature maps as the target highlight feature map, determine the target first initial feature map corresponding to the target highlight feature map;

[0134] Calculate the first product between the target highlighted feature map and the target first initial feature map;

[0135] After completing multiple second-highlight feature maps as target highlight feature maps, multiple first products are obtained;

[0136] Calculate the second product between the initial feature map of the target and the first highlighted feature map;

[0137] Based on multiple first and second products, the feature enhancement map corresponding to the initial feature map of the target is obtained;

[0138] Using any initial feature map from multiple initial feature maps as the target initial feature map, we obtain feature enhancement maps corresponding to each of the multiple initial feature maps, thus obtaining multiple feature enhancement maps.

[0139] In some embodiments, the feature fusion module 704 may specifically be used for:

[0140] Calculate the sum of multiple first and second products to obtain the feature enhancement map corresponding to the target initial feature map.

[0141] This application also provides an electronic device that integrates any of the prohibited items detection devices provided in this application. For example... Figure 8 As shown, it illustrates a structural schematic diagram of the electronic device involved in the embodiments of this application, specifically:

[0142] The electronic device may include components such as a processor 801 with one or more processing cores, a memory 802 with one or more computer-readable storage media, a power supply 803, and an input unit 804. Those skilled in the art will understand that... Figure 8 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0143] The processor 801 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 802, and by calling data stored in the memory 802, it performs various functions and processes data, thereby providing overall monitoring of the electronic device. Optionally, the processor 801 may include one or more processing cores; preferably, the processor 801 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 801.

[0144] The memory 802 can be used to store software programs and modules. The processor 801 executes various functional applications and data processing by running the software programs and modules stored in the memory 802. The memory 802 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 802 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 802 may also include a memory controller to provide the processor 801 with access to the memory 802.

[0145] The electronic device also includes a power supply 803 that supplies power to the various components. Preferably, the power supply 803 can be logically connected to the processor 801 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 803 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0146] The electronic device may also include an input unit 804, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0147] Although not shown, the electronic device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 801 in the electronic device loads the executable files corresponding to the processes of one or more application programs into the memory 802 according to the following instructions, and the processor 801 runs the application programs stored in the memory 802 to realize various functions, as follows:

[0148] The process involves: acquiring the security inspection image to be inspected; extracting features from the security inspection image to obtain multiple initial feature maps; acquiring the highlighted feature map corresponding to each initial feature map to obtain multiple highlighted feature maps, which are used to enhance the image features in the initial feature maps; fusing the multiple initial feature maps and the multiple highlighted feature maps to obtain multiple enhanced feature maps, which are used to further enhance the image features in the initial feature maps; and performing contraband detection based on the multiple enhanced feature maps to obtain the contraband detection result of the security inspection image.

[0149] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0150] Therefore, embodiments of this application provide a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk, etc. A computer program is stored thereon, which is loaded by a processor to execute the steps in any of the contraband detection methods provided in embodiments of this application. For example, the computer program loaded by the processor can execute the following steps:

[0151] The process involves: acquiring the security inspection image to be inspected; extracting features from the security inspection image to obtain multiple initial feature maps; acquiring the highlighted feature map corresponding to each initial feature map to obtain multiple highlighted feature maps, which are used to enhance the image features in the initial feature maps; fusing the multiple initial feature maps and the multiple highlighted feature maps to obtain multiple enhanced feature maps, which are used to further enhance the image features in the initial feature maps; and performing contraband detection based on the multiple enhanced feature maps to obtain the contraband detection result of the security inspection image.

[0152] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the detailed descriptions of other embodiments above, which will not be repeated here.

[0153] In practice, each of the above units or structures can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units or structures, please refer to the previous method embodiments, which will not be repeated here.

[0154] The foregoing has provided a detailed description of a method, apparatus, electronic device, and storage medium for detecting contraband provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for detecting contraband, characterized in that, The methods for detecting prohibited items include: Acquire the security inspection image to be inspected; Feature extraction is performed on the security inspection image to obtain multiple initial feature maps corresponding to the security inspection image; Obtain the highlight feature map corresponding to each of the plurality of initial feature maps to obtain a plurality of highlight feature maps, which are used to enhance the image features in the initial feature maps; The plurality of initial feature maps and the plurality of highlight feature maps are fused to obtain a plurality of feature enhancement maps, which are used to further enhance the image features in the initial feature maps; Contraband detection is performed based on the multiple feature enhancement maps to obtain the contraband detection result of the security inspection image; The step of extracting features from the security inspection image to obtain multiple initial feature maps corresponding to the security inspection image includes: The security inspection image is used to extract features using a preset feature recognition model to obtain multiple initial feature maps corresponding to the security inspection image; the preset feature recognition model includes multiple convolutional layers of different sizes, and each convolutional layer extracts one initial feature map. The process of fusing the plurality of initial feature maps and the plurality of highlight feature maps includes fusing the initial feature maps extracted by multiple convolutional layers of different sizes, as well as the highlight feature maps corresponding to the plurality of initial feature maps. The plurality of highlight feature maps include a first highlight feature map and a second highlight feature map; the fusion of the plurality of initial feature maps and the plurality of highlight feature maps to obtain a plurality of feature enhancement maps includes: Using any second highlight feature map among multiple second highlight feature maps as the target highlight feature map, determine the target first initial feature map corresponding to the target highlight feature map; Calculate the first product between the target highlighted feature map and the target first initial feature map; After completing multiple second-highlight feature maps as target highlight feature maps, multiple first products are obtained; Calculate the second product between the target initial feature map and the first highlighted feature map corresponding to the target initial feature map; the target initial feature map is any initial feature map among the plurality of initial feature maps; Based on the plurality of first products and second products, the feature enhancement map corresponding to the target initial feature map is obtained; Using any initial feature map from the plurality of initial feature maps as the target initial feature map, feature enhancement maps corresponding to each of the plurality of initial feature maps are obtained, resulting in a plurality of feature enhancement maps.

2. The method for detecting prohibited items according to claim 1, characterized in that, The step of obtaining the highlighted feature map corresponding to each of the plurality of initial feature maps to obtain a plurality of highlighted feature maps includes: Using any initial feature map among the plurality of initial feature maps as the target initial feature map, determine the first highlighted feature map corresponding to the target initial feature map; Among the multiple convolutional layers of different sizes, the target convolutional layer corresponding to the target initial feature map is determined, so as to determine the size of the target convolutional layer; Based on the size of the target convolutional layer, determine the second highlighted feature map corresponding to each of the other initial feature maps in the plurality of initial feature maps, excluding the target initial feature map.

3. The method for detecting prohibited items according to claim 2, characterized in that, The step of determining the second highlighted feature map corresponding to each of the other initial feature maps (excluding the target initial feature map) in the plurality of initial feature maps according to the size of the target convolutional layer includes: Based on the size of the target convolutional layer, the other initial feature maps are sampled to transform them into the same size as the target convolutional layer, resulting in multiple first initial feature maps. Each of the plurality of first initial feature maps is then determined to have its corresponding second highlighted feature map, resulting in a plurality of second highlighted feature maps.

4. The method for detecting contraband according to claim 1, characterized in that, The step of obtaining the feature enhancement map corresponding to the initial feature map of the target based on the plurality of first products and second products includes: The sum of the plurality of first products and second products is calculated to obtain the feature enhancement map corresponding to the target initial feature map.

5. The method for detecting prohibited items according to claim 1, characterized in that, The method further includes: If contraband is detected, obtain the waybill number corresponding to the contraband, generate and issue a wanted and interception alarm corresponding to the waybill number; Packages containing prohibited items are opened for inspection to manually confirm whether they contain prohibited items.

6. A contraband detection device, characterized in that, The prohibited item detection device includes: The image acquisition module is used to acquire security inspection images to be inspected. The feature recognition module is used to extract features from the security inspection image to obtain multiple initial feature maps corresponding to the security inspection image; The feature enhancement module is used to obtain a highlight feature map corresponding to each of the plurality of initial feature maps, thereby obtaining a plurality of highlight feature maps. The highlight feature maps are used to enhance the image features in the initial feature maps. The plurality of highlight feature maps include a first highlight feature map and a second highlight feature map. The feature fusion module is used to fuse the multiple initial feature maps and the multiple highlight feature maps to obtain multiple feature enhancement maps, which are used to further enhance the image features in the initial feature maps; The detection module is used to detect contraband based on the multiple feature enhancement maps to obtain the contraband detection result of the security inspection image; The feature recognition module can be used to: extract features from security images using a preset feature recognition model to obtain multiple initial feature maps corresponding to the security images; the preset feature recognition model includes multiple convolutional layers of different sizes, and one convolutional layer extracts one initial feature map. The process of fusing the plurality of initial feature maps and the plurality of highlight feature maps includes fusing the initial feature maps extracted by multiple convolutional layers of different sizes, as well as the highlight feature maps corresponding to the plurality of initial feature maps. The feature fusion module is specifically used to determine the target first initial feature map corresponding to the target high-brightness feature map by taking any second high-brightness feature map from multiple second high-brightness feature maps as the target high-brightness feature map; calculate the first product between the target high-brightness feature map and the target first initial feature map; after taking multiple second high-brightness feature maps as the target high-brightness feature map, multiple first products are obtained; calculate the second product between the target initial feature map and the first high-brightness feature map corresponding to the target initial feature map; the target initial feature map is any initial feature map among the multiple initial feature maps; based on the multiple first products and second products, the feature enhancement map corresponding to the target initial feature map is obtained; taking any initial feature map from the multiple initial feature maps as the target initial feature map, the feature enhancement maps corresponding to each of the multiple initial feature maps are obtained, resulting in multiple feature enhancement maps.

7. An electronic device, characterized in that, The electronic device includes: One or more processors; Memory; and One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the contraband detection method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, It contains a computer program that is loaded by a processor to perform the steps of the contraband detection method according to any one of claims 1 to 5.

Images