Passenger station security check package photo and X-ray image association method and device

Abstract

The invention provides a passenger station security check package photo and X-ray image association method and device, and the method comprises the steps: obtaining a target package object image corresponding to each passenger station security check package photo, and constructing each training sample based on a package X-ray image corresponding to each target package object image and a label; and training a twin network with a 16-time down-sampling rate based on a mixed attention mechanism by using the training sample, so as to train the twin network into a correlation model used for outputting correlation recognition result data between the input target parcel object image and parcel X-ray image according to the input target parcel object image and parcel X-ray image. According to the invention, effective training of a passenger station security check package photo and X-ray image association model can be realized, and the accuracy and effectiveness of associating package photos and package X-ray images from different sources by using the model can be effectively improved; therefore, the accuracy and the reliability of application such as railway security check automatic sorting by adopting the associated recognition result can be effectively improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method and device for associating a security inspection package photo with an X-ray image at a passenger station. Background Art

[0002] In scenarios where package security inspection is required, such as civil aviation, stations, and logistics transit, the security inspection automatic sorting system has been partially piloted. The security inspection automatic sorting system can automatically sort packages to be opened for inspection to another area to facilitate staff inspection. Among them, the execution basis of package security inspection requires associating the package with the package X-ray image, and then quickly locating the location of each item in the package and inspecting it based on the X-ray image associated with the package.

[0003] Currently, in the existing method of associating parcels with parcel X-ray images, all security inspection parcels need to be placed in different storage baskets, and the electronic tags of the storage baskets are used to achieve the correspondence between the parcels and the X-ray images, thereby achieving automatic sorting of the parcels.

[0004] However, for large user groups such as railway passenger security inspection and non-user scenarios such as logistics transfer centers, placing each package in a storage basket for security inspection requires a lot of manpower and time costs. Therefore, it is necessary to automatically associate the photos of the passenger station security inspection packages with the X-ray images. Although models that can perform image recognition such as convolutional neural networks have been widely used, due to the huge style differences between the X-ray images of packages and the visible light photos of packages and the different acquisition equipment, the direct use of existing convolutional neural networks cannot guarantee the association effect between the two.

[0005] Based on this, there is an urgent need to design a method that can automatically associate the security inspection package photos and X-ray images at the passenger station and improve the accuracy of the association results. Summary of the invention

[0006] In view of this, the embodiments of the present application provide a method and device for associating photos of package inspections at passenger stations with X-ray images to eliminate or improve one or more defects existing in the prior art.

[0007] One aspect of the present application provides a method for associating a passenger station security inspection package photo with an X-ray image, comprising: Obtaining the target package physical image corresponding to each of the security inspection package photos at each passenger station, and constructing each training sample based on the package X-ray image and label corresponding to each of the target package physical images, wherein the label is used to indicate whether the target package physical image is associated with the corresponding package X-ray image; Each of the training samples is used to train a twin network with a 16-fold downsampling rate based on a hybrid attention mechanism, so as to train the twin network into a package photo and X-ray image association model, so that the package photo and X-ray image association model is used to output the association recognition result data between the target package physical image and the package X-ray image according to the input target package physical image and the package X-ray image.

[0008] In some embodiments of the present application, the step of obtaining the physical image of the target package corresponding to each of the security inspection package photos at each passenger station includes: Based on the edge position data of the conveyor belt, the photos of the security inspection packages at the passenger station collected by the photo collection device above the conveyor belt are cropped to obtain the conveyor belt area photos corresponding to each of the security inspection package photos at the passenger station for showing that the packages are located on the conveyor belt; Performing angle correction processing on each of the conveyor belt area photos respectively to obtain a target image corresponding to each of the conveyor belt area photos; Performing background segmentation processing on each of the target images respectively to obtain a package physical image corresponding to each of the target images; Normalization processing is performed on each of the package physical images and the source package X-ray images corresponding to each of the package physical images to obtain the target package physical images corresponding to each of the package physical images and the package X-ray images corresponding to each of the source package X-ray images.

[0009] In some embodiments of the present application, the angle correction processing is performed on each of the conveyor belt area photos to obtain the target image corresponding to each of the conveyor belt area photos, including: The top surface of the conveyor belt is set as the source plane, and the plane perpendicular to the axis of the camera of the photo acquisition device is set as the target plane; Performing perspective transformation processing on each of the conveyor belt area photos located on the source plane collected by the photo collection device, so as to transform each of the conveyor belt area photos into each target image located on the target plane; Based on a preset angle transformation rule, the angle of each target image is adjusted.

[0010] In some embodiments of the present application, the performing background segmentation processing on each of the target images to obtain a package physical image corresponding to each of the target images includes: Inputting each of the target images into a pre-trained instance segmentation model, so that the instance segmentation model performs background segmentation on each of the target images and outputs a package physical image corresponding to each of the target images as a foreground; Wherein, the instance segmentation model includes: YOLOv8n-Seg model.

[0011] In some embodiments of the present application, the normalizing process of each of the package physical images and the source package X-ray images corresponding to each of the package physical images to obtain the target package physical images corresponding to each of the package physical images and the package X-ray images corresponding to each of the source package X-ray images includes: Edge filling processing is performed on each of the physical package images and the source package X-ray images corresponding to each of the physical package images to complete each of the physical package images to a preset size and form a corresponding target package physical image, and each of the source package X-ray images is completed to a preset size and form a corresponding package X-ray image.

[0012] In some embodiments of the present application, constructing each training sample based on the X-ray image of the package and the label corresponding to each of the physical images of the target package includes: Each of the target package physical images and the package X-ray images corresponding to the target package physical images are respectively taken as an original positive sample pair; positive sample expansion is performed on each of the original positive sample pairs to obtain a plurality of expanded positive sample pairs corresponding to the original positive sample pair, and labels for indicating the association between the target package physical images and the corresponding package X-ray images are respectively set for the original positive sample pairs and each of the expanded positive sample pairs to obtain a plurality of training samples respectively serving as positive samples; wherein the positive sample expansion includes: rotating the original positive sample pairs at different angles, changing the sizes at different magnifications, and randomly adjusting the colors; And, for each of the target package physical image, an unrelated package X-ray image is randomly set to obtain an original negative sample pair; for each of the original negative samples, negative sample expansion is performed to obtain multiple expanded negative sample pairs corresponding to the original negative sample pair, and labels used to indicate that the target package physical image and the corresponding package X-ray image are unrelated are respectively set for the original negative sample pair and each of the expanded negative sample pairs to obtain multiple training samples respectively serving as negative samples; wherein the negative sample expansion includes: rotating the original negative sample pair at multiple different angles, changing the size at different magnifications, and randomly adjusting the color.

[0013] In some embodiments of the present application, the 16-fold downsampling rate twin network based on the hybrid attention mechanism includes: a first residual network with a 16-fold downsampling rate, a second residual network with a 16-fold downsampling rate, and a prediction layer connected between the output ends corresponding to the first residual network and the second residual network, respectively; The input of the first residual network is the target package physical image, and the output is the physical image feature corresponding to the target package physical image; The input of the second residual network is the package X-ray image, and the output is the X-ray image features corresponding to the package X-ray image; The input of the prediction layer is the physical image features and the X-ray image features, and the output is the association recognition result data between the physical image of the target package and the X-ray image features; The first residual network and the second residual network both include: sequentially connected convolution kernels, maximum pooling layers, first residual blocks, second residual blocks, third residual blocks, fourth residual blocks, fifth residual blocks, residual blocks based on a hybrid attention mechanism, adaptive pooling modules, and fully connected layers; The downsampling rates of the convolution kernel, the maximum pooling layer, the third residual block and the fifth residual block are all 2; the downsampling rates of the first residual block, the second residual block, the fourth residual block and the residual block based on the hybrid attention mechanism are all 1.

[0014] In some embodiments of the present application, the method for associating the passenger station security inspection package photo with the X-ray image further includes: Acquire the photos of the passenger station security inspection packages currently collected by the photo collection equipment arranged at the exit of the security inspection instrument and above the conveyor belt, and the X-ray images of the source packages collected by the security inspection instrument; The passenger station security inspection package photo is cropped, angle corrected, and background segmented to obtain a package physical image corresponding to the passenger station security inspection package photo; Normalizing the package physical image and the source package X-ray image to obtain a target package physical image corresponding to the package physical image and a package X-ray image corresponding to the source package X-ray image; The target package physical image and the package X-ray image are input into the package photo and X-ray image association model, so that the package photo and X-ray image association model outputs association recognition result data between the target package physical image and the package X-ray image.

[0015] The second aspect of the present application provides a device for associating a security inspection package photo with an X-ray image at a passenger station, comprising: A sample construction module is used to obtain the target package physical image corresponding to each passenger station security inspection package photo, and construct each training sample based on the package X-ray image and label corresponding to each target package physical image, wherein the label is used to indicate whether the target package physical image is associated with the corresponding package X-ray image; A model training module is used to use each of the training samples to train a twin network with a 16-fold downsampling rate based on a hybrid attention mechanism, so as to train the twin network into a package photo and X-ray image association model, so that the package photo and X-ray image association model is used to output the association recognition result data between the target package physical image and the package X-ray image according to the input target package physical image and the package X-ray image.

[0016] In some embodiments of the present application, the passenger station security inspection package photo and X-ray image association device further includes: An image acquisition module, used to acquire the photos of the passenger station security inspection packages currently acquired by the photo acquisition equipment arranged at the exit of the security inspection instrument and above the conveyor belt, and the X-ray images of the source packages acquired by the security inspection instrument; An image preprocessing module, used for performing cropping, angle correction and background segmentation processing on the passenger station security inspection package photo to obtain a package physical image corresponding to the passenger station security inspection package photo; an image normalization module, used to perform normalization processing on the package physical image and the source package X-ray image to obtain a target package physical image corresponding to the package physical image and a package X-ray image corresponding to the source package X-ray image; The model association module is used to input the target package physical image and the package X-ray image into the package photo and X-ray image association model, so that the package photo and X-ray image association model outputs the association recognition result data between the target package physical image and the package X-ray image.

[0017] The third aspect of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method for associating passenger station security inspection package photos with X-ray images when executing the computer program.

[0018] The fourth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for associating the passenger station security inspection package photo with the X-ray image.

[0019] The fifth aspect of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the method for associating passenger station security inspection package photos with X-ray images.

[0020] The present application provides a method for associating passenger station security inspection package photos and X-ray images, by obtaining the target package physical image corresponding to each passenger station security inspection package photo, and constructing each training sample based on the package X-ray image and label corresponding to each of the target package physical images, wherein the label is used to indicate whether the target package physical image is associated with the corresponding package X-ray image; each of the training samples is used to train a twin network with a 16-fold downsampling rate based on a hybrid attention mechanism, so as to train the twin network to be used for inputting the target package physical image and the package X-ray image, corresponding to the input A package photo and X-ray image association model that can output the association recognition result data between the target package physical image and the package X-ray image can realize the effective training of the package photo and X-ray image association model, and can effectively improve the accuracy and effectiveness of associating passenger station security inspection package photos and package X-ray images from different sources using the model, and can effectively improve the automation and intelligence of associating passenger station security inspection package photos and package X-ray images from different sources, thereby effectively improving the accuracy and reliability of applications such as railway security inspection automatic sorting using the association recognition results, and effectively saving labor costs and time costs.

[0021] Additional advantages, purposes, and features of the present application will be partially described in the following description, and will become partially apparent to those skilled in the art after studying the following, or may be learned from the practice of the present application. The purposes and other advantages of the present application can be achieved and obtained by the structures specifically pointed out in the specification and the drawings.

[0022] Those skilled in the art will appreciate that the objectives and advantages that can be achieved with the present invention are not limited to the above specific description, and the above and other objectives that can be achieved by the present invention will be more clearly understood from the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention. The components in the drawings are not drawn to scale, but are only for illustrating the principles of the present invention. In order to facilitate the illustration and description of some parts of the present invention, the corresponding parts in the drawings may be enlarged, that is, they may become larger relative to other components in the exemplary device actually manufactured according to the present invention. In the drawings: Figure 1 This is a schematic diagram of a first flow chart of a method for associating a passenger station security inspection package photo with an X-ray image in one embodiment of the present application.

[0024] Figure 2 This is a second flow chart of the method for associating baggage photos and X-ray images at a passenger station security inspection in one embodiment of the present application.

[0025] Figure 3 This is a third flow chart of the method for associating baggage photos and X-ray images at a passenger station security inspection in one embodiment of the present application.

[0026] Figure 4 This is a diagram showing an example of taking photos of packages for security inspection at a passenger station.

[0027] Figure 5 This is a schematic diagram of the architecture of a twin network with a 16x downsampling rate based on a hybrid attention mechanism in one embodiment of the present application.

[0028] Figure 6 Schematic diagram of residual block architecture corresponding to the first residual block, the second residual block, the third residual block, the fourth residual block and the fifth residual block in one embodiment of the present application.

[0029] Figure 7 A schematic diagram of a residual block architecture corresponding to a residual block based on a hybrid attention mechanism in an embodiment of the present application.

[0030] Figure 8 This is a fourth flow chart of the method for associating baggage photos and X-ray images at a passenger station security inspection in one embodiment of the present application.

[0031] Fig. 9 The present invention is a schematic diagram showing an example of the application process of the method for associating baggage photos and X-ray images at a passenger station security inspection in one embodiment of the present application.

[0032] Fig.10 This is a schematic diagram of the first structure of a device for associating baggage photos and X-ray images at a passenger station security inspection in one embodiment of the present application.

[0033] Fig.11 This is a second structural schematic diagram of the device for associating baggage photos and X-ray images at a passenger station security inspection in one embodiment of the present application. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0035] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, only structures and / or processing steps closely related to the solutions according to the present invention are shown in the accompanying drawings, while other details that are not closely related to the present invention are omitted.

[0036] It should be emphasized that the term “include / comprises” when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0037] It should also be noted that, unless otherwise specified, the term “connection” herein may refer not only to a direct connection but also to an indirect connection involving an intermediate.

[0038] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0039] Automatic security sorting refers to the automatic separation of parcels that need to be manually opened for inspection, eliminating the process of manually opening and inspecting parcels, which is of great significance to reducing the burden of security inspection and improving security inspection efficiency. Existing automatic security sorting is mainly used in the field of civil aviation security inspection, relying on storage baskets and electronic tags, that is, passengers must place their parcels in storage baskets to realize automatic sorting of parcels. However, this method is not applicable to railway security inspection. Therefore, this application studies the method of associating security inspection package photos and X-ray images at security inspection passenger stations, which is particularly suitable for scenarios with large user groups such as passenger stations, to support automatic sorting of parcels without storage baskets.

[0040] It should be noted that associating a package with an X-ray image of the package can be referred to as package-image association, which refers to associating the physical photo of the package with the X-ray image to form a one-to-one correspondence. Obviously, based on the package-image association, it is possible to form a correspondence between each package and the X-ray image without relying on the storage basket, thereby determining the sorting direction of each package on the sorting device. Package-image association is a highly specialized scenario task, and conventional methods have poor association effects. Usually, technicians in this field will combine the speed of the security scanner conveyor belt to estimate the time difference between the generation time of the security inspection X-ray image and the time when the package arrives at the exit of the security scanner to determine the photo taking time and obtain the physical photo. However, the photo taking time accuracy of the above method is limited, and there are multiple packages in the photo, so the association effect is not good.

[0041] In order to solve the above problems, some technicians have considered using machine vision methods to associate X-ray images with photos of passenger station security inspection packages, but they simply input the two images into the convolutional neural network, without performing targeted processing according to the specific scene of railway passenger station security inspection, resulting in failure to meet on-site use requirements. Specifically, due to the huge difference in style between the package X-ray image and the visible light photo, the simple use of the existing convolutional neural network cannot guarantee the association effect. Based on this, the embodiments of the present application respectively provide a method for associating passenger station security inspection package photos with X-ray images, a passenger station security inspection package photo and X-ray image association device for executing the passenger station security inspection package photo and X-ray image association method, an electronic device, a computer-readable storage medium and a computer program product, which can realize the automatic association of passenger station security inspection package photos and X-ray images and improve the accuracy of the association results.

[0042] The details are described in detail through the following examples.

[0043] Based on this, the embodiment of the present application provides a method for associating a passenger station security inspection package photo with an X-ray image, which can be implemented by a passenger station security inspection package photo and X-ray image associating device, see Figure 1 The method for associating the passenger station security inspection package photo with the X-ray image specifically includes the following contents: Step 100: Obtain the target package physical image corresponding to each passenger station security inspection package photo, and construct each training sample based on the package X-ray image and label corresponding to each of the target package physical images, wherein the label is used to indicate whether the target package physical image is associated with the corresponding package X-ray image.

[0044] In step 100, the passenger station security inspection package photo can be collected by a photo collection device set at the exit of the security inspection instrument and above the conveyor belt in the passenger station and other scenes, and then pre-processed and normalized by the passenger station security inspection package photo and X-ray image association device. The photo collection device can be a camera and other devices. The package X-ray image can be collected by the same security inspection instrument corresponding to the photo collection device, and then normalized by the passenger station security inspection package photo and X-ray image association device. The pre-processing and normalization process will be described in detail in the subsequent embodiments.

[0045] In one or more embodiments of the present application, each of the training samples includes a target package physical image, a package X-ray image and a label. If the label indicates that the target package physical image is associated with the corresponding package X-ray image, then the training sample is a positive sample. If the label indicates that the target package physical image is not associated with the corresponding package X-ray image, then the training sample is a negative sample.

[0046] Step 200: Use each of the training samples to train a twin network with a 16-fold downsampling rate based on a hybrid attention mechanism to train the twin network into a package photo and X-ray image association model, so that the package photo and X-ray image association model is used to output the association recognition result data between the target package physical image and the package X-ray image according to the input target package physical image and the package X-ray image.

[0047] In step 200, due to the huge difference in styles between the package X-ray image and the visible light photograph, simply using the existing convolutional neural network cannot guarantee the association effect. Therefore, the present application adopts a twin network convolutional neural network with a low sampling rate (16 times downsampling rate) in step 200, and introduces a hybrid attention mechanism to achieve effective training of the package photo and X-ray image association model, and can effectively improve the accuracy and effectiveness of using the model to associate passenger station security inspection package photos and package X-ray images from different sources.

[0048] It can be understood that the association recognition result data is used to indicate whether the target package physical image is associated with the corresponding package X-ray image, and the data may be composed of probability values ​​corresponding to the first identifier, the second identifier and the third identifier, respectively, wherein the first identifier is used to indicate that the target package physical image is associated with the corresponding package X-ray image, the second identifier is used to indicate that the target package physical image is not associated with the corresponding package X-ray image, and the third identifier is used to indicate that the association result cannot be identified.

[0049] In one or more embodiments of the present application, whether the bus station security inspection package photo and the X-ray image are associated refers to whether the two are for the same package, that is, if the packages contained in the bus station security inspection package photo and the X-ray image point to the same package, then the two are associated. It can be understood that in the present application, the association can also be referred to as association.

[0050] From the above description, it can be seen that the method for associating passenger station security package photos and X-ray images provided in the embodiment of the present application can realize the effective training of the package photo and X-ray image association model, and can effectively improve the accuracy and effectiveness of associating passenger station security package photos and package X-ray images from different sources using the model, and can effectively improve the automation and intelligence of associating passenger station security package photos and package X-ray images from different sources, thereby effectively improving the accuracy and reliability of applications such as railway security inspection automatic sorting using the association recognition results, and effectively saving labor costs and time costs.

[0051] On the basis of realizing automatic association between the security package photos and the X-ray images of the packages at the passenger station and improving the association accuracy, it should also be considered that the security package photos and X-ray images of the passenger station are heterogeneous images, resulting in a large difference between the two, and the shooting angle of the security package photos of the passenger station is inconsistent with the scanning angle of the X-ray images, which will affect the performance of the association model between the security package photos and the X-ray images of the packages to a certain extent. At the same time, the camera of the railway passenger station security inspection instrument mostly has angle changes and deformations when shooting large packages, and there are a lot of interfering background problems around. In addition, due to the sensitivity of scanning imaging, the image contour of the X-ray image may deviate from the actual package, which further aggravates the difficulty of associating the two. If the image is directly sent to the convolutional neural network, the association effect is difficult to guarantee.

[0052] Therefore, in order to solve the problem that the shooting angle of the passenger station security inspection package photo is inconsistent with the scanning angle of the X-ray image, and the camera of the railway passenger station security inspection instrument mostly has angle changes and deformations when shooting large packages, and there are a lot of interference backgrounds around, in order to further improve the model training effect and association accuracy, in a method for associating a passenger station security inspection package photo with an X-ray image provided in an embodiment of the present application, see Figure 2 Step 100 of the method for associating the passenger station security inspection package photo with the X-ray image specifically includes the following contents: Step 110: Based on the edge position data of the conveyor belt, the photos of the security inspection packages at each passenger station collected by the photo collection device above the conveyor belt are cropped to obtain a conveyor belt area photo corresponding to each of the passenger station security inspection package photos, which is used to show that the package is located on the conveyor belt.

[0053] Step 120: Perform angle correction processing on each of the conveyor belt area photos to obtain a target image corresponding to each of the conveyor belt area photos.

[0054] Step 130: performing background segmentation processing on each of the target images to obtain a package physical image corresponding to each of the target images.

[0055] Step 140: normalizing each of the package physical images and the source package X-ray images corresponding to each of the package physical images to obtain a target package physical image corresponding to each of the package physical images and a package X-ray image corresponding to each of the source package X-ray images.

[0056] In order to further improve the effectiveness and reliability of angle correction, and to further improve the model training effect and association accuracy, in a method for associating a passenger station security inspection package photo with an X-ray image provided in an embodiment of the present application, see Figure 3Step 120 of the method for associating the passenger station security inspection package photo with the X-ray image specifically includes the following contents: Step 121: Set the top surface of the conveyor belt as the source plane, and set the plane perpendicular to the axis of the camera of the photo acquisition device as the target plane.

[0057] Step 122: Perform perspective transformation processing on each of the conveyor belt area photos located on the source plane collected by the photo acquisition device, so as to transform each of the conveyor belt area photos into each target image located on the target plane.

[0058] Step 123: Based on a preset angle transformation rule, adjust the angle of each target image.

[0059] Specifically, the embodiment of the present application solves the problem that the camera shooting angle is inconsistent with the security scanner scanning angle, designs a perspective transformation method and an angle change strategy to correct the passenger station security inspection package photo, see Figure 4 , set the top surface of the conveyor belt as the source plane M, set the plane perpendicular to the axis of the camera of the photo acquisition device as the target plane N, transform the two-dimensional conveyor belt area photo in the source plane M into three-dimensional space, and then project it from the three-dimensional space to the two-dimensional target image in the target plane N, so as to realize the transformation from the conveyor belt area photo to the target image.

[0060] Among them, the perspective change matrix is ​​expressed as follows: (1) In formula (1), They are the x-axis, y-axis, and z-axis coordinates of the conveyor belt area photo in the three-dimensional coordinate system. For the two-dimensional conveyor belt area photo, w is 1. is the coordinate of the x-axis, y-axis and z-axis transformed to three-dimensional space. T is , is the perspective transformation matrix, where t 11 ,t 12 ,t 13 ,t 21 ,t 22 ,t 23 ,t 31 ,t 32 ,t 33 are all elements in the perspective transformation matrix.

[0061] The coordinates in the corresponding target image can be calculated from the coordinates in the three-dimensional space ,in: (2) (3) By analyzing equations (1) to (3), it can be seen that by providing the four coordinate points in the conveyor belt area photo and their corresponding coordinates in the target image, the perspective transformation matrix T can be calculated to achieve the transformation correction from the conveyor belt area photo to the target image.

[0062] In order to further improve the effectiveness and reliability of background segmentation, and to further improve the model training effect and association accuracy, in a method for associating a passenger station security inspection package photo with an X-ray image provided in an embodiment of the present application, see Figure 3 Step 130 of the method for associating the passenger station security inspection package photo with the X-ray image specifically includes the following contents: Step 131: Input each of the target images into a pre-trained instance segmentation model, so that the instance segmentation model performs background segmentation on each of the target images and outputs a package physical image corresponding to each of the target images as the foreground; wherein the instance segmentation model includes: a YOLOv8n-Seg model.

[0063] Specifically, the embodiment of the present application uses the YOLOv8n-seg model to perform instance segmentation to reduce background interference in the surrounding environment and the conveyor belt of the security inspection instrument in the security inspection package photo of the passenger station. The YOLOv8n-Seg model is used for instance segmentation to eliminate background area interference. The YOLOv8n-Seg model is trained by the following method: (1) Collect photos of packages for security inspection at the passenger station on site; (2) Labeling photos of passenger station security inspection packages to form a segmentation training set; (3) Train the YOLOv8n-Seg model.

[0064] For the input image, the length and width are normalized to 640 pixels, and finally, instance segmentation is achieved in three segment branches. Among them, the 80×80×64 branch is used to achieve the segmentation of smaller objects, the 40×40×128 branch is used to achieve the segmentation of medium-sized objects, and the 20×20×256 branch is used to achieve the segmentation of larger objects.

[0065] On the basis of having realized the automatic association between the security inspection package photo and the package X-ray image at the passenger station and improving the association accuracy, it should also be considered in the training and reasoning links of the network model. Most mainstream methods will normalize the input image size and stretch and deform (resize) it to a specific size, but the above method will result in the loss of image shape information, resulting in a significant decrease in on-site association accuracy. Therefore, the embodiment of the present application also proposes to use edge padding to complete the image to a specific size, maintain the original shape information of the image, so as to improve the association accuracy.

[0066] Specifically, in order to avoid the loss of image shape information and to further improve the model training effect and association accuracy, in a method for associating a passenger station security inspection package photo with an X-ray image provided in an embodiment of the present application, see Figure 3 Step 140 in the method for associating the passenger station security inspection package photo with the X-ray image specifically includes the following contents: Step 141: Perform edge filling processing on each of the physical package images and the source package X-ray images corresponding to each of the physical package images, so as to complete each of the physical package images to a preset size and form a corresponding target package physical image, and complete each of the source package X-ray images to a preset size and form a corresponding package X-ray image.

[0067] That is to say, in the task of associating the baggage photos and X-ray images at the passenger station security check, the shape of the target contains important information, so the input image cannot be resized and normalized. After analysis, all the segmented physical photos and most of the X-ray images are less than 1024 pixels in size, so step 141 uses the Padding normalization operation on all input images, and the length and width after normalization are both 1024 pixels. Therefore, in order to solve the problem that the color, contour, appearance, and structure of heterogeneous images are very different and it is difficult to extract common features artificially, the embodiment of the present application designs a 16-fold downsampling rate twin network based on a hybrid attention mechanism, and uses edge filling (padding) instead of the commonly used stretching deformation (resize) normalization method to pre-process the input image.

[0068] On the basis of realizing the automatic association between the security inspection package photos and the package X-ray images of the passenger station and improving the association accuracy, it should also be considered that due to the high specificity of the scene, the data collection cost should be considered, and less data should be used for training and learning as much as possible. Based on this, in order to achieve the use of less data for training the model and meet the model association accuracy to reduce the model training cost, in a method for associating the security inspection package photos and X-ray images of the passenger station provided in an embodiment of the present application, see Figure 2 or Figure 3 Step 100 of the method for associating the passenger station security inspection package photo with the X-ray image further specifically includes the following contents: Step 150: each of the target package physical image and the package X-ray image corresponding to the target package physical image is respectively regarded as an original positive sample pair; positive sample expansion is performed for each of the original positive sample pairs to obtain a plurality of expanded positive sample pairs corresponding to the original positive sample pair, and labels for indicating the association between the target package physical image and the corresponding package X-ray image are respectively set for the original positive sample pair and each of the expanded positive sample pairs to obtain a plurality of training samples respectively serving as positive samples; wherein the positive sample expansion includes: rotating the original positive sample pair at different angles, changing the size at different magnifications, and randomly adjusting the color; And, step 160: randomly setting an unrelated package X-ray image for each of the target package physical images to obtain an original negative sample pair; performing negative sample expansion for each of the original negative samples to obtain multiple expanded negative sample pairs corresponding to the original negative sample pair, and setting labels for indicating that the target package physical image and the corresponding package X-ray image are unrelated for the original negative sample pair and each of the expanded negative sample pairs to obtain multiple training samples respectively serving as negative samples; wherein the negative sample expansion includes: rotating the original negative sample pair at multiple different angles, changing the size at different magnifications, and randomly adjusting the color.

[0069] In one example, before model training, photos of security inspection packages and security inspection images of passenger stations can be collected on site to form original image pairs; then image enhancement is performed to expand the data set. Among them, for each original image pair, it is itself a positive sample pair; the two images are rotated 90 degrees at the same time to form the second positive sample pair; the two images are rotated 180 degrees at the same time and reduced to 0.7 times at the same time to form the third positive sample pair; the two images are rotated 270 degrees at the same time and reduced to 0.7 times at the same time to form the fourth positive sample pair. Therefore, each pair of original images can generate 4 positive sample pairs. For each X-ray image in the original image pair, randomly associate it with another photo of the actual package to form a negative sample pair; rotate the photo of the package at the passenger station security check by 90 degrees to form the second negative sample pair; rotate the photo of the package at the passenger station security check by 90 degrees and reduce it to 0.7 times to form the third negative sample pair; rotate the photo of the package at the passenger station security check by 180 degrees and reduce the X-ray image to 0.7 times to form the fourth negative sample pair. Therefore, each pair of original images can generate 4 negative sample pairs. You can also randomly adjust the color of the package photo at the passenger station security check and the X-ray image, and randomly adjust the hue and saturation proportionally between 0.75 and 1 to increase the generalization of the sample as much as possible. That is to say, in this example, one original image pair can be expanded to obtain 8 enhanced image pairs, and the expanded data can be used for training.

[0070] In order to further realize the effective training of the package photo and X-ray image association model, and to effectively improve the accuracy and effectiveness of associating the passenger station security package photos and package X-ray images from different sources using the model, in a passenger station security package photo and X-ray image association method provided in an embodiment of the present application, a 16-fold down-sampling twin network based on a hybrid attention mechanism is designed to realize the association of passenger station security package photos and X-ray images. In order to improve computational efficiency while maintaining more detailed features of the image, a residual network with a 16-fold down-sampling rate is designed. In order to enhance the network's ability to extract common features in visible light photos and X-ray images, a hybrid attention mechanism is added before the adaptive pooling layer. Through the above two designs, it is ensured that the network can better adapt to the package image association task at the passenger station site.

[0071] See also Figure 5 , the twin network with 16 times downsampling rate based on hybrid attention mechanism in the method for associating the security inspection package photo and X-ray image of the passenger station includes: a first residual network with 16 times downsampling rate, a second residual network with 16 times downsampling rate, and a prediction layer connected to the output ends corresponding to the first residual network and the second residual network, respectively, and the prediction layer is written as Pred module; The input of the first residual network is the target package physical image, and the output is the physical image feature corresponding to the target package physical image; The input of the second residual network is the package X-ray image, and the output is the X-ray image features corresponding to the package X-ray image; The input of the prediction layer is the physical image features and the X-ray image features, and the output is the association recognition result data between the physical image of the target package and the X-ray image features; The first residual network and the second residual network both include: sequentially connected convolution kernel Conv (7, 7, 64, 2), maximum pooling layer Maxpool / 2, first residual block Block (64, 1), second residual block Block (64, 1), third residual block Block (128, 2), fourth residual block Block (128, 1), fifth residual block Block (256, 2), residual block AttentionBlock (265, 1) based on hybrid attention mechanism, adaptive pooling module Adaptive Pooling and fully connected layer FC (256, 128); The downsampling rates of the convolution kernel Conv (7, 7, 64, 2), the maximum pooling layer Maxpool / 2, the third residual block Block (128, 2) and the fifth residual block Block (256, 2) are all 2; the downsampling rates of the first residual block Block (64, 1), the second residual block Block (64, 1), the fourth residual block Block (128, 1) and the residual block AttentionBlock (265, 1) based on the hybrid attention mechanism are all 1.

[0072] Among them, Conv (7,7,64,2) represents a convolution kernel with a length and width of 7, an output channel dimension of 64, and a downsampling rate of 2; Maxpool / 2 represents a maximum pooling layer with a downsampling rate of 2; Adaptive Pooling module is an adaptive pooling module, which can output a feature value of length 256; FC (256,128) represents a fully connected layer with an input dimension of 256 and an output dimension of 128, which can output a feature value of length 128 for each input image; Pred module is a prediction layer, which analyzes the 128-dimensional feature values ​​of the two image inputs to determine whether the two images are associated, and then outputs the distance (margine value) between the two images. Then, based on the output margin value, it can be determined whether the two images belong to the same package. If the margin is greater than the threshold, they do not belong to the same package; if the margin is less than the threshold, they belong to the same package.

[0073] See also Figure 6 , the residual block architectures of the first residual block Block (64, 1), the second residual block Block (64, 1), the third residual block Block (128, 2), the fourth residual block Block (128, 1) and the fifth residual block Block (256, 2) are the same, and the Block (c, d) represents a residual block with an output channel dimension of c and a downsampling rate of d, c and d are both positive integers, and are formed by connecting the convolution kernel Conv (3, 3, c, d) and the convolution kernel Conv (3, 3, c, 1), wherein x represents the input of the residual block, Conv (3, 3, c, d) represents a convolution kernel with a length and width of 3, an output channel dimension of c and a downsampling rate of d, Conv (3, 3, c, 1) represents a convolution kernel with a length and width of 3, an output channel dimension of c and a downsampling rate of 1, and relu refers to a linear rectification function. It can also be written as ReLU.

[0074] See also Figure 7In the residual block architecture of the residual block AttentionBlock(265, 1) based on the hybrid attention mechanism, AttentionBlock(c, d) represents a residual block with an output channel dimension of c and a downsampling rate of d, to which a hybrid attention mechanism is added. It is composed of the convolution kernel Conv(3, 3, c, d), the convolution kernel Conv(3, 3, c, 1) and the module where the hybrid attention mechanism is located.

[0075] Specifically, when training a 16-fold downsampling twin network based on a hybrid attention mechanism, two images, the target package physical image and the package X-ray image, are input simultaneously, and features are extracted from the two images respectively to obtain two feature vectors. Finally, the distance between the two feature vectors is estimated and used as the loss. Therefore, this application can use the Contrastive Loss function as the loss function for training, which is defined as follows: (4) In formula (4) (4-19), L represents contrastive loss, and N represents the length of the feature value. y represents the true label. For two related images, y is 1, and for two unrelated images, y is 0. m is the hyperparameter margin, which represents the distance that the model is expected to pull apart unrelated image pairs. When this distance is greater than m, the loss function value is 0. d represents the Euclidean distance between the feature values ​​of two images, which is defined as follows: (5) In formula (5), , They represent the feature values ​​of the target package physical image and the package X-ray image respectively. , They respectively represent the value of the i-th bit in the eigenvalues ​​corresponding to the physical image of the target package and the X-ray image of the package.

[0076] In order to further improve the accuracy and effectiveness of associating the photos of passenger station security inspection packages from different sources with the X-ray images of the packages, and to improve the accuracy and reliability of applications such as railway security inspection automatic sorting using the associated recognition results, in the method for associating the photos of passenger station security inspection packages with X-ray images provided in the present application, the physical photos of the packages are derived from the camera at the exit of the security scanner, and the X-ray images are derived from the security scanner itself. There are multiple mechanisms for determining the moment when the physical photos of the packages are taken, which can be estimated based on the time difference or triggered by the installation of sensors. Regardless of the source of the photos of passenger station security inspection packages, they can be associated with X-ray images through the method.

[0077] See also Figure 8 and Fig. 9The method for associating the security inspection package photo with the X-ray image at the passenger station further specifically includes the following contents after step 200: Step 300: Acquire the photos of the passenger station security inspection packages currently collected by the photo acquisition equipment arranged at the exit of the security inspection instrument and above the conveyor belt, and the X-ray image of the source package collected by the security inspection instrument.

[0078] Step 400: cropping, angle correction and background segmentation are performed on the passenger station security inspection package photo to obtain a package physical image corresponding to the passenger station security inspection package photo.

[0079] Step 500: normalizing the package physical image and the source package X-ray image to obtain a target package physical image corresponding to the package physical image and a package X-ray image corresponding to the source package X-ray image.

[0080] Step 600: Input the target package physical image and the package X-ray image into a package photo and X-ray image association model, so that the package photo and X-ray image association model outputs association recognition result data between the target package physical image and the package X-ray image.

[0081] From the above description, it can be seen that the method for associating passenger station security package photos and X-ray images provided in the embodiment of the present application can effectively improve the accuracy and effectiveness of associating passenger station security package photos and package X-ray images from different sources using the model, and can effectively improve the degree of automation and intelligence of associating passenger station security package photos and package X-ray images from different sources, thereby effectively improving the accuracy and reliability of applications such as railway security inspection automatic sorting using the associated recognition results, and effectively saving labor costs and time costs.

[0082] From the software level, the present application also provides a device for associating passenger station security inspection package photos and X-ray images for executing all or part of the passenger station security inspection package photos and X-ray image association method, see Fig.10 The passenger station security inspection package photo and X-ray image association device specifically includes the following contents: The sample construction module 10 is used to obtain the target package physical image corresponding to each of the passenger station security inspection package photos, and to construct each training sample based on the package X-ray image and label corresponding to each of the target package physical images, wherein the label is used to indicate whether the target package physical image is associated with the corresponding package X-ray image; The model training module 20 is used to use each of the training samples to train a twin network with a 16-fold downsampling rate based on a hybrid attention mechanism, so as to train the twin network into a package photo and X-ray image association model, so that the package photo and X-ray image association model is used to output the association recognition result data between the target package physical image and the package X-ray image according to the input target package physical image and the package X-ray image.

[0083] The embodiment of the device for associating baggage photos and X-ray images at passenger station security inspections provided in the present application can be specifically used to execute the processing flow of the embodiment of the method for associating baggage photos and X-ray images at passenger station security inspections in the above-mentioned embodiment. Its functions will not be described in detail here, and reference may be made to the detailed description of the embodiment of the method for associating baggage photos and X-ray images at passenger station security inspections.

[0084] The part of the bus station security inspection package photo and X-ray image association device that associates the bus station security inspection package photo with the X-ray image can be executed in the server or completed in the client device. The specific selection can be based on the processing capability of the client device and the limitations of the user's usage scenario. This application does not limit this. If all operations are completed in the client device, the client device may also include a processor for specific processing of the association between the bus station security inspection package photo and the X-ray image.

[0085] The client device may have a communication module (i.e., a communication unit) that can communicate with a remote server to achieve data transmission with the server. The server may include a server on the task scheduling center side, and other implementation scenarios may also include a server on an intermediate platform, such as a server on a third-party server platform that has a communication link with the task scheduling center server. The server may include a single computer device, or a server cluster consisting of multiple servers, or a server structure of a distributed device.

[0086] The server and the client device may communicate with each other using any suitable network protocol, including network protocols that have not yet been developed on the date of filing this application. The network protocols may include, for example, TCP / IP, UDP / IP, HTTP, HTTPS, etc. Of course, the network protocols may also include, for example, RPC (Remote Procedure Call Protocol) and REST (Representational State Transfer) protocols used on top of the above protocols.

[0087] From the above description, it can be seen that the passenger station security package photo and X-ray image association device provided in the embodiment of the present application can realize the effective training of the package photo and X-ray image association model, and can effectively improve the accuracy and effectiveness of associating passenger station security package photos and package X-ray images from different sources using the model, and can effectively improve the automation and intelligence of associating passenger station security package photos and package X-ray images from different sources, thereby effectively improving the accuracy and reliability of applications such as railway security inspection automatic sorting using the association recognition results, and effectively saving labor costs and time costs.

[0088] In order to further improve the accuracy and effectiveness of associating the passenger station security inspection package photos and package X-ray images from different sources, and to improve the accuracy and reliability of applications such as railway security inspection automatic sorting using the associated recognition results, in the passenger station security inspection package photo and X-ray image association device provided in this application, see Fig.11 The passenger station security inspection package photo and X-ray image association device also specifically includes the following contents: The image acquisition module 30 is used to obtain the photos of the passenger station security inspection packages currently collected by the photo acquisition equipment arranged at the exit of the security inspection instrument and above the conveyor belt, and the X-ray image of the source package collected by the security inspection instrument.

[0089] The image preprocessing module 40 is used to perform cropping, angle correction and background segmentation processing on the passenger station security inspection package photo to obtain a package physical image corresponding to the passenger station security inspection package photo.

[0090] The image normalization module 50 is used to perform normalization processing on the package physical image and the source package X-ray image to obtain a target package physical image corresponding to the package physical image and a package X-ray image corresponding to the source package X-ray image.

[0091] The model association module 60 is used to input the target package physical image and the package X-ray image into the package photo and X-ray image association model, so that the package photo and X-ray image association model outputs the association recognition result data between the target package physical image and the package X-ray image.

[0092] The embodiment of the present application also provides an electronic device, which may include a processor, a memory, a receiver and a transmitter, wherein the processor is used to execute the method for associating the passenger station security inspection package photo with the X-ray image mentioned in the above embodiment, wherein the processor and the memory may be connected via a bus or other means, taking the bus connection as an example. The receiver may be connected to the processor and the memory via a wired or wireless manner.

[0093] The processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0094] As a non-transitory computer-readable storage medium, the memory can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as the program instructions / modules corresponding to the method for associating the security inspection package photo with the X-ray image at the passenger station in the embodiment of the present application. The processor executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory, that is, the method for associating the security inspection package photo with the X-ray image at the passenger station in the above method embodiment is realized.

[0095] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0096] The one or more modules are stored in the memory, and when executed by the processor, perform the method for associating passenger station security inspection package photos and X-ray images in the embodiment.

[0097] In some embodiments of the present application, the user equipment may include a processor, a memory, and a transceiver unit, which may include a receiver and a transmitter. The processor, memory, receiver, and transmitter may be connected through a bus system. The memory is used to store computer instructions, and the processor is used to execute the computer instructions stored in the memory to control the transceiver unit to send and receive signals.

[0098] As an implementation method, the functions of the receiver and the transmitter in the present application can be considered to be implemented through a transceiver circuit or a dedicated chip for transceiver, and the processor can be considered to be implemented through a dedicated processing chip, a processing circuit or a general chip.

[0099] As another implementation method, it is possible to use a general-purpose computer to implement the server provided in the embodiment of the present application, that is, to store the program code for implementing the functions of the processor, receiver, and transmitter in a memory, and the general-purpose processor implements the functions of the processor, receiver, and transmitter by executing the code in the memory.

[0100] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the aforementioned method for associating the security inspection package photo with the X-ray image at the passenger station are implemented. The computer-readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0101] The embodiment of the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the aforementioned method for associating a security inspection package photo with an X-ray image at a passenger station.

[0102] It should be understood by those skilled in the art that the exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.

[0103] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0104] In the present application, features described and / or illustrated for one embodiment may be used in the same manner or in a similar manner in one or more other embodiments, and / or combined with features of other embodiments or replace features of other embodiments.

[0105] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the embodiments of the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for associating a passenger station security inspection package photo with an X-ray image, characterized in that: include: Obtaining the target package physical image corresponding to each of the security inspection package photos at each passenger station, and constructing each training sample based on the package X-ray image and label corresponding to each of the target package physical images, wherein the label is used to indicate whether the target package physical image is associated with the corresponding package X-ray image; Each of the training samples is used to train a twin network with a 16-fold downsampling rate based on a hybrid attention mechanism, so as to train the twin network into a package photo and X-ray image association model, so that the package photo and X-ray image association model is used to output the association recognition result data between the target package physical image and the package X-ray image according to the input target package physical image and the package X-ray image.

2. The method for associating a passenger station security inspection package photo with an X-ray image according to claim 1, characterized in that: The step of obtaining the physical images of the target packages corresponding to the security inspection package photos of each passenger station includes: Based on the edge position data of the conveyor belt, the photos of the security inspection packages at the passenger station collected by the photo collection device above the conveyor belt are cropped to obtain the conveyor belt area photos corresponding to each of the security inspection package photos at the passenger station for showing that the packages are located on the conveyor belt; Performing angle correction processing on each of the conveyor belt area photos respectively to obtain a target image corresponding to each of the conveyor belt area photos; Performing background segmentation processing on each of the target images respectively to obtain a package physical image corresponding to each of the target images; Normalization processing is performed on each of the package physical images and the source package X-ray images corresponding to each of the package physical images to obtain the target package physical images corresponding to each of the package physical images and the package X-ray images corresponding to each of the source package X-ray images.

3. The method for associating a passenger station security inspection package photo with an X-ray image according to claim 2, characterized in that: The step of performing angle correction processing on each of the conveyor belt area photos to obtain a target image corresponding to each of the conveyor belt area photos includes: The top surface of the conveyor belt is set as the source plane, and the plane perpendicular to the axis of the camera of the photo acquisition device is set as the target plane; Performing perspective transformation processing on each of the conveyor belt area photos located on the source plane collected by the photo collection device, so as to transform each of the conveyor belt area photos into each target image located on the target plane; Based on a preset angle transformation rule, the angle of each target image is adjusted.

4. The method for associating a passenger station security inspection package photo with an X-ray image according to claim 2, characterized in that: The performing background segmentation processing on each of the target images to obtain the package physical image corresponding to each of the target images includes: Inputting each of the target images into a pre-trained instance segmentation model, so that the instance segmentation model performs background segmentation on each of the target images and outputs a package physical image corresponding to each of the target images as a foreground; Wherein, the instance segmentation model includes: YOLOv8n-Seg model.

5. The method for associating a passenger station security inspection package photo with an X-ray image according to claim 2, characterized in that: The normalizing process of each of the package physical images and the source package X-ray images corresponding to each of the package physical images to obtain the target package physical images corresponding to each of the package physical images and the package X-ray images corresponding to each of the source package X-ray images includes: Edge filling processing is performed on each of the physical package images and the source package X-ray images corresponding to each of the physical package images to complete each of the physical package images to a preset size and form a corresponding target package physical image, and each of the source package X-ray images is completed to a preset size and form a corresponding package X-ray image.

6. The method for associating a passenger station security inspection package photo with an X-ray image according to claim 1, characterized in that: The step of constructing each training sample based on the X-ray image of the package and the label corresponding to each of the physical images of the target package includes: Each of the target package physical images and the package X-ray images corresponding to the target package physical images are respectively taken as an original positive sample pair; positive sample expansion is performed on each of the original positive sample pairs to obtain a plurality of expanded positive sample pairs corresponding to the original positive sample pair, and labels for indicating the association between the target package physical images and the corresponding package X-ray images are respectively set for the original positive sample pairs and each of the expanded positive sample pairs to obtain a plurality of training samples respectively serving as positive samples; wherein the positive sample expansion includes: rotating the original positive sample pairs at different angles, changing the sizes at different magnifications, and randomly adjusting the colors; And, for each of the target package physical image, an unrelated package X-ray image is randomly set to obtain an original negative sample pair; for each of the original negative samples, negative sample expansion is performed to obtain multiple expanded negative sample pairs corresponding to the original negative sample pair, and labels used to indicate that the target package physical image and the corresponding package X-ray image are unrelated are respectively set for the original negative sample pair and each of the expanded negative sample pairs to obtain multiple training samples respectively serving as negative samples; wherein the negative sample expansion includes: rotating the original negative sample pair at multiple different angles, changing the size at different magnifications, and randomly adjusting the color.

7. The method for associating a passenger station security inspection package photo with an X-ray image according to any one of claims 1 to 6, characterized in that: The 16-fold downsampling twin network based on the hybrid attention mechanism includes: a first residual network with a 16-fold downsampling rate, a second residual network with a 16-fold downsampling rate, and a prediction layer connected to the output ends corresponding to the first residual network and the second residual network respectively; The input of the first residual network is the target package physical image, and the output is the physical image feature corresponding to the target package physical image; The input of the second residual network is the package X-ray image, and the output is the X-ray image features corresponding to the package X-ray image; The input of the prediction layer is the physical image features and the X-ray image features, and the output is the association recognition result data between the physical image of the target package and the X-ray image features; The first residual network and the second residual network both include: sequentially connected convolution kernels, maximum pooling layers, first residual blocks, second residual blocks, third residual blocks, fourth residual blocks, fifth residual blocks, residual blocks based on a hybrid attention mechanism, adaptive pooling modules, and fully connected layers; The downsampling rates of the convolution kernel, the maximum pooling layer, the third residual block and the fifth residual block are all 2; the downsampling rates of the first residual block, the second residual block, the fourth residual block and the residual block based on the hybrid attention mechanism are all 1.

8. The method for associating a passenger station security inspection package photo with an X-ray image according to claim 2, characterized in that: Also includes: Acquire the photos of the passenger station security inspection packages currently collected by the photo collection equipment arranged at the exit of the security inspection instrument and above the conveyor belt, and the X-ray images of the source packages collected by the security inspection instrument; The passenger station security inspection package photo is cropped, angle corrected, and background segmented to obtain a package physical image corresponding to the passenger station security inspection package photo; Normalizing the package physical image and the source package X-ray image to obtain a target package physical image corresponding to the package physical image and a package X-ray image corresponding to the source package X-ray image; The target package physical image and the package X-ray image are input into the package photo and X-ray image association model, so that the package photo and X-ray image association model outputs association recognition result data between the target package physical image and the package X-ray image.

9. A device for associating photos of security inspection packages with X-ray images at passenger stations, characterized in that: include: A sample construction module is used to obtain the target package physical image corresponding to each passenger station security inspection package photo, and construct each training sample based on the package X-ray image and label corresponding to each target package physical image, wherein the label is used to indicate whether the target package physical image is associated with the corresponding package X-ray image; A model training module is used to use each of the training samples to train a twin network with a 16-fold downsampling rate based on a hybrid attention mechanism, so as to train the twin network into a package photo and X-ray image association model, so that the package photo and X-ray image association model is used to output the association recognition result data between the target package physical image and the package X-ray image according to the input target package physical image and the package X-ray image.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method for associating a passenger station security inspection package photo with an X-ray image as described in any one of claims 1 to 8 is implemented.

Images