Security article automatic sorting system and method

Abstract

The application provides an automatic sorting system and method for security check articles, and relates to the technical field of security check. The system comprises: a sensor processing module, which is used for acquiring an infrared sensor blocking time ratio; an image acquisition module, which is used for acquiring first image information, second image information and target X-ray image information; a first processing module, which is used for inputting the second image information and the target X-ray image information into a first twin network to obtain a first similarity measurement value; a second processing module, which is used for inputting the second image information and the first image information into a second twin network to obtain a second similarity measurement value; and a sorting module, which is used for acquiring a fusion discrimination coefficient according to the infrared sensor blocking time ratio, the first similarity measurement value and the second similarity measurement value, and sorting the security check articles based on the fusion discrimination coefficient. The application improves the security check efficiency and quality.

Description

TECHNICAL FIELD

[0001] The present application relates to the security technology field, and in particular to a security article automatic sorting system and method. BACKGROUND

[0002] In recent years, with the gradual application of dual-view security inspection instruments and intelligent image judgment technologies, the technical means related to check-in image judgment have been continuously strengthened.

[0003] However, the handlers in passenger security inspection, as the key positions for specific implementation of package opening inspection, have not been paid attention to in terms of technical means and equipment devices. Due to a large number of packages, the existing security handling operation has the problems of mutual interference between package searching and package opening inspection tasks, and easy package searching errors, which leads to high work intensity of handlers and reduces security inspection efficiency and quality.

[0004] Therefore, there is an urgent need for a security article automatic sorting system and method to solve the above problems. SUMMARY

[0005] In view of the problems in the prior art, the present application provides a security article automatic sorting system and method.

[0006] The present application provides a security article automatic sorting system, comprising:

[0007] A sensor processing module is configured to obtain an infrared sensor blocking time ratio, wherein the infrared sensor blocking time ratio is calculated based on a first infrared sensor blocking duration and a second infrared sensor blocking duration; the first infrared sensor blocking duration represents a time duration during which a security article blocks an infrared sensor inside a security inspection instrument; and the second infrared sensor blocking duration represents a time duration during which the security article blocks an infrared sensor at an exit of the security inspection instrument.

[0008] An image acquisition module is configured to obtain first image information, second image information, and target X-ray image information, wherein the first image information is obtained by image acquisition of the security article inside the security inspection instrument; the second image information is obtained by image acquisition of the security article at the exit of the security inspection instrument; and the target X-ray image information is X-ray image information of the security article.

[0009] A first processing module is configured to input the second image information and the target X-ray image information into a first twin network to obtain a first similarity measurement value between the second image information and the target X-ray image information output by the first twin network.

[0010] a second processing module, configured to input the second image information and the first image information into a second twin network to obtain a second similarity measurement value between the second image information and the first image information output by the second twin network, wherein the first twin network and the second twin network are constructed based on a convolutional neural network;

[0011] a sorting module, configured to obtain a fusion discrimination coefficient according to the infrared sensor blocking time ratio, the first similarity measurement value and the second similarity measurement value, and sort the security inspection article based on the fusion discrimination coefficient.

[0012] According to the security inspection article automatic sorting system provided in the application, the sensor processing module comprises a first infrared sensor, a second infrared sensor, a blocking time length calculation unit and a blocking time ratio processing unit, wherein:

[0013] The first infrared sensor is arranged in the interior of the security inspection instrument, configured to generate a first infrared blocking start signal when the security inspection article blocks the first infrared sensor during the process of entering the security inspection instrument from the security inspection instrument entrance conveying belt, and generate a first infrared blocking end signal when the security inspection article no longer blocks the first infrared sensor;

[0014] The second infrared sensor is arranged at the security inspection instrument exit, configured to generate a second infrared blocking start signal when the security inspection article blocks the second infrared sensor during the process of leaving the security inspection instrument from the security inspection instrument exit conveying belt, and generate a second infrared blocking end signal when the security inspection article no longer blocks the second infrared sensor;

[0015] The blocking time length calculation unit is configured to calculate the first infrared sensor blocking duration according to the first infrared blocking end signal and the first infrared blocking start signal, and calculate the second infrared sensor blocking duration according to the second infrared blocking end signal and the second infrared blocking start signal;

[0016] The blocking time ratio processing unit is configured to calculate the infrared sensor blocking time ratio according to the ratio between the first infrared sensor blocking duration and the second infrared sensor blocking duration when it is determined that the first infrared sensor blocking duration is less than the second infrared sensor blocking duration, and calculate the infrared sensor blocking time ratio according to the ratio between the second infrared sensor blocking duration and the first infrared sensor blocking duration when it is determined that the first infrared sensor blocking duration is greater than or equal to the second infrared sensor blocking duration.

[0017] According to the security article automatic sorting system provided by the application, the image acquisition module comprises a first image acquisition unit, a second image acquisition unit and an X-ray generator, wherein:

[0018] The first image acquisition unit is arranged in the security instrument and is used for acquiring images of the security article to obtain the first image information when the first infrared shielding end signal is received.

[0019] The second image acquisition unit is arranged at the exit of the security instrument and is used for acquiring images of the security article to obtain the second image information when the second infrared shielding end signal is received.

[0020] The X-ray generator is arranged in the security instrument and is used for detecting the security article by X-ray after the security article enters the security instrument to obtain the target X-ray image information.

[0021] According to the security article automatic sorting system provided by the application, the system further comprises an image preprocessing module, wherein:

[0022] The image preprocessing module is used for performing first preprocessing on the first image information, the second image information and the target X-ray image information respectively to obtain first preprocessed first image information, first preprocessed second image information and first preprocessed target X-ray image information, wherein the first preprocessing comprises size adjustment processing and image padding processing.

[0023] According to the security article automatic sorting system provided by the application, the first twin network and the second twin network are constructed based on an improved ShuffleNet model, and a convolution block attention module is arranged in the improved ShuffleNet model, wherein:

[0024] The first twin network is obtained by training the improved ShuffleNet model through a second sample and sample X-ray image information corresponding to the second sample, wherein the second sample represents image information of the security article passing through the exit of the security instrument collected in a historical period.

[0025] The second twin network is obtained by training the improved ShuffleNet model through the second sample and a first sample corresponding to the second sample, wherein the first sample represents image information of the security article passing through the inside of the security instrument collected in a historical period.

[0026] According to the automatic sorting system for security articles provided by the application, the system further comprises an image data enhancement module and a training set construction module, wherein:

[0027] The image data enhancement module is configured to perform second preprocessing on the first sample, the sample X-ray image information and the second sample to obtain a second preprocessed first sample, second preprocessed sample X-ray image information and second preprocessed second sample; wherein the second preprocessing comprises image equal-ratio reduction processing, image padding processing, image color adjustment processing and image size processing.

[0028] According to the automatic sorting system for security articles provided by the application, the sorting module comprises a fusion discriminant coefficient calculation unit and a sorting discriminant unit, wherein:

[0029] The fusion discriminant coefficient calculation unit is configured to perform weighted summation processing on the infrared sensor blocking time ratio, the first similarity measurement value, the second similarity measurement value and a weight coefficient to obtain the fusion discriminant coefficient;

[0030] The sorting discriminant unit is configured to, when it is determined that the fusion discriminant coefficient is greater than or equal to a preset threshold value, execute a corresponding sorting operation on the security article based on a target sorting execution instruction; and when it is determined that the fusion discriminant coefficient is less than the preset threshold value, generate an artificial disposal prompt signal corresponding to the security article.

[0031] The application further provides an automatic sorting method for security articles, comprising:

[0032] An infrared sensor blocking time ratio is obtained, wherein the infrared sensor blocking time ratio is calculated based on a first infrared sensor blocking duration and a second infrared sensor blocking duration; the first infrared sensor blocking duration represents a duration for which a security article blocks an infrared sensor inside a security instrument; and the second infrared sensor blocking duration represents a duration for which the security article blocks an infrared sensor at an outlet of the security instrument.

[0033] First image information, second image information and target X-ray image information are obtained, wherein the first image information is obtained by image acquisition of the security article inside the security instrument; the second image information is obtained by image acquisition of the security article at the outlet of the security instrument; and the target X-ray image information is X-ray image information of the security article.

[0034] The second image information and the target X-ray image information are input into a first twin network to obtain a first similarity measurement value between the second image information and the target X-ray image information output by the first twin network.

[0035] inputting the second image information and the first image information into a second twin network to obtain a second similarity measurement value between the second image information and the first image information output by the second twin network, wherein the first twin network and the second twin network are constructed based on a convolutional neural network;

[0036] According to the shielding time ratio, the first similarity measurement value and the second similarity measurement value, a fusion discrimination coefficient is obtained, and the security inspection article is sorted based on the fusion discrimination coefficient.

[0037] The security inspection article automatic sorting system and method provided by the application obtain a shielding time ratio calculated based on the infrared sensor shielding time length inside and outside the security inspection instrument, and image information and target X-ray image information collected inside and outside the security inspection instrument; then the image information and the target X-ray image information are respectively input into two twin networks constructed based on a convolutional neural network to obtain similarity measurement values; finally, a fusion discrimination coefficient is obtained according to the similarity measurement values, so that the security inspection article is sorted according to the fusion discrimination coefficient, and the security inspection efficiency and quality are improved. BRIEF DESCRIPTION OF DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the application or the prior art, the drawings needed in the embodiments or the prior art description will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0039] Figure 1 It is a schematic diagram of the operation mode of the existing railway security inspection;

[0040] Figure 2 It is a structural schematic diagram of the security inspection article automatic sorting system provided by the application;

[0041] Figure 3 It is a schematic diagram of the overall architecture of the security inspection article automatic sorting system provided by the application;

[0042] Figure 4 It is a design schematic diagram of the twin network provided by the application;

[0043] Figure 5 It is a flowchart of the security inspection article automatic sorting method provided by the application;

[0044] Figure 6 It is a structural schematic diagram of the electronic device provided by the application. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below in combination with the drawings in the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0046] After decades of development, railway passenger security inspection has formed an operation mode mainly based on manual inspection and security inspection instrument machine inspection. Figure 1 The existing railway security operation mode is shown in the schematic diagram, which can be referred to as Figure 1 The main includes four posts of guide (not shown in Figure 1 ), check-in staff, hand inspector and disposal staff, and the division of labor is clear.

[0047] Guide: responsible for order maintenance in front of the security inspection instrument, to ensure smooth and orderly security inspection site.

[0048] Check-in staff: responsible for security inspection instrument check-in work, observing the image of the package passing through the security inspection instrument, and identifying the prohibited and restricted items or suspicious images.

[0049] Hand inspector: mainly for personal inspection of the inspected personnel, to find hidden prohibited and restricted items on the passenger's body in time.

[0050] Disposal staff: according to the notification of hand inspector and check-in staff, to check and confirm the suspected prohibited and restricted items.

[0051] In recent years, dual-view security inspection instruments and intelligent image judgment technologies have been gradually applied to railway passenger security inspection, and the technical means related to check-in image judgment have been continuously strengthened. However, the disposal staff as the key post of package opening inspection, their related technical means and equipment have long been neglected.

[0052] Each security inspection channel usually configures 1 disposal staff, whose work mainly includes two parts: one is to quickly and accurately find the disposal package according to the prompt of the check-in staff; the other is to open the package for inspection according to the prompt after finding the package, to confirm whether the package contains prohibited and restricted items.

[0053] The disposal staff needs to frequently take the package from the conveyor belt to the disposal table for opening inspection. In this process, there are many problems, which are embodied in the following aspects:

[0054] Task interference: if the disposal staff has not completed the previous opening inspection task, and there is a package to be opened and inspected in the middle, he needs to interrupt the current task and go to the conveyor belt to take the next package. This not only affects the disposal process of the current passenger and reduces the passenger experience, but also affects the security inspection quality.

[0055] Package prone to wrong taking: due to the limited time for the package to stay at the exit of the security instrument, the disposal staff has a tight time to find the package to be checked, and is prone to wrong taking of the package, resulting in confusion on site.

[0056] High work intensity: the disposal staff needs to continuously run between the conveying belt and the disposal table, and frequently bend down, which is high in work intensity and prone to mistakes in the busy work.

[0057] The existing security disposal operation has problems such as mutual interference of the tasks of finding the package and opening the package for inspection, package prone to wrong taking, and high work intensity of the disposal staff. Therefore, the package that needs to be opened for inspection and the package that does not need to be opened for inspection need to be automatically and accurately separated according to the judgment conclusion of the check-in staff, the step of finding the package by the disposal staff is saved, the problems of wrong taking and missing taking are avoided, the disposal staff is focused on the task of opening the package for inspection, the labor intensity is reduced, and the security and service quality are improved.

[0058] The automatic sorting technology is currently widely used in the field of civil aviation security and logistics. In the field of civil aviation security, the automatic sorting is mainly realized by using the baskets with RFID tags, and the passengers are required to put the packages to be checked in the baskets, and the automatic sorting is realized by using the RFID tags on the baskets. In the field of logistics, the automatic sorting is mainly realized by using the bar code labels on the packages, and the packages are positioned and sorted by automatically scanning the bar codes.

[0059] However, the railway station security has the characteristics of short time and large passenger flow, and if the above-mentioned methods are used, whether the passengers put the packages in the baskets or the packages are pasted with bar code labels, the work is too large, and it is not suitable for the railway station security scene. Therefore, it is necessary to develop a railway station security automatic sorting system that can sort the packages according to the information of the packages themselves.

[0060] In the current railway passenger security sorting process, when a large number of packages pass through the security instrument quickly, overlapping or shielding of the packages is prone to occur. When the check-in staff observes the X-ray images of these packages, due to the mutual interference of the packages, the images are complex and it is difficult to clearly and accurately distinguish the internal items of each package. In this case, the check-in staff can only observe and analyze the images by relying on their own experience, and then judge whether the further opening and inspection by the disposal staff is needed. However, the experience judgment has subjectivity and limitations, and there are differences in the experience level and judgment standard of different check-in staff, and even the same check-in staff may make mistakes in different states. This leads to the fact that some packages that should be opened for inspection may be missed, and some packages that do not need to be opened for inspection may be misjudged, which greatly increases the work intensity of the disposal staff, reduces the efficiency and accuracy of the security, and brings potential risks to the safety of railway passenger transportation.

[0061] Figure 2 The structure diagram of the automatic sorting system of the security articles provided by the present application is shown in the figure, Figure 2As shown, the present application provides an automatic sorting system for security inspection articles, comprising:

[0062] The sensor processing module 201 is configured to obtain an infrared sensor blocking time ratio, wherein the infrared sensor blocking time ratio is calculated based on a first infrared sensor blocking duration and a second infrared sensor blocking duration; the first infrared sensor blocking duration represents a duration that the security inspection article blocks the infrared sensor inside the security inspection instrument; and the second infrared sensor blocking duration represents a duration that the security inspection article blocks the infrared sensor at the outlet of the security inspection instrument.

[0063] The image acquisition module 202 is configured to obtain first image information, second image information and target X-ray image information, wherein the first image information is obtained by image acquisition of the security inspection article inside the security inspection instrument; the second image information is obtained by image acquisition of the security inspection article at the outlet of the security inspection instrument; and the target X-ray image information is X-ray image information of the security inspection article.

[0064] The first processing module 203 is configured to input the second image information and the target X-ray image information into a first twin network to obtain a first similarity measurement value between the second image information and the target X-ray image information output by the first twin network.

[0065] The second processing module 204 is configured to input the second image information and the first image information into a second twin network to obtain a second similarity measurement value between the second image information and the first image information output by the second twin network, wherein the first twin network and the second twin network are constructed based on a convolutional neural network.

[0066] The sorting module 205 is configured to obtain a fusion discrimination coefficient according to the infrared sensor blocking time ratio, the first similarity measurement value and the second similarity measurement value, and sort the security inspection article based on the fusion discrimination coefficient.

[0067] In the present application, the sensor processing module 201 during the operation of the security inspection instrument is equipped with two key infrared sensors, one of which is located inside the security inspection instrument, and the other of which is located at the outlet of the security inspection instrument (or can be located at the inlet of the sorting device connected to the outlet of the security inspection instrument).

[0068] When the security check object passes through the security check instrument, it will shield the two infrared sensors in turn. The system will record the time length of the security check object shielding the infrared sensor inside the security check instrument, that is, the first infrared sensor shielding duration; and the time length of shielding the infrared sensor at the outlet of the security check instrument, that is, the second infrared sensor shielding duration. Then, the infrared sensor shielding time ratio is calculated based on the two time length data, which can reflect the passing situation of the security check object in the security check instrument from the time dimension, providing an important basis for subsequent judgment.

[0069] In the present application, in order to comprehensively obtain the image features of the security check object, the image acquisition module 202 will perform image acquisition at different positions. The first image information is obtained when the security check object is in the interior of the security check instrument, which reflects the state and appearance features of the object in the interior of the security check instrument. The second image information is obtained when the security check object reaches the outlet of the security check instrument (if one of the image acquisition devices is arranged at the entrance of the sorting device connected to the outlet of the security check instrument, it is acquired at the entrance of the sorting device), at this time the object may have undergone a certain conveying process, and its state may be different from that in the interior.

[0070] In the present application, the target X-ray image information is obtained by scanning the security check object using X-ray technology, which can penetrate the surface of the object and show the structure and composition of the interior of the object, providing key information for accurate identification of the object.

[0071] Further, the second image information obtained by the above sensor processing module 201 and the target X-ray image information are input into the first twin network pre-constructed in the first processing module 203. In the present application, the first twin network is constructed based on a convolutional neural network, which has strong feature extraction and similarity comparison capabilities.

[0072] Inside the first twin network, the two input images are subjected to feature extraction and analysis, then the similarity between them is calculated, and the first similarity measurement value is output. The first similarity measurement value can measure the similarity between the appearance image and the X-ray image of the security check object at the outlet, which is helpful for judging whether the features of the object are consistent.

[0073] Similarly, the second image information and the first image information are input into the second twin network in the second processing module 204. The second twin network is also constructed based on a convolutional neural network, and its structure and function are similar to those of the first twin network. The present application processes and analyzes the two images through the second twin network, calculates the similarity between them, and outputs the second similarity measurement value. The second similarity measurement value reflects the appearance change of the security check object inside the security check instrument and at the outlet (or at the entrance of the sorting device), which can be used to judge whether the object has undergone abnormal changes in the conveying process.

[0074] Finally, the sorting module 205 calculates a fusion discriminant coefficient by a specific algorithm or model by synthesizing the infrared sensor blocking time ratio, the first similarity measure and the second similarity measure obtained in the above embodiments. In the present application, the fusion discriminant coefficient comprehensively considers various information of the security inspection article in the time dimension and the image feature dimension, and can more comprehensively and accurately reflect the attributes and state of the security inspection article. Finally, the sorting module 205 automatically sorts the security inspection article according to the size or range of the fusion discriminant coefficient in combination with the image judgment conclusion at the security inspection instrument, for example, separates the suspicious article and the normal article which need to be opened and inspected by the variable direction conveyor belt in the sorting device, so as to improve the efficiency and accuracy of the security inspection and ensure the quality of the security inspection work.

[0075] The security inspection article automatic sorting system provided by the present application obtains the blocking time ratio calculated based on the infrared sensor blocking time length in the security inspection instrument and at the outlet, and the image information and the target X-ray image information collected in the security inspection instrument and at the outlet; then the security inspection article automatic sorting system respectively inputs the above information into two twin networks based on convolutional neural network to obtain the similarity measure; finally, the security inspection article automatic sorting system obtains the fusion discriminant coefficient according to the similarity measure, so as to sort and inspect the security inspection article according to the fusion discriminant coefficient, thereby improving the efficiency and quality of the security inspection.

[0076] On the basis of the above embodiments, the sensor processing module comprises a first infrared sensor, a second infrared sensor, a blocking time length calculation unit and a blocking time ratio processing unit, wherein:

[0077] The first infrared sensor is arranged in the interior of the security inspection instrument, and is used to generate a first infrared blocking start signal when the security inspection article blocks the first infrared sensor while entering the security inspection instrument from the security inspection instrument entrance conveyor belt, and generate a first infrared blocking end signal when the security inspection article no longer blocks the first infrared sensor;

[0078] The second infrared sensor is arranged at the outlet of the security inspection instrument, and is used to generate a second infrared blocking start signal when the security inspection article blocks the second infrared sensor while leaving the security inspection instrument from the security inspection instrument outlet conveyor belt, and generate a second infrared blocking end signal when the security inspection article no longer blocks the second infrared sensor;

[0079] The blocking time length calculation unit is used to calculate the first infrared sensor blocking duration according to the first infrared blocking end signal and the first infrared blocking start signal, and calculate the second infrared sensor blocking duration according to the second infrared blocking end signal and the second infrared blocking start signal;

[0080] The occlusion time ratio processing unit is configured to: when it is determined that the first infrared sensor occlusion duration is less than the second infrared sensor occlusion duration, calculate the infrared sensor occlusion time ratio according to the ratio between the first infrared sensor occlusion duration and the second infrared sensor occlusion duration; and when it is determined that the first infrared sensor occlusion duration is greater than or equal to the second infrared sensor occlusion duration, calculate the infrared sensor occlusion time ratio according to the ratio between the second infrared sensor occlusion duration and the first infrared sensor occlusion duration.

[0081] In the present application, the sensor processing module is an important component of the security inspection system, which is mainly responsible for monitoring the conveying of the security inspection articles in the security inspection instrument by using the infrared sensor, and obtaining a parameter reflecting the conveying characteristics of the articles, i.e., the infrared sensor occlusion time ratio, through a series of calculations, to provide key data for subsequent analysis and processing of the security inspection articles.

[0082] The sensor processing module comprises four parts: a first infrared sensor, a second infrared sensor, an occlusion duration calculation unit, and an occlusion time ratio processing unit.

[0083] Specifically, the first infrared sensor is arranged inside the security inspection instrument. When the security inspection article enters the security inspection instrument from the entrance conveying belt, the article will gradually form an occlusion to the first infrared sensor as it moves. Once the occlusion occurs, the first infrared sensor will generate a first infrared occlusion start signal, which indicates that the security inspection article has started to enter the interior of the security inspection instrument and has an effect on the interior sensor. When the security inspection article continues to convey and no longer occludes the first infrared sensor, the sensor will generate a first infrared occlusion end signal, indicating that the security inspection article has left the occlusion range of the first infrared sensor, i.e., has completed the conveying process in the specific area inside the security inspection instrument. These two signals can accurately define the occlusion time period of the security inspection article to the first infrared sensor when conveying inside the security inspection instrument.

[0084] The second infrared sensor is arranged at the exit of the security inspection instrument (or the entrance of the sorting device). When the security inspection article is ready to leave the security inspection instrument from the exit conveying belt, it will occlude the second infrared sensor. At this time, the second infrared sensor will generate a second infrared occlusion start signal, indicating that the security inspection article has reached the exit of the security inspection instrument and has an effect on the exit sensor. When the security inspection article completely leaves and no longer occludes the second infrared sensor, the sensor will generate a second infrared occlusion end signal, indicating that the security inspection article has successfully passed through the exit of the security inspection instrument. These two signals are used to determine the occlusion time period of the security inspection article to the second infrared sensor at the exit of the security inspection instrument.

[0085] The shielding duration calculation unit is mainly used to calculate the shielding duration of the security article to the two sensors according to the correlation signals generated by the first infrared sensor and the second infrared sensor. For the first infrared sensor, after the shielding duration calculation unit receives the first infrared shielding end signal and the first infrared shielding start signal, the time difference between the two signals is calculated to obtain the shielding duration of the first infrared sensor, which reflects the time spent by the security article in the specific area inside the security instrument. Similarly, for the second infrared sensor, the shielding duration calculation unit calculates the time difference between the second infrared shielding end signal and the second infrared shielding start signal to obtain the shielding duration of the second infrared sensor, which reflects the time experienced by the security article in the outlet of the security instrument.

[0086] The shielding time ratio processing unit is responsible for further calculating the infrared sensor shielding time ratio according to the two shielding durations calculated in the above embodiment. This ratio can comprehensively reflect the transmission time relationship of the security article at different positions in the security instrument, and provide an important basis for judging the state and characteristics of the security article.

[0087] In the present application, when it is determined that the shielding duration of the first infrared sensor is less than the shielding duration of the second infrared sensor, it means that the transmission time of the security article inside the security instrument is relatively short, and the transmission time at the outlet is relatively long. At this time, the infrared sensor shielding time ratio is calculated according to the ratio between the shielding duration of the first infrared sensor and the shielding duration of the second infrared sensor.

[0088] When it is determined that the shielding duration of the first infrared sensor is greater than or equal to the shielding duration of the second infrared sensor, it means that the transmission time of the security article inside the security instrument is relatively long, or the transmission time at the outlet is relatively short or even equal. In this case, the infrared sensor shielding time ratio is calculated according to the ratio between the shielding duration of the second infrared sensor and the shielding duration of the first infrared sensor. Through this flexible calculation method, the characteristics of the security article in different transmission stages can be more accurately reflected, and the reliability and practicability of the data can be improved.

[0089] On the basis of the above embodiment, the image acquisition module comprises a first image acquisition unit, a second image acquisition unit and an X-ray generator, wherein:

[0090] The first image acquisition unit is arranged inside the security instrument and is used to acquire the image of the security article to obtain the first image information when the first infrared shielding end signal is received;

[0091] The second image acquisition unit is arranged at the exit of the security instrument, and is used for acquiring images of the security article to obtain the second image information when the second infrared shielding end signal is received.

[0092] The X-ray generator is arranged in the security instrument, and is used for detecting the security article by X-ray after the security article enters the security instrument to obtain the target X-ray image information.

[0093] In the present application, the image acquisition module is a key part of the security system for acquiring different types of image information of the security article, and is composed of the first image acquisition unit, the second image acquisition unit and the X-ray generator. The image acquisition module acquires images at a specific time of the security article transmission through signal linkage with the sensor processing module, so as to obtain image data that can comprehensively reflect the appearance and internal structure of the security article, and provide an important basis for subsequent security analysis and article sorting.

[0094] Specifically, the first image acquisition unit is arranged in the security instrument, and the first image acquisition unit can be a camera with automatic light compensation function. The first image acquisition unit is associated with the first infrared sensor in the sensor processing module. When the security article enters the security instrument from the entrance conveying belt, the first infrared sensor will be shielded, and when the article continues to convey and no longer shields the first infrared sensor, the first infrared sensor will generate a first infrared shielding end signal. The first image acquisition unit starts image acquisition operation immediately after receiving the first infrared shielding end signal.

[0095] At this time, the security article has completed the transmission in the specific area of the security instrument, and the first image acquisition unit acquires images of the security article to obtain the first image information, which can reflect the appearance state of the security article after the transmission in the security instrument, including the shape, color and surface characteristics of the article, etc., to provide basic data for subsequent analysis of whether the appearance of the article is normal.

[0096] The second image acquisition unit is arranged at the exit of the security instrument (or the entrance of the sorting device), and the second image acquisition unit can be a camera with automatic light compensation function. The second image acquisition unit cooperates with the second infrared sensor in the sensor processing module, and when the security article is ready to leave the security instrument from the exit conveying belt, the second infrared sensor will be shielded, and when the article completely leaves and no longer shields the second infrared sensor, the second infrared sensor will generate a second infrared shielding end signal. The second image acquisition unit starts image acquisition after receiving the second infrared shielding end signal.

[0097] The second image information records the appearance of the security check article when it is about to leave the security check instrument. By comparison with the first image information, whether the appearance of the security check article changes in the conveying process of the security check instrument can be analyzed, for example, whether the article falls, is damaged or is added with other objects, which helps to more accurately judge the safety of the security check article.

[0098] The X-ray generator is arranged inside the security check instrument. When the security check article enters the inside of the security check instrument, the X-ray generator starts to work, which does not need to rely on the end signal of the infrared sensor to trigger like the image acquisition unit, but is automatically started after the event of the article entering the inside of the security check instrument.

[0099] The X-ray generator emits X-rays to the security check article, and after the X-rays penetrate the article, different densities of substances have different absorption degrees of the X-rays, thereby forming different images on the detector. Through processing and analysis of these images, target X-ray image information is obtained. The target X-ray image information can show the internal structure of the security check article, for example, whether there is contraband in the article, the material distribution of the article and the like, which is an important basis for judging whether the article is safe in the security check process. By combination with the first image information and the second image information, the characteristics of the security check article can be more comprehensively and accurately understood, and the accuracy and reliability of the security check are improved.

[0100] Figure 3 The overall architecture schematic diagram of the security check article automatic sorting system provided by the present application can refer to Figure 3 In the present application, the core idea of the design of the security check automatic sorting system of the railway passenger station is to realize automatic sorting by analyzing the package itself directly, without relying on other media such as the storage basket and the bar code.

[0101] The present application combines the actual security check of the existing railway passenger station, designs the security check article automatic sorting system of the railway passenger station, and in Figure 3 the present application, mainly shows three subjects of the passenger, the check-in staff and the disposal staff, and shows eight modules of the low-speed conveying belt (newly added module), the security check instrument entrance conveying belt, the security check instrument, the security check instrument exit conveying belt, the variable direction conveying belt (newly added module), the package opening inspection package conveying belt (newly added module), the safe package conveying belt (newly added module) and the disposal table. At the same time, four devices of the infrared sensor 1 (i.e. the first infrared sensor, which is a newly added device), the infrared sensor 2 (i.e. the second infrared sensor, which is a newly added device), the automatic light supplement camera in the security check instrument (i.e. the first image acquisition unit, which is a newly added device) and the sorting device automatic light supplement camera (i.e. the second image acquisition unit, which is a newly added device) are shown.

[0102] In the present application, the sorting control software installed in the disposal table can receive the following information: the X-ray image of the parcel sent by the security instrument and the judgment conclusion of the X-ray image, the shielding signal of the infrared sensor 1 (including the start signal and the end signal), the start and stop signal of the conveying belt of the security instrument and the shielding signal of the infrared sensor 2 (including the start signal and the end signal).

[0103] Further, according to the above information received, the sorting control software can control the photographing of the automatic light supplement camera in the security instrument and the photographing of the automatic light supplement camera of the sorting device, and further control the conveying direction of the variable direction conveying belt according to the sorting logic. At the same time, the sorting control software can send the X-ray image and the visible light photo of all the parcels to the disposal table, which is convenient for the disposal personnel to work.

[0104] In an embodiment, the sorting logic process of the automatic sorting system of the security instrument provided by the present application can be described with reference to the following figure: Figure 3

[0105] Step S1, when the parcel shields the infrared sensor 1, the security instrument sends the parcel shielding infrared sensor 1 start signal to the sorting device, that is, the first infrared shielding start signal;

[0106] Step S2, when the parcel no longer shields the infrared sensor 1 after the parcel advances with the entrance conveying belt of the security instrument, the security instrument sends the parcel shielding infrared sensor 1 end signal to the sorting device, that is, the first infrared shielding end signal;

[0107] Step S3, when the sorting control software receives the parcel shielding infrared sensor 1 end signal, the automatic light supplement camera in the security instrument is controlled to take a photo of the parcel, and the parcel photo 1, that is, the first image information, is obtained;

[0108] Step S4, the sorting control software receives the X-ray image of the parcel sent by the security instrument, that is, the target X-ray image information;

[0109] Step S5, when the parcel shields the infrared sensor 2 while the parcel continues to be conveyed forward with the exit conveying belt of the security instrument, the sorting control software receives the parcel shielding infrared sensor 2 start signal, that is, the second infrared shielding start signal;

[0110] Step S6, when the parcel continues to be conveyed forward and no longer shields the infrared sensor 2, the sorting control software receives the parcel shielding infrared sensor 2 end signal, that is, the second infrared shielding end signal;

[0111] Step S7, when the sorting control software receives the parcel shielding infrared sensor 2 end signal, the automatic light supplement camera of the sorting device is controlled to take a photo of the parcel, and the parcel photo 2, that is, the second image information, is obtained;

[0112] Step S8, the sorting control software executes the comparison and judgment logic of the parcel to be sorted:​

[0113] For each package passing through the security inspection device, the following four-dimensional information related to the machine passing 4 is involved: package shielding infrared sensor 1 duration t1, package photo 1, X-ray image, and whether the package is opened for inspection. The judgment conclusion (which can be judged by the value clerk according to the X-ray image).

[0114] For each package passing through the sorting device, the following two-dimensional information related to the sorting 2 is involved: package shielding infrared sensor 2 duration t2 and package photo 2.

[0115] The essence of the sorting logic is to perform sorting operations according to the four-dimensional information of the machine passing 4 and the two-dimensional information of the sorting 2:

[0116] First, calculate the infrared sensor shielding time ratio k: according to the package shielding infrared sensor 1 start signal and the package shielding infrared sensor 1 end signal, the package shielding infrared sensor 1 duration t1 is obtained, that is, the first infrared sensor shielding duration; according to the package shielding infrared sensor 2 start signal and the package shielding infrared sensor 2 end signal, the package shielding infrared sensor 2 duration t2 is obtained, that is, the second infrared sensor shielding duration; if t1 < t2, then the infrared sensor shielding time ratio k = t1 / t2; otherwise, the infrared sensor shielding time ratio k = t2 / t1, where 0 < k ≤ 1.

[0117] Then, calculate the similarity between package photo 2 and X-ray image: construct twin network 1, that is, the first twin network; adjust the size and fill the package photo 2 and the X-ray image, then input the twin network 1, get the similarity measure value d1, that is, the first similarity measure value, 0 ≤ d1 ≤ 1, close to 1 means similar, close to 0 means different.

[0118] Further, calculate the similarity between package photo 2 and package photo 1: construct twin network 2, that is, the second twin network; adjust the size and fill the package photo 2 and the package photo 1, then input the twin network 2, get the similarity measure value d2, that is, the second similarity measure value, 0 ≤ d2 ≤ 1, close to 1 means similar, close to 0 means different.

[0119] Finally, different weights are given to k, d1 and d2 to obtain a fusion discrimination coefficient K, K = a x k + b x d1 + c x d2, wherein a, b and c are weights. If K is greater than or equal to a preset threshold thresh, the parcel at the sorting device corresponds to the current X-ray image, and sorting is performed according to the image discrimination conclusion, for example, if the image discrimination conclusion is to open the parcel for inspection, the current parcel is conveyed to the parcel conveying belt for opening the parcel for inspection by the variable direction conveying belt, and if the parcel does not need to be opened for inspection, the current parcel is conveyed to the safety parcel conveying belt by the variable direction conveying belt. If K is less than the preset threshold thresh, it indicates that the parcel at the sorting device does not correspond to the current X-ray image, at which time a corresponding artificial disposal prompt signal is generated and sent to the disposal station to prompt the artificial judgment.

[0120] On the basis of the above-mentioned embodiments, the system further comprises an image preprocessing module, wherein:

[0121] The image preprocessing module is configured to perform first preprocessing on the first image information, the second image information and the target X-ray image information respectively to obtain first preprocessed first image information, first preprocessed second image information and first preprocessed target X-ray image information, wherein the first preprocessing comprises size adjustment processing and image padding processing.

[0122] Since the first image information, the second image information and the target X-ray image information obtained from the image acquisition module may have inconsistent sizes and may have blank areas, these problems will directly affect the accuracy and efficiency of subsequent image analysis (such as similarity measurement value calculation). In the present application, the image preprocessing module is used to uniformly preliminarily process the three kinds of image information, eliminate the adverse factors in the images, and lay a foundation for more accurate subsequent image analysis and processing.

[0123] Specifically, the first image information reflects the appearance state of the security inspection article after being conveyed inside the security inspection instrument. However, due to factors such as parameter setting of the acquisition device and shooting angle, the size thereof may not meet the requirements of subsequent processing, and there may be some blank or irregular areas.

[0124] The second image information records the appearance of the security inspection article when it is about to leave the security inspection instrument. Similarly, this image may also have problems such as inappropriate size and blank areas.

[0125] The target X-ray image information may also have size differences and incomplete images (blank areas) due to device characteristics and article placement.

[0126] In the present application, the size adjustment processing is used to make the first image information, the second image information and the target X-ray image information have the same size specification. For example, bilinear interpolation, bicubic interpolation, etc. are used to realize the enlargement or reduction of the image size.

[0127] The blank areas of the three images are processed by constant filling, edge filling and mirror filling, etc., so that the images reach the regular state.

[0128] After the processing of the image preprocessing module, the preprocessed image has a uniform size, and the blank area is reasonably filled, which is more in line with the input requirements of the subsequent image analysis algorithm, and can improve the accuracy and reliability of the entire security system for analyzing and judging security articles.

[0129] On the basis of the above embodiment, the first twin network and the second twin network are constructed based on an improved ShuffleNet model, and a convolution block attention module is arranged in the improved ShuffleNet model, wherein:

[0130] The first twin network is obtained by training the improved ShuffleNet model through a second sample and sample X-ray image information corresponding to the second sample, wherein the second sample represents image information of the security article collected at the exit of the security instrument in a historical period.

[0131] The second twin network is obtained by training the improved ShuffleNet model through the second sample and a first sample corresponding to the second sample, wherein the first sample represents image information of the security article collected inside the security instrument in a historical period.

[0132] In the present application, in order to realize more accurate analysis and judgment of security articles, the first twin network and the second twin network constructed based on the improved ShuffleNet model are adopted. ShuffleNet is a lightweight convolutional neural network model, which has the characteristics of small calculation amount and high efficiency, and is suitable for running on devices with limited resources. In the present application, the improved ShuffleNet model is optimized on this basis, and a convolution block attention module (CBAM) is particularly introduced, which can enhance the attention ability of the model to important features in the image, and improve the extraction and recognition effect of the model to image features.

[0133] Figure 4 The design schematic of the twin network provided in the present application can refer to Figure 4As shown, in the present application, the twin network 1 and the twin network 2 adopt similar structures and are obtained by training different data, wherein the twin network 1 is obtained by training a pair of parcel photo 2 (sample data) and X-ray image (sample data), and the twin network 2 is obtained by training a pair of parcel photo images (including parcel photo 1 and parcel photo 2, both of which are sample data).

[0134] Taking the twin network 1 as an example for description:

[0135] In the training process of the twin network 1, the parcel photo 2 and the X-ray image are taken as two inputs. First, a 3x3 convolution operation is performed based on the convolution layer Conv, and the stride is 2. Then, the maximum pooling (MaxPool) is performed, and the stride is 2. Further, the features are processed through the stage layers Stage2, Stage3 and Stage4, and then the CBAM is used to enhance the expression ability of the features.

[0136] After the features are enhanced by the CBAM, a 1x1 convolution operation is performed to further adjust the channel number of the feature map, and then the global pooling (GlobalPool) is used to convert the feature map into a fixed-length feature vector. Finally, the feature vector is mapped to 1024 dimensions through the full connection layer (FC), and then mapped to 128 dimensions, and the feature vectors of the two branches are combined to perform prediction (Pred).

[0137] By introducing the CBAM, the improved ShuffleNet model can more accurately capture the key features in the image, and improve the performance and accuracy of the model.

[0138] On the basis of the above embodiment, the system further comprises an image data enhancement module and a training set construction module, wherein:

[0139] The image data enhancement module is configured to perform second preprocessing on the first sample, the sample X-ray image information and the second sample to obtain a second preprocessed first sample, a second preprocessed sample X-ray image information and a second preprocessed second sample, wherein the second preprocessing includes image equal-ratio reduction processing, image padding processing, image color adjustment processing and image size processing.

[0140] In the present application, the image data enhancement module can improve data utilization and network learning ability. In the package map matching scenario, the original first sample, sample X-ray image information and second sample may have problems such as limited data volume and single feature, which can easily cause overfitting phenomenon of the network in the training process, and the network cannot learn the key features in the data well, thereby affecting the accuracy of package map matching. The image data enhancement module generates more diversified and representative image data by performing a series of second preprocessing operations on the original images, provides rich and high-quality data for the subsequent training set construction module, and helps to improve the network's learning ability of key features in the data and enhance the generalization performance of the model.

[0141] Specifically, the second preprocessing includes the following processes:

[0142] Image equal ratio reduction processing: The original photo size is usually 1920x1080 pixels, while the security image size is smaller than this specification. Moreover, the detailed information of the image has relatively small influence on network learning. By reducing the image in equal ratio, the data volume in subsequent network operation can be effectively reduced, the consumption of computing resources can be reduced, and the operation efficiency can be improved, without causing substantial influence on the key package map matching features (such as package outline).

[0143] In the present application, equal ratio reduction operation is performed on all input first samples, sample X-ray image information and second samples, and they are reduced to 1 / 2 of the original size. For example, after processing, the size of the originally large image becomes 960x540 pixels, which not only retains the main features of the image, but also reduces the computational burden.

[0144] Image padding processing (horizontal alignment and vertical random padding):

[0145] In the matching process of package photos and X-ray images, the similarity of package outlines in the two images is mainly relied on. In order to enable images of different sizes to be uniformly processed and analyzed in the network, normalization processing is needed. The padding method can fill the image to a uniform size without changing the original image proportion, while avoiding distortion of the outline information, and can synchronize the horizontal position information, increase the random amount in the vertical direction, enrich the data volume, and improve the network's recognition ability of packages in different positions.

[0146] In the present application, all photos are processed by equal reduction, and the size is 960x540 pixels. At this time, only the sample X-ray image information needs to be padded to 960x540 pixels. Since the photos are taken by a sensor trigger mechanism, the horizontal position wrapped in the photo is basically on the left side, but the vertical position is uncertain. Therefore, when padding, the X-ray image is placed on the left side in the horizontal direction, and the vertical position is determined randomly. In this way, the integrity of the image contour information is ensured, and the diversity of the data in the vertical position is increased.

[0147] Image color adjustment processing (random color adjustment):

[0148] Since the model mainly focuses on the contour information of the package, the color information of the package has little effect on the matching result. By randomly adjusting the color of the image, the present application can generate image data with different color characteristics, increase the diversity of the data, and enable the network to learn more general features instead of relying only on specific color information, thereby improving the generalization ability of the data and reducing the risk of overfitting.

[0149] In the present application, the brightness, contrast, saturation, and hue of the first sample, sample X-ray image information, and second sample are randomly adjusted. For example, randomly change the brightness value of the image to make the image look brighter or darker; adjust the contrast to enhance or weaken the difference between light and dark areas in the image; modify the saturation to make the colors of the image more vibrant or more dull; change the hue to make the image exhibit different color tendencies. By these random adjustments, a large number of image data with different color characteristics are generated.

[0150] Image size processing (random size adjustment):

[0151] In order to further improve the diversity of the data and avoid overfitting phenomenon in the training process of the network, the size of the image needs to be randomly adjusted. Through small-scale proportional scaling, the network can be exposed to package images of different sizes, learn the features of the package at different scales, enhance the recognition and matching ability of the network for packages of different sizes, and improve the robustness and generalization performance of the model.

[0152] In the present application, only the sample X-ray image information is randomly adjusted in size. A random scale between 0.97 and 1.03 is selected, and the X-ray image is scaled proportionally. For example, the image may be reduced to 0.98 times the original size, or it may be enlarged to 1.02 times the original size. This small-scale random scaling neither changes the features of the image too much nor effectively increases the diversity of the data.

[0153] After preprocessing by the image data enhancement module, the obtained preprocessed image data has more rich features and diversity in terms of size, color, position, etc., which can provide high-quality data support for the subsequent training set construction module, and help to train a package image matching model with better performance and stronger generalization ability.

[0154] On the basis of the above embodiment, the sorting module comprises a fusion discriminant coefficient calculation unit and a sorting discriminant unit, wherein:

[0155] The fusion discriminant coefficient calculation unit is configured to perform weighted sum processing on the infrared sensor blocking time ratio, the first similarity measure value, the second similarity measure value and a weight coefficient to obtain the fusion discriminant coefficient.

[0156] The sorting discriminant unit is configured to, when it is determined that the fusion discriminant coefficient is greater than or equal to a preset threshold value, execute a corresponding sorting operation on the security article based on a target sorting execution instruction; and when it is determined that the fusion discriminant coefficient is less than the preset threshold value, generate an artificial disposal prompt signal corresponding to the security article.

[0157] In the present application, the sorting module is used to determine whether the package at the sorting device corresponds to the current X-ray image, and accordingly decides the important task of subsequent operation (automatic sorting or artificial disposal). By fusing multiple information and performing comprehensive analysis, the sorting module can improve the accuracy and reliability of sorting, and ensure the efficient operation of the security process.

[0158] In the present application, the fusion discriminant coefficient calculation unit needs to receive three key input information, which are the infrared sensor blocking time ratio, the first similarity measure value and the second similarity measure value. Among them, the infrared sensor blocking time ratio reflects the relative relationship between the passing time of the package at the security instrument and the sorting device, and to some extent can reflect the motion state and characteristics of the package.

[0159] The first similarity measure value can be used to determine whether the appearance of the package at the sorting device has a high matching degree with the internal structure reflected by the current X-ray image.

[0160] The second similarity measure value helps to further verify the consistency of the package from the perspective of the appearance of the package.

[0161] The weight coefficients (a, b, c) are pre-set parameters used to adjust the importance of the infrared sensor blocking time ratio, the first similarity measure value and the second similarity measure value in the calculation of the fusion discriminant coefficient. Different weight distribution can be optimized according to actual demand and system characteristics, in order to highlight the influence of certain key information on the sorting decision.

[0162] Further, the fusion discriminant coefficient calculation unit performs weighted sum processing on the above three input information and the corresponding weight coefficients, comprehensively considers the time characteristics of the package in the security inspection process, the similarity of the package appearance and the X-ray image, and the similarity of the package appearance at different positions, and obtains a fusion discriminant coefficient K.

[0163] When the sorting discriminant unit determines that the fusion discriminant coefficient K is greater than or equal to the preset threshold thresh, it indicates that the package at the sorting device and the current X-ray image have high correspondence. In this case, the sorting discriminant unit will execute the corresponding sorting operation on the security inspection object based on the target sorting execution instruction. The target sorting execution instruction can be preset according to the image judgment conclusion (such as whether to open the package for inspection, etc.), and the sorting operation can be sorting the package to different channels or areas to complete the subsequent processing flow.

[0164] If the sorting discriminant unit determines that the fusion discriminant coefficient K is less than the preset threshold thresh, it indicates that the correspondence between the package at the sorting device and the current X-ray image is low. At this time, the sorting discriminant unit will generate an artificial handling prompt signal corresponding to the security inspection object, and send the signal to the handling station. After the handling station receives the signal, it will prompt the staff to make a manual judgment, and further confirm the condition of the package by manual, and make corresponding processing decision, to ensure the accuracy and safety of security inspection.

[0165] Figure 5 The flowchart of the automatic sorting method of the security inspection object provided by the present application is shown in FIG. 1. Figure 5 The present application provides an automatic sorting method of security inspection object, which comprises the following steps:

[0166] In step 501, the infrared sensor blocking time ratio is obtained, wherein the infrared sensor blocking time ratio is calculated based on the first infrared sensor blocking duration and the second infrared sensor blocking duration; the first infrared sensor blocking duration represents the duration that the security inspection object blocks the infrared sensor inside the security inspection instrument; and the second infrared sensor blocking duration represents the duration that the security inspection object blocks the infrared sensor at the outlet of the security inspection instrument.

[0167] In the present application, two key infrared sensors are provided during the operation of the security inspection instrument, one of which is located inside the security inspection instrument, and the other is located at the outlet of the security inspection instrument (or can be located at the inlet of the sorting device connected to the outlet of the security inspection instrument).

[0168] When the security article passes through the security instrument, it will shield the two infrared sensors in turn. The system will record the time length of the security article shielding the internal infrared sensor of the security instrument, that is, the first infrared sensor shielding duration; and the time length of shielding the infrared sensor at the outlet of the security instrument, that is, the second infrared sensor shielding duration. Then, the infrared sensor shielding time ratio is calculated based on the two time length data, which can reflect the passing situation of the security article in the security instrument from the time dimension, providing an important basis for subsequent judgment.

[0169] In step 502, first image information, second image information and target X-ray image information are obtained, wherein the first image information is obtained by image acquisition of the security article inside the security instrument; the second image information is obtained by image acquisition of the security article at the outlet of the security instrument; and the target X-ray image information is X-ray image information of the security article.

[0170] In the present application, in order to comprehensively obtain the image features of the security article, the system will perform image acquisition at different positions. The first image information is obtained when the security article is inside the security instrument, which reflects the state and appearance features of the article inside the security instrument. The second image information is obtained when the security article reaches the outlet of the security instrument (for example, one of the image acquisition devices is arranged at the entrance of the sorting device connected to the outlet of the security instrument, and the image acquisition is performed at the entrance of the sorting device), at which time the article may have undergone a certain conveying process, and its state may be different from that inside.

[0171] In the present application, the target X-ray image information is obtained by scanning the security article using X-ray technology, which can penetrate the surface of the article and display the structure and composition inside the article, providing key information for accurate identification of the article.

[0172] In step 503, the second image information and the target X-ray image information are input into the first twin network to obtain a first similarity measurement value between the second image information and the target X-ray image information output by the first twin network.

[0173] Further, the second image information and the target X-ray image information obtained in the above embodiment are input into the first twin network constructed in advance. In the present application, the first twin network is constructed based on a convolutional neural network, which has strong feature extraction and similarity comparison capability.

[0174] Inside the first twin network, the two input images are subjected to feature extraction and analysis, and then the similarity between them is calculated, and a first similarity measurement value is output. The first similarity measurement value can measure the similarity between the appearance image and the X-ray image of the security article at the outlet, which is helpful for judging whether the features of the article are consistent.

[0175] Step 504, input the second image information and the first image information into the second twin network to obtain a second similarity measurement value between the second image information and the first image information output by the second twin network, wherein the first twin network and the second twin network are constructed based on a convolutional neural network.

[0176] Similarly, the second image information and the first image information are input into the second twin network. The second twin network is also constructed based on a convolutional neural network, and its structure and function are similar to those of the first twin network. The present application processes and analyzes the two images through the second twin network, calculates the similarity between them, and outputs a second similarity measurement value. The second similarity measurement value reflects the appearance change of the security inspection article inside the security inspection instrument and at the exit (or at the entrance of the sorting device), which can be used to judge whether the article has undergone abnormal changes in the conveying process.

[0177] Step 505, according to the infrared sensor blocking time ratio, the first similarity measurement value and the second similarity measurement value, a fusion discrimination coefficient is obtained, and the security inspection article is sorted based on the fusion discrimination coefficient.

[0178] Finally, the infrared sensor blocking time ratio, the first similarity measurement value and the second similarity measurement value obtained in the above embodiments are integrated, and a fusion discrimination coefficient is calculated through a specific algorithm or model. In the present application, the fusion discrimination coefficient comprehensively considers various information of the security inspection article in the time dimension and the image feature dimension, and can more comprehensively and accurately reflect the properties and state of the security inspection article. Finally, according to the size or range of the fusion discrimination coefficient, combined with the image judgment conclusion at the security inspection instrument, the system will automatically sort the security inspection article, for example, through the variable direction conveyor belt in the sorting device, to separate the suspicious articles that need to be opened and checked from the normal articles, thereby improving the efficiency and accuracy of security inspection and ensuring the quality of safety inspection work.

[0179] The security inspection article automatic sorting method provided by the present application obtains the blocking time ratio calculated based on the infrared sensor blocking time inside and at the exit of the security inspection instrument, and the image information and target X-ray image information collected inside and at the exit of the security inspection instrument; then the above information is input into two twin networks constructed based on a convolutional neural network respectively to obtain similarity measurement values; finally, a fusion discrimination coefficient is obtained according to the similarity measurement values, so that the security inspection article is sorted according to the fusion discrimination coefficient, thereby improving the efficiency and quality of security inspection.

[0180] Figure 6 The structure schematic diagram of the electronic device provided by the present application is as follows, Figure 6As shown, the electronic device can include a processor 601, a communications interface 602, a memory 603, and a communications bus 604, wherein the processor 601, the communications interface 602, and the memory 603 complete mutual communication through the communications bus 604. The processor 601 can invoke a logical instruction in the memory 603 to execute the security article automatic sorting method, which includes: obtaining an infrared sensor shielding time ratio, wherein the infrared sensor shielding time ratio is calculated based on a first infrared sensor shielding duration and a second infrared sensor shielding duration; the first infrared sensor shielding duration represents a duration that a security article shields an infrared sensor inside a security instrument; the second infrared sensor shielding duration represents a duration that the security article shields an infrared sensor at an exit of the security instrument; obtaining first image information, second image information, and target X-ray image information, wherein the first image information is obtained by image collection of the security article inside the security instrument; the second image information is obtained by image collection of the security article at the exit of the security instrument; and the target X-ray image information is X-ray image information of the security article; inputting the second image information and the target X-ray image information into a first twin network to obtain a first similarity measurement value between the second image information and the target X-ray image information output by the first twin network; inputting the second image information and the first image information into a second twin network to obtain a second similarity measurement value between the second image information and the first image information output by the second twin network, wherein the first twin network and the second twin network are constructed based on a convolutional neural network; obtaining a fusion discrimination coefficient according to the infrared sensor shielding time ratio, the first similarity measurement value, and the second similarity measurement value, and sorting the security article based on the fusion discrimination coefficient.

[0181] In addition, the logic instructions in the memory 603 described above can be implemented in the form of software functional units and sold or used as independent products, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a plurality of instructions for making a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0182] In another aspect, the present application also provides a computer program product, which comprises a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, enable the computer to perform the automatic sorting method of security articles provided by the above-mentioned methods, the method comprising: obtaining an infrared sensor blocking time ratio, wherein the infrared sensor blocking time ratio is calculated based on a first infrared sensor blocking duration and a second infrared sensor blocking duration; the first infrared sensor blocking duration represents the duration of the security article blocking the infrared sensor inside the security instrument; the second infrared sensor blocking duration represents the duration of the security article blocking the infrared sensor at the outlet of the security instrument; obtaining first image information, second image information and target X-ray image information, wherein the first image information is obtained by image acquisition of the security article inside the security instrument; the second image information is obtained by image acquisition of the security article at the outlet of the security instrument; the target X-ray image information is the X-ray image information of the security article; inputting the second image information and the target X-ray image information into a first twin network to obtain a first similarity measure value between the second image information and the target X-ray image information output by the first twin network; inputting the second image information and the first image information into a second twin network to obtain a second similarity measure value between the second image information and the first image information output by the second twin network, wherein the first twin network and the second twin network are constructed based on a convolutional neural network; obtaining a fusion discrimination coefficient according to the infrared sensor blocking time ratio, the first similarity measure value and the second similarity measure value, and sorting the security article based on the fusion discrimination coefficient.

[0183] In yet another aspect, the present application also provides a non-transitory computer readable storage medium having stored thereon a computer program, which, when executed by a processor, implements the method for automatically sorting security articles provided by the above embodiments, the method comprising: obtaining an infrared sensor blocking time ratio, wherein the infrared sensor blocking time ratio is calculated based on a first infrared sensor blocking duration and a second infrared sensor blocking duration; the first infrared sensor blocking duration represents a duration that a security article blocks an infrared sensor inside a security instrument; the second infrared sensor blocking duration represents a duration that the security article blocks an infrared sensor at an exit of the security instrument; obtaining first image information, second image information, and target X-ray image information, wherein the first image information is obtained by image acquisition of the security article inside the security instrument; the second image information is obtained by image acquisition of the security article at the exit of the security instrument; and the target X-ray image information is X-ray image information of the security article; inputting the second image information and the target X-ray image information into a first twin network to obtain a first similarity measurement value between the second image information and the target X-ray image information output by the first twin network; inputting the second image information and the first image information into a second twin network to obtain a second similarity measurement value between the second image information and the first image information output by the second twin network, wherein the first twin network and the second twin network are constructed based on a convolutional neural network; obtaining a fusion discrimination coefficient according to the infrared sensor blocking time ratio, the first similarity measurement value, and the second similarity measurement value, and sorting the security article based on the fusion discrimination coefficient.

[0184] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., they can be located in one place, or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the present embodiment according to actual needs. Those skilled in the art can understand and implement it without creative labor.

[0185] Those skilled in the art can clearly understand the technical solutions of the various embodiments from the above description of the embodiments, and the various embodiments can be implemented by means of software with the necessary general hardware platforms, and of course, can also be implemented by hardware. Based on such understanding, the above technical solutions, essentially or in other words, the part of the prior art that makes a contribution, can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, and the like, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0186] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for some technical features therein; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An automatic security article sorting system, characterized by, The method comprises the following steps: a sensor processing module is configured to obtain an infrared sensor blocking time ratio, wherein the infrared sensor blocking time ratio is calculated based on a first infrared sensor blocking duration and a second infrared sensor blocking duration; the first infrared sensor blocking duration represents a time duration during which an inspection article blocks an infrared sensor inside a security inspection instrument; the second infrared sensor blocking duration represents a time duration during which the inspection article blocks an infrared sensor at an exit of the security inspection instrument; the infrared sensor blocking time ratio can comprehensively reflect the conveying time relationship of the inspection article at different positions in the security inspection instrument, and is used to determine the state and characteristics of the inspection article; an image acquisition module is configured to obtain first image information, second image information, and target X-ray image information, wherein the first image information is obtained by image acquisition of the inspection article inside the security inspection instrument; the second image information is obtained by image acquisition of the inspection article at the exit of the security inspection instrument; and the target X-ray image information is X-ray image information of the inspection article; a first processing module is configured to input the second image information and the target X-ray image information into a first twin network to obtain a first similarity measurement value between the second image information and the target X-ray image information output by the first twin network; a second processing module is configured to input the second image information and the first image information into a second twin network to obtain a second similarity measurement value between the second image information and the first image information output by the second twin network, wherein the first twin network and the second twin network are constructed based on a convolutional neural network; a sorting module is configured to obtain a fusion discrimination coefficient based on the infrared sensor blocking time ratio, the first similarity measurement value, and the second similarity measurement value, and sort the inspection article based on the fusion discrimination coefficient.

2. The automatic security item sorting system of claim 1, wherein, The sensor processing module comprises a first infrared sensor, a second infrared sensor, a blocking duration calculation unit, and a blocking time ratio processing unit, wherein: the first infrared sensor is arranged inside the security inspection instrument, and is configured to generate a first infrared blocking start signal when the inspection article blocks the first infrared sensor as the inspection article enters the security inspection instrument from a security inspection instrument entry conveyor belt, and generate a first infrared blocking end signal when the inspection article no longer blocks the first infrared sensor; the second infrared sensor is arranged at the exit of the security inspection instrument, and is configured to generate a second infrared blocking start signal when the inspection article blocks the second infrared sensor as the inspection article exits the security inspection instrument from a security inspection instrument exit conveyor belt, and generate a second infrared blocking end signal when the inspection article no longer blocks the second infrared sensor; The occlusion duration calculation unit is configured to calculate the first infrared sensor occlusion duration according to the first infrared occlusion end signal and the first infrared occlusion start signal, and calculate the second infrared sensor occlusion duration according to the second infrared occlusion end signal and the second infrared occlusion start signal. The occlusion time ratio processing unit is configured to calculate the infrared sensor occlusion time ratio according to a ratio between the first infrared sensor occlusion duration and the second infrared sensor occlusion duration when it is determined that the first infrared sensor occlusion duration is less than the second infrared sensor occlusion duration, and calculate the infrared sensor occlusion time ratio according to a ratio between the second infrared sensor occlusion duration and the first infrared sensor occlusion duration when it is determined that the first infrared sensor occlusion duration is greater than or equal to the second infrared sensor occlusion duration.

3. The automatic security item sorting system of claim 2, wherein, The image acquisition module comprises a first image acquisition unit, a second image acquisition unit and an X-ray generator, wherein: The first image acquisition unit is arranged inside the security instrument and is configured to acquire an image of the security article to obtain the first image information when the first infrared occlusion end signal is received. The second image acquisition unit is arranged at the exit of the security instrument and is configured to acquire an image of the security article to obtain the second image information when the second infrared occlusion end signal is received. The X-ray generator is arranged inside the security instrument and is configured to perform X-ray detection on the security article to obtain the target X-ray image information after the security article enters the security instrument.

4. The automatic security item sorting system of claim 1, wherein, The system further comprises an image preprocessing module, wherein: The image preprocessing module is configured to perform first preprocessing on the first image information, the second image information and the target X-ray image information respectively to obtain first preprocessed first image information, first preprocessed second image information and first preprocessed target X-ray image information, wherein the first preprocessing comprises size adjustment processing and image padding processing.

5. The automatic security item sorting system of claim 1, wherein, The first twin network and the second twin network are constructed based on an improved ShuffleNet model, and a convolution block attention module is arranged in the improved ShuffleNet model, wherein: The first twin network is obtained by training the improved ShuffleNet model based on a second sample and sample X-ray image information corresponding to the second sample, wherein the second sample represents image information of the security article passing through the exit of the security instrument collected in a historical period; The second twin network is obtained by training the improved ShuffleNet model based on the second sample and a first sample corresponding to the second sample, wherein the first sample represents image information of the security article passing through the inside of the security instrument collected in a historical period.

6. The automatic security item sorting system of claim 5, wherein, The system further comprises an image data enhancement module and a training set construction module, wherein: The image data enhancement module is configured to perform second preprocessing on the first sample, the sample X-ray image information, and the second sample to obtain a second preprocessed first sample, second preprocessed sample X-ray image information, and second preprocessed second sample. The second preprocessing includes image equal-ratio reduction processing, image padding processing, image color adjustment processing, and image size processing.

7. The automatic security item sorting system of claim 1, wherein, The sorting module includes a fusion discriminant coefficient calculation unit and a sorting discriminant unit, wherein: The fusion discriminant coefficient calculation unit is configured to perform weighted summation processing on the infrared sensor blocking time ratio, the first similarity measure value, the second similarity measure value, and a weight coefficient to obtain the fusion discriminant coefficient. The sorting discriminant unit is configured to, when determining that the fusion discriminant coefficient is greater than or equal to a preset threshold value, perform a corresponding sorting operation on the security article based on a target sorting execution instruction; and when determining that the fusion discriminant coefficient is less than the preset threshold value, generate an artificial handling prompt signal corresponding to the security article.

8. An automatic sorting method of security articles, characterized by, It includes: An infrared sensor blocking time ratio is obtained, wherein the infrared sensor blocking time ratio is calculated based on a first infrared sensor blocking duration and a second infrared sensor blocking duration. The first infrared sensor blocking duration represents the duration for which a security article blocks an infrared sensor inside a security instrument. The second infrared sensor blocking duration represents the duration for which the security article blocks an infrared sensor at the outlet of the security instrument. The infrared sensor blocking time ratio can comprehensively reflect the conveying time relationship of a security article at different positions in a security instrument, and is used to determine the state and characteristics of the security article. First image information, second image information, and target X-ray image information are obtained, wherein the first image information is obtained by image acquisition of the security article inside the security instrument. The second image information is obtained by image acquisition of the security article at the outlet of the security instrument. The target X-ray image information is X-ray image information of the security article. The second image information and the target X-ray image information are input into a first twin network to obtain a first similarity measure value between the second image information and the target X-ray image information output by the first twin network. The second image information and the first image information are input into a second twin network to obtain a second similarity measure value between the second image information and the first image information output by the second twin network, wherein the first twin network and the second twin network are constructed based on a convolutional neural network. According to the infrared sensor blocking time ratio, the first similarity measure value, and the second similarity measure value, a fusion discriminant coefficient is obtained, and the security article is sorted based on the fusion discriminant coefficient.

Images