Key point detection method, device and storage medium for X-ray video stream

Abstract

The present invention relates to the field of image processing technology, and more specifically to a key point detection method, device and storage medium for X-ray video streams, the method comprising: when a bag is sent into a security inspection device, a control unit starts an X-ray transmitting device to transmit an X-ray beam of a first energy to penetrate the interior of the bag, an X-ray array receiving device receives an X-ray beam of residual energy to generate a first video stream, and obtains a first bag image, obtains multiple items and item shapes in the first bag image, and also obtains a first residual energy, calculates a first attenuation coefficient of the first item after being irradiated by the X-ray beam of the first energy; by creating and training a material classification model, a second classification result of the first item is obtained; the control unit finally determines whether the first bag image contains dangerous items, and issues a prompt in time when the first bag image contains dangerous items. The present invention can improve the accuracy and efficiency of key point detection.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and more specifically to a key point detection method, device and storage medium for X-ray video stream. Background Art

[0002] In the field of security inspection, X-ray imaging technology is widely used to detect potentially dangerous items in luggage, bags, etc. Traditional X-ray image analysis relies on manual inspection, which is inefficient and susceptible to operator fatigue and subjective judgment.

[0003] Similar prior art includes Chinese patent application with publication number CN118429660A, which discloses a key point detection method, including obtaining an image to be detected, wherein the image to be detected contains an object to be detected; inputting the image to be detected into a key point detection model to obtain image key points of the object to be detected, wherein the key point detection model predicts a distance matrix between reference key points and image key points, and determines image key points based on the distance matrix. By inputting the image to be detected into the key point detection model, the key point detection model predicts a distance matrix between reference key points and image key points, thereby determining image key points based on the distance matrix. A similar prior art also includes a Chinese patent application with publication number CN116778522A, which discloses a key point detection method and a key point detection device, wherein the key point detection method includes: performing initial processing on the input image sequence to obtain an initial multi-frame model heat map, outputting it as an iterative multi-frame model heat map to subsequent iterative processing, performing one or more iterative processing on the image sequence until each frame in the image sequence has been processed, in each iterative processing: obtaining a current frame heat map and a neighboring frame heat map based on the current frame and neighboring frames to be processed in the image sequence, respectively; post-processing the iterative multi-frame model heat map to obtain a post-processed iterative multi-frame model heat map; obtaining a multi-frame model heat map including the corrected key points of the current frame based on the post-processed iterative multi-frame model heat map, the current frame heat map and the neighboring frame heat map; and using the obtained multi-frame model heat map to update the iterative multi-frame model heat map and output it to the next iterative processing. Although the above two inventions can realize accurate detection of key points on an image, they do not take into account that in a security inspection environment, not only the accuracy of key point detection is required, but also the efficiency of detection needs to be improved.

[0004] Therefore, the present invention proposes a key point detection method, device and storage medium for X-ray video stream to improve the accuracy and efficiency of key point detection. Summary of the invention

[0005] In order to better solve the above problems, the present invention provides a key point detection method for an X-ray video stream, wherein a rotatable X-ray transmitting device is installed at a middle position near the top on the upper left side of the inner side of the security inspection device, and an X-ray array receiving device is installed at a position near the bottom on the lower right side of the inner side of the security inspection device, wherein the transmitting direction of the X-ray transmitting device is relative to the receiving direction of the X-ray array receiving device, and the method is implemented by executing the following steps:

[0006] Step S1: When a bag is sent into the security inspection device, the control unit starts the X-ray emitting device, and controls the X-ray emitting device to emit an X-ray beam set to a first energy within a preset time to penetrate the interior of the bag, the X-ray array receiving device receives the X-ray beam with residual energy after penetrating the bag, and generates a first video stream based on the X-ray beam with residual energy, processes each frame of the first video stream, and uses image recognition technology to obtain a first bag image from the first video stream;

[0007] Step S2: based on the grayscale value of the first luggage image, a plurality of items and shapes of the items in the first luggage image are obtained, and the residual energy of the X-ray beam received by the X-ray array receiving device after penetrating the first item in the first luggage within the preset time is obtained as the first residual energy, and a first attenuation coefficient of the first item after being irradiated by the X-ray beam of the first energy is calculated;

[0008] Step S3: creating and training a material classification model, inputting the first attenuation coefficient and the first energy of the first object into the material classification model, obtaining a first classification result of the first object, and when the first classification result of the first object includes a plurality of first materials, obtaining a target energy and a target attenuation coefficient of the first object based on the attenuation coefficients of the plurality of first materials under irradiation with X-ray beams of different energies, and obtaining a second classification result of the first object based on the target attenuation coefficient of the first object, the target energy and the material classification model;

[0009] Step S4: Send the classification results of each item in the first luggage image and the shape of each item to the processing unit, the processing unit determines whether each item in the first luggage image is a dangerous item, and sends the judgment result to the control unit, the control unit finally determines whether the first luggage image contains dangerous items based on the judgment result sent by the processing unit, and promptly issues a reminder when the first luggage image contains the dangerous items, wherein the classification result includes the first classification result and the second classification result.

[0010] As a preferred technical solution of the present invention, the step S3 of creating and training a material classification model includes:

[0011] Creating a material classification model, obtaining first samples of multiple different materials, and attenuation coefficients of each of the first sample materials for X-ray beams of different preset energies, storing all the first sample materials, the first sample materials corresponding to X-ray beams of different preset energies, and the attenuation coefficients under irradiation with X-ray beams of different preset energies in a sample database, clustering the attenuation coefficients corresponding to different first sample materials under irradiation with X-rays of each preset energy, and obtaining a plurality of clusters;

[0012] The first sample material in the cluster is used as a cluster sample material, and the number N of the cluster sample materials in the cluster is calculated, and M groups of test attenuation coefficients of each cluster sample material under the X-ray irradiation of the preset energy are obtained again through a testing unit, and the X-ray beam of the preset energy and the corresponding M test attenuation coefficients are added to the sample database, wherein the values ​​of M and N are in direct proportion, and the value range of N is a positive integer greater than or equal to 1 and less than or equal to 20, and the material classification model is trained based on the sample database;

[0013] Based on the second attenuation coefficient of the first sample for an X-ray beam of a second energy, the difference between the second attenuation coefficients of each of the first samples for the X-ray beam of the second energy is compared; when the difference is greater than or equal to a first threshold, multiple second samples are taken near the material cluster to which the first sample belongs, and the attenuation coefficients of the second samples for X-ray beams of different energies are used as input data of the material classification model; the material classification model is continuously trained until the difference between the attenuation coefficients of samples of different materials for the same energy is less than the first threshold, wherein the material of the second sample is different from that of the first sample.

[0014] As a preferred technical solution of the present invention, step S3 further includes:

[0015] In the material classification model, when there are multiple materials whose similarity between the first attenuation coefficient of the first object and the attenuation coefficient of the material corresponding to the first energy is greater than or equal to a second threshold, the first classification result of the first object obtained includes multiple first materials;

[0016] A plurality of attenuation coefficients corresponding to each of the first materials after irradiation with an X-ray beam of different energies are obtained from the sample database, and when a difference between the attenuation coefficients of any two different first materials after irradiation with an X-ray beam of a third energy is greater than or equal to a third threshold, the smallest third energy is used as the target energy, wherein the target energy is greater than or equal to the first energy.

[0017] As a preferred technical solution of the present invention, the step S3 of obtaining the target attenuation coefficient of the first object based on the target energy includes:

[0018] Based on the X-ray emitting device emitting an X-ray beam set to the target energy within the preset time to penetrate the interior of the luggage, a second video stream is generated, a second luggage image is acquired based on the second video stream, and a target attenuation coefficient of the first object in the second luggage image after being irradiated by the X-ray beam of the target energy is calculated, wherein the first object in the first luggage image and the first object in the second luggage image are the same object, and the preset time is less than or equal to 5s.

[0019] As a preferred technical solution of the present invention, the processing unit in step S3 determines whether each item in the luggage image is a dangerous item, including:

[0020] The processing unit receives the classification results and shapes of each item in the luggage image, and compares the classification results and shapes of the items with the shapes and materials in a preset list of dangerous goods one by one, determines whether each of the items is a dangerous good, and sends the determination result to the control unit.

[0021] As a preferred technical solution of the present invention, determining whether each of the items is a dangerous item further includes:

[0022] When the first item is determined to be a dangerous item, the determination result is directly returned, and other items in the luggage image except the first item are no longer determined;

[0023] When the first object is determined to be a non-dangerous object, other objects in the luggage image except the first object continue to be determined until all objects in the luggage image are confirmed to be non-dangerous objects, and then the determination result is returned.

[0024] As a preferred technical solution of the present invention, in step S2, a first attenuation coefficient of the first object after being irradiated by the X-ray beam of the first energy is calculated:

[0025] The first attenuation coefficient of the first object after being irradiated by the X-ray beam of the first energy is calculated by the following formula:

[0026]

[0027] in, represents the first attenuation coefficient of the first object after being irradiated by the X-ray beam of the first energy, h represents the height of the first object, and the height of the first object is obtained based on the shape of the first object, W1 represents the first energy, and W x1 represents the first remaining energy.

[0028] The present invention also provides a computing device, the device comprising:

[0029] Memory and processor;

[0030] The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions. When the computer executable instructions are executed by the processor, the above-mentioned key point detection method for X-ray video stream is implemented.

[0031] The present invention also provides a storage medium, wherein the storage medium stores computer executable instructions, and when the computer executable instructions are executed by a processor, the above-mentioned key point detection method for an X-ray video stream is implemented.

[0032] Compared with the prior art, the beneficial effects of the present invention are at least as follows:

[0033] 1. The technical solution of the present invention is that when the luggage is sent into the security inspection equipment, the control unit starts the X-ray emitting device to emit an X-ray beam of the first energy to penetrate the interior of the luggage, and the X-ray array receiving device receives the X-ray beam of the remaining energy to generate a first video stream, and obtains the first luggage image from the first video stream, and preliminarily identifies the items in the luggage without causing excessive radiation. Based on the grayscale value of the first luggage image, multiple items and the shapes of the items in the first luggage image are obtained, and the X-ray beam of the remaining energy received by the X-ray array receiving device after penetrating the first item in the first luggage within a preset time is obtained as the first residual energy, and the first attenuation coefficient of the first item after being irradiated by the X-ray beam of the first energy is calculated; by analyzing the grayscale value of the luggage image, multiple items in the luggage can be automatically identified and segmented, and their shape features can be extracted, and their materials can be classified to further identify the items in the luggage.

[0034] 2. The technical solution of the present invention creates and trains a material classification model, inputs the first attenuation coefficient and the first energy of the first object into the material classification model, obtains the first classification result of the first object, and when the first classification result of the first object includes multiple first materials, obtains the target energy and the target attenuation coefficient of the first object based on the attenuation coefficients of the multiple first materials under the irradiation of X-ray beams of different energies, and obtains the second classification result of the first object based on the target attenuation coefficient, target energy and material classification model of the first object; further determines whether there are dangerous goods in the luggage, and improves the accuracy of dangerous goods detection. The classification result of each item in the first luggage image and the shape of each item are sent to the processing unit, and the processing unit determines whether each item in the first luggage image is a dangerous item, and sends the judgment result to the control unit. The control unit finally determines whether the first luggage image contains dangerous goods, and promptly issues a reminder when the first luggage image contains dangerous goods. Through the coordination between the above steps, the present invention can improve the accuracy and efficiency of key point detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0036] Figure 1 The figure is a flowchart of the steps of the key point detection method for X-ray video stream in the present invention. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0038] It is understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but unless otherwise specified, these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, without departing from the scope of this application, a first xx script may be referred to as a second xx script, and similarly, a second xx script may be referred to as a first xx script.

[0039] In the field of security inspection, X-ray imaging technology is widely used to detect potentially dangerous items in luggage, bags and other items. Traditional X-ray image analysis relies on manual inspection, which is inefficient and easily affected by operator fatigue and subjective judgment. The present invention takes the dangerous items in the bags to be inspected as the key point of inspection, which is explained in detail below.

[0040] In view of the above technical problems, the present invention proposes Figure 1 The key point detection method for X-ray video stream shown in the figure is to install a rotatable X-ray transmitting device at the middle position near the top of the upper left side of the inner side of the security inspection device, and to install an X-ray array receiving device at the lower right side of the inner side of the security inspection device near the bottom. The transmitting direction of the X-ray transmitting device is relative to the receiving direction of the X-ray array receiving device, and is implemented by executing the following steps:

[0041] Step S1: When a bag is sent into the security inspection equipment, the control unit starts the X-ray emitting device, and controls the X-ray emitting device to emit an X-ray beam set to a first energy within a preset time to penetrate the inside of the bag, the X-ray array receiving device receives the X-ray beam with residual energy after penetrating the bag, and generates a first video stream based on the X-ray beam with residual energy, processes each frame of the first video stream, and uses image recognition technology to obtain a first bag image from the first video stream.

[0042] Specifically, in the above scheme, the transmitting device starts to emit an X-ray beam set to the first energy according to preset parameters, such as emission angle, intensity, duration, etc. The first energy is a smaller energy or an energy less than a set threshold, so as to quickly detect whether there are dangerous goods in the luggage while reducing radiation to the human body, which helps to preliminarily determine whether further detailed inspection is needed without causing excessive radiation. The emitted X-ray beam penetrates the interior of the luggage and interacts with the items in the luggage. Since items of different materials absorb and scatter X-rays to different degrees, the energy of the X-ray beam is attenuated when penetrating the items in the luggage. The X-ray array receiving device near the bottom on the lower right side of the inner side of the security inspection equipment accurately measures the received X-ray energy distribution, converts the received X-ray signal into an electrical signal, and generates a first video stream through image processing technology. The first video stream reflects the shape of the items inside the luggage in real time. Each frame of the first video stream is processed, including denoising, contrast enhancement, edge detection and other steps to improve the quality of the acquired first luggage image. Image recognition technology (such as machine learning, deep learning algorithm) is used to automatically identify and extract the first luggage image from the processed first video stream. The first luggage image contains information such as the outline and shape of the items inside the luggage. Based on this information, different items in the luggage can be accurately identified, including metals, liquids, explosives and other dangerous goods.

[0043] Step S2: Based on the grayscale value of the first luggage image, multiple items and item shapes in the first luggage image are obtained, and the X-ray beam with residual energy received by the X-ray array receiving device after penetrating the first item in the first luggage within a preset time is obtained as the first residual energy, and the first attenuation coefficient of the first item after being irradiated by the X-ray beam of the first energy is calculated.

[0044] Specifically, by analyzing the grayscale value of the luggage image, multiple items in the luggage can be automatically identified and segmented, and their shape features can be extracted. However, judging whether an item is a dangerous item based only on the shape features of the item may be inaccurate, and further judgment based on the material of the item is required. According to the residual energy of the X-ray after penetrating the item, the attenuation coefficient of each item to the X-ray can be calculated, and based on the feature that different materials have different attenuation coefficients for the same energy, the material of the item can be distinguished, thereby improving the accuracy of dangerous goods detection. The classification method of the material of the item will be described in detail below. The grayscale value of each pixel in the luggage image is different, indicating that the X-ray energy received by the pixel is different. Within the preset time, the X-ray array receiving device will continuously receive the X-ray beam with residual energy after penetrating the luggage. By measuring the difference in X-ray energy before and after penetrating the first item, that is, the difference between the first residual energy and the initial emission energy, the attenuation of the first item to the X-ray can be calculated. According to the definition of the attenuation and the physical properties of the X-ray, the first attenuation coefficient of the first item can be calculated, providing a basis for subsequent material classification and dangerous goods identification. The calculation method of the attenuation coefficient will be described below.

[0045] Step S3: Create and train a material classification model, input the first attenuation coefficient and the first energy of the first object into the material classification model, and obtain a first classification result of the first object. When the first classification result of the first object includes multiple first materials, obtain the target energy and the target attenuation coefficient of the first object based on the attenuation coefficients of the multiple first materials under the irradiation of X-ray beams of different energies, and obtain the second classification result of the first object based on the target attenuation coefficient, target energy and material classification model of the first object.

[0046] Specifically, since the attenuation coefficient under a single energy may not be sufficient to distinguish certain materials with similar attenuation characteristics, it is necessary to improve the accuracy of material classification through comprehensive analysis of attenuation coefficients under multiple energies. Specifically, a material classification model is created using a machine learning or deep learning algorithm, the model input includes the attenuation coefficient of the object and the corresponding X-ray energy information, and the output is the possible material classification of the object. The model is trained by using a training data set containing multiple material samples so that the model can learn the attenuation characteristics of different materials under different energies. After the first attenuation coefficient and the first energy information of the first object are input into the preliminarily trained material classification model, the model outputs the first classification result of the first object. The first classification result may contain only one material or multiple materials. If there is only one material, it can be determined that the material is the material corresponding to the object. If the first classification result includes multiple materials whose attenuation coefficients are similar to a certain value, that is, it is not easy to distinguish between the multiple materials, it is impossible to determine which material the object corresponds to, so further analysis is required to clarify the material corresponding to the object. According to the attenuation coefficient database of different materials at different energies, the possible attenuation coefficients of the materials in these first classification results at different energies are calculated, and a target energy is selected. The target energy can maximize the difference in attenuation coefficients between different materials to separate the materials. The attenuation coefficients at the target energy are re-entered into the material classification model to obtain a more accurate second classification result. The parameters and structure of the material classification model can also be adjusted through continuous optimization and iteration based on the actual detection results. The trained material classification model is integrated into the key point detection system of the X-ray video stream. During the real-time detection process, the model is automatically called for material classification, and the classification results are used to further determine whether there are dangerous goods in the luggage, thereby improving the accuracy of dangerous goods detection.

[0047] Step S4: Send the classification result of each item in the first luggage image and the shape of each item to the processing unit. The processing unit determines whether each item in the first luggage image is a dangerous item, and sends the judgment result to the control unit. The control unit finally determines whether the first luggage image contains dangerous items based on the judgment result sent by the processing unit, and promptly issues a reminder when the first luggage image contains dangerous items. The classification result includes a first classification result and a second classification result.

[0048] Specifically, by combining the classification results and shape information of the items obtained in the previous steps, it is determined whether there are dangerous items in the luggage, and a reminder is issued to the staff in time. The classification results (including the first classification results and the second classification results) and shape information of each item in the first luggage image and the second luggage image (if the second luggage image is obtained) are organized into a unified format and sent to the processing unit for subsequent dangerous goods judgment. After receiving the data, the processing unit classifies and judges each item according to the preset dangerous goods list and rules. The classification results of the items are compared with the dangerous goods list, and a comprehensive evaluation is performed based on the shape and other information of the items to reduce misjudgment and missed judgment. If it is found that a certain item belongs to a category in the dangerous goods list, or its shape, size and other characteristics meet the characteristics of dangerous goods, the item is marked as a dangerous item. The control unit finally determines whether the luggage image contains dangerous items based on the judgment result. If dangerous items are contained, a reminder signal is issued in time, such as an audible and visual alarm, a warning message displayed on the screen, etc., thereby improving the accuracy and efficiency of key point detection.

[0049] Through the coordination of the above steps, the present invention can improve the accuracy and efficiency of key point detection. The present invention can also be applied to different types of bag and article detection requirements, and can flexibly respond to different security inspection scenarios and requirements by adjusting the parameters of the X-ray emission device and the image processing algorithm.

[0050] Furthermore, the creation and training of the material classification model in the above step S3 includes:

[0051] A material classification model is created, first samples of multiple different materials and the attenuation coefficient of each first sample material for X-ray beams of different preset energies are obtained, all first sample materials, X-ray beams of different preset energies corresponding to the first sample materials, and the attenuation coefficients under irradiation with X-ray beams of different preset energies are stored in a sample database, the attenuation coefficients corresponding to different first sample materials under irradiation with X-rays of each preset energy are clustered, and multiple clusters are obtained.

[0052] The first sample material in the cluster cluster is used as the cluster sample material, and the number N of cluster sample materials in the cluster cluster is calculated, and M groups of test attenuation coefficients of each cluster sample material under X-ray irradiation of preset energy are obtained through the test unit again, and the X-ray beam of preset energy and the corresponding M test attenuation coefficients are added to the sample database, wherein the values ​​of M and N are in direct proportion, and the value range of N is a positive integer greater than or equal to 1 and less than or equal to 20, and the material classification model is trained based on the sample database.

[0053] Based on the second attenuation coefficient of the first sample for the X-ray beam of the second energy, the difference between the second attenuation coefficients of each first sample for the X-ray beam of the second energy is compared. When the difference is greater than or equal to a first threshold, multiple second samples are taken near the material cluster to which the first sample belongs, and the attenuation coefficients of the second samples for X-ray beams of different energies are used as input data of the material classification model. The material classification model is continuously trained until the difference between the attenuation coefficients of samples of different materials for the same energy is less than the first threshold, wherein the material of the second sample is different from that of the first sample.

[0054] Specifically, first samples of multiple different materials are collected to train the material classification model, and the first samples are representative and can cover most of the materials that may appear in the target application field. For each sample, an X-ray source is used to emit X-ray beams of different energies, and the attenuation coefficient of the sample is measured at each energy. The attenuation coefficient reflects the absorption characteristics of different materials to X-rays of different energies, and is an important basis for material classification. The attenuation coefficient of each first sample to X-ray beams of different energies and different energies are used as input data and input into the material classification model. Based on different energies and the attenuation coefficients corresponding to different energies, a cluster analysis is performed on multiple first samples. The clustering algorithm divides the samples into multiple material clusters, each of which represents a class of similar materials. Afterwards, the classification result of each first sample is determined based on the material cluster to which it belongs.

[0055] If a material cluster includes multiple similar materials, in order to make the output result of the material classification model more accurate, it is necessary to further train the material classification model by adding more refined samples. Specifically, based on the attenuation coefficient of the first sample to the second energy (or other specific energy) X-ray beam, the differences between different samples are compared. When the difference is greater than or equal to the first threshold, it indicates that the current material model may not be able to accurately distinguish the materials of these samples. Take multiple second samples near the material cluster to which the samples with significant differences belong. The material of the second sample should be different from that of the original first sample to increase the training accuracy of the model. Use the newly added second sample and its corresponding attenuation coefficient as input data to continue training the material classification model. Through multiple iterative training, the difference between the attenuation coefficients of different material samples to the same energy X-ray beam is gradually reduced until the difference is less than the first threshold. This method can make the material classification model more accurate in classifying items, so that the final judgment result of luggage is more accurate.

[0056] Furthermore, the above step S3 also includes:

[0057] In the material classification model, when there are multiple materials whose similarity between the first attenuation coefficient of the first object and the attenuation coefficient of the material corresponding to the first energy is greater than or equal to the second threshold, the first classification result of the first object obtained includes multiple first materials.

[0058] A plurality of attenuation coefficients corresponding to each first material after irradiation with an X-ray beam of different energies are obtained from a sample database, and when a difference between the attenuation coefficients of any two different first materials after irradiation with an X-ray beam of a third energy is greater than or equal to a third threshold, the minimum third energy is used as the target energy, wherein the target energy is greater than or equal to the first energy.

[0059] Specifically, the first classification result of the first object output by the material classification model may include only one material or multiple materials. If there is only one material, it can be determined that the material is the material corresponding to the object. If the first classification result includes multiple materials whose attenuation coefficients have a similarity greater than the second threshold, for example, there are multiple materials with a similarity greater than 80%, that is, it is not easy to distinguish between the multiple materials. At this time, it is impossible to determine which material the object corresponds to. Therefore, further analysis is needed to clarify the material corresponding to the object.

[0060] Retrieve the attenuation coefficient of the first material listed in the first classification result from the sample database after irradiation with X-ray beams of different energies. For any two different first materials, calculate the difference in their attenuation coefficients after irradiation with the same energy X-ray beam. This difference reflects the degree of difference in the attenuation coefficients of the two materials at this energy. Define a third threshold value, which is used to determine whether the difference in the attenuation coefficients of the two materials at a certain energy is large enough to distinguish them. The selection of the third threshold value needs to be based on the needs of the actual application, the detection accuracy of the system, and the statistical characteristics of the sample data. Traverse all energies, and for each energy, check the attenuation coefficient difference between all possible pairs of first materials. If there is at least one pair of materials whose attenuation coefficient difference is greater than or equal to the third threshold, the candidate energy is regarded as a valid target energy. If there are multiple energies that meet the conditions, the energy with the largest difference can be selected. Use the selected target energy to re-irradiate the items in the bag, and measure the attenuation coefficients of these items at the target energy. These attenuation coefficients are input into the material classification model to obtain a more accurate second classification result for the items. The target energy refers to the X-ray beam emitted by the X-ray device when the attenuation coefficients of different materials are greatly different, so as to maximize the distinction between materials.

[0061] Furthermore, in the above step S3, based on the target energy, obtaining the target attenuation coefficient of the first object includes:

[0062] Based on the X-ray emitting device emitting an X-ray beam set to a target energy to penetrate the interior of a bag within a preset time, a second video stream is generated, a second bag image is acquired based on the second video stream, and a target attenuation coefficient of the first object in the second bag image after being irradiated by the X-ray beam of the target energy is calculated, wherein the first object in the first bag image and the first object in the second bag image are the same object, and the preset time is less than or equal to 5s.

[0063] Specifically, when the material of the object cannot be accurately determined, the energy of the X-ray emission device is set to the previously determined target energy, which is calculated based on the difference in attenuation coefficients of different materials at different energies, in order to maximize the differentiation between materials. Within a preset time, the X-ray emission device emits an X-ray beam with the target energy to penetrate the interior of the bag. At the same time, the X-ray beam after penetrating the bag is captured to generate a second video stream, extract a second bag image, and locate the first object (i.e., the same object as in the first bag image) in the second bag image. For each pixel or area of ​​the first object in the second bag image, the target attenuation coefficient is calculated based on the target energy of the X-ray beam, the energy after penetrating the bag, and the material characteristics of the bag and the object (such as density, thickness, etc.), so as to accurately classify the objects by material.

[0064] Furthermore, the processing unit in the above step S3 determines whether each item in the luggage image is a dangerous item, including:

[0065] The processing unit receives the classification results and shapes of each item in the luggage image, and compares the classification results and shapes of the items with the shapes and materials in the preset dangerous goods list one by one to determine whether each item is a dangerous good, and sends the determination result to the control unit.

[0066] Specifically, the processing unit receives the output from the material classification module, including the classification results (such as material, etc.) and shapes (such as size, outline, etc.) of each item in the luggage image, and also obtains a preset dangerous goods list, which contains detailed information of various dangerous goods, including their shape characteristics, material descriptions, possible size ranges, etc., and is established in advance according to safety regulations and dangerous goods identification standards. For each item in the luggage image, the processing unit compares its classification results and shape information with the items in the preset dangerous goods list one by one. The comparison process may involve matching in multiple dimensions, including but not limited to: material matching: checking whether the material of the item matches a certain material in the dangerous goods list; shape matching: comparing whether the shape and size of the item match the description in the dangerous goods list. Other features such as color, texture, marking, packaging method, etc. can also be compared as needed. If the classification results and shape information of an item are highly matched with an entry in the preset dangerous goods list in multiple dimensions, the processing unit will determine that the item is a dangerous item. If the matching degree is less than a certain threshold, or no matching entry is found, the processing unit will determine that the item is not a dangerous item. The processing unit sends the judgment result of each item (i.e. whether it is a dangerous item) to the control unit. The control unit can trigger subsequent security inspection processes based on the judgment result, such as alarm, bag interception, further manual inspection, etc.

[0067] Furthermore, determining whether each item is a dangerous good also includes:

[0068] When the first item is judged as a dangerous item, the judgment result is directly returned, and other items in the bag image except the first item are no longer judged;

[0069] When the first object is determined to be a non-dangerous object, other objects in the luggage image except the first object continue to be determined until all objects in the luggage image are confirmed to be non-dangerous objects, and then the determination result is returned.

[0070] Specifically, in the security inspection process of bag images, the process of the processing unit judging each item as dangerous goods does contain an efficient judgment logic, which can minimize unnecessary inspection time and resource consumption while ensuring safety, thereby improving detection efficiency.

[0071] Furthermore, in the above step S2, a first attenuation coefficient of the first object after being irradiated by an X-ray beam of a first energy is calculated:

[0072] The first attenuation coefficient of the first object after being irradiated by an X-ray beam of the first energy is calculated by the following formula:

[0073]

[0074] in, represents the first attenuation coefficient of the first object after being irradiated by the X-ray beam of the first energy, h represents the height of the first object, and the height of the first object is obtained based on the shape of the first object, W1 represents the first energy, and W x1 Represents the first residual energy.

[0075] The present invention also provides a computing device, the device comprising:

[0076] Memory and processor;

[0077] The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions. When the computer executable instructions are executed by the processor, the above-mentioned key point detection method for X-ray video stream is implemented.

[0078] The present invention also provides a computer storage medium, which stores program instructions, wherein when the program instructions are executed, the device where the computer storage medium is located is controlled to execute the above-mentioned key point detection method for X-ray video stream.

[0079] In summary, when a bag is sent into the security inspection equipment, the control unit starts the X-ray emitting device to emit an X-ray beam of the first energy to penetrate the inside of the bag, and the X-ray array receiving device receives the X-ray beam of the remaining energy to generate a first video stream, and obtains the first bag image from the first video stream, and preliminarily identifies the items in the bag without causing excessive radiation. Based on the grayscale value of the first bag image, multiple items and the shapes of the items in the first bag image are obtained, and the X-ray beam of the remaining energy received by the X-ray array receiving device after penetrating the first item in the first bag within a preset time is obtained as the first residual energy, and the first attenuation coefficient of the first item after being irradiated by the X-ray beam of the first energy is calculated; by analyzing the grayscale value of the bag image, multiple items in the bag can be automatically identified and segmented, and their shape features can be extracted, and their materials can be classified to further identify the items in the bag.

[0080] By creating and training a material classification model, the first attenuation coefficient and the first energy of the first object are input into the material classification model to obtain the first classification result of the first object. When the first classification result of the first object includes multiple first materials, the target energy and the target attenuation coefficient of the first object are obtained based on the attenuation coefficients of the multiple first materials under the irradiation of X-ray beams of different energies. The second classification result of the first object is also obtained based on the target attenuation coefficient, target energy and material classification model of the first object; further determine whether there are dangerous goods in the luggage to improve the accuracy of dangerous goods detection. The classification result of each item in the first luggage image and the shape of each item are sent to the processing unit. The processing unit determines whether each item in the first luggage image is a dangerous item, and sends the judgment result to the control unit. The control unit finally determines whether the first luggage image contains dangerous goods, and issues a prompt when the first luggage image contains dangerous goods. Through the coordination between the above steps, the present invention can improve the accuracy and efficiency of key point detection.

[0081] It should be understood that, although each step in the flow chart of each embodiment of the present invention is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0082] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the above-mentioned program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0083] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0084] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

[0085] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A key point detection method for an X-ray video stream, wherein a rotatable X-ray transmitting device is installed at a middle position near the top on the upper left side of the inner side of a security inspection device, and an X-ray array receiving device is installed at a position near the bottom on the lower right side of the inner side of the security inspection device, wherein the transmitting direction of the X-ray transmitting device is opposite to the receiving direction of the X-ray array receiving device, and the method is characterized in that: The method comprises: Step S1: When a bag is sent into the security inspection device, the control unit starts the X-ray emitting device, and controls the X-ray emitting device to emit an X-ray beam set to a first energy within a preset time to penetrate the interior of the bag, the X-ray array receiving device receives the X-ray beam with residual energy after penetrating the bag, and generates a first video stream based on the X-ray beam with residual energy, processes each frame of the first video stream, and uses image recognition technology to obtain a first bag image from the first video stream; Step S2: based on the grayscale value of the first luggage image, a plurality of items and shapes of the items in the first luggage image are obtained, and the residual energy of the X-ray beam received by the X-ray array receiving device after penetrating the first item in the first luggage within the preset time is obtained as the first residual energy, and a first attenuation coefficient of the first item after being irradiated by the X-ray beam of the first energy is calculated; Step S3: creating and training a material classification model, inputting the first attenuation coefficient and the first energy of the first object into the material classification model, obtaining a first classification result of the first object, and when the first classification result of the first object includes a plurality of first materials, obtaining a target energy and a target attenuation coefficient of the first object based on the attenuation coefficients of the plurality of first materials under irradiation with X-ray beams of different energies, and obtaining a second classification result of the first object based on the target attenuation coefficient of the first object, the target energy and the material classification model; Step S4: Send the classification results of each item in the first luggage image and the shape of each item to the processing unit, the processing unit determines whether each item in the first luggage image is a dangerous item, and sends the judgment result to the control unit, the control unit finally determines whether the first luggage image contains dangerous items based on the judgment result sent by the processing unit, and promptly issues a reminder when the first luggage image contains the dangerous items, wherein the classification result includes the first classification result and the second classification result.

2. The method according to claim 1, characterized in that: The step S3 of creating and training a material classification model includes: Creating a material classification model, obtaining first samples of multiple different materials, and attenuation coefficients of each of the first sample materials for X-ray beams of different preset energies, storing all the first sample materials, the first sample materials corresponding to X-ray beams of different preset energies, and the attenuation coefficients under irradiation with X-ray beams of different preset energies in a sample database, clustering the attenuation coefficients corresponding to different first sample materials under irradiation with X-rays of each preset energy, and obtaining a plurality of clusters; The first sample material in the cluster is used as a cluster sample material, and the number N of the cluster sample materials in the cluster is calculated, and M groups of test attenuation coefficients of each cluster sample material under the X-ray irradiation of the preset energy are obtained again through a testing unit, and the X-ray beam of the preset energy and the corresponding M test attenuation coefficients are added to the sample database, wherein the values ​​of M and N are in direct proportion, and the value range of N is a positive integer greater than or equal to 1 and less than or equal to 20, and the material classification model is trained based on the sample database; Based on the second attenuation coefficient of the first sample for an X-ray beam of a second energy, the difference between the second attenuation coefficients of each of the first samples for the X-ray beam of the second energy is compared; when the difference is greater than or equal to a first threshold, multiple second samples are taken near the material cluster to which the first sample belongs, and the attenuation coefficients of the second samples for X-ray beams of different energies are used as input data of the material classification model; the material classification model is continuously trained until the difference between the attenuation coefficients of samples of different materials for the same energy is less than the first threshold, wherein the material of the second sample is different from that of the first sample.

3. The method according to claim 2, characterized in that The step S3 further comprises: In the material classification model, when there are multiple materials whose similarity between the first attenuation coefficient of the first object and the attenuation coefficient of the material corresponding to the first energy is greater than or equal to a second threshold, the first classification result of the first object obtained includes multiple first materials; A plurality of attenuation coefficients corresponding to each of the first materials after irradiation with an X-ray beam of different energies are obtained from the sample database, and when a difference between the attenuation coefficients of any two different first materials after irradiation with an X-ray beam of a third energy is greater than or equal to a third threshold, the smallest third energy is used as the target energy, wherein the target energy is greater than or equal to the first energy.

4. The method according to claim 1, characterized in that: The step S3 of obtaining a target attenuation coefficient of the first object based on the target energy includes: Based on the X-ray emitting device emitting an X-ray beam set to the target energy within the preset time to penetrate the interior of the luggage, a second video stream is generated, a second luggage image is acquired based on the second video stream, and a target attenuation coefficient of the first object in the second luggage image after being irradiated by the X-ray beam of the target energy is calculated, wherein the first object in the first luggage image and the first object in the second luggage image are the same object, and the preset time is less than or equal to 5s.

5. The method according to claim 1, characterized in that The processing unit in step S3 determines whether each item in the luggage image is a dangerous item, including: The processing unit receives the classification results and shapes of each item in the luggage image, and based on the classification results and shapes of the items, compares them one by one with the shapes and materials in a preset list of dangerous goods, determines whether each of the items is a dangerous good, and sends the determination result to the control unit.

6. The method according to claim 4, characterized in that Determining whether each of the items is a dangerous good also includes: When the first item is determined to be a dangerous item, the determination result is directly returned, and other items in the luggage image except the first item are no longer determined; When the first article is determined to be a non-dangerous article, other articles in the luggage image except the first article continue to be determined until all articles in the luggage image are confirmed to be non-dangerous articles, and then the determination result is returned.

7. The method according to claim 1, characterized in that In step S2, a first attenuation coefficient of the first object after being irradiated by the X-ray beam of the first energy is calculated: The first attenuation coefficient of the first object after being irradiated by the X-ray beam of the first energy is calculated by the following formula: in, represents the first attenuation coefficient of the first object after being irradiated by the X-ray beam of the first energy, h represents the height of the first object, and the height of the first object is obtained based on the shape of the first object, W1 represents the first energy, and W x1 represents the first remaining energy.

8. A computing device, characterized in that: The device comprises: Memory and processor; The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions. When the computer executable instructions are executed by the processor, the key point detection method for X-ray video stream described in any one of claims 1-7 is implemented.

9. A computer storage medium, characterized in that: The storage medium stores program instructions, wherein when the program instructions are executed, the device where the storage medium is located is controlled to execute the key point detection method for X-ray video stream according to any one of claims 1 to 7.

Images