Retriever generation method, target detection method and ray scanning detection system

Abstract

Provided in the present disclosure are a retriever generation method and apparatus, a target detection method and apparatus, an electronic device and a ray scanning detection system, which can be applied to the field of image processing. The generation method comprises: constructing a sample library, involving: acquiring first sample data, which comprises a first image and a first label, wherein the first label represents the category of a first detection target, and acquiring second sample data, which comprises a second image and a second label, wherein the second label represents the category of a second detection target, the first sample data and the second sample data respectively being positive sample data and negative sample data; constructing a feature library, involving: using a feature extraction model to extract first and second image features, forming a first and a second image feature-label relationship, and on the basis of the first and the second image feature-label relationship, generating the feature library; and on the basis of the feature library, generating a retriever.

Description

RETRIEVER GENERATION METHOD, TARGET DETECTION METHOD AND RAY SCANNING DETECTION SYSTEM [1] This application claims priority to Chinese Patent Application No. 202410430802.X, filed on April 11, 2024, the entire contents of which are incorporated herein by reference.TECHNICAL FIELD[2] The present disclosure relates to the field of image processing and / or object detection technologies, and in particular, to a retriever generation method and apparatus, an object detection method and apparatus, an electronic device, a radiation scanning detection system, a computer-readable storage medium, and a program product. BACKGROUND[3] The widespread application of radiation scanning detection technology has greatly facilitated the identification of prohibited items. However, rapidly increasing parcel volumes and throughput requirements have placed tremendous pressure on conventional manual image interpretation, creating an urgent need for more automated identification methods. Intelligent recognition technology for radiation scanning images has accordingly emerged. This technology employs deep learning-based object detection methods to automatically identify objects in acquired radiation scanning images and issue alarms, thereby substantially reducing the human workload.[4] Deep learning-based methods for recognizing radiation scanning images have been developed in related technologies. However, existing deep learning applications typically suffer from certain deficiencies or problems. For example, retraining a deep learning model generally requires a large number of labeled samples, while labeled data for certain rare classes may be very difficult to obtain. As another example, current mainstream algorithms generally lack the capability of continuous learning. Once a network model has been deployed, its structure and parameters are fixed and no longer change, and data encountered after deployment cannot be fully utilized to optimize the model.[5] The information disclosed in this section is provided solely to facilitate an understanding of the background of the inventive concepts of the present disclosure. Therefore, such information may include information that does not constitute prior art. SUMMARY[6] The present disclosure provides a retriever generation method and apparatus, an object detection method and apparatus, an electronic device, a radiation scanning detection system, a computer-readable storage medium, and a program product.[7] In an aspect, a method for generating a retriever for a radiation scanning detection system is provided, including: [8] constructing a sample library, including: obtaining first sample data, where the first sample data includes a first image and a first label, the first image is an image including at least one first detection target, and the first label indicates a class of the at least one first detection target; and obtaining second sample data, where the second sample data includes a second image and a second label, the second image is an image including at least one second detection target, and the second label indicates a class of the at least one second detection target, where the first sample data is positive sample data, and the second sample data is negative sample data;[9] constructing a feature library, including: extracting a first image feature of the first image and a second image feature of the second image by using a pre-trained feature extraction model; forming a first feature-label association based on the first image feature and the first label, and forming a second feature-label association based on the second image feature and the second label; and generating the feature library based on the first feature-label association and the second feature-label association; and

[10] generating a retriever based on the feature library.

[11] According to some exemplary embodiments, the constructing a sample library further includes: obtaining third sample data, where the third sample data includes a third image and a third label, the third image is a radiation scanning image including at least one third detection target, the third label indicates a class of the at least one third detection target, and the third sample data includes positive sample data and / or negative sample data.

[12] According to some exemplary embodiments, the constructing a feature library further includes: extracting a third image feature of the third image by using the pre-trained feature extraction model; forming a third feature-label association based on the third image feature and the third label; and storing the third feature-label association into the feature library.

[13] According to some exemplary embodiments, the first image feature includes at least one of a sparse feature and a dense feature; and / or the second image feature includes at least one of a sparse feature and a dense feature; and / or the third image feature includes at least one of a sparse feature and a dense feature.

[14] According to some exemplary embodiments, the sparse feature includes a region feature represented using bounding box annotation; and / or the dense feature includes a region feature represented using pixel-level mask annotation.

[15] According to some exemplary embodiments, the obtaining first sample data includes: obtaining a plurality of first images for an application scenario of the radiation scanning detection system, where each of the plurality of first images includes at least one first detection target, and the first detection target is a representative detection target suitable for the application scenario of the radiation scanning detection system; and labeling the class of the first detection target in the plurality of first images to form a plurality of first labels.

[16] According to some exemplary embodiments, the obtaining second sample data includes: obtaining a plurality of second images according to the first sample data, where each of the plurality of second images includes at least one second detection target, the second detection target is similar to the first detection target, and the class of the second detection target is different from the class of the first detection target; and labeling the class of the second detection target in the plurality of second images to form a plurality of second labels.

[17] According to some exemplary embodiments, the obtaining a plurality of second images includes: obtaining the plurality of second images by at least one of active acquisition, retrieval of historical data, and synthesis using a generative model.

[18] According to some exemplary embodiments, the first sample data and the second sample data are obtained before the radiation scanning detection system is placed into operational service.

[19] According to some exemplary embodiments, the third sample data is obtained during the operational service of the radiation scanning detection system.

[20] According to some exemplary embodiments, the obtaining third sample data includes: spot-checking detection results of the radiation scanning detection system during the operational service of the radiation scanning detection system; in response to a missed detection and / or a false alarm existing in the detection results, obtaining an image associated with the missed detection and / or the false alarm as the third image; and labeling the class of the third detection target in the third images to form a plurality of third labels.

[21] In another aspect, an object detection method for a radiation scanning detection system is provided, including: acquiring a radiation scanning image by the radiation scanning detection system, where the radiation scanning image includes at least one detection target; extracting an image feature of the radiation scanning image by using a pre-trained feature extraction model; performing a similarity search with respect to the image feature of the radiation scanning image by using a retriever generated by the method described above; and in response to the image feature of the radiation scanning image matching an image feature in the feature library, obtaining a class corresponding to the image feature of the radiation scanning image and determining the class to be a class of the at least one detection target included in the radiation scanning image.

[22] According to some exemplary embodiments, the performing a similarity search includes: searching, by using a nearest neighbor search algorithm, the feature library of the retriever for an image feature matching the image feature of the radiation scanning image.

[23] According to some exemplary embodiments, the image features in the feature library include sparse features and dense features, and the performing a similarity search includes: searching, with respect to the image feature of the radiation scanning image, the feature library of the retriever to determine whether there exists a sparse feature matching the image feature of the radiation scanning image; and in response to existence of a sparse feature matching the image feature of the radiation scanning image, further searching, with respect to the image feature of the radiation scanning image, the feature library of the retriever for a dense feature matching the image feature of the radiation scanning image; or where the performing a similarity search includes: searching, with respect to the image feature of the radiation scanning image, the feature library of the retriever only for a dense feature matching the image feature of the radiation scanning image; or where the performing a similarity search includes: searching, with respect to the image feature of the radiation scanning image, the feature library of the retriever only for a sparse feature matching the image feature of the radiation scanning image.

[24] According to some exemplary embodiments, the method further includes: during the acquisition of the radiation scanning image, further acquiring metadata corresponding to the radiation scanning image, where the metadata includes device parameters for acquiring the radiation scanning image.

[25] According to some exemplary embodiments, the radiation scanning detection system includes a multi-view scanning detection system, and the method further includes: based on the acquired radiation scanning image and the class of the at least one detection target included in the radiation scanning image, forming two-dimensional scanning images from a plurality of views; and performing three-dimensional fusion on the two-dimensional scanning images from the plurality of views based on geometric relationships and the identified at least one detection target to form a three-dimensional scanning image, where a three-dimensional identification result for the at least one detection target is displayed in the three-dimensional scanning image; or where the method further includes: based on the acquired radiation scanning image and the class of the at least one detection target included in the radiation scanning image, forming two-dimensional scanning images from a plurality of views; and obtaining a three-dimensional identification result for the at least one detection target based on geometric relationships and the identified at least one detection target without forming a three-dimensional scanning image by fusion.

[26] According to some exemplary embodiments, the method further includes: based on the acquired radiation scanning image and the class of the at least one detection target included in the radiation scanning image, determining a physical property of the at least one detection target according to the class; and performing three-dimensional fusion on the radiation scanning image based on the physical property and the identified at least one detection target to form a three-dimensional scanning image, where a three-dimensional identification result for the at least one detection target is displayed in the three-dimensional scanning image.

[27] In yet another aspect, an apparatus for generating a retriever for a radiation scanning detection system is provided, including:

[28] a first construction module configured to construct a sample library, where the first construction module includes: a first data obtaining sub-module configured to obtain first sample data, where the first sample data includes a first image and a first label, the first image is a radiation scanning image including at least one first detection target, and the first label indicates a class of the at least one first detection target; and a second data obtaining sub-module configured to obtain second sample data, where the second sample data includes a second image and a second label, the second image is a radiation scanning image including at least one second detection target, the second label indicates a class of the at least one second detection target, the first sample data is positive sample data, and the second sample data is negative sample data;

[29] a second construction module configured to construct a feature library, where the second construction module includes: an extraction sub-module configured to extract a first image feature of the first image and a second image feature of the second image by using a pre-trained feature extraction model; an association forming sub-module configured to form a first feature-label association based on the first image feature and the first label and to form a second feature-label association based on the second image feature and the second label; and a feature library generation sub-module configured to generate the feature library based on the first feature-label association and the second feature-label association; and

[30] a retriever generation module configured to generate a retriever based on the feature library.

[31] In still another aspect, an object detection apparatus for a radiation scanning detection system is provided, including: an image acquisition module configured to acquire a radiation scanning image by the radiation scanning detection system, where the radiation scanning image includes at least one detection target; a feature extraction module configured to extract an image feature of the radiation scanning image by using a pre-trained feature extraction model; a retrieval module configured to perform a similarity search with respect to the image feature of the radiation scanning image by using a retriever generated by the method described above; and an object detection module configured to, in response to the image feature of the radiation scanning image matching an image feature in the feature library, obtain a class corresponding to the image feature of the radiation scanning image and determine the class to be a class of the at least one detection target included in the radiation scanning image.

[32] In yet another aspect, an electronic device is provided, including: one or more processors; and a storage device storing one or more programs, where the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method described above.

[33] In a further aspect, a radiation scanning detection system is provided, including: a radiation source configured to emit radiation; a detector configured to receive radiation emitted from the radiation source and passed through an object under inspection; and a data processing device communicatively connected to the detector, where the data processing device includes: one or more processors; and a storage device storing one or more programs, where the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method described above.

[34] In yet a further aspect, a computer-readable storage medium having executable instructions thereon is provided, where the executable instructions, when executed by a processor, cause the processor to perform the method described above.

[35] In still a further aspect, a computer program product including a computer program is provided, where the computer program, when executed by a processor, causes the processor to implement the method described above. BRIEF DESCRIPTION OF THE DRAWINGS

[36] The present disclosure will be described in detail with reference to the following drawings to facilitate a better understanding of the present disclosure.

[37] FIG. 1 is a schematic structural diagram of a radiation scanning detection system according to some exemplary embodiments of the present disclosure.

[38] FIG. 2 is a schematic diagram of a CT scanning apparatus that implements a reconstruction scanning mode according to some exemplary embodiments of the present disclosure.

[39] FIG. 3 illustrates an example of attenuation signal data obtained according to an embodiment of the present disclosure.

[40] FIG. 4 is a schematic flowchart of a method for generating a retriever for a radiation scanning detection system according to an embodiment of the present disclosure.

[41] FIG. 5 is a schematic flowchart of constructing a sample library according to an embodiment of the present disclosure.

[42] FIG. 6 is a schematic flowchart of constructing a feature library according to an embodiment of the present disclosure.

[43] FIG. 7 is a schematic diagram of a first image according to some exemplary embodiments of the present disclosure.

[44] FIG. 8 is a schematic diagram of a first image according to other exemplary embodiments of the present disclosure.

[45] FIG. 9 is a schematic flowchart of an object detection method for a radiation scanning detection system according to an embodiment of the present disclosure.

[46] FIG. 10 is a schematic flowchart of an object detection method for a radiation scanning detection system according to an embodiment of the present disclosure, in which a retriever generation process and an object detection process are schematically illustrated.

[47] FIG. 11 is a structural block diagram of an apparatus for generating a retriever for a radiation scanning detection system according to an exemplary embodiment of the present disclosure.

[48] FIG. 12 is a structural block diagram of an object detection apparatus for a radiation scanning detection system according to an exemplary embodiment of the present disclosure.

[49] FIG. 13 schematically illustrates a structural block diagram of an electronic device suitable for implementing the retriever generation method or the object detection method according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION OF EMBODIMENTS

[50] Specific embodiments of the present disclosure will be described in detail below. It should be noted that the embodiments described herein are provided merely for illustration and are not intended to limit the present disclosure. Numerous specific details are set forth in the following description to provide a thorough understanding of the present disclosure. However, it will be apparent to those of ordinary skill in the art that the present disclosure may be practiced without these specific details. In other instances, well-known structures, materials, or methods have not been described in detail so as not to obscure the present disclosure.

[51] Throughout this specification, any reference to "one embodiment", "an embodiment", "one example", or "an example" means that a particular feature, structure, or characteristic described in connection with the embodiment or example is included in at least one embodiment of the present disclosure. Thus, appearances of the phrases "in one embodiment", "in an embodiment", "one example", or "an example" in various places throughout this specification do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, or characteristics may be combined in any suitable combination and / or sub-combination in one or more embodiments or examples. Those of ordinary skill in the art will also understand that the term "and / or", as used herein, includes any and all combinations of one or more of the associated listed items.

[52] The terminology used herein is for the purpose of describing specific embodiments only and is not intended to limit the present disclosure. The terms "comprise", "include", and the like, as used herein, specify the presence of stated features, steps, operations, and / or components, but do not preclude the presence or addition of one or more other features, steps, operations, and / or components.

[53] Unless otherwise defined, all terms used herein (including technical and scientific terms) have the meanings commonly understood by those of ordinary skill in the art. It should be noted that the terms used herein should be construed as having meanings consistent with the context of this specification and should not be construed in an idealized or overly rigid manner.

[54] In the field of computer vision, an important application scenario of deep learning methods is object detection. For an input image, it is possible to, by using an algorithm, provide position information (such as a bounding box or edge contour) of an object and a class to which the object belongs, which has important applications in prohibited item recognition, autonomous driving, anomaly detection, and other fields.

[55] The inventors have found through research that, with the development of radiation scanning detection technology and deep learning technology, intelligent recognition technology for radiation scanning images has emerged. The technology employs a deep learning-based object detection method to automatically recognize objects in acquired radiation scanning images and issue alarms, thereby substantially reducing the human workload. For example, computed tomography (CT) scanning technology employs a reconstruction algorithm to obtain information about the internal three-dimensional structure of an object, effectively resolving the object overlap problem encountered in digital radiography (DR). CT scanning may inspect an internal structure of an object without contact, at moderate cost, and with intuitive results, and has achieved significant success in medical imaging, security inspection, industrial nondestructive testing, and many other fields. In particular, dual-energy material identification technology for security CT may distinguish material information such as the atomic number of an object, greatly improving the capability to detect dangerous items, and is an important and indispensable baggage-inspection technique for security CT.

[56] The widespread application of CT has presented a significant challenge for identifying prohibited items. Rapidly increasing parcel volumes and throughput requirements have placed tremendous pressure on conventional manual image interpretation, creating an urgent need for more automated recognition methods. Intelligent CT recognition technology has accordingly emerged and employs machine learning technology to recognize CT scan data and issue alarms. In 2012, deep learning methods based on artificial neural networks began advancing rapidly across various subfields of computer vision and demonstrated outstanding performance in image classification, object detection, super-resolution, and other tasks. These results fully demonstrated the powerful capability of deep learning to discover complex structures automatically from image data without manually extracting features. It should be noted that the multilayer neural networks underlying deep learning models were not recently proposed. Three major factors have driven the breakthrough results achieved by deep learning in recent years: complex deep network architectures, massive training datasets represented by ImageNet, and advances in high-performance computing technology. Among these factors, data is regarded as the fuel of deep learning and has a decisive effect on ultimate algorithm performance.

[57] The inventors have further found that, despite the rapid development of deep learning, most deep learning applications still have deficiencies. For example, current mainstream algorithms generally lack capability of continuous learning. Once deployed, the structure and parameters of a network model are fixed and no longer change, and data encountered after deployment cannot be fully utilized to optimize the model. A neural network model obtained by conventional end-to-end training generally has the following characteristics.

[58] (1) An end-to-end neural network is a function approximator for training data. A deep network learns feature representations and a decision surface under a particular distribution of training data. Although training may provide powerful feature-learning and modeling capabilities, the nature of end-to-end learning also creates a high degree of dependence on the training data. The neural network model does not explicitly separate feature extraction from modeling, but instead performs fitting as a whole. Consequently, when the classes change, the model must relearn and adapt to a new data distribution.

[59] (2) A neural network requires a large amount of training data. Although various regularization methods may reduce the model's dependence on the amount of data to some extent, compared with conventional machine learning methods, the demand of a network model for labeled samples has not been fundamentally resolved. This demand mainly results from the high complexity of the network architecture, because a large number of parameters must be constrained using large-scale data. A network is therefore difficult to adapt to few-shot scenarios.

[60] (3) It is difficult to extend the classes of an end-to-end neural network model. A complex nonlinear network architecture provides a model with powerful recognition capability, but also makes the model sensitive to the training data distribution. Thus, even in the same classification task, a slight change in the training data distribution may have a large effect on the model. Therefore, the model has difficulty in generalizing to new classes or incrementally adding new classes and often must be retrained as a whole.

[61] (4) Network training is computationally expensive. Each fine-tuning of a deep network requires iterative optimization involving a large amount of forward- and back-propagation computation. Consequently, the network has low adaptation efficiency and is difficult to update rapidly online.

[62] (5) Interpretability is poor. A deep network is a black-box system, and its internal representations and bases for prediction are difficult to explain. This limits the trustworthiness of the model.

[63] Taken together, these deficiencies mean that, when a current deep learning-based image recognition system needs to recognize a new class, the entire deep neural network model must be retrained. Retraining a neural network model generally requires a large number of labeled samples, while labeled data for certain rare classes may be very difficult to obtain.

[64] In sharp contrast, humans may learn new knowledge while solving problems and thereby continuously improve. If a network has capability of continuous learning, that is, the capability to learn autonomously while operating online, the algorithm performance may be continuously improved during use. The core of continuous learning is learning from field data obtained during online operation. It is therefore valuable to develop a recognition system capable of rapidly adding a new class using only a small number of samples.

[65] The inventors have found that, in practical security inspection scenarios, the initially required prohibited-item recognition function is often limited to only a few classes, such as knives, firearms, and explosives. As requirements continue to change during field use and unexpected events occur, the classes of objects to be detected must be continuously updated. Repeating the entire process of collecting data, manually labeling the data, adjusting an algorithm, and redeploying the algorithm after each update incurs enormous and unacceptable costs in both time and money. Furthermore, the daily data flow at a detection site is enormous, and field data is highly authentic and diverse. If data for certain fixed classes that require routine detection may be used effectively, system detection performance may be greatly improved. The algorithm may also be adjusted more conveniently according to characteristics of data from the region of use, thereby further optimizing system performance.

[66] On this basis, some exemplary embodiments of the present disclosure provide a method for generating a retriever for a radiation scanning detection system. The method includes: constructing a sample library, where the constructing of the sample library includes obtaining first sample data and second sample data. The first sample data includes a first image and a first label, the first image is a radiation scanning image including at least one first detection target, and the first label indicates a class of the at least one first detection target. The second sample data includes a second image and a second label, the second image is a radiation scanning image including at least one second detection target, and the second label indicates a class of the at least one second detection target. The first sample data is positive sample data, and the second sample data is negative sample data. The method further includes constructing a feature library, where the constructing of the feature library includes: extracting a first image feature of the first image and a second image feature of the second image using a pre-trained feature extraction model; forming a first feature-label association based on the first image feature and the first label and forming a second feature-label association based on the second image feature and the second label; and generating the feature library based on the first feature-label association and the second feature-label association. The method further includes generating the retriever based on the feature library. In embodiments of the present disclosure, it is possible to only construct a few-shot sample library. Feature representations may be obtained through an image feature extraction network using only a small number of samples, and a feature library corresponding to the classes may be constructed.

[67] Some exemplary embodiments of the present disclosure further provide an object detection method for a radiation scanning detection system. The method includes obtaining, by the radiation scanning detection system, a radiation scanning image including at least one detection target; extracting an image feature of the radiation scanning image using a pre-trained feature extraction model; performing a similarity search for the image feature of the radiation scanning image using a retriever generated by the method described above; and in response to the image feature of the radiation scanning image matching an image feature in the feature library, obtaining a class corresponding to the image feature of the radiation scanning image and determining the class to be a class of the at least one detection target included in the radiation scanning image.

[68] In embodiments of the present disclosure, retrieval-augmented detection technology is employed to effectively combine the advantages of a retrieval model and a detection model so as to implement rapid online learning under few-shot conditions. Further, optimizing the sample library based on adversarial samples improves the robustness of model predictions.

[69] The embodiments of the present disclosure are described in detail below using CT scanning detection as an example. It should be understood that the embodiments are not limited to CT scanning detection scenarios and may be applied to various scanning detection scenarios, such as X-ray security inspection and terahertz (millimeter-wave) body security inspection. As another example, the embodiments may be applied to scanning detection scenarios involving various objects under inspection, including, but not limited to, vehicle scanning detection, baggage / parcel scanning detection, and body scanning detection. The scanning detection scenarios described herein are not exhaustive and should not be construed as limiting the scope of the present disclosure.

[70] It should be noted that the embodiments of the present disclosure are not limited to object detection tasks and may also be applied to image classification, image segmentation, and other tasks. The dimensionality of the data representing an object under inspection is likewise not limited. The data may be three-dimensional data used as an example in the embodiments below, two-dimensional data, pseudo-three-dimensional data (depth data), or other data. Thus, unless stated otherwise, the term "image recognition" as used herein includes, but is not limited to, object detection, image classification, image segmentation, and other tasks.

[71] FIG. 1 is a schematic structural diagram of a radiation scanning detection system according to some exemplary embodiments of the present disclosure. A CT scanning apparatus is schematically illustrated in FIG. 1 as an example of the radiation scanning detection system. As shown in FIG. 1, the CT scanning apparatus according to the present embodiment includes a gantry 20, a support mechanism 40, a controller 50, and a data processing device 60 (such as a computer). The gantry 20 includes a radiation source 10, such as an X-ray machine, that emits inspection X-rays, and a detection and acquisition device 30. The support mechanism 40 carries an object under inspection 70 (such as baggage to be inspected) through a scanning region between the radiation source 10 and the detection and acquisition device 30 of the gantry 20, while the gantry 20 rotates about a direction in which the object under inspection 70 advances. Radiation emitted from the radiation source 10 may thereby pass through the object under inspection 70 to perform a CT scan of the object under inspection 70. The detection and acquisition device 30 includes, for example, a detector and a data acquisition device having an integrated modular structure, such as a flat-panel detector, for detecting radiation transmitted through the detection target to obtain an analog signal, converting the analog signal into a digital signal, and outputting projection data of the object under inspection 70 for the X-rays. The controller 50 is configured to control the various parts of the overall system to operate synchronously. The data processing device 60 is configured to process and reconstruct data acquired by the data acquisition device and output a result.

[72] As shown in FIG. 1, the radiation source 10 is disposed on one side of a location at which an object to be inspected may be placed, and the detection and acquisition device 30 is disposed on the other side of the object under inspection 70. The detection and acquisition device 30 includes a detector and a data acquisition device and is configured to obtain transmission data and / or multi-angle projection data of the object under inspection 70. The data acquisition device includes a data amplification and shaping circuit that may operate in a (current) integration mode or a pulse (counting) mode. A data output cable of the detection and acquisition device 30 is connected to the controller 50 and the data processing device 60 so that acquired data is stored in the data processing device 60 in response to a trigger command.

[73] In embodiments of the present disclosure, a radiation scanning detection module may be deployed in the data processing device 60. An image recognition model or an object detection model is deployed in the radiation scanning detection module. The radiation scanning detection module may recognize acquired field data (such as a radiation scanning image) using the image recognition model or object detection model. The recognition may include object detection, such as detection of a target object or an object of interest in the radiation scanning image. For example, the target object or object of interest may be any of various prohibited items. It should be understood that the particular class of the target object or object of interest depends on the scanning detection site at which the radiation scanning detection system is deployed. In other words, the particular class of the target object or object of interest is determined by a user of the radiation scanning detection system according to specific scanning detection needs, and the class may be dynamically adjusted as the scanning detection needs change.

[74] In embodiments of the present disclosure, the radiation source 10 may be, for example, an X-ray machine, and a suitable focal-spot size of the X-ray machine may be selected according to the desired imaging resolution. In other embodiments, a linear accelerator or the like may be used to generate an X-ray beam instead of an X-ray machine.

[75] The detection and acquisition device 30 includes an X-ray detector, a data acquisition circuit, and the like. The X-ray detector may be a solid state detector, a gas detector, or another detector, and the embodiments of the present disclosure are not limited in this respect. The data acquisition circuit includes a readout circuit, an acquisition trigger circuit, a data transmission circuit, and the like.

[76] The combination of the controller 50 and the data processing device 60 may include, for example, a computer device on which a control program and a data processing program are installed. The combination is configured to: control operation of the CT scanning apparatus, including mechanical rotation, electrical control, safety interlock control, and the like; reconstruct a CT image from projection data; train an image recognition model or object detection model; and recognize a radiation scanning image using the trained image recognition model or object detection model.

[77] FIG. 2 is a schematic diagram of a CT scanning apparatus that implements a reconstruction scanning mode according to some exemplary embodiments of the present disclosure. As shown in FIG. 2, after X-rays emitted from the radiation source 10 pass through a portion of the object under inspection 70 in a field of view 45, the X-rays are received by the detection and acquisition device 30, converted into electrical signals, and then converted into digital signals representing attenuation values. The digital signals are used as projection data and reconstructed by a computer to obtain a high-quality image.

[78] According to embodiments of the present disclosure, the CT scanning apparatus described above performs a CT scan of an object under inspection to obtain raw attenuation signals. Attenuation signal data corresponding to one row of detectors may also be displayed as a two-dimensional image. FIG. 3 illustrates an example of attenuation signal data obtained according to an embodiment of the present disclosure. The horizontal axis of the raw attenuation signal image shown in FIG. 3 represents a pixel sequence in a detector column direction (for example, from 1 to 256), and the vertical axis represents angle (for example, from 1 degree to 360 degrees). The raw attenuation signals become projection data after preprocessing. For example, the CT scanning apparatus may perform preprocessing such as a negative logarithmic transformation to obtain the projection data. The data processing device 60 then executes a reconstruction program. For example, a mainstream analytical reconstruction method in the field may be employed to obtain a reconstructed image, which is a radiation scanning image as referred to in embodiments of the present disclosure.

[79] In embodiments of the present disclosure, the data processing device 60 may perform image recognition (such as object detection) on the reconstructed image (that is, the radiation scanning image) to identify an object of interest (such as a prohibited item) in the radiation scanning image. More specifically, a radiation scanning detection module may be deployed in the data processing device 60, and an image recognition model or object detection model may be deployed in the radiation scanning detection module. The radiation scanning detection module may recognize the radiation scanning image using the image recognition model or object detection model.

[80] FIG. 4 is a schematic flowchart of a method for generating a retriever for a radiation scanning detection system according to an embodiment of the present disclosure. In embodiments of the present disclosure, the retriever may be generated in advance and, as a part of a detection model, applied to a radiation scanning detection system (such as the CT scanning apparatus shown in FIG. 1) deployed at a scanning detection site. As the amount of sample data acquired at the scanning detection site increases, the retriever may be updated. By way of example, referring to FIG. 4, the retriever generation method may include operations S410 to S430 described below.

[81] In operation S410, a sample library may be constructed.

[82] FIG. 5 is a schematic flowchart of constructing a sample library according to an embodiment of the present disclosure. By way of example, referring to FIG. 5, constructing the sample library may include sub-operations S4101 to S4102.

[83] In sub-operation S4101, first sample data is obtained. The first sample data includes a first image and a first label. The first image is an image including at least one first detection target, and the first label indicates a class of the at least one first detection target.

[84] In some exemplary embodiments of the present disclosure, the detection and acquisition device 30 includes a detector configured to detect X-rays, and the first image may be a radiation scanning image, such as an X-ray fluoroscopic image or a CT image. Alternatively or additionally, the detection and acquisition device 30 may include a sensor configured to detect visible light, such as an industrial camera, a surveillance camera, or another image sensor, and the first image may be a visible light image.

[85] In some exemplary embodiments of the present disclosure, in sub-operation S4101, a plurality of first images may be obtained for an application scenario of the radiation scanning detection system, where each of the plurality of first images includes at least one first detection target, and the first detection target is a representative detection target suitable for the application scenario of the radiation scanning detection system; classes of the first detection targets in the plurality of first images are labeled to form a plurality of first labels.

[86] In embodiments of the present disclosure, the first sample data may be obtained before the radiation scanning detection system is placed into operational service. As used herein, being "placed into operational service" may be understood to mean that the radiation scanning detection system is deployed at a user's actual detection site and performs actual detection work.

[87] For example, in embodiments of the present disclosure, the detection and acquisition device 30 supports two operating modes: an offline batch acquisition mode and an online acquisition mode. In the offline batch acquisition mode, the radiation scanning detection system may collect samples (such as the first images described above) in advance, and the samples collected in advance are manually labeled. In embodiments of the present disclosure, the samples collected in advance may be a small number of key samples with class labels obtained by labeling a new class. During a subsequent detection stage, the system may rapidly adapt to the new class using the small number of samples of the known class. In the online acquisition mode, the system acquires image inputs in real time for recognition while the system is deployed and in use.

[88] In sub-operation S4102, second sample data is obtained, where the second sample data includes a second image and a second label, the second image is a radiation scanning image including at least one second detection target, and the second label indicates a class of the at least one second detection target. The first sample data is positive sample data, and the second sample data is negative sample data.

[89] In some exemplary embodiments of the present disclosure, the detection and acquisition device 30 includes a detector configured to detect X-rays, and the second image may be a radiation scanning image, such as an X-ray fluoroscopic image or a CT image. Alternatively or additionally, the detection and acquisition device 30 may include a sensor configured to detect visible light, such as an industrial camera, a surveillance camera, or another image sensor, and the second image may be a visible light image.

[90] In some exemplary embodiments of the present disclosure, in sub-operation S4102, a plurality of second images may be obtained according to the first sample data, where each of the plurality of second images includes at least one second detection target that is similar to the first detection target but belongs to a different class from the first detection target; classes of the second detection targets in the plurality of second images are labeled to form a plurality of second labels.

[91] In embodiments of the present disclosure, the first sample data is positive sample data. For example, when the radiation scanning detection system is applied for baggage security inspection or parcel inspection, the positive sample data may be an image including a first detection target, and the first detection target may be an object of interest to the user or an object warranting particular attention in the baggage security inspection or parcel inspection scenario.

[92] FIG. 7 is a schematic diagram of a first image according to some exemplary embodiments of the present disclosure. Referring to FIG. 7, the first image includes first detection targets of two classes: a tablet computer and a wristwatch. In this scenario, the first image feature may include a first annotation box 11 labeling the tablet computer, and the first label may include "tablet computer". The first image feature may further include a first annotation box 12 labeling the wristwatch, and the first label may further include "wristwatch".

[93] FIG. 8 is a schematic diagram of a first image according to other exemplary embodiments of the present disclosure. Referring to FIG. 8, the first image includes first detection targets of three classes: a first class of detection target, a second class of detection target (such as a first knife), and a third class of detection target (such as a second knife). In this scenario, the first image feature may include a first annotation box 13 labeling the second knife, and the first label may include "knife".

[94] It should be noted that, the classes of the detection targets and the annotation boxes illustrated in FIG. 7 and FIG. 8 are merely schematic examples provided to illustrate embodiments of the present disclosure and should not be construed as limiting the embodiments of the present disclosure. In other embodiments, an image may include detection targets of other classes and corresponding annotation boxes.

[95] In embodiments of the present disclosure, the second sample data is negative sample data. For example, when the radiation scanning detection system is used for baggage security inspection or parcel inspection, the negative sample data may be an image including a second detection target, and the second detection target is similar to the first detection target but belongs to a different class from the first detection target. For example, in the scenario shown in FIG. 7, the shape of the second detection target may be the same as or similar to the shape of the tablet computer or wristwatch, while the second detection target is not actually a tablet computer or wristwatch. In the scenario shown in FIG. 8, the shape of the second detection target may be the same as or similar to the shape of a firearm, the first knife (such as a kitchen knife) or the second knife (such as a dagger), while the second detection target is not actually a prohibited item.

[96] In embodiments of the present disclosure, the second sample data is adversarial sample data. By providing the second sample data, it is possible to provide information for controlling a decision boundary of the system. In other words, in embodiments of the present disclosure, for each manually labeled positive sample (that is, the first sample data), an adversarial sample likely to cause a false alarm may be actively added as a negative sample. These adversarial samples may help the detection model better define the decision boundary and improve the robustness of model predictions.

[97] In some exemplary embodiments, obtaining the plurality of second images may include obtaining the plurality of second images by at least one of active acquisition, retrieval of historical data, and synthesis using a generative model. In other words, sources of the adversarial samples may include, but are not limited to, active acquisition by the user, retrieval by the model from historical user data, and data synthesis using a generative model. The generative model may include, but is not limited to, a generative adversarial network (GAN), a diffusion model, or another generative model.

[98] In embodiments of the present disclosure, the second sample data may be obtained before the radiation scanning detection system is placed into operational service.

[99] Referring again to FIG. 4, in operation S420, a feature library may be constructed.

[100] FIG. 6 is a schematic flowchart of constructing a feature library according to an embodiment of the present disclosure. By way of example, referring to FIG. 6, constructing the feature library may include sub-operations S4201 to S4203.

[101] In sub-operation S4201, a first image feature of the first image and a second image feature of the second image are extracted using a pre-trained feature extraction model.

[102] In sub-operation S4202, a first feature-label association is formed based on the first image feature and the first label, and a second feature-label association is formed based on the second image feature and the second label.

[103] In sub-operation S4203, a feature library is generated based on the first feature-label association and the second feature-label association.

[104] Referring again to FIG. 4, in operation S430, a retriever may be generated based on the feature library.

[105] By way of example, referring again to FIG. 5, constructing the sample library may further include sub-operation S4103. In sub-operation S4103, third sample data is obtained, where the third sample data includes a third image and a third label, the third image is a radiation scanning image including at least one third detection target, and the third label indicates a class of the at least one third detection target. The third sample data includes positive sample data and / or negative sample data.

[106] In embodiments of the present disclosure, the third sample data may include false-alarm samples. When a human inspection finds that the detection model has made an incorrect determination for certain samples, the misclassified samples may be saved, labeled, and stored in the sample library as error correction samples. Alternatively or additionally, the third sample data may include missed-detection samples. By way of example, a missed-detection sample may include a dangerous item that should have triggered an alarm but did not trigger an alarm during actual operation. When a human inspection finds such missed-detection samples, the missed-detection samples may be saved, labeled, and stored in the sample library as positive samples. Because these samples are obtained mainly from erroneous cases, they form a sample set that grows during operation. In other words, the third sample data may be obtained during the operational service of the radiation scanning detection system.

[107] More specifically, in sub-operation S4103, detection results of the radiation scanning detection system are spot-checked during the operational service of the radiation scanning detection system; in response to a missed detection and / or false alarm being present in the detection results, an image associated with the missed detection and / or false alarm is obtained as the third image, and classes of the third detection targets in the third images are labeled to form a plurality of third labels.

[108] In embodiments of the present disclosure, according to misidentification cases, difficult samples are selected to generate adversarial samples for feature library augmentation. The adversarial samples are generated to augment the feature library.

[109] In embodiments of the present disclosure, only a few-shot sample library needs to be constructed during generation of the retriever. That is, feature representations may be obtained through an image feature extraction network using only a small number of samples, and a feature library corresponding to the classes may be constructed.

[110] Further, sample optimization is performed based on adversarial samples. Specifically, to improve recognition accuracy, a sample-library optimization method based on adversarial samples is designed. Each time a manually labeled positive sample is added, an adversarial sample that would cause a false alarm under the current library is subsequently added as a negative sample. Such an adversarial sample pair may better define the decision boundary.

[111] By way of example, in sub-operation S4201, a third image feature of the third image is further extracted using the pre-trained feature extraction model. In sub-operation S4202, a third feature-label association is further formed based on the third image feature and the third label. In sub-operation S4203, the feature library is further generated based on the third feature-label association.

[112] It should be noted that, in sub-operation S4203, if no feature library exists and a feature library is to be newly constructed, the feature library may be generated based on at least one of the first feature-label association, the second feature-label association, and the third feature-label association. If a feature library already exists, at least one of the first feature-label association, the second feature-label association, and the third feature-label association may be stored in the existing feature library.

[113] In some exemplary embodiments of the present disclosure, an image preprocessing operation may be performed prior to the feature extraction operation.

[114] For example, the preprocessing operation may include format conversion, size adjustment, and the like. As another example, at least one of the first image, the second image, and the third image may be subjected to preprocessing such as rotation correction and denoising to ensure image quality.

[115] Rotation correction may help rectify image skew. For example, a rotation correction method based on image edge contours may be used to perform rotation correction on at least one of the first image, the second image, and the third image. Specifically, the rotation correction method based on image edge contours may include the following steps.

[116] (1) Image grayscale conversion: converting a color image into a grayscale image to facilitate subsequent processing.

[117] (2) Threshold binarization: converting the grayscale image into a binary image so that a target and a background in the image are clearly distinguishable.

[118] (3) Contour detection: detecting an edge contour of the target in the binary image.

[119] (4) Extracting a bounding rectangle of the contour: identifying a minimum rectangular region that encloses the target contour, where an edge of the rectangular region may be used to estimate an image skew angle.

[120] (5) Obtaining a rotation angle from the extracted bounding rectangle: calculating an angle between a long side of the bounding rectangle and a horizontal line, where the calculated angle is the image skew angle.

[121] (6) Rotating the image using an affine transformation: rotating the image to a normal orientation using an affine transformation according to the calculated skew angle.

[122] For example, a rotation correction method based on a Fourier transform and Hough line detection may be used to perform rotation correction on at least one of the first image, the second image, and the third image. Specifically, the rotation correction method based on Hough line detection may include the following steps.

[123] (1) Image preprocessing and edge detection: first preprocessing the image, for example, by grayscale conversion and binarization, and then detecting edges in the image using an edge detection algorithm (such as a Sobel operator).

[124] (2) Hough transform: the Hough transform is a technique for detecting straight lines and curves in an image and is used here to detect straight lines in the image. The Hough transform is based on point-line duality and converts lines in an image space into accumulation points in a parameter space, thereby detecting whether a curve having a given property exists in a given image.

[125] (3) Calculating a skew angle: the Hough transform provides a polar-coordinate representation (ρ, θ) of each straight line in the image, where θ represents the skew angle of the straight line. The skew angles of all detected straight lines may be statistically analyzed and averaged to obtain the skew angle of the entire image.

[126] (4) Image rotation: rotating the image using an image rotation algorithm (such as an affine transformation) according to the calculated skew angle to correct the image.

[127] (5) Outputting the corrected image.

[128] In some exemplary embodiments, at least one of the first image, the second image, and the third image may be denoised using a filter-based method, a model-based method, or a learning-based method.

[129] Specifically, in a filter-based method, a designed low-pass filter (such as a median filter or a Wiener filter) may be used to remove image noise. For example, a median filter is a nonlinear smoothing filter. Its basic principle is to replace a value of a point in a digital image or digital sequence with the median of values of points in a neighborhood of the point, thereby removing isolated noise points. In a model-based method, a distribution of natural images or noise may be modeled; then the modeled distribution is used as a prior to obtain a clear image and an optimization algorithm. A learning-based method focuses on learning an underlying mapping from noisy images to clean images and may be divided into a conventional learning-based method and a deep network-based learning method.

[130] Examples of filter-based image denoising methods include the following. (1) Mean filter: a mean filter replaces an original pixel value with an average value of neighboring pixels and linearly smooths the entire image. (2) Median filter: a median filter operates in a nonlinear manner and replaces a pixel value with a median value from a convolution kernel, and is suitable for processing salt-and-pepper noise. A main function of the median filter is to replace a pixel having a grayscale value that differs greatly from those of surrounding pixels with a value close to the values of the surrounding pixels, thereby removing isolated noise points. (3) Gaussian filter: a Gaussian filter is a linear filter, and coefficients in its convolution kernel decrease as the distance from the center of the kernel increases. Compared with a mean filter, a Gaussian filter blurs the overall image to a lesser degree and may effectively suppress noise and smooth the image. (4) Bilateral filter: a bilateral filter is a nonlinear filtering method and provides a compromise that combines spatial proximity and pixel-value similarity in an image, thereby preserving edges.

[131] For example, in some exemplary embodiments, at least one of the first image, the second image, and the third image may be denoised using a median filter. The denoising may include the following steps.

[132] (1) Selecting a neighborhood window: for each pixel, selecting a neighborhood window defined as a rectangular region having a size of w x w.

[133] (2) Reading corresponding pixel grayscale values under the kernel: reading corresponding pixel grayscale values under the kernel.

[134] (3) Sorting: sorting the grayscale values in descending order.

[135] (4) Selecting a median value: selecting the grayscale value of a middle pixel in the grayscale value sequence.

[136] (5) Assigning the value to the pixel at the center of the kernel: assigning the median value to the pixel at the center of the kernel.

[137] In embodiments of the present disclosure, in sub-operation S4201, the first image feature of the first image, the second image feature of the second image, and the third image feature of the third image are extracted using the pre-trained feature extraction model.

[138] In embodiments of the present disclosure, a pre-trained model is used to extract features of the sample images, the features are stored in the feature library, and feature-label index associations are constructed to form the retriever. Different image feature representations are stored in the feature library according to different annotation types.

[139] In some exemplary embodiments, a feature extraction model may be pre-trained to extract a feature representation from an input image. The feature extraction model may employ any neural network architecture having general representation capability as a backbone, such as a Transformer architecture including a Vision Transformer, or a convolutional neural network such as ResNet. The model may be pre-trained on a large-scale image dataset to learn general feature representations of images. The generality of the feature extraction model may reduce dependence on the number of samples when the model is updated.

[140] Optionally, the feature extraction model may use a Vision Transformer as the backbone. A Vision Transformer (ViT) is a model that applies a Transformer architecture to image classification. Extracting image features using a ViT may include the following sub-operations.

[141] (1) Image patching: the ViT divides an input image into a plurality of fixed-size patches, linearly embeds each patch, adds a positional embedding, and inputs the resulting vector sequence into a standard Transformer encoder.

[142] (2) Image patch embedding: image patching is a preprocessing procedure. To convert a vector dimension of N x (P^2 x C) into a two-dimensional input of size N x D, an image patch embedding operation is also performed. Similar to word embedding in NLP, patch embedding is also a method of converting a high-dimensional vector into a low-dimensional vector.

[143] (3) Positional encoding: to preserve spatial position information among the input image patches, a positional encoding vector is added to the image patch embeddings.

[144] (4) Transformer encoder: after an embedded input vector integrating class vector appending, image patch embedding, and positional encoding is obtained, the vector may be input directly into a Transformer encoder that mainly includes an MSA portion and an MLP portion.

[145] (5) MLP classification: an MLP is used as a classification head.

[146] This approach has the advantages of effectively extracting image features and being highly robust to complex images and noise.

[147] It should be noted that, in deep learning, a "backbone" generally refers to a main portion of a neural network, namely, the network used for feature extraction. The backbone network is typically a feature extraction network whose function is to extract information from an image for use by a subsequent network. A "patch" as used herein refers to a small block into which an input image is divided. When a Vision Transformer (ViT) is used to process an image, the input image is divided into a plurality of fixed-size patches, and each patch is processed. This reduces computational complexity while preserving local image characteristics.

[148] Optionally, the feature extraction model may employ ResNet as the backbone. ResNet, namely a residual network, is a deep convolutional neural network whose main characteristic is the introduction of skip connections (also known as residual connections or shortcut connections). The specific operations for extracting image features using ResNet may include the following sub-operations.

[149] (1) Convolutional layers: a basic building block of ResNet is a residual block. Each residual block includes two or three convolutional layers configured to extract image features.

[150] (2) Residual connection: in each residual block, an input is directly connected to an output through a skip connection. The skip connection allows the network to learn a residual function between the input and the output, thereby effectively mitigating vanishing gradient and representational bottleneck problems in deep neural networks.

[151] (3) Pooling layer: a pooling layer is typically provided after a convolutional layer to reduce feature dimensionality and decrease computational cost.

[152] (4) Fully connected layer: one or more fully connected layers are typically provided at the end of the network to map the extracted features to the target classes.

[153] (5) Activation function: an activation function, such as ReLU, is typically provided after a convolutional layer and fully connected layer to increase the nonlinear representation capability of the network.

[154] (6) Batch normalization: a batch normalization layer is typically provided after a convolutional layer to accelerate network training and prevent overfitting.

[155] In embodiments of the present disclosure, image feature extraction is intended to extract useful data or information from an image to obtain a "non-image" representation or description of the image, such as a numerical value, vector, or symbol. For example, the image features may include, but are not limited to: (1) an edge, which is a location at which image brightness changes sharply and generally corresponds to an object boundary; (2) texture, which reflects microscopic structure and local illumination changes in an image; (3) color, which is an intuitive image feature important to object recognition and classification; (4) shape, which describes an external-form characteristic of an object and is an important basis for object recognition; and (5) a corner, which is a location at which image grayscale changes sharply and has high stability.

[156] In embodiments of the present disclosure, the first image feature includes at least one of a sparse feature and a dense feature; and / or the second image feature includes at least one of a sparse feature and a dense feature; and / or the third image feature includes at least one of a sparse feature and a dense feature.

[157] By way of example, the sparse feature includes a region feature represented using bounding box annotation; and / or the dense feature includes a region feature represented using pixel-level mask annotation.

[158] Specifically, bounding box annotation may employ a rectangular box to represent a position of a detection target in an image. For example, the rectangular box may be represented by coordinates of an upper-left corner and a lower-right corner or by a center, width, and height. This annotation method may help a computer understand the position and size of an object in an image for further image analysis and processing. In this case, each image may include a plurality of object instances, and each instance has a respective bounding box. For each bounding box region, a corresponding region feature is extracted using a pre-trained network and stored in the feature library. Consequently, each object instance corresponds to a sparse local feature. During prediction, a feature is extracted for each detected candidate bounding box and matched against features in the feature library.

[159] For pixel-level mask annotation, such fine-grained annotation may delineate each semantic region in an image, and dense feature extraction may be used. Specifically, the image is divided into a grid (such as a 16 x 16 grid), each grid cell is treated as an image patch, and features are extracted from each image patch using the pre-trained network. A dense image-patch feature map is thereby obtained for the image. Such a feature map contains abundant local detail information. The feature map is stored directly when the features are constructed. During prediction, a dense feature map is likewise extracted from an input image and compared with feature maps in the feature library.

[160] In embodiments of the present disclosure, a retriever that includes a feature library may be generated by the method described above. The retriever may be independently deployed in the radiation scanning detection system. For example, the retriever may be deployed in the radiation scanning detection system in the form of a software module including the feature library, or may be deployed in the radiation scanning detection system in the form of independent hardware.

[161] The retriever according to embodiments of the present disclosure is an online learning system capable of rapidly adding a new class using only a small number of labeled samples. The retriever has at least one of the following advantages.

[162] (1) A low sample volume requirement: a new class may be recognized without a large number of labeled samples, substantially reducing the manual labeling workload.

[163] (2) Rapid adaptation: a new class may be recognized in a very short time according to a small number of new samples, without repeated network training.

[164] (3) High accuracy: recognition accuracy is not significantly reduced by a small sample volume and remains at an acceptable level.

[165] (4) Ease of deployment: the retriever may be integrated as an independent module into an existing system without modifying the overall pipeline.

[166] (5) High scalability: new classes may be continuously added to the system without a rapid decrease in performance as the number of classes increases.

[167] The methods provided in embodiments of the present disclosure combine the advantages of conventional machine learning and deep learning technologies. A shared pre-trained feature model reduces the sample volume requirement; nearest neighbor search enables rapid adaptation to new classes without repeated training of a deep network; adversarial samples improve the accuracy of the decision boundary. These techniques effectively address the difficulty that current deep learning-based image recognition systems have in handling new classes and will promote the practical application of the technology in more scenarios.

[168] On this basis, embodiments of the present disclosure further provide an object detection method for a radiation scanning detection system. FIG. 9 is a schematic flowchart of an object detection method for a radiation scanning detection system according to an embodiment of the present disclosure. By way of example, referring to FIG. 9, the object detection method may include operations S910 to S940 described below.

[169] In operation S910, a radiation scanning image is acquired by the radiation scanning detection system, where the radiation scanning image includes at least one detection target.

[170] By way of example, after the radiation scanning detection system shown in FIG. 1 is deployed at an inspection site, the radiation scanning detection system may be controlled to enter operational service. Specifically, the support mechanism 40 carries an object under inspection (such as baggage to be inspected) 70 to pass through a scanning region between the radiation source 10 and the detection and acquisition device 30 of the gantry 20, while the gantry 20 rotates about the direction in which the object under inspection 70 advances. Radiation emitted from the radiation source 10 may thereby pass through the object under inspection 70 to perform a CT scan of the object under inspection 70. The detection and acquisition device 30 includes a detector configured to detect X-rays. The detector detects radiation emitted from the radiation source 10 and passed through the object under inspection 70 to acquire a radiation scanning image, such as an X-ray fluoroscopic image or a CT image.

[171] In this operation, during the acquisition of the radiation scanning image, metadata corresponding to the radiation scanning image is also acquired. The metadata includes the time at which the image was acquired and / or device parameters to acquire the image, which may facilitate subsequent processing.

[172] In operation S920, an image feature of the radiation scanning image is extracted using a pre-trained feature extraction model.

[173] In some exemplary embodiments of the present disclosure, the feature extraction model used in operation S920 is the same as the feature extraction model used in the retriever generation method described above.

[174] In embodiments of the present disclosure, the retriever generation stage and the current detection stage use a unified image feature representation. A general-purpose feature extraction network is obtained by pre-training on a large number of images. The network does not need to be fine-tuned for a particular class and learns to extract general feature representations of images to provide a unified feature description for images of different classes. The feature extraction thereby supports nearest neighbor-based search.

[175] In operation S930, a similarity search is performed with respect to the image feature of the radiation scanning image by using a retriever generated by a method according to any embodiment described above.

[176] In some exemplary embodiments of the present disclosure, the similarity search may include searching the feature library of the retriever, using a nearest neighbor search algorithm, for an image feature matching the image feature of the radiation scanning image.

[177] Nearest neighbor search is a method of finding, in a dataset, a data point closest to a given target. By way of example, the nearest neighbor search may use array sorting or a hierarchical navigable small world (HNSW) search algorithm to provide search and recognition with real-time responsiveness.

[178] Specifically, nearest neighbor search using array sorting may include the following main steps.

[179] (1) Calculating distances between the scanning image feature and features in the feature library. For example, an L2 distance or a cosine distance between the scanning image feature and each feature in the feature library may be calculated.

[180] (2) Sorting the distances calculated in step (1).

[181] (3) Selecting the features corresponding to the shortest k distances as k nearest neighbors.

[182] Specifically, HNSW is a graph-based approximate nearest neighbor search algorithm having extremely high search speed and excellent recall. Nearest neighbor search algorithm using HNSW may include the following main steps.

[183] (1) Graph construction: the HNSW algorithm first constructs a hierarchical graph where each layer is a proximity graph in which two vertices are connected according to their proximity (i.e., closer vertices are connected). Vectors are inserted one by one during graph construction.

[184] (2) Search: during the search phase, the process enters the top layer and identifies the longest links. The corresponding vertices tend to be high-level vertices (having links spanning multiple layers), which means that the search begins by default at a coarse stage of an NSW graph. The search traverses edges in each layer as in an NSW graph and greedily moves to the nearest vertex until reaching a local minimum. Unlike an NSW search, the HNSW search then moves to the current vertex in the next lower layer and resumes searching. This process is repeated until a local minimum is found in the bottom layer ( layer 0).

[185] In embodiments of the present disclosure, classification is implemented based on nearest neighbor search. In other words, an input image is processed by the same feature extraction network to obtain a feature representation. A nearest neighbor search is then performed against samples of the respective classes in the feature library, and a final class label is determined from the search result. This operation has at least one of the following advantages.

[186] (1) Interpretable similarity search. The retrieval model outputs the most similar sample, thereby providing an intuitive explanation for the basis of classification. An end-to-end neural network, by contrast, has poor interpretability. Multi-evidence aggregation is also supported. Instead of depending on a single nearest neighbor, classification aggregates a plurality of neighboring "witnesses" and makes a multi-evidence decision through similarity voting, thereby improving robustness.

[187] (2) An optional efficient nearest neighbor index. An efficient algorithm, such as HNSW, may be used to index features in a knowledge base in advance, thereby ensuring time and space efficiency for large-scale searches. Compression may also be used to reduce storage and search overhead.

[188] In some exemplary embodiments, image features in the feature library may include sparse features and dense features. For example, the sparse features include region features represented using bounding box annotations, and the dense features include region features represented using pixel-level mask annotations.

[189] Optionally, in operation S930, the similarity search includes: with respect to the image feature of the radiation scanning image, searching the feature library of the retriever only to determine whether there exists a sparse feature matching the image feature of the radiation scanning image.

[190] Optionally, in operation S930, the similarity search includes: with respect to the image feature of the radiation scanning image, searching the feature library of the retriever only for a dense feature matching the image feature of the radiation scanning image.

[191] Optionally, in operation S930, the similarity search includes: with respect to the image feature of the radiation scanning image, searching the feature library of the retriever to determine whether there exists a sparse feature matching the image feature of the radiation scanning image, and in response to existence of a sparse feature matching the image feature of the radiation scanning image, further searching the feature library of the retriever for a dense feature matching the image feature of the radiation scanning image.

[192] In embodiments of the present disclosure, support for multiple types of feature representations allows a feature library to be constructed according to different annotation granularities. Dense features contain richer details but occupy more storage, while sparse features are more compact but have weaker fitting capability. A feature type may be selected according to actual annotation conditions and storage constraints. Meanwhile, multiple feature types may also be combined for cascaded retrieval, thereby ensuring accuracy while controlling overhead.

[193] In operation S940, in response to the image feature of the radiation scanning image matching an image feature in the feature library, a class corresponding to the image feature of the radiation scanning image is obtained and determined to be a class of the at least one detection target included in the radiation scanning image.

[194] In other words, in embodiments of the present disclosure, a nearest labeled sample to the feature vector of a query image is found in the feature library, and the class of the nearest labeled sample is returned.

[195] FIG. 10 is a schematic flowchart of an object detection method for a radiation scanning detection system according to an embodiment of the present disclosure, in which a retriever generation process and an object detection process are schematically illustrated. By way of example, referring to FIG. 10, the object detection method may include operations S1010 to S1070 described below.

[196] In operation S1010, a labeled sample is obtained. For example, the labeled sample may include the first sample data, second sample data, and third sample data described above, and may be in the form of an image and / or voxels.

[197] In operation S1020, the labeled sample is input into an encoder to extract a feature vector. For example, the encoder may include the feature extraction model described above.

[198] In operation S1030, a feature library is generated based on a feature-label mapping relationship, or the feature-label mapping relationship is stored in the feature library.

[199] In operation S1040, a sample to be detected is obtained. For example, the sample to be detected includes a radiation scanning image and may be in the form of an image and / or voxels.

[200] In operation S1050, the sample to be detected is input into an encoder to extract a feature vector. For example, the encoder may include the feature extraction model described above.

[201] In operation S1060, a similarity search is performed for the feature of the radiation scanning image using the retriever.

[202] In operation S1070, in response to the image feature of the radiation scanning image matching an image feature in the feature library, a class of the detection target is output.

[203] In some exemplary embodiments of the present disclosure, the object detection method may further include: based on the acquired radiation scanning image and the class of the at least one detection target included in the radiation scanning image, fusing the radiation scanning image with the class of the at least one detection target to form a fusion image.

[204] Specifically, after the class of the at least one detection target is determined in operation S940 by retrieval-feature matching, the determined class of the detection target may be represented in the radiation scanning image by text, color, an annotation, or the like. In this manner, the radiation scanning image and the class of the at least one detection target are fused to form the fusion image.

[205] In some exemplary embodiments of the present disclosure, the object detection method may further include: based on the acquired radiation scanning image and the class of the at least one detection target included in the radiation scanning image, determining a physical property of the at least one detection target according to the class; and fusing the radiation scanning image based on the physical property and the identified at least one detection target to form a fusion image, where a three-dimensional identification result for the at least one detection target is displayed in the three-dimensional scanning image.

[206] For example, the physical property may include, but is not limited to, a density, an atomic number, or the like of the detection target. Specifically, after the class of the at least one detection target is determined in operation S940 by retrieval-feature matching, a value of a physical property, such as the density or atomic number of a material corresponding to the class, may be obtained from prior knowledge or industry knowledge. The physical property may then be represented in the radiation scanning image by text, color, an annotation, or the like. In this manner, the radiation scanning image, the class of the at least one detection target, and the physical property are fused to form a fusion image.

[207] In this embodiment, the three-dimensional identification result may be optimized based on the physical property in the fusion image. For example, when a three-dimensional mask in a fused scanning image corresponds to a suspected class of all-metal prohibited item, voxels whose corresponding physical properties do not conform to those of metal may be filtered out, thereby making the identification mask more compact and precise.

[208] In some exemplary embodiments of the present disclosure, the radiation scanning detection system includes a multi-view scanning detection system.

[209] The object detection method may further include: based on the acquired radiation scanning image and the class of the at least one detection target included in the radiation scanning image, forming two-dimensional scanning images from a plurality of views; and performing three-dimensional fusion on the two-dimensional scanning images from the plurality of views based on geometric relationships and the identified at least one detection target to form a three-dimensional scanning image in which a three-dimensional identification result for the at least one detection target is displayed.

[210] By way of example, the object detection method may further include: based on the acquired radiation scanning image and the class of the at least one detection target included in the radiation scanning image, forming two-dimensional scanning images from a plurality of views; and obtaining a three-dimensional identification result for the at least one detection target based on geometric relationships and the identified at least one detection target without forming a three-dimensional scanning image by fusion. For example, the two-dimensional scanning images from the plurality of views may each identify a two-dimensional identification result for the same detection target. Based on geometric relationships among the two-dimensional scanning images from the plurality of views and the two-dimensional identification results, a three-dimensional identification result for the detection target may be calculated without forming a three-dimensional scanning image by fusion.

[211] Specifically, after the class of the at least one detection target is determined in operation S940 by retrieval-feature matching, two-dimensional scanning images from a plurality of views and a class of a detection target in each two-dimensional scanning image may be obtained. Three-dimensional image reconstruction may then be performed by converting the two-dimensional images into a three-dimensional image using a three-dimensional reconstruction algorithm. In this manner, the detection target and its relationship to connected structures may be displayed stereoscopically from multiple angles, thereby addressing deficiencies of two-dimensional images.

[212] Embodiments of the present disclosure may implement retrieval-detection fusion. Specifically, a fusion strategy may be designed according to task similarities and task differences and may be optimized for a device. For example, multi-view projection data may be subjected to geometry-based three-dimensional fusion, and an industry knowledge-based optimization may be performed on the three-dimensional fusion result, thereby ultimately forming an enhanced detection output.

[213] In embodiments of the present disclosure, the feature library and the retriever may be provided by a developer, and features and labels related to the third sample data in the feature library may be updated online. The retriever is deployed in a scanning detection system. In practical use, the radiation scanning detection system in which the retriever is deployed is arranged at a scanning detection site for use by a user. For example, the scanning detection site may be a security inspection site at an airport or subway station, or an inspection site at a port, wharf, or vehicle checkpoint. Embodiments of the present disclosure do not particularly limit the scanning detection site.

[214] Embodiments of the present disclosure provide a retrieval-augmented detection scheme that mainly uses the following technical measures.

[215] (1) Pre-training a general-purpose image feature model: a general-purpose feature extraction model is trained in advance using large-scale image data to learn image representations. Parameters of the model may be fixed, and the model may be used to extract features of different new classes.

[216] (2) Extracting and storing few-shot features: features of a new class may be extracted from the pre-trained model using only a small number of samples of the new class. The features and their annotations form a feature library for the class.

[217] (3) Nearest neighbor-based classification: a feature of an input image is extracted using the same model, and a nearest neighbor sample is searched for in the feature library to determine the most likely class.

[218] (4) Adversarial sample augmentation: the feature library is optimized by adding adversarial samples, thereby increasing the discriminability of samples in the feature library.

[219] In embodiments of the present disclosure, retrieval-augmented detection effectively combines the advantages of a retrieval model and a detection model to implement rapid online learning under few-shot conditions. Optimizing the sample library based on adversarial samples also improves the robustness of model predictions.

[220] Retrieval-augmented detection according to embodiments of the present disclosure has at least one of the following main advantages.

[221] (1) Improved few-shot adaptability: the retrieval model may construct a feature library using only a small number of samples and does not require large-scale labeled data for training.

[222] (2) Faster model testing: after the feature library and retriever are constructed offline, only a rapid similarity search is required during the prediction stage (that is, the detection stage), thereby avoiding repeated execution of a complex neural network.

[223] (3) Improved detection model predictions: boundary information provided by adversarial samples may help correct erroneous model predictions and improve system robustness.

[224] (4) Lower deployment and use barriers: the retrieval model may be independent of a particular detection model and may be migrated and extended more easily.

[225] Retrieval-augmented detection according to embodiments of the present disclosure combines the efficient adaptability of a retrieval model (that is, a retriever) with the accuracy and reliability of prior knowledge. It effectively addresses the dependence of current deep learning systems on large amounts of data and the poor interpretability of such systems, and promotes the development of detection systems toward few-shot operation, high interpretability, and high robustness.

[226] Based on the methods described above, embodiments of the present disclosure further provide an apparatus for generating a retriever for a radiation scanning detection system. The apparatus is described in detail below with reference to FIG. 11.

[227] FIG. 11 is a structural block diagram of an apparatus for generating a retriever for a radiation scanning detection system according to an exemplary embodiment of the present disclosure. Referring to FIG. 11, the apparatus 1100 includes a first construction module 1110, a second construction module 1120, and a retriever generation module 1130.

[228] The first construction module 1110 is configured to construct a sample library. In some exemplary embodiments, the first construction module 1110 may be configured to perform operation S410 and its sub-operations described above, and repeated description thereof is omitted.

[229] The first construction module 1110 may include a first data obtaining sub-module 1111 and a second data obtaining sub-module 1112.

[230] The first data obtaining sub-module 1111 is configured to obtain first sample data, where the first sample data includes a first image and a first label, the first image is a radiation scanning image including at least one first detection target, and the first label indicates a class of the at least one first detection target. In some exemplary embodiments, the first data obtaining sub-module 1111 may be configured to perform operation S4101 and its sub-operations described above, and repeated description thereof is omitted.

[231] The second data obtaining sub-module 1112 is configured to obtain second sample data, where the second sample data includes a second image and a second label, the second image is a radiation scanning image including at least one second detection target, and the second label indicates a class of the at least one second detection target. The first sample data is positive sample data, and the second sample data is negative sample data. In some exemplary embodiments, the second data obtaining sub-module 1112 may be configured to perform operation S4102 and its sub-operations described above, and repeated description thereof is omitted.

[232] Optionally, the first construction module 1110 may further include a third data obtaining sub-module 1113.

[233] The third data obtaining sub-module 1113 is configured to obtain third sample data, where the third sample data includes a third image and a third label, the third image is a radiation scanning image including at least one third detection target, and the third label indicates a class of the at least one third detection target. The third sample data includes positive sample data and / or negative sample data. In some exemplary embodiments, the third data obtaining sub-module 1113 may be configured to perform operation S4103 and its sub-operations described above, and repeated description thereof is omitted.

[234] The second construction module 1120 is configured to construct a feature library. In some exemplary embodiments, the second construction module 1120 may be configured to perform operation S420 and its sub-operations described above, and repeated description thereof is omitted.

[235] The second construction module 1120 may include an extraction sub-module 1121, an association forming sub-module 1122, and a feature library generation sub-module 1123.

[236] The extraction sub-module 1121 is configured to extract a first image feature of the first image and a second image feature of the second image using a pre-trained feature extraction model. In some exemplary embodiments, the extraction sub-module 1121 may be configured to perform operation S4201 and its sub-operations described above, and repeated description thereof is omitted.

[237] The association forming sub-module 1122 is configured to form a first feature-label association based on the first image feature and the first label and to form a second feature-label association based on the second image feature and the second label. In some exemplary embodiments, the association forming sub-module 1122 may be configured to perform operation S4202 and its sub-operations described above, and repeated description thereof is omitted.

[238] The feature library generation sub-module 1123 is configured to generate a feature library based on the first feature-label association and the second feature-label association. In some exemplary embodiments, the feature library generation sub-module 1123 may be configured to perform operation S4203 and its sub-operations described above, and repeated description thereof is omitted.

[239] The retriever generation module 1130 is configured to generate a retriever based on the feature library. In some exemplary embodiments, the retriever generation module 1130 may be configured to perform operation S430 and its sub-operations described above, and repeated description thereof is omitted.

[240] FIG. 12 is a structural block diagram of an object detection apparatus for a radiation scanning detection system according to an exemplary embodiment of the present disclosure. As shown in FIG. 12, the object detection apparatus 1200 includes an image acquisition module 1210, a feature extraction module 1220, a retrieval module 1230, and an object detection module 1240.

[241] The image acquisition module 1210 is configured to acquire, by the radiation scanning detection system, a radiation scanning image including at least one detection target. In some exemplary embodiments, the image acquisition module 1210 may be configured to perform operation S910 and its sub-operations described above, and repeated description thereof is omitted.

[242] The feature extraction module 1220 is configured to extract an image feature of the radiation scanning image using a pre-trained feature extraction model. In some exemplary embodiments, the feature extraction module 1220 may be configured to perform operation S920 and its sub-operations described above, and repeated description thereof is omitted.

[243] The retrieval module 1230 is configured to perform a similarity search for the image feature of the radiation scanning image using a retriever generated by the methods described above. In some exemplary embodiments, the retrieval module 1230 may be configured to perform operation S930 and its sub-operations described above, and repeated description thereof is omitted.

[244] The object detection module 1240 is configured to, in response to the image feature of the radiation scanning image matching an image feature in the feature library, obtain a class corresponding to the image feature of the radiation scanning image and determine the class to be a class of the at least one detection target included in the radiation scanning image. In some exemplary embodiments, the object detection module 1240 may be configured to perform operation S940 and its sub-operations described above, and repeated description thereof is omitted.

[245] According to embodiments of the present disclosure, any number of the first construction module 1110, the second construction module 1120 and the retriever generation module 1130 included in the apparatus 1100 and their respective sub-modules, as well as any number of the image acquisition module 1210, the feature extraction module 1220, the retrieval module 1230 and the object detection module 1240 included in the object detection apparatus 1200 and their sub-modules, may be combined into one module for implementation, or any one of the modules may be split into a plurality of modules for implementation. Alternatively, at least part of the functions of one or more of these modules may be combined with at least part of the functions of other modules and implemented in one module. According to embodiments of the present disclosure, at least one of the first construction module 1110, the second construction module 1120 and the retriever generation module 1130 included in the apparatus 1100 and their respective sub-modules, as well as at least one of the image acquisition module 1210, the feature extraction module 1220, the retrieval module 1230 and the object detection module 1240 included in the object detection apparatus 1200 and their sub-modules, may be implemented at least partially as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or may be implemented by hardware or firmware in any other reasonable manner of integrating or encapsulating the circuit, or may be implemented by any one of the three implementation modes of software, hardware and firmware or an appropriate combination thereof. Alternatively, at least one of the first construction module 1110, the second construction module 1120 and the retriever generation module 1130 included in the apparatus 1100 and their respective sub-modules, as well as at least one of the image acquisition module 1210, the feature extraction module 1220, the retrieval module 1230 and the object detection module 1240 included in the object detection apparatus 1200 and their sub-modules, may be at least partially implemented as a computer program module that may perform corresponding functions when executed.

[246] FIG. 13 schematically shows a structural block diagram of an electronic device suitable for implementing the retriever generation method or the object detection method according to exemplary embodiments of the present disclosure.

[247] As shown in FIG. 13, an electronic device 1300 according to embodiments of the present disclosure includes a processor 1301, which may execute various appropriate actions and processing according to the program stored in a read only memory (ROM) 1302 or the program loaded into a random access memory (RAM) 1303 from a storage part 1308. The processor 1301 may include, for example, a general-purpose microprocessor (for example, CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application specific integrated circuit (ASIC)), and the like. The processor 1301 may further include an on-board memory for caching purposes. The processor 1301 may include a single processing unit or multiple processing units for executing different actions of the method flow according to embodiments of the present disclosure.

[248] For example, the electronic device may be a data processing device of the radiation scanning detection system.

[249] Various programs and data required for the operation of the device 1300 are stored in the RAM 1303. The processor 1301, the ROM 1302 and the RAM 1303 are connected to each other through a bus 1304. The processor 1301 executes various operations of the method flow according to embodiments of the present disclosure by executing the programs in the ROM 1302 and / or the RAM 1303. It should be noted that the program may also be stored in one or more memories other than the ROM 1302 and the RAM 1303. The processor 1301 may also execute various operations of the method flow according to embodiments of the present disclosure by executing the programs stored in the one or more memories.

[250] According to embodiments of the present disclosure, the electronic device 1300 may further include an input / output (I / O) interface 1305 which is also connected to the bus 1304. The device 1300 may further include one or more of the following components connected to the I / O interface 1305: an input part 1306 including a keyboard, a mouse, etc.; an output part 1307 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage part 1308 including a hard disk, etc.; and a communication part 1309 including a network interface card such as a LAN card, a modem, and the like. The communication part 1309 performs communication processing via a network such as the Internet. A drive 1310 is also connected to the I / O interface 1305 as required. A removable medium 1311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and the like, is installed on the drive 1310 as required, so that the computer program read therefrom is installed into the storage part 1308 as needed.

[251] The present disclosure further provides a computer-readable storage medium, which may be included in the apparatus / device / system described in the above embodiments; or exist alone without being assembled into the apparatus / device / system. The above-mentioned computer-readable storage medium carries one or more programs that when executed, perform the methods according to embodiments of the present disclosure.

[252] According to embodiments of the present disclosure, the computer-readable storage medium may be a non-transitory computer-readable storage medium, for example, may include but not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer-readable storage medium may be any tangible medium that contains or stores programs that may be used by or in combination with an instruction execution system, apparatus or device. For example, according to embodiments of the present disclosure, the computer-readable storage medium may include the above-mentioned ROM 1302 and / or RAM 1303 and / or one or more memories other than the ROM 1302 and RAM 1303.

[253] Embodiments of the present disclosure further include a computer program product, which contains a computer program. The computer program contains program code for performing the method shown in the flowchart. When the computer program product runs in a computer system, the program code causes the computer system to implement the methods provided in embodiments of the present disclosure.

[254] When the computer program is executed by the processor 1301, the above-mentioned functions defined in the system / apparatus of the embodiments of the present disclosure are performed. According to the embodiments of the present disclosure, the above-described systems, apparatuses, modules, units, etc. may be implemented by computer program modules.

[255] In an embodiment, the computer program may rely on a tangible storage medium such as an optical storage device and a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, downloaded and installed through the communication part 1309, and / or installed from the removable medium 1311. The program code contained in the computer program may be transmitted by any suitable network medium, including but not limited to a wireless one, a wired one, or any suitable combination of the above.

[256] In such an embodiment, the computer program may be downloaded and installed from the network through the communication part 1309, and / or installed from the removable medium 1311. When the computer program is executed by the processor 1301, the above-mentioned functions defined in the system of embodiments of the present disclosure are performed. According to embodiments of the present disclosure, the above-described systems, devices, apparatuses, modules, units, etc. may be implemented by computer program modules.

[257] According to embodiments of the present disclosure, the program code for executing the computer programs provided by the embodiments of the present disclosure may be written in any combination of one or more programming languages. In particular, these computing programs may be implemented using high-level procedures and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages include, but are not limited to, Java, C++, Python, "C" language or similar programming languages. The program code may be completely executed on the user computing device, partially executed on the user device, partially executed on the remote computing device, or completely executed on the remote computing device or server. In a case of involving a remote computing device, the remote computing device may be connected to a user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[258] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions, and operations of the system, method, and computer program product according to various embodiments of the present disclosure. In this regard, each block in the flowcharts or block diagrams may represent a part of a module, a program segment, or a code, which part includes one or more executable instructions for implementing the specified logical function. It should be further noted that, in some alternative implementations, the functions noted in the blocks may also occur in a different order from that noted in the accompanying drawings. For example, two blocks shown in succession may actually be executed substantially in parallel, or they may sometimes be executed in a reverse order, depending on the functions involved. It should be further noted that each block in the block diagrams or flowcharts, and the combination of blocks in the block diagrams or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[259] The retriever generation method and apparatus, the object detection method and apparatus, the electronic device, the radiation scanning detection system, the computer-readable storage medium, and the program product provided in embodiments of the present disclosure have at least one of the following advantageous effects.

[260] (1) A low training-sample requirement and easy system expansion. The retrieval model (that is, the retriever) may construct a feature library using only a small number of samples, avoiding the dependence of a deep neural network on a large number of labeled samples and substantially reducing manual labeling. According to such methods, classes may be continuously added to the class library, and the system may progressively evolve to support recognition of more classes. The system therefore has good scalability, while the cost of updating the application system is reduced.

[261] (2) Rapid adaptation. After the feature library is constructed offline, only the feature library needs to be updated for real-time prediction, and a new class may be recognized through sample search. The time cost of repeated network training is avoided, greatly accelerating adaptation to a new class.

[262] (3) High accuracy. An adversarial sample strategy may better optimize the feature library and thereby improve recognition accuracy. Prediction robustness is improved so that the overall false-alarm rate of the system may be maintained at a desirable level even under few-shot conditions.

[263] (4) Strong interpretability and user-friendly interaction. The retrieval model may output the most similar sample, intuitively explaining the basis for classification. The feature library may be continuously enhanced according to feedback to implement human-machine collaboration. System interpretability and trustworthiness are thereby improved.

[264] (5) Modular design and convenient deployment. The retrieval model is decoupled from the detection model, and different technical solutions may be selected for the two models as needed. System modularity is thereby improved, facilitating migration and upgrades. The retrieval model may also be connected as an independent module to a related system to provide incremental adaptation to a new class.

[265] Embodiments of the present disclosure have been described above. These embodiments are provided for purposes of illustration and are not intended to limit the scope of the present disclosure. Although the embodiments have been described separately above, the measures in the respective embodiments may advantageously be used in combination. The scope of the present disclosure is defined by the appended claims and their equivalents. Those of ordinary skill in the art may make various substitutions and modifications without departing from the scope of the present disclosure, and such substitutions and modifications should fall within the scope of the present disclosure.What is claimed is: 1.A method for generating a retriever for a radiation scanning detection system, comprising:constructing a sample library, comprising:obtaining first sample data, wherein the first sample data comprises a first image and a first label, the first image is an image comprising at least one first detection target, and the first label indicates a class of the at least one first detection target; andobtaining second sample data, wherein the second sample data comprises a second image and a second label, the second image is an image comprising at least one second detection target, and the second label indicates a class of the at least one second detection target, wherein the first sample data is positive sample data, and the second sample data is negative sample data;constructing a feature library, comprising:extracting a first image feature of the first image and a second image feature of the second image by using a pre-trained feature extraction model;forming a first feature-label association based on the first image feature and the first label, and forming a second feature-label association based on the second image feature and the second label; andgenerating the feature library based on the first feature-label association and the second feature-label association; andgenerating the retriever based on the feature library. 2.The method of claim 1, wherein the constructing a sample library further comprises:obtaining third sample data, wherein the third sample data comprises a third image and a third label, the third image is a radiation scanning image comprising at least one third detection target, the third label indicates a class of the at least one third detection target, and the third sample data comprises positive sample data and / or negative sample data. 3.The method of claim 2, wherein the constructing a feature library further comprises:extracting a third image feature of the third image by using the pre-trained feature extraction model;forming a third feature-label association based on the third image feature and the third label; andstoring the third feature-label association into the feature library. 4.The method of claim 3, wherein:the first image feature comprises at least one of a sparse feature and a dense feature; and / orthe second image feature comprises at least one of a sparse feature and a dense feature; and / orthe third image feature comprises at least one of a sparse feature and a dense feature. 5.The method of claim 4, wherein:the sparse feature comprises a region feature represented using bounding box annotation; and / or the dense feature comprises a region feature represented using pixel-level mask annotation. 6.The method of any one of claims 1 to 5, wherein the obtaining first sample data comprises:obtaining a plurality of first images for an application scenario of the radiation scanning detection system, wherein each of the plurality of first images comprises at least one first detection target, and the first detection target is a representative detection target suitable for the application scenario of the radiation scanning detection system; andlabeling the class of the first detection target in the plurality of first images to form a plurality of first labels. 7.The method of any one of claims 1 to 5, wherein the obtaining second sample data comprises:obtaining a plurality of second images according to the first sample data, wherein each of the plurality of second images comprises at least one second detection target, the second detection target is similar to the first detection target, and the class of the second detection target is different from the class of the first detection target; andlabeling the class of the second detection target in the plurality of second images to form a plurality of second labels. 8.The method of claim 7, wherein the obtaining a plurality of second images comprises:obtaining the plurality of second images by at least one of active acquisition, retrieval of historical data, and synthesis using a generative model. 9.The method of any one of claims 1 to 5, wherein the first sample data and the second sample data are obtained before the radiation scanning detection system is placed into operational service. 10.The method of any one of claims 2 to 5, wherein the third sample data is obtained during operational service of the radiation scanning detection system. 11.The method of claim 10, wherein the obtaining third sample data comprises:spot-checking detection results of the radiation scanning detection system during the operational service of the radiation scanning detection system;in response to a missed detection and / or a false alarm existing in the detection results, obtaining an image associated with the missed detection and / or the false alarm as the third image; andlabeling the class of the third detection target in the third images to form a plurality of third labels. 12.An object detection method for a radiation scanning detection system, comprising:acquiring a radiation scanning image by the radiation scanning detection system, wherein the radiation scanning image comprises at least one detection target;extracting an image feature of the radiation scanning image by using a pre-trained feature extraction model; performing a similarity search with respect to the image feature of the radiation scanning image by using a retriever generated by the method of any one of claims 1 to 11; andin response to the image feature of the radiation scanning image matching an image feature in the feature library, obtaining a class corresponding to the image feature of the radiation scanning image and determining the class to be a class of the at least one detection target comprised in the radiation scanning image. 13.The method of claim 12, wherein the performing a similarity search comprises:searching, by using a nearest neighbor search algorithm, the feature library of the retriever for an image feature matching the image feature of the radiation scanning image. 14.The method of claim 12, wherein the image features in the feature library comprise sparse features and dense features, and the performing a similarity search comprises:searching, with respect to the image feature of the radiation scanning image, the feature library of the retriever to determine whether there exists a sparse feature matching the image feature of the radiation scanning image; andin response to existence of a sparse feature matching the image feature of the radiation scanning image, further searching, with respect to the image feature of the radiation scanning image, the feature library of the retriever for a dense feature matching the image feature of the radiation scanning image; orwherein the performing a similarity search comprises:searching, with respect to the image feature of the radiation scanning image, the feature library of the retriever only for a dense feature matching the image feature of the radiation scanning image; orwherein the performing a similarity search comprises:searching, with respect to the image feature of the radiation scanning image, the feature library of the retriever only for a sparse feature matching the image feature of the radiation scanning image. 15.The method of any one of claims 12 to 14, further comprising: during the acquisition of the radiation scanning image, further acquiring metadata corresponding to the radiation scanning image, wherein the metadata comprises device parameters for acquiring the radiation scanning image. 16.The method of any one of claims 12 to 14, wherein the radiation scanning detection system comprises a multi-view scanning detection system, and the method further comprises:based on the acquired radiation scanning image and the class of the at least one detection target comprised in the radiation scanning image, forming two-dimensional scanning images from a plurality of views; andperforming three-dimensional fusion on the two-dimensional scanning images from the plurality of views based on geometric relationships and the identified at least one detection target to form a three-dimensional scanning image, wherein a three-dimensional identification result for the at least one detection target is displayed in the three-dimensional scanning image; orwherein the method further comprises:based on the acquired radiation scanning image and the class of the at least one detection target comprised in the radiation scanning image, forming two-dimensional scanning images from a plurality of views; andobtaining a three-dimensional identification result for the at least one detection target based on geometric relationships and the identified at least one detection target without forming a three-dimensional scanning image by fusion. 17.The method of any one of claims 12 to 14, further comprising:based on the acquired radiation scanning image and the class of the at least one detection target comprised in the radiation scanning image, determining a physical property of the at least one detection target according to the class; andperforming three-dimensional fusion on the radiation scanning image based on the physical property and the identified at least one detection target to form a three-dimensional scanning image, wherein a three-dimensional identification result for the at least one detection target is displayed in the three-dimensional scanning image. 18.An apparatus for generating a retriever for a radiation scanning detection system, comprising:a first construction module configured to construct a sample library, wherein the first construction module comprises:a first data obtaining sub-module configured to obtain first sample data, wherein the first sample data comprises a first image and a first label, the first image is a radiation scanning image comprising at least one first detection target, and the first label indicates a class of the at least one first detection target; anda second data obtaining sub-module configured to obtain second sample data, wherein the second sample data comprises a second image and a second label, the second image is a radiation scanning image comprising at least one second detection target, the second label indicates a class of the at least one second detection target, the first sample data is positive sample data, and the second sample data is negative sample data;a second construction module configured to construct a feature library, wherein the second construction module comprises:an extraction sub-module configured to extract a first image feature of the first image and a second image feature of the second image by using a pre-trained feature extraction model;an association forming sub-module configured to form a first feature-label association based on the first image feature and the first label and to form a second feature-label association based on the second image feature and the second label; anda feature library generation sub-module configured to generate the feature library based on the first feature-label association and the second feature-label association; anda retriever generation module configured to generate a retriever based on the feature library. 19.An object detection apparatus for a radiation scanning detection system, comprising:an image acquisition module configured to acquire a radiation scanning image by the radiation scanning detection system, wherein the radiation scanning image comprises at least one detection target;a feature extraction module configured to extract an image feature of the radiation scanning image by using a pre-trained feature extraction model;a retrieval module configured to perform a similarity search with respect to the image feature of the radiation scanning image by using a retriever generated by the method of any one of claims 1 to 11; andan object detection module configured to, in response to the image feature of the radiation scanning image matching an image feature in the feature library, obtain a class corresponding to the image feature of the radiation scanning image and determine the class to be a class of the at least one detection target comprised in the radiation scanning image. 20.An electronic device, comprising:one or more processors; anda storage device storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method of any one of claims 1 to 17. 21.A radiation scanning detection system, comprising:a radiation source configured to emit radiation;a detector configured to receive radiation emitted from the radiation source and passed through an object under inspection; anda data processing device communicatively connected to the detector, wherein the data processing device comprises:one or more processors; anda storage device storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method of any one of claims 1 to 17. 22.A computer-readable storage medium having executable instructions thereon, wherein the executable instructions, when executed by a processor, cause the processor to perform the method of any one of claims 1 to 17. 23.A computer program product comprising a computer program, wherein the computer program, when executed by a processor, causes the processor to implement the method of any one of claims 1 to 17.ABSTRACT The present disclosure provides a retriever generation method and apparatus, an object detection method and apparatus, an electronic device, and a radiation scanning detection system, which may be applied to image processing. The generation method includes: constructing a sample library, including: obtaining first sample data, the first sample data including a first image and a first label indicating a class of a first detection target; and obtaining second sample data, the second sample data including a second image and a second label indicating a class of a second detection target, the first and second sample data being positive and negative sample data, respectively. The method further includes: constructing a feature library, including: extracting first and second image features by using a feature extraction model; forming first and second feature-label associations, and generating the feature library based on the first and second feature-label associations. The method further includes generating a retriever based on the feature library.1 / 9 FIG. 1 FIG. 22 / 9 FIG. 33 / 9 FIG. 44 / 9FIG. 5 FIG. 65 / 9 FIG. 7 FIG. 86 / 9FIG. 97 / 9 FIG. 108 / 9 FIG. 119 / 9FIG. 12 FIG. 13 

Claims

1. A method for generating a retriever for a radiographic scanning detection system, comprising: Building a sample library, wherein building the sample library includes: Acquire first sample data, where the first sample data includes a first image and a first label, wherein the first image is an image including at least one first detection target, and the first label indicates a category of the at least one first detection target; and Acquire second sample data, where the second sample data includes a second image and a second label, the second image is an image including at least one second detection target, the second label indicates a category of the at least one second detection target, the first sample data is positive sample data, and the second sample data is negative sample data; Constructing a feature library, wherein constructing the feature library comprises: Extracting first image features of the first image and second image features of the second image using a pre-trained feature extraction model; forming a first feature-label relationship based on the first image feature and the first label, and forming a second feature-label relationship based on the second image feature and the second label; and generating a feature library based on the first feature-label relationship and the second feature-label relationship; and Based on the feature library, a retriever is generated.

2. The method according to claim 1, wherein The constructing of the sample library further includes: acquiring third sample data, the third sample data including a third image and a third label, the third image being a radiographic scanning image including at least one third detection target, the third label indicating a category of the at least one third detection target, wherein the third sample data includes positive sample data and / or negative sample data.

3. The method according to claim 2, wherein: The constructing feature library further comprises: extracting third image features of the third image using a pre-trained feature extraction model; Based on the third image feature and the third label, forming a third feature-label relationship; and The third feature-label relationship is stored in the feature library.

4. The method according to claim 3, wherein: The first image feature includes at least one of a sparse feature and a dense feature; and / or, the second image feature includes at least one of a sparse feature and a dense feature; and / or, the third image feature includes at least one of a sparse feature and a dense feature.

5. The method according to claim 4, wherein The sparse features include regional features represented by bounding box annotation; and / or the dense features include regional features represented by pixel-level mask annotation.

6. The method according to any one of claims 1 to 5, wherein The obtaining of the first sample data includes: Acquire a plurality of first images for an application scenario of the ray scanning detection system, wherein each of the plurality of first images includes at least one first detection target, and the first detection target is a typical detection target suitable for the application scenario of the ray scanning detection system; and The categories of the first detection targets in the plurality of first images are marked to form a plurality of first labels.

7. The method according to any one of claims 1 to 5, wherein The obtaining of the second sample data includes: Acquire a plurality of second images according to the first sample data, wherein each of the plurality of second images includes at least one second detection target, the second detection target is similar to the first detection target, and the second detection target is of a different category from the first detection target; and The categories of the second detection targets in the plurality of second images are marked to form a plurality of second labels.

8. The method according to claim 7, wherein: The acquiring of the plurality of second images comprises acquiring the plurality of second images by at least one of actively collecting, retrieving historical data, and synthesizing based on a generative model.

9. The method according to any one of claims 1 to 5, wherein The first sample data and the second sample data are acquired before the ray scanning detection system is formally put into operation.

10. The method according to any one of claims 2 to 5, wherein: The third sample data is obtained during the formal operation of the ray scanning detection system.

11. The method according to claim 10, wherein: The obtaining of the third sample data includes: During the formal operation of the radiographic scanning detection system, spot-check the detection results of the radiographic scanning detection system; In response to the detection result having omissions and / or false alarms, acquiring an image having omissions and / or false alarms as a third image; and The categories of the third detection targets in the third image are marked to form a plurality of third labels.

12. A target detection method for a ray scanning detection system, comprising: Acquiring a radiographic scanning image through the radiographic scanning detection system, wherein the radiographic scanning image includes at least one detection target; Extracting image features of the X-ray scan image using a pre-trained feature extraction model; performing similarity retrieval based on the image features of the X-ray scan image using a retriever generated by the method according to any one of claims 1 to 11; as well as In response to the image features of the ray scanning image matching the image features in the feature library, a category corresponding to the image features of the ray scanning image is obtained, and the category is determined to be a category of at least one detection target included in the ray scanning image.

13. The method according to claim 12, wherein: The similarity search includes: using a nearest neighbor search algorithm to search for image features that match the image features of the radiographic image in the feature library of the retriever.

14. The method according to claim 12, wherein: The image features in the feature library include sparse features and Dense features; The similarity search includes: searching, for the image features of the radiographic image, a feature library of the retriever to determine whether there are sparse features that match the image features of the radiographic image; and In response to the existence of sparse features matching the image features of the radiographic image, further searching the feature library of the retriever for dense features matching the image features of the radiographic image with respect to the image features of the radiographic image; or The similarity search includes: For the image features of the radiographic image, only dense features matching the image features of the radiographic image are searched in the feature library of the retriever; or, The similarity search includes: With respect to the image features of the radiographic scanning image, only sparse features matching the image features of the radiographic scanning image are searched in the feature library of the retriever.

15. The method according to any one of claims 12 to 14, wherein The method includes: in the process of acquiring the X-ray scanning image, further acquiring meta information corresponding to the X-ray scanning image, wherein the meta information includes equipment parameters for acquiring the image.

16. The method according to any one of claims 12 to 14, wherein: The ray scanning detection system includes a multi-view scanning detection system; The method further includes: forming two-dimensional scanning images of multiple viewing angles based on the acquired ray scanning image and the category of at least one detection target included in the ray scanning image; and performing three-dimensional fusion on the two-dimensional scanning images of the multiple viewing angles based on the geometric relationship and the identified at least one detection target to form a three-dimensional scanning image, wherein the three-dimensional recognition result of the at least one detection target is displayed in the three-dimensional scanning image; or The method also includes: forming two-dimensional scanning images from multiple perspectives based on the acquisition of the X-ray scanning image and the category of at least one detection target included in the X-ray scanning image; and obtaining a three-dimensional recognition result of the at least one detection target based on the geometric relationship and the identified at least one detection target without fusing the three-dimensional scanning image.

17. The method according to any one of claims 12 to 14, wherein: The method also includes: determining the physical properties of the at least one detection target according to the category based on the acquisition of the X-ray scanning image and the category of at least one detection target included in the X-ray scanning image; and performing three-dimensional fusion of the X-ray scanning image based on the physical properties and the identified at least one detection target to form a three-dimensional scanning image, wherein the three-dimensional recognition result of the at least one detection target is displayed in the three-dimensional scanning image.

18. A device for generating a retriever for a radiographic scanning detection system, comprising: The first building module is used to build a sample library, and the first building module includes: a first data acquisition submodule, configured to acquire first sample data, wherein the first sample data includes a first image and a first label, the first image being a radiographic scan image including at least one first detection target, and the first label indicating a category of the at least one first detection target; and a second data acquisition submodule, configured to acquire second sample data, wherein the second sample data includes a second image and a second label, the second image is a radiographic scan image including at least one second detection target, the second label indicates a category of the at least one second detection target, the first sample data is positive sample data, and the second sample data is negative sample data; The second building module is used to build a feature library, and the second building module includes: an extraction submodule, configured to extract first image features of the first image and second image features of the second image using a pre-trained feature extraction model; a relationship forming submodule, configured to form a first feature-label relationship based on the first image feature and the first label, and to form a second feature-label relationship based on the second image feature and the second label; and a feature library generating submodule, configured to generate a feature library based on the first feature-label relationship and the second feature-label relationship; and The retriever generation module is used to generate a retriever based on the feature library.

19. A target detection device for a ray scanning detection system, comprising: An image acquisition module, configured to acquire a radiographic scanning image through the radiographic scanning detection system, wherein the radiographic scanning image includes at least one detection target; A feature extraction module, configured to extract image features of the X-ray scan image using a pre-trained feature extraction model; a retrieval module configured to perform similarity retrieval based on image features of the X-ray scan image using a retriever generated by the method according to any one of claims 1 to 11; as well as The target detection module is used to obtain the category corresponding to the image feature of the ray scanning image in response to the image feature of the ray scanning image matching the image feature in the feature library, and determine that the category is the category of at least one detection target included in the ray scanning image.

20. An electronic device comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors are enabled to execute the method according to any one of claims 1 to 17.

21. A radiographic scanning detection system, comprising: A ray source, used for emitting rays; a detector, configured to receive the radiation emitted from the radiation source and passing through the detection object; as well as a data processing device, the data processing device being in communication with the detector, The data processing device includes: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the method according to any one of claims 1 to 17.

22. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to perform the method according to any one of claims 1 to 17.

23. A computer program product comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 17 is implemented.

Images