Object identification method for three-dimensional CT data, apparatus and ray scanning detection system

Abstract

Provided is an object identification method for three-dimensional CT data, the method comprising: acquiring three-dimensional CT data; segmenting the three-dimensional CT data along a plurality of segmentation directions to obtain m slice sequences, wherein at least one of the m slice sequences comprises n slices, m being a positive integer greater than or equal to 1, and n being a positive integer greater than or equal to 2; generating a plurality of two-dimensional images on the basis of the m slice sequences; performing object identification on the plurality of two-dimensional images to obtain a two-dimensional semantic description set of an object; and increasing the dimensionality of the two-dimensional semantic description set to obtain a three-dimensional identification result of the object.

Description

Target recognition method and device for three-dimensional CT data, and ray scanning detection system

[0001] This application claims priority to Chinese Patent Application No. 202411231116.6, filed September 3, 2024, the contents of which are incorporated herein by reference in their entirety. TECHNICAL FIELD

[0002] The present disclosure relates to the technical field of image processing and / or object detection, and in particular to a target recognition method and device for three-dimensional CT data, an electronic device, a ray scanning detection system, a computer readable storage medium, and a program product. BACKGROUND

[0003] In related technologies, security CT (Computed Tomography) devices can provide three-dimensional images with more abundant information and are widely used in luggage security checks in customs, airports, train stations, subway stations, and other places.

[0004] Due to the large amount of CT image data, high spatial dimension, and large and difficult training and reasoning volume, the target automatic recognition method based on three-dimensional images faces the problems of large computing resource demand and obvious reasoning delay. In addition, in various security application scenarios, the requirements for recognition accuracy and speed are also increasing. Therefore, how to accurately and quickly realize the automatic recognition of targets in CT images is a challenging task.

[0005] It should be noted that the above information disclosed in this section is only for understanding the background of the inventive concept of the present disclosure, and therefore, the above information can include information that does not constitute prior art. SUMMARY

[0006] A target recognition method and device for three-dimensional CT data, an electronic device, a ray scanning detection system, a computer readable storage medium, and a program product are proposed.

[0007] In one aspect, a target recognition method for three-dimensional CT data is provided, the method comprising: obtaining three-dimensional CT data; slicing the three-dimensional CT data along a plurality of slicing directions to obtain m slice sequences, wherein at least one of the m slice sequences includes n slices, m is a positive integer greater than or equal to 1, and n is a positive integer greater than or equal to 2; generating a plurality of two-dimensional images according to the m slice sequences; performing target recognition on the plurality of two-dimensional images to obtain a two-dimensional semantic description set of a target; and upgrading the two-dimensional semantic description set to obtain a three-dimensional recognition result of the target.

[0008] According to some exemplary embodiments, the generating a plurality of two-dimensional images according to the m slice sequences comprises: processing at least one slice sequence comprising the n slices by using a first deep learning model to output at least one two-dimensional image.

[0009] According to some exemplary embodiments, the processing at least one slice sequence comprising the n slices by using the first deep learning model to output at least one two-dimensional image comprises: processing a first slice sequence comprising n1 slices by using the first deep learning model to output k1 two-dimensional images, wherein n1 is greater than or equal to 2, and k1 is greater than or equal to 1 and less than n1; and / or, processing a second slice sequence comprising n2 slices by using the first deep learning model to output k2 two-dimensional images, wherein n2 is greater than or equal to 2, and k2 is greater than or equal to 1 and less than n2.

[0010] According to some exemplary embodiments, the method further comprises: obtaining a mapping relationship between pixels in the two-dimensional images and voxels in the three-dimensional CT data during the generating the plurality of two-dimensional images.

[0011] According to some exemplary embodiments, the target recognition for the plurality of two-dimensional images to obtain a two-dimensional semantic description set of the target comprises:

[0012] The target recognition for the plurality of two-dimensional images is performed by using at least one of an image processing method, a classical machine learning method, and a deep learning method for two-dimensional images, or in a combined manner, to obtain the two-dimensional semantic description set of the target.

[0013] According to some exemplary embodiments, the method further comprises: determining the plurality of segmentation directions according to at least one of an occlusion condition of a target, a complexity of a target category, a shape, a position, and a pose of the target.

[0014] According to some exemplary embodiments, at least two slice sequences in the m slice sequences correspond to non-overlapping segmentation regions; and / or, at least two slice sequences in the m slice sequences correspond to partially overlapping segmentation regions.

[0015] According to some exemplary embodiments, the method further comprises: determining the number n of slices included in at least one slice sequence in the m slice sequences according to a size of a target.

[0016] According to some exemplary embodiments, the dimensionality upgrading of the two-dimensional semantic description set to obtain a three-dimensional recognition result of the target comprises:

[0017] mapping the two-dimensional semantic description set to a three-dimensional space by using a back projection method to obtain a three-dimensional probability map; and

[0018] perform feature extraction on the three-dimensional probability map to obtain a three-dimensional recognition result of the target.

[0019] According to some exemplary embodiments, the dimensionality increasing of the set of two-dimensional semantic descriptions to obtain a three-dimensional recognition result of the target comprises:

[0020] extracting a candidate three-dimensional region of interest from the three-dimensional CT data;

[0021] mapping the set of two-dimensional semantic descriptions to a three-dimensional space using a back-projection method to obtain a three-dimensional probability map; and

[0022] processing the candidate three-dimensional region of interest and the three-dimensional probability map using a three-dimensional target recognition method to obtain a three-dimensional recognition result of the target.

[0023] According to some exemplary embodiments, the plurality of two-dimensional images further comprises a two-dimensional scan image, which is acquired by a two-dimensional imaging device.

[0024] According to some exemplary embodiments, the method further comprises projecting the obtained three-dimensional recognition result of the target to the two-dimensional scan image.

[0025] According to some exemplary embodiments, the mapping of the set of two-dimensional semantic descriptions to a three-dimensional space using a back-projection method to obtain a three-dimensional probability map comprises: performing the mapping of the set of two-dimensional semantic descriptions to a three-dimensional space by voxel driving or pixel driving, obtaining a semantic feature matrix, and compressing the semantic feature matrix into a three-dimensional probability map.

[0026] According to some exemplary embodiments, the voxel driving comprises: corresponding each voxel in the three-dimensional CT data to a pixel in each of the two-dimensional images, querying and accumulating the two-dimensional semantic description information corresponding to the pixel, and generating the semantic feature matrix.

[0027] According to some exemplary embodiments, the pixel driving comprises: each pixel in the two-dimensional image corresponds to a straight line in the three-dimensional CT data, traversing each pixel or each pixel in a region of interest in each two-dimensional image, propagating the two-dimensional semantic description information corresponding to the pixel along the straight line to the three-dimensional space, and generating the semantic feature matrix, wherein the region of interest is given by the set of two-dimensional semantic descriptions.

[0028] According to some exemplary embodiments, in the voxel driving or the pixel driving, the correspondence between the voxels and the pixels is obtained by a mapping function or a lookup table.

[0029] According to some exemplary embodiments, the feature extraction is performed on the three-dimensional probability map to obtain a three-dimensional recognition result of the target, including: binarizing the three-dimensional probability map to obtain a three-dimensional binary map; performing connected region analysis on the three-dimensional binary map to obtain a connected region; and generating a three-dimensional image semantic description set for the connected region.

[0030] According to some exemplary embodiments, the connected region analysis includes: performing connected component labeling on the three-dimensional binary map, and performing a mask operation on each labeled region to obtain the connected region.

[0031] According to some exemplary embodiments, the generating of the three-dimensional image semantic description set for the connected region includes: extracting all probability values in the connected region, performing principal component analysis to obtain an analysis set, taking the analysis set as an effective voxel region of the object, and counting a three-dimensional image semantic description set.

[0032] According to some exemplary embodiments, the three-dimensional image semantic description set is in units of one or more of a voxel, a three-dimensional region of interest, and a three-dimensional CT image, and includes: class information and / or confidence; or the three-dimensional image semantic description set is in units of a three-dimensional region of interest and / or a three-dimensional CT image, and includes: at least one of class information, location information of the target, and confidence.

[0033] According to some exemplary embodiments, the location information includes a three-dimensional bounding box.

[0034] According to some exemplary embodiments, the two-dimensional semantic description set is in units of one or more of a pixel, a region of interest, and a two-dimensional image, and includes: class information and / or confidence; or the two-dimensional semantic description set is in units of a region of interest and / or a two-dimensional image, and includes: at least one of class information, confidence, and location information of the target.

[0035] According to some exemplary embodiments, the target recognition for each of the plurality of two-dimensional images includes: using at least one of an image processing method, a classical machine learning method, and a deep learning method for two-dimensional images or a combination thereof to perform target recognition.

[0036] In another aspect, provided is an object recognition device for three-dimensional CT data, the device comprising: an acquisition module configured to acquire three-dimensional CT data; a slicing module configured to slice the three-dimensional CT data along a plurality of slicing directions to obtain m slice sequences, wherein at least one of the m slice sequences comprises n slices, m is a positive integer greater than or equal to 1, and n is a positive integer greater than or equal to 2; a two-dimensional image generation module configured to generate a plurality of two-dimensional images according to the m slice sequences; a two-dimensional recognition module configured to perform object recognition on the plurality of two-dimensional images to obtain a two-dimensional semantic description set of an object; and a dimension upgrading module configured to upgrade the two-dimensional semantic description set to obtain a three-dimensional recognition result of the object.

[0037] In yet another aspect, provided is an electronic device comprising: one or more processors; a memory storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method as described above.

[0038] In yet another aspect, provided is a radiation scanning detection system comprising: a radiation source configured to emit radiation; a detector configured to receive radiation emitted from the radiation source and passing through a detection object; and a data processing device in communication connection with the detector, wherein the data processing device comprises: one or more processors; a memory storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method as described above.

[0039] In yet another aspect, provided is a computer-readable storage medium having stored thereon a computer program, which, when executed by a processor, implements the steps of the method as described above.

[0040] In yet another aspect, provided is a computer program product comprising a computer program, which, when executed by a processor, implements the steps of the method as described above. BRIEF DESCRIPTION OF DRAWINGS

[0041] For better understanding of the present disclosure, the present disclosure will be described in detail in accordance with the following drawings:

[0042] FIG. 1 is a structural schematic diagram of a radiation scanning detection system according to some exemplary embodiments of the present disclosure.

[0043] FIG. 2 is a schematic diagram of a CT scanning device implementing a reconstruction scanning mode according to some exemplary embodiments of the present disclosure.

[0044] FIG. 3 shows an example of attenuation signal data obtained according to an embodiment of the present disclosure.

[0045] FIG. 4 is a schematic flowchart of a target recognition method for three-dimensional CT data according to an embodiment of the present disclosure.

[0046] FIG. 5 is a schematic diagram of a segmentation direction in a target recognition method for three-dimensional CT data according to an embodiment of the present disclosure.

[0047] FIGS. 6A and 6B are schematic diagrams of a dimension reduction fusion process for single-channel and multi-channel, respectively, in a target recognition method for three-dimensional CT data according to an embodiment of the present disclosure.

[0048] FIG. 6C schematically shows a network structure diagram of a first deep learning model according to some example embodiments of the present disclosure.

[0049] FIG. 7 is a schematic flowchart of obtaining a three-dimensional recognition structure in a target recognition method according to example embodiments of the present disclosure.

[0050] FIG. 8 is a detailed flowchart of obtaining a three-dimensional recognition structure in a target recognition method according to example embodiments of the present disclosure.

[0051] FIG. 9 is a schematic flowchart of a target recognition method for three-dimensional CT data according to other example embodiments of the present disclosure.

[0052] FIG. 10 is a structural block diagram of a target recognition apparatus for three-dimensional CT data according to some example embodiments of the present disclosure.

[0053] FIG. 11 schematically shows a structural block diagram of an electronic device suitable for implementing the target recognition method according to example embodiments of the present disclosure. DETAILED DESCRIPTION

[0054] Specific embodiments of the present disclosure will be described in detail below, it should be noted that the embodiments described herein are only used for illustration and do not limit the present disclosure. In the following description, a large number of specific details are set forth in order to provide a thorough understanding of the present disclosure. However, it is obvious to those skilled in the art that the present disclosure does not necessarily have to be implemented with these specific details. In other instances, well-known structures, materials or methods are not specifically described in order to avoid obscuring the present disclosure.

[0055] Reference throughout this specification to "one embodiment", "an embodiment", "one example", or "an example", means that a particular feature, structure, or characteristic described in connection with the embodiment or example is included in at least one embodiment of the disclosure. Thus, appearances of the phrases "in one embodiment" or "in an embodiment" or "one example" or "an example" in various places throughout this specification are not necessarily all referring to the same embodiment or example. Furthermore, the particular features, structures, or characteristics can be combined in any suitable

[0056] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The use of the terms "including," "comprising," or "having" and the like in the detailed description are meant to be inclusive in a manner similar to the term "comprising," unless otherwise noted.

[0057] All terms used herein including technical and scientific terms have the meanings commonly understood by one of ordinary skill in the art unless otherwise defined. It should be noted that the use of terms such as "including", "comprising", "having" and the like are meant to be inclusive in a manner similar to the term "comprising" unless otherwise noted.

[0058] In the following, embodiments of the disclosure are described in detail taking CT scan detection as an example. It should be understood that the embodiments of the disclosure are not limited to the CT scan detection scenario, and can be applied to various scan detection scenarios. For example, it can be applied to scan detection scenarios including various different inspection objects, including but not limited to vehicle scan detection, luggage / parcel scan detection, human body scan detection, etc. It should be noted that the description of the scan detection scenario here is not an exhaustive description, and the description here should not be understood as limiting the protection scope of the disclosure.

[0059] It should be noted that the embodiments of the disclosure are not limited to the target detection task, and can also be applied to target classification, image segmentation, etc. tasks. Thus, in this paper, unless otherwise stated, the expression "target recognition" includes but is not limited to various tasks such as target detection, target classification, image segmentation, etc.

[0060] It should also be noted that in this paper, the expression "target" can include but is not limited to, for example, target objects of interest, items of interest, etc. in a scan detection scenario.

[0061] FIG. 1 is a structural schematic diagram of a ray scanning detection system according to some exemplary embodiments of the present disclosure. In FIG. 1, a CT scanning device is schematically shown as an example of the ray scanning detection system. As shown in FIG. 1, the CT scanning device according to the present embodiment includes a gantry 20, a carrying mechanism 40, a controller 50, a data processing device 60 (e.g., a computer), etc. The gantry 20 includes a ray source 10, such as an X-ray machine, which emits X-rays for examination, and a detection and acquisition device 30. The carrying mechanism 40 carries a detection object (e.g., checked luggage) 70 through a scanning region between the ray source 10 and the detection and acquisition device 30 of the gantry 20 while the gantry 20 rotates around the advancing direction of the detection object 70, so that the rays emitted by the ray source 10 can penetrate the detection object 70, and the detection object 70 is subjected to CT scanning. The detection and acquisition device 30, for example, includes a detector and a data acquisition device having an integral module structure, such as a flat panel detector, for detecting the rays transmitted through the detection object 70 (e.g., checked luggage), obtaining analog signals, and converting the analog signals into digital signals, so as to output projection data of the detection object 70 against X-rays. The controller 50 is used to control the synchronous operation of various parts of the entire system. The data processing device 60 is used to process the data collected by the data acquisition device, process and reconstruct the data, and output the results.

[0062] As shown in FIG. 1, the ray source 10 is placed on one side where the detection object can be placed, and the detection and acquisition device 30 is placed on the other side of the detection object 70, including a detector and a data acquisition device, for obtaining transmission data and / or multi-angle projection data of the detection object 70. The data acquisition device includes a data amplification shaping circuit therein, which can work in an (electric current) integration mode or a pulse (counting) mode. The data output cable of the detection and acquisition device 30 is connected to the controller 50 and the data processing device 60, and the collected data is stored in the data processing device 60 according to a trigger command.

[0063] In the embodiments of the present disclosure, a ray scanning detection module can be deployed in the data processing device 60, and an image recognition model or a target recognition model can be deployed in the ray scanning detection module. The ray scanning detection module can utilize the image recognition model or the target recognition model to recognize the collected field data (e.g., a ray scanning image), and the recognition can include target detection, for example, detecting a target object or an object of interest in the ray scanning image. For example, the target object or the object of interest can be various kinds of contraband. It should be understood that the specific categories of the target object or the object of interest depend on the scanning detection field where the ray scanning detection system is arranged, or in other words, the specific categories of the target object or the object of interest are determined by the user (also referred to as the user herein) of the ray scanning detection system according to specific scanning detection needs, which can be dynamically adjusted according to scanning detection needs.

[0064] It should be noted that, for the convenience of description, the target object or the object of interest herein can be collectively referred to as a target.

[0065] In embodiments of the present disclosure, the ray source 10 can be an X-ray machine, for example, and a suitable X-ray machine focus size can be selected according to the resolution of imaging. In other embodiments, an X-ray machine can not be used, but an X-ray beam can be generated using a linear accelerator or the like.

[0066] The detection and acquisition device 30 includes an X-ray detector and a data acquisition circuit, etc. The X-ray detector can use a solid detector, or a gas detector or other detectors, and embodiments of the present disclosure are not limited thereto. The data acquisition circuit includes a readout circuit, an acquisition trigger circuit, a data transmission circuit, etc.

[0067] The combination of the controller 50 and the data processing device 60, for example, includes a computer device installed with a control program and a data processing program, responsible for completing the control of the operation process of the CT scanning device, including mechanical rotation, electrical control, safety interlock control, etc., reconstructing a CT image from projection data, training an image recognition model or a target recognition model, and identifying a ray scanning image using the trained image recognition model or target recognition model, etc.

[0068] FIG. 2 is a schematic diagram of a CT scanning device implementing a reconstruction scanning mode according to some example embodiments of the present disclosure. As shown in FIG. 2, a portion of the detection object 70 in the X-ray transmission field of view 45 emitted from the ray source 10 is received by the detection and acquisition device 30, converted into an electrical signal and further converted into a digital signal representing an attenuation value, as projection data, and reconstructed by a computer to obtain a high-quality image.

[0069] According to embodiments of the present disclosure, the detection object 70 is CT scanned using the above-mentioned CT scanning device to obtain a raw attenuation signal. The attenuation signal data corresponding to a row of detectors can also be displayed in the form of a two-dimensional image. FIG. 3 shows an example of attenuation signal data obtained according to embodiments of the present disclosure. As shown in FIG. 3, the horizontal axis direction of the raw attenuation signal represents a detector column direction pixel sequence (e.g., from 1 to 256), and the vertical axis represents an angle (e.g., from 1 degree to 360 degrees). The raw attenuation signal is preprocessed to become projection data. For example, the projection data can be preprocessed by the CT scanning device, such as negative logarithmic transformation, to obtain the projection data. Then, the data processing device 60 executes a reconstruction program, for example, a mainstream analytical reconstruction method in the related art can be used to obtain a reconstructed image, which is the ray scanning image mentioned in embodiments of the present disclosure.

[0070] In an embodiment of the present disclosure, the data processing apparatus 60 can perform target recognition (e.g., target detection) on the reconstructed image (i.e., the ray scanning image) described above to identify a target (e.g., contraband) in the ray scanning image. Specifically, a ray scanning detection module can be deployed in the data processing apparatus 60, and an image recognition model or a target recognition model can be deployed in the ray scanning detection module. The ray scanning detection module can identify the ray scanning image by using the image recognition model or the target recognition model.

[0071] The applicant has found through research that three-dimensional CT data usually includes a large number of voxels, is sparse, and also includes certain noise and redundant information. Identifying a three-dimensional CT target can face a problem of a huge amount of computation. In a security check scene, it is difficult to identify a target that has a strong three-dimensional shape feature but a complex material composition and physical property. Meanwhile, the real-time performance and accuracy of three-dimensional CT data target recognition are also required to be high in the security check scene.

[0072] A related art also proposes a target recognition method for three-dimensional CT data. In this method, a three-dimensional CT data is cut into multiple sub-volumes along three orthogonal coordinate axis directions. A maximum value of each sub-volume is taken along a certain coordinate axis direction to convert into a two-dimensional slice. A two-dimensional slice set is obtained after traversing the CT data. Then, a detector is used to obtain a prediction result on each two-dimensional slice. Next, three-dimensional prediction results are obtained by accumulating and fusing the two-dimensional prediction results in three directions. Finally, a final three-dimensional recognition result is obtained through three-dimensional post-processing.

[0073] In this related art, the three-dimensional CT data is cut into multiple sub-volumes along three orthogonal coordinate axis directions, and a maximum value of each sub-volume is taken to convert into a two-dimensional slice. The applicant has further found that this dimension reduction method at least has the following disadvantages: (1) When each sub-volume is converted into a two-dimensional image, only the maximum value along a certain coordinate axis direction is retained, which can lose some target information. For example, when a hand gun with a main body made of engineering plastic is blocked by a safety product made of metal, the two-dimensional image retains the information of the metal safety product rather than the target information. Therefore, this method can be not conducive to the recognition of targets with different material compositions or a small equivalent atomic number of density. (2) The dimension reduction directions are only three orthogonal coordinate axis directions. For various inspected objects with different poses and shapes and sizes, only three directions are used for dimension reduction, which can not fully display part of the structure of the target. This can affect the accuracy and completeness of detection, and thus cause poor recognition accuracy.

[0074] In addition, the applicant proposes a security CT target recognition method (see the published patent of the same applicant as the present disclosure, with the authorized publication number CN113792623B and the invention name of "security CT target recognition method and device"). In the security CT target recognition method, a plurality of two-dimensional images are generated by reducing the dimensionality of three-dimensional CT data, then target recognition is performed on the plurality of two-dimensional images to obtain a two-dimensional semantic description set of the target, and the obtained two-dimensional semantic description set is dimensionally upgraded to finally obtain a three-dimensional recognition result of the target, which realizes effective recognition of the security CT target. In the security CT target recognition method, the method of reducing dimensionality from three-dimensional data to two-dimensional for recognition and then upgrading to generate a three-dimensional result is proposed, which can not only reduce the computational complexity during recognition, but also use the two-dimensional target recognition technology with higher accuracy and better performance for target recognition, so that the target with complex material composition and physical properties and shape features can be effectively recognized, and a three-dimensional recognition result with rich information is also given.

[0075] The applicant has further found that in the above method, a plurality of directions are set for the three-dimensional CT data and projection or rendering is performed according to the plurality of directions, so as to reduce dimensionality to generate a plurality of two-dimensional reduced dimensionality views. At this time, the target recognition is performed by using a limited number of two-dimensional images, which may cause insufficient expression of three-dimensional features and loss of part of the three-dimensional shape features, spatial position information, etc. in some cases. For example, when a plurality of objects are overlapped and placed, the spatial information may be partially lost in some angle projection.

[0076] Therefore, an embodiment of the present disclosure proposes a target recognition method for three-dimensional CT data, which can slice the three-dimensional CT data along a plurality of slicing directions, convert the slice sequence into a two-dimensional image, i.e. a thin layer slice, perform two-dimensional target recognition by using the thin layer slice, and then upgrade to a three-dimensional target recognition result. The thin layer slice not only better retains the significant features of the target, but also reduces the data dimension, which is beneficial to improve the recognition accuracy and recognition speed, and weaken the influence of security product shielding on target recognition.

[0077] For example, some exemplary embodiments of the present disclosure provide a target recognition method for three-dimensional CT data, which includes: acquiring three-dimensional CT data; slicing the three-dimensional CT data along a plurality of slicing directions to obtain m slice sequences, wherein at least one of the m slice sequences includes n slices, m is a positive integer greater than or equal to 1, and n is a positive integer greater than or equal to 2; generating a plurality of two-dimensional images according to the m slice sequences; performing target recognition on the plurality of two-dimensional images to obtain a two-dimensional semantic description set of the target; and upgrading the two-dimensional semantic description set to obtain a three-dimensional recognition result of the target.

[0078] In this embodiment, a dimension reduction method of thin slice is used to efficiently obtain a two-dimensional semantic description set, and then a three-dimensional recognition result is obtained by dimension increase, thereby effectively improving the speed and accuracy of three-dimensional CT recognition.

[0079] FIG. 4 is a schematic flowchart of a target recognition method for three-dimensional CT data according to an embodiment of the present disclosure. Referring to FIG. 4, the target recognition method for three-dimensional CT data can include the following operations or steps S410-S450.

[0080] Referring to FIG. 4, in operation or step S410, three-dimensional CT data is obtained.

[0081] With reference to FIGS. 1-3, a portion of the detection object 70 in the X-ray transmission field of view 45 emitted from the ray source 10 is received by the detection and acquisition device 30, converted into an electrical signal and further converted into a digital signal representing an attenuation value as projection data; then, a mainstream analytical reconstruction method in the related technical field can be used to obtain a three-dimensional reconstruction image.

[0082] In an embodiment of the present disclosure, the three-dimensional CT data can include the three-dimensional reconstruction image.

[0083] Referring to FIG. 4, in operation or step S420, the three-dimensional CT data is sliced along multiple slicing directions to obtain m slice sequences, wherein at least one of the m slice sequences includes n slices, m is a positive integer greater than or equal to 1, and n is a positive integer greater than or equal to 2.

[0084] In some exemplary embodiments, for a three-dimensional CT data to be recognized, the three-dimensional CT data is sliced along multiple slicing directions to obtain multiple slice sequences, each slice sequence including a predetermined number of slices, which can more completely show the shape and other information of a target in different poses and weaken the influence of security product occlusion on target recognition as much as possible.

[0085] In embodiments of the present disclosure, the method can further include determining a plurality of segmentation directions according to at least one of an occlusion condition of the target, a complexity of a target category, a shape of the target, a position, and a pose of the target. That is, in embodiments of the present disclosure, the segmentation directions are not limited to a few directions, for example, are not limited to three orthogonal coordinate axis directions, and the segmentation directions can be flexibly set according to the complexity of a specific security inspection scene, the condition of the inspected object, and the condition of the target. In addition, the number of segmentation directions can be increased according to the complexity of the security inspection scene or the complexity of the inspected object category, for example, for some scenes with disordered placed objects, more segmentation directions can be added. Based on at least one of the above factors, a plurality of segmentation directions are flexibly determined, for various inspected objects with different poses and different shapes and sizes, the dimensionality can be reduced along the directions that meet the target requirements, so that the partial structure of the target can be fully displayed. In this way, the accuracy and completeness of target recognition can be improved.

[0086] FIG. 5 is a schematic diagram of segmentation directions in a target recognition method for three-dimensional CT data according to an embodiment of the present disclosure.

[0087] For example, referring to FIG. 5, a plurality of segmentation directions are set for three-dimensional CT volume data, and optionally, a certain volume data preprocessing operation can be performed, which can include filtering invalid voxels (for example, according to prior knowledge of target gray scale distribution, set the pixel values that are greatly different from the distribution to 0). For example, when setting the segmentation directions, assume that the three orthogonal coordinate axis directions in the coordinate system O1 are The plurality of segmentation directions can include three orthogonal coordinate axis directions The XY coordinate axis is a new coordinate system O2 obtained by rotating the original coordinate system O1 around the Z axis in the XOY plane by a certain angle (for example, 45 degrees) with the original point O as the base point. At this time, the original point O and the Z axis of the new coordinate system coincide with the original coordinate system O1, and the X and Y axes of the new coordinate system are X' and Y', which have an angle of 45 degrees with the X and Y axes of the original coordinate system O1. The plurality of segmentation directions can also include the direction In this example, five segmentation directions are set, three directions in the coordinate system O1 are two directions in the coordinate system O2 are

[0088] In embodiments of the present disclosure, the method can further include determining the number n of slices included in at least one of the m slice sequences according to the size of the target. For example, the number of slices included in a certain slice sequence is determined according to prior knowledge such as the size of the target in the slice direction corresponding to the slice sequence. Through such design, the slice sequence can fully display the partial structure of the target, thereby improving the accuracy and completeness of target recognition.

[0089] It should be noted that the setting of the cutting direction is not limited to the above several, and can also be set by rotating one or more of the X, Y and Z axes by a certain angle. That is, in the embodiments of the present disclosure, the specific direction and number of the cutting direction are not particularly limited.

[0090] Continuing to refer to FIG. 5, the three-dimensional CT data is cut into a plurality of slice sequences along the above plurality of cutting directions. For example, for the cutting direction , the corresponding slice sequence is M y ×N z ×n, that is, the YZ slice sequence. It should be understood that the plane in which each slice in the YZ slice sequence is located is perpendicular to the cutting direction . Among them, M y and N z are the sizes of the YZ slice, and n is the number of slices.

[0091] In the embodiments of the present disclosure, after the cutting direction is determined, a plurality of slice sequences can be obtained by traversing all the cutting directions. For example, in the embodiment shown in FIG. 5, five slice sequences can be obtained by traversing the five determined cutting directions .

[0092] It should be noted that in some embodiments, the cutting direction corresponding to any two slice sequences of the m slice sequences is different. In other embodiments, the cutting direction corresponding to at least two slice sequences of the m slice sequences can be the same. That is, in the embodiments of the present disclosure, the m slice sequences are cut along m’ cutting directions, where m’ is a positive integer less than or equal to m.

[0093] It should also be noted that in the embodiments of the present disclosure, in at least one slice sequence including n slices, the length and width dimensions of each slice are each significantly greater than the sum of the thicknesses of the plurality of slices along the cutting direction, for example, in the above YZ slice sequence, the length and width dimensions M y and N z of each slice are each significantly greater than the sum of the thicknesses of the n slices along the cutting direction . It should be noted that “significantly greater than” here can be understood as: the length and width dimensions of the slice are each 5 times, for example, 10 times, greater than the sum of the thicknesses of the plurality of slices along the cutting direction. In this way, in the embodiments of the present disclosure, the combination of each slice sequence presents a thin cuboid, which is beneficial to generating a two-dimensional image of a thin layer slice in the next step.

[0094] In some embodiments of the present disclosure, the m slice sequences correspond to non-overlapping segmentation regions. By setting the segmentation regions corresponding to the slice sequences to be non-overlapping, a smaller number of slice sequences can be used to correspond to a larger region of three-dimensional CT data, which is conducive to improving the efficiency of identification.

[0095] In some embodiments of the present disclosure, the m slice sequences correspond to partially overlapping segmentation regions. By setting the segmentation regions corresponding to the slice sequences to be partially overlapping, the features of the target are repeatedly displayed in the thin layer slices generated subsequently, which is conducive to improving the accuracy of target identification.

[0096] It should be noted that the "segmentation region corresponding to the slice sequence" herein can be understood as a region formed by the length, width of the slice sequence perpendicular to the slicing direction, and the thickness (i.e. height) of the slice sequence along the slicing direction, for example, the region is a cuboid region.

[0097] Referring to FIG. 4, in operation or step S430, a plurality of two-dimensional images are generated according to the m slice sequences.

[0098] Specifically, in operation or step S430, at least one slice sequence including n slices can be processed by using a first deep learning model to output at least one two-dimensional image. For example, the above-mentioned five slice sequences can be processed by using the first deep learning model respectively to output at least five two-dimensional images. Illustratively, processing each of the above-mentioned five slice sequences by using the first deep learning model can output at least one two-dimensional image.

[0099] In some example embodiments of the present disclosure, in operation or step S430, the first slice sequence including n1 slices can be processed by using the first deep learning model to output k1 two-dimensional images, where n1 is greater than or equal to 2, and k1 is greater than or equal to 1 and less than n1. Additionally or optionally, in operation or step S430, the second slice sequence including n2 slices can also be processed by using the first deep learning model to output k2 two-dimensional images, where n2 is greater than or equal to 2, and k2 is greater than or equal to 1 and less than n2. That is, in operation or step S430, the i-th slice sequence including ni slices can be processed by using the first deep learning model to output ki two-dimensional images, where i is a positive integer greater than or equal to 1, ni is greater than or equal to 2, and ki is greater than or equal to 1 and less than ni. In this embodiment, the single slice sequence input to the first deep learning model includes multiple slices, and the number of two-dimensional images output by the first deep learning model can be configured according to actual conditions, and the number of two-dimensional images output by the first deep learning model is less than the number of slices included in the single slice sequence. In this way, the two-dimensional images (i.e., thin layer slices) are generated from the slice sequence by using the deep learning model, so that the thin layer slices can better retain the significant features of the target, while also reducing the data dimension.

[0100] FIG. 6C schematically illustrates a network architecture diagram of the first deep learning model, according to some example embodiments of the present disclosure. In FIG. 6C, conv, up-conv, +BN, ReLU, max pool represent the abbreviations of some operations and layers. Specifically, conv is the abbreviation of convolutional, representing a convolutional layer or a convolutional operation. A convolutional layer is a core component in a model, which extracts features of input data through filters (or called convolutional kernels). A convolutional operation can capture local spatial features and is able to automatically recognize patterns in images through learned weights. up-conv is the abbreviation of up-convolution, representing a transposed convolution or an up-sampling convolution. BN is the abbreviation of “batch normalization”, which is a technique used to improve the training process by normalizing the input of a layer to speed up the convergence rate, reduce the problem of gradient vanishing or exploding, and help prevent overfitting. A BN layer normalizes the input data of each feature channel to have a fixed mean and variance. ReLU is the abbreviation of “Rectified Linear Unit”, which is an activation function used to introduce nonlinearity in a neural network. max pool is the abbreviation of “maximum pooling”, representing a max pooling layer or a max pooling operation. Max pooling is used to reduce the spatial dimension of data while preserving the most important features. N represents the batch size. The batch size is the number of samples in each batch, which can affect the stability and efficiency of the training process. C represents the number of channels. In image data, each channel can represent different color information (such as the red, green, and blue channels in an RGB image), or in deeper levels of feature maps, each channel can represent different features or filter responses.

[0101] It should be noted that the network structure of the first deep learning model shown in FIG. 6C is only exemplary, and the first deep learning model in the embodiments of the present disclosure is not limited to this network structure. For example, referring to FIG. 6C, the network structure of the first deep learning model includes an encoder and a decoder, for example, the encoding performed by the encoder includes 3x3x3 convolution, followed by a ReLu activation function, followed by 2x2x2 max pooling, and the step of each dimension is 2; for example, the decoding operation performed by the decoder includes 3x3x3 convolution, and each convolution is followed by a ReLu activation function. Continuing to refer to FIG. 6C, in the last layer, the number of output channels is reduced to the specified number of channels by 1x1x1 convolution. During training, the label is a two-dimensional image generated by rendering the target slice sequence, and the loss function is the mean square error loss function. For example, during inference, the input is a slice sequence of 640x640x32, and the output is a two-dimensional image of 640x640.

[0102] That is, in the embodiments of the present disclosure, the network structure of the first deep learning model can include an encoder and a decoder. In this embodiment, in operation or step S430, the at least one slice sequence can be encoded by the encoder to obtain first features; then, the first features are decoded by the decoder to obtain second features; finally, the number of output channels of the second features is reduced to a specified number of channels by a convolution operation to obtain at least one two-dimensional image, wherein the number of the at least one two-dimensional image corresponds to the specified number of channels.

[0103] In the embodiments of the present disclosure, in the at least one slice sequence including n slices, the length and width dimensions of each slice are significantly greater than the sum of the thicknesses of the plurality of slices along the slicing direction, respectively. Accordingly, the two-dimensional image generated according to the at least one slice sequence including n slices can be regarded as a two-dimensional image corresponding to a thin layer slice, i.e., the two-dimensional image reflects the characteristics of the target in the thin layer slice. That is, the at least one slice sequence including n slices represents part of three-dimensional data, and the generated two-dimensional image reduces the dimension of this part of three-dimensional data to two-dimensional data, and the two-dimensional data is generated by fusing the target characteristics of the n slices, so the first deep learning network can be understood as a dimension reduction fusion network or a dimension reduction fusion module.

[0104] In this dimension reduction fusion network or dimension reduction fusion module, a three-dimensional slice sequence is generated into a two-dimensional image using deep learning technology, and in the embodiments of the present disclosure, the two-dimensional image can be referred to as a thin layer slice.

[0105] Specifically, each of the m slice sequences can be input into a dimension reduction fusion module based on deep learning technology to reduce and fuse the slice sequence with high-dimensional sparse voxels into a thin layer slice including refined information.

[0106] It should be noted that the generated two-dimensional image has the following characteristics: the length and width of the slices in the slice sequence are significantly greater than the thickness, and the dimensionality reduction two-dimensional image generated under such conditions is called a thin slice. The thin slice not only better preserves the significant features of the target, but also reduces the data dimension, and is also conducive to improving the recognition accuracy and recognition speed, and weakening the influence of security product shielding on target recognition. In addition, compared with the conversion method using the maximum value in the related art, the method according to the embodiments of the present disclosure has more advantages in identifying targets with a variety of different material compositions and complex physical properties, and is not limited to identifying prohibited items such as guns and knives mainly composed of metal materials.

[0107] FIGS. 6A and 6B are schematic diagrams of the dimensionality reduction fusion process of single-channel and multi-channel in the target recognition method for three-dimensional CT data according to embodiments of the present disclosure.

[0108] For example, referring to FIG. 6A, a slice sequence includes n slices, and the n slices of the slice sequence are input into the first deep learning model. After being processed by the first deep learning model, a two-dimensional image is output, that is, the channel number of the thin slice is single-channel.

[0109] For example, referring to FIG. 6B, a slice sequence includes n slices, and the n slices of the slice sequence are input into the first deep learning model. After being processed by the first deep learning model, a plurality of two-dimensional images are output, that is, the channel number of the thin slice is multi-channel.

[0110] In embodiments of the present disclosure, the channel number of the thin slice is flexibly set according to the characteristics of the target, which can be single-channel or multi-channel.

[0111] In some embodiments of the present disclosure, the method further includes: obtaining a mapping relationship between pixels in the two-dimensional image and voxels in the three-dimensional CT data in the process of generating the plurality of two-dimensional images. That is, in the process of generating the thin slice, the position information of the corresponding three-dimensional space is also saved, that is, the mapping relationship between the thin slice pixels and the three-dimensional CT voxel region, which is used for subsequent dimensionality increasing fusion.

[0112] Optionally or additionally, in some embodiments of the present disclosure, the ray scanning detection system further includes a detection device that can generate a two-dimensional scanning image, for example, can include a DR detection device that can generate a two-dimensional DR image. The two-dimensional DR image is obtained by a DR imaging device that is independently and additionally configured with the CT detection device. The two-dimensional DR image is an image of the same security inspection object as the three-dimensional CT data.

[0113] Correspondingly, the method further includes: obtaining a two-dimensional scanning image generated by the ray scanning detection system, for example, a two-dimensional DR image.

[0114] Referring to FIG. 4, in operation or step S440, target recognition is performed on the plurality of two-dimensional images to obtain a two-dimensional semantic description set of the target.

[0115] In an embodiment of the present disclosure, for each one or more thin layer slices of the acquired direction, a two-dimensional target recognition method is used to obtain a two-dimensional semantic description set of the target.

[0116] For example, the two-dimensional target recognition method can be a two-dimensional image processing method, or a traditional machine learning method, or a method based on deep neural network target detection, image segmentation or classification, etc.

[0117] For example, at least one two-dimensional image obtained in step S430 can be input as input into a second deep learning model (such as a neural network model), and after processing by the second deep learning model, a two-dimensional semantic description set is obtained as output.

[0118] Specifically, a two-dimensional position detection of the target can be performed using a deep learning based target detection neural network. The convolutional neural network used in the target detection task is a typical structure of deep learning in computer vision tasks, and such a convolutional neural network has the characteristics of local connection, weight sharing, and spatial resampling. These characteristics make the convolutional neural network have a certain degree of translation scale invariance. Here, the two-dimensional semantic description set includes at least one of class information and / or confidence in units of pixels, regions of interest, and / or two-dimensional images, or the two-dimensional semantic description set includes at least one of class information, confidence, and position information of the target in units of regions of interest and / or two-dimensional images. The class information indicates the class to which the target belongs. The position information can include center coordinates, bounding boxes, etc. The confidence indicates the possibility of the existence of the target, which can be a normalized scalar or vector.

[0119] That is, the two-dimensional semantic description set can include at least one of class information, confidence, etc. of a certain pixel, class information, position information of the target, confidence, etc. of a certain region of interest, and class information, position information of the target, confidence, etc. of a certain two-dimensional image. The at least one information can be included in a group, or can be included in different groups respectively.

[0120] In addition, the two-dimensional semantic description set can include other semantic information such as the pose of the target, the number of targets, etc. in addition to the class information, the confidence, and the position information.

[0121] It should be noted that in the embodiments of the present disclosure, the two-dimensional image recognition method for the two-dimensional image is not particularly limited, as long as it is a method capable of obtaining the above-mentioned two-dimensional semantic description set based on the two-dimensional image.

[0122] Alternatively or additionally, in some embodiments of the present disclosure, in operation or step S440, target recognition is performed on the plurality of two-dimensional images and at least one two-dimensional scan image (for example, a two-dimensional DR image), and a two-dimensional semantic description set of the target is obtained.

[0123] For example, the at least one two-dimensional image obtained in step S430 and the at least one two-dimensional scan image can be input as input into a second deep learning model (for example, a neural network model), and after processing of the second deep learning model, a two-dimensional semantic description set is obtained as output.

[0124] That is, in some embodiments, not only the two-dimensional reduced dimensionality image generated from the three-dimensional CT data is used for target recognition, but also the two-dimensional DR image directly obtained by the scanning detection system is used for target recognition.

[0125] Referring to FIG. 4, in operation or step S450, the two-dimensional semantic description set is dimensionally upgraded to obtain a three-dimensional recognition result of the target.

[0126] FIG. 7 is a schematic flowchart of obtaining a three-dimensional recognition structure in a target recognition method according to an exemplary embodiment of the present disclosure.

[0127] Specifically, as shown in FIG. 7, operation or step S450 can include sub-operation or sub-step S451 and sub-operation or sub-step S452.

[0128] In sub-operation or sub-step S451, the two-dimensional semantic description set is mapped to a three-dimensional space by back projection to obtain a three-dimensional probability map. It should be understood that back projection can be considered as the inverse process of projection.

[0129] Alternatively, the back projection process can be implemented by voxel driving or pixel driving, etc. Specifically, by voxel driving or pixel driving, a semantic feature matrix can be obtained, and the semantic feature matrix can be compressed into a three-dimensional probability map.

[0130] For example, voxel driving includes: corresponding each voxel in the three-dimensional CT data to a pixel in each two-dimensional image, querying and accumulating the two-dimensional semantic description information corresponding to the pixel, and generating a semantic feature matrix.

[0131] The mapping relationship between the voxels and the pixels can establish a mapping function or a lookup table to improve the calculation speed. For example, in an embodiment of the present disclosure, during the generation of the thin layer slice, the position information of the corresponding three-dimensional space, i.e., the mapping relationship between the thin layer slice pixels and the three-dimensional CT voxel region, is saved at the same time, and these mapping relationships can be used to establish a mapping function or a lookup table here to improve the calculation speed of mapping from the two-dimensional pixels to the three-dimensional voxels.

[0132] As described above, according to the voxel driving, each voxel in the three-dimensional CT data is traversed, and the semantic feature matrix thereof is obtained in sequence, and finally the three-dimensional probability map is obtained by compressing the semantic feature matrix.

[0133] In this embodiment, the voxel driving can operate in parallel for each voxel, so the operation speed is fast, and the real-time performance of security check can be improved.

[0134] For another example, the pixel driving includes: each pixel in a two-dimensional image corresponds to a straight line in the three-dimensional CT data, each pixel or each pixel in a region of interest in each two-dimensional image is traversed, the two-dimensional semantic description information corresponding to the pixel is propagated along the straight line into the three-dimensional space, and a semantic feature matrix is generated, wherein the region of interest is given by the two-dimensional semantic description set. The corresponding relationship between the voxels and the pixels can also be obtained through a mapping function or a lookup table.

[0135] As described above, according to the pixel driving, each pixel in the plurality of two-dimensional images obtains the semantic feature matrix thereof in sequence, and finally the three-dimensional probability map is obtained by compressing the semantic feature matrix.

[0136] In this embodiment, the pixel driving can also operate in parallel according to the pixels, which is also beneficial to improve the operation speed and improve the real-time performance of security check.

[0137] According to the above description of the voxel driving and the pixel driving, it can be known that the semantic feature matrix is generated from the two-dimensional semantic description information according to the spatial corresponding relationship thereof, and is a matrix obtained by digitizing and intensifying the two-dimensional semantic description information. For example, for the category information in the two-dimensional semantic description set, the semantic feature matrix can be obtained according to the category of each target. For example, it can be assumed that the corresponding value in the semantic feature matrix is 1 when it belongs to the category, and the corresponding value in the semantic feature matrix is 0 when it does not belong to the category. In addition, the semantic feature matrix can also be obtained in a similar manner for other semantic information in the two-dimensional semantic description set.

[0138] The typical method for compressing the semantic feature matrix includes weighted average, principal component analysis, etc. At this time, the input is the semantic feature matrix, and the output is the probability map.

[0139] As an example, assuming there are two two-dimensional images, thus having two sets of two-dimensional semantic descriptions, where the semantic information on a certain pixel (or region of interest or two-dimensional image) is represented by a numerical value as 1 or 0, the back projection method can be used to map to the three-dimensional space to generate a semantic feature matrix corresponding to the three-dimensional space, the value in this matrix is a vector composed of 0 or 1, and the weighted average method can be used to calculate the probability map value of the voxel in the corresponding three-dimensional space. For example, the semantic feature matrix value of a certain voxel is v = [0, 1], and under the condition of the same weight, the probability map value of the voxel is 0.5. When compressing the semantic feature matrix corresponding to all target categories, the dimension of the output probability map value is determined by the number of target categories. The way to obtain the probability map value described here is only an example, and the probability map value can also be obtained in other ways. For example, the weights can be different, and the semantic feature matrix values are weighted with different weights to obtain the probability map value.

[0140] As another example, one or more vectors in the three-dimensional semantic feature matrix can also be used as input variables in principal component analysis, and the input variables are subjected to principal component analysis to obtain output variables as principal components, and the output variables are normalized as the probability map value of the corresponding voxel.

[0141] By using the operation method as described above, not only the real-time performance of the operation can be ensured, but also the two-dimensional recognition results can be effectively integrated to improve the final recognition effect.

[0142] In the sub-operation or sub-step S452, feature extraction is performed on the three-dimensional probability map to obtain a three-dimensional recognition result of the target.

[0143] In an embodiment of the present disclosure, at least one of the image processing method, the classical machine learning method, the deep learning method or the combination thereof is used for feature extraction on the three-dimensional probability map, so as to obtain a three-dimensional image semantic description set as the three-dimensional recognition result.

[0144] As an example, the three-dimensional probability map is input into a deep learning model as an input, and the three-dimensional recognition result such as confidence and three-dimensional bounding box is obtained as an output. The deep learning model used here can use a classification neural network with fewer layers or a target detection network and the like. By using such a technology, the amount of information contained in the original three-dimensional CT data is effectively simplified and abstracted after the above steps, which is closer to the final goal of contraband identification, and a simple feature extraction method can quickly and accurately extract a three-dimensional semantic description set.

[0145] For example, the three-dimensional image semantic description set includes, in units of voxels, three-dimensional regions of interest, and one or more of three-dimensional CT images: class information and / or confidence; or the three-dimensional image semantic description set includes, in units of three-dimensional regions of interest and / or three-dimensional CT images: at least one of class information, location information of the target, and confidence. The location information of the target in the three-dimensional CT image can include a three-dimensional bounding box.

[0146] That is, the three-dimensional image semantic description set includes: class information of a certain voxel belonging to a target, confidence, etc.; class information of a certain three-dimensional region of interest (VOI) including a target, location information of the target, confidence, etc.; and at least one of class information, location information of the target, and confidence of a certain three-dimensional CT image including a target. The at least one information can be information included in a group, or information included in different groups respectively.

[0147] Since the three-dimensional image semantic description set is generated from the three-dimensional probability map generated based on the two-dimensional semantic description set, the types of semantic information included in the three-dimensional image semantic description set and the types of semantic information included in the two-dimensional semantic description set are consistent or convertible.

[0148] In the embodiments of the present disclosure, by the dimensionality increasing processing described above, the two-dimensional semantic description set is dimensionality increased to obtain the three-dimensional recognition result of the target, thereby solving the problem of significant reduction of information amount when only two-dimensional recognition is performed by dimensionality reduction, and being able to reduce the loss of information amount while adopting two-dimensional recognition, and taking into account the real-time performance and accuracy of security inspection.

[0149] FIG. 8 is a detailed flowchart of obtaining a three-dimensional recognition structure in a target recognition method according to an exemplary embodiment of the present disclosure. As another example of the sub-operation or sub-step S452, for example, an image processing method can be adopted, as shown in FIG. 8, and the sub-operation or sub-step S452 can include sub-operations or sub-steps S4521-S4523.

[0150] In the sub-operation or sub-step S4521, the three-dimensional probability map is binarized to obtain a three-dimensional binary map.

[0151] In the sub-operation or sub-step S4522, the three-dimensional binary map is subjected to connected region analysis to obtain a connected region.

[0152] As an example, the three-dimensional binary map can be subjected to connected component labeling, and for each labeled region, a mask operation is performed to obtain a connected region.

[0153] In the sub-operation or sub-step S4523, a three-dimensional image semantic description set is generated for the connected region.

[0154] In this embodiment, the three-dimensional image semantic description set can include a three-dimensional bounding box. By including the three-dimensional bounding box, the spatial boundary of the target on the three-dimensional image can be given, the position, range, pose, shape, etc. of the target can be more intuitively shown, and the accuracy of the judgment of the security officer on whether the target is a dangerous article is improved.

[0155] As an example, all probability values in the connected region can be extracted, principal component analysis is performed to obtain an analysis set, and the analysis set is taken as an object effective voxel region. The three-dimensional image semantic description set is counted for the effective voxel region. In this way, the accuracy of three-dimensional recognition can be further improved.

[0156] Optionally or additionally, in operation or step S450, not only the two-dimensional semantic description set of the two-dimensional reduced dimension image is dimensioned, but also the two-dimensional semantic description set of the two-dimensional DR image is dimensioned, thereby obtaining a three-dimensional recognition result.

[0157] It should be understood that the two-dimensional DR image is a two-dimensional image of different principles and properties from the two-dimensional reduced dimension image generated by reducing the three-dimensional CT data. By also using such a two-dimensional DR image for target recognition, the amount of information used for recognition can be increased, thereby improving the accuracy of recognition.

[0158] FIG. 9 is a schematic flowchart of a target recognition method for three-dimensional CT data according to some other exemplary embodiments of the present disclosure.

[0159] Optionally or additionally, as shown in FIG. 9, after operation or step S450, the method can further include operation or step S460. In operation or step S460, the three-dimensional recognition result generated in operation or step S450 is projected onto a two-dimensional DR image, and is output as a recognition result of the two-dimensional DR image.

[0160] Due to work habits and needs, some security personnel want to confirm the recognition result on the two-dimensional DR image. However, if the recognition result of the two-dimensional DR image is directly used, when the target on the DR image is severely occluded or has a special placement pose, the information of the target is not complete, which affects the recognition accuracy. The three-dimensional recognition result effectively integrates the semantic information of several two-dimensional views, and is more accurate and reliable. Therefore, by projecting the three-dimensional recognition result onto the two-dimensional DR image as the recognition result for output, the work needs of the security personnel to confirm the recognition result through the two-dimensional DR image are met, and the accuracy of the recognition result is improved.

[0161] In some embodiments of the present disclosure, the result of operation or step S450 and the result of operation or step S460 can also be output simultaneously. In this embodiment, the three-dimensional recognition result and the recognition result on the two-dimensional DR image can be compared and verified with each other, thereby facilitating the security personnel to more accurately judge whether the target is a dangerous article.

[0162] In the embodiments of the present disclosure, a plurality of slice sequences are generated by dimension reduction from three-dimensional CT data, a plurality of thin layer slices are generated according to the slice sequences, target recognition is performed by using a plurality of two-dimensional images including the plurality of thin layer slices, a two-dimensional semantic description set is obtained, and three-dimensional recognition results are obtained by dimension increase of the two-dimensional semantic description set, that is, three-dimensional results are generated by dimension reduction from three-dimensional first, recognition by thin layer slices, and then dimension increase, thereby, both the target with complex material composition and physical properties and the target with shape features can be effectively recognized based on two-dimensional recognition, and the two-dimensional recognition results can be effectively integrated to give three-dimensional recognition results with rich information, so that the recognition effect of the target can be improved, and the real-time requirement of security check can be met.

[0163] Based on the above method, the embodiments of the present disclosure further provide a target recognition device for three-dimensional CT data. The device will be described in detail below in combination with FIG. 10.

[0164] FIG. 10 is a structural block diagram of a target recognition device for three-dimensional CT data according to some exemplary embodiments of the present disclosure. Referring to FIG. 10, the device 900 includes an acquisition module 910, a slicing module 920, a two-dimensional image generation module 930, a two-dimensional recognition module 940, and a dimension increase module 950.

[0165] The acquisition module 910 is configured to acquire three-dimensional CT data. In some exemplary embodiments, the acquisition module 910 can be configured to perform the operation S410 and its sub-operations described above, which will not be repeated here.

[0166] The slicing module 920 is configured to slice the three-dimensional CT data along a plurality of slicing directions to obtain m slice sequences, wherein at least one of the m slice sequences includes n slices, m is a positive integer greater than or equal to 1, and n is a positive integer greater than or equal to 2. In some exemplary embodiments, the slicing module 920 can be configured to perform the operation S420 and its sub-operations described above, which will not be repeated here.

[0167] For example, the slicing module 920 is configured to determine the plurality of slicing directions according to at least one of the occlusion of the target, the complexity of the target category, the shape, position, and attitude of the target.

[0168] For example, at least two of the m slice sequences correspond to non-overlapping slicing regions; and / or, at least two of the m slice sequences correspond to partially overlapping slicing regions.

[0169] For example, the slicing module 920 is further configured to determine the number n of slices included in at least one of the m slice sequences according to the size of the target.

[0170] The two-dimensional image generation module 930 is configured to generate a plurality of two-dimensional images according to the m slice sequences. In some example embodiments, the two-dimensional image generation module 930 can be configured to perform the operation S430 and sub-operations thereof as described above, and details are not repeated here.

[0171] For example, the two-dimensional image generation module 930 is configured to process at least one slice sequence including n slices by using a first deep learning model to output at least one two-dimensional image.

[0172] For example, the two-dimensional image generation module 930 is configured to obtain a mapping relationship between pixels in the two-dimensional images and voxels in the three-dimensional CT data during the process of generating the plurality of two-dimensional images.

[0173] The two-dimensional recognition module 940 is configured to perform target recognition on the plurality of two-dimensional images to obtain a two-dimensional semantic description set of the target. In some example embodiments, the two-dimensional recognition module 940 can be configured to perform the operation S440 and sub-operations thereof as described above, and details are not repeated here.

[0174] For example, the two-dimensional recognition module 940 is configured to perform target recognition on the plurality of two-dimensional images by using at least one of or a combination of an image processing method, a classical machine learning method, and a deep learning method for two-dimensional images to obtain a two-dimensional semantic description set of the target.

[0175] The dimensionality lifting module 950 is configured to lift the dimensionality of the two-dimensional semantic description set to obtain a three-dimensional recognition result of the target. In some example embodiments, the dimensionality lifting module 950 can be configured to perform the operation S450 and sub-operations thereof as described above, and details are not repeated here.

[0176] For example, the dimensionality lifting module 950 is configured to map the two-dimensional semantic description set to a three-dimensional space by using a back projection method to obtain a three-dimensional probability map, and perform feature extraction on the three-dimensional probability map to obtain the three-dimensional recognition result of the target.

[0177] For example, the dimensionality lifting module 950 is configured to extract a candidate three-dimensional region of interest from the three-dimensional CT data, map the two-dimensional semantic description set to a three-dimensional space by using a back projection method to obtain a three-dimensional probability map, and process the candidate three-dimensional region of interest and the three-dimensional probability map by using a three-dimensional target recognition method to obtain the three-dimensional recognition result of the target.

[0178] Optionally or additionally, in some embodiments, the plurality of two-dimensional images further include a two-dimensional scan image, and the two-dimensional scan image is obtained by a two-dimensional imaging device.

[0179] Correspondingly, the apparatus can further include a projection module configured to project the obtained three-dimensional recognition result of the target to the two-dimensional scan image.

[0180] According to an embodiment of the present disclosure, the acquisition module 910, the segmentation module 920, the two-dimensional image generation module 930, the two-dimensional recognition module 940 and the dimensionality rising module 950 and any number of sub-modules thereof comprised in the apparatus 900 can be combined in one module, or any one of them can be split into multiple modules. Alternatively, at least part of the function of one or more of these modules can be combined with at least part of the function of other modules, and implemented in one module. According to an embodiment of the present disclosure, at least one of the acquisition module 910, the segmentation module 920, the two-dimensional image generation module 930, the two-dimensional recognition module 940 and the dimensionality rising module 950 and sub-modules thereof comprised in the apparatus 900 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on board, a system on package, an application specific integrated circuit (ASIC), or any other reasonable way of hardware or firmware by integrating or packaging circuits, or implemented in any one of software, hardware and firmware or in a proper combination of any number of them. Alternatively, at least one of the acquisition module 910, the segmentation module 920, the two-dimensional image generation module 930, the two-dimensional recognition module 940 and the dimensionality rising module 950 and sub-modules thereof comprised in the apparatus 900 can be at least partially implemented as computer program modules which can perform corresponding functions when they are run.

[0181] FIG. 11 schematically shows a structural block diagram of an electronic device suitable for implementing the target recognition method according to an exemplary embodiment of the present disclosure.

[0182] As shown in FIG. 11, the electronic device 1100 according to an embodiment of the present disclosure includes a processor 1101 which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 1102 or loaded from a storage portion 1108 to a random access memory (RAM) 1103. The processor 1101 can include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset, and / or a special-purpose microprocessor (such as an application specific integrated circuit (ASIC)), and the like. The processor 1101 can also include an on-board memory for cache use. The processor 1101 can include a single processing unit or a plurality of processing units for performing different actions of the method processes according to embodiments of the present disclosure.

[0183] For example, the electronic device can be the data processing apparatus 60 of the ray scanning detection system.

[0184] In the RAM 1103, various programs and data required for the operation of the electronic device 1100 are stored. The processor 1101, the ROM 1102, and the RAM 1103 are connected to each other via the bus 1104. The processor 1101 performs various operations of the method flow according to the embodiments of the present disclosure by executing the programs in the ROM 1102 and / or the RAM 1103. It should be noted that the programs can also be stored in one or more memories other than the ROM 1102 and the RAM 1103. The processor 1101 can also perform various operations of the method flow according to the embodiments of the present disclosure by executing the programs stored in the one or more memories.

[0185] According to an embodiment of the present disclosure, the electronic device 1100 can further include an input / output (I / O) interface 1105, which is also connected to the bus 1104. The electronic device 1100 can further include one or more of the following components connected to the I / O interface 1105: an input part 1106 including a keyboard, a mouse, etc.; an output part 1107 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage part 1108 including a hard disk, etc.; and a communication part 1109 including a network interface card such as a LAN card, a modem, etc. The communication part 1109 performs communication processing via a network such as the Internet. A drive 1110 is also connected to the I / O interface 1105 as necessary. A removable medium 1111 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is mounted on the drive 1110 as necessary, so that a computer program read out therefrom is installed in the storage part 1108 as necessary.

[0186] The present disclosure also provides a computer readable storage medium, which can be included in the device / apparatus / system described in the above embodiments; or can exist separately without being assembled into the device / apparatus / system. The above computer readable storage medium carries one or more programs, when the one or more programs are executed, the method according to the embodiments of the present disclosure is implemented.

[0187] According to an embodiment of the present disclosure, the computer readable storage medium can be a nonvolatile computer readable storage medium, for example, can include, but is not limited to, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any appropriate combination thereof. In the present disclosure, the computer readable storage medium can be any tangible medium that includes or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present disclosure, the computer readable storage medium can include one or more memories such as the ROM 1102 and / or the RAM 1103 described above and / or one or more memory other than the ROM 1102 and the RAM 1103.

[0188] Embodiments of the present disclosure also include a computer program product including a computer program that includes program code for executing the methods illustrated in the flowcharts. When the computer program product is run in the computer system, the program code is used to cause the computer system to implement the methods provided by the embodiments of the present disclosure.

[0189] The above-described functions defined in the system / apparatus / module / units of the embodiments of the present disclosure are performed when the computer program is executed by the processor 1101. According to an embodiment of the present disclosure, the above-described system, apparatus, module, unit, etc. can be implemented by the computer program modules.

[0190] In one embodiment, the computer program can rely on a tangible storage medium such as an optical storage device, a magnetic storage device, etc. In another embodiment, the computer program can also be transmitted, distributed, and / or downloaded in the form of a signal over a network medium and installed and / or installed from a removable medium 1111 through the communication part 1109. The program code included in the computer program can be transmitted using any appropriate network medium, including but not limited to wireless, wired, etc., or any appropriate combination thereof.

[0191] In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 1109 and / or installed from the removable medium 1111. When the computer program is executed by the processor 1101, the above-described functions defined in the system of the embodiments of the present disclosure are performed. According to an embodiment of the present disclosure, the above-described system, apparatus, device, module, unit, etc. can be implemented by the computer program modules.

[0192] According to the embodiments of the present disclosure, the program code for executing the computer program provided by the embodiments of the present disclosure can be written in any combination of one or more programming languages, and specifically, the computer program can be implemented by using a high-level procedure and / or object-oriented programming language, and / or assembly / machine language. The programming language includes, but is not limited to, a programming language such as Java, C++, python, "C" language or a similar programming language. The program code can be executed completely on a user computing device, partially on a user device, partially on a remote computing device, or completely on a remote computing device or server. In the case involving a remote computing device, the remote computing device can be connected to the user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, connected through the Internet by using an Internet service provider).

[0193] The flowcharts and block diagrams in the drawings illustrate the possible implementation architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each block in the flowcharts or block diagrams can represent a module, a program segment, or a part of code including one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in different orders than those noted in the drawings. For example, two blocks that are shown in succession can actually be executed substantially in parallel, and sometimes they can be executed in reverse order, depending on the involved functions. It should also be noted that each block in the block diagrams or flowcharts, and the combination of blocks in the block diagrams or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0194] The target identification method and device, the electronic device, the ray scanning detection system, the computer readable storage medium and the program product provided by the embodiments of the present disclosure have at least one aspect of the following beneficial effects:

[0195] In the security check CT detection process, the three-dimensional data is large in scale, the materials of the to-be-detected articles are various in type, the shapes and sizes of the to-be-detected articles are different, and the background of the luggage is complex. The target is often blocked or surrounded. How to efficiently use the three-dimensional CT image to realize fast and accurate automatic identification has great challenges. In the embodiments of the present disclosure, a dimension reduction method of thin layer slicing is used to efficiently obtain a two-dimensional semantic description set, and then a three-dimensional recognition result is obtained by dimension increasing, so that the speed and accuracy of three-dimensional CT automatic identification are effectively improved.

[0196] The above describes embodiments of the present disclosure. However, these embodiments are merely for illustrative purposes, and are not intended to limit the scope of the present disclosure. Although each embodiment is described above separately, this does not mean that the measures in each embodiment cannot be used advantageously in combination. The scope of the present disclosure is defined by the appended claims and their equivalents. Those skilled in the art can make various substitutions and modifications without departing from the scope of the present disclosure, and these substitutions and modifications should all fall within the scope of the present disclosure.

Claims

1. A target recognition method for three-dimensional CT data, comprising: obtaining three-dimensional CT data; slicing the three-dimensional CT data along a plurality of slicing directions to obtain m slice sequences, wherein at least one of the m slice sequences comprises n slices, m is a positive integer greater than or equal to 1, and n is a positive integer greater than or equal to 2; generating a plurality of two-dimensional images according to the m slice sequences; performing target recognition on the plurality of two-dimensional images to obtain a two-dimensional semantic description set of a target; and dimensionally upgrading the two-dimensional semantic description set to obtain a three-dimensional recognition result of the target.

2. The method of claim 1, wherein, The generating a plurality of two-dimensional images according to the m slice sequences comprises: processing at least one slice sequence comprising the n slices by using a first deep learning model to output at least one two-dimensional image.

3. The method of claim 2, wherein, The processing at least one slice sequence comprising the n slices by using a first deep learning model to output at least one two-dimensional image comprises: processing a first slice sequence comprising n1 slices by using the first deep learning model to output k1 two-dimensional images, wherein n1 is greater than or equal to 2, and k1 is greater than or equal to 1 and less than n1; and / or processing a second slice sequence comprising n2 slices by using the first deep learning model to output k2 two-dimensional images, wherein n2 is greater than or equal to 2, and k2 is greater than or equal to 1 and less than n2.

4. The method of any one of claims 1-3, wherein, The method further comprises: obtaining a mapping relationship between pixels in the two-dimensional images and voxels in the three-dimensional CT data in the process of generating the plurality of two-dimensional images.

5. The method of any one of claims 1-3, wherein, The performing target recognition on the plurality of two-dimensional images to obtain a two-dimensional semantic description set of a target comprises: performing target recognition on the plurality of two-dimensional images by using at least one of or in combination of an image processing method, a classical machine learning method, and a deep learning method for two-dimensional images to obtain the two-dimensional semantic description set of the target.

6. The method of any one of claims 1-3, wherein, The method further comprises: determining the plurality of slicing directions according to at least one of a target occlusion condition, a target category complexity, a target shape, a target position, and a target pose.

7. The method of any one of claims 1-3, wherein, At least two of the m slice sequences correspond to non-overlapping slicing regions; and / or At least two of the m slice sequences correspond to partially overlapping slicing regions.

8. The method of any one of claims 1-3, wherein, The method further comprises: determining the number n of slices included in at least one of the m slice sequences according to a target size.

9. The method of any one of claims 1-3, wherein, The dimensionally upgrading the two-dimensional semantic description set to obtain a three-dimensional recognition result of the target comprises: mapping the two-dimensional semantic description set to a three-dimensional space by using back projection to obtain a three-dimensional probability map; and performing feature extraction on the three-dimensional probability map to obtain the three-dimensional recognition result of the target.

10. The method of any one of claims 1-3, wherein, The dimensionally upgrading the two-dimensional semantic description set to obtain a three-dimensional recognition result of the target comprises: extracting a candidate three-dimensional region of interest from the three-dimensional CT data; mapping the two-dimensional semantic description set to a three-dimensional space by using back projection to obtain a three-dimensional probability map; and The three-dimensional target recognition method is used to process the candidate three-dimensional region of interest and the three-dimensional probability map to obtain a three-dimensional recognition result of the target.

11. The method of any one of claims 1-3, wherein, The plurality of two-dimensional images further includes a two-dimensional scan image, which is acquired by a two-dimensional imaging device.

12. The method of any one of claims 1-3, wherein, The method further includes projecting the obtained three-dimensional recognition result of the target to the two-dimensional scan image.

13. A target recognition apparatus for three-dimensional CT data, comprising: an acquisition module configured to acquire three-dimensional CT data; a slicing module configured to slice the three-dimensional CT data along a plurality of slicing directions to obtain m slice sequences, wherein at least one of the m slice sequences includes n slices, m is a positive integer greater than or equal to 1, and n is a positive integer greater than or equal to 2; a two-dimensional image generation module configured to generate a plurality of two-dimensional images according to the m slice sequences; a two-dimensional recognition module configured to perform target recognition on the plurality of two-dimensional images to obtain a two-dimensional semantic description set of a target; and a dimension lifting module configured to lift the dimension of the two-dimensional semantic description set to obtain a three-dimensional recognition result of the target.

14. An electronic device, comprising: one or more processors; a memory configured to store one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1-12.

15. A ray scanning detection system, comprising: a ray source configured to emit rays; a detector configured to receive rays emitted from the ray source and passing through a detection object; and a data processing apparatus in communication connection with the detector, wherein the data processing apparatus comprises one or more processors and a memory configured to store one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1-12.

16. A computer readable storage medium having stored thereon a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1-12.

17. A computer program product comprising a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1-12. ​

Images