Cascade CT data target identification method and device and ray scanning detection system

Abstract

The invention provides a cascaded CT data target identification method. The method comprises the following steps: acquiring three-dimensional CT data; performing initial identification on the three-dimensional CT data to obtain a first object description set of a target; and according to the first object description set of the target, performing fine identification on the target by using a cascaded fine identification method to obtain a fine identification result of the target, and performing fine recognition on the target by using at least one of a first deep learning network for the three-dimensional CT data and a second deep learning network for the two-dimensional data to obtain a fine recognition result of the target.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of image processing and / or object detection, and particularly relates to a cascaded CT data object identification method and device, electronic equipment, ray scanning detection system, computer readable storage medium and program product. BACKGROUND

[0002] In the related art, security CT (Computed Tomography) equipment can provide three-dimensional images with more abundant information, and is widely used in luggage security inspection in customs, airports, railway stations, subway stations and other places.

[0003] With the development of science and technology, the accuracy and real-time requirement of target recognition in the security field is getting higher and higher. On the one hand, three-dimensional CT data contains more abundant information, on the other hand, three-dimensional CT data usually contains a large number of voxels, which is sparse, and also contains certain noise and redundant information. Identifying three-dimensional CT objects will face the problem of huge amount of calculation. In the security scene, it is difficult to identify targets with strong three-dimensional shape features but complex material composition and physical properties, and the real-time and accuracy requirements of three-dimensional CT data object identification in the security scene are also higher.

[0004] Therefore, how to effectively use three-dimensional CT data to identify targets is one of the key issues to improve the accuracy and real-time of the algorithm.

[0005] It should be noted that the above information disclosed in this part is only for understanding the background of the inventive concept of the present disclosure, therefore, the above information can contain information which does not constitute prior art. SUMMARY

[0006] In view of at least one aspect of the above technical problems, a cascaded CT data object identification method and device, electronic equipment, ray scanning detection system, computer readable storage medium and program product are provided.

[0007] In one aspect, a cascaded CT data object identification method is provided, the method comprising: obtaining three-dimensional CT data; performing preliminary identification on the three-dimensional CT data to obtain a first object description set of a target; and performing fine identification on the target according to the first object description set of the target by using a cascaded fine identification method to obtain a fine identification result of the target, wherein in the cascaded fine identification method, at least one of a first deep learning network for three-dimensional CT data and a second deep learning network for two-dimensional data can be selectively used to perform fine identification on the target to obtain the fine identification result of the target.

[0008] According to some exemplary embodiments, the first object description set comprises one or more of three-dimensional voxel data, point cloud data, feature vectors, and semantic descriptions.

[0009] According to some exemplary embodiments, the semantic description comprises at least one of category information, position information, and confidence of the target.

[0010] According to some exemplary embodiments, the preliminary identification of the three-dimensional CT data to obtain the first object description set of the target comprises: processing the three-dimensional CT data using a three-dimensional segmentation network based on deep learning to obtain the first object description set of the target; or processing the three-dimensional CT data using a three-dimensional target detection network to obtain the first object description set of the target; or processing the three-dimensional CT data in a target feature library comparison manner to obtain the first object description set of the target.

[0011] According to some exemplary embodiments, the preliminary identification of the three-dimensional CT data to obtain the first object description set of the target comprises: a slice region extraction step of obtaining data of one tomographic slice of the target to be detected; preprocessing the tomographic slice data; segmenting the preprocessed tomographic slice data into a plurality of regions with similar physical properties; in response to a certain segmented region being a slice region, extracting the slice region; a slice object formation step of judging whether the extracted slice region in the current tomographic slice can be merged with the extracted slice region in the previous tomographic slice; in response to the extracted slice region in the current tomographic slice being able to be merged with the extracted slice region in the previous tomographic slice, merging to form a slice object; and repeatedly executing the slice region extraction step and the slice object formation step to process the data of each tomographic slice using a slice-by-slice analysis method until all the data of the tomographic slices are processed to obtain the first object description set of the target.

[0012] Optionally, the preliminary identification of the three-dimensional CT data to obtain the first object description set of the target comprises: a block region extraction step of obtaining data of one tomographic slice of the target to be detected; preprocessing the tomographic slice data; segmenting the preprocessed tomographic slice data into a plurality of regions with similar physical properties; in response to a certain segmented region being a block region, extracting the block region; a block object formation step of judging whether the extracted block region in the current tomographic slice can be merged with the extracted block region in the previous tomographic slice; in response to the extracted block region in the current tomographic slice being able to be merged with the extracted block region in the previous tomographic slice, merging to form a block object; and repeatedly executing the block region extraction step and the block object formation step to process the data of each tomographic slice using a slice-by-slice analysis method until all the data of the tomographic slices are processed to obtain the object description set of the block target.

[0013] According to some exemplary embodiments, in the cascaded fine recognition method, a first deep learning network for three-dimensional CT data and a second deep learning network for two-dimensional data are used to perform fine recognition of the target in order to obtain a fine recognition result of the target.

[0014] According to some exemplary embodiments, the method further includes: generating a second object description set based on a first object description set of the target, wherein the first object description set is a first object description set for three-dimensional data, and the second object description set is a first object description set for two-dimensional data; the step of using a first deep learning network for three-dimensional CT data and a second deep learning network for two-dimensional data to perform fine recognition of the target specifically includes: using the first object description set as input to the first deep learning network for three-dimensional CT data, and processing the first object description set using the first deep learning network; and using the second object description set as input to the second deep learning network for two-dimensional data, and processing the second object description set using the second deep learning network.

[0015] According to some exemplary embodiments, using the first object description set as input to a first deep learning network for three-dimensional CT data includes: using one or more of the three-dimensional voxel data, point cloud data, feature vectors, and semantic descriptions corresponding to the target in the first object description set as input to the first deep learning network.

[0016] According to some exemplary embodiments, the input to the first deep learning network further includes: background region information in three-dimensional CT image data; and / or, the three-dimensional CT data includes a first three-dimensional image acquired by a first CT detection device; and the input to the first deep learning network further includes second three-dimensional data acquired by the first CT detection device or other detection devices different from the first CT detection device, wherein the second three-dimensional data includes a second three-dimensional image, the second three-dimensional image corresponds to the first three-dimensional image, and information in the first object description set of the target is mapped to the second three-dimensional image.

[0017] According to some exemplary embodiments, the second object description set includes slice data of the three-dimensional region of interest of the target along any direction, one or more of the feature vectors and semantic descriptions corresponding to the target in the slice data; and / or, the second object description set includes a two-dimensional projection image obtained by projecting the three-dimensional region of interest of the target, one or more of the feature vectors and semantic descriptions corresponding to the target in the two-dimensional projection image; and / or, the second object description set includes a two-dimensional projection image obtained by projecting the entire three-dimensional CT data, one or more of the feature vectors and semantic descriptions corresponding to the target in the two-dimensional projection image; and / or, the three-dimensional CT data includes a first three-dimensional image obtained by a first CT detection device; and the second object description set includes a two-dimensional image obtained by the first CT detection device or a detection device different from the first CT detection device, one or more of the feature vectors and semantic descriptions corresponding to the target in the two-dimensional image.

[0018] According to some exemplary embodiments, the step of processing the first object description set using the first deep learning network specifically includes: performing feature extraction using a pre-trained first deep learning network to obtain a feature map; and / or, the step of processing the second object description set using the second deep learning network specifically includes: performing feature extraction using a pre-trained second deep learning network to obtain a feature map.

[0019] According to some exemplary embodiments, the extracted features include at least one of shape, texture, and color in the 3D CT data.

[0020] According to some exemplary embodiments, processing the first object description set using the first deep learning network specifically includes: operating on the feature map using the first deep learning network, wherein the operation includes at least one of feature map fusion and feature map dimensionality reduction; and / or, processing the second object description set using the second deep learning network specifically includes: operating on the feature map using the second deep learning network, wherein the operation includes at least one of feature map fusion and feature map dimensionality reduction.

[0021] According to some exemplary embodiments, the step of processing the first object description set using the first deep learning network further includes: applying the feature map to a classification, segmentation, or detection task to obtain a three-dimensional image semantic description set; and / or, the step of processing the second object description set using the second deep learning network further includes: applying the feature map to a classification, segmentation, or detection task to obtain a two-dimensional image semantic description set.

[0022] According to some exemplary embodiments, a cascaded fine recognition method is used to perform fine recognition of the target to obtain a fine recognition result of the target, specifically including: a step of executing the first deep learning network to process the input in a single execution; and using the output of the first deep learning network after a single execution as the fine recognition result of the target; and / or, a step of executing the second deep learning network to process the input in a single execution; and using the output of the second deep learning network after a single execution as the fine recognition result of the target.

[0023] According to some exemplary embodiments, a cascaded fine recognition method is used to perform fine recognition of the target to obtain a fine recognition result of the target. Specifically, it includes: iteratively executing the first deep learning network to process the input; and using the output of the first deep learning network after multiple iterations as the fine recognition result of the target; and / or iteratively executing the second deep learning network to process the input; and using the output of the second deep learning network after multiple iterations as the fine recognition result of the target.

[0024] According to some exemplary embodiments, the fine identification result of the target is expressed in units of one or more voxels, three-dimensional regions of interest, and three-dimensional CT images, and includes: category information and / or confidence level; or, the fine identification result of the target is expressed in units of three-dimensional regions of interest and / or three-dimensional CT images, and includes: at least one of category information, confidence level, and target location information.

[0025] On the other hand, a cascaded CT data target recognition device is provided, the device comprising: an acquisition module for acquiring three-dimensional CT data; a preliminary recognition module for performing preliminary recognition on the three-dimensional CT data to obtain a first object description set of the target; and a fine recognition module for performing fine recognition on the target based on the first object description set of the target using a cascaded fine recognition method to obtain a fine recognition result of the target, wherein, in the cascaded fine recognition method, at least one of a first deep learning network for three-dimensional CT data and a second deep learning network for two-dimensional data is used to perform fine recognition on the target to obtain a fine recognition result of the target.

[0026] In another aspect, an electronic device is provided, comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method described above.

[0027] In another aspect, a radiation scanning detection system is provided, comprising: a radiation source for emitting radiation; a detector for receiving radiation emitted from the radiation source and passing through a target object; and a data processing device communicatively connected to the detector, wherein the data processing device includes: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method described above.

[0028] In another aspect, a computer-readable storage medium is provided, on which a computer program is stored, characterized in that the computer program, when executed by a processor, implements the steps of the method described above.

[0029] In another aspect, a computer program product is provided, comprising a computer program, characterized in that, when the computer program is executed by a processor, it implements the steps of the method described above.

[0030] In the embodiments of this disclosure, after obtaining the first object description set of the target using the initial identification method, the cascaded fine identification method performs further identification, effectively utilizing the three-dimensional information of the CT data, which can further distinguish between contraband and safe items, and improve the accuracy of target identification. Attached Figure Description

[0031] To better understand the present invention, it will be described in detail with reference to the following drawings:

[0032] Figure 1 This is a schematic diagram of the structure of a X-ray scanning detection system according to some exemplary embodiments of the present invention.

[0033] Figure 2 This is a schematic diagram of a CT scanning apparatus that implements a reconstruction scanning method according to some exemplary embodiments of the present disclosure.

[0034] Figure 3 An example of attenuated signal data obtained according to an embodiment of the present disclosure is shown.

[0035] Figure 4 This is a schematic flowchart of a cascaded CT data target recognition method according to an embodiment of the present disclosure.

[0036] Figure 5 This is a schematic flowchart of the initial identification operation in a cascaded CT data target recognition method according to an embodiment of the present disclosure.

[0037] Figure 6 This is a schematic flowchart of the fine recognition operation in a cascaded CT data target recognition method according to an embodiment of the present disclosure.

[0038] Figure 7This is an architecture diagram of a cascaded deep learning network used in a target recognition method according to an exemplary embodiment of the present disclosure.

[0039] Figure 8 This is a schematic flowchart of a cascaded CT data target recognition method according to other embodiments of the present disclosure.

[0040] Figure 9 This is a structural block diagram of a cascaded CT data target recognition device according to an exemplary embodiment of the present disclosure.

[0041] Figure 10 The diagram schematically illustrates a structural block diagram of an electronic device suitable for implementing the target recognition method according to exemplary embodiments of the present disclosure. Detailed Implementation

[0042] Specific embodiments of the present invention will now be described in detail. It should be noted that the embodiments described herein are for illustrative purposes only and are not intended to limit the invention. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the invention. However, it will be apparent to those skilled in the art that these specific details are not necessary to practice the invention. In other instances, well-known structures, materials, or methods have not been specifically described to avoid obscuring the invention.

[0043] Throughout this specification, references to "an embodiment," "an example," or "an example" mean that a particular feature, structure, or characteristic described in connection with that embodiment or example is included in at least one embodiment of the invention. Therefore, the phrases "in an embodiment," "in an embodiment," "an example," or "an example" appearing in various places throughout the specification do not necessarily refer to the same embodiment or example. Furthermore, specific features, structures, or characteristics can be combined in one or more embodiments or examples in any suitable combination and / or sub-combination. Moreover, those skilled in the art will understand that the term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0044] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0045] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0046] The embodiments of this disclosure will now be described in detail using CT scan detection as an example. It should be understood that the embodiments of this disclosure are not limited to CT scan detection scenarios, but can be applied to various scanning detection scenarios. For example, it can be applied to scanning detection scenarios that include various different inspection objects, including but not limited to vehicle scanning detection, luggage / parcel scanning detection, human body scanning detection, etc. It should be noted that the description of scanning detection scenarios here is not an exhaustive description, and the description here should not be construed as limiting the scope of protection of this disclosure.

[0047] It should be noted that the embodiments disclosed herein are not limited to object detection tasks, but can also be applied to tasks such as object classification and image segmentation. Therefore, unless otherwise stated herein, the term "object recognition" includes, but is not limited to, various tasks such as object detection, object classification, and image segmentation.

[0048] It should also be noted that in this article, the term "target" may include, but is not limited to, objects of interest or items of interest in a scanning detection scene.

[0049] Figure 1 This is a schematic diagram of the structure of a radiographic scanning detection system according to some exemplary embodiments of the present invention. Figure 1 In the diagram, a CT scanning device is schematically shown as an example of the aforementioned X-ray scanning detection system. For example... Figure 1 As shown, the CT scanning apparatus according to this embodiment includes: a gantry 20, a support mechanism 40, a controller 50, and a data processing device 60 (e.g., a computer). The gantry 20 includes a radiation source 10, such as an X-ray machine, that emits X-rays for examination, and a detection and acquisition device 30. The support mechanism 40 carries the object to be examined (e.g., luggage to be inspected) 70 through the scanning area between the radiation source 10 and the detection and acquisition device 30 of the gantry 20. Simultaneously, the gantry 20 rotates about the direction of travel of the object to be examined 70, so that the radiation emitted by the radiation source 10 can pass through the object to be examined 70, allowing for a CT scan of the object. The detection and acquisition device 30 includes, for example, a detector and a data acquisition unit with an integrated modular structure, such as a flat panel detector, for detecting the radiation passing through the object to be examined 70 (e.g., luggage to be inspected), obtaining analog signals, and converting the analog signals into digital signals, thereby outputting projection data of the object to be examined 70 against the X-rays. The controller 50 is used to control the synchronous operation of all parts of the entire system. The data processing unit 60 is used to process the data collected by the data acquisition unit, process and reconstruct the data, and output the results.

[0050] like Figure 1As shown, the X-ray source 10 is placed on one side where the object to be inspected can be placed, and the detection and acquisition device 30 is placed on the other side of the object to be inspected 70. The device includes a detector and a data acquisition unit for acquiring transmission data and / or multi-angle projection data of the object to be inspected 70. The data acquisition unit includes a data amplification and shaping circuit, which can operate in (current) integration mode or pulse (counting) mode. The data output cable of the detection and acquisition device 30 is connected to the controller 50 and the data processing device 60, and the acquired data is stored in the data processing device 60 according to a trigger command.

[0051] In embodiments of this disclosure, the data processing device 60 may deploy a radiographic scanning detection module. This module may include an image recognition model or a target recognition model. The radiographic scanning detection module can use the image recognition model or the target recognition model to identify the acquired on-site data (e.g., radiographic scanning images). This identification may include target detection, for example, detecting target objects or objects of interest in the radiographic scanning images. For example, the target objects or objects of interest may be various contraband. It should be understood that the specific category of the target objects or objects of interest depends on the scanning and detection site where the radiographic scanning detection system is deployed. In other words, the specific category of the target objects or objects of interest is determined by the user of the radiographic scanning detection system (also referred to herein as the user) according to specific scanning and detection needs, and it can be dynamically adjusted according to the scanning and detection requirements.

[0052] It should be noted that, for ease of description, the target object or object of interest in this article can be uniformly referred to as the target.

[0053] In embodiments of this disclosure, the radiation source 10 may be, for example, an X-ray machine, and a suitable X-ray machine focal size may be selected based on the imaging resolution. In other embodiments, an X-ray machine may not be used; instead, a linear accelerator or the like may be used to generate the X-ray beam.

[0054] The detection and acquisition device 30 includes an X-ray detector and a data acquisition circuit, etc. The X-ray detector can be a solid-state detector, a gas detector, or other detectors, and the embodiments disclosed herein are not limited thereto. The data acquisition circuit includes a readout circuit, an acquisition trigger circuit, and a data transmission circuit, etc.

[0055] The combination of controller 50 and data processing device 60 includes, for example, a computer device equipped with control and data processing programs, which is responsible for controlling the operation of the CT scanning device, including mechanical rotation, electrical control, safety interlock control, etc., reconstructing CT images from projection data, training image recognition models or target recognition models, and using the trained image recognition models or target recognition models to recognize X-ray scan images, etc.

[0056] Figure 2 This is a schematic diagram of a CT scanning apparatus implementing a reconstruction scanning method according to some exemplary embodiments of the present disclosure. Figure 2 As shown, a portion of the object 70 in the X-ray transmission field 45 emitted from the X-ray source 10 is received by the detection and acquisition device 30, converted into an electrical signal, and then into a digital signal representing the attenuation value. This signal is used as projection data and reconstructed by a computer to obtain a high-quality image.

[0057] According to embodiments of this disclosure, the object 70 is scanned using the aforementioned CT scanning device to obtain the original attenuation signal. The attenuation signal data corresponding to a row of detectors can also be displayed as a two-dimensional image. Figure 3 An example of attenuated signal data obtained according to embodiments of the present disclosure is shown. For example... Figure 3 The horizontal axis of the raw attenuation signal represents the pixel sequence along the detector column direction (e.g., from 1 to 256), while the vertical axis represents the angle (e.g., from 1 degree to 360 degrees). The raw attenuation signal is preprocessed to become projection data. For example, the projection data can be preprocessed by a CT scanning device using a negative logarithmic transformation to obtain the projection data. Then, the data processing device 60 executes a reconstruction procedure, for example, employing mainstream analytical reconstruction methods in the relevant technical field to obtain a reconstructed image, which is the X-ray scan image mentioned in the embodiments of this disclosure.

[0058] In embodiments of this disclosure, the data processing device 60 can perform target recognition (e.g., target detection) on the reconstructed image (i.e., the X-ray scan image) to identify targets (e.g., contraband) in the X-ray scan image. Specifically, the data processing device 60 may deploy a X-ray scan detection module, which is equipped with an image recognition model or a target recognition model. The X-ray scan detection module can use the image recognition model or the target recognition model to identify the X-ray scan image.

[0059] The applicant's research revealed that 3D CT data typically contains a massive number of voxels, exhibiting sparsity, and also includes a certain amount of noise and redundant information. Identifying 3D CT targets presents a significant computational challenge. In security inspection scenarios, identifying targets with strong 3D shape features but complex material composition and physical properties is particularly difficult. Furthermore, security inspection scenarios demand high real-time performance and accuracy in 3D CT data target identification.

[0060] To address the above issues, the applicant has proposed a method for identifying flakes and blocks of contraband, explosives, or drugs (see the same applicant's published patent, authorized publication number CN103971084B, entitled "Method for Identifying Flakes and Blocks of Contraband, Explosives, or Drugs," which will be referred to as Patent Document 1 below). Patent Document 1 simplifies the complex problem of three-dimensional target extraction into a more technologically mature two-dimensional target extraction problem. It employs layer-by-layer tomographic analysis to directly measure the length, thickness, and other geometric properties of the two-dimensional regions of flakes and blocks of contraband, accurately and quickly locating the suspect object. Then, the tomographic region is gradually increased to grow into a three-dimensional object, and its physical properties are determined, ultimately obtaining the three-dimensional identification result of the target, thus achieving effective identification of flakes and blocks of contraband, explosives, or drugs.

[0061] Further research by the applicant revealed that the aforementioned method, utilizing a limited number of two-dimensional views for target identification, results in insufficient representation of three-dimensional features, potentially leading to the loss of some three-dimensional shape features and spatial texture information. Furthermore, due to the use of real-time tomographic data analysis, different tomographic data are independent during the analysis process; information from later tomographic data has no impact on the analysis results of earlier tomographic data, causing significant information loss. If, based on the method in Patent Document 1, the three-dimensional object grown after tomographic analysis could be combined with comprehensive, three-dimensional CT raw data for further identification and judgment of suspects, it would be possible to further distinguish between contraband and safe goods, improving the accuracy of identification.

[0062] To address this, embodiments of this disclosure propose a cascaded target recognition method for CT data, which effectively utilizes the feature representation of three-dimensional objects to further improve recognition accuracy. In the method provided by embodiments of this disclosure, the three-dimensional CT data is first initially identified to obtain a first object description set of the target. Then, a cascaded fine recognition method is used, taking the already obtained first object description set of the target as input to further identify the three-dimensional region of interest. Furthermore, if the pre-obtained three-dimensional recognition results do not contain the three-dimensional region of interest of the target, the method provided by embodiments of this disclosure can also handle this, using the entire or a portion of the three-dimensional CT data as input to the cascaded fine recognition method for identification.

[0063] For example, some exemplary embodiments of this disclosure provide a cascaded CT data target recognition method, the method comprising: acquiring three-dimensional CT data; performing initial recognition on the three-dimensional CT data to obtain a first object description set of the target; and performing fine recognition on the target using a cascaded fine recognition method based on the first object description set of the target to obtain a fine recognition result of the target, wherein, in the cascaded fine recognition method, at least one of a first deep learning network for three-dimensional CT data and a second deep learning network for two-dimensional data is used to perform fine recognition on the target to obtain a fine recognition result of the target.

[0064] In this embodiment, after obtaining the first object description set of the target using the initial identification method, the cascaded fine identification method performs further identification, effectively utilizing the three-dimensional information of the CT data, which can further distinguish between contraband and safe items, and improve the accuracy of target identification.

[0065] Figure 4 This is a schematic flowchart of a cascaded CT data target recognition method according to embodiments of the present disclosure. Exemplarily, refer to... Figure 4 The cascaded CT data target recognition method may include the following operations or steps S410~S430.

[0066] Reference Figure 4 In operation or step S410, three-dimensional CT data is acquired.

[0067] Combined with reference Figures 1 to 3 After a portion of the object 70 is detected in the X-ray transmission field 45 emitted from the X-ray source 10, it is received by the detection and acquisition device 30, converted into an electrical signal, and then into a digital signal representing the attenuation value, as projection data; then, the mainstream analytical reconstruction method in the relevant technical field can be used to obtain a three-dimensional reconstructed image.

[0068] In embodiments of this disclosure, the three-dimensional CT data may include the three-dimensional reconstructed image.

[0069] Reference Figure 4 In operation or step S420, the three-dimensional CT data is initially identified to obtain a first object description set of the target, wherein the number of targets can be N, where N is a positive integer greater than or equal to 1.

[0070] Figure 5 This is a schematic flowchart of the initial identification operation in a cascaded CT data target recognition method according to an embodiment of the present disclosure. Exemplarily, refer to... Figure 5 The initial identification operation may include the following sub-operations or sub-steps S510~S540.

[0071] ReferenceFigure 5 In sub-operation or sub-step S510, a thin-slice region extraction step is performed. Specifically, data of a fracture of the detection object is acquired; the fracture data is preprocessed; the preprocessed fracture data is divided into several regions with similar physical properties; in response to a certain segmented region being a thin-slice region, the thin-slice region is extracted.

[0072] Optionally, in sub-operation or sub-step S510, a block region extraction step is performed. Specifically, data from a fault of the detection object is acquired; the fault data is preprocessed; the preprocessed fault data is divided into several regions with similar physical properties; in response to a segmented region being a block region, the block region is extracted.

[0073] In sub-operation or sub-step S520, the thin-sheet object formation step is performed. Specifically, it is determined whether the thin-sheet region extracted from the current fault can be merged with the thin-sheet region extracted from the previous fault; in response to the fact that the thin-sheet region extracted from the current fault can be merged with the thin-sheet region extracted from the previous fault, the thin-sheet object is formed by merging.

[0074] Optionally, in sub-operation or sub-step S520, a block object formation step is performed. Specifically, it is determined whether the block region extracted from the current fault can be merged with the block region extracted from the previous fault; in response to the block region extracted from the current fault being able to be merged with the block region extracted from the previous fault, the block is merged to form a block object.

[0075] In sub-operation or sub-step S530, determine whether the current fault is the last fault.

[0076] In sub-operation or sub-step S540, in response to the current fault not being the last fault, the thin-section region extraction step and the thin-section object formation step are repeated, and the data of each fault is processed using a fault-by-fault analysis method.

[0077] Optionally, in sub-operation or sub-step S540, in response to the current fault not being the last fault, the block region extraction step and the block object formation step are repeated, and the data of each fault is processed using a fault-by-fault analysis method.

[0078] In other words, in this initial identification operation, the data of each fault layer is extracted and processed layer by layer until the data of all fault layers are processed to obtain the first object description set of the target.

[0079] It should be noted that the specific implementation of the initial identification operation (i.e., operation S420 above) or step in the target identification method of cascaded CT data according to the embodiments of this disclosure can refer to the method described in Patent Document 1. That is, without conflict, the method described in Patent Document 1 is incorporated herein in whole or in part as the initial identification method in the embodiments of this disclosure.

[0080] It should also be noted that the initial identification operation (i.e., operation S420 above) or steps in the target identification method for cascaded CT data according to the embodiments of this disclosure are not limited to the above-described implementation.

[0081] For example, the above operation S420 may include: processing the three-dimensional CT data using a deep learning-based three-dimensional segmentation network to obtain a first set of object descriptions of the target.

[0082] For example, the above operation S420 may include: processing the three-dimensional CT data using a three-dimensional target detection network to obtain a first set of object descriptions of the target.

[0083] For example, the above operation S420 may include: processing the three-dimensional CT data by comparing it with a target feature library to obtain a first object description set of the target.

[0084] In embodiments of this disclosure, after the initial identification operation, the first object description set of the target obtained may include one or more of three-dimensional voxel data, point cloud data, feature vectors, and semantic descriptions. The semantic description includes at least one of the target's category information, location information, and confidence level.

[0085] Reference Figure 4 In operation or step S430, the target is finely identified using a cascaded fine identification method based on the first object description set of the target, so as to obtain the fine identification result of the target.

[0086] In embodiments of this disclosure, in the cascaded fine recognition method, at least one of a first deep learning network for three-dimensional CT data and a second deep learning network for two-dimensional data may be selectively used to perform fine recognition of the target in order to obtain a fine recognition result of the target.

[0087] In embodiments of this disclosure, a first deep learning network and / or a second deep learning network can be selectively configured to perform fine recognition based on the specific application scenario of the method (e.g., a specific security check scenario) and at least one factor in the first object description set of the target.

[0088] For example, in some embodiments of this disclosure, based on the characteristics of the first object description set of the target, in the cascaded fine recognition method, a first deep learning network for three-dimensional CT data may be selectively used to perform fine recognition of the target to obtain fine recognition results of the target.

[0089] For example, in some embodiments of this disclosure, based on the characteristics of the first object description set of the target, in the cascaded fine recognition method, a second deep learning network for two-dimensional data may be selectively used to perform fine recognition of the target in order to obtain fine recognition results of the target.

[0090] For example, in some embodiments of this disclosure, based on the characteristics of the first object description set of the target, in the cascaded fine recognition method, both a first deep learning network for three-dimensional CT data and a second deep learning network for two-dimensional data may be selectively used to perform fine recognition of the target in order to obtain fine recognition results of the target.

[0091] For example, the method further includes: generating a second object description set based on a first object description set of the target, wherein the first object description set is a first object description set for three-dimensional data, and the second object description set is a first object description set for two-dimensional data.

[0092] In some exemplary embodiments, the second object description set includes one or more of the following: slice data of the three-dimensional region of interest of the target along any direction, feature vectors corresponding to the target in the slice data, and semantic descriptions.

[0093] For example, the three-dimensional region of interest may include a three-dimensional suspected region. Correspondingly, the second object description set may include all or part of the slice data of the three-dimensional suspected region along any direction, along with its feature vectors, semantic descriptions, and other description sets. Specifically, the three-dimensional data can be sliced ​​along any direction at any resolution to obtain a complete slice of the three-dimensional data in that direction. For ease of description, this complete slice can be denoted as the first data form. Then, region extraction is performed on the above slices. Methods that can be used include, but are not limited to, Region Proposal Networks (RPNs), sliding windows, etc., to obtain all or part of the slice data set. This all or part of the slice data set can be denoted as the second data form. The final presented slice data form can be either the first data form or the second data form alone, or a combination of the first and second data forms.

[0094] In some exemplary embodiments, the second object description set includes one or more of the following: a two-dimensional projected image obtained by projecting a three-dimensional region of interest of the target, a feature vector corresponding to the target in the two-dimensional projected image, and a semantic description.

[0095] For example, a three-dimensional suspected area can be projected to obtain one or more two-dimensional images, and the feature vectors and semantic descriptions corresponding to the suspected object can be projected onto the two-dimensional images in a corresponding manner to obtain a two-dimensional projected image.

[0096] In some exemplary embodiments, the second object description set includes one or more of the following: a two-dimensional projection image obtained by projecting the entire three-dimensional CT data, a feature vector corresponding to the target in the two-dimensional projection image, and a semantic description.

[0097] For example, one or more two-dimensional images can be obtained by projecting the overall three-dimensional data, and the three-dimensional data, feature vectors, and semantic descriptions corresponding to the suspect object can be mapped onto the two-dimensional images. The two-dimensional image data and the mapped suspect object description information can be used as a second object description set, which can then be used as input for precise identification.

[0098] In some exemplary embodiments, the 3D CT data includes 3D images acquired by a first CT detection device. The second object description set includes one or more of the following: 2D images acquired by the first CT detection device or other detection devices different from the first CT detection device, feature vectors corresponding to targets in the 2D images, and semantic descriptions. For example, the other detection devices may include a DR detection device.

[0099] It should be noted that, in the embodiments of this disclosure, in the cascaded precision identification method, the input may include one or more of the following: (1) the three-dimensional suspected area data corresponding to the suspected object and its feature vector, semantic description and other description set, where the three-dimensional suspected area data corresponding to the suspected object refers to the volume data or point cloud data corresponding to the local area in the original three-dimensional CT data; (2) background area information in the three-dimensional CT image data; (3) other three-dimensional data corresponding to the original three-dimensional image obtained through the original device hardware, and the three-dimensional data, feature vector and semantic description corresponding to the suspected object are mapped to the new three-dimensional image in a predetermined manner, thereby obtaining the three-dimensional image; (4) the three-dimensional of the target. (5) A two-dimensional projection image obtained by projecting the three-dimensional region of interest of the target in any direction, and one or more of the feature vector and semantic description corresponding to the target in the two-dimensional projection image; (6) A two-dimensional projection image obtained by projecting the entire three-dimensional CT data, and one or more of the feature vector and semantic description corresponding to the target in the two-dimensional projection image; (7) A two-dimensional image obtained by the first CT detection device or other detection device different from the first CT detection device, and one or more of the feature vector and semantic description corresponding to the target in the two-dimensional image.

[0100] It should be noted that the above input (3) can be understood as follows: the three-dimensional CT data includes a first three-dimensional image acquired by the first CT detection device; the input of the first deep learning network also includes second three-dimensional data acquired by the first CT detection device or other detection devices different from the first CT detection device, wherein the second three-dimensional data includes a second three-dimensional image, the second three-dimensional image corresponds to the first three-dimensional image, and the information in the first object description set of the target is mapped to the second three-dimensional image. The "other detection devices" here may include the second CT detection device, or other types of detection devices capable of acquiring three-dimensional images.

[0101] In some exemplary embodiments, in the cascaded fine recognition method, a first deep learning network for three-dimensional CT data and a second deep learning network for two-dimensional data are used to perform fine recognition of the target in order to obtain a fine recognition result of the target.

[0102] For example, all or at least a portion of the first object description set is used as input to the first deep learning network, and all or at least a portion of the second object description set is used as input to the second deep learning network. For instance, the input to the first deep learning network may include at least one of the inputs (1), (2), and (3) above, and the input to the second deep learning network may include at least one of the inputs (2), (4), (5), (6), and (7) above.

[0103] Figure 6 This is a schematic flowchart illustrating the fine recognition operation in a cascaded CT data target recognition method according to embodiments of the present disclosure. Exemplarily, refer to... Figure 6 The precise identification operation may include the following sub-operations or sub-steps S610~S640. It should be noted that, in Figure 6 In the process, some of the sub-operations or sub-steps S610 to S640 are optional operations or steps.

[0104] In some exemplary embodiments of this disclosure, in sub-operation or sub-step S620, feature extraction is performed using a pre-trained deep learning network for 3D CT data (hereinafter referred to as a first deep learning network) and / or a pre-trained deep learning network for 2D data (hereinafter referred to as a second deep learning network) to obtain feature maps. That is, the cascaded 3D target recognition method may include a 3D target recognition method based on a deep learning network and / or a 2D target recognition method based on a deep learning network.

[0105] In the embodiments of this disclosure, the cascaded three-dimensional target recognition method can identify the three-dimensional region of interest, supports input of various three-dimensional data such as single-channel and multi-channel, and is compatible with various information including original three-dimensional CT data, the three-dimensional image semantic description set, etc.

[0106] Optionally, in sub-operation or sub-step S610, before performing feature extraction, the initial identification result (e.g., the initial identification result includes a three-dimensional region of interest) can be preprocessed. The preprocessing method includes, but is not limited to, denoising, morphological transformation, geometric transformation, and image type conversion, supporting the conversion of three-dimensional volume data into point cloud data. For example, the preprocessing may include format conversion and size adjustment. Furthermore, rotation correction and denoising may be performed on the three-dimensional region of interest to ensure image quality.

[0107] Rotation correction can help correct image tilt. For example, a rotation correction method based on image edge contours can be used to correct the rotation of at least one of the first, second, and third images. Specifically, the rotation correction method based on image edge contours may include the following steps: image grayscale conversion: converting a color image to a grayscale image for subsequent processing; threshold binarization: converting a grayscale image to a binary image, making the target and background in the image clearly distinguishable; contour detection: detecting the edge contour of the target in the binary image; extracting the contour enclosing matrix: finding the smallest rectangular region enclosing the target contour, the edges of which can be used to estimate the tilt angle of the image; obtaining the deflection angle through the extracted enclosing matrix: calculating the angle between the long side of the enclosing matrix and the horizontal line, this angle is the tilt angle of the image; and deflecting the image using an affine transformation: rotating the image to a normal angle using an affine transformation based on the calculated tilt angle.

[0108] For example, rotation correction of at least one of the first image, the second image, and the third image can be performed using a rotation correction method based on Fourier transform and Hough line detection. Specifically, the rotation correction method based on Hough line detection may include the following steps: image preprocessing and edge detection: First, the image is preprocessed, such as grayscale and binarization, and then an edge detection algorithm (e.g., the Sobel operator) is used to detect edges in the image; Hough transform: The Hough transform is a technique for detecting lines and curves in an image. Here, it is used to detect lines in the image. The basic principle of the Hough transform is to utilize the duality of points and lines to transform lines in image space into points of convergence in parameter space, thereby detecting whether a curve with a given property exists in a given image; calculating the tilt angle: Through the Hough transform, the polar coordinate representation (ρ, θ) of each line in the image can be obtained. Where θ represents the tilt angle of the line. The tilt angle of the entire image can be obtained by statistically analyzing the tilt angles of all detected lines and then averaging them; Image rotation: Based on the calculated tilt angle, the image can be rotated using an image rotation algorithm (such as affine transformation) to correct it; Output the corrected image.

[0109] In some exemplary embodiments, the 3D region of interest can be denoised using filter-based, model-based, or learning-based methods. Denoising is an operation to reduce noise interference in an image or data. In 3D region of interest preprocessing, the goal of denoising is to eliminate or reduce irrelevant information introduced due to the acquisition process, sensor noise, or other interference factors. For example, denoising methods may include median filtering, Gaussian filtering, mean filtering, etc., which can smooth data and remove noise to improve the accuracy and stability of subsequent processing.

[0110] In some exemplary embodiments, morphological transformations can be used to preprocess the three-dimensional region of interest. Morphological transformations are a class of operations based on image shape and structure, used to alter the geometry of an image or extract shape features of interest. In three-dimensional region of interest preprocessing, morphological transformations can be used to remove small spots or isolated noise, fill holes, connect adjacent regions, etc. For example, morphological transformations include dilation, erosion, opening, and closing operations.

[0111] In some exemplary embodiments, geometric transformations can be used to preprocess the three-dimensional region of interest. Geometric transformations refer to operations performed on the position, scale, rotation, or affine transformations of an object in three-dimensional space to better match or adjust an image or data. In three-dimensional region of interest preprocessing, geometric transformations can be used to correct or align data to conform to a specific reference coordinate system or geometry. For example, translation, rotation, scaling, mirroring, and other transformation operations can be performed.

[0112] In some exemplary embodiments, image type conversion can be used to preprocess the 3D region of interest. For example, image type conversion can be an operation that converts raw 3D volume data into point cloud data. In 3D region of interest preprocessing, this conversion can help transform the volume data representation into a series of discrete 3D point coordinates for subsequent point cloud processing and analysis. For example, image type conversion methods may include voxelization, surface reconstruction, and sampling.

[0113] In the cascaded precision identification method, when the input includes "background region information in 3D CT image data", traditional image processing or statistical analysis methods can be used to preprocess the background region information in the 3D CT image data to obtain relevant analysis results of the background region. For example, traditional image processing methods include, but are not limited to, edge detection, texture analysis, feature descriptors, and statistical analysis methods. The obtained background region analysis results include feature descriptors, value distribution, and statistical information. The above analysis results can be directly used as part of the precision identification input, or as a screening or enhancement condition for the precision identification input. For example, (1) the feature descriptors are used as part of the precision identification input and input into precision identification together with other suspect object descriptions; (2) the value distribution and statistical information are used as screening conditions for precision identification input, and some suspect objects are screened as input for precision identification based on the above information and preset conditions.

[0114] The preprocessing method described above can improve data quality, reduce noise interference, adjust geometric structure, or convert 3D volume data into point cloud data, providing more accurate and reliable input for subsequent 3D recognition and analysis.

[0115] It's important to note that 3D volumetric data refers to data represented in 3D space using voxels as the basic unit. A voxel can be understood as a pixel in 3D space, and each voxel contains attribute or numerical information at a specific location in space. 3D volumetric data can be constructed by dividing a 3D object or scene into a series of voxels and assigning attribute values ​​to each voxel. 3D volumetric data can be represented in the form of a cube or cuboid, providing detailed information about the object's internal and external structure.

[0116] Point cloud data is a dataset consisting of the coordinates of a set of discrete 3D points. Each point contains its position information in 3D space and often includes other attributes such as color, normal vector, or intensity. Point cloud data can be acquired in various ways, such as through LiDAR scanning, stereo vision, or 3D reconstruction techniques. Point cloud data can represent the surface geometry of a 3D object or the scattered distribution of points in a scene, providing a description of the object's shape, surface details, and spatial location. Point cloud data has wide applications in computer vision, computer graphics, robotics, and other fields, such as object detection and recognition, 3D reconstruction, and environmental perception.

[0117] In summary, 3D volume data is 3D data with volume information represented by voxels, while point cloud data is a dataset composed of discrete 3D point coordinates, used to represent the surface geometry of an object or the scattered distribution of points in a scene.

[0118] It should be noted that there are some differences between 3D volume data and point cloud data in terms of representation and data structure: (1) Representation: 3D volume data uses voxels as the basic unit, dividing the 3D space into regular cubes or cuboids. Each voxel contains attribute or numerical information at a specific location in the space. Point cloud data is a dataset composed of a set of discrete 3D point coordinates. Each point contains position information in the 3D space. (2) Data structure: 3D volume data is usually stored in the form of a voxel grid and is represented using data structures such as multidimensional arrays or voxel trees. The attribute value of each voxel can represent information such as the density, color, and material of the object. Point cloud data is stored in the form of a set of points. Each point can contain additional attributes, such as color, normal vector, or intensity. (3) Data density: Since 3D volume data is represented by a regular voxel grid, it can provide detailed information about the internal and external structure of the object, including density distribution, shape, etc. Point cloud data is a discrete set of point coordinates, which usually only provides geometric information of the object's surface. The density and distribution of points depend on the method and parameters for collecting or generating the point cloud. (4) Data scale: Since point cloud data only stores discrete point coordinates, its storage space is more compact than that of 3D volume data. Therefore, point cloud data may be more suitable for storage and processing for large-scale 3D scenes or complex objects.

[0119] Reference Figure 6 In sub-operation or sub-step S620, a first deep learning network for three-dimensional CT data is used, specifically, a specially designed feature extraction network is used to extract features from the preprocessed three-dimensional data; and / or, a second deep learning network for two-dimensional data is used, specifically, a specially designed feature extraction network is used to extract features from the preprocessed two-dimensional data.

[0120] In the embodiments of this disclosure, the extracted features include, but are not limited to, the shape, texture, and color of the three-dimensional data. The shape of the three-dimensional data corresponds to the macrostructure of the target, the texture corresponds to the microstructure or detailed structure of the target, and the color corresponds to the physical properties of the target such as Z-value and density.

[0121] In embodiments of this disclosure, useful feature representations are extracted from input data using neural networks or other methods. These features may include information such as the shape, texture, and color of an object, used to describe and express the key characteristics of the input data. In other words, embodiments of this disclosure can comprehensively utilize various information from 3D data, which is beneficial for further improving recognition accuracy. In deep learning networks for 3D CT data, by using 3D convolutional neural networks (3D CNNs) or attention-based feature extraction methods, local and global features in the input data can be learned, extracting feature maps with discriminative and expressive capabilities.

[0122] In some exemplary embodiments, in the sub-operation or sub-step S620, feature extraction may use a three-dimensional convolutional neural network or an attention-based feature extraction method, which may be used alone or in combination to extract feature maps of three-dimensional CT data; and / or, feature extraction may use a two-dimensional convolutional neural network or an attention-based feature extraction method, which may be used alone or in combination to extract feature maps of two-dimensional data.

[0123] If the target's 3D image semantic description set contains 3D mask, voxel category information, and confidence information, then it also supports accurate generation of feature vectors. That is, in the sub-operation or sub-step S620, the 3D mask, category information, and confidence information contained in the first 3D image semantic description set can be used to generate feature vectors.

[0124] If the target's image semantic description set contains a two-dimensional mask, pixel category information, and confidence information, then it also supports the accurate generation of feature vectors. That is, in the sub-operation or sub-step S620, the two-dimensional mask, category information, and confidence information contained in the first image semantic description set can be used to generate feature vectors.

[0125] In the training process of feature extraction networks, various data augmentation techniques can be applied to increase data diversity, including but not limited to rotation, translation, scaling, flipping, and data fusion. Data augmentation helps to improve the robustness and generalization of the network.

[0126] Optionally, in the sub-operation or sub-step S630, operations can be performed on the extracted feature maps, including but not limited to feature map fusion and feature map dimensionality reduction. These operations can be selected and combined according to the specific task and network architecture. The aim is to improve the model's performance and generalization ability by appropriately manipulating and processing the feature maps, thereby better adapting to various complex data and application scenarios.

[0127] Specifically, if the network has feature maps with multiple branches or multiple levels, feature map fusion operations can be performed to obtain more comprehensive and multi-scale feature representations. Fusion operations can use methods such as weighted summation of feature maps, feature map concatenation, feature pyramids, and feature fusion based on Transformer encoding and decoding.

[0128] For example, in a feature map weighted summation fusion method, feature maps from different levels or sources are weighted and summed according to certain weights. Each feature map is multiplied by its corresponding weight, and they are then added together to obtain the fused feature map. The weights can be learned or set manually.

[0129] For example, in feature map concatenation fusion, feature maps from different levels or sources are concatenated along the channel dimension. For instance, if two feature maps have C1 and C2 channels respectively, the concatenated feature map will have C1 + C2 channels. This method increases the channel dimension of the feature maps, allowing the network to utilize information from multiple levels or sources simultaneously.

[0130] For example, in the feature pyramid fusion method, the feature pyramid is achieved by extracting and fusing feature maps at different scales. Information at different scales is captured by using convolutional kernels of different sizes or feature maps of different levels. These feature maps are then fused together to obtain a multi-scale feature representation.

[0131] For example, in feature fusion based on Transformer encoding and decoding, Transformer is a neural network model based on a self-attention mechanism that can be used to encode and decode sequential data. In feature fusion, the encoder-decoder structure of Transformer can be used to fuse feature maps. The encoder encodes the input feature map into a context-aware feature representation, and the decoder decodes these feature representations into the final fused feature map.

[0132] It should be noted that these feature fusion methods can be selected and applied according to the specific task and network structure. Their goal is to improve the expressive power and performance of the network by effectively combining feature information from different sources.

[0133] Specifically, in order to improve computational efficiency or reduce feature dimensions, feature maps can be subjected to feature map dimensionality reduction operations to reduce resolution or spatial dimensions. For example, the operation methods may include, but are not limited to, global average pooling, downsampling, convolutional dimensionality reduction (such as 1×1 convolution, depthwise separable convolution, etc.).

[0134] In the embodiments of this disclosure, the first deep learning network for 3D CT data may include, but is not limited to, 3D convolutional neural networks, such as 3D U-Net, V-Net, 3D DenseNet, 3D ResNet, etc. It should be noted that 3D U-Net is a 3D convolutional neural network based on the U-Net architecture. It has an encoder-decoder structure and extracts and recovers features through stacked convolution, pooling, and upsampling operations, effectively capturing contextual information at different scales. V-Net is a volumetric segmentation network based on 3D CNN, specifically designed for medical image segmentation tasks. It introduces techniques such as residual connections and batch normalization, learning image feature representations through multiple 3D convolutional and deconvolutional layers and generating pixel-level segmentation results. 3D DenseNet is a network structure that extends 2D DenseNet to three dimensions. It connects the feature maps of each layer with the feature maps of all previous layers through dense connections, promoting feature transfer and information flow, thereby improving model performance and stability. 3D ResNet is a 3D convolutional neural network that extends 2D ResNet. It utilizes the ideas of residual connections and deep networks, and constructs the network by stacking multiple residual blocks, which can effectively solve the problems of gradient vanishing and model degradation, and improve the representation and learning capabilities of features.

[0135] Figure 7 This is an architecture diagram of a cascaded deep learning network used in a target recognition method according to an exemplary embodiment of the present disclosure.

[0136] Reference Figure 7The cascaded deep learning network is a three-dimensional convolutional neural network based on the U-Net architecture. Its structure consists of an encoder and a decoder, extracting features through stacked convolutions, pooling, and upsampling operations. The cascaded deep learning network includes an encoder part and a decoder part. The encoder part is a typical convolutional network, involving repeated applications of convolutions, each followed by a ReLU activation function and a max-pooling operation. For example, in this network, each layer in the encoder part may include two 3×3×3 convolutions, each followed by a ReLU layer, and then a 2×2×2 max-pooling layer with a stride of 2 in each direction. The decoder part mainly includes upsampling and convolution operations. Upsampling is used to restore the image resolution, while convolution is used to combine this information into a precise output. In the decoder part, each layer contains a 2×2×2 deconvolutional layer with a stride of 2, followed by two 3×3×3 convolutional layers, each followed by a ReLU layer. The cascaded deep learning network also includes skip connections to connect corresponding layer feature maps from the encoder and decoder parts, providing richer contextual information. At the network's output layer, 1×1×1 convolutional layers and appropriate activation functions (e.g., sigmoid or softmax) are used to generate the final result.

[0137] The cascaded deep learning network is characterized by a symmetrical structure between the encoder and decoder. Skip connections allow for the fusion of low-level and high-level features, thereby improving segmentation accuracy and detail preservation. Furthermore, the network's depth and width can be adjusted to adapt to different tasks and datasets, enhancing performance and generalization capabilities.

[0138] In some exemplary embodiments of this disclosure, in sub-operation or sub-step S640, the feature map is applied to a classification, segmentation, or detection task to obtain a set of three-dimensional image semantic descriptions. Here, "feature map" can be a feature map directly extracted by sub-operation or sub-step S620, or a feature map after extraction by sub-operation or sub-step S620 and processing by S630 of sub-operation or sub-step S620.

[0139] Specifically, feature maps can be mapped to class probability distributions using multilayer perceptrons or other classifiers to achieve classification tasks. Additionally or alternatively, deconvolution, bilinear interpolation, or other methods can be used to restore the feature maps to their original image size and assign semantic labels to each voxel, thereby achieving voxel-level segmentation. Additionally or alternatively, object detection heads can be used to perform classification and regression operations on the feature maps, predicting the class and location of objects and generating class information, confidence scores, and location information to achieve object detection tasks.

[0140] In embodiments of this disclosure, the three-dimensional image semantic description set, based on one or more of voxels, three-dimensional regions of interest, and three-dimensional CT images, includes: category information and / or confidence level; or, based on three-dimensional regions of interest and / or three-dimensional CT images, the three-dimensional image semantic description set includes at least one of: category information, confidence level, and target location information. For example, the target location information in a three-dimensional CT image may include a three-dimensional bounding box.

[0141] Research has shown that the semantic description set of the 3D image obtained after refined recognition is more accurate than that obtained after initial recognition, representing a correction and improvement to the semantic description set obtained in the previous stage. In the embodiments of this disclosure, the refined recognition method utilizes a two-dimensional deep learning network, which can improve the efficiency of refined recognition without significantly reducing recognition accuracy.

[0142] In the embodiments of this disclosure, the target recognition method can directly obtain the final recognition conclusion by using the above-described fine recognition method in a single execution. Specifically, it involves executing at least one of the first deep learning network and the second deep learning network in a single execution step to process the input; and using the output of at least one of the first deep learning network and the second deep learning network after a single execution as the fine recognition result of the target.

[0143] Figure 8 This is a schematic flowchart of a cascaded CT data target recognition method according to other embodiments of the present disclosure.

[0144] In embodiments of this disclosure, reference is made to Figure 8 The target recognition method can iteratively apply the aforementioned fine recognition method until a final recognition conclusion is obtained. Specifically, it iteratively executes at least one of the first deep learning network and the second deep learning network to process the input; and uses the output of at least one of the first deep learning network and the second deep learning network after multiple iterations as the fine recognition result of the target.

[0145] During the iterative use of the above-described fine recognition method, the output of the i-th round of fine recognition (i is a positive integer greater than or equal to 1) can be used as the input of the (i+1)-th round of fine recognition. Alternatively, the output of the i-th round of fine recognition and the result of the initial recognition in the above operation or step S420 can be used together as the input of the (i+1)-th round of fine recognition. Or, the output of the i-th round of fine recognition can be used as a condition for filtering the input information of the (i+1)-th round of fine recognition.

[0146] It should be noted that, in the embodiments of this disclosure, the termination condition for iteratively using the above-described precision identification method may include: the number of iterations reaches a preset number of iterations, and the obtained precision identification output reaches a preset precision.

[0147] Based on the above method, embodiments of this disclosure also provide a cascaded CT data target recognition device. The following will be combined with... Figure 9 The device is described in detail.

[0148] Figure 9 This is a structural block diagram of a cascaded CT data target recognition device according to an exemplary embodiment of the present disclosure. (Refer to...) Figure 9 The device 900 includes an acquisition module 910, a preliminary identification module 920, and a fine identification module 930.

[0149] The acquisition module 910 is used to acquire three-dimensional CT data. In some exemplary embodiments, the acquisition module 910 can be used to perform the operation S410 and its sub-operations described above, which will not be repeated here.

[0150] The initial identification module 920 is used to perform initial identification on the three-dimensional CT data to obtain a first object description set of the target. In some exemplary embodiments, the initial identification module 920 can be used to perform the operation S420 and its sub-operations described above, which will not be repeated here.

[0151] The fine recognition module 930 is used to perform fine recognition of the target based on the first object description set of the target using a cascaded fine recognition method to obtain a fine recognition result of the target. In some exemplary embodiments, the fine recognition module 930 can be used to perform the operation S430 and its sub-operations described above, which will not be repeated here.

[0152] In the fine recognition method executed by the fine recognition module 930, at least one of a first deep learning network for three-dimensional CT data and a second deep learning network for two-dimensional data is used to perform fine recognition of the target in order to obtain a fine recognition result of the target.

[0153] In some embodiments, the precision recognition module 930 is configured to: perform feature extraction using a pre-trained first deep learning network for 3D CT data and / or a second deep learning network for 2D data to obtain a feature map. Specifically, feature vectors can be generated using the 3D mask, category information, and confidence level contained in the first 3D image semantic description set.

[0154] For example, the extracted features include at least one of shape, texture, and color from the 3D CT data.

[0155] Optionally, the precision recognition module 930 is further configured to: operate on the feature map using the first deep learning network and / or the second deep learning network, wherein the operation includes at least one of feature map fusion and feature map dimensionality reduction.

[0156] For example, the precision recognition module 930 is further configured to: apply the feature map to a classification, segmentation, or detection task to obtain a set of three-dimensional image semantic descriptions.

[0157] In embodiments of this disclosure, the three-dimensional image semantic description set is based on one or more of voxels, three-dimensional regions of interest, and three-dimensional CT images, and includes: category information and / or confidence level; or, the three-dimensional image semantic description set is based on three-dimensional regions of interest and / or three-dimensional CT images, and includes at least one of: category information, confidence level, and target location information.

[0158] According to embodiments of this disclosure, any plurality of modules, including the acquisition module 910, the initial identification module 920, and the fine identification module 930 and their sub-modules, of the device 900 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least some of the functions of one or more of these modules can be combined with at least some of the functions of other modules and implemented in one module. According to embodiments of this disclosure, at least one of the acquisition module 910, the initial identification module 920, and the fine identification module 930 and their sub-modules of the device 900 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or implemented in hardware or firmware by any other reasonable means of integrating or packaging the circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, at least one of the acquisition module 910, the initial identification module 920, and the fine identification module 930 and their sub-modules included in the device 900 can be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0159] Figure 10 The diagram schematically illustrates a structural block diagram of an electronic device suitable for implementing the target recognition method according to exemplary embodiments of the present disclosure.

[0160] like Figure 10As shown, an electronic device 1000 according to an embodiment of the present disclosure includes a processor 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage portion 1008 into a random access memory (RAM) 1003. The processor 1001 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 1001 may also include onboard memory for caching purposes. The processor 1001 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0161] For example, the electronic device may be the data processing device 60 of the X-ray scanning detection system.

[0162] RAM 1003 stores various programs and data required for the operation of electronic device 1000. Processor 1001, ROM 1002, and RAM 1003 are interconnected via bus 1004. Processor 1001 performs various operations of the method flow according to embodiments of the present disclosure by executing programs in ROM 1002 and / or RAM 1003. It should be noted that the programs may also be stored in one or more memories other than ROM 1002 and RAM 1003. Processor 1001 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in said one or more memories.

[0163] According to embodiments of this disclosure, the electronic device 1000 may further include an input / output (I / O) interface 1005, which is also connected to a bus 1004. The electronic device 1000 may also include one or more of the following components connected to the I / O interface 1005: an input section 1006 including a keyboard, mouse, etc.; an output section 1007 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card, modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 1010 as needed so that computer programs read from it can be installed into the storage section 1008 as needed.

[0164] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0165] According to embodiments of this disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this disclosure, the computer-readable storage medium may include ROM 1002 and / or RAM 1003 and / or one or more memories other than ROM 1002 and RAM 1003 described above.

[0166] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the methods provided in the embodiments of this disclosure.

[0167] When the computer program is executed by the processor 1001, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0168] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 1009, and / or installed from a removable medium 1011. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0169] In such an embodiment, the computer program can be downloaded and installed from a network via communication section 1009, and / or installed from removable medium 1011. When the computer program is executed by processor 1001, it performs the functions defined in the system of this disclosure embodiment. According to embodiments of this disclosure, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0170] According to embodiments of this disclosure, program code for executing the computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can execute entirely on a user's computing device, partially on a user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0171] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0172] The target identification method and apparatus, electronic device, X-ray scanning detection system, computer-readable storage medium, and program product provided in this disclosure have at least one of the following beneficial effects:

[0173] After acquiring the 3D CT data, an initial identification is performed, followed by a cascaded 3D target recognition method. This method supports using the results obtained from the initial identification as input for the cascaded fine identification method. By cascading the initial and fine identification 3D target recognition methods, the feature representation of 3D objects can be enhanced, further improving recognition accuracy.

[0174] In particular, in the field of security CT inspection, 3D CT data provides more comprehensive, accurate, and visualized information. Effectively utilizing 3D CT data to identify contraband improves the accuracy of identification algorithms, thereby enabling more precise identification of potential threats and concealed items. In the method provided in this disclosure, after obtaining a first set of descriptions of suspected objects using an initial identification method, a cascaded fine identification method is used. This effectively utilizes the 3D information of the CT data to further distinguish between contraband and safe items, improving identification accuracy.

[0175] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of this disclosure is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A method for identifying a target in CT data in a cascade, characterized by The method comprises: acquiring three-dimensional CT data; performing preliminary identification on the three-dimensional CT data to obtain a first object description set of a target; and performing fine identification on the target according to the first object description set of the target by using a cascaded fine identification method to obtain a fine identification result of the target, wherein in the cascaded fine identification method, at least one of a first deep learning network for three-dimensional CT data and a second deep learning network for two-dimensional data is selectively used to perform fine identification on the target to obtain the fine identification result of the target.

2. The method of claim 1, wherein, The first object description set comprises one or more of three-dimensional voxel data, point cloud data, feature vectors and semantic descriptions.

3. The method of claim 2, wherein, The semantic description comprises at least one of category information, position information and confidence of the target.

4. The method of any one of claims 1-3, wherein, The preliminary identification on the three-dimensional CT data to obtain the first object description set of the target specifically comprises: processing the three-dimensional CT data by using a three-dimensional segmentation network based on deep learning to obtain the first object description set of the target; or processing the three-dimensional CT data by using a three-dimensional target detection network to obtain the first object description set of the target; or processing the three-dimensional CT data by using a target feature library comparison method to obtain the first object description set of the target.

5. The method of any one of claims 1-3, wherein, The preliminary identification on the three-dimensional CT data to obtain the first object description set of the target specifically comprises: a slice region extraction step of acquiring data of a slice of an object to be detected; pre-processing the slice data; segmenting the pre-processed slice data into a plurality of regions with similar physical properties; and extracting a slice region in response to the slice region being a slice region; a slice object formation step of judging whether the slice region extracted in the current slice can be merged with the slice region extracted in the previous slice; and forming a slice object in response to the slice region extracted in the current slice being able to be merged with the slice region extracted in the previous slice; and repeatedly performing the slice region extraction step and the slice object formation step to process data of each slice by using a slice-by-slice analysis method until data of all slices are processed to obtain the first object description set of the target, or The preliminary identification on the three-dimensional CT data to obtain the first object description set of the target specifically comprises: a block region extraction step of acquiring data of a slice of a detection object; pre-processing the slice data; segmenting the pre-processed slice data into a plurality of regions with similar physical properties; and extracting a block region in response to the block region being a block region; a block object formation step of judging whether the block region extracted in the current slice can be merged with the block region extracted in the previous slice; and forming a block object in response to the block region extracted in the current slice being able to be merged with the block region extracted in the previous slice; and The block region extraction step and the block object formation step are repeatedly performed, and data of each slice is processed by a slice-by-slice analysis method until data of all slices are processed, so as to obtain a target object description set of the block target.

6. The method of any one of claims 1-3, wherein, In the cascade fine identification method, the target is finely identified by using a first deep learning network for three-dimensional CT data and a second deep learning network for two-dimensional data, so as to obtain a fine identification result of the target.

7. The method of claim 6, wherein, The method further comprises: generating a second object description set according to the first object description set of the target, wherein the first object description set is a first object description set for three-dimensional data, and the second object description set is a first object description set for two-dimensional data; The fine identification of the target by using the first deep learning network for three-dimensional CT data and the second deep learning network for two-dimensional data specifically comprises: The first object description set is taken as input of the first deep learning network for three-dimensional CT data, and the first object description set is processed by using the first deep learning network; and The second object description set is taken as input of the second deep learning network for two-dimensional data, and the second object description set is processed by using the second deep learning network.

8. The method of claim 7, wherein, Taking the first object description set as input of the first deep learning network for three-dimensional CT data comprises: One or more of three-dimensional voxel data, point cloud data, feature vectors and semantic descriptions corresponding to the target in the first object description set are taken as input of the first deep learning network.

9. The method of claim 7 or 8, wherein, The input of the first deep learning network further comprises: background region information in three-dimensional CT image data; and / or The three-dimensional CT data comprises first three-dimensional image data acquired by a first CT detection device; and the input of the first deep learning network further comprises second three-dimensional data acquired by the first CT detection device or other detection devices different from the first CT detection device, wherein the second three-dimensional data comprises second three-dimensional image data, the second three-dimensional image data corresponds to the first three-dimensional image data, and information in the first object description set of the target is mapped into the second three-dimensional image data.

10. The method of claim 7 or 8, wherein, The second object description set comprises one or more of slice data of a three-dimensional region of interest of the target along any direction, feature vectors and semantic descriptions corresponding to the target in the slice data; and / or The second object description set comprises one or more of two-dimensional projection image data obtained by projecting the three-dimensional region of interest of the target, feature vectors and semantic descriptions corresponding to the target in the two-dimensional projection image data; and / or The second object description set comprises one or more of two-dimensional projection image data obtained by projecting the three-dimensional CT data as a whole, feature vectors and semantic descriptions corresponding to the target in the two-dimensional projection image data; and / or The three-dimensional CT data comprises first three-dimensional image data acquired by a first CT detection device; and and the second object description set comprises one or more of a two-dimensional image acquired by the first CT detection device or other detection device different from the first CT detection device, a feature vector corresponding to the target in the two-dimensional image, and a semantic description.

11. The method of claim 7 or 8, wherein, The processing of the first object description set by the first deep learning network specifically comprises: feature extraction by using a pre-trained first deep learning network to obtain a feature map; and / or, The processing of the second object description set by the second deep learning network specifically comprises: feature extraction by using a pre-trained second deep learning network to obtain a feature map.

12. The method of claim 11, wherein, The extracted features comprise at least one of shape, texture and color in the three-dimensional CT data.

13. The method of claim 11, wherein, The processing of the first object description set by the first deep learning network specifically comprises: operation of the first deep learning network on the feature map, wherein the operation comprises at least one of feature map fusion and feature map dimension reduction; and / or, The processing of the second object description set by the second deep learning network specifically comprises: operation of the second deep learning network on the feature map, wherein the operation comprises at least one of feature map fusion and feature map dimension reduction.

14. The method of claim 7 or 8, wherein, The processing of the first object description set by the first deep learning network further specifically comprises: application of the feature map to a classification, segmentation or detection task to obtain a three-dimensional image semantic description set; and / or, The processing of the second object description set by the second deep learning network further specifically comprises: application of the feature map to a classification, segmentation or detection task to obtain a two-dimensional image semantic description set.

15. The method of any one of claims 1-3, 7, 8, 12, and 13, wherein, The fine recognition of the target by using the cascaded fine recognition method to obtain a fine recognition result of the target specifically comprises: Single execution of the step of processing the input by the first deep learning network; and output of the first deep learning network after single execution as the fine recognition result of the target; and / or, Single execution of the step of processing the input by the second deep learning network; and output of the second deep learning network after single execution as the fine recognition result of the target.

16. The method of any one of claims 1-3, 7, 8, 12, and 13, wherein, The fine recognition of the target by using the cascaded fine recognition method to obtain a fine recognition result of the target specifically comprises: iterative execution of the step of processing the input by the first deep learning network; and output of the first deep learning network after iterative execution for multiple times as the fine recognition result of the target; and / or, Iterative execution of the step of processing the input by the second deep learning network; and output of the second deep learning network after iterative execution for multiple times as the fine recognition result of the target.

17. The method of any one of claims 1-3, 7, 8, 12, and 13, wherein, The fine recognition result of the target is in units of one or more of voxels, three-dimensional regions of interest and three-dimensional CT images, and contains: class information and / or confidence; or, The fine recognition result of the target is in units of three-dimensional regions of interest and / or three-dimensional CT images, and contains: at least one of class information, confidence and position information of the target.

18. A device for identifying a target in CT data, characterized by The apparatus comprises: an acquisition module configured to acquire three-dimensional CT data; a preliminary identification module configured to perform preliminary identification on the three-dimensional CT data to obtain a first object description set of a target; and a fine identification module configured to perform fine identification on the target according to the first object description set of the target by using a cascaded fine identification method to obtain a fine identification result of the target, wherein in the cascaded fine identification method, at least one of a first deep learning network for three-dimensional CT data and a second deep learning network for two-dimensional data is used to perform fine identification on the target to obtain the fine identification result of the target. 19.An electronic device comprising: one or more processors; a memory configured to store one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1-17. 20.A radiation scanning detection system comprising: a radiation source configured to emit radiation; a detector configured to receive radiation emitted from the radiation source and passing through a detection object; and a data processing apparatus in communication connection with the detector, wherein the data processing apparatus comprises one or more processors and a memory configured to store one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1-17. The computer program is executed by a processor to implement the steps of the method according to any one of claims 1-17.

21. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by a processor to implement the steps of the method according to any one of claims 1-17.

22. A computer program product comprising a computer program, characterized in that, The computer program is executed by a processor to implement the steps of the method according to any one of claims 1-17.

Images