Scan image reconstruction method and scan imaging system for adapting incomplete data

Abstract

This application provides a scanning image reconstruction method and scanning imaging system adapted to incomplete data, relating to the fields of image processing, artificial intelligence, and radiation imaging technology. The method includes: in response to a ray scanning of an object to be imaged, acquiring projection data of the object, wherein the projection data is incomplete projection data in a projection space; transforming the incomplete projection data into first template data according to the mapping relationship between the projection space and a template data space; pre-completing the first template data to obtain second template data; acquiring confidence information of the second template data; inputting the second template data and confidence information as input to a pre-trained data processing model; processing the input using a first sub-model to obtain third template data; and performing image reconstruction based on the third template data to obtain a reconstructed image. This method can improve the efficiency and accuracy of data completion while ensuring the fidelity of the original sampled projection, and is beneficial for obtaining high-quality reconstructed images.

Description

Technical Field

[0001] This application relates to the fields of image processing technology, artificial intelligence technology, and radiation imaging technology, and more specifically, to a method for reconstructing scanned images and a scanning imaging system adapted to incomplete data. Background Technology

[0002] X-ray scanning imaging is widely used in medical, security inspection, and industrial non-destructive testing fields. Obtaining high-quality reconstruction usually relies on dense projection sampling within a sufficient angular range. However, due to constraints of actual scanning conditions and the influence of non-ideal factors, incomplete data scanning is becoming increasingly common. Typical cases include: (1) sparse angle scanning: reducing the angle sampling density while ensuring sufficient angular coverage can effectively reduce radiation dose; (2) limited angle scanning: in dynamic imaging or high-throughput inspection and in-situ scanning of large targets, the projection angle coverage is insufficient due to limitations of scanning time and physical space. In addition, in the architecture of certain specific scanning imaging systems, due to the irregular geometric arrangement of the X-ray source and detector, the above two situations may coexist, forming a more complex problem of incomplete data reconstruction. How to effectively reconstruct images to obtain high-quality reconstructed images for incomplete data has always been one of the important topics of concern for researchers in related fields.

[0003] It should be noted that the information disclosed in this section is only used to understand the background of the inventive concept of this application. Therefore, the above information may include information that does not constitute prior art. Summary of the Invention

[0004] In view of at least one aspect of the above-mentioned technical problems, embodiments of this application provide a scanning image reconstruction method adapted to incomplete data. The method includes: in response to a ray scanning of an object to be imaged, acquiring projection data of the object to be imaged, wherein the projection data is incomplete projection data in a projection space; transforming the incomplete projection data into first template data according to a mapping relationship between the projection space and a template data space, wherein the first template data includes m1 first data points in the template data space, where m1 is a positive integer greater than 1; pre-completing the first template data to obtain second template data, wherein the second... The template data includes m2 second data points in the template data space, where m2 is a positive integer greater than m1; the confidence information of the m2 second data points is obtained; the second template data and the confidence information are used as input to a pre-trained data processing model, wherein the data processing model includes a first sub-model, which is used to complete the second template data; the input is processed using the first sub-model to obtain third template data, wherein the third template data includes m3 third data points in the template data space, where m3 is a positive integer greater than 1; image reconstruction is performed based on the third template data to obtain a reconstructed image.

[0005] For example, the step of performing image reconstruction based on the third template data to obtain a reconstructed image includes: performing image reconstruction directly based on the third template data to obtain a reconstructed image.

[0006] For example, the step of performing image reconstruction based on the third template data to obtain a reconstructed image includes: transforming the third template data into complete projection data in the projection space according to the mapping relationship between the projection space and the template data space; and performing image reconstruction based on the complete projection data to obtain a reconstructed image.

[0007] For example, the data processing model further includes a second sub-model for repairing detailed structures in the reconstructed image; image reconstruction based on the complete projection data to obtain a reconstructed image includes: processing the complete projection data using an analytical reconstruction operator to obtain a first reconstructed image; and processing the first reconstructed image using the second sub-model to obtain a second reconstructed image.

[0008] For example, the confidence information includes the position confidence of the m2 second data points in the template data space.

[0009] For example, obtaining the location confidence score includes: for the i-th second data point among the m2 second data points, calculating the minimum distance between the i-th second data point and each of the m1 first data points, where 1≤i≤m2; and calculating the location confidence score based on the minimum distance.

[0010] For example, calculating the location confidence based on the minimum distance includes: calculating the location confidence based on the following weighted confidence function.

[0011] ,

[0012] in, The hyperparameter for adjusting the magnitude of the position confidence level, Let be the minimum distance between the i-th second data point and each of the m1 first data points. This is the weighted confidence function.

[0013] For example, the confidence information also includes the noise confidence of the line integral values ​​corresponding to the m2 second data points.

[0014] For example, obtaining the noise confidence score includes: for the i-th second data point among the m2 second data points, obtaining the line integral value corresponding to the i-th second data point. According to the line integral value The confidence function of the noise metric is defined as follows. Calculate the noise confidence level.

[0015] ,

[0016] in, The hyperparameter is used to adjust the magnitude of the noise confidence level.

[0017] For example, obtaining the confidence information of the m2 second data points includes: according to the following confidence function Calculate the confidence information of the i-th second data point among the m2 second data points.

[0018] .

[0019] For example, the data processing model includes multiple parameters; the method further includes: calculating the gradient value of the loss function with respect to each parameter based on a predefined loss function; and fine-tuning the at least one parameter in response to the gradient value of at least one parameter being greater than a predefined gradient threshold.

[0020] For example, fine-tuning the at least one parameter includes: fine-tuning the at least one parameter using the loss function, prior constraints on the reconstructed image, and constraints on the amount of fine-tuning for the at least one parameter.

[0021] For example, the method further includes: constructing a mapping relationship between the projection space and the template data space based on the projection geometry of the ray scanning system used for ray scanning and the projection viewpoint distribution of the rays.

[0022] For example, the template data space includes at least one of Radon space, Fourier space, and data space using orthogonal basis decomposition.

[0023] For example, the pre-completion of the first template data includes: pre-completion of the first template data using at least one of conjugate projection information and interpolation algorithm.

[0024] For example, the incomplete projection data includes projection data generated by at least one of sparse angle scanning and finite angle scanning.

[0025] For example, performing image reconstruction based on the complete projection data to obtain a reconstructed image includes: processing the complete projection data using a parsing reconstruction operator to obtain the reconstructed image.

[0026] For example, m3 is greater than or equal to m2.

[0027] For example, the object to be imaged includes items to be inspected in a security inspection scenario.

[0028] On the other hand, a scanning imaging system is provided, the system comprising: an imaging channel, at least a portion of which extends along a first direction, the imaging channel being used to place an object to be imaged during imaging; a radiation source for emitting radiation; a detector for detecting radiation emitted from the radiation source and passing through the object to be imaged, and generating projection data corresponding to the detected radiation, wherein the projection data is incomplete projection data in a projection space; and a data processing device for generating a reconstructed image based on the projection data using the method described above.

[0029] For example, the scanning imaging system includes a CT scanning imaging system.

[0030] The above methods or systems map projection data to a template data space and pre-complete incomplete data in the template data space, which is beneficial for handling non-standard and irregular missing data and can adapt to various data sampling distributions. By using deep learning models, the confidence information of data sampling and the pre-complete results of the template data space are fully utilized to improve the efficiency and accuracy of data completion while ensuring the fidelity of the original sampled projection. In addition, high-quality reconstructed images are obtained by using analytical reconstruction or iterative image reconstruction. Attached Figure Description

[0031] The above and other objects, features and advantages of this application will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0032] Figure 1 A schematic diagram illustrating the projection relationship between the X-ray source, the object to be imaged, and the detector is shown.

[0033] Figure 2A This is a schematic diagram of the structure of a scanning imaging system according to some exemplary embodiments of this application;

[0034] Figure 2B This is a schematic diagram of the structure of a scanning imaging system according to some other exemplary embodiments of this application;

[0035] Figure 3A This is a schematic diagram of the structure of a slip-ring CT scanning imaging system according to some exemplary embodiments of this application;

[0036] Figure 3B A planar schematic diagram of a linear trajectory scan performed in a scanning imaging system according to an embodiment of this application is shown;

[0037] Figure 3C This is a schematic diagram of the structure of a CT scanning imaging system based on a three-segment coplanar scanning structure according to some exemplary embodiments of this application;

[0038] Figure 4 This is a basic flowchart of the training and test inference phases of the data processing model used in the scanned image reconstruction method according to an embodiment of this application.

[0039] Figure 5A This is a flowchart of a training method for training a data processing model used in a scanned image reconstruction method according to an embodiment of this application;

[0040] Figure 5B This is a flowchart of a test reasoning method for testing the data processing model used in the scanned image reconstruction method according to the embodiments of this application;

[0041] Figure 6This is a flowchart of a method for reconstructing scanned images with incomplete data according to an embodiment of this application;

[0042] Figure 7 The flowchart illustrates the application of the scanned image reconstruction method for adapting incomplete data according to embodiments of this application in an exemplary scenario;

[0043] Figure 8 An exemplary confidence information graph obtained according to the method of an embodiment of this application is illustrated schematically;

[0044] Figure 9 This schematically illustrates the backbone network structure using Transformers;

[0045] Figure 10 This illustration schematically shows a confidence information graph obtained by the method according to an embodiment of this application in an exemplary application scenario;

[0046] Figure 11 The illustration shows an image in an exemplary application scenario, where, Figure 11 In the image, (a) is the true image of the object to be imaged. Figure 11 (b) in the image is the reconstructed image obtained using methods from related techniques. Figure 11 (c) in the figure represents the reconstructed image obtained using the method according to the embodiments of this application;

[0047] Figure 12 This illustration schematically shows a confidence information graph obtained by the method according to an embodiment of this application in another exemplary application scenario;

[0048] Figure 13 The illustration shows an image in another exemplary application scenario, in which, Figure 13 In the image, (a) is the true image of the object to be imaged. Figure 13 (b) in the image is the reconstructed image obtained using methods from related techniques. Figure 13 (c) in the figure represents the reconstructed image obtained using the method according to the embodiments of this application;

[0049] Figure 14 The illustration shows an image in yet another exemplary application scenario, in which, Figure 14 In the image, (a) is the true image of the object to be imaged. Figure 14 (b) in the image is the reconstructed image obtained using methods from related techniques. Figure 14 (c) in the figure represents the reconstructed image obtained using the method according to the embodiments of this application;

[0050] Figure 15 The illustration shows an image in yet another exemplary application scenario, in which, Figure 15In the image, (a) is the true image of the object to be imaged. Figure 15 (b) in the image is the reconstructed image obtained using methods from related techniques. Figure 15 (c) in the figure represents the reconstructed image obtained using the method according to the embodiments of this application;

[0051] Figure 16 A block diagram of an electronic device suitable for implementing the method according to an embodiment of this application is illustrated schematically. Detailed Implementation

[0052] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0053] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0054] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0055] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0056] It should be noted that, in this application, computed tomography (CT) imaging refers to the process of using X-rays to perform a tomographic scan of the object to be imaged, then converting the analog signals received by the detector into digital signals, calculating the attenuation coefficient of each pixel by an electronic computer, and then reconstructing the image, thereby displaying the tomographic structure of each part of the object to be imaged.

[0057] X-ray DR imaging (Digital Radiography, abbreviated as DR) refers to a two-dimensional projection imaging technique. It utilizes the penetration of X-rays into an object, where the transmitted signal is directly received by a digital detector and converted into a digital image. Specifically, X-rays can pass through an object from one direction and be captured by a detector on the other side. Because different materials absorb X-rays to varying degrees, the resulting image is based on these differences. It should be understood that in X-ray DR imaging, since all information is obtained from a single angle, a two-dimensional image is obtained; in this application, this two-dimensional image can be referred to as a two-dimensional projection image. In this two-dimensional projection image, the internal structures of the object along the direction of the X-ray output are superimposed.

[0058] X-ray tomography involves acquiring X-ray data from multiple angles and using mathematical algorithms to reconstruct cross-sections (e.g., slices) or three-dimensional volume data of an object's interior. Specifically, X-rays need to pass through the object from multiple angles, which can be achieved using a distributed X-ray source system that emits X-rays from multiple fixed locations. The data collected by the detectors is then used to reconstruct a three-dimensional image of the object's interior, or to reconstruct an image of a specific cross-section or slice within the object.

[0059] The following describes the embodiments of this application in detail, taking a scanning imaging system for objects such as luggage and parcels or a specific tissue of a living organism as an example. It should be understood that the embodiments of this application are not limited to imaging scenarios for objects such as luggage and parcels or a specific tissue of a living organism; they can be applied to various scanning imaging scenarios. For example, they can be applied to scanning imaging scenarios involving various different inspection objects, including but not limited to vehicle scanning imaging, luggage / parcel scanning imaging, human or animal scanning imaging, organ / tissue scanning imaging, small object scanning imaging, and large object scanning imaging such as containers. It should be noted that the description of scanning imaging scenarios here is not exhaustive, and the exemplary descriptions below should not be construed as limiting the scope of protection of this application.

[0060] Figure 1 A schematic diagram illustrating the projection relationship between the X-ray source, the object to be imaged, and the detector. (Refer to...) Figure 1In the embodiments of this application, rays emitted from the ray source 1 (e.g., X-rays, gamma rays, etc.) are incident on the object to be imaged 120, and the rays transmitted through the object to be imaged 120 are detected by the detector 2. A spatial point P1 on the object to be imaged is projected onto an image point P2 on the detector 2 via the ray source 1. In forward projection (also known as front projection), the pixel values ​​of the spatial points on the object to be imaged 120 are known, and the projection values ​​of the image points on the detector 2 are obtained. In backward projection (also known as back projection), the projection values ​​of the image points on the detector 2 are known, and the pixel values ​​of the spatial points on the object to be imaged 120 are obtained.

[0061] In some exemplary embodiments, the scanning imaging system may include only one scanning stage, i.e., only one source detection system. Figure 2A This is a schematic diagram of the structure of a scanning imaging system according to some exemplary embodiments of this application. (Refer to...) Figure 2A The scanning imaging system according to embodiments of this application may include a scanning stage, a transport mechanism 110, a control device 140, and a data processing device 130. For example, the scanning stage may include an X-ray source and a detector.

[0062] Figure 2B This is a schematic diagram of the structure of a scanning imaging system according to some other exemplary embodiments of this application. (Refer to...) Figure 2B The scanning imaging system according to embodiments of this application may include multiple scanning stages (e.g., first scanning stage A, second scanning stage B, ...), a transport mechanism 110, a control device 140, and a data processing device 130. For example, the scanning stage may include an X-ray source and a detector.

[0063] Figure 3A This is a schematic diagram of the structure of a slip-ring CT scanning imaging system according to some exemplary embodiments of this application. Figure 3A As shown, a scanning imaging system according to some exemplary embodiments of this application may include: an X-ray source 1, a slip ring 150, a detector 2, a transmission mechanism 110, a control device 140, and a data processing device 130, etc. It should be understood that the X-ray source 1, the detector 2, and the slip ring 150 can be configured... Figure 2A or Figure 2B One of the scan levels.

[0064] For example, the slip ring 150 may include a rotating end and a fixed end. The rotating end of the slip ring 150 may include a conductive brush and a conductive ring, with a cantilever spring maintaining contact pressure to ensure reliable electrical contact during rotation. The fixed end of the slip ring 150 may include a data receiving electrode plate, forming capacitive coupling with the rotating end for non-contact transmission, avoiding signal attenuation caused by mechanical wear. The X-ray source 1 and detector 2 may be mounted on the slip ring 150. Specifically, the X-ray source 1 and detector 2 may be located at the rotating end of the slip ring 150; for example, the X-ray source 1 and detector 2 may be symmetrically fixed to the rotating end of the slip ring 150. Driven by the slip ring 150, the X-ray source 1 and detector 2 can rotate around a predetermined rotation axis. In the embodiments of this application, the scanning imaging system can perform CT scans on the object 120 to be imaged within a field of view centered on a predetermined rotation axis. Exemplarily, the object 120 to be imaged may include luggage, packages, biological tissue, etc.; however, the embodiments of this application do not impose particular limitations on the specific form of the object to be imaged.

[0065] For example, the radiation source 1 can be an X-ray machine. The detector 2 can be used to receive at least a portion of the radiation beam emitted by the X-ray machine. The detector 2 can include multiple detector modules for detecting the radiation beam passing through the object to be imaged 120, obtaining analog signals, and converting the analog signals into digital signals to output projection data of the object to be imaged 120 relative to the X-rays. The control device 140 is used to control the synchronous operation of various parts of the entire system. The data processing device 130 is used to process the projection data acquired by the detector 2, process and reconstruct the projection data, and output the results (e.g., a reconstructed image).

[0066] like Figure 3A As shown, X-ray source 1 is placed on one side of the object 120 to be imaged, and detector 2 is placed on the other side of the object 120. During the operation of the scanning imaging system, the focal point of X-ray source 1 emits a X-ray beam, and detector 2 acquires the transmission data and / or multi-angle projection data of the object 120. For example, the data acquisition unit in detector 2 may include a data amplification and shaping circuit, which can operate in (current) integration mode or pulse (counting) mode. The data output cable of detector 2 is connected to control device 140 and data processing device 130, and the acquired data is stored in data processing device 130 according to trigger commands.

[0067] The transmission mechanism 110 carries the object to be imaged 120 and passes through the scanning area between the X-ray source 1 and the detector 2 via the slip ring 150. Simultaneously, the slip ring 150 rotates around the translational direction of the object to be imaged 120, allowing the X-ray beam emitted by the X-ray source 1 to pass through the object to be imaged 120 and perform a CT scan. In other words, the transmission mechanism 110 can cause relative motion between the X-ray source 1 and the object to be imaged 120, causing the X-ray source 1 to move relative to the object to be imaged 120 along a predetermined scanning trajectory.

[0068] In some embodiments, during the scanning process of the X-ray source 1, the slip ring 150 drives the X-ray source 1 and detector 2 to rotate, while the object to be imaged 120 remains stationary; that is, the X-ray source 1 moves along a circular trajectory relative to the object to be imaged 120. In other embodiments, during the scanning process of the X-ray source 1, the slip ring 150 drives the X-ray source 1 and detector 2 to rotate, and the object to be imaged 120 translates under the drive of the conveying mechanism 110; that is, the X-ray source 1 moves along a helical trajectory relative to the object to be imaged 120.

[0069] Figure 3B A planar schematic diagram of linear trajectory scanning performed in a scanning imaging system according to an embodiment of this application is shown. Figure 2A , Figure 2B and Figure 3B As shown, the object to be imaged 120 moves along a straight trajectory between the X-ray source 1 and the detector 2. During this movement, the X-ray source 1 emits X-rays according to the commands of the control system, penetrating the object to be imaged 120. The detector 2 receives the transmitted signal and acquires projection data under the control of the control device 140. It should be noted that... Figure 3B The scanning structure shown, including the radiation source and detector, can correspond to... Figure 2A and Figure 2B One scan level is shown in the image.

[0070] like Figure 3B As shown, exemplarily, the radiation source 1 may include a radiation source such as an X-ray accelerator, an X-ray machine, or a radioactive isotope, and corresponding auxiliary equipment. Optionally, in order to make the horizontal beam angle (i.e., the beam fan angle) greater than 90 degrees, for example, between 90 degrees and 180 degrees, two or more radiation sources may be used, selected according to the size of the object 120 to be imaged and the application scenario.

[0071] The conveying mechanism 110, such as a conveyor belt, can carry and smoothly transport the object to be imaged 120, and is used to move the carried object 120 along a straight trajectory during the scanning imaging process. Optionally, the conveying mechanism 110 moves the X-ray source and detector along a straight trajectory during the scanning imaging process, or moves the object to be imaged towards the X-ray source and detector. That is, the movement of the object to be imaged and the movement of the X-ray source and detector are relative movements.

[0072] Detector 2 may include a detector array for acquiring transmission projection data of rays by receiving rays that pass through the object to be imaged. Detector 2 may also include readout circuitry and a logic control unit for reading out the projection data on the detector array. The detector array may include multiple solid-state detector units, multiple gas detector units, or multiple semiconductor detector units.

[0073] In some exemplary embodiments of this application, the scanning imaging system may further include a multi-segment coplanar scanning structure. Below, a CT scanning imaging system with a three-segment scanning structure is used as an example to describe the embodiments of this application in detail. It should be understood that the following CT scanning imaging system based on a three-segment coplanar scanning structure is merely an exemplary structure for illustrative purposes. Those skilled in the art should understand that the number and position of the X-ray source and detector can be adjusted according to specific needs to accommodate objects of different sizes and shapes to be imaged.

[0074] Figure 3C This is a schematic diagram of a CT scanning imaging system based on a three-segment coplanar scanning structure according to some exemplary embodiments of this application. It should be noted that... Figure 3C The scanning structure shown, including the radiation source and detector, can correspond to... Figure 2A and Figure 2B One scan level is shown in the image.

[0075] Combined with reference Figure 2A , Figure 2B and Figure 3CA scanning imaging system according to some exemplary embodiments of this application may include: a radiation source 1, a detector 2, and an imaging channel 160, etc. The imaging channel 160 extends along a first direction z and is used to place the object to be imaged 120 during the imaging process. For example, at least a portion of a support device may be disposed in the imaging channel 160, and the object to be imaged 120 may be placed on the support device. During scanning, the object to be imaged 120 may remain stationary in the imaging channel 160. As another example, at least a portion of a transport mechanism 110 is disposed in the imaging channel 160. The transport mechanism 110 carries the object to be imaged 120 through the scanning area between the radiation source 1 and the detector 2. The transport mechanism 110 can move the object to be imaged 120 along the first direction z, while the radiation emitted by the radiation source 1 can pass through the object to be imaged 120 to perform a CT scan.

[0076] Combined with reference Figure 2A , Figure 2B and Figure 3C In some exemplary embodiments, the X-ray source 1 of the CT scanning imaging system may include multiple distributed X-ray sources, for example, M distributed X-ray sources, where M is a positive integer greater than or equal to 2. Figure 3C In the exemplary embodiment shown, M=5, that is, five distributed radiation sources are provided. In this application, for ease of description, the five distributed radiation sources are respectively numbered as the first distributed radiation source 11, the second distributed radiation source 12, the third distributed radiation source 13, the fourth distributed radiation source 14, and the fifth distributed radiation source 15.

[0077] In the embodiments of this application, the distributed X-ray source includes multiple target points (also known as focal points). For example, multiple X-ray target points can be arranged at a high density within a single X-ray tube vacuum chamber. Compared to a traditional single X-ray source, a distributed X-ray source has multiple independent X-ray emission points, which can operate independently and be controlled individually. For example, the distributed X-ray source developed by the applicant of this application can integrate hundreds of X-ray target points within a single X-ray tube, each of which can be independently controlled and quickly switched as needed. For example, the emission of each target point can be precisely controlled, including parameters such as X-ray intensity and emission time, allowing for flexible adjustment of X-ray output based on different scanning locations, object density, and other factors during CT scans, thereby obtaining higher-quality images. For example, the intensity distribution of the X-ray beam can be modulated by controlling the X-ray intensity of different target points to adapt to the imaging requirements of different objects. Furthermore, by employing a multi-target distributed X-ray source, rapid scanning imaging capabilities can be achieved. Due to the presence of multiple X-ray target points, the distributed X-ray source can simultaneously or rapidly scan target objects sequentially from multiple different locations, greatly improving scanning speed. Compared to traditional CT scans, this method can acquire complete image information of an object in a shorter time, which is significant for scenarios requiring rapid imaging, such as medical and security checks. Furthermore, it enables high-quality imaging results; the collaborative work of multiple targets and precise control allow distributed X-ray sources to provide higher resolution and lower noise images. When imaging complex objects or minute structures, it can more clearly reveal the object's internal structure and details, providing more accurate information for doctors' diagnoses and other applications.

[0078] In some exemplary embodiments, the CT scanning imaging system may include multiple detectors 2, for example, N detectors, where N is a positive integer greater than or equal to 2. Figure 3C In the exemplary embodiment shown, N=3, that is, three detectors are provided. In this application, for ease of description, the three detectors are respectively numbered as the first detector 21, the second detector 22, and the third detector 23.

[0079] Reference Figure 3C In some exemplary embodiments, among the M distributed radiation sources, at least one distributed radiation source includes qs target points 101, where qs is a positive integer greater than or equal to 2. For example, each of the M distributed radiation sources includes more than two target points. Figure 3C In the illustrated embodiment, qs=6, meaning a distributed radiation source comprises 6 target points. Exemplarily, qs target points are configured to be activated in a predetermined order to emit radiation.

[0080] It should be understood that the five distributed X-ray sources and three detectors mentioned here are merely exemplary embodiments and should not be construed as limiting the number of distributed X-ray sources and detectors in the embodiments of this application. In other embodiments, the CT scanning imaging system may include fewer distributed X-ray sources and detectors, such as two, or more, such as six, eight, ten, or more. When more than two distributed X-ray sources and detectors are set, X-rays can be emitted and received simultaneously or sequentially from different angles and positions, thereby improving the speed and resolution of imaging. Alternatively, three, four, or five distributed X-ray sources and detectors can be set. The combination of more than three distributed X-ray sources and detectors can provide a more comprehensive and detailed scan for imaging complex objects. For example, setting six distributed X-ray sources and detectors can achieve X-ray coverage over a larger area, which has significant advantages for imaging large objects or specific application scenarios requiring high resolution. In some cases, more than eight distributed X-ray sources and detectors can be set. The configuration of more than eight distributed X-ray sources and detectors can realize more complex scanning modes and multi-angle imaging, providing richer image information for security inspection scenarios, doctors' diagnoses, and researchers' analyses. It should be understood that as the number of distributed X-ray sources increases, CT scanning imaging systems can achieve more refined imaging, making them more adept at observing minute structures and performing high-precision detection tasks. In the embodiments of this application, the number of distributed X-ray sources and detectors can be flexibly adjusted according to specific application needs to meet the CT imaging requirements of different fields and scenarios.

[0081] It should also be understood that Figure 3C The distributed X-ray source in the figures, including six target points, and the number of target points shown in other figures are merely exemplary embodiments and should not be construed as a limitation on the number of target points in the embodiments of this application. In other embodiments, the distributed X-ray source of the CT scanning imaging system may include fewer target points, such as fewer than five, or more target points, such as 10, 20, 50, 100, hundreds, or more. In the embodiments of this application, the number of target points in the distributed X-ray source can be flexibly adjusted according to specific application requirements to meet the CT imaging requirements in different fields and scenarios.

[0082] It should be noted that, unless otherwise specified, the terms "first," "second," "third," etc., used in this application are merely for the convenience of referring to different components, such as the distributed radiation source and detector mentioned here, and should not be construed as imposing any limitation on the structure of the distributed radiation source and detector. In the embodiments of this application, the structures of the first distributed radiation source 11, the second distributed radiation source 12, the third distributed radiation source 13, the fourth distributed radiation source 14, and the fifth distributed radiation source 15 may be identical, or some of the distributed radiation sources may have the same structure while others may have different structures. For example, the first distributed radiation source 11, the second distributed radiation source 12, the third distributed radiation source 13, the fourth distributed radiation source 14, and the fifth distributed radiation source 15 may include the same number of target points, or the number of target points included in some of the distributed radiation sources may differ from the number of target points included in others, or any two distributed radiation sources may include different numbers of target points. In the embodiments of this application, the first detector 21, the second detector 22 and the third detector 23 may have the same structure, or some of the detectors may have the same structure while the other part of the detectors may have different structures.

[0083] Combined with reference Figure 2A , Figure 2B and Figure 3C In this application, for ease of description, the first direction z, the second direction x, the third direction y, the circumferential direction c of the imaging channel, and the radial direction r are described respectively. For example, the first direction z corresponds to the extension direction of the imaging channel 160, or, when the object to be imaged 120 moves, its direction of movement. The second direction x and the third direction y intersect, and the first direction z is perpendicular to both the second direction x and the third direction y. For example, the second direction x corresponds to the width direction of the scanning imaging system, and the third direction y corresponds to the height direction of the scanning imaging system. The circumferential direction c of the imaging channel intersects with the radial direction r, and the first direction z is perpendicular to both the circumferential direction c and the radial direction r of the imaging channel. For example, the radial direction r of the imaging channel 160 corresponds to the direction from the center of the imaging channel 160 to the outer periphery of the imaging channel 160, and the circumferential direction c of the imaging channel 160 corresponds to the direction surrounding the imaging channel 160. It should be noted that, in the embodiments of this application, the definitions and descriptions of various coordinate systems and directions are merely exemplary descriptions for the convenience of describing the embodiments of this application, and they are not intended to limit the embodiments of this application.

[0084] Reference Figure 2A , Figure 2B and Figure 3CFive distributed X-ray sources 11 to 15 are arranged along the circumferential direction c of the imaging channel 160. Exemplarily, the five distributed X-ray sources 11 to 15 are arranged at intervals along the circumferential direction c of the imaging channel 160. Three detectors 21 to 23 are arranged along the circumferential direction c of the imaging channel 160. Exemplarily, the three detectors 21 to 23 are arranged continuously or at intervals along the circumferential direction c of the imaging channel 160.

[0085] It should be understood that Figure 3C A schematic diagram of a plane perpendicular to the first direction z, i.e. Figure 3C The plane of the paper is perpendicular to the first direction z. For example... Figure 3C As shown, in a plane perpendicular to the first direction z, five distributed ray sources 11 to 15 and three detectors 21 to 23 surround and form a closed or nearly closed polygon.

[0086] In the above embodiments, circular trajectory, spiral trajectory, single-segment straight line trajectory and multi-segment straight line trajectory are used as examples to describe the scanning imaging system. It should be noted that the scanning imaging system according to the embodiments of this application is not limited to this. For example, in other embodiments, the scanning imaging system according to the embodiments of this application may include a tomographic imaging system.

[0087] It should be understood that, in the embodiments of this application, during the scanning imaging process, the X-ray source 1 emits a X-ray beam to scan the object to be imaged 120; in response to the X-ray beam scanning the object to be imaged 120, the detector 2 can acquire the projection data of the object to be imaged 120. The data processing device 130 can process the projection data acquired by the detector 2, process and reconstruct the projection data, and generate a reconstructed image.

[0088] The inventors discovered that in various scanning imaging systems, incomplete data can occur due to constraints from actual scanning conditions and non-ideal factors. For example, incomplete data scanning can include sparse-view CT scans, limited-angle CT scans, combinations of sparse-view CT scans and limited-angle CT scans. In sparse-view CT scans, the angle sampling density can be reduced while ensuring complete coverage of the projection angle range (e.g., 0°~180° or 0°~360°). That is, within the complete scanning angle range, the number of projection data acquisitions is reduced, and the angle interval between adjacent projections becomes larger. The characteristics of sparse-view CT scans include: complete angle coverage, only reduced angle sampling density. Sparse-view CT scans can effectively reduce the number of X-ray projections, thereby reducing radiation dose and shortening scan time. In limited-angle CT scans, the projection angle range cannot completely cover the standard scanning range; projection data can only be acquired within a limited angle range. The characteristics of limited-angle CT scans include: incomplete scanning angle coverage, with obvious missing angle intervals. For example, limited-angle CT scanning can be applied in the following scenarios: constrained by factors such as scanning time and physical space, for example, dynamic organ imaging (such as scanning during heartbeat) requires a very short time to complete and cannot cover the full angle; in in-situ inspection of large industrial components, the scanning imaging system cannot rotate around the component at all angles; and in the field of security inspection, the angle range is limited to improve throughput and shorten scanning time. In addition, in certain specific system architectures, due to the irregular geometric arrangement of the X-ray source and detector, both sparse-view CT scanning and limited-angle CT scanning may coexist, resulting in more complex scanning situations with incomplete data.

[0089] For incomplete data obtained in the above situations, image reconstruction is still required in scanning imaging systems to obtain reconstructed images. The inventors discovered that traditional analytical reconstruction methods for image reconstruction with incomplete data suffer from severe artifacts and structural defects. Model-driven iterative reconstruction based on Bayesian theory organically combines imaging physics models, statistical properties, and image priors, which can improve the quality of reconstructed images to a certain extent. Such methods construct a joint optimization problem by combining data consistency terms and sparse priors. However, these manually designed constraints usually have limited effectiveness, potentially leading to limited imaging quality or even the introduction of new artifacts in practical applications. In recent years, deep learning methods have demonstrated powerful capabilities in solving various incomplete data reconstruction problems. Its basic idea is to train a neural network based on large-scale data, extract shared features and introduce implicit priors to assist in reconstruction. The relevant methods are mainly divided into three categories: (1) Single data domain method, which uses a neural network to complete the sine curve in the projection domain to estimate the fully sampled projection data, or performs artifact suppression on the degraded reconstructed image in the image domain to improve image quality. This type of method is computationally efficient, but it is difficult to explicitly fuse imaging physics and ensure consistency with projection data; (2) Cross-data domain method, which constructs a joint optimization mechanism between the projection domain and the image domain and incorporates the constraints of the imaging model; (3) Deep unfolding network alternately uses data consistency constraints and data-driven priors in iterative optimization and multi-stage learning, and deeply fuses the imaging physics model and data-driven learning.

[0090] However, further research by the inventors revealed the following problems with the three types of methods: First, they are only designed for specific imaging geometries and data sampling distributions, making it difficult to handle multiple different data missing problems simultaneously, resulting in poor reconstruction performance and robustness. Second, most of these methods are designed for regular data missing problems, making it difficult to handle non-standard geometries, non-uniform sampling, and other data sampling conditions. Third, when there are significant differences between the training data and the actual application scenario (e.g., the structure of the scanned object, the projection geometry, the physical parameters of the X-ray source and detector, the data sampling distribution, and noise), the performance of the methods will drop significantly, severely affecting the quality of the reconstructed image.

[0091] To address at least one aspect of the aforementioned problems, embodiments of this application provide a method for reconstructing scanned images using incomplete data. The method includes: in response to a ray scanning of an object to be imaged, acquiring projection data of the object, wherein the projection data is incomplete projection data in a projection space; transforming the incomplete projection data into first template data according to a mapping relationship between the projection space and a template data space, wherein the first template data includes m1 first data points in the template data space, where m1 is a positive integer greater than 1; and pre-completing the first template data to obtain second template data, wherein the second template... The data includes m2 second data points in the template data space, where m2 is a positive integer greater than m1; confidence information of the m2 second data points is obtained; the second template data and confidence information are used as input to a pre-trained data processing model, wherein the data processing model includes a first sub-model, which is used to complete the template data in the template data space; the input is processed using the first sub-model to obtain third template data, wherein the third template data includes m3 third data points in the template data space, where m3 is a positive integer greater than 1; image reconstruction is performed based on the third template data to obtain a reconstructed image.

[0092] In the embodiments of this application, the projection data is mapped to the template data space, and incomplete data is pre-completed in the template data space, which is beneficial for handling non-standard and irregular missing data and can adapt to various data sampling distributions. By using a deep learning model, the confidence information of data sampling and the pre-completed results of the template data space are fully utilized to improve the efficiency and accuracy of data completion while ensuring the fidelity of the original sampled projection. In addition, high-quality reconstructed images are obtained by using analytical reconstruction or iterative image reconstruction.

[0093] In embodiments of this application, the data processing model may include a deep learning model, such as a deep neural network model. The data processing model may include a training phase and a test / inference phase.

[0094] The training phase is used to enable the model to learn patterns for performing specific tasks from massive amounts of data. This process requires preparing a large amount of labeled or unlabeled training data and defining a validation set for monitoring training effectiveness. Then, the model's network structure is defined and all parameters are initialized. During training, data is first input into the model to obtain prediction results. Then, the error between the predicted and true values ​​is calculated using a loss function. Next, using the backpropagation algorithm, each parameter of the model is iteratively updated along the opposite direction of the loss function's gradient using optimizers such as gradient descent. This cycle of "forward propagation—calculate loss—backward propagation—parameter update" can be repeated multiple times until the model's performance on the validation set reaches the expected level. At this point, training is complete, and a fixed set of optimal parameters is obtained.

[0095] The testing and inference phase is the stage for verifying the model's generalization ability and putting it into practical application. It is divided into two parts: testing and inference. The testing phase is used to evaluate the model's generalization ability. For example, a set of test data that has never been used in training can be input into the trained model. Metrics such as accuracy and recall are used to judge the model's performance on unfamiliar data. Simultaneously, error cases are analyzed to identify overfitting, underfitting, and other problems, determining whether the model meets the application criteria. If the metrics of each computational unit of the pre-trained model reach the pre-set thresholds, the pre-trained model is considered well-suited for this test scenario and can be directly used for inference. Otherwise, the model can be fine-tuned by adjusting some parameters of the pre-trained model. The inference phase is the actual working stage of the model. At this point, the model's parameters are completely fixed and no further updates are made. New, real-time input data is preprocessed in the same way as during training, then input into the model to perform forward propagation calculations. Finally, post-processing of the output results yields usable prediction conclusions. It should be understood that the inference stage does not require backpropagation, so the demand for computing power is much lower than that of the training stage. It also focuses more on low latency and fast response. In practical applications, the running efficiency will be further optimized through model quantization, pruning and other means.

[0096] Figure 4 This is a basic flowchart of the training and test inference phases of the data processing model used in the scanned image reconstruction method according to embodiments of this application. Figure 5A This is a flowchart of a training method for training a data processing model used in a scanned image reconstruction method according to embodiments of this application. (Refer to...) Figure 4 and Figure 5A The training method for training the data processing model used in the scanned image reconstruction method according to the embodiments of this application may include steps S510 to S550.

[0097] In step S510, projection data, projection geometry information, and projection viewing angle distribution information of the scanning imaging system used to acquire the projection data are obtained. For example, this can be achieved using... Figures 3A-3C The projection data acquired by any one of the provided scanning imaging systems is used as the training dataset, and accordingly, Figures 3A-3CThe projection geometry information and projection viewpoint distribution information of any of the provided scanning imaging systems can be used as training data corresponding to the aforementioned projection data. For example, the projection geometry information may include, but is not limited to: the relative motion relationship between the X-ray source or detector and the object to be imaged, for example, it can be characterized by a circular trajectory, a spiral trajectory, a straight line trajectory, or a multi-segment straight line trajectory; the relative positional relationship between the X-ray source, detector, and object to be imaged, which can be characterized by relative angles or relative distances. For example, the projection viewpoint distribution information may include, but is not limited to: information used to characterize the scanning angle, such as finite angle scanning, sparse angle scanning, and a hybrid scanning of sparse and finite angles; specific scanning angles, acquisition angle intervals, etc.

[0098] by Figure 3C Taking the multi-segment linear trajectory scanning imaging system shown as an example, step S510 is described exemplarily. Figure 3C As shown, the angle of a distributed radiation source is The target coordinates within a distributed radiation source are: The fan angle of the rays emitted from the target is The radial distance from the origin O to a distributed ray source is For example, the angle of a distributed radiation source. The target coordinates within a distributed ray source can be represented by the angle of the line connecting the center of the distributed ray source and the origin O relative to the third direction y. This can be represented by the distance between the target point and the center of the distributed radiation source along the target point's orientation. The fan angle of the radiation emitted from the target point... This can be represented by the angle of the emitted ray relative to the main ray beam (or central ray beam) emitted from the target point. The radial distance from the origin O to a distributed ray source. It can be represented by the length of the line segment connecting the center of the distributed ray source and the origin O.

[0099] exist Figure 3C In the CT scanning imaging system with multiple straight-line trajectories shown, for any given ray, its projection geometry and projection angle distribution information can be obtained using... According to a specific scanning beam strategy, the projection space formed by each ray can be represented as: ,in, This represents the total number of target points included in a distributed radiation source. This indicates that The serial number of each target point Between 1~ between, Represents the i-th distributed ray source. The projection geometry and projection viewpoint distribution information corresponding to each target point This represents the projection space formed by each ray. Correspondingly, in Figure 3C In the CT scanning imaging system shown, by controlling the target points of each X-ray source to emit X-rays and controlling the detector to acquire projection data through a specific scanning beam-emitting strategy, projection data of the object to be imaged can be obtained. It should be understood that this projection data is incomplete data in the projection space; for ease of description, the incomplete data in the projection space is denoted as... Incomplete data It is a projection dataset that includes multiple projection data.

[0100] In step S520, the incomplete projection data is transformed into first template data according to the mapping relationship between the projection space and the template data space. The first template data includes m1 first data points in the template data space, where m1 is a positive integer greater than 1.

[0101] For example, the template data space may include at least one of Radon space, Fourier space, and data space using orthogonal basis decomposition.

[0102] It should be understood that in incomplete scanning imaging scenarios such as those with limited or sparse angles, the characteristics of the projection space lead to poor results when directly performing completion within the projection space, resulting in severe artifacts in the reconstructed image. Specifically, in the projection space of scenarios such as CT scans, X-ray attenuation projection values ​​at different angles are arranged into a data space using a two-dimensional grid of "angle-detector channel". In this data space, the data correlation between pixel values ​​of adjacent angles and adjacent detector channels is weak, and completion lacks reliable constraints. For example, for limited-angle scans, there is no continuous gradient between the projection data of missing angle intervals and the projection data of existing angle intervals, and direct interpolation (such as linear interpolation) may lead to a complete deviation from the true projection distribution. Furthermore, for sparse-angle scans, the angle intervals are too large, and the sampling points of the projection data are sparse and discrete, making it impossible to capture the continuous changes in the angle dimension during completion. In addition, the completion error in the projection space is directly transmitted to the image space through reconstruction algorithms such as filtered backprojection and is amplified. For example, if incorrect projection values ​​are used to fill in missing angles in the projection space, the reconstructed image will show obvious stripe artifacts or angle artifacts, or even be completely distorted.

[0103] In the embodiments of this application, by converting incomplete data to template data spaces such as Radon space, Fourier space, and data space using orthogonal basis decomposition, the structured features of the data become more obvious, which is beneficial to improving the accuracy and stability of data completion.

[0104] The following description, using the template data space as an example, illustrates step S520. (Refer to the reference...) Figure 3C Projected spatial coordinates With Lardon spatial coordinates The mapping relationship between them can be expressed by the following formula: Based on the above equation, a data domain transformation operator from the projection space to the template data space can be defined. This maps data sampling points (i.e., projected data) within the projection space P to data points within the Radon space R. For example, this can be achieved through a data domain transformation operator. The above incomplete data Transformed into data in the Radon space, for ease of description, the data in the Radon space here is denoted as... It should be understood that the data in the Radon space... The data is incomplete and includes multiple (e.g., m1) first data points.

[0105] In step S530, incomplete data, for example, in the Radon space, are processed. The first template data is pre-completed to obtain the second template data, wherein the second template data may include m2 second data points in the template data space, where m2 is a positive integer greater than m1.

[0106] For example, incomplete projection data in the projection space is mapped to the template data space. After obtaining the incomplete data in the template data space, data pre-completion can be performed on the incomplete data in the template data space using conjugate projection information and interpolation to obtain the complete data in the template data space, i.e., the second template data. For example, incomplete projection data in the projection space can be... Mapping to the Radon space yields incomplete data within that space. Then, two-dimensional linear interpolation can be used to process incomplete data in the Radon space. Perform data pre-completion to obtain pre-completion data in the Radon space. This refers to the second template data. The pre-complete data in this Radon space. This includes multiple (e.g., m²) second data points. The second template data is obtained by pre-completing the first template data; it should be understood that it includes more data points than the first template data.

[0107] In step S540, the confidence information of m2 second data points in the second template data is obtained.

[0108] In the embodiments of this application, the second template data is the data generated after pre-completing the first template data in the template data space. A portion of the m2 second data points are the original m1 first data points in the first template data, and this portion of data points has a high confidence level. Another portion of the m2 second data points are data points generated by interpolation and other processing of the m1 first data points, and the confidence level of this portion of data points is related to the distance between the newly generated data points and the original data points. Based on this, a discrete sampling point distribution of complete data sampling can be defined in the template data space. The confidence level metric function q is constructed by using the distance d between the original sampling points (i.e., the m1 first data points in the first template data) and the complete sampling points (i.e., the m2 second data points in the second template data) to evaluate the sampling confidence level of the m2 second data points located at different positions in the template data space. It should be understood that the confidence level metric function q can satisfy the following basic characteristics: (1) when d=0, q=1; (2) the larger d is, the smaller q is; (3) the minimum value of q is 0. Therefore, the confidence information of the m2 second data points in the second template data can be obtained.

[0109] In step S550, the pre-built data processing model is trained using the confidence information of the m2 second data points in the second template data.

[0110] For example, the data processing model may include at least one of a deep neural network for data completion and a deep neural network for data completion and image reconstruction. Regarding the backbone network structure of the data processing model, network structures such as U-Net (U-shaped network) and Transformer (a neural network architecture based on self-attention mechanism) can be selected, and generative artificial intelligence methods (such as generative adversarial networks, diffusion models, etc.) can be combined to learn in a single data domain (projection domain, image domain, Fourier domain, etc.) or multiple data domains. For the confidence information guidance mechanism, a method such as "pre-completed results of the template data space (i.e., the aforementioned second template data) + confidence information" input can be adopted, combined with an attention mechanism, to enhance the deep neural network's ability to perceive data distribution.

[0111] For example, the data processing model may include a first sub-model, which is used to complete the template data in the template data space. Second template data and confidence information can be used as input, for example, the second template data and confidence information can be used as dual-channel input to a pre-trained data processing model, specifically, the first sub-model included in the data processing model. The first sub-model processes the input (e.g., the dual-channel input) to obtain third template data, wherein the third template data includes m3 third data points in the template data space, where m3 is a positive integer greater than 1, for example, m3 is a positive integer greater than or equal to m2. Through the completion processing of the first sub-model, the second template data can be further completed, and the pre-completed result of the second template data can be corrected, i.e., the accuracy of the completed data can be improved.

[0112] For example, in an embodiment of this application, the third template data can also be transformed into complete projection data in the projection space according to the mapping relationship between the projection space and the template data space. Then, image reconstruction is performed based on the complete projection data to obtain a reconstructed image.

[0113] Optionally or additionally, in embodiments of this application, the data processing model may further include a second sub-model, which is used to repair the detailed structures in the reconstructed image. That is, image reconstruction based on complete projection data to obtain a reconstructed image includes: processing the complete projection data using an analytical reconstruction operator to obtain a first reconstructed image; and processing the first reconstructed image using a second sub-model to obtain a second reconstructed image. The processing by the second sub-model can improve the quality of the reconstructed image.

[0114] In the example of this application, by optimizing the network architecture of the data processing model to include a backbone network structure and a guidance mechanism for confidence information, the confidence information of data sampling and the pre-completion results of the template data space can be fully utilized to achieve efficient completion of the projection data. To balance the data fidelity of the original projection and the "translation-invariant" characteristic of the template data space, and to leverage the advantages of neural network learning, data completion can be performed first in the template data space, and then the completion result can be converted back to the original projection space. This ensures the fidelity of the original sampled projection while improving the efficiency and accuracy of data completion.

[0115] Figure 5B This is a flowchart of a test inference method for testing the data processing model used in the scanned image reconstruction method according to embodiments of this application. (Refer to...) Figure 4 , Figure 5A and Figure 5B The test reasoning method for testing the data processing model used in the scanned image reconstruction method according to the embodiments of this application may include steps S560 to S590.

[0116] In step S560, data used in the test inference phase is acquired. It should be understood that the data used in the test inference phase is different from the data used in the training phase. For example, the data used in the test inference phase may include projection data acquired in a real-world application scenario, projection geometry information of the scanning imaging system used to acquire the projection data, and projection viewpoint distribution information.

[0117] In step S570, the generalization performance of the data processing model pre-trained in the above training phase is evaluated. For example, an index can be defined to evaluate the parameter generalization performance of the pre-trained data processing model, and the generalization performance index of each computational unit of the pre-trained data processing model can be calculated using the data used in the test and inference phase. It should be noted that the computational unit here can be defined as a convolutional kernel, convolutional layer, or a module composed of multiple network layers, etc., as needed.

[0118] In step S580, based on the evaluation results of generalization performance, it is determined whether the pre-trained data processing model needs fine-tuning. For example, if the metrics of each computational unit in the pre-trained data processing model all reach the preset threshold, it is determined that the pre-trained data processing model is well applicable to the test scenario, and the pre-trained data processing model does not need fine-tuning; if the metrics of each computational unit in the pre-trained data processing model do not reach the preset threshold, it is determined that the pre-trained data processing model needs fine-tuning.

[0119] In step S590, in response to determining that the pre-trained data processing model needs fine-tuning, the parameters of the pre-trained data processing model in the training phase are fine-tuned.

[0120] In embodiments of this application, the data processing model may include multiple parameters. In step S590, the method may include: calculating the gradient value of the loss function with respect to each parameter based on a predefined loss function; and fine-tuning the at least one parameter in response to the gradient value of at least one parameter being greater than a predefined gradient threshold. Exemplarily, fine-tuning the at least one parameter may include: using the loss function, prior constraints on the reconstructed image, and constraints on the amount of fine-tuning for the at least one parameter.

[0121] For example, in response to determining that the pre-trained data processing model does not require fine-tuning, the data processing model can be moved to the inference stage or the application stage.

[0122] The following example, using the data processing model including a neural network, will be used to further illustrate steps S580 and S590.

[0123] For example, in step S580, for the pre-trained neural network The initial values ​​of the network parameters are Define the loss function This loss function is used to evaluate the generalization performance of each computational unit in the neural network. The data used in the test inference phase may include projection data acquired in real-world application scenarios, projection geometry information of the scanning imaging system used to collect this projection data, and projection viewpoint distribution information.

[0124] For projection data (i.e., incomplete data) acquired in practical application scenarios If complete projection data of the object to be imaged can be obtained, the reconstruction result will be successful. Then the loss function can be defined as the supervised loss function expressed by the following equation. :

[0125] ,

[0126] in, Indicates: When the network parameters are In this case, using a pre-trained neural network Processing projection data (i.e., incomplete data) acquired in real-world application scenarios The result; The L2 norm, also known as the Euclidean norm, is calculated for vectors by taking the square root of the sum of the squares of the differences between the elements.

[0127] For projection data (i.e., incomplete data) acquired in practical application scenarios If complete projection data of the object to be imaged cannot be obtained, the reconstruction result will be... If only incomplete projection data can be obtained, then the forward projection matrix of the imaging system can be used. For imaging models, a self-supervised loss function can be constructed using the following formula:

[0128] ,

[0129] in, Indicates: When the network parameters are In this case, using a pre-trained neural network Processing projection data (i.e., incomplete data) acquired in real-world application scenarios The result; Indicates: To Projection data obtained after performing orthographic projection; The L2 norm, also known as the Euclidean norm, is calculated for vectors by taking the square root of the sum of the squares of the differences between the elements.

[0130] Based on the above loss function, the following formula can be used to calculate the computational units in the pre-trained neural network. gradient value :

[0131] ,

[0132] Where the gradient value is the loss function Parameters of this computing unit The partial derivatives reflect the parameters Change and loss function The relationship of change; Indicates the first Each computing unit.

[0133] In some exemplary embodiments, the gradient threshold can be reasonably set by referring to the gradient values ​​of each computational unit in the training dataset. When the gradient value calculated using the above formula is lower than the set gradient threshold, the neural network can be directly used for inference. When the gradient value calculated using the above formula is higher than the set gradient threshold, the parameters of the computational units exceeding the threshold are... Use the following fine-tuning steps to make fine adjustments.

[0134] In this step, generalization performance can be evaluated using gradients to determine the fine-tuning range. In other words, gradient values ​​can be used to determine whether computational units in the neural network need fine-tuning. The dataset from the real-world application scenario is input into the pre-trained neural network, and the gradient values ​​of each computational unit (e.g., weights of convolutional layers, activation parameters, etc.) are calculated. When the gradient value is below a threshold, it indicates that the parameter has a small impact on the loss function in the current real-world scenario, and the parameter is suitable for the scenario; in this case, the parameter can be directly retained without fine-tuning. When the gradient value is above a threshold, it indicates that the parameter has a large impact on the loss function in the current real-world scenario, and the parameters of the pre-trained neural network are not well-suited to the scenario; these parameters are marked as... It needs to be fine-tuned locally.

[0135] For example, in step S590, the loss function defined above is used. This is combined with prior constraints on the reconstructed image (such as TV constraints and low-rank constraints). And constraints on the fine-tuning of parameters in pre-trained neural networks (such as low-rank constraints). The parameters of the pre-trained neural network are locally fine-tuned to obtain an image reconstruction network adapted to real-world application scenarios. This fine-tuning process can be expressed by the following formula:

[0136] ,

[0137] in, The fine-tuned model parameters obtained after final optimization are the output of this optimization problem. It is a hyperparameter used to balance prior constraints. The weights in the function are used to control the strength of the influence of prior constraints on model optimization. It is a hyperparameter used to balance the constraints of parameter fine-tuning. The weights in the function are used to control the strength of the constraints on the fine-tuning of the pre-trained parameters. These are the initial values ​​for the parameters that need to be fine-tuned in the pre-trained model.

[0138] The above fine-tuning formula describes a local fine-tuning approach for a pre-trained model, which involves fine-tuning some parameters based on the pre-trained model. This yields optimal model parameters that fit the current image reconstruction task. The optimization logic is: to minimize the reconstruction loss. At the same time, two regular expression constraints are added, the first one... This is a priori constraint on the reconstructed image, used to ensure that the generated image conforms to the prior knowledge of image quality. The second term... Hyperparameters are constraints on the amount of parameter fine-tuning, used to prevent the model from overfitting the current task, thereby preserving the generalization ability of the pre-trained model. and The importance of these two constraints in the overall optimization objective is controlled separately to achieve a balance between reconstruction performance and preservation of pre-trained knowledge.

[0139] In the embodiments of this application, the pre-trained neural network performs well on general datasets, but its generalization performance may decrease when directly transferred to the reconstruction scenario of incomplete projection data in reality. To address this, the entire neural network is not trained from scratch. Instead, the neural network is evaluated by gradient evaluation, then the computational units that need fine-tuning are selected, and constrained local fine-tuning is performed. This allows the neural network to quickly adapt to the real-world scenario while reducing the amount of computation.

[0140] In the embodiments of this application, performance is improved through neural network parameter adaptation. Specifically, projection data of the object to be imaged in a real-world application scenario or paired reconstructed image data is used to fine-tune computational units with poor generalization ability in the pre-trained neural network. During fine-tuning, a CT imaging model based on the actual projection data can be combined to model noise, scattering, etc., thereby improving the accuracy of the loss function. Furthermore, perceptual loss can be combined to further improve image quality. In addition, to fully retain the effective information from the pre-training process and achieve efficient fine-tuning, sparsity constraints on the parameter changes of the neural network can be introduced during the fine-tuning process.

[0141] Figure 6This is a flowchart of a scanned image reconstruction method for adapting incomplete data according to an embodiment of this application. Figure 7 The flowchart illustrating the application of the scanned image reconstruction method for adapting incomplete data according to embodiments of this application in an exemplary scenario is shown. For example, Figure 7 Exemplary scenarios could include applying a CT scanning imaging system with multiple straight-line trajectories to the scanning of biological tissues. (See reference...) Figure 6 and Figure 7 The scan image reconstruction method for adapting incomplete data according to the embodiments of this application may include steps S610 to S670.

[0142] In step S610, projection data can be acquired in response to ray scanning of the object to be imaged. For example, projection data can be acquired using... Figures 3A-3C Any of the provided scanning imaging systems can scan the object 120 to be imaged to obtain projection data. For example, it is possible to utilize... Figure 3C The CT scanning imaging system shown uses multiple straight-line trajectories to scan biological tissues, for example, in... Figure 7 In, for example, the complete projection data reconstruction results of the imaging object 120 of a biological tissue. The object to be imaged, 120, can be exemplarily characterized, for example, as a biological tissue. When utilizing... Figure 3C The CT scanning imaging system shown, employing multiple straight-line trajectories, can acquire projection data after scanning the object 120 to be imaged. It should be understood that this projection data This refers to incomplete projection data in the projection space.

[0143] For example, before or during step S610, data for acquiring the projection may also be obtained. The projection geometry and projection viewpoint distribution information of the scanning imaging system, for example... Figures 3A-3C The projection geometry information and projection viewing angle distribution information of any one of the provided scanning imaging systems are provided. For example, the projection geometry information may include, but is not limited to: the relative motion relationship between the X-ray source or detector and the object to be imaged, for example, it can be characterized by a circular trajectory, a spiral trajectory, a straight line trajectory, or a multi-segment straight line trajectory; the relative positional relationship between the X-ray source, detector, and object to be imaged, which can be characterized by relative angles or relative distances. For example, the projection viewing angle distribution information may include, but is not limited to: information used to characterize the scanning angle, such as finite angle scanning, sparse angle scanning, or a mixture of sparse and finite angle scanning; specific scanning angles, acquisition angle intervals, etc.

[0144] by Figure 3C Taking the multi-segment linear trajectory scanning imaging system shown as an example, step S610 is described exemplarily. Figure 3CAs shown, the angle of a distributed radiation source is The target coordinates within a distributed radiation source are: The fan angle of the rays emitted from the target is The radial distance from the origin O to a distributed ray source is For example, the angle of a distributed radiation source. The target coordinates within a distributed ray source can be represented by the angle of the line connecting the center of the distributed ray source and the origin O relative to the third direction y. This can be represented by the distance between the target point and the center of the distributed radiation source along the target point's orientation. The fan angle of the radiation emitted from the target point... This can be represented by the angle of the emitted ray relative to the main ray beam (or central ray beam) emitted from the target point. The radial distance from the origin O to a distributed ray source. It can be represented by the length of the line segment connecting the center of the distributed ray source and the origin O.

[0145] exist Figure 3C In the CT scanning imaging system with multiple straight-line trajectories shown, for any given ray, its projection geometry and projection angle distribution information can be obtained using... According to a specific scanning beam strategy, the projection space formed by each ray can be represented as: ,in, This represents the total number of target points included in a distributed radiation source. This indicates that The serial number of each target point Between 1~ between, Represents the i-th distributed ray source. The projection geometry and projection viewpoint distribution information corresponding to each target point This represents the projection space formed by each ray. Correspondingly, in Figure 3C In the CT scanning imaging system shown, by controlling the target points of each X-ray source to emit X-rays and controlling the detector to collect projection data through a specific scanning beam strategy, the projection data of the object to be imaged can be obtained. It should be understood that this projection data is incomplete data in the projection space.

[0146] In step S620, the incomplete projection data is transformed into first template data according to the mapping relationship between the projection space and the template data space. The first template data may include m1 first data points in the template data space, where m1 is a positive integer greater than 1.

[0147] For example, the template data space may include at least one of Radon space, Fourier space, and data space using orthogonal basis decomposition.

[0148] It should be noted that in CT image reconstruction, the Radon space is a mathematical space used to describe projection data. It is a space defined based on the Radon transform and used to connect projection data and the reconstructed image. For CT scanning imaging, the projection value received by the detector when the X-ray beam passes through the object being imaged is essentially the line integral of the attenuation coefficients of all materials along the X-ray propagation path. The Radon transform maps functions in the image space (the distribution of attenuation coefficients at points on a two-dimensional plane) to a set of line integral values ​​at different angles and positions in the projection space. Each element in the Radon space corresponds to a set of projection parameters and a corresponding projection value; for example, the projection parameters may include angle parameters. and position parameters Angular parameters The projection direction and position parameters used to characterize the ray. Used to characterize the positional offset of the projected line on a straight line perpendicular to the projection direction. In other words, the Radon space can be understood as a two-dimensional parameter space. In Radon space, each point corresponds to a unique integral value of the projection line. When dealing with incomplete CT reconstruction problems such as those with limited or sparse angles, data completion or optimization in Radon space can closely match the actual physical process of scanning, ensuring that the completed data meets the integral constraints of X-ray projection, thereby reducing artifacts in the reconstructed image.

[0149] Fourier space, also known as frequency space, is a mathematical space used to transform image or projection data from the spatial domain (spatial location domain) to the frequency domain. It is based on the center slice theorem of CT imaging, which states that the result of a two-dimensional Fourier transform in image space has a strict correspondence with the one-dimensional Fourier transform result of the projection data at each angle: after a one-dimensional Fourier transform of the projection data at each angle, it corresponds to a radial straight line passing through the origin in two-dimensional Fourier space. Each point in Fourier space represents not the spatial location information of the data, but rather its frequency characteristics: low-frequency components correspond to the overall outline and structure of the image, while high-frequency components correspond to the edges, details, and other fine information of the image. When processing incomplete projection data, the advantage of Fourier space lies in the natural smoothness of its frequency distribution. It can utilize the prior law of continuous low frequencies and sparse high frequencies to complete the frequency slices corresponding to missing angles, and then restore them to image space through inverse Fourier transform. This method can efficiently preserve the key structure of the image while suppressing reconstruction artifacts.

[0150] The data space decomposed using orthogonal basis decomposition is a space formed by decomposing the original data into a linear combination of a set of orthogonal basis functions. This is based on the orthogonality of the basis functions, which makes the data features simpler and easier to process. For example, orthogonal bases can include wavelet bases, cosine bases, etc. These basis functions are independent of each other and have no redundant information. Taking projection data as an example, when projection data is projected onto such an orthogonal basis, a set of basis coefficients is obtained, and the space composed of these basis coefficients is the orthogonal basis decomposition data space. In CT incomplete data reconstruction, sparsity constraints can be used to accurately reconstruct the distribution of basis coefficients from a small amount of projection data. Then, the image can be reconstructed through a linear combination of basis functions. This method can significantly reduce the requirement for the number of projection angles and is well-suited for scenarios with sparse angle scanning.

[0151] It should be understood that in incomplete scanning imaging scenarios such as those with limited or sparse angles, the characteristics of the projection space lead to poor results when directly performing completion within the projection space, resulting in severe artifacts in the reconstructed image. Specifically, in the projection space of scenarios such as CT scans, X-ray attenuation projection values ​​at different angles are arranged into a data space using a two-dimensional grid of "angle-detector channel". In this data space, the data correlation between pixel values ​​of adjacent angles and adjacent detector channels is weak, and completion lacks reliable constraints. For example, for limited-angle scans, there is no continuous gradient between the projection data of missing angle intervals and the projection data of existing angle intervals, and direct interpolation (such as linear interpolation) may lead to a complete deviation from the true projection distribution. Furthermore, for sparse-angle scans, the angle intervals are too large, and the sampling points of the projection data are sparse and discrete, making it impossible to capture the continuous changes in the angle dimension during completion. In addition, the completion error in the projection space is directly transmitted to the image space through reconstruction algorithms such as filtered backprojection and is amplified. For example, if incorrect projection values ​​are used to fill in missing angles in the projection space, the reconstructed image will show obvious stripe artifacts or angle artifacts, or even be completely distorted.

[0152] In the embodiments of this application, by converting incomplete data to template data spaces such as Radon space, Fourier space, and data space using orthogonal basis decomposition, the structured features of the data become more obvious, which is beneficial to improving the accuracy and stability of data completion.

[0153] The following description, using the template data space as an example, illustrates step S620. (Refer to the reference...) Figure 3C Projected spatial coordinates With Lardon spatial coordinates The mapping relationship between them can be expressed by the following formula: Based on the above equation, a data domain transformation operator from the projection space to the template data space can be defined. This maps data sampling points (i.e., projected data) within the projection space P to data points within the Radon space R. For example, this can be achieved through a data domain transformation operator. The above incomplete data Transformed into data in the Radon space, for ease of description, the data in the Radon space here is denoted as... It should be understood that the data in the Radon space... The data is incomplete and includes multiple (e.g., m1) first data points.

[0154] In step S630, incomplete data, for example, in the Radon space, are processed. The first template data is pre-completed to obtain the second template data, wherein the second template data may include m2 second data points in the template data space, where m2 is a positive integer greater than m1.

[0155] For example, incomplete projection data in the projection space is mapped to the template data space. After obtaining the incomplete data in the template data space, data pre-completion can be performed on the incomplete data in the template data space using conjugate projection information and interpolation to obtain the complete data in the template data space, i.e., the second template data. For example, incomplete projection data in the projection space can be... Mapping to the Radon space yields incomplete data within that space. Then, two-dimensional linear interpolation can be used to process incomplete data in the Radon space. Perform data pre-completion to obtain pre-completion data in the Radon space. This refers to the second template data. The pre-complete data in this Radon space. This includes multiple (e.g., m²) second data points. The second template data is obtained by pre-completing the first template data; it should be understood that it includes more data points than the first template data.

[0156] In step S640, the confidence information of m2 second data points in the second template data is obtained.

[0157] In the embodiments of this application, the second template data The data is generated after pre-completing the first template data in the template data space. Some of the m2 second data points are the original m1 first data points in the first template data, and the confidence of this part of the data points is relatively high. The other part of the m2 second data points are data points generated by interpolation and other processing of the m1 first data points. The confidence of this part of the data points is related to the distance between the newly generated data points and the original data points. Based on this, the discrete sampling point distribution of complete data sampling can be defined in the template data space. The confidence metric function q is constructed by the distance d between the original sampling points (i.e., the m1 first data points in the first template data) and the complete sampling points (i.e., the m2 second data points in the second template data) to evaluate the sampling confidence of the m2 second data points at different positions in the template data space. It should be understood that the confidence metric function q can satisfy the following basic characteristics: (1) when d=0, q=1; (2) the larger d is, the smaller q is; (3) the minimum value of q is 0. Therefore, the confidence information of the m2 second data points in the second template data can be obtained.

[0158] In some exemplary embodiments, the confidence information may include the position confidence of m2 second data points in the template data space.

[0159] For example, in step S640, obtaining the location confidence information of m2 second data points in the second template data may include: for the i-th second data point among the m2 second data points, calculating the minimum distance between the i-th second data point and each of the m1 first data points, where 1≤i≤m2; and calculating the location confidence based on the minimum distance.

[0160] For example, calculating the location confidence based on the minimum distance may include: calculating the location confidence based on the following weighted confidence function.

[0161] ,

[0162] in, The hyperparameter for adjusting the magnitude of the position confidence level, Let be the minimum distance between the i-th second data point and each of the m1 first data points. This is the weighted confidence function.

[0163] In some exemplary embodiments, the confidence information may further include the noise confidence of the line integral values ​​corresponding to m2 second data points.

[0164] For example, in step S640, obtaining the noise confidence information of m2 second data points in the second template data may include: for the i-th second data point among the m2 second data points, obtaining the line integral value corresponding to the i-th second data point. According to the line integral value The confidence function of the noise metric is defined as follows. Calculate the noise confidence level.

[0165] ,

[0166] in, The hyperparameter is used to adjust the magnitude of the noise confidence level.

[0167] For example, in step S640, obtaining the confidence information of the m2 second data points includes: according to the following confidence function Calculate the confidence information of the i-th second data point among the m2 second data points.

[0168] .

[0169] In some exemplary embodiments, a confidence information map can be generated based on the calculated confidence information (e.g., confidence values) of each data point.

[0170] Figure 8 An exemplary confidence information graph obtained according to the method of embodiments of this application is illustrated schematically. (Referring to...) Figure 7 and Figure 8 In the confidence infographic, the horizontal axis represents the location parameter. The vertical axis represents the angle parameter. , Figure 8 The color scale on the right side indicates that different colors represent different confidence levels. For example, yellow represents a higher confidence level, blue represents a lower confidence level, and green represents a middle confidence level. As the color transitions from blue to yellow, the confidence level gradually increases. In the coordinate system consisting of the horizontal and vertical axes, multiple arrays of dispersion values ​​are distributed. The different colors at each discrete point represent the corresponding confidence level. It should be noted that... Figure 7 In It can be understood as Figure 8 The confidence infographic shown is a scaled-down version. It should be noted that... Figure 7 and Figure 8 The confidence infographics in the graphs all correspond to confidence functions. In this application, for ease of description, the confidence infographic is also referred to as a confidence infographic. .

[0171] In step S650, the second template data and confidence information are used as input to the pre-trained data processing model. For example, as... Figure 7 As shown, the second template data can be... Confidence Infographic As input, it is fed into a pre-trained data processing model.

[0172] In some exemplary embodiments, reference is made to Figure 7 The data processing model may include a first sub-model 71, which is used to complete the template data in the template data space.

[0173] For example, in step S660, the input is processed using the first sub-model to obtain third template data, wherein the third template data includes m3 third data points in the template data space, where m3 is a positive integer greater than 1. For example, the second template data can be processed using the first sub-model. Confidence Infographic The dual-channel input yields the third template data. For example, m3 can be a positive integer greater than or equal to m2.

[0174] In the embodiments of this application, in step S630, incomplete data in the template data space is pre-completed using conjugate projection information and interpolation to obtain second template data. The completed data in the second template data may contain inaccuracies or incompleteness; therefore, the completion in step S630 is called pre-completeness. In step S650, the first sub-model in the pre-trained data processing model can further complete or correct the pre-completed second template data, thereby obtaining a third template data that is more complete and accurate than the second template data. In other words, through the completion processing of the first sub-model in the data processing model, the second template data can be further completed, thus improving the completeness of the pre-completed data; and the pre-completed results of the second template data can be corrected or adjusted, thus improving the accuracy of the pre-completed data.

[0175] In step S670, image reconstruction can be performed based on the third template data to obtain a reconstructed image.

[0176] In some exemplary embodiments, in step S670, image reconstruction can be performed directly based on the third template data to obtain a reconstructed image. That is, it is not necessary to transform the third template data in the Ladon space to the projection space; image reconstruction can be performed directly using the third template data in the Ladon space to obtain a reconstructed image.

[0177] In some other exemplary embodiments, step S670 may include: transforming the third template data into complete projection data in the projection space according to the mapping relationship between the projection space and the template data space; and performing image reconstruction based on the complete projection data to obtain a reconstructed image.

[0178] For example, in conjunction with reference Figure 3C and Figure 7 Taking the template data space as the Radon space as an example, the mapping relationship between the projection space and the Radon space can be expressed by the following formula: Based on this formula, a data domain transformation operator from the template data space to the projection space can be defined. Then, through the data domain transformation operator , the third template data Transformed into complete projection data in projection space .

[0179] For example, an analytical reconstruction operator can be used based on complete projection data. Image reconstruction is performed to obtain a reconstructed image. It should be understood that the process of reconstructing an image from projection data is essentially about using mathematical methods to inversely reconstruct the density distribution inside an object by taking the intensity attenuation information obtained from rays passing through it at different angles. This process can be understood as follows: first, the original ray intensity data is converted into integral values ​​of the material density, for example, using a filtered back-projection algorithm. The processed projection data is then mapped back to the image space along the ray direction, and the results from all angles are superimposed to obtain a clear reconstructed image. Alternatively, iterative reconstruction algorithms can be used. For example, starting from an initial image guess, an algebraic reconstruction algorithm or a synchronous iterative reconstruction algorithm is used to continuously compare the generated virtual projection with the actually acquired projection, gradually correcting the image until the two are sufficiently close.

[0180] Optional or additional, see reference Figure 7 The data processing model may further include a second sub-model 72, which is used to repair the detailed structures in the reconstructed image. That is, in step S670, image reconstruction based on complete projection data to obtain the reconstructed image may include: processing the complete projection data using an analytical reconstruction operator. The first reconstructed image is obtained by using an analytical reconstruction operator; the second sub-model 72 is then used to process the first reconstructed image to obtain a second reconstructed image. In this way, a preliminary reconstructed image can be obtained first using the analytical reconstruction operator, and then the detailed structures in the preliminary reconstructed image can be further repaired through the processing of the second sub-model to obtain the final reconstructed image. This can improve the quality of the reconstructed image.

[0181] In the embodiments of this application, in order to balance the data fidelity of the original projection and the "translation-invariant" characteristic of the template data space, and to facilitate the advantages of neural network learning, data completion can be performed first in the template data space, and then the completion result can be converted back to the original projection space. This improves the efficiency and accuracy of data completion while ensuring the fidelity of the original sampled projection. By optimizing the network architecture of the data processing model, including a backbone network structure and a confidence information guidance mechanism, the confidence information of the data sampling and the pre-completion result of the template data space can be fully utilized to achieve efficient completion of the projection data.

[0182] In embodiments of this application, the data processing model may include at least one of a deep neural network for data completion and a deep neural network for data completion and image reconstruction. Regarding the backbone network structure of the data processing model, network structures such as U-Net (U-shaped network) and Transformer (a neural network architecture based on self-attention mechanism) can be selected, and generative artificial intelligence methods (such as generative adversarial networks, diffusion models, etc.) can be combined to learn in a single data domain (projection domain, image domain, Fourier domain, etc.) or multiple data domains. For the confidence information guidance mechanism, a dual-channel input method of "pre-completed results of the template data space (i.e., the aforementioned second template data) + confidence information" can be adopted, combined with an attention mechanism, to enhance the deep neural network's ability to perceive data distribution. For example, in some exemplary embodiments, at least one backbone network structure of the first sub-model 71 and the second sub-model 72 of the data processing model may be selected to use a Transformer network structure.

[0183] Figure 9 The diagram schematically illustrates a backbone network structure using a Transformer, which may be employed by at least one of the first sub-model 71 and the second sub-model 72. This backbone network structure may be an example neural network system 700. The neural network system 700 may be an example of a system implemented as a computer program on one or more computers at one or more locations. The neural network system 700 may receive an input sequence 701 and process the input sequence 701 to transform it into an output sequence 702. The input sequence 701 has a respective network input at each of a plurality of input positions in input order, and the output sequence 702 has a respective network output at each of a plurality of output positions in output order. That is, the input sequence 701 has a plurality of inputs arranged in input order, and the output sequence 702 has a plurality of outputs arranged in output order. As described above, the neural network system 700 can perform any of a variety of tasks that require processing sequential inputs to generate sequential outputs.

[0184] The neural network system 700 may include an attention-based sequence transformation neural network, which may include an encoder neural network 710 and a decoder neural network 750.

[0185] An encoder neural network 710 is configured to receive an input sequence 701 and generate a respective encoded representation for each network input in the input sequence. Generally, the encoded representation is a vector or other ordered set of numerical values. A decoder neural network 750 is configured to use the encoded representations of the network inputs to generate an output sequence 702. It should be understood that both the encoder 710 and the decoder 750 are attention-based. In some cases, neither the encoder nor the decoder includes any convolutional layers or any recurrent layers. The encoder neural network 710 includes a sequence of embedding layers 720 and one or more encoder subnetworks 730. Exemplarily, the encoder neural network may include multiple encoder subnetworks 730. The embedding layer 720 is configured to, for each network input in the input sequence, map the network input to a numerical representation of the network input in the embedding space, for example, mapping it to a vector in the embedding space. The embedding layer 720 then provides the numerical representation of the network input to a first subnetwork in the sequence of encoder subnetworks 730, i.e., to the first encoder subnetwork 730 out of N encoder subnetworks 730.

[0186] For example, embedding layer 720 is configured to map each network input to an embedding representation of the network input, and then combine the embedding representation of the network input with the positional embeddings of the input positions of the network inputs in the input order to generate a combined embedding representation of the network inputs. That is, each position in the input sequence has a corresponding embedding, and for each network input, embedding layer 720 combines the embedding representation of the network inputs in the input sequence with the positional embeddings of the network inputs. This positional embedding allows the model to fully utilize the order of the input sequence without relying on recursion or convolution.

[0187] Each encoder subnet in encoder subnet 730 is configured to receive a respective encoder subnet input for each of a plurality of input positions and generate a respective subnet output for each of the plurality of input positions. The encoder subnet output generated by the last encoder subnet in the sequence is then used as the encoded representation of the network input. For the first encoder subnet in the sequence, the encoder subnet input is the numerical representation generated by the embedding layer 720, and for each encoder subnet in the sequence other than the first encoder subnet, the encoder subnet input is the encoder subnet output of the preceding encoder subnet in the sequence.

[0188] Each encoder subnet 730 includes an encoder self-attention sublayer 732. The encoder self-attention sublayer 732 is configured to receive a subnet input at each of a plurality of input positions, and for each specific input position in the order of input, apply an attention mechanism to the encoder subnet input at that input position using one or more queries derived from the encoder subnet input at that specific input position to generate a respective output for that specific input position. In some cases, the attention mechanism is a multi-head attention mechanism.

[0189] In some implementations, each of the encoder subnets 730 further includes a residual connection layer and a layer normalization layer. The residual connection layer combines the output of the encoder self-attention sublayer with the input of the encoder self-attention sublayer to generate an encoder self-attention residual output. The layer normalization layer applies layer normalization to the encoder self-attention residual output. These two layers... Figure 9 These are collectively referred to as "addition and normalization" operations.

[0190] The encoder subnet may also include a feedforward layer, which is configured to operate separately at each position in the input sequence.

[0191] Once the encoder neural network 710 has generated the encoded representation, the decoder neural network 750 is configured to generate the output sequence in an autoregressive manner. That is, the decoder neural network 750 generates the output sequence by generating the network output at the corresponding output position at each of the multiple generation time steps, conditioned on (i) the encoded representation and (ii) the network output at the output position preceding the output position in the output order.

[0192] Specifically, for a given output position, the decoder neural network generates an output that defines a probability distribution over the possible network outputs at that given output position. The decoder neural network can then select a network output for the output position either by sampling from the probability distribution or by selecting the network output with the highest probability.

[0193] Because the decoder neural network 750 is autoregressive, at each generation time step, the decoder 750 operates on the network outputs that have been generated prior to the generation time step; that is, it operates on the network outputs at the output positions preceding the corresponding output positions in the output order. In some implementations, to ensure this is true during both inference and training, at each generation time step, the decoder neural network 750 shifts the already generated network outputs to the right by one output order position (i.e., introduces a position offset into the sequence of already generated network outputs) and masks certain operations so that the position only concerns positions in the output sequence up to and including that position. While the remainder of the description below describes how the various components of the decoder 750 operate on data at output positions preceding the given output position when generating a given output at a given output position, it should be understood that this type of adjustment can be efficiently implemented using the shift described above.

[0194] The decoder neural network 750 may include an embedding layer 760, a sequence of decoder subnets 770, a linear layer 780, and a softmax layer 790. For example, the decoder neural network may include multiple decoder subnets 770.

[0195] The embedding layer 760 is configured to, at each generation time step, map the network output to a numerical representation of the network output in the embedding space for each network output at a position preceding the current output position in the output order. The embedding layer 760 then provides the numerical representation of the network output to a first subnet 770 in the sequence of decoder subnets, i.e., to a first decoder subnet 770 among a plurality of decoder subnets.

[0196] In some implementations, embedding layer 760 is configured to map each network output to an embedded representation of the network output, and to combine the embedded representations of the network outputs with the positional embeddings of the output locations of the network outputs in the output order to generate a combined embedded representation of the network outputs. This combined embedded representation is then used as a numerical representation of the network outputs. Embedding layer 760 generates the combined embedded representation in the same manner as described above with reference to embedding layer 720.

[0197] Each decoder subnet 770 is configured to: at each generation time step, receive the respective decoder subnet input for each of the plurality of output positions preceding the corresponding output position, and generate the respective decoder subnet output for each of the plurality of output positions preceding the corresponding output position (or equivalently, when the output sequence has been shifted to the right, each network output is at a position up to and including the current output position).

[0198] Specifically, each decoder subnet 770 may include two different attention sublayers: a decoder self-attention sublayer (i.e., a multi-head attention layer for the mask) 772 and an encoder-decoder attention sublayer (i.e., a multi-head attention layer) 774.

[0199] The decoder self-attention sublayer 772 is configured to, at each generation time step, receive input from each output position preceding the corresponding output position, and for each specific output position, apply an attention mechanism to the input at the output positions preceding the corresponding position using one or more queries derived from the input at the specific output position, to generate an updated representation for the specific output position. In other words, the decoder self-attention sublayer 772 applies a masked attention mechanism so that it does not focus on or process any data not preceding the current output position in the output sequence.

[0200] The encoder-decoder attention sublayer 774 is configured to receive inputs for each output position preceding the corresponding output position at each generation time step, and for each output position, apply an attention mechanism to the encoded representation at the input position using one or more queries derived from the input of the output position to generate an updated representation of the output position. Therefore, the encoder-decoder attention sublayer 774 applies attention to the encoded representation, while the encoder-decoder attention sublayer 772 applies attention to the input at the output position.

[0201] Figure 10 The illustration schematically depicts a confidence information graph obtained by the method according to an embodiment of this application in an exemplary application scenario. Figure 11 An image is schematically shown in an exemplary application scenario, wherein (a) is a ground truth image of the object to be imaged, (b) is a reconstructed image obtained using a method in the related art, and (c) is a reconstructed image obtained using a method according to an embodiment of this application.

[0202] Figure 12 The illustration schematically shows a confidence information graph obtained by the method according to an embodiment of this application in another exemplary application scenario. Figure 13 The illustration shows an image in another exemplary application scenario, where (a) is a ground truth image of the object to be imaged, (b) is a reconstructed image obtained using a method in the related art, and (c) is a reconstructed image obtained using a method according to an embodiment of this application.

[0203] It should be noted that, in Figure 10 and Figure 12 In this example, for the sake of brevity, the coordinate system and color scale have been omitted to show a more concise and exemplary confidence infographic. Figure 10 and Figure 12 The coordinate system and color scale in the image can be referenced. Figure 8It should also be noted that, Figures 10 to 13 The exemplary application scenario targeted can be a scenario covered by the training data.

[0204] In this exemplary application scenario, reconstructed images can be obtained using methods from related technologies and methods according to embodiments of this application, respectively. For example, in the process of obtaining reconstructed images using the method according to embodiments of this application, in step S650, as follows... Figure 7 As shown, the second template data can be... and Figure 10 or Figure 12 The confidence information map shown is used as a dual-channel input to a pre-trained data processing model. The data processing model may include a first sub-model 71, which is used to complete the template data in the template data space. In step S660, the dual-channel input is processed using the first sub-model to obtain third template data, wherein the third template data includes m3 third data points in the template data space, where m3 is a positive integer greater than or equal to m2. In step S670, the third template data can be transformed into complete projection data in the projection space according to the mapping relationship between the projection space and the template data space. In step S670, the complete projection data is processed using an analytical reconstruction operator. To obtain a first reconstructed image; to process the first reconstructed image using a second sub-model 72 to obtain a second reconstructed image, for example... Figure 11 The image shown in (c) and Figure 13 The image shown in (c) is shown in the middle.

[0205] By comparison Figure 11 , Figure 13 From (a), (b), and (c), we can see that... Figure 11 The reconstructed image shown in (c) is larger than Figure 11 The reconstructed image shown in (b) is closer to... Figure 11 The truth image shown in (a) is as follows: Figure 13 The reconstructed image shown in (c) is larger than Figure 13 The reconstructed image shown in (b) is closer to... Figure 13 The truth image shown in (a) is an example. Figure 11 , Figure 13 In the truth image shown in (a), regions A1 and B1 are exemplarily listed, with region A2 in the upper right corner being a magnified view of region A1, and region B2 in the lower right corner being a magnified view of region B1. Figure 11 , Figure 13 In the reconstructed image shown in (b) using methods from related techniques, Figure 11 , Figure 13 Region A2 in (b) and Figure 11 , Figure 13 In (a), the pixels in region A2 are clearly inconsistent. Figure 11 , Figure 13 Region B2 in (b) and Figure 11 , Figure 13 In (a), multiple pixels in region B2 are clearly inconsistent. Figure 11 , Figure 13 In the reconstructed image shown in (c) obtained using the method according to the embodiments of this application, compared to Figure 11 , Figure 13 Region A2 in (b) Figure 11 , Figure 13 Region A2 in (c) and Figure 11 , Figure 13 In (a), the consistency of multiple pixels in region A2 is higher; compared to Figure 11 , Figure 13 Region B2 in (b) Figure 11 , Figure 13 Region B2 in (c) and Figure 11 , Figure 13 In (a), the consistency of multiple pixels in region B2 is higher. That is to say, in application scenarios where the training data is covered, the reconstructed image obtained by the method according to the embodiments of this application is of higher quality than the traditional image reconstruction method for incomplete data.

[0206] Figure 14 The illustration shows an image in yet another exemplary application scenario, where (a) is a ground truth image of the object to be imaged, (b) is a reconstructed image obtained using a method in the related art, and (c) is a reconstructed image obtained using a method according to an embodiment of this application. Figure 15 The illustration shows an image in another exemplary application scenario, where (a) is a ground truth image of the object to be imaged, (b) is a reconstructed image obtained using a method in the related art, and (c) is a reconstructed image obtained using a method according to an embodiment of this application.

[0207] It should be noted that, Figures 14 to 15 The exemplary application scenario targeted can be a scenario not covered by training data.

[0208] By comparison Figure 14 , Figure 15 From (a), (b), and (c), we can see that... Figure 14 The reconstructed image shown in (c) is larger than Figure 14 The reconstructed image shown in (b) is closer to... Figure 14 The truth image shown in (a) is as follows: Figure 15 The reconstructed image shown in (c) is larger than Figure 15 The reconstructed image shown in (b) is closer to... Figure 15 The truth image shown in (a) is an example. Figure 14 , Figure 15 In the truth image shown in (a), regions C1 and D1 are exemplarily illustrated. Figure 14 , Figure 15 In the reconstructed image shown in (b) using methods from related techniques, Figure 14 Region C1 in (b) and Figure 14 In (a), the pixels in region C1 are clearly inconsistent. Figure 15 Region D1 in (b) and Figure 15 In (a), multiple pixels in region D1 are clearly inconsistent. Figure 14 , Figure 15 In the reconstructed image shown in (c) obtained using the method according to the embodiments of this application, compared to Figure 14 Region C1 in (b) Figure 14 Region C1 in (c) and Figure 14 In (a), the consistency of multiple pixels in region C1 is higher; compared to Figure 15 Region D1 in (b) Figure 15 Region D1 in (c) and Figure 15 In (a), the consistency of multiple pixels in region D1 is higher. That is to say, in application scenarios where the training data is not covered, the reconstructed image obtained by the method according to the embodiments of this application is of higher quality than the traditional image reconstruction method for incomplete data.

[0209] In combination with the above Figures 10 to 15 As can be seen, the method according to the embodiments of this application demonstrates superior reconstruction performance in both application scenarios where the training data is covered and application scenarios where the training data is not covered. In particular, for application scenarios where the training data is not covered, the reconstructed images generated by traditional methods exhibit severe artifacts and loss of detail, while the reconstructed images generated by the method according to the embodiments of this application maintain good reconstruction performance, indicating that it has better robustness and can adapt to various different data sampling distributions.

[0210] It should be noted that the numbering of each step in the above method is not a restriction on the order of the method. In the absence of conflict, the steps in the method can be executed in parallel or in a different order than that described in this application.

[0211] Figure 16 A block diagram of an electronic device suitable for implementing the methods described above, according to an embodiment of this application, is illustrated schematically. Figure 16The illustrated electronic device is merely an example and should not be construed as limiting the functionality or scope of use of the embodiments of this application. For example, either the data processing device 130 or the control device 140 may be implemented as the described electronic device. In embodiments of this application, the data processing device 130 and the control device 140 may be implemented separately; for example, the data processing device 130 may be implemented as... Figure 16 The illustrated electronic device, control unit 140, can be implemented as a display device with a display screen. In other embodiments, data processing unit 130 and control unit 140 can be implemented as an integrated device, that is, they can be unified into a single electronic device. For example, data processing unit 130 and control unit 140 can be implemented as... Figure 16 The illustrated electronic device is in the form of a display screen. The embodiments of this application do not impose particular limitations on the specific implementation of the data processing device 130 and the control device 140.

[0212] like Figure 16 As shown, an electronic device 1600 according to an embodiment of this application includes a processor 1601, which can perform various appropriate actions and processes according to a program stored in a read-only memory 1602 or a program loaded from a storage portion 1608 into a random access memory 1603. The processor 1601 may include, for example, a general-purpose microprocessor, an instruction set processor and / or an associated chipset and / or a dedicated microprocessor. The processor 1601 may also include onboard memory for caching purposes. The processor 1601 may include a single processing unit or multiple processing units for executing different steps of the method flow according to an embodiment of this application.

[0213] Random access memory 1603 stores various programs and data required for the operation of electronic device 1600. Processor 1601, read-only memory 1602, and random access memory 1603 are interconnected via bus 1604. Processor 1601 executes various steps of the method flow according to embodiments of this application by executing programs in read-only memory 1602 and / or random access memory 1603. It should be noted that the programs may also be stored in one or more memories other than read-only memory 1602 and random access memory 1603. Processor 1601 may also execute various steps of the method flow according to embodiments of this application by executing programs stored in said one or more memories.

[0214] According to embodiments of this application, the electronic device 1600 may further include an input / output interface 1605, which is also connected to a bus 1604. The electronic device 1600 may also include one or more of the following components connected to the input / output interface 1605: an input section 1606 including a keyboard, mouse, etc.; an output section 1607 including a cathode ray tube, liquid crystal display, etc., and a speaker, etc.; a storage section 1608 including a hard disk, etc.; and a communication section 1609 including a network interface card, such as a local area network card, modem, etc. The communication section 1609 performs communication processing via a network such as the Internet. A drive 1610 is also connected to the input / output interface 1605 as needed. A removable medium 1611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 1610 as needed so that computer programs read from it can be installed into the storage section 1608 as needed.

[0215] Embodiments of this application also provide a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of this application.

[0216] According to embodiments of this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including but not limited to: portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination thereof. In embodiments of this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this application, the computer-readable storage medium may include the read-only memory 1602, and / or random access memory 1603, and / or one or more memories other than read-only memory 1602 and random access memory 1603 described above.

[0217] Embodiments of this application also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the methods provided in the embodiments of this application.

[0218] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 1609, and / or installed from a removable medium 1611. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0219] In embodiments of this application, the computer program can be downloaded and installed from a network via communication section 1609, and / or installed from removable medium 1611. When the computer program is executed by processor 1601, it performs the functions defined in the system of embodiments of this application. According to embodiments of this application, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0220] According to embodiments of this application, program code for executing the computer programs provided in the embodiments of this application can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0221] The embodiments of this application have been described above. However, these embodiments are merely illustrative and not intended to limit the scope of this application. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Without departing from the scope of this application, those skilled in the art can make various substitutions and modifications, all of which should fall within the scope of this application.

Claims

1. A method for reconstructing scanned images using incomplete data, characterized in that, The method includes: In response to a ray scan of an object to be imaged, projection data of the object to be imaged is acquired, wherein the projection data is incomplete projection data in the projection space; Based on the mapping relationship between the projection space and the template data space, the incomplete projection data is transformed into first template data, wherein the first template data includes m1 first data points in the template data space, where m1 is a positive integer greater than 1, and the template data space includes at least one of Radon space, Fourier space and data space using orthogonal basis decomposition. The first template data is pre-completed to obtain the second template data, wherein the second template data includes m2 second data points in the template data space, where m2 is a positive integer greater than m1; Obtain the confidence information of the m2 second data points; The second template data and the confidence information are used as inputs to a pre-trained data processing model, wherein the data processing model includes a first sub-model, which is used to complete the second template data; The input is processed using the first sub-model to obtain third template data, wherein the third template data includes m3 third data points in the template data space, where m3 is a positive integer greater than 1; Image reconstruction is performed based on the third template data to obtain a reconstructed image.

2. The method according to claim 1, characterized in that, The step of reconstructing the image based on the third template data to obtain the reconstructed image includes: The image is reconstructed directly based on the third template data to obtain the reconstructed image.

3. The method according to claim 1, characterized in that, The step of reconstructing the image based on the third template data to obtain the reconstructed image includes: Based on the mapping relationship between the projection space and the template data space, the third template data is transformed into complete projection data in the projection space; image reconstruction is performed based on the complete projection data to obtain a reconstructed image.

4. The method according to claim 3, characterized in that, The data processing model also includes a second sub-model, which is used to repair and reconstruct the detailed structures in the image. Image reconstruction based on the complete projection data to obtain a reconstructed image includes: processing the complete projection data using an analytical reconstruction operator to obtain a first reconstructed image; and processing the first reconstructed image using a second sub-model to obtain a second reconstructed image.

5. The method according to claim 3 or 4, characterized in that, The confidence information includes the position confidence of the m2 second data points in the template data space.

6. The method according to claim 5, characterized in that, Obtaining the location confidence includes: For the i-th second data point among the m2 second data points, calculate the minimum distance between the i-th second data point and each of the m1 first data points, where 1≤i≤m2; The location confidence is calculated based on the minimum distance.

7. The method according to claim 6, characterized in that, The step of calculating the location confidence score based on the minimum distance includes: calculating the location confidence score according to the following weighted confidence function. ， in, The hyperparameter for adjusting the magnitude of the position confidence level, Let be the minimum distance between the i-th second data point and each of the m1 first data points. This is the weighted confidence function.

8. The method according to claim 7, characterized in that, The confidence information also includes the noise confidence of the line integral values ​​corresponding to the m2 second data points.

9. The method according to claim 8, characterized in that, Obtaining the noise confidence level includes: For the i-th second data point among the m2 second data points, obtain the line integral value corresponding to the i-th second data point. ; According to the line integral value The confidence function of the noise metric is defined as follows. Calculate the noise confidence level. ， in, The hyperparameter is used to adjust the magnitude of the noise confidence level.

10. The method according to claim 9, characterized in that, The step of obtaining the confidence information of the m2 second data points includes: according to the following confidence function. Calculate the confidence information of the i-th second data point among the m2 second data points. 。 11. The method according to any one of claims 1-4 and 6-10, characterized in that, The data processing model includes multiple parameters; The method further includes: Based on a predefined loss function, the gradient value of the loss function with respect to each parameter is calculated; In response to a gradient value of at least one parameter being greater than a predefined gradient threshold, the at least one parameter is fine-tuned.

12. The method according to claim 11, characterized in that, The fine-tuning of the at least one parameter includes: The at least one parameter is fine-tuned using the loss function, prior constraints on the reconstructed image, and constraints on the fine-tuning amount of the at least one parameter.

13. The method according to any one of claims 1-4, 6-10 and 12, characterized in that, The method further includes: constructing a mapping relationship between the projection space and the template data space based on the projection geometry of the ray scanning system used for ray scanning and the projection angle distribution of the rays.

14. The method according to any one of claims 1-4, 6-10 and 12, characterized in that, The pre-completion of the first template data includes: using at least one of conjugate projection information and interpolation algorithm to pre-complete the first template data.

15. The method according to any one of claims 1-4, 6-10 and 12, characterized in that, The incomplete projection data includes projection data generated by at least one of sparse angle scanning and finite angle scanning.

16. The method according to claim 3, characterized in that, Reconstructing an image from the complete projection data to obtain a reconstructed image includes: processing the complete projection data using an analytical reconstruction operator to obtain the reconstructed image.

17. The method according to claim 1, characterized in that, m3 is greater than or equal to m2.

18. The method according to any one of claims 1-4, 6-10 and 12, characterized in that, The objects to be imaged include items to be inspected in security inspection scenarios.

19. A scanning imaging system, characterized in that, The system includes: An imaging channel, at least a portion of which extends along a first direction, the imaging channel being used to place an object to be imaged during the imaging process; A radiation source, wherein the radiation source is used to emit radiation; A detector is used to detect rays emitted from the ray source and passing through the object to be imaged, and to generate projection data corresponding to the detected rays, wherein the projection data is incomplete projection data in the projection space; A data processing apparatus for generating a reconstructed image based on the projection data using the method described in any one of claims 1-18.

20. The system according to claim 19, characterized in that, The scanning imaging system includes a CT scanning imaging system.

Images