Method, device and electronic equipment for identifying tiny lesions based on super-resolution technology

Abstract

The present application relates to a method, device and electronic device for identifying tiny lesions based on super-resolution technology. The method comprises: acquiring at least one first image of a target object; the first image includes the lesion area of ​​the target object; reconstructing each first image through an image reconstruction model to obtain a second image; wherein the image reconstruction model is obtained by updating the observation model by the regularization optimization term of the second image; the regularization optimization term is used to indicate the regularization of the second image in different dimensions of the same space; the resolution of the first image is lower than the resolution of the second image. By adopting this method, the image details of the second image in different dimensions can be captured and supplemented by regularizing the second image in different dimensions of the same space, so that the second image can clearly display the tiny structure and / or boundary information in the lesion area of ​​the target object, improve the clarity and detail accuracy of the second image, and improve the image reconstruction effect.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method and device for identifying tiny lesions based on super-resolution technology. Background Art

[0002] Super-resolution (SR) is an image processing technique designed to recover the detailed information of high-quality, high-resolution images from observed low-quality, low-resolution images. This technique can significantly improve image quality and has important applications in fields such as high-definition television, medical imaging, and remote sensing satellite imaging.

[0003] In the field of medical imaging, accurately identifying and quantitatively analyzing the structure of tiny lesions is crucial for diagnosis and treatment. Traditional super-resolution reconstruction techniques, including those based on reconstruction and / or deep learning, ignore high-frequency information such as edges and textures due to the physical limitations of imaging equipment and the influence and limitations of various factors during the scanning process. This results in insufficient image detail, making it difficult to clearly display the structure and / or edge information of tiny lesions.

[0004] Therefore, how to accurately model the blur in the image acquisition process so as to clearly display the structure and / or boundaries of tiny lesions is an urgent problem that needs to be solved. Summary of the Invention

[0005] Based on this, it is necessary to provide a method, device and electronic equipment for identifying tiny lesions based on super-resolution technology to address the above technical problems.

[0006] In a first aspect, the present application provides a method for identifying micro-lesions based on super-resolution technology, the method comprising:

[0007] Acquire at least one first image of a target object; the first image includes a lesion area of ​​the target object;

[0008] reconstructing each of the first images using an image reconstruction model to obtain a second image;

[0009] The image reconstruction model is obtained by updating the observation model by the regularized optimization term of the second image; the regularized optimization term is used to indicate the regularization of the second image in different dimensions of the same space; and the resolution of the first image is lower than that of the second image.

[0010] In one embodiment, the process of establishing the image reconstruction model includes:

[0011] The observation model is established according to the downsampling parameters, the blur parameters, and the geometric transformation parameters; wherein the observation model is used to indicate the mapping relationship between the first image and the second image; the expression of the observation model is as follows:

[0012] ;

[0013] in, used to indicate the total number of the first images; for indicating the first image; Used to indicate the downsampling operator from high-resolution space to low-resolution space; Used to indicate fuzzy operators; Used to indicate geometric transformation operators; used to indicate the second image;

[0014] Regularizing the X-axis dimension, the Y-axis dimension, and the Z-axis dimension of the second image according to the Manhattan norm to obtain the regularized optimization term;

[0015] According to the regular optimization term, the observation model is updated to obtain the image reconstruction model; wherein the expression of the regular optimization term is as follows:

[0016] ;

[0017] in, Used to indicate the regularization function; Used to indicate the balance impact factor; A vector indicating the second image in the X-axis dimension; A vector indicating the second image in the Y-axis dimension; A vector indicating the second image in the Z-axis dimension; The expression is as follows:

[0018] ;

[0019] in, To indicate calculation The gradient on .

[0020] In one embodiment, the method of determining the fuzzy parameter includes:

[0021] Using Gaussian distribution to establish point spread functions of the first image in the X-axis dimension, the Y-axis dimension, and the Z-axis dimension;

[0022] The blur parameter is determined according to the point spread function; wherein the expression of the point spread function is as follows:

[0023] ;

[0024] in, Used to indicate the coordinates of the first image in the n-axis dimension.

[0025] In one embodiment, the image reconstruction model is expressed as follows:

[0026] ;

[0027] in, for indicating the first image; A vector indicating the first image; Used to indicate the balance impact factor; A vector indicating the second image in the X-axis dimension; A vector indicating the second image in the Y-axis dimension; A vector indicating the second image in the Z-axis dimension.

[0028] In one embodiment, reconstructing the first image using the image reconstruction model to obtain the second image includes:

[0029] determining a mean image of each of the first images as an initial image;

[0030] Obtaining an initial residual and an initial iteration direction according to the initial image and an expression of the image reconstruction model;

[0031] The initial residual is iteratively solved according to the initial iteration direction using a conjugate gradient descent algorithm until the residual meets a preset requirement, thereby obtaining the second image.

[0032] In one embodiment, the iteratively solving the expression of the image reconstruction model using the conjugate gradient descent algorithm and the initial image until the residual meets a preset requirement to obtain the second image includes:

[0033] In the i-th iteration process, the i-th iteration direction is obtained according to the second candidate image obtained after the i-1-th solution; wherein i is a positive integer; i is greater than 1;

[0034] Perform iterative solution based on the second candidate image after the i-1th solution and the i-th iteration direction to obtain the second candidate image after the i-th solution;

[0035] Obtaining an i-th residual according to the second candidate image obtained after the i-th solution and an expression of the image reconstruction model;

[0036] If the i-th residual is less than or equal to a preset threshold, the second candidate image obtained after the i-th solution is determined to be the second image.

[0037] In one embodiment, the method further comprises:

[0038] Performing three-dimensional reconstruction on the first image and the second image respectively to obtain a first three-dimensional image before super-resolution reconstruction and a second three-dimensional image after super-resolution reconstruction;

[0039] Obtaining a differential three-dimensional image by subtracting the second three-dimensional image from the first three-dimensional image; wherein the differential three-dimensional image includes at least one difference voxel;

[0040] Determine the region corresponding to the difference voxels that is greater than or equal to a preset threshold as a suspicious region and filter out a target region from the suspicious region;

[0041] Re-constructing the target area in three dimensions to obtain a target reconstructed area;

[0042] The target reconstructed region is superimposed on the second three-dimensional image to obtain a target high-resolution image.

[0043] In a second aspect, the present application further provides a device for identifying micro-lesions based on super-resolution technology, the device comprising:

[0044] An acquisition module, configured to acquire at least one first image of a target object; the first image includes a lesion area of ​​the target object;

[0045] a processing module, configured to reconstruct each of the first images using an image reconstruction model to obtain a second image;

[0046] The image reconstruction model is obtained by updating the observation model by the regularized optimization term of the second image; the regularized optimization term is used to indicate the regularization of the second image in different dimensions of the same space; and the resolution of the first image is lower than that of the second image.

[0047] In a third aspect, the present application further provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method for identifying small lesions based on super-resolution technology described in any embodiment of the present application is implemented.

[0048] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for identifying small lesions based on super-resolution technology described in any embodiment of the present application.

[0049] The above-mentioned method, device, and electronic device for identifying tiny lesions based on super-resolution technology obtain a reconstructed high-resolution second image by inputting at least one first image containing the lesion area of ​​the target object into an image reconstruction model for reconstruction. In this way, the method for identifying tiny lesions based on super-resolution technology of the present application can capture and supplement the image details of the second image in different dimensions by regularizing the second image in different dimensions of the same space, so that the second image can clearly display the tiny structure and / or boundary information in the lesion area of ​​the target object, improve the clarity and detail accuracy of the second image, and improve the image reconstruction effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is a diagram showing an application environment of a method for identifying micro-lesions based on super-resolution technology according to an exemplary embodiment;

[0051] Figure 2 is a flow chart of a method for identifying micro-lesions based on super-resolution technology according to an exemplary embodiment;

[0052] Figure 3 is a schematic diagram showing a first three-dimensional image according to an exemplary embodiment;

[0053] Figure 4 is a schematic diagram showing a second three-dimensional image according to an exemplary embodiment;

[0054] Figure 5 is a schematic diagram showing a suspicious area according to an exemplary embodiment;

[0055] Figure 6 is a schematic diagram showing a target reconstruction area according to an exemplary embodiment;

[0056] Figure 7 is a schematic diagram showing a target high-resolution image according to an exemplary embodiment;

[0057] Figure 8A is a schematic diagram of a cross-section before reconstruction according to an exemplary embodiment;

[0058] Figure 8B is a schematic diagram of a reconstructed cross section according to an exemplary embodiment;

[0059] Figure 9 is a flow chart of a method for identifying micro-lesions based on super-resolution technology according to an exemplary embodiment;

[0060] Figure 10 is a structural block diagram of a device for identifying micro-lesions based on super-resolution technology according to an exemplary embodiment;

[0061] Figure 11 It is a diagram showing the internal structure of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0063] The terms "first", "second" and "third" in the embodiments of the present application are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first", "second" and "third" may explicitly or implicitly include at least one of the features. In the description of the present application, "at least one" is used to indicate one or more; "plurality" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally also include steps or units that are not listed, or may optionally also include other steps or units inherent to these processes, methods, products or devices.

[0064] References to "embodiments" herein mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0065] like Figure 1 As shown, this method is applied to Figure 1 The electronic device in the embodiment is used as an example for explanation. The method described in the embodiments of the present application is applied to electronic devices and / or applications located in electronic devices; the electronic devices can be any mobile terminals or fixed terminals. The terminal can be a device that provides voice and / or data connectivity to the user. Exemplarily, the terminal can be an Internet of Things terminal, such as a sensor device, a mobile phone or a so-called "cellular" phone and a computer with an Internet of Things terminal, for example, a fixed, portable, pocket-sized, handheld, or computer-built-in device. The application can be, but is not limited to, various software programs, quick applications and / or applets.

[0066] In one embodiment, Figure 2As shown, a method for identifying micro-lesions based on super-resolution technology is provided, comprising the following steps:

[0067] S201 , acquiring at least one first image of a target object; the first image includes a lesion area of ​​the target object.

[0068] In this embodiment, the target object may be any object with life characteristics. Optionally, the target object may include but is not limited to at least one of humans, animals, and plants.

[0069] In this embodiment, the lesion refers to a tissue or part of the target subject's body where a lesion occurs.

[0070] In some embodiments, the target object is a human body, and the lesion may be located in organs or tissues such as the prostate, kidney, heart, and lung.

[0071] In this embodiment, the first image may be a low-resolution image in any medical image format.

[0072] Optionally, the medical image format may include, but is not limited to, at least one of the Digital Imaging and Communications in Medicine (DICOM) format, the Neuroimaging Informatics Technology Initiative (NIFTI), the Mayo Medical Imaging (Analyze) format, and the Nearly Raw Raster Data (NRRD) format.

[0073] In some embodiments, acquiring at least one first image of the target object includes at least one of the following:

[0074] In response to an image acquisition operation, acquiring the first image of the target object through a plurality of acquisition viewing angles;

[0075] In response to an image acquisition operation, the first image of the target object is acquired through a plurality of acquisition modes.

[0076] Optionally, the acquisition perspective may include but is not limited to at least one of a level perspective, a top perspective, an upward perspective, a side perspective, and an oblique side perspective. For example, the acquisition perspective may be a level perspective and tilted 20° to the right.

[0077] In some embodiments, the electronic device acquires multiple first images of the lesion area containing the target object through acquisition perspectives such as the transverse plane, the coronal plane, the sagittal plane, and the oblique plane.

[0078] In the embodiment of the present application, the acquisition method may include but is not limited to at least one of a spin echo (SE) sequence, a gradient echo (GRE) sequence, a parallel acquisition technique (PAT), a rapid imaging sequence acquisition, and a diffusion weighted imaging technique (DWI).

[0079] S202, reconstructing each of the first images using an image reconstruction model to obtain a second image;

[0080] The image reconstruction model is obtained by updating the observation model by the regularized optimization term of the second image; the regularized optimization term is used to indicate the regularization of the second image in different dimensions of the same space; and the resolution of the first image is lower than that of the second image.

[0081] In this embodiment, the regularization optimization term is used to instruct the expansion of the image variable parameters in the regularization term to different dimensions in the three-dimensional space. The regularization optimization term can simulate supplementary information of image details of the second image in different dimensions during the image reconstruction process, thereby achieving denoising and reconstruction effects of different granularities.

[0082] In this embodiment, the observation model can indicate the sampling model used in the imaging process. The observation model is used to model the sampled signals to address the problem of how to better reconstruct images in situations such as incomplete information and noise interference. The observation model can be used to model and describe the relationship between the sampled signals and the image, thereby mathematically solving a reconstruction result that is closer to the actual image. For example, the observation model can indicate the sampling model used in magnetic resonance imaging.

[0083] In this embodiment, the first image is used to indicate a low-resolution image before reconstruction, and the second image is used to indicate a high-resolution image after reconstruction.

[0084] In this embodiment, the dimension can be used to indicate different establishment directions of the coordinate system in the same space. For example, the dimension can be an X-axis dimension, a Y-axis dimension, and / or a Z-axis dimension.

[0085] In this embodiment, regularized optimization terms in the image reconstruction model can be used to simulate supplementary information of image details in different dimensions of a high-resolution image during the image reconstruction process, such as information about the structure of tiny lesions. Compared to traditional methods for identifying tiny lesions based on super-resolution technology, this application uses an image reconstruction model to directly reconstruct a high-resolution image that displays both detailed image information and supplementary information, improving image accuracy and providing clearer and more detailed information for clinical diagnosis and treatment.

[0086] In the above-mentioned method for identifying tiny lesions based on super-resolution technology, at least one first image containing the lesion area of ​​the target object is input into the image reconstruction model for reconstruction, thereby obtaining a reconstructed high-resolution second image. In this way, the method for identifying tiny lesions based on super-resolution technology of the present application can capture and supplement the image details of the second image in different dimensions by regularizing the second image in different dimensions of the same space, so that the second image can clearly display the tiny structure and / or boundary information in the lesion area of ​​the target object, improve the clarity and detail accuracy of the second image, and enhance the image reconstruction effect.

[0087] In some embodiments, the process of establishing the image reconstruction model includes:

[0088] The observation model is established according to the downsampling parameters, the blur parameters, and the geometric transformation parameters; wherein the observation model is used to indicate the mapping relationship between the first image and the second image; the expression of the observation model is as follows:

[0089] ;

[0090] in, used to indicate the total number of the first images; for indicating the first image; Used to indicate the downsampling operator from high-resolution space to low-resolution space; Used to indicate fuzzy operators; Used to indicate geometric transformation operators; used to indicate the second image;

[0091] Regularizing the X-axis dimension, the Y-axis dimension, and the Z-axis dimension of the second image according to the Manhattan norm to obtain the regularized optimization term;

[0092] According to the regular optimization term, the observation model is updated to obtain the image reconstruction model; wherein the expression of the regular optimization term is as follows:

[0093] ;

[0094] in, Used to indicate the regularization function; Used to indicate the balance impact factor; A vector indicating the second image in the X-axis dimension; A vector indicating the second image in the Y-axis dimension; A vector indicating the second image in the Z-axis dimension; The expression is as follows:

[0095] ;

[0096] in, To indicate calculation The gradient on .

[0097] In this embodiment, the downsampling operator, also known as the downsampling operator, is primarily used to reduce the size of the image and the number of sampling points in the matrix. The downsampling process typically involves screening and aggregating the original image data. Downsampling methods may include alternate bit-sampling and region merging. Optionally, in the alternate bit-sampling method, one point is taken at a certain interval in each row and column, thereby reducing the amount of image data. Optionally, in the region merging method, the pixel values ​​within a certain area are aggregated by averaging or maximizing the pixel values ​​to obtain a pixel value representing the area.

[0098] In this embodiment, the fuzzy operator is primarily used to process the fuzziness of an image, achieving a blurred image effect. This blurring can be global or local. The fuzzy operator is used to simulate and reverse the blur effect in an image.

[0099] In this embodiment, geometric transformation operators are primarily used to alter geometric properties of an image, such as its shape, position, and orientation. These transformations can be global or local, depending on the specific form of the transformation function. For example, affine transformations and projective transformations can be considered geometric transformation operators. These operators can map pixels in the original image to new positions based on a specific transformation matrix or function, thereby generating a transformed image. By adjusting the transformation parameters, various effects such as scaling, rotation, translation, and tilting of the image can be achieved.

[0100] In this embodiment, the mapping relationship is used to indicate a process of simulating a second image to obtain a first image after undergoing shifting, blurring, and downsampling operations.

[0101] In some embodiments, establishing the observation model according to the downsampling parameters, the blur parameters, and the geometric transformation parameters includes:

[0102] The observation model is obtained by eliminating the random noise value in the initial model through the expectation maximization model; wherein the expression of the initial model is as follows:

[0103] ;

[0104] in, used to indicate the total number of the first images; for indicating the first image; Used to indicate the downsampling operator from high-resolution space to low-resolution space; Used to indicate fuzzy operators; Used to indicate geometric transformation operators; used to indicate the second image; Indicates the random noise value.

[0105] In this embodiment, the Manhattan norm (L1 norm) is a measure of the length of a vector in Manhattan space. The Manhattan norm can help the algorithm better handle noise and outliers in the data, thereby improving the robustness and accuracy of the model. Furthermore, when processing sparse data, the Manhattan norm is generally better able to preserve the sparsity of the data.

[0106] For example, the regular optimization term It can be 0.5.

[0107] In this embodiment, Used to balance the impact between regularization and data fidelity. For example, =0.01.

[0108] In some embodiments, the image reconstruction model is expressed as follows:

[0109] ;

[0110] in, for indicating the first image; A vector indicating the first image; Used to indicate the balance impact factor; A vector indicating the second image in the X-axis dimension; A vector indicating the second image in the Y-axis dimension; A vector indicating the second image in the Z-axis dimension.

[0111] In an embodiment of the present application, by regularizing the different dimensions of the second image, supplementary information of the image details of the second image in different dimensions can be simulated, so that when the second image is reconstructed using the image reconstruction model updated by the regularization optimization term, the minute structure and / or boundary information of the second image can be clearly displayed, thereby improving the clarity and detail accuracy of the second image and improving the image reconstruction effect.

[0112] In some embodiments, the method of determining the blur parameter includes:

[0113] Using Gaussian distribution to establish point spread functions of the first image in the X-axis dimension, the Y-axis dimension, and the Z-axis dimension;

[0114] The blur parameter is determined according to the point spread function; wherein the expression of the point spread function is as follows:

[0115] ;

[0116] in, Used to indicate the coordinates of the first image in the n-axis dimension.

[0117] In this embodiment, the Gaussian distribution, also known as the normal distribution, is an important probability distribution in statistics. The normal distribution has the characteristics of a bell-shaped curve, with low peaks at both ends, a high center, and symmetry. The point spread function is established based on the Gaussian distribution. On the one hand, the Gaussian distribution has a smooth characteristic, which makes the point spread process smoother, reducing sudden changes or jumps, and reducing jagged edges or noise in the image. On the other hand, the Gaussian distribution is differentiable and integrable, making it easy to mathematically model and calculate.

[0118] In this embodiment, the point spread function (PSF) is a mathematical function that describes the imaging quality of an optical system. It represents the brightness distribution of a point light source on the imaging surface after it is imaged by the optical system. The PSF describes the imaging system's response to a point source or object.

[0119] In this embodiment, parameters in different dimensions Different sizes. Parameters It can be determined based on multiple experiments. For example, in the X-axis dimension or the Y-axis dimension, you can set ; In the Z-axis dimension, you can set ;in, Used to indicate the physical distance between two adjacent first images in the Z-axis dimension.

[0120] In medical imaging, such as magnetic resonance imaging (MRI), accurately estimating the PSF is crucial for image reconstruction quality, as it directly affects the clarity and accuracy of the resulting composite image. Therefore, accurately modeling image blur using the PSF is crucial.

[0121] In an embodiment of the present application, when the image is blurred during the imaging process due to some reasons, since the rationality of the parameters directly affects the reconstruction effect of the image details, it is possible to accurately simulate the blurring effect in image imaging by establishing Gaussian distributions separately in different dimensions of the same space and determining the parameter sizes corresponding to the dimensions respectively, so as to make the details of the lesion area in the reconstructed second image clear and specific, thereby improving the reconstruction effect of the image details.

[0122] In some embodiments, before reconstructing the first image using the image reconstruction model, the method further includes:

[0123] Determining at least one image stack based on each of the first images of the same dimension; wherein the image stack includes a target image stack;

[0124] The image brightness of each image stack is adjusted to the target brightness of the target image stack; and / or the coordinate system of each image stack is adjusted to the coordinate system of the target image stack.

[0125] In an embodiment of the present application, by preprocessing each first image (for example, adjusting the image brightness and / or adjusting the coordinate system), the interference caused by the inconsistency of image brightness or image coordinate system between the first images to image reconstruction can be reduced, thereby further improving the image reconstruction effect and ensuring the clarity of the second image.

[0126] In some embodiments, reconstructing the first image using the image reconstruction model to obtain the second image includes:

[0127] determining a mean image of each of the first images as an initial image;

[0128] Obtaining an initial residual and an initial iteration direction according to the initial image and an expression of the image reconstruction model;

[0129] The initial residual is iteratively solved according to the initial iteration direction using a conjugate gradient descent algorithm until the residual meets a preset requirement, thereby obtaining the second image.

[0130] In this embodiment, the conjugate gradient descent algorithm is an iterative algorithm used to solve the linear equation system AX = b. Here, A is a large sparse matrix that indicates the downsampling and blurring process from the high-resolution image (the second image) to the low-resolution image (the first image); X represents the reconstructed high-resolution image (represented as a vector); and b represents the observed low-resolution image (represented as a vector). The core of the conjugate gradient descent algorithm is to keep the iteration direction consistent with the conjugate direction of the previous iteration.

[0131] Exemplarily, the electronic device determines the initial image X0 based on the average of the first images; a method for determining the initial residual The method is as follows:

[0132] ;

[0133] ;

[0134] in, Used to indicate the balance impact factor; A vector indicating the second image in the X-axis dimension; A vector indicating the second image in the Y-axis dimension; A vector indicating the second image in the Z-axis dimension.

[0135] A method to determine the initial iteration direction The method is: .

[0136] In some embodiments, the electronic device first inputs an initial image to obtain a first-solved image. The first-solved image is then input to obtain a second-solved image. Each iteration uses the previously solved image as input to obtain the latest solved image. The iterative reconstruction is completed until the residual meets preset requirements, at which point the currently solved image is determined to be the second image.

[0137] In some embodiments, the iteratively solving the expression of the image reconstruction model using the conjugate gradient descent algorithm and the initial image until the residual meets a preset requirement to obtain the second image includes:

[0138] In the i-th iteration process, the i-th iteration direction is obtained according to the second candidate image obtained after the i-1-th solution; wherein i is a positive integer; i is greater than 1;

[0139] Perform iterative solution based on the second candidate image after the i-1th solution and the i-th iteration direction to obtain the second candidate image after the i-th solution;

[0140] Obtaining an i-th residual according to the second candidate image obtained after the i-th solution and an expression of the image reconstruction model;

[0141] If the i-th residual is less than or equal to a preset threshold, the second candidate image obtained after the i-th solution is determined to be the second image.

[0142] Exemplary, a method for determining the direction of the i-th iteration The method is:

[0143] ;

[0144] in, Used to indicate the second candidate image after the i-1th solution. A method for determining the second candidate image after the i-th solution The method is ; A method to determine the i-th residual The method is .

[0145] In an embodiment of the present application, the second image is obtained by iteratively solving the conjugate gradient descent algorithm, and the image after the previous solution can be used as the input for the next time. The convergence speed is fast, and the number of iterations can be effectively reduced; and sparsity can be effectively utilized to reduce storage requirements and have higher stability.

[0146] In some embodiments, the method further comprises:

[0147] Performing three-dimensional reconstruction on the first image and the second image respectively to obtain a first three-dimensional image before super-resolution reconstruction and a second three-dimensional image after super-resolution reconstruction;

[0148] Obtaining a differential three-dimensional image by subtracting the second three-dimensional image from the first three-dimensional image; wherein the differential three-dimensional image includes at least one difference voxel;

[0149] Determine the region corresponding to the difference voxels that is greater than or equal to a preset threshold as a suspicious region and filter out a target region from the suspicious region;

[0150] Re-constructing the target area in three dimensions to obtain a target reconstructed area;

[0151] The target reconstructed region is superimposed on the second three-dimensional image to obtain a target high-resolution image.

[0152] In the embodiment of the present application, three-dimensional reconstruction is used to indicate the process of converting a two-dimensional medical image into a three-dimensional image.

[0153] In some embodiments, the three-dimensional reconstruction may include but is not limited to at least one of surface reconstruction, volume rendering, and voxel interpolation.

[0154] In the embodiment of the present application, a voxel is the smallest unit that can be segmented in a three-dimensional space. The content of a voxel may include but is not limited to at least one of a density value, a label value, opacity, color, and volume flow rate.

[0155] In some embodiments, as Figure 3 and Figure 4 As shown, Figure 3 is a schematic diagram of a first three-dimensional image, Figure 4This is a schematic diagram of a second 3D image. Because the first image is a raw sequence of images and has not undergone super-resolution reconstruction, the edges and details (for example, the microscopic structure of a lesion) in the first 3D image reconstructed by the electronic device based on the first image will be missing or blurred. The second image, on the other hand, is an image obtained through super-resolution reconstruction. The edges and details in the second 3D image reconstructed by the electronic device based on the second image are relatively clear and rich. Therefore, by subtracting the second 3D image from the first 3D image, the electronic device can obtain differential voxels to highlight edges and details.

[0156] In one embodiment, the first image includes a prostate lesion region of the target object, and the difference voxels may include the lesion region, a noise region generated by super-resolution interpolation, and a non-lesion prostate microstructure region.

[0157] Exemplarily, the preset threshold may be 2 mm, 3 mm, or 4 mm, etc.

[0158] For example, Figure 5 As shown, the suspicious area 501 is a tiny structure in the figure, characterized by an irregular shape. There may be multiple suspicious areas, and the target area is screened out from the suspicious areas based on images such as computed tomography (CT), positron emission tomography (PET), and multi-parameter magnetic resonance imaging.

[0159] In some embodiments, the electronic device segments the target area in different directions to obtain point cloud data of different planes (horizontal, sagittal, and coronal); the electronic device converts the point cloud data into voxel representation and uses the Poisson equation to reconstruct the surface; by solving the Poisson equation, a smooth surface model can be obtained, so that the surface model is relatively matched with the point cloud data, thereby achieving the "wrapping" of the point cloud data to highlight the 3D effect of the lesion. For example, Figure 6 As shown, Figure 6 Schematic diagram of the target reconstruction area.

[0160] For example, the target high-resolution image after superimposing the target reconstruction area onto the second three-dimensional image is as follows: Figure 7 shown.

[0161] In some embodiments, electronic devices can use volume rendering technology to perform 3D rendering modeling on a high-resolution target image. Volume rendering technology uses the volumetric information of a 3D image or model as a basis for rendering, thereby achieving realistic transparency, light refraction, and shadow effects. This allows for intuitive and accurate 3D rendering of the lesion's structure and adjacent relationships, ultimately resulting in a visual presentation.

[0162] Alternatively, volume rendering technology is used to refer to a method of displaying information in a three-dimensional space in the form of images using computer technology. Volume rendering technology can simulate objects and scenes in the real world, using effects such as light, shadow, and material to enable users to view and interact with data or models more intuitively.

[0163] Exemplarily, the volume rendering technology may be a volume rendering technology based on OpenGL (Open Graphics Library).

[0164] Traditional two-dimensional images struggle to accurately depict an organ's three-dimensional structure, geometry, and spatial relationship with surrounding tissue. In contrast, the present invention utilizes three-dimensional reconstruction and other processing techniques to ultimately produce a high-resolution, three-dimensional image of the target organ or tissue. This precisely displays the three-dimensional structure of the organ or tissue, enabling visualization of tiny lesions within the organ or tissue, facilitating subsequent clinical diagnosis and treatment. Furthermore, OpenGL-based volume rendering technology enables real-time rendering, improving image processing efficiency and ensuring timely delivery.

[0165] In some embodiments, the method further comprises:

[0166] Display an image reconstruction interface; the image reconstruction interface includes an image display area; the image display area is used to display the first image, the second image, the first three-dimensional image, the second three-dimensional image, the differential three-dimensional image and / or the target high-resolution image.

[0167] For example, Figure 8A and Figure 8B shown. Figure 8A is the original image of the cross section, i.e. the first image; Figure 8B It is the image after cross-sectional super-resolution reconstruction, that is, the second image.

[0168] In the embodiments of the present application, the following provides specific examples in combination with any of the above embodiments:

[0169] Specific example 1: Figure 9 FIG. 1 is an exemplary method for identifying micro-lesions based on super-resolution technology, wherein the micro-lesions are prostate lesions. Figure 9 As shown, the method is executed by an electronic device, and the method includes:

[0170] 1) Obtain image stack data of the prostate in different directions and input them into the image reconstruction model. The image stack in each direction includes multiple first images; here, the direction can be the dimension in the above embodiment.

[0171] 2) Preprocessing the input image stack data to obtain an initialization volume; here, the initialization volume can be understood as the initial image in the above embodiment.

[0172] 3) The initialization volume is iteratively solved using the conjugate gradient algorithm until the second image is obtained.

[0173] 4) Performing three-dimensional reconstruction on the first image and the second image respectively to obtain a first three-dimensional image before super-resolution reconstruction and a second three-dimensional image after super-resolution reconstruction.

[0174] 5) Subtracting the second 3D image from the first 3D image to obtain a differential 3D image; wherein the differential 3D image includes at least one difference voxel.

[0175] 6) Determine the area corresponding to the voxels with a difference greater than or equal to a preset threshold as the suspicious area and filter out the target area from the suspicious area.

[0176] 7) Reconstruct the target area in three dimensions to obtain the target reconstructed area.

[0177] 8) The target reconstructed area is superimposed on the second 3D image to obtain a high-resolution target image. The high-resolution target image is then 3D rendered and modeled using volume rendering technology. The structural morphology and adjacent relationships of the prostate and lesions are intuitively and accurately rendered in 3D, and finally visualized as a 3D volume of the nuclear resonance image.

[0178] In the above-mentioned method for identifying tiny lesions based on super-resolution technology, at least one first image containing the lesion area of ​​the target object is input into the image reconstruction model for reconstruction, thereby obtaining a reconstructed high-resolution second image. In this way, the method for identifying tiny lesions based on super-resolution technology of the present application can capture and supplement the image details of the second image in different dimensions by regularizing the second image in different dimensions of the same space, so that the second image can clearly display the tiny structure and / or boundary information in the lesion area of ​​the target object, improve the clarity and detail accuracy of the second image, and enhance the image reconstruction effect.

[0179] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0180] Based on the same inventive concept, the embodiments of the present application also provide a super-resolution technology-based small lesion identification device for implementing the above-mentioned super-resolution technology-based small lesion identification method. The implementation solution provided by this device is similar to the implementation solution described in the above-mentioned method. Therefore, the specific limitations of one or more embodiments of the super-resolution technology-based small lesion identification device provided below can be referred to the above-mentioned limitations of the super-resolution technology-based small lesion identification method, and will not be repeated here.

[0181] In some embodiments, as Figure 10 As shown, the present application also provides a device for identifying micro-lesions based on super-resolution technology, the device comprising:

[0182] An acquisition module 110 is configured to acquire at least one first image of a target object; the first image includes a lesion area of ​​the target object;

[0183] A processing module 120 is configured to reconstruct each of the first images using an image reconstruction model to obtain a second image;

[0184] The image reconstruction model is obtained by updating the observation model by the regularized optimization term of the second image; the regularized optimization term is used to indicate the regularization of the second image in different dimensions of the same space; and the resolution of the first image is lower than that of the second image.

[0185] In some embodiments, the process of establishing the image reconstruction model includes:

[0186] The observation model is established according to the downsampling parameters, the blur parameters, and the geometric transformation parameters; wherein the observation model is used to indicate the mapping relationship between the first image and the second image; the expression of the observation model is as follows:

[0187] ;

[0188] in, used to indicate the total number of the first images; for indicating the first image; Used to indicate the downsampling operator from high-resolution space to low-resolution space; Used to indicate fuzzy operators; Used to indicate geometric transformation operators; used to indicate the second image;

[0189] Regularizing the X-axis dimension, the Y-axis dimension, and the Z-axis dimension of the second image according to the Manhattan norm to obtain the regularized optimization term;

[0190] According to the regular optimization term, the observation model is updated to obtain the image reconstruction model; wherein the expression of the regular optimization term is as follows:

[0191] ;

[0192] in, Used to indicate the regularization function; Used to indicate the balance impact factor; A vector indicating the second image in the X-axis dimension; A vector indicating the second image in the Y-axis dimension; A vector indicating the second image in the Z-axis dimension; The expression is as follows:

[0193] ;

[0194] in, To indicate calculation The gradient on .

[0195] In some embodiments, the method of determining the blur parameter includes:

[0196] Using Gaussian distribution to establish point spread functions of the first image in the X-axis dimension, the Y-axis dimension, and the Z-axis dimension;

[0197] The blur parameter is determined according to the point spread function; wherein the expression of the point spread function is as follows:

[0198] ;

[0199] in, Used to indicate the coordinates of the first image in the n-axis dimension.

[0200] In some embodiments, the image reconstruction model is expressed as follows:

[0201] ;

[0202] in, for indicating the first image; A vector indicating the first image; Used to indicate the balance impact factor; A vector indicating the second image in the X-axis dimension; A vector indicating the second image in the Y-axis dimension; A vector indicating the second image in the Z-axis dimension.

[0203] In some embodiments, the processing module 120 is configured to perform the following steps:

[0204] determining a mean image of each of the first images as an initial image;

[0205] Obtaining an initial residual and an initial iteration direction according to the initial image and an expression of the image reconstruction model;

[0206] The initial residual is iteratively solved according to the initial iteration direction using a conjugate gradient descent algorithm until the residual meets a preset requirement, thereby obtaining the second image.

[0207] In some embodiments, the processing module 120 is further configured to perform the following steps:

[0208] In the i-th iteration process, the i-th iteration direction is obtained according to the second candidate image obtained after the i-1-th solution; wherein i is a positive integer; i is greater than 1;

[0209] Perform iterative solution based on the second candidate image after the i-1th solution and the i-th iteration direction to obtain the second candidate image after the i-th solution;

[0210] Obtaining an i-th residual according to the second candidate image obtained after the i-th solution and an expression of the image reconstruction model;

[0211] If the i-th residual is less than or equal to a preset threshold, the second candidate image obtained after the i-th solution is determined to be the second image.

[0212] In some embodiments, the apparatus further comprises a reconstruction module; the reconstruction module is configured to perform the following steps:

[0213] Performing three-dimensional reconstruction on the first image and the second image respectively to obtain a first three-dimensional image before super-resolution reconstruction and a second three-dimensional image after super-resolution reconstruction;

[0214] Obtaining a differential three-dimensional image by subtracting the second three-dimensional image from the first three-dimensional image; wherein the differential three-dimensional image includes at least one difference voxel;

[0215] Determine the region corresponding to the difference voxels that is greater than or equal to a preset threshold as a suspicious region and filter out a target region from the suspicious region;

[0216] Re-constructing the target area in three dimensions to obtain a target reconstructed area;

[0217] The target reconstructed region is superimposed on the second three-dimensional image to obtain a target high-resolution image.

[0218] Each module in the above-mentioned image reconstruction device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in an electronic device in the form of hardware, or can be stored in a memory in the electronic device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0219] In one embodiment, an electronic device is provided. The electronic device may be a terminal, and its internal structure diagram may be as follows: Figure 11 As shown. The electronic device includes a processor, memory, a communication interface, a display, and an input device connected via a system bus. The processor of the electronic device provides computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The communication interface of the electronic device is used to communicate with an external terminal via wired or wireless communication. The wireless communication can be achieved via Wi-Fi, a mobile cellular network, NFC (near-field communication), or other technologies. When executed by the processor, the computer program implements a method for identifying microscopic lesions based on super-resolution technology. The display of the electronic device can be a liquid crystal display or an electronic ink display. The input device of the electronic device can be a touch layer covering the display, or keys, a trackball, or a touchpad provided on the electronic device housing, or an external keyboard, touchpad, or mouse.

[0220] Those skilled in the art will understand that Figure 11 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the electronic device to which the solution of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0221] In one embodiment, an electronic device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the following steps when executing the computer program: the computer program implements the steps in the above method embodiments.

[0222] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0223] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0224] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), data processing logic devices based on quantum computing, and the like.

[0225] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0226] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A method for identifying micro-lesions based on super-resolution technology, characterized in that: The method comprises: Acquire at least one first image of a target object; the first image includes a lesion area of ​​the target object; Performing super-resolution reconstruction on each of the first images using an image reconstruction model to obtain a second image, including: determining a mean image of each of the first images as an initial image; Obtaining an initial residual and an initial iteration direction according to the initial image and an expression of the image reconstruction model; Using a conjugate gradient descent algorithm, the initial residual is iteratively solved according to the initial iteration direction until the residual meets a preset requirement, thereby obtaining the second image; The image reconstruction model is obtained by updating the observation model using a regularized optimization term of the second image; the regularized optimization term is used to indicate regularization of the second image in different dimensions of the same space; and the resolution of the first image is lower than that of the second image. The process of establishing the image reconstruction model includes: The observation model is established according to the downsampling parameters, the blur parameters, and the geometric transformation parameters; wherein the observation model is used to indicate the mapping relationship between the first image and the second image; the expression of the observation model is as follows: ； in, used to indicate the total number of the first images; for indicating the first image; Used to indicate the downsampling operator from high-resolution space to low-resolution space; Used to indicate fuzzy operators; Used to indicate geometric transformation operators; used to indicate the second image; Regularizing the X-axis dimension, the Y-axis dimension, and the Z-axis dimension of the second image according to the Manhattan norm to obtain the regularized optimization term; According to the regular optimization term, the observation model is updated to obtain the image reconstruction model; wherein the expression of the regular optimization term is as follows: ； in, Used to indicate the regularization function; Used to indicate the balance impact factor; A vector indicating the second image in the X-axis dimension; A vector indicating the second image in the Y-axis dimension; A vector indicating the second image in the Z-axis dimension; The expression is as follows: ； in, To indicate calculation The gradient on The expression of the image reconstruction model is as follows: ； in, for indicating the first image; A vector indicating the first image; Used to indicate the balance impact factor; A vector indicating the second image in the X-axis dimension; A vector indicating the second image in the Y-axis dimension; A vector indicating the second image in the Z-axis dimension.

2. The method according to claim 1, characterized in that Methods for determining fuzzy parameters include: Using Gaussian distribution to establish point spread functions of the first image in the X-axis dimension, the Y-axis dimension, and the Z-axis dimension; The blur parameter is determined according to the point spread function; wherein the expression of the point spread function is as follows: ； in, used to indicate the coordinate of the first image in the n-axis dimension, in the X-axis dimension or the Y-axis dimension, ; In the Z-axis dimension, , Used to indicate the physical distance between two adjacent first images in the Z-axis dimension.

3. The method according to claim 1, characterized in that The method of iteratively solving the expression of the image reconstruction model using the conjugate gradient descent algorithm and the initial image until the residual meets a preset requirement to obtain the second image includes: In the i-th iteration process, the i-th iteration direction is obtained according to the second candidate image obtained after the i-1-th solution; wherein i is a positive integer; i is greater than 1; Perform iterative solution based on the second candidate image after the i-1th solution and the i-th iteration direction to obtain the second candidate image after the i-th solution; Obtaining an i-th residual according to the second candidate image obtained after the i-th solution and an expression of the image reconstruction model; If the i-th residual is less than or equal to a preset threshold, the second candidate image obtained after the i-th solution is determined to be the second image.

4. The method according to claim 1, wherein The method further comprises: Performing three-dimensional reconstruction on the first image and the second image respectively to obtain a first three-dimensional image before super-resolution reconstruction and a second three-dimensional image after super-resolution reconstruction; Obtaining a differential three-dimensional image by subtracting the second three-dimensional image from the first three-dimensional image; wherein the differential three-dimensional image includes at least one difference voxel; Determine the region corresponding to the difference voxels that is greater than or equal to a preset threshold as a suspicious region and filter out a target region from the suspicious region; Re-constructing the target area in three dimensions to obtain a target reconstructed area; The target reconstructed region is superimposed on the second three-dimensional image to obtain a target high-resolution image.

5. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images