Medical image reconstruction method and device and related equipment

Abstract

The invention belongs to the technical field of digital medical treatment, provides a medical image reconstruction method and device, computer equipment and a computer readable storage medium, and aims to solve the problem of insufficient resolution when a traditional technology in digital medical treatment is used for reconstructing a medical image. The initial medical image is converted into an image block sequence, an initial image block sequence is obtained, the initial image block sequence comprises initial image blocks and position codes of the initial image blocks in the initial medical image, the resolution of the initial image blocks is increased, target image blocks are obtained, and the target image blocks are obtained according to the position codes; and reconstructing all the target image blocks corresponding to the initial image block sequence into a whole image to obtain the target medical image, so that the image data corresponding to the medical image can be subjected to efficient super-resolution reconstruction, the resolution and definition of the complex medical image are greatly improved, and the technical level of digital medical treatment is improved.

Description

Technical Field

[0001] The present application relates to the field of digital medical technology, and in particular to a medical image reconstruction method, apparatus, computer equipment, and computer-readable storage medium. Background Art

[0002] Medical imaging plays a vital role in clinical diagnosis and research. However, due to the limitations of imaging equipment and technology, traditional medical imaging often has the problem of insufficient resolution. These low-resolution medical images affect doctors' accurate identification and diagnosis of lesions, and thus affect the formulation of treatment plans. Therefore, improving the resolution of medical images has important clinical significance.

[0003] In traditional medical image processing technology, methods for super-resolution reconstruction of low-resolution medical images include interpolation, dictionary-based learning methods, and convolutional neural networks (CNNs). Although these methods have improved the resolution of images to a certain extent, they still have limitations when processing complex medical images, resulting in technical problems such as insufficient resolution when super-resolution reconstruction of medical images. Summary of the invention

[0004] The present application provides a medical image reconstruction method, apparatus, computer equipment and computer-readable storage medium, which can solve the technical problem of insufficient resolution in medical image reconstruction in traditional digital medical technology.

[0005] In a first aspect, the present application provides a medical image reconstruction method, comprising: obtaining an initial medical image; based on a preset generative pre-trained conversion model, converting the initial medical image into an image block sequence to obtain an initial image block sequence, wherein the initial image block sequence includes an initial image block and a position code of the initial image block in the initial medical image; increasing the resolution of the initial image block to obtain a target image block; and according to the position code, reconstructing all the target image blocks corresponding to the initial image block sequence into a whole image to obtain a target medical image.

[0006] In a second aspect, the present application provides a medical image reconstruction device, comprising: a first acquisition unit, used to acquire an initial medical image; a first conversion unit, used to convert the initial medical image into an image block sequence based on a preset generative pre-trained conversion model, to obtain an initial image block sequence, wherein the initial image block sequence includes an initial image block and a position code of the initial image block in the initial medical image; a first enlargement unit, used to increase the resolution of the initial image block to obtain a target image block; and a first construction unit, used to reconstruct all the target image blocks corresponding to the initial image block sequence into a whole image according to the position code to obtain a target medical image.

[0007] In a third aspect, the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the medical image reconstruction method when executing the computer program.

[0008] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor performs the steps of the medical image reconstruction method.

[0009] The present application provides a medical image reconstruction method, apparatus, computer equipment and computer-readable storage medium. The method obtains a preset system architecture of a business by responding to a display instruction of a system architecture of a business, wherein the preset system architecture includes constituent elements, obtains resource information corresponding to the constituent elements, and determines to display the constituent elements when the resource information meets the preset display conditions, constructs a target system architecture corresponding to the business with all constituent elements determined to be displayed, and displays the target system architecture. Compared with the fixed system architecture diagram used to display financial business in traditional technology, the embodiment of the present application can dynamically determine whether to display the constituent element according to the resource information of the constituent element each time the system architecture is displayed, and realizes the dynamic display of the system architecture according to the resource information, so that the display of the system architecture can reflect the resource information of the constituent elements, and the resource information reflects the operation of the business, so that the system architecture of the displayed business can intuitively and accurately reflect the operation status and survivability of the business, so that the system architecture of the business is as consistent as possible with the operation of the business status, so as to improve the accuracy of medical image reconstruction, so that the front-end user can intuitively, accurately and quickly grasp the system architecture corresponding to the current operation of the business. For example, in digital medicine, for financial business, the real operation status of the financial business can be accurately displayed through the system architecture of the financial business. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0011] Figure 1 A schematic diagram of a process for reconstructing medical images provided in an embodiment of the present application;

[0012] Figure 2 A schematic diagram of a first sub-process of the medical image reconstruction method provided in an embodiment of the present application;

[0013] Figure 3A schematic diagram of a second sub-process of the medical image reconstruction method provided in an embodiment of the present application;

[0014] Figure 4 A schematic block diagram of a medical image reconstruction device provided in an embodiment of the present application;

[0015] Figure 5 A schematic block diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0016] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0017] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0018] The embodiment of the present application provides a medical image reconstruction method, which can be applied to but not limited to computer devices such as laptop computers, desktop computers, servers, etc., and can be used in medical image reconstruction including but not limited to digital medicine.

[0019] In the face of the technical problem of insufficient resolution when reconstructing medical images with traditional technologies in digital medicine, the inventors proposed a medical image reconstruction method of the embodiment of the present application. The core idea of ​​the embodiment of the present application is: converting the image data of the medical image itself into an ordered sequence of image blocks, and encoding and decoding the image block sequence based on a generative pre-trained conversion model to enlarge and refine each image block in the image block sequence, and then recombining the enlarged and refined image blocks in the corresponding order to generate a clearer medical image, thereby realizing super-resolution reconstruction of medical images, and significantly improving the clarity and resolution of the reconstructed medical images. Compared with traditional medical image super-resolution reconstruction methods, the embodiment of the present application can show higher reconstruction accuracy and efficiency when processing complex medical images, and can better meet the needs of digital medicine for super-resolution reconstruction of medical images.

[0020] Some embodiments of the present application are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0021] See also Figure 1 , Figure 1 The following is a flow chart of a medical image reconstruction method provided in an embodiment of the present application. Figure 1 As shown, the method includes but is not limited to the following steps S11-S14:

[0022] S11. Obtaining initial medical images.

[0023] Explanationally, initial medical images are generally low-resolution images, including but not limited to CT, MRI (magnetic resonance imaging), X-ray and other types of image data. Since low-resolution initial medical images will affect doctors' accurate identification and diagnosis of lesions, it is necessary to improve the resolution of initial medical images to reconstruct medical images.

[0024] S12. Based on a preset generative pre-trained conversion model, convert the initial medical image into an image block sequence to obtain an initial image block sequence, wherein the initial image block sequence includes initial image blocks and position codes of the initial image blocks in the initial medical image.

[0025] Explanatory, a generative pre-trained transformer model is pre-built, that is, a preset generative pre-trained transformer model. Among them, the generative pre-trained transformer model is a resolution conversion model of medical images based on a generative pre-trained model (Generative Pre-trained Transformer, GPT), wherein GPT is a deep learning model based on Transformer, GPT usually adopts a multi-layer Transformer structure to predict the probability distribution of the next word, and generates natural language text through language patterns learned in a large text corpus. It has the ability to generate high-quality text through pre-training learning of a large amount of text data. The core component of the GPT model is Transformer, which consists of two parts: encoder and decoder, wherein the encoder is responsible for encoding the input data into hidden representation, and the decoder generates output data based on the hidden representation.

[0026] In order to adapt the GPT architecture to medical image processing, the initial medical image is sliced ​​and divided into blocks, and multiple image blocks are used to convert the initial medical image into an image block sequence to obtain an initial image block sequence, so that the image data is regarded as a special kind of "text" data. The powerful modeling ability of Transformer can be used to process and generate high-resolution medical images, where the initial image block sequence contains the initial image block and the position encoding of the initial image block in the initial medical image.

[0027] Exemplarily, based on the input layer included in the preset generative pre-trained transformation model, the image data corresponding to the initial medical image is converted into sequence data suitable for Transformer processing. This can be done by dividing the image data corresponding to the initial medical image into small patches (patches) and converting them into a one-dimensional sequence. For example, assuming that the input low-resolution image size is 128*128 pixels, it can be divided into 16*16 small patches, each small patch contains 16*16=256 pixels, and the entire image will be divided into 8*8=64 small patches. Then, each small patch is converted into a 256-dimensional vector, so that the entire image becomes a sequence of 64 256-dimensional vectors. This division method allows the GPT model to process these image patches like text "vocabulary", enabling the model to better capture the details in the image and generate high-resolution images during reconstruction.

[0028] S13: increasing the resolution of the initial image block to obtain a target image block.

[0029] Explanatory, based on a preset generative pre-trained conversion model, the resolution of the initial image block is increased to obtain a target image block, where the target image block is an image block with an enlarged image resolution.

[0030] Since the preset generative pre-trained conversion model based on GPT consists of two parts: encoder and decoder, increasing the resolution of the initial image block includes the following process:

[0031] 1) Encode the initial image block sequence.

[0032] The encoder uses multiple self-attention layers to process the sequence data corresponding to the input initial image block sequence and extract the feature representation of the image block sequence. The self-attention mechanism can capture the long-distance dependencies between different parts of the image corresponding to the image block sequence and enhance the overall understanding of the image.

[0033] 2) Decode the initial image block sequence.

[0034] The decoder receives the hidden representation generated by the encoder and the input image block sequence, gradually decodes these representations, and generates new high-resolution image blocks, as shown in the following formula (1):

[0035] y t =Decoder(y ＜t h t ), formula (1);

[0036] Among them, y t is the t-th output image patch, y ＜t represents the previously generated image patch sequence, h tis the hidden representation of the encoder.

[0037] 3) Multi-layer self-attention mechanism:

[0038] The decoder also uses the self-attention mechanism. Each layer in the decoder contains a self-attention mechanism, which captures the global information of the image by calculating the correlation between each image block and other image blocks, as shown in the following formula (2):

[0039]

[0040] Where Q, K, and V are query, key, and value matrices respectively, k is the scaling factor.

[0041] The decoder uses a self-attention mechanism and combines the hidden representation of the encoder to enlarge and refine the image blocks decoded at each layer to generate a clearer image. Finally, the decoder recombines these high-resolution image blocks into a complete image and gradually generates high-resolution image blocks.

[0042] For example, assuming that the initial input medical image is a low-resolution 64x64 pixel image, the decoder will generate a 128x128 or higher resolution image. Each generated image block is gradually enlarged during the decoding process and combined with context information to generate the final high-resolution block. Through the layer-by-layer self-attention mechanism and feedforward network, the image blocks gradually become detailed until all image blocks are reconstructed to generate high-resolution image blocks.

[0043] S14. Reconstruct all the target image blocks corresponding to the initial image block sequence into a whole image according to the position code to obtain a target medical image.

[0044] Explanatory, after the resolution of the initial image block is increased to generate a high-resolution image block and obtain the target image block, all target image blocks corresponding to the initial image block sequence are reconstructed into a whole image based on the position encoding of each image block when converting the initial medical image into an image block sequence to obtain a target medical image, so as to recombine these high-resolution image blocks of the reconstructed resolution into a complete image. The target medical image is an image with enlarged and refined resolution, and the target medical image becomes more detailed and clear.

[0045] In an embodiment of the present application, an initial medical image is obtained, and based on a preset generative pre-trained conversion model, the initial medical image is converted into an image block sequence to obtain an initial image block sequence, wherein the initial image block sequence includes an initial image block and a position code of the initial image block in the initial medical image, the resolution of the initial image block is increased to obtain a target image block, and all target image blocks corresponding to the initial image block sequence are reconstructed into a whole image according to the position code to obtain a target medical image. By converting the initial medical image into an image block sequence, the image data corresponding to the initial medical image is regarded as a special "text", so that the GPT architecture is adapted to medical image processing, and can make full use of the massive data and strong GPT large model. It has a large generation capacity and can perform efficient super-resolution reconstruction of image data corresponding to medical images, thereby improving the clarity and resolution of medical images and the quality of medical images. Moreover, through the global modeling capability of Transformer, it can ensure that the reconstructed target medical images are consistent at the global level, rather than just improving the clarity of local areas. Especially when processing complex medical images, compared with traditional medical image resolution reconstruction methods, it greatly improves the resolution and clarity of complex medical images, showing higher reconstruction accuracy and efficiency of complex medical images, which can better meet the needs of clinical diagnosis and research, improve the technical level of digital medicine, and enable digital medicine to assist clinical diagnosis and research.

[0046] In one embodiment, see Figure 2 , Figure 2 This is a schematic diagram of the first sub-process of the medical image reconstruction method provided in the embodiment of the present application. Figure 2 As shown, in this embodiment, the initial medical image is converted into an image block sequence to obtain an initial image block sequence, including:

[0047] S21, performing image block cutting on the initial medical image to obtain initial image blocks;

[0048] S22, marking the initial image block with a position code according to the position of the initial image block in the initial medical image, to obtain a position code corresponding to the initial image block;

[0049] S23, converting the initial image block into an initial image block vector;

[0050] S24. Arrange all the initial image block vectors into a sequence according to the position coding to obtain an initial image block sequence corresponding to the initial medical image.

[0051] Explanatory, firstly, the initial medical image is cut into image blocks to obtain the initial image blocks. Then, according to the position of the initial image blocks in the initial medical image, the initial image blocks are annotated with position codes to obtain the position codes corresponding to the initial image blocks. The position codes are also the position order of the initial image blocks in the initial medical image, so that the spatial structure information of the initial medical image is retained through the position codes, that is, the initial image blocks can be restored to the initial medical image through the position codes. Then, the initial image blocks are converted into initial image block vectors. The initial image block vectors corresponding to the initial image blocks can be obtained by converting the initial image blocks into pixel matrices according to pixels, and then converting the pixel matrices into one-dimensional vectors. Since the position codes are also the position order of the initial image blocks in the initial medical image, according to the position codes, all the initial image block vectors are arranged into a sequence sorted according to the position codes to obtain the initial image block sequence corresponding to the initial medical image. The initial image block sequence is a sequence composed of all the initial image blocks contained in the initial medical image according to the order corresponding to the position codes.

[0052] For example, assuming that the image size of the input low-resolution initial medical image is 128*128 pixels, it can be divided into 16x16 small image blocks, each small image block contains 16*16=256 pixels, and the image corresponding to the entire initial medical image will be divided into 8*8=64 small image blocks. Then, each small image block is converted into a 256-dimensional one-dimensional vector based on the pixels, so that the image corresponding to the entire initial medical image becomes a sequence of 64 256-dimensional one-dimensional vectors. This division method allows the GPT model to process these image blocks like text "vocabulary", allowing the GPT model to better capture the details in the image and generate high-resolution images during reconstruction.

[0053] In the embodiment of the present application, the initial medical image is cut into image blocks and each initial image block is position-coded and labeled, so that the spatial structure information of the initial medical image is retained while the initial medical image is converted into a sequence of initial image blocks, so that the preset generative pre-trained conversion model can process the initial image blocks corresponding to the initial medical image, and after the initial image blocks are processed, the spatial structure of the processed initial image blocks can be restored according to the spatial structure information of the initial medical image corresponding to the position code, so as to obtain a clearer image after resolution reconstruction, thereby improving the clarity and resolution of the medical image. Especially when processing complex medical images, compared with traditional medical image resolution reconstruction methods, the resolution and clarity of complex medical images are greatly improved, showing higher reconstruction accuracy and efficiency of complex medical images, which can better meet the needs of clinical diagnosis and research, improve the technical level of digital medicine, and enable digital medicine to assist clinical diagnosis and research.

[0054] In one embodiment, increasing the resolution of the initial image block to obtain a target image block includes:

[0055] Acquire the main image blocks and the auxiliary image blocks in all the initial image blocks;

[0056] The main image block is enlarged by using a first resolution, and the auxiliary image block is enlarged by using a second resolution to obtain a target image block, wherein the first resolution is greater than the second resolution.

[0057] Explanatory, when converting the initial medical image into a sequence of image blocks, when the initial medical image is sliced ​​to obtain image blocks, the obtained image blocks are divided into primary image blocks and secondary image blocks. The primary image block refers to the main, important, and key image part related to the lesion in the initial medical image, and the secondary image block refers to the image block of the non-primary image block part in the initial medical image. For initial medical images including but not limited to CT, MRI, X-ray, etc., since the image pattern is fixed, generally speaking, the position of the main, important, and key image part related to the lesion in the initial medical image is also fixed. This part is more important and critical than other parts for clinical diagnosis and research. Based on this, the image blocks contained in the initial medical image are divided into primary image blocks and secondary image blocks. When training the preset generative pre-trained conversion model, the primary image blocks and secondary image blocks contained in the training samples can be mainly labeled by medical experts, so that the preset generative pre-trained conversion model can identify the primary image blocks and secondary image blocks corresponding to each type of initial medical image including but not limited to CT, MRI, X-ray, etc.

[0058] On the basis of dividing the initial image blocks contained in the initial medical image into main image blocks and sub-image blocks, when the resolution of the initial image blocks is increased, the main image blocks and sub-image blocks in all the initial image blocks are obtained, and the main image blocks are increased by a first resolution, and the sub-image blocks are increased by a second resolution to obtain target image blocks, wherein the first resolution is greater than the second resolution, so that the main image blocks are focused on and refined, and the sub-image blocks are also enlarged and refined. On the basis of global refinement and enlargement of the entire image, the main local image corresponding to the main image block is made to achieve a higher resolution and thus clearer, so as to highlight the key points, thereby enhancing the processing capability of local features, thereby ensuring the clear visibility of the lesion area, and improving the accuracy and reliability of the doctor's diagnosis during auxiliary diagnosis.

[0059] In the embodiment of the present application, the initial image blocks contained in the initial medical image are divided into main image blocks and sub-image blocks, and the main image blocks are enlarged and refined with a higher resolution than the sub-image blocks, so that on the basis of global refinement and enlargement of the entire image, the main and key local images corresponding to the main image blocks can achieve higher resolution and thus be clearer, so as to highlight the key points, thereby enhancing the processing capability of local features, thereby ensuring clear visibility of the lesion area, and improving the auxiliary diagnosis so that the doctor's diagnosis is more accurate and reliable. Especially when processing complex medical images, compared with traditional medical image resolution reconstruction methods, the resolution and clarity of complex medical images are greatly improved, showing higher reconstruction accuracy and efficiency of complex medical images, which can better meet the needs of clinical diagnosis and research, improve the technical level of digital medicine, and enable digital medicine to assist clinical diagnosis and research.

[0060] In one embodiment, see Figure 3 , Figure 3 This is a schematic diagram of the second sub-process of the medical image reconstruction method provided in the embodiment of the present application. Figure 3 As shown, in this embodiment, the initial medical image is converted into an image block sequence to obtain an initial image block sequence, including:

[0061] S31, obtaining the image type corresponding to the initial medical image;

[0062] S32, identifying key image blocks contained in the initial medical image according to the image type, obtaining main image blocks, and marking the main image blocks;

[0063] S33. Identify non-critical image blocks contained in the initial medical image according to the image type, obtain secondary image blocks, and label the secondary image blocks.

[0064] Explanatory, for different types of initial medical images including but not limited to CT, MRI, X-ray, etc., for the same type of initial medical images, since the image pattern and format are fixed, generally speaking, the position of the main, important, and key image parts related to the lesions in the initial medical images is also fixed. This part is more important and critical than other parts for clinical diagnosis and research. Based on this, the initial medical images can be classified according to attributes such as but not limited to CT, MRI, X-ray, etc. For the attributes for classifying the initial medical images, as long as the attributes can distinguish different types of initial medical images, there is no limitation here. Then, the image blocks contained in each type of initial medical image are divided into main image blocks and sub-image blocks. When training the preset generative pre-trained conversion model, the main image blocks and sub-image blocks contained in the training samples can be mainly labeled by medical experts, so that the preset generative pre-trained conversion model can identify the main image blocks and sub-image blocks corresponding to each type of initial medical image including but not limited to CT, MRI, X-ray, etc.

[0065] When converting the initial medical image into an image block sequence based on a preset generative pre-trained conversion model, the image type corresponding to the initial medical image is obtained, and according to the image type, the key image blocks contained in the initial medical image are identified to obtain the main image block, and the main image block is annotated, wherein the main image block includes a plurality of image blocks, and according to the image type, the non-key image blocks contained in the initial medical image are identified to obtain the sub-image block, and the sub-image block is annotated, wherein the sub-image block includes a plurality of image blocks, thereby dividing the image blocks obtained by dividing the initial medical image into main image blocks and sub-image blocks according to the image type. Since medical images of the same image type have the same attributes, and medical images with the same attributes have common features including but not limited to the same image pattern and format, therefore, based on the image type, the main image blocks and sub-image blocks contained in the initial medical image are identified. For the initial medical image of the same image type, the accuracy of the main image blocks and the sub-image blocks can be improved, thereby enhancing the key and main local feature processing capabilities corresponding to the main image blocks, and assisting doctors in more accurate identification and analysis of lesion sites.

[0066] The embodiments of the present application can improve the accuracy of the main image blocks and the sub-image blocks contained in the initial medical image according to the image type, thereby enhancing the key and main local feature processing capabilities corresponding to the main image blocks, and assisting doctors in more accurately identifying and analyzing the lesion sites. Especially when processing complex medical images, compared with traditional medical image resolution reconstruction methods, the resolution and clarity of complex medical images are greatly improved, showing higher reconstruction accuracy and efficiency of complex medical images, which can better meet the needs of clinical diagnosis and research, improve the technical level of digital medicine, and enable digital medicine to assist clinical diagnosis and research.

[0067] In one embodiment, before converting the initial medical image into an image block sequence to obtain the initial image block sequence, the method further includes:

[0068] Acquire a preset medical image training sample, wherein the preset medical image training sample includes a preset image type to which the preset medical image training sample belongs, and the preset medical image training sample is annotated with a preset main image block and a preset main image block identifier corresponding thereto, and a preset sub-image block and a preset sub-image block identifier corresponding thereto;

[0069] The preset generative pre-trained conversion model is trained using the preset medical image training samples.

[0070] Explanatory, a preset medical image training sample is obtained, the preset medical image training sample includes a preset image type to which the preset medical image training sample belongs, and the preset medical image training sample is annotated with a preset main image block and its corresponding preset main image block identifier, and a preset sub-image block and its corresponding preset sub-image block identifier. The preset medical image training sample is generally annotated by a medical expert, and then the preset medical image training sample is used to train the preset generative pre-trained conversion model, so that the preset generative pre-trained conversion model can, according to the preset image type, when the initial medical image is cut into image blocks to obtain the initial image block, perform main image block recognition on the initial image block and annotate it with the corresponding preset main image block identifier, perform sub-image block recognition on the initial image block and annotate it with the corresponding preset sub-image block identifier. When the image type is combined with the main and sub-image block division to perform recognition training of the main image block and the sub-image block, the accuracy of the preset generative pre-trained conversion model for the main image block and the sub-image block can be improved.

[0071] In the embodiment of the present application, by adopting preset medical image training samples that include preset image types and are marked with preset main image blocks and their corresponding preset main image block identifiers, and preset secondary image blocks and their corresponding preset secondary image block identifiers to train a preset generative pre-trained conversion model, the accuracy of the preset generative pre-trained conversion model for the main image blocks and the secondary image blocks can be improved, so that on the basis of global refinement and amplification of the entire image, the main and key local images corresponding to the main image blocks can achieve higher resolution and thus be clearer, so as to highlight the key points, enhance the processing capability of local features, thereby ensuring that the lesion area is clearly visible, and improve the auxiliary diagnosis so that the doctor's diagnosis is more accurate and reliable. Especially when processing complex medical images, compared with traditional medical image resolution reconstruction methods, the resolution and clarity of complex medical images are greatly improved, showing higher reconstruction accuracy and efficiency of complex medical images, which can better meet the needs of clinical diagnosis and research, improve the technical level of digital medicine, and enable digital medicine to assist clinical diagnosis and research.

[0072] In one embodiment, according to the position coding, all the target image blocks corresponding to the initial image block sequence are reconstructed into a whole image to obtain a target medical image, including:

[0073] Aligning all the target image blocks corresponding to the initial image block sequence according to the position code to obtain a first medical image;

[0074] Fusing all the target image blocks contained in the first medical image to obtain a second medical image;

[0075] All image blocks contained in the second medical image are combined into a complete image to obtain a target medical image.

[0076] Explanatory, since the position coding reflects the spatial structure information of the initial medical image, and there is a corresponding relationship between the initial image block and the target image block, the position coding can determine the corresponding position of the corresponding target image block in the initial medical image. Based on this, according to the position coding, all target image blocks corresponding to the initial image block sequence are aligned to place the target image block at the corresponding position of the initial image block in the initial medical image, thereby ensuring that each target image block is placed in the correct image area to obtain a first medical image. The spatial structure of the first medical image corresponds to and is consistent with the spatial structure of the initial medical image. Therefore, the first medical image is an image after the resolution of the initial medical image is enlarged and refined.

[0077] After placing each target image block in the correct image area, since there may be discontinuous edges or overlapping areas between different target image blocks, these target image blocks are fused using interpolation or weighted averaging methods to fuse all target image blocks contained in the first medical image, ensuring a smooth transition between adjacent target image blocks without obvious splicing marks, and obtaining a second medical image. All image blocks contained in the second medical image are then combined into a complete image, and the spatial structure of the initial medical image is restored to obtain a target medical image, which is an image after resolution enlargement and refinement corresponding to the spatial structure of the initial medical image.

[0078] In the embodiment of the present application, all target image blocks corresponding to the initial image block sequence are aligned and fused to reconstruct all target image blocks into a complete image. Compared with the simple splicing in the traditional technology, the context information between image blocks provided by the self-attention mechanism contained in the preset generative pre-trained conversion model can be fully utilized to adaptively adjust the fusion between adjacent target image blocks to ensure that the transition between adjacent target image blocks is smoother and more natural. At the same time, based on the global modeling capability of the preset generative pre-trained conversion model, the reconstructed medical image not only improves the clarity of the local area, but also ensures that the reconstructed medical image is consistent at the global level, so as to improve the auxiliary diagnosis and make the doctor's diagnosis more accurate and reliable. Especially when processing complex medical images, compared with the traditional medical image resolution reconstruction method, the resolution and clarity of the complex medical images are greatly improved, showing higher reconstruction accuracy and efficiency of complex medical images, which can better meet the needs of clinical diagnosis and research, improve the technical level of digital medicine, and enable digital medicine to assist clinical diagnosis and research.

[0079] In one embodiment, reconstructing all the target image blocks corresponding to the initial image block sequence into a whole image to obtain a target medical image includes:

[0080] Reconstructing all the target image blocks corresponding to the initial image block sequence into a whole block image to obtain an initial whole block image;

[0081] The initial whole block image is subjected to artifact removal and / or contrast enhancement to obtain a target medical image.

[0082] Explanatory, all target image blocks corresponding to the initial image block sequence are reconstructed into a whole block image to obtain an initial whole block image, and the initial whole block image is subjected to artifact removal and / or contrast enhancement to obtain a target medical image, wherein artifact removal can be achieved by, but not limited to, frequency domain filtering, regularization techniques (such as total variation regularization, etc.), and iterative reconstruction methods to reduce noise and artifacts while maintaining image edge features. Contrast enhancement can be achieved by, but not limited to, histogram equalization, contrast-limited adaptive histogram equalization (CLAHE), nonlinear mapping (such as gamma correction), and other methods to highlight image details.

[0083] In the embodiment of the present application, by reconstructing all the target image blocks corresponding to the initial image block sequence into a whole image, an initial whole image is obtained, and the initial whole image is de-artifacted and / or contrast enhanced, the image quality of the target medical image obtained by resolution reconstruction can be improved, so as to ensure the authenticity of the lesions and anatomical structures, avoid introducing new artifacts or distortions, and ensure the clear visibility of the lesion area, so as to improve the accuracy and reliability of the doctor's diagnosis during auxiliary diagnosis, so as to better assist clinical diagnosis and research, especially when processing complex medical images, compared with traditional medical image resolution reconstruction methods, the resolution and clarity of complex medical images are greatly improved, showing higher reconstruction accuracy and efficiency of complex medical images, which can better meet the needs of clinical diagnosis and research, improve the technical level of digital medicine, and enable digital medicine to assist clinical diagnosis and research.

[0084] It should be noted that the medical image reconstruction method described in the above embodiments can recombine the technical features contained in different embodiments as needed to obtain a combined implementation plan, but all of them are within the protection scope required by this application.

[0085] See also Figure 4 , Figure 4 A schematic block diagram of a medical image reconstruction device provided in an embodiment of the present application. Corresponding to the above-mentioned medical image reconstruction method, an embodiment of the present application also provides a medical image reconstruction device. Figure 4 As shown, the medical image reconstruction device includes a unit for executing the above-mentioned medical image reconstruction method, and the medical image reconstruction device can be configured in a computer device. Figure 4 The medical image reconstruction device 40 includes a first acquisition unit 41, a first conversion unit 42, a first enlargement unit 43 and a first construction unit 44.

[0086] Wherein, the first acquisition unit 41 is used to acquire an initial medical image;

[0087] A first conversion unit 42, configured to convert the initial medical image into an image block sequence based on a preset generative pre-trained conversion model to obtain an initial image block sequence, wherein the initial image block sequence includes initial image blocks and position codes of the initial image blocks in the initial medical image;

[0088] A first increasing unit 43, configured to increase the resolution of the initial image block to obtain a target image block;

[0089] The first constructing unit 44 is used to reconstruct all the target image blocks corresponding to the initial image block sequence into a whole image according to the position code to obtain a target medical image.

[0090] In one embodiment, the first conversion unit 42 includes:

[0091] A cutting subunit, used for cutting the initial medical image into image blocks to obtain initial image blocks;

[0092] A first labeling subunit is used to label the initial image block with a position code according to the position of the initial image block in the initial medical image, so as to obtain a position code corresponding to the initial image block;

[0093] A conversion subunit, used for converting the initial image block into an initial image block vector;

[0094] The arrangement subunit is used to arrange all the initial image block vectors into a sequence according to the position code to obtain an initial image block sequence corresponding to the initial medical image.

[0095] In one embodiment, the first increasing unit 43 includes:

[0096] A first acquisition subunit, used for acquiring the main image blocks and the auxiliary image blocks in all the initial image blocks;

[0097] The enlarging subunit is used to enlarge the main image block with a first resolution and to enlarge the auxiliary image block with a second resolution to obtain a target image block, wherein the first resolution is greater than the second resolution.

[0098] In one embodiment, the first conversion unit 42 includes:

[0099] A second acquisition subunit is used to acquire the image type corresponding to the initial medical image;

[0100] A second annotation subunit is used to identify key image blocks contained in the initial medical image according to the image type, obtain a main image block, and annotate the main image block;

[0101] The identification subunit is used to identify non-critical image blocks contained in the initial medical image according to the image type, obtain sub-image blocks, and mark the sub-image blocks.

[0102] In one embodiment, the medical image reconstruction device 40 further includes:

[0103] A second acquisition unit is used to acquire a preset medical image training sample, wherein the preset medical image training sample includes a preset image type to which the preset medical image training sample belongs, and the preset medical image training sample is annotated with a preset main image block and a preset main image block identifier corresponding thereto, and a preset sub-image block and a preset sub-image block identifier corresponding thereto;

[0104] A training unit is used to train the preset generative pre-trained conversion model using the preset medical image training samples.

[0105] In one embodiment, the first construction unit 44 includes:

[0106] an alignment subunit, configured to align all the target image blocks corresponding to the initial image block sequence according to the position code to obtain a first medical image;

[0107] a fusion subunit, configured to fuse all the target image blocks contained in the first medical image to obtain a second medical image;

[0108] The combining subunit is used to combine all the image blocks contained in the second medical image into a complete image to obtain a target medical image.

[0109] In one embodiment, the first construction unit 44 includes:

[0110] A reconstruction subunit, used for reconstructing all the target image blocks corresponding to the initial image block sequence into a whole block image to obtain an initial whole block image;

[0111] The modification subunit is used to remove artifacts and / or enhance the contrast of the initial whole image to obtain a target medical image.

[0112] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the above-mentioned medical image reconstruction device and each unit can refer to the corresponding description in the aforementioned method embodiment, and for the convenience and brevity of description, it will not be repeated here.

[0113] At the same time, the division and connection methods of the various units in the above-mentioned medical image reconstruction device are only used for illustration. In other embodiments, the medical image reconstruction device can be divided into different units as needed, and the various units in the medical image reconstruction device can also be connected in different orders and methods to complete all or part of the functions of the above-mentioned medical image reconstruction device.

[0114] The medical image reconstruction device can be implemented in the form of a computer program. Figure 5 Runs on the computer device shown.

[0115] See also Figure 5 , Figure 5 1 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device 500 may be a computer device such as a desktop computer or a server, or may be a component or part in other devices.

[0116] See also Figure 5 The computer device 500 includes a processor 502, a memory and a network interface 505 connected via a system bus 501, wherein the memory may include a non-volatile storage medium 503 and an internal memory 504, and the memory may also be a volatile storage medium.

[0117] The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. When the computer program 5032 is executed, the processor 502 can execute the above-mentioned medical image reconstruction method.

[0118] The processor 502 is used to provide computing and control capabilities to support the operation of the entire computer device 500 .

[0119] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute the above-mentioned medical image reconstruction method.

[0120] The network interface 505 is used to communicate with other devices over the network. Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the present application scheme, and does not constitute a limitation on the computer device 500 to which the present application scheme is applied. The specific computer device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components. For example, in some embodiments, the computer device may only include a memory and a processor. In such an embodiment, the structure and function of the memory and the processor are the same as those of the present application scheme. Figure 5 The embodiments shown are consistent and will not be described again here.

[0121] The processor 502 is used to run a computer program 5032 stored in a memory to implement the following steps: obtaining an initial medical image; based on a preset generative pre-trained conversion model, converting the initial medical image into an image block sequence to obtain an initial image block sequence, wherein the initial image block sequence includes an initial image block and a position code of the initial image block in the initial medical image; increasing the resolution of the initial image block to obtain a target image block; and according to the position code, reconstructing all the target image blocks corresponding to the initial image block sequence into a whole image to obtain a target medical image.

[0122] In one embodiment, when converting the initial medical image into an image block sequence to obtain an initial image block sequence, the processor 502 specifically implements the following steps:

[0123] Performing image block cutting on the initial medical image to obtain initial image blocks;

[0124] According to the position of the initial image block in the initial medical image, the initial image block is marked with a position code to obtain a position code corresponding to the initial image block;

[0125] Converting the initial image block into an initial image block vector;

[0126] According to the position coding, all the initial image block vectors are arranged into a sequence to obtain an initial image block sequence corresponding to the initial medical image.

[0127] In one embodiment, when the processor 502 increases the resolution of the initial image block to obtain the target image block, the processor 502 specifically implements the following steps:

[0128] Acquire the main image blocks and the auxiliary image blocks in all the initial image blocks;

[0129] The main image block is enlarged by using a first resolution, and the auxiliary image block is enlarged by using a second resolution to obtain a target image block, wherein the first resolution is greater than the second resolution.

[0130] In one embodiment, when converting the initial medical image into an image block sequence to obtain an initial image block sequence, the processor 502 specifically implements the following steps:

[0131] Acquire the image type corresponding to the initial medical image;

[0132] According to the image type, identifying key image blocks contained in the initial medical image, obtaining main image blocks, and marking the main image blocks;

[0133] According to the image type, non-critical image blocks included in the initial medical image are identified to obtain secondary image blocks, and the secondary image blocks are labeled.

[0134] In one embodiment, before converting the initial medical image into an image block sequence to obtain an initial image block sequence, the processor 502 further implements the following steps:

[0135] Acquire a preset medical image training sample, wherein the preset medical image training sample includes a preset image type to which the preset medical image training sample belongs, and the preset medical image training sample is annotated with a preset main image block and a preset main image block identifier corresponding thereto, and a preset sub-image block and a preset sub-image block identifier corresponding thereto;

[0136] The preset generative pre-trained conversion model is trained using the preset medical image training samples.

[0137] In one embodiment, when the processor 502 reconstructs all the target image blocks corresponding to the initial image block sequence into a whole image according to the position coding to obtain the target medical image, the processor 502 specifically implements the following steps:

[0138] Aligning all the target image blocks corresponding to the initial image block sequence according to the position code to obtain a first medical image;

[0139] Fusing all the target image blocks contained in the first medical image to obtain a second medical image;

[0140] All image blocks contained in the second medical image are combined into a complete image to obtain a target medical image.

[0141] In one embodiment, when the processor 502 reconstructs all the target image blocks corresponding to the initial image block sequence into a whole image to obtain a target medical image, the processor 502 specifically implements the following steps:

[0142] Reconstructing all the target image blocks corresponding to the initial image block sequence into a whole block image to obtain an initial whole block image;

[0143] The initial whole block image is subjected to artifact removal and / or contrast enhancement to obtain a target medical image.

[0144] It should be understood that in the embodiment of the present application, the processor 502 may be a central processing unit (CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0145] Those skilled in the art will appreciate that all or part of the processes in the method for implementing the above embodiment can be completed by a computer program, which can be stored in a computer-readable storage medium. The computer program is executed by at least one processor in the computer system to implement the process steps of the embodiment of the above method.

[0146] Therefore, the present application also provides a computer-readable storage medium. The computer-readable storage medium may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, and the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor executes the following steps:

[0147] A computer program product, when running on a computer, enables the computer to execute the steps of the medical image reconstruction method described in the above embodiments.

[0148] The computer-readable storage medium may be an internal storage unit of the aforementioned device, such as a hard disk or memory of the device. The computer-readable storage medium may also be an external storage device of the device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit of the device and an external storage device.

[0149] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0150] The storage medium is a physical, non-transient storage medium, for example, it can be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk, etc., which can store computer programs.

[0151] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0152] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0153] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs. The units in the device of the embodiment of the present application can be combined, divided and deleted according to actual needs. In addition, the functional units in the various embodiments of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0154] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for an electronic device (which can be a personal computer, terminal, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application.

[0155] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A medical image reconstruction method, characterized in that: include: Obtain initial medical images; Based on a preset generative pre-trained conversion model, the initial medical image is converted into an image block sequence to obtain an initial image block sequence, wherein the initial image block sequence includes initial image blocks and position codes of the initial image blocks in the initial medical image; Increasing the resolution of the initial image block to obtain a target image block; According to the position coding, all the target image blocks corresponding to the initial image block sequence are reconstructed into a whole image to obtain a target medical image.

2. The medical image reconstruction method according to claim 1, characterized in that: Converting the initial medical image into an image block sequence to obtain an initial image block sequence includes: Performing image block cutting on the initial medical image to obtain initial image blocks; According to the position of the initial image block in the initial medical image, the initial image block is marked with a position code to obtain a position code corresponding to the initial image block; Converting the initial image block into an initial image block vector; According to the position coding, all the initial image block vectors are arranged into a sequence to obtain an initial image block sequence corresponding to the initial medical image.

3. The medical image reconstruction method according to claim 1 or 2, characterized in that: Increasing the resolution of the initial image block to obtain a target image block, comprising: Acquire the main image blocks and the auxiliary image blocks in all the initial image blocks; The main image block is enlarged by using a first resolution, and the auxiliary image block is enlarged by using a second resolution to obtain a target image block, wherein the first resolution is greater than the second resolution.

4. The medical image reconstruction method according to claim 3, characterized in that: Converting the initial medical image into an image block sequence to obtain an initial image block sequence includes: Acquire the image type corresponding to the initial medical image; According to the image type, identifying key image blocks contained in the initial medical image, obtaining main image blocks, and marking the main image blocks; According to the image type, non-critical image blocks included in the initial medical image are identified to obtain secondary image blocks, and the secondary image blocks are labeled.

5. The medical image reconstruction method according to claim 4, characterized in that: Before converting the initial medical image into an image block sequence to obtain the initial image block sequence, the method further includes: Acquire a preset medical image training sample, wherein the preset medical image training sample includes a preset image type to which the preset medical image training sample belongs, and the preset medical image training sample is annotated with a preset main image block and a preset main image block identifier corresponding thereto, and a preset sub-image block and a preset sub-image block identifier corresponding thereto; The preset generative pre-trained conversion model is trained using the preset medical image training samples.

6. The medical image reconstruction method according to claim 1, characterized in that: According to the position code, all the target image blocks corresponding to the initial image block sequence are reconstructed into a whole image to obtain a target medical image, including: Aligning all the target image blocks corresponding to the initial image block sequence according to the position code to obtain a first medical image; Fusing all the target image blocks contained in the first medical image to obtain a second medical image; All image blocks contained in the second medical image are combined into a complete image to obtain a target medical image.

7. The medical image reconstruction method according to claim 1 or 6, characterized in that: Reconstructing all the target image blocks corresponding to the initial image block sequence into a whole image to obtain a target medical image, including: Reconstructing all the target image blocks corresponding to the initial image block sequence into a whole block image to obtain an initial whole block image; The initial whole block image is subjected to artifact removal and / or contrast enhancement to obtain a target medical image.

8. A medical image reconstruction device, characterized in that: include: A first acquisition unit, used for acquiring an initial medical image; A first conversion unit, configured to convert the initial medical image into an image block sequence based on a preset generative pre-trained conversion model to obtain an initial image block sequence, wherein the initial image block sequence includes initial image blocks and position codes of the initial image blocks in the initial medical image; A first enlarging unit, configured to enlarge the resolution of the initial image block to obtain a target image block; The first construction unit is used to reconstruct all the target image blocks corresponding to the initial image block sequence into a whole image according to the position code to obtain a target medical image.

9. A computer device, characterized in that: The computer device comprises a memory and a processor connected to the memory; the memory is used to store a computer program; the processor is used to run the computer program to execute the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 can be implemented.

Images