Medical image generation method and device, equipment and storage medium

Abstract

The embodiment of the invention provides a medical image generation method and device, equipment and a storage medium, and relates to the technical field of image processing. The method comprises the following steps: inputting image medical parameters into an image parameter prediction model for data processing to obtain an image parameter vector, obtaining a plurality of coordinate parameters corresponding to a target medical image, inputting the coordinate parameters and the image parameter vector into a pixel mapping model for pixel value prediction to obtain a predicted pixel value of each coordinate parameter, and filling the predicted pixel value to obtain a target medical image, if the coordinate parameter is a three-dimensional coordinate, determining that the target medical image is a three-dimensional image, otherwise, determining that the target medical image is a two-dimensional image. An image parameter vector capable of guiding a generation process is generated by using specific image medical conditions, and pixel value prediction and image generation can be directly performed in a space according to specific coordinate parameters no matter a two-dimensional mode or a three-dimensional mode, so that the method is suitable for various different mode requirements.

Description

Technical Field

[0001] This application relates to the field of image processing technologies, and in particular, to methods, devices, equipment, and storage media for generating medical imaging images. Background Art

[0002] With the development of artificial intelligence and deep learning technologies, analyzing medical images using medical imaging-related models has become an indispensable technical means in the fields of medical research, clinical disease diagnosis, and treatment. However, the training performance of medical imaging-related models highly depends on a large amount of training data directly related to them, but the acquisition cost of medical imaging training data is relatively high.

[0003] In related technologies, generative models such as diffusion models are usually used to generate images based on base images to provide more abundant training data. However, medical imaging images include two-dimensional and three-dimensional modalities, and most of these generative methods are only applicable to a single generative task. For images of different modalities, different generative models need to be trained separately. Moreover, since higher resolution is usually required in the three-dimensional modality, different generative models are often cascaded during the generation process to gradually improve the resolution of the generated images, resulting in low generation efficiency. Summary of the Invention

[0004] The main objective of the embodiments of this application is to propose methods, devices, equipment, and storage media for generating medical imaging images, which can generate medical imaging images of different modalities simultaneously and improve the efficiency of image generation in the three-dimensional modality.

[0005] To achieve the above objective, a first aspect of the embodiments of this application proposes a method for generating medical imaging images, which is executed by a medical imaging image generation model. The medical imaging image generation model includes an image parameter prediction model and a pixel mapping model. The method includes:

[0006] Obtain image medical parameters indicating the generation direction of the target medical imaging image, and input the image medical parameters into the image parameter prediction model for data processing to obtain an image parameter vector;

[0007] Obtain a plurality of coordinate parameters corresponding to the target medical imaging image, and input the coordinate parameters and the image parameter vector into the pixel mapping model for pixel value prediction to obtain a predicted pixel value corresponding to each coordinate parameter;

[0008] Fill the predicted pixel values to obtain the target medical imaging image. If the coordinate parameters are three-dimensional coordinates, the target medical imaging image is a three-dimensional image; otherwise, the target medical imaging image is a two-dimensional image.

[0009] In some embodiments, the training steps of the medical image generation model at least include:

[0010] Initialize the mapping system parameters of the pixel mapping model and the prediction system parameters of the image parameter prediction model;

[0011] Obtain at least one medical image sample image, generate sample coordinate parameters corresponding to the medical image sample image, and initialize the image parameter sample vector, where the medical image sample image is a two-dimensional sample image or a three-dimensional sample image;

[0012] Update the mapping system parameters and the prediction system parameters based on the medical image sample image, the image parameter sample vector, and the sample coordinate parameters until the trained image parameter prediction model and the pixel mapping model are obtained.

[0013] In some embodiments, the updating the mapping system parameters and the prediction system parameters based on the medical image sample image, the image parameter sample vector, and the sample coordinate parameters until the trained image parameter prediction model and the pixel mapping model are obtained includes:

[0014] Fix the mapping system parameters, input the image parameter sample vector and the sample coordinate parameters into the pixel mapping model for prediction, and generate a first predicted image based on the prediction result;

[0015] Calculate a first loss value based on the medical image sample image and the first predicted image, adjust the prediction system parameters based on the first loss value, and use the updated image parameter prediction model to predict the updated image parameter sample vector until a first training termination condition is reached, obtaining the trained image parameter prediction model;

[0016] Fix the prediction system parameters, input the image parameter sample vector and the sample coordinate parameters into the pixel mapping model for prediction, and generate a second predicted image based on the prediction result;

[0017] Calculate a second loss value based on the medical image sample image and the second predicted image, adjust the mapping system parameters based on the second loss value until a second training termination condition is reached, obtaining the trained pixel mapping model.

[0018] In some embodiments, the image parameter prediction model at least includes a feature extraction network, and the obtaining the updated image parameter vector by using the updated image parameter prediction model includes:

[0019] Obtain the medical condition parameters of the medical image sample image and the conditional embedding features corresponding to the medical condition parameters;

[0020] Obtain the Gaussian noise feature corresponding to the current time step and the time embedding feature corresponding to the current time step;

[0021] Accumulate the time embedding feature and the conditional embedding feature to obtain a reference feature, and jointly input the reference feature and the Gaussian noise feature into the feature extraction network for data processing to obtain the image parameter vector.

[0022] In some embodiments, the inputting the coordinate parameter and the image parameter vector into the pixel mapping model for pixel value prediction to obtain a predicted pixel value corresponding to each coordinate parameter includes:

[0023] Input the coordinate parameter into a first linear layer for feature extraction to obtain a first eigenvalue;

[0024] Perform feature fusion on the first eigenvalue and the image parameter vector to obtain a second eigenvalue;

[0025] Input the second eigenvalue into a second linear layer for feature extraction to obtain a third eigenvalue;

[0026] Perform feature fusion on the third eigenvalue and the image parameter vector to obtain a fourth eigenvalue;

[0027] Input the fourth eigenvalue into an output layer for prediction to obtain the predicted pixel value.

[0028] In some embodiments, the obtaining a plurality of coordinate parameters corresponding to the target medical image includes:

[0029] Obtain the target resolution of the target medical image, and divide the target medical image into a plurality of coordinate grids according to the target resolution;

[0030] Select an indication coordinate position from each coordinate grid to obtain the coordinate parameter.

[0031] In some embodiments, the obtaining the target resolution of the target medical image and dividing the target medical image into a plurality of coordinate grids according to the target resolution includes:

[0032] Obtain the regional resolution of different image regions of the target medical image, and the target resolution includes all the regional resolutions;

[0033] Based on the regional resolution, divide the corresponding image region into regional grids, and obtain all the regional grids to obtain the coordinate grids, and the sizes of the regional grids in different image regions are not exactly the same.

[0034] To achieve the above object, a second aspect of the embodiments of the present application provides a medical image generation device, which is executed by a medical image generation model. The medical image generation model includes an image parameter prediction model and a pixel mapping model. The device includes:

[0035] An image parameter acquisition module: configured to acquire image medical parameters indicating the generation direction of a target medical image, and input the image medical parameters into the image parameter prediction model for data processing to obtain an image parameter vector;

[0036] A pixel prediction module: configured to acquire a plurality of coordinate parameters corresponding to the target medical image, input the coordinate parameters and the image parameter vector into the pixel mapping model for pixel value prediction, and obtain a predicted pixel value corresponding to each of the coordinate parameters;

[0037] An image generation module: configured to fill the predicted pixel values to obtain the target medical image. If the coordinate parameters are three-dimensional coordinates, the target medical image is a three-dimensional image; otherwise, the target medical image is a two-dimensional image.

[0038] To achieve the above object, a third aspect of the embodiments of the present application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the method described in the first aspect above is implemented.

[0039] To achieve the above object, a fourth aspect of the embodiments of the present application provides a storage medium, which is a storage medium that stores a computer program. When the computer program is executed by a processor, the method described in the first aspect above is implemented.

[0040] The medical image generation method, device, equipment, and storage medium proposed in the embodiments of this application are executed by a medical image generation model. The medical image generation model includes an image parameter prediction model and a pixel mapping model. By obtaining image medical parameters indicating the generation direction of the target medical image, and inputting the image medical parameters into the image parameter prediction model for data processing to obtain an image parameter vector, obtaining multiple coordinate parameters corresponding to the target medical image, inputting the coordinate parameters and the image parameter vector into the pixel mapping model for pixel value prediction, obtaining the predicted pixel value corresponding to each coordinate parameter, and filling the predicted pixel values to obtain the target medical image. If the coordinate parameters are three-dimensional coordinates, the target medical image is a three-dimensional image; otherwise, the target medical image is a two-dimensional image. In the embodiments of this application, specific image medical conditions are used to generate an image parameter vector that can guide the generation process. Then, according to different modality generation requirements, the corresponding two-dimensional or three-dimensional coordinate parameters are determined, and these coordinate parameters are used to define the generation area of the target medical image. Next, the pixel values of the coordinate parameters are predicted using the image parameter vector to achieve the generation process. During the entire operation process, there is no need to input a base image. Whether it is a two-dimensional modality or a three-dimensional modality, pixel value prediction and image generation can be directly performed in space according to the preset logic, so it is applicable to various different modality requirements. In addition, since the three-dimensional coordinate parameters corresponding to the target medical image are obtained and pixel value prediction is directly performed in the three-dimensional space, the error accumulation situation that may occur in the cascade generation method is effectively avoided. At the same time, the computational complexity is reduced, the processing time is reduced, and the efficiency of image generation in the three-dimensional modality is significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a schematic diagram of the medical image generation model provided by the embodiments of this application.

[0042] Figure 2 is a flowchart of the medical image generation method provided by the embodiments of this application.

[0043] Figure 3 is a schematic diagram of the structure of the image parameter prediction model provided by the embodiments of this application.

[0044] Figure 4 is a flowchart of inputting the coordinate parameters and the image parameter vector into the pixel mapping model for pixel value prediction to obtain the predicted pixel value corresponding to each coordinate parameter provided by the embodiments of this application.

[0045] Figure 5 is a schematic diagram of the pixel mapping model provided by the embodiments of this application.

[0046] Figure 6 is a flowchart of obtaining multiple coordinate parameters corresponding to the target medical image provided by the embodiments of this application.

[0047] Figure 7 It is a flowchart of the training steps of the medical image generation model provided by the embodiments of the present application.

[0048] Figure 8 It is a schematic diagram of the training process provided by the embodiments of the present application.

[0049] Figure 9 It is a flowchart of updating the mapping system parameters and the prediction system parameters based on the medical image sample images, the image parameter sample vectors and the sample coordinate parameters until the trained image parameter prediction model and the pixel mapping model are obtained, provided by the embodiments of the present application.

[0050] Figure 10 It is a flowchart of obtaining the updated image parameter vector by using the updated image parameter prediction model, provided by the embodiments of the present application.

[0051] Figure 11 It is a block diagram of the structure of the medical image generation device provided by another embodiment of the present application.

[0052] Figure 12 It is a schematic diagram of the hardware structure of the electronic device provided by the embodiments of the present application. Detailed implementation manners

[0053] In order to make the objectives, technical solutions and advantages of the present application more clear and understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0054] It should be noted that although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order from the module division in the device or the order in the flowchart.

[0055] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0056] First, several nouns involved in the present application are analyzed:

[0057] Artificial Intelligence (AI): It is a new technical science that studies and develops theories, methods, technologies, and application systems for simulating, extending, and expanding human intelligence; artificial intelligence is a branch of computer science. Artificial intelligence attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a way similar to human intelligence. The research in this field includes robots, speech recognition, image recognition, natural language processing, and expert systems, etc. Artificial intelligence can simulate the information process of human consciousness and thinking. Artificial intelligence also uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, sense the environment, acquire knowledge, and use knowledge to obtain the best results in terms of theories, methods, technologies, and application systems.

[0058] With the development of artificial intelligence and deep learning technologies, using medical image-related models to analyze medical images has become an indispensable technical means in the fields of medical research, clinical disease diagnosis, and treatment. However, the training performance of medical image-related models highly depends on a large amount of training data directly related to them, but the acquisition cost of medical image training data is relatively high.

[0059] In related technologies, generative models such as diffusion models are usually used to generate images based on the base image to provide more abundant training data. However, medical image images include two-dimensional and three-dimensional modalities, and most of these generative methods are only applicable to a single generative task. For images of different modalities, different generative models need to be trained separately. Moreover, since there are usually high requirements for resolution in the three-dimensional modality, during the generation process, different generative models are often cascaded to gradually improve the resolution of the generated image, which leads to cumulative errors and low generation efficiency at the same time.

[0060] Based on this, the embodiments of the present application provide a method, device, equipment, and storage medium for generating medical image images, which use specific image medical conditions to generate an image parameter vector that can guide the generation process. Then, according to different modality generation requirements, the corresponding two-dimensional or three-dimensional coordinate parameters are determined, and these coordinate parameters are used to define the generation area of the target medical image image. Next, the image parameter vector is used to predict the pixel values of the coordinate parameters to achieve the generation process. During the entire operation process, there is no need to input the base image, and both two-dimensional and three-dimensional modalities can directly perform pixel value prediction and image generation in space according to the preset logic, so it is applicable to various different modality requirements. In addition, since the three-dimensional coordinate parameters corresponding to the target medical image image are obtained and pixel value prediction is directly performed in the three-dimensional space, the error accumulation situation that may occur in the cascaded generation method is effectively avoided. At the same time, the computational complexity is reduced, the processing time is reduced, and the efficiency of image generation in the three-dimensional modality is significantly improved.

[0061] Embodiments of the present application provide a method, apparatus, device, and storage medium for generating medical imaging images, which will be specifically described through the following embodiments. First, the method for generating medical imaging images in the embodiments of the present application will be described.

[0062] Embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, Artificial Intelligence (AI) is a theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a way similar to human intelligence. Artificial intelligence also studies the design principles and implementation methods of various intelligent machines, enabling the machines to have the functions of perception, reasoning, and decision-making.

[0063] Artificial intelligence technology is an interdisciplinary subject with a wide range of fields, including both hardware-level technologies and software-level technologies. Basic artificial intelligence technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation / interaction systems, and mechatronics. Artificial intelligence software technologies mainly include several major directions such as computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0064] The method for generating medical imaging images provided by the embodiments of the present application relates to the field of image processing technology. The method for generating medical imaging images provided by the embodiments of the present application can be applied to a terminal, or to a server, or can be a computer program running on a terminal or a server. For example, the computer program can be a native program or software module in an operating system; it can be a local (Native) application (Application, APP), that is, a program that needs to be installed in an operating system to run, such as a client that supports the generation of medical imaging images, that is, a program that only needs to be downloaded to a browser environment to run; it can also be a small program that can be embedded in any APP. In short, the above computer program can be any form of application program, module, or plug-in. Among them, the terminal communicates with the server through a network. The method for generating medical imaging images can be executed by the terminal or the server, or by the terminal and the server in cooperation.

[0065] In some embodiments, the terminal may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart watch, or the like. The server may be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms; it may also be a service node in a blockchain system, and the service nodes in the blockchain system form a Peer To Peer (P2P) network, and the P2P protocol is an application layer protocol running on top of the Transmission Control Protocol (TCP) protocol. The terminal and the server may be connected through communication connection means such as Bluetooth, Universal Serial Bus (USB), or a network, and this embodiment does not limit this here.

[0066] This application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and so on. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This application can also be practiced in a distributed computing environment where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0067] First, the medical image generation model in the embodiments of this application will be described below.

[0068] In one embodiment, referring to Figure 1 , Figure 1 is a schematic diagram of the medical image generation model provided by the embodiments of this application. Among them, the medical image generation model includes an image parameter prediction model and a pixel mapping model. First, the image parameter prediction model generates an image parameter vector based on the image medical parameters, and then the pixel mapping model directly generates a three-dimensional or two-dimensional target medical image based on the image parameter vector according to the coordinate parameters.

[0069] Next, in Figure 1Based on this, the medical image generation method in the embodiments of the present application is described.

[0070] In one embodiment, referring to Figure 2 , Figure 2 FIG. is an optional flowchart of the medical image generation method provided by the embodiments of the present application. Figure 2 The method in may include but is not limited to steps 110 to 130. At the same time, it can be understood that the order of steps 110 to 130 in this embodiment is not specifically limited, and the order of steps can be adjusted according to actual needs, or some steps can be reduced or added. Figure 2 In

[0071] Step 110: Obtain image medical parameters for indicating the generation direction of the target medical image, and input the image medical parameters into the image parameter prediction model for data processing to obtain an image parameter vector.

[0072] In one embodiment, the image medical parameters are used to indicate the generation direction of the target medical image. The image medical conditions may include the imaging site (such as the heart, lungs, brain, etc.), the desired imaging quality (such as resolution, contrast, etc.), and the imaging technology used (such as X-ray imaging, magnetic resonance imaging, ultrasonic imaging, etc.). For example, "generate an X-ray chest radiograph with a certain resolution", "generate a fundus pathological section", "generate a three-dimensional lung CT image", etc. Different image medical parameters can indicate different types of target medical images. It can be understood that the image medical parameters can be set according to actual needs, and this embodiment is only for illustration and does not limit its specific form and content.

[0073] Taking "generate an X-ray chest radiograph with a certain resolution" as an example, according to this image medical parameter, it has a clear directivity. According to this image medical parameter, it can be determined that the target medical image will at least contain information related to the chest position and is presented as a grayscale image. The morphology and density of tissues and organs in the chest are reflected through the display of different grayscales, and the size of the image is limited by the resolution. Another example is the image medical parameter of "generate a fundus pathological section", which indicates that the generated target medical image focuses on the fundus tissue and can clearly show the microscopic pathological changes of fundus structures such as the retina and choroid.

[0074] In one embodiment, after having the image medical parameters, input them into the image parameter prediction model for data processing to obtain an image parameter vector M. Referring to Figure 3 , Figure 3It is a schematic structural diagram of an image parameter prediction model provided by an embodiment of the present application. Among them, the image parameter prediction model includes: a conditional embedding layer, a temporal embedding layer, and a feature extraction network. Here, both the conditional embedding layer and the temporal embedding layer can be a convolutional neural network or a Transformer architecture, and the feature extraction network can be generated by a UNET network structure.

[0075] Referring to Figure 3 , after receiving the image medical parameters, the image parameter prediction model inputs the image medical parameters into the conditional embedding layer for feature extraction to obtain medical features. Next, the current time information is obtained and input into the temporal embedding layer for feature extraction to obtain temporal features. At the same time, a noise feature corresponding to the current time information is generated, and this noise feature can be obtained according to Gaussian noise. After adding the medical feature and the temporal feature, the added result and the noise feature are simultaneously input into the feature extraction network for feature extraction to obtain the medical parameter vector M.

[0076] In one embodiment, the current time information can reflect the physiological state changes at different time points, the development process of the disease, and the imaging time sequence, etc. After obtaining the current time information, it is input into the temporal embedding layer, and the temporal embedding layer adopts a temporal encoding technique to convert the time information into continuous temporal features with time series characteristics. This encoding method can accurately capture the continuity and periodicity of time, enabling the image parameter prediction model to deeply learn the influence of time information in the process of generating the target medical image.

[0077] In one embodiment, the medical parameter vector can indicate the image features of the target medical image. Among them, according to the actual generation requirements, the medical parameter vector can be pixel-related parameters, such as indicating the intensity value, variance, histogram, etc. of each pixel in the target medical image, and the basic visual features of the target medical image can be described according to the pixel-related parameters. The medical parameter vector can also be feature extraction parameters, such as edge intensity and direction information, the number and position information of corner points related to corner detection, local feature descriptors, etc. Among them, the edge intensity and direction information can indicate the contour of the object in the generated target medical image, the number and position information of corner points can locate key feature points in the process of generating the target medical image, and the local feature descriptor can indicate specific image targets. Therefore, the target medical image that meets the image medical parameters can be guided to be generated through the medical parameter vector.

[0078] Step 120: Obtain multiple coordinate parameters corresponding to the target medical image, input the coordinate parameters and the image parameter vector into the pixel mapping model for pixel value prediction, and obtain the predicted pixel value corresponding to each coordinate parameter.

[0079] In one embodiment, for the target medical image to be generated, it can be regarded as an image composed of pixel points, and each pixel point corresponds to a coordinate value. In this case, multiple coordinate parameters corresponding to the target medical image can be obtained. Specifically, if the target medical image is a two-dimensional image, such as common X-ray images, ultrasound images, etc., the Cartesian coordinate system is adopted. This coordinate system consists of a horizontal axis (x-axis) and a vertical axis (y-axis) that are perpendicular to each other. Among them, the x value represents the position of the pixel point in the horizontal direction, and the y value represents its position in the vertical direction. The corresponding coordinate parameter is a two-dimensional coordinate value, which can be expressed as (x, y). When the target medical image is a three-dimensional image, such as a three-dimensional organ model reconstructed by CT or a three-dimensional brain image generated by MRI, a vertical axis (z-axis) perpendicular to the xy plane is additionally added on the basis of the two-dimensional Cartesian coordinate system. At this time, the coordinate parameter becomes a three-dimensional coordinate value, such as (x, y, z). In the embodiment of the present application, three-dimensional coordinate values are used to represent the coordinate parameters, and it is set that when z = 0, it means that the coordinate parameter corresponds to a two-dimensional coordinate value.

[0080] In one embodiment, referring to Figure 4 , Figure 4 is a flowchart of predicting pixel values corresponding to each coordinate parameter by inputting coordinate parameters and an image parameter vector into a pixel mapping model provided by the embodiment of the present application, which specifically includes the following steps:

[0081] Step 410: Input the coordinate parameter into the first linear layer for feature extraction to obtain a first eigenvalue.

[0082] In one embodiment, referring to Figure 5 , Figure 5 is a schematic diagram of the pixel mapping model provided by the embodiment of the present application. Among them, the pixel mapping model sequentially includes: a first linear layer, a second linear layer, a third linear layer, and an output layer. Specifically, the first linear layer receives the coordinate parameter for feature processing and performs a linear transformation on the input coordinate parameter through a series of weight matrices and bias vectors. When the coordinate parameter is input into the first linear layer, the coordinate parameter performs a matrix multiplication operation with a pre-trained weight matrix and adds a bias vector to convert the originally simple position information into a more representative numerical representation that can better reflect the internal characteristics of the medical image, and obtain a first eigenvalue. For example, for the coordinate parameters related to a lung CT image, after being processed by the first linear layer, the first eigenvalue may be able to indicate information such as the relative relationship of different regions of the lung in terms of spatial position and the potential connection between these positions and possible lesions.

[0083] Step 420: Perform feature fusion on the first eigenvalue and the image parameter vector to obtain a second eigenvalue.

[0084] In one embodiment, since the image parameter vector contains the image feature information of the target medical image, the first eigenvalue and the image parameter vector are subjected to feature fusion to integrate information from different dimensions. The feature fusion here can be concatenation or element-wise addition to obtain a second eigenvalue. The obtained second eigenvalue synthesizes the spatial position information reflected by the coordinate parameters and the essential image features carried by the image parameter vector, facilitating the subsequent generation of a more accurate target medical image. For example, in brain MRI image generation, the second eigenvalue obtained by fusing the first eigenvalue reflecting the positions of different regions of the brain with the image parameter vector containing information such as MRI imaging sequences and contrast can more accurately depict the characteristic manifestations of different regions of the brain under specific imaging conditions.

[0085] Step 430: Input the second eigenvalue into the second linear layer for feature extraction to obtain a third eigenvalue.

[0086] In one embodiment, referring to Figure 5 , the second linear layer is used to further extract and optimize the features of the second eigenvalue. Similar to the first linear layer, the second linear layer performs a linear transformation on the second eigenvalue through a weight matrix and a bias vector to obtain a third eigenvalue.

[0087] Step 440: Perform feature fusion on the third eigenvalue and the image parameter vector to obtain a fourth eigenvalue.

[0088] In one embodiment, fusing the image parameter vector again is to further strengthen the guiding role of the image parameter vector in the entire analysis process and fully explore the potential connection between the third eigenvalue and the image parameter vector. The feature fusion here can also adopt methods such as concatenation and element-wise addition, and then through a non-linear transformation to obtain a fourth eigenvalue. The fourth eigenvalue not only integrates the spatial position information and the essential features of the image, but also fuses and optimizes this information at a higher level, improving the accuracy of the prediction result.

[0089] Step 450: Input the fourth eigenvalue into the output layer for prediction to obtain a predicted pixel value.

[0090] In one embodiment, referring to Figure 5 , after obtaining the fourth eigenvalue, it is input into the output layer for prediction. Among them, the role of the output layer is to convert the fourth eigenvalue obtained through multiple layers of processing and fusion into the final prediction result, that is, the predicted pixel value. The output layer here can adopt a specific activation function, such as the softmax function or the sigmoid function, and use the output layer to calculate the possible value probabilities or specific values of each pixel point of the target medical image according to the fourth eigenvalue.

[0091] In one embodiment, since the pixel value prediction in the embodiments of the present application directly predicts the coordinate values of coordinate parameters in a two-dimensional space or a three-dimensional space, that is, directly generates an image. Therefore, during the generation process, the image resolution can be arbitrarily adjusted by adjusting the mapping size of the coordinate parameters on the target medical image. Refer to Figure 6 , Figure 6 is a flowchart for obtaining a plurality of coordinate parameters corresponding to a target medical image provided by the embodiments of the present application, specifically including the following steps:

[0092] Step 610: Obtain the target resolution of the target medical image, and divide the target medical image into a plurality of coordinate grids according to the target resolution.

[0093] In one embodiment, the target resolution of the target medical image can be set according to actual generation requirements. Among them, the same resolution can be used for the entire image. At this time, each part of the target medical image will be presented with the same pixel density, ensuring the overall consistency and uniformity of the image. In addition, different resolutions can also be set according to different regions. For example, some unimportant regions located at the edge can use a lower resolution to save computing power, while a higher resolution can be used in important regions to improve the generation accuracy. For example, in a lung CT image, the edge region of the lung is mainly some relatively regular thoracic tissues, which contribute less to the key information for diagnosis. Therefore, a lower resolution can be used to reduce the number of pixels, thereby saving computing power and storage resources. For the central region of the lung, especially near important structures such as bronchi and blood vessels and parts where lesions may exist, a higher resolution is required to capture more subtle textures, nodules and other features and improve the accuracy of diagnosis.

[0094] Therefore, the embodiments of the present application divide the target medical image, and set appropriate regional resolutions for each region according to the characteristics and requirements of each region. At this time, the regional resolutions of different image regions of the target medical image obtained constitute the target resolution. It can be understood that the regional resolutions can be the same or different.

[0095] In one embodiment, after obtaining the regional resolution of different image regions, the corresponding image regions are divided into regional grids based on the regional resolution. Each regional grid includes at least one pixel point, and the regional resolution determines the number of pixel points in the regional grid. Moreover, the sizes of the regional grids located in different image regions are not exactly the same, and different regional resolutions will result in differences in the sizes of the regional grids located in different image regions. For example, in a high-resolution image region, in order to fully capture detailed information, the size of the regional grid will be relatively small to ensure that the pixels contained in each grid have a high density and accuracy; while in a low-resolution image region, in order to save computing resources and storage space, the size of the regional grid will be correspondingly increased. Although the pixel density in each grid is low, it can still meet the requirements for describing and analyzing this image region. After dividing the regional grids, all the regional grids are obtained and integrated to obtain a coordinate grid.

[0096] Step 620: Select an indicating coordinate position from each coordinate grid to obtain coordinate parameters.

[0097] In one embodiment, the coordinate position of a specific point is selected from each coordinate grid as the indicating coordinate position, and the indicating coordinate position is used as the coordinate parameter to uniquely identify the coordinate grid. In this representation, the pixel values of all pixel points within each coordinate grid are the same as the pixel value of the selected indicating coordinate position, making the coordinate grid a basic unit with a unified attribute. During the generation process, only the prediction of the single pixel value of each coordinate grid needs to be concerned, without having to process the complex information of other pixel points within the coordinate grid, which can improve the generation efficiency.

[0098] Among them, the indicating coordinate position can be any point in the coordinate grid. In the embodiments of the present application, multiple selection methods can be set. For example, the center of the coordinate grid is selected as the indicating coordinate position. The center of the coordinate grid can comprehensively reflect the pixel information in all directions within the grid to a certain extent. For example, when judging whether there is a lesion in the area covered by a coordinate grid, using the pixel value at the center position as a representative can more comprehensively consider the overall situation of this area. The vertex of the coordinate grid can also be selected as the indicating coordinate position. The coordinate position of the vertex can be more conveniently used to calculate and judge the spatial relationship between the grid and the surrounding area. For example, when analyzing a medical image of a bone structure, the edge and contour of the bone can be more accurately determined through the vertex coordinates. It can be understood that the embodiments of the present application do not make specific limitations on the indicating coordinate position, which can be set according to actual needs, and are not limited to one selection method, and multiple selection methods can be used in combination.

[0099] Step 130: Fill in the predicted pixel values to obtain the target medical image. If the coordinate parameter is a three-dimensional coordinate, the target medical image is a three-dimensional image; otherwise, the target medical image is a two-dimensional image.

[0100] In one embodiment, after obtaining the predicted pixel value corresponding to each coordinate parameter, fill the predicted pixel value in the coordinate grid corresponding to the coordinate parameter to obtain the target medical image. Among them, the dimension of the target medical image is determined by the dimension of the coordinate parameter. If the coordinate parameter is a three-dimensional coordinate, the obtained target medical image is a three-dimensional image at this time. If the coordinate parameter is a two-dimensional coordinate, the obtained target medical image is a two-dimensional image.

[0101] In one embodiment, since the image parameter vector is a vector with a fixed length, when the pixel mapping model generates the target medical image, the operation amount can also be reduced by inputting the coordinate parameters in batches, and there is no need to adopt the method of cascade generation.

[0102] Next, the training process of the medical image generation model is described. Refer to Figure 7 , Figure 7 which is the flowchart of the training steps of the medical image generation model provided by the embodiment of the present application, including at least the following steps:

[0103] Step 710: Initialize the mapping system parameters of the pixel mapping model and the prediction system parameters of the image parameter prediction model.

[0104] In one embodiment, according to the inference process of the above medical image generation model, the core idea of the pixel mapping model in the embodiment of the present application is to regard a two-dimensional image or a three-dimensional image as a mapping function F from the coordinates (x, y, [z]) of pixel points to pixel values v. Among them, if it is a two-dimensional image, the coordinates are expressed as (x, y, [z]=0]), and the mapping function is expressed as: v x,y,z =F(x, y, z), where v x,y,z represents the pixel value corresponding to the coordinates (x, y, x).

[0105] Next, since the image parameter vector M output by the image parameter prediction model is required in the mapping process, a pixel mapping model with learnable parameters θ is generated according to the established mapping function, expressed as: F θ (x, y, z, M). During the training process, the mapping system parameter θ is adjusted to make F θ approach F.

[0106] At the beginning of training, initialize the mapping system parameter θ of the pixel mapping model and the prediction system parameters of the image parameter prediction model. Here, the prediction system parameter is the model weight of the image parameter prediction model.

[0107] Step 720: Obtain at least one medical image sample image, generate sample coordinate parameters corresponding to the medical image sample image, and initialize the image parameter sample vector.

[0108] In one embodiment, the medical image sample image can be a two-dimensional sample image or a three-dimensional sample image. Multiple medical image sample images are obtained to form a training data set. During training, the training data set is divided into multiple training batches, and each training batch is used for the training process. It can be understood that each medical image sample image also includes corresponding medical condition parameters, which correspond to the image medical parameters in the above inference process and are used to guide the generation direction.

[0109] Next, determine the corresponding sample coordinate parameters based on the medical image sample image. Similar to the inference process, the sample coordinate parameters can be obtained according to the above calculation method of the coordinate parameters. The image parameter sample vector corresponds to the image parameter vector in the inference process, and here it can be initialized with an all-zero vector or with random numerical values.

[0110] Step 730: Update the mapping system parameters and the prediction system parameters based on the medical image sample image, the image parameter sample vector, and the sample coordinate parameters until a trained image parameter prediction model and a pixel mapping model are obtained.

[0111] In one embodiment, the training process needs to adjust the image parameter prediction model so that it generates an image parameter vector that can reflect the image features of the medical image sample image as much as possible. At the same time, adjust the pixel mapping model, and the goal is to make the image generated by the pixel mapping model as close as possible to the medical image sample image.

[0112] In one embodiment, refer to Figure 8 , Figure 8 which is a schematic diagram of the training process provided by an embodiment of the present application. It can be seen from Figure 8 that the training process is divided into two layers. First, the image parameter prediction model is trained, and then the pixel mapping model is trained.

[0113] In one embodiment, refer to Figure 9 , Figure 9 which is a flowchart for updating the mapping system parameters and the prediction system parameters based on the medical image sample image, the image parameter sample vector, and the sample coordinate parameters until a trained image parameter prediction model and a pixel mapping model are obtained provided by an embodiment of the present application. The specific steps are as follows:

[0114] Step 910: Fix the mapping system parameters, input the image parameter sample vector and the sample coordinate parameters into the pixel mapping model for prediction, and generate a first predicted image according to the prediction result.

[0115] In one embodiment, taking one training batch as an example, when training an image parameter prediction model with N medical image sample images, with reference to Figure 8 , first fix the mapping system parameters of the pixel mapping model, sequentially select medical image sample images, input the current image parameter sample vector and the corresponding sample coordinate parameters into the pixel mapping model for prediction, and generate a first predicted image according to the prediction result.

[0116] Step 920: Calculate a first loss value based on the medical image sample image and the first predicted image, adjust the prediction system parameters based on the first loss value, and use the updated image parameter prediction model to predict an updated image parameter sample vector until a first training termination condition is reached, obtaining a trained image parameter prediction model.

[0117] In one embodiment, calculate the image difference between the selected medical image sample image and the first predicted image to obtain a first loss value. If the first loss value does not meet the loss convergence condition, adjust the prediction system parameters corresponding to the image parameter prediction model. After adjustment, select the next medical image sample image for prediction.

[0118] In the next prediction process, use the updated image parameter prediction model to predict an updated image parameter sample vector, recalculate the first loss value until a first training termination condition is reached, obtaining a trained image parameter prediction model. Here, the first training termination condition can be understood as that the image parameter sample vector output by the image parameter prediction model can guide the first predicted image to be as close as possible to the corresponding medical image sample image during the image generation process. At this time, the first loss value converges. It can be understood that the first image parameter sample vector is an initial value.

[0119] In one embodiment, with reference to Figure 10 , Figure 10 is a flowchart for obtaining an updated image parameter vector using the updated image parameter prediction model provided by an embodiment of the present application, specifically including the following steps:

[0120] Step 1010: Obtain the medical condition parameters of the medical image sample image and the conditional embedding features corresponding to the medical condition parameters.

[0121] Step 1020: Obtain the Gaussian noise features corresponding to the current time step and the time embedding features corresponding to the current time step.

[0122] Step 1030: Accumulate the time embedding features and the conditional embedding features to obtain a reference feature, and jointly input the reference feature and the Gaussian noise features into a feature extraction network for data processing to obtain an image parameter vector.

[0123] In one embodiment, referring to Figure 3 , the medical condition parameter input condition embedding layer is input with medical condition parameters to obtain condition embedding features corresponding to the medical condition parameters. The time step during the training process is used as the current time information. The time step reflects different stages during the training process, and each time step corresponds to a specific state and learning progress of the model during the training process. The current time step is input into the time embedding layer. Using encoding techniques such as sine and cosine position encoding, the discrete information of the current time step is converted into continuous time embedding features with time series characteristics. At the same time, it is also necessary to obtain the Gaussian noise features corresponding to the Gaussian noise of the current time step. Finally, the time embedding features and the condition embedding features are accumulated to obtain reference features, and the reference features and the Gaussian noise features are jointly input into the feature extraction network for data processing to obtain the image parameter vector.

[0124] Step 930: Fix the prediction system parameters, input the image parameter sample vector and the sample coordinate parameters into the pixel mapping model for prediction, and generate the second predicted image according to the prediction result.

[0125] In one embodiment, after the image parameter prediction model is trained, the prediction system parameters can be fixed and remain unchanged, and a stable image parameter sample vector can be generated in each iteration. Referring to Figure 8 , the image parameter sample vector and the sample coordinate parameters are input into the pixel mapping model for prediction, and the second predicted image is generated according to the prediction result.

[0126] Step 940: Calculate the second loss value according to the medical image sample and the second predicted image, and adjust the mapping system parameters based on the second loss value until the second training termination condition is reached, and obtain the trained pixel mapping model.

[0127] In one embodiment, according to the selected medical image sample, the image difference between it and the second predicted image is calculated to obtain the second loss value. If the second loss value does not meet the loss convergence condition, the mapping system parameters of the pixel mapping model are adjusted. After the adjustment, the next medical image sample is selected for prediction. During the next prediction process, the second loss value is recalculated until the second training termination condition is reached, and the trained pixel mapping model is obtained. The second training termination condition here can be understood as that the second predicted image output by the pixel mapping model is as close as possible to the corresponding medical image sample. At this time, the second loss value converges.

[0128] The technical solution provided by the embodiments of the present application is executed by a medical image generation model. The medical image generation model includes an image parameter prediction model and a pixel mapping model. By obtaining image medical parameters indicating the generation direction of the target medical image, and inputting the image medical parameters into the image parameter prediction model for data processing to obtain an image parameter vector, obtaining a plurality of coordinate parameters corresponding to the target medical image, inputting the coordinate parameters and the image parameter vector into the pixel mapping model for pixel value prediction, obtaining a predicted pixel value corresponding to each coordinate parameter, and filling the predicted pixel values to obtain the target medical image. If the coordinate parameters are three-dimensional coordinates, the target medical image is a three-dimensional image; otherwise, the target medical image is a two-dimensional image.

[0129] In the embodiments of the present application, specific image medical conditions are used to generate an image parameter vector that can guide the generation process. Then, according to different modality generation requirements, the corresponding two-dimensional or three-dimensional coordinate parameters are determined, and these coordinate parameters are used to define the generation area of the target medical image. Next, the pixel values of the coordinate parameters are predicted using the image parameter vector to achieve the generation process. Throughout the entire operation process, there is no need to input a base image. Whether it is a two-dimensional modality or a three-dimensional modality, pixel value prediction and image generation can be directly performed in space according to the preset logic, so it is applicable to various different modality requirements.

[0130] In addition, the cascaded generation method is usually a process of generating in stages and gradually. In this process, the error of each stage may accumulate to the next stage, ultimately resulting in a large deviation between the generated image and the actual requirements. In the embodiments of the present application, the three-dimensional coordinate parameters corresponding to the target medical image are directly obtained in the three-dimensional space, and pixel value prediction is performed, effectively avoiding the error accumulation situation that may occur in the cascaded generation method. For example, when generating a three-dimensional target medical image of the liver, the cascaded generation method may generate errors in the shape, position, etc. of the liver at different stages. As the generation process progresses, these errors accumulate continuously, resulting in a difference between the finally generated image and the real liver structure. In this embodiment, the operation is directly performed in the three-dimensional space, which can more accurately capture the three-dimensional characteristics of the liver and generate a target medical image that is more in line with the actual situation.

[0131] Moreover, since the cascaded process requires multiple stages of calculation and processing, involving a large amount of data transmission and storage of intermediate results, the computational complexity is high and the processing time is long. In the embodiments of the present application, by directly performing pixel value prediction in the three-dimensional space, the intermediate links are reduced and the computational amount is decreased. At the same time, since there is no need to input a base image, the process of processing the base image is avoided, significantly improving the efficiency of image generation in the three-dimensional modality.

[0132] The embodiment of the present application also provides a medical image generation device, which can implement the above medical image generation method. Referring to Figure 11 , the device includes:

[0133] An image parameter acquisition module 1110: configured to acquire image medical parameters for indicating the generation direction of a target medical image, and input the image medical parameters into an image parameter prediction model for data processing to obtain an image parameter vector.

[0134] A pixel prediction module 1120: configured to acquire a plurality of coordinate parameters corresponding to the target medical image, input the coordinate parameters and the image parameter vector into a pixel mapping model for pixel value prediction, and obtain a predicted pixel value corresponding to each coordinate parameter.

[0135] An image generation module 1130: configured to fill the predicted pixel values to obtain the target medical image. If the coordinate parameters are three-dimensional coordinates, the target medical image is a three-dimensional image; otherwise, the target medical image is a two-dimensional image.

[0136] The specific implementation manner of the medical image generation device in this embodiment is basically the same as that of the above medical image generation method, and will not be elaborated here.

[0137] The embodiment of the present application also provides an electronic device, including:

[0138] At least one memory;

[0139] At least one processor;

[0140] At least one program;

[0141] The program is stored in the memory, and the processor executes the at least one program to implement the medical image generation method described above in the present application. The electronic device can be any intelligent terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), an in-vehicle computer, etc.

[0142] Please refer to Figure 12 , Figure 12 which schematically shows the hardware structure of an electronic device in another embodiment. The electronic device includes:

[0143] A processor 1201, which can be implemented by using a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, etc., and is configured to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0144] The memory 1202 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM), etc. The memory 1202 can store an operating system and other application programs. When implementing the technical solutions provided in the embodiments of this specification through software or firmware, the relevant program codes are stored in the memory 1202, and the processor 1201 is called to execute the medical image generation method of the embodiments of this application;

[0145] The input / output interface 1203 is used to implement information input and output;

[0146] The communication interface 1204 is used to implement communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or through wireless means (such as mobile network, WIFI, Bluetooth, etc.);

[0147] The bus 1205 transmits information between the various components of the device (such as the processor 1201, the memory 1202, the input / output interface 1203, and the communication interface 1204);

[0148] Among them, the processor 1201, the memory 1202, the input / output interface 1203, and the communication interface 1204 achieve communication connections with each other inside the device through the bus 1205.

[0149] The embodiments of this application also provide a storage medium. The storage medium is a storage medium that stores a computer program. When the computer program is executed by a processor, the above-mentioned medical image generation method is implemented.

[0150] As a non-transitory storage medium, the memory can be used to store non-transitory software programs and non-transitory computer executable programs. In addition, the memory can include high-speed random access memory, and can also include non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory optionally includes a memory remotely set relative to the processor, and these remote memories can be connected to the processor through a network. Examples of the above networks include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0151] The medical image generation method, device, equipment, and storage medium proposed in the embodiments of the present application are executed by a medical image generation model. The medical image generation model includes an image parameter prediction model and a pixel mapping model. By obtaining image medical parameters indicating the generation direction of the target medical image, and inputting the image medical parameters into the image parameter prediction model for data processing to obtain an image parameter vector, obtaining a plurality of coordinate parameters corresponding to the target medical image, inputting the coordinate parameters and the image parameter vector into the pixel mapping model for pixel value prediction, obtaining a predicted pixel value corresponding to each coordinate parameter, and filling the predicted pixel values to obtain the target medical image. If the coordinate parameters are three-dimensional coordinates, the target medical image is a three-dimensional image; otherwise, the target medical image is a two-dimensional image. In the embodiments of the present application, specific image medical conditions are used to generate an image parameter vector that can guide the generation process. Then, according to different modality generation requirements, the corresponding two-dimensional or three-dimensional coordinate parameters are determined, and these coordinate parameters are used to define the generation area of the target medical image. Next, the pixel values of the coordinate parameters are predicted using the image parameter vector to achieve the generation process. During the entire operation process, there is no need to input a base image. Whether it is a two-dimensional modality or a three-dimensional modality, pixel value prediction and image generation can be directly performed in space according to the preset logic, so it is applicable to various different modality requirements. In addition, since the three-dimensional coordinate parameters corresponding to the target medical image are obtained and pixel value prediction is directly performed in the three-dimensional space, the error accumulation situation that may occur in the cascade generation method is effectively avoided. At the same time, the computational complexity is reduced, the processing time is reduced, and the efficiency of image generation in the three-dimensional modality is significantly improved.

[0152] The embodiments described in the embodiments of the present application are for more clearly explaining the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art know that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of the present application are equally applicable to similar technical problems.

[0153] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than those shown, or combine certain steps, or different steps.

[0154] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0155] Those of ordinary skill in the art will understand that all or some of the steps in the methods disclosed above, and the functional modules / units in systems and devices, can be implemented as software, firmware, hardware, or a suitable combination thereof.

[0156] As used in the specification of this application and the above-mentioned drawings, the terms "first", "second", "third", "fourth", etc. (if any) are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data may be interchanged under appropriate circumstances so that the embodiments of this application described herein can be implemented in an order different from those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0157] It should be understood that in this application, "at least one (item)" means one or more, and "a plurality" means two or more. "And / or" is used to describe the association relationship of associated objects and indicates that three relationships may exist. For example, "A and / or B" may mean: only A exists, only B exists, and both A and B exist at the same time. Among them, A and B can be singular or plural. The character " / " generally means that the associated objects before and after are in an "or" relationship. "At least one (one) of the following" or a similar expression means any combination of these items, including any combination of single items (ones) or plural items (ones). For example, at least one (one) of a, b, or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or plural.

[0158] In several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the above-mentioned unit division is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling, direct coupling, or communication connection to each other can be through some interfaces, and the indirect coupling or communication connection of devices or units can be in electrical, mechanical, or other forms.

[0159] The units described above as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed over multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0160] In addition, each functional unit in various embodiments of the present application may be integrated into a processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0161] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes multiple instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in various embodiments of the present application. The foregoing storage medium includes: various media that can store programs such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs.

[0162] The preferred embodiments of the embodiments of the present application have been described above with reference to the accompanying drawings. However, this does not limit the scope of the rights of the embodiments of the present application. Any modification, equivalent replacement, and improvement made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the rights of the embodiments of the present application.

Claims

1. A method for generating a medical image, characterized in that: The method is performed by a medical image generation model, wherein the medical image generation model includes an image parameter prediction model and a pixel mapping model, and the method includes: Acquire image medical parameters for indicating the generation direction of the target medical image, and input the image medical parameters into the image parameter prediction model for data processing to obtain an image parameter vector; Acquire a plurality of coordinate parameters corresponding to the target medical image, input the coordinate parameters and the image parameter vector into the pixel mapping model to perform pixel value prediction, and obtain a predicted pixel value corresponding to each of the coordinate parameters; The predicted pixel values ​​are filled to obtain the target medical image. If the coordinate parameters are three-dimensional coordinates, the target medical image is a three-dimensional image; otherwise, the target medical image is a two-dimensional image.

2. The medical image generation method according to claim 1, characterized in that: The training step of the medical image generation model at least includes: Initializing mapping system parameters of the pixel mapping model and prediction system parameters of the image parameter prediction model; Acquire at least one medical image sample image, generate sample coordinate parameters corresponding to the medical image sample image, and initialize an image parameter sample vector, wherein the medical image sample image is a two-dimensional sample image or a three-dimensional sample image; The mapping system parameters and the prediction system parameters are updated based on the medical imaging sample images, the image parameter sample vectors and the sample coordinate parameters until the trained image parameter prediction model and the pixel mapping model are obtained.

3. The medical image generation method according to claim 2, characterized in that: The updating of the mapping system parameters and the prediction system parameters based on the medical image sample image, the image parameter sample vector and the sample coordinate parameters until the trained image parameter prediction model and the pixel mapping model are obtained includes: Fixing the mapping system parameters, inputting the image parameter sample vector and the sample coordinate parameters into the pixel mapping model for prediction, and generating a first predicted image according to the prediction result; Calculating a first loss value according to the medical image sample image and the first predicted image image, adjusting the prediction system parameters based on the first loss value, and using the updated image parameter prediction model to predict the updated image parameter sample vector until a first training termination condition is reached, thereby obtaining the trained image parameter prediction model; Fixing the prediction system parameters, inputting the image parameter sample vector and the sample coordinate parameters into the pixel mapping model for prediction, and generating a second predicted image according to the prediction result; A second loss value is calculated according to the medical image sample image and the second predicted image, and the mapping system parameters are adjusted based on the second loss value until a second training termination condition is reached, thereby obtaining the trained pixel mapping model.

4. The medical imaging method according to claim 3, characterized in that: The image parameter prediction model at least includes a feature extraction network, and the updated image parameter vector is obtained by using the updated image parameter prediction model, including: Acquire medical condition parameters of the sample medical image and conditional embedding features corresponding to the medical condition parameters; Obtaining a Gaussian noise feature corresponding to a current time step and a time embedding feature corresponding to the current time step; The temporal embedding feature and the conditional embedding feature are accumulated to obtain a reference feature, and the reference feature and the Gaussian noise feature are input into the feature extraction network for data processing to obtain the image parameter vector.

5. The medical image generation method according to claim 1, characterized in that: The step of inputting the coordinate parameters and the image parameter vector into the pixel mapping model to predict pixel values, and obtaining a predicted pixel value corresponding to each of the coordinate parameters, comprises: Inputting the coordinate parameters into a first linear layer for feature extraction to obtain a first eigenvalue; Performing feature fusion on the first eigenvalue and the image parameter vector to obtain a second eigenvalue; Inputting the second eigenvalue into a second linear layer for feature extraction to obtain a third eigenvalue; Performing feature fusion on the third eigenvalue and the image parameter vector to obtain a fourth eigenvalue; The fourth eigenvalue is input into the output layer for prediction to obtain the predicted pixel value.

6. The medical image generation method according to claim 1, characterized in that: The step of obtaining a plurality of coordinate parameters corresponding to the target medical image includes: Acquiring a target resolution of the target medical image, and dividing the target medical image into a plurality of coordinate grids according to the target resolution; An indicated coordinate position is selected from each of the coordinate grids to obtain the coordinate parameters.

7. The medical image generation method according to claim 6, characterized in that: The step of obtaining a target resolution of the target medical image and dividing the target medical image into a plurality of coordinate grids according to the target resolution includes: Acquire regional resolutions of different image regions of the target medical image, wherein the target resolution includes all the regional resolutions; The corresponding image region is divided into regional grids based on the regional resolution, and all the regional grids are acquired to obtain the coordinate grid. The sizes of the regional grids located in different image regions are not completely the same.

8. A medical image generation device, characterized in that: The method is performed by a medical image generation model, wherein the medical image generation model includes an image parameter prediction model and a pixel mapping model, and the device includes: An image parameter acquisition module is used to acquire image medical parameters for indicating the generation direction of a target medical image, and input the image medical parameters into the image parameter prediction model for data processing to obtain an image parameter vector; Pixel prediction module: used for obtaining a plurality of coordinate parameters corresponding to the target medical image, inputting the coordinate parameters and the image parameter vector into the pixel mapping model for pixel value prediction, and obtaining a predicted pixel value corresponding to each coordinate parameter; Image generation module: used to fill the predicted pixel value to obtain the target medical image. If the coordinate parameter is a three-dimensional coordinate, the target medical image is a three-dimensional image, otherwise the target medical image is a two-dimensional image.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the medical imaging image generation method according to any one of claims 1 to 7 when executing the computer program.

10. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the medical imaging image generating method according to any one of claims 1 to 7 is implemented.

Images