Model training method, image recovery method, system, electronic device, and storage medium

Abstract

The application discloses a model training method, an image recovery method, a system, an electronic device and a storage medium, and relates to the technical fields of artificial intelligence and medical treatment. The method comprises the following steps: acquiring training data, wherein the training data comprises degraded image samples and real image samples; performing recovery processing on the degraded image samples by using an initial recovery model to obtain target semantic features and a recovered image, wherein the decoding part of the initial recovery model comprises an anatomical semantic codebook, and the anatomical semantic codebook is used for constraining the anatomical structure of the recovered image; determining target loss based on the degraded image samples, the real image samples, the recovered image and the target semantic features; and performing parameter adjustment on the initial recovery model according to the target loss to obtain a target recovery model. The application solves the technical problems of poor anatomical structure consistency and low image recovery accuracy of the model in the recovery processing of the degraded image.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence technology and medical technology, and more specifically, to a model training method, an image restoration method, a system, an electronic device, and a storage medium. Background Technology

[0002] In recent years, artificial intelligence (AI) models have been widely used to restore or improve the visual quality and usability of degraded images, thereby supporting tasks with high requirements for structural reliability (such as medical diagnosis and remote image analysis). Current technologies typically perform pixel-level optimization during model training to enhance the visual effect of the restored images. However, this approach has drawbacks, including the possibility that the restored images generated by the model may exhibit distortions in their overall shape that do not conform to the actual anatomical structure. In other words, the model suffers from poor anatomical consistency and low image restoration accuracy in degraded image restoration.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This application provides a model training method, an image restoration method, a system, an electronic device, and a storage medium to at least solve the technical problems of poor anatomical structure consistency and low image restoration accuracy in the restoration of degraded images.

[0005] According to one aspect of the embodiments of this application, a model training method is provided, comprising: acquiring training data, wherein the training data includes degraded image samples and real image samples; performing restoration processing on the degraded image samples using an initial restoration model to obtain target semantic features and a restored image, wherein the decoding part of the initial restoration model includes an anatomical semantic codebook, the anatomical semantic codebook being used to constrain the anatomical structure of the restored image; determining a target loss based on the degraded image samples, real image samples, the restored image, and the target semantic features; and adjusting the parameters of the initial restoration model according to the target loss to obtain a target restoration model.

[0006] According to another aspect of the embodiments of this application, an image restoration method is provided, comprising: acquiring a degraded image to be restored; performing restoration processing on the degraded image using a target restoration model to obtain a restored image, wherein the target restoration model is generated according to the model training method of any one of the above.

[0007] According to another aspect of the embodiments of this application, a computer-aided diagnostic method is provided, comprising: acquiring a medical image to be restored of a target object, wherein the category of the medical image to be restored includes at least one of the following: computed tomography (CT) image, magnetic resonance imaging (MRI) image, and positron emission tomography (PET) image; performing restoration processing on the medical image to be restored using a target restoration model to obtain a restored medical image, wherein the target restoration model is generated according to the model training method of any one of the above; and performing diagnostic analysis on the target object based on the restored medical image to obtain auxiliary diagnostic suggestions.

[0008] According to another aspect of the embodiments of this application, an image restoration method is provided, comprising: obtaining an image restoration dialog request through a first application programming interface, wherein the request data carried in the image restoration dialog request includes: a degraded image to be restored; and returning an image restoration dialog response through a second application programming interface, wherein the response data carried in the image restoration dialog response includes: a restored image, wherein the restored image is obtained by restoring the degraded image using a target restoration model, and the target restoration model is generated according to the model training method of any one of the above.

[0009] According to another aspect of the embodiments of this application, an image restoration system is provided, comprising: a client for sending a degraded image to be restored; a server connected to the client for performing restoration processing on the degraded image using a target restoration model to obtain a restored image, wherein the target restoration model is generated according to the model training method of any one of the above; the client is also used to output the restored image.

[0010] According to another aspect of the embodiments of this application, an electronic device is provided, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the method of any one of the above when it runs.

[0011] According to another aspect of the embodiments of this application, a computer-readable storage medium is provided, the computer-readable storage medium including a stored executable program, wherein, when the executable program is running, the device where the storage medium is located executes any of the methods described above.

[0012] According to another aspect of the embodiments of this application, a computer program product is provided, including a computer program that, when executed by a processor, implements the method according to any one of the above.

[0013] In this embodiment, by introducing an anatomical semantic codebook into the decoding section, the generation process of the restored image is constrained by the semantics of the anatomical structure, thereby guiding the restored image to maintain similarity or consistency with the anatomical structure represented by the anatomical semantic codebook in terms of structural morphology. By jointly determining the target loss based on real image samples, the restored image, and target semantic features, the model can balance pixel-level reconstruction and semantic structure matching during the optimization process, enhancing the restored image's ability to fit the real anatomical structure. Thus, this embodiment achieves the goal of introducing an anatomical semantic codebook into the decoding section to constrain the anatomical structure and ensure the accuracy of the restored image, thereby improving the model's image restoration accuracy and solving the technical problems of poor anatomical structure consistency and low image restoration accuracy in the restoration of degraded images.

[0014] It is worth noting that the general description above and the detailed description that follow are merely for illustrative purposes and do not constitute a limitation on this application. Attached Figure Description

[0015] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a hardware structure block diagram of a computing device for implementing the methods provided in the embodiments of this application; Figure 2 This is a schematic diagram of the system structure in an application scenario; Figure 3 This is a flowchart of a model training method according to an embodiment of this application; Figure 4 This is a schematic diagram of an optional image restoration model training scheme according to an embodiment of this application; Figure 5 This is a schematic diagram of an optional frequency domain feedforward unit according to an embodiment of this application; Figure 6 This is a schematic diagram of an optional dynamic frequency domain mask attention portion according to an embodiment of this application; Figure 7 This is a schematic diagram of an optional semantic decoder according to an embodiment of this application; Figure 8 This is a flowchart of an image restoration method according to an embodiment of this application; Figure 9 This is a flowchart of a computer-aided diagnostic method according to an embodiment of this application; Figure 10 This is a flowchart of another image restoration method according to an embodiment of this application; Figure 11 This is a structural block diagram of a model training device according to an embodiment of this application; Figure 12 This is a structural block diagram of an image restoration apparatus according to an embodiment of this application; Figure 13 This is a structural block diagram of a computer-aided diagnostic device according to an embodiment of this application; Figure 14 This is a structural block diagram of another image restoration apparatus according to an embodiment of this application; Figure 15 This is a structural block diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0016] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0017] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0018] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0019] First, some nouns or terms that appear in the description of the embodiments of this application shall be interpreted as follows.

[0020] Integrated (also known as All-in-One) medical image restoration refers to a unified model architecture capable of handling various medical image restoration tasks. For example, this model architecture can meet the image enhancement needs of different degradation types, including denoising, super-resolution, and low-dose reconstruction.

[0021] Semantic Codebook: A set of vectors used to represent the semantics of discretized anatomical structures. Each code vector in the semantic codebook can correspond to a typical biological anatomical morphological pattern, serving as a reference benchmark for anatomical structure constraints.

[0022] Vector quantization refers to the process of mapping semantic features in a continuous space to nearest-neighbor discrete vectors in a predefined semantic codebook, aiming to achieve the structuring and standardization of semantic representation.

[0023] Dynamic Frequency Mask Attention (DFMA) is an attention mechanism for adaptively separating low-frequency structural components from high-frequency detail components in an image spectrum. It achieves differentiated processing of different frequency components through frequency domain masking.

[0024] Frequency Feedforward Block (FFB): This refers to a processing module that performs channel-level energy recalibration of characteristic components in the frequency domain, aiming to enhance the response strength of diagnostic related frequency bands.

[0025] Semantic Loss: A constraint function calculated based on the similarity distance between the reconstructed image and the real image in the semantic codebook space, used to guide the reconstruction result to tend towards the real distribution in semantic structure.

[0026] The method embodiment provided in Embodiment 1 of this application can be executed on a mobile terminal, computer terminal, or similar computing device. Figure 1 This is a hardware structure block diagram of a computing device for implementing the methods provided in the embodiments of this application. For example... Figure 1As shown, the computing device 10 may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) (processor 102 may include, but is not limited to, a microprocessor or a field-programmable gate array (FPGA), etc.), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, the computing device 10 may also include: a display, an input / output interface (I / O interface), a Universal Serial Bus (USB) port (which may be included as one of the ports of a computer bus), a network interface, a cursor control device (such as a mouse, touchpad, etc.), a keyboard, a power supply, and / or a camera.

[0027] Those skilled in the art will understand that Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, the arithmetic unit 10 may also include a ratio Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0028] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element of the computing device 10. As involved in the embodiments of this application, the data processing circuits serve as processor control (e.g., selection of a variable resistor termination path connected to an interface).

[0029] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the model training method in the embodiments of this application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the above-mentioned model training method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computing device 10 via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0030] The transmission device 106 is used to connect to a network via a network interface to receive or send data. Specific examples of the network described above may include wired and / or wireless networks provided by the communication provider of the computing device 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module for wireless communication with the Internet.

[0031] like Figure 1 The display shown can be, for example, a touchscreen liquid crystal display (LCD), which allows the user to interact with the user interface of the computing device 10.

[0032] Figure 1 The hardware structure block diagram shown can serve not only as an exemplary block diagram of the aforementioned computing device 10, but also as an exemplary block diagram of the aforementioned server. In one optional embodiment, Figure 2 It is a kind of use Figure 1 The diagram shows a system architecture for an application scenario where the computing device acts as a transmitter or receiver. Figure 2 As shown, the computing device 10 can be connected to one or more servers (e.g., security servers, resource servers, game servers, etc.) via a data network connection or electronically. In an alternative embodiment, the computing device 10 can be any mobile computing device, etc. The data network connection can be a local area network (LAN) connection, a wide area network (WAN) connection, an Internet connection, or other types of data network connection. The computing device 10 can execute network services that connect to a server (e.g., a security server) or a group of servers 20. The network server is a network-based user service, such as social networks, cloud resources, email, online payments, or other online applications.

[0033] Under the aforementioned operating environment, this application provides the following: Figure 3 The model training method shown. Figure 3 This is a flowchart of a model training method according to an embodiment of this application. Figure 3 As shown, the method may include the following steps S301 to S304.

[0034] Step S301: Obtain training data, wherein the training data includes degraded image samples and real image samples.

[0035] The execution entity of the model training method provided in this application embodiment can be a model training system deployed on a computing device. When the model training method is applied to a specific scenario, the model training system is used to train a deep neural network model capable of restoring medical images to solve degradation problems such as noise, low resolution, or insufficient dosage in medical images.

[0036] The training data described above is the dataset used to train the initial recovery model. The training data includes paired image samples, namely pairs of degraded image samples and real image samples. Degraded image samples can be medical images with image quality defects. Real image samples are high-quality medical images that are defect-free or have minimal defects, corresponding to the degraded image samples. For example, a degraded image sample can be a low-dose computed tomography (CT) image, and the corresponding real image sample can be a high-quality CT image at a standard dose. Another example is a low-resolution magnetic resonance imaging (MRI) image, and the corresponding real image sample can be a high-resolution MRI image. Yet another example is a low-dose positron emission tomography (PET) image, and the corresponding real image sample can be a high-quality positron emission tomography (PET) image at a full dose. The training data can be obtained from publicly available medical image datasets or provided by medical institutions.

[0037] Step S302: The degraded image sample is restored using the initial restoration model to obtain the target semantic features and the restored image. The decoding part of the initial restoration model includes an anatomical semantic codebook, which is used to constrain the anatomical structure of the restored image.

[0038] The aforementioned initial restoration model is the image restoration model to be trained. It processes input degraded image samples and outputs a high-quality restored image. Internally, the initial restoration model includes an encoding part and a decoding part. Specifically, the decoding part of the initial restoration model includes an anatomical semantic codebook.

[0039] The aforementioned anatomical semantic codebook is a learnable data structure that can further contain multiple discrete vector representations (also called code vectors). These vector representations can characterize typical semantic patterns in anatomical structures. Therefore, the anatomical semantic codebook can constrain the anatomical structure of the reconstructed image during the decoding process. This means that during the generation of the reconstructed image, the anatomical semantic codebook imposes constraints on image features and semantic features, making the generated reconstructed image more realistic and reasonable in terms of anatomical structure. For example, the anatomical semantic codebook can constrain: the structural morphology of bones, the boundaries and relative positions of internal organs, the direction of blood vessels and nerves, etc. Through the constraints of the anatomical semantic codebook, the reconstructed image generated by the initial reconstruction model has better anatomical plausibility.

[0040] The aforementioned target semantic features are semantic features generated during the decoding process, carrying anatomical structural information of the reconstructed image. The reconstructed image is the output image of the initial reconstruction model, which is visually clearer than the degraded image sample and structurally closer to the real image sample.

[0041] In the application scenario, degraded image samples are input into the encoding part of the initial restoration model. The encoding part encodes the degraded image samples and extracts image features. These image features are then passed to the decoding part of the initial restoration model. In the decoding part, the image features are modulated based on the anatomical semantic codebook, and semantic features are used for collaborative constraints. The restored image is then obtained through layer-by-layer decoding. During the decoding process, target semantic features are extracted from a specified location in the decoding part (e.g., after the convolutional denoising layer). These target semantic features are output along with the restored image and used for subsequent target loss calculations. Thus, the initial restoration model completes the transformation from degraded image samples to restored images. In this process, the extraction of target semantic features and the constraints of the anatomical semantic codebook jointly ensure the rationality and realism of the anatomical structure of the restored image.

[0042] Step S303: Determine the target loss based on the degraded image sample, the real image sample, the restored image, and the target semantic features.

[0043] The target loss described above is a metric used to evaluate the performance of the initial reconstruction model. This target loss can include multiple sub-losses. For example, it can include a reconstruction loss, which measures the pixel-level difference between the reconstructed image and the original image sample. Or, it can include a semantic loss, which measures the semantic plausibility of the reconstructed image. The target loss can be a combination of the reconstruction and semantic losses.

[0044] For example, in one scenario, the pixel-level difference between the restored image and the real image sample can be calculated as the reconstruction loss; a first distance between the target semantic features and the semantic embedding of the real image sample, and a second distance between the target semantic features and the semantic embedding of the degraded image sample are calculated, and the semantic loss is calculated based on the first and second distances. Further, the reconstruction loss and the semantic loss are added or weighted to obtain the target loss. This target loss can guide subsequent parameter adjustments. Based on this, the above-mentioned target loss can support pixel-level reconstruction and preservation of semantic structure consistency during parameter adjustment and optimization.

[0045] Step S304: Adjust the parameters of the initial recovery model according to the target loss to obtain the target recovery model.

[0046] The parameter tuning described above follows the backpropagation and gradient descent process in machine learning. Based on the calculated target loss, the gradients of each parameter in the initial recovery model are calculated. Then, these gradients are used to update the parameters of the initial recovery model. The purpose of parameter tuning is to gradually reduce the value of the target loss.

[0047] In some application scenarios, steps S301 to S304 can be repeatedly executed on the training dataset (containing multiple sample pairs) to perform multiple rounds of iterative training on the initial restoration model, thereby gradually optimizing the model parameters. When the training ends, the target restoration model is obtained. The above target restoration model can be used to restore new degraded medical images and generate high-quality restored images.

[0048] The model training method provided in this application can be applied to scenarios such as medical imaging diagnostic systems, radiotherapy planning systems, and medical image database management systems. For example, in medical imaging diagnostic systems, based on low-dose computed tomography (CT) or low-resolution magnetic resonance imaging (MRI) images, a target restoration model can be used to generate restored images with clear anatomical structures, accurately assisting in the identification of small lesions during the diagnostic process and reducing missed diagnoses and misdiagnoses. In radiotherapy planning systems, high-fidelity reference images can be generated based on the patient's low-dose localization images for target delineation and dose calculation, improving the accuracy of radiotherapy. Overall, the embodiments of this application can provide high-quality, structurally accurate image restoration capabilities for various medical imaging applications, helping to improve diagnostic accuracy and treatment planning quality.

[0049] Based on steps S301 to S304 above, this embodiment of the application introduces an anatomical semantic codebook into the decoding section, constraining the generation process of the restored image by the semantics of the anatomical structure. This guides the restored image to maintain similarity or consistency with the anatomical structure represented by the anatomical semantic codebook in terms of structural morphology. By jointly determining the target loss based on real image samples, the restored image, and target semantic features, the model can balance pixel-level reconstruction and semantic structure matching during optimization, enhancing the ability of the restored image to fit the real anatomical structure. Thus, this embodiment of the application achieves the goal of introducing an anatomical semantic codebook into the decoding section to constrain the anatomical structure and ensure the accuracy of the restored image, thereby improving the technical effect of the model's image restoration accuracy and solving the technical problems of poor anatomical structure consistency and low image restoration accuracy in the restoration processing of degraded images.

[0050] The following will further explain other optional embodiments of the model training method in the embodiments of this application.

[0051] As an optional embodiment, the initial restoration model further includes an encoding part and a prompt generator; in step S302, the degraded image sample is restored using the initial restoration model to obtain a restored image, including the following method steps: Step S321, in the encoding part, the degraded image sample is encoded to obtain intermediate image features; Step S322, the intermediate image features are input to the prompt generator to generate initial semantic features; Step S323, in the decoding part, the intermediate image features and the initial semantic features are decoded based on the anatomical semantic codebook to obtain target semantic features and the restored image.

[0052] The aforementioned encoding section can be a front-end module in the initial restoration model used to extract image features. It receives degraded image samples as input, performs feature extraction step by step through multiple encoding layers, and outputs intermediate image features. The intermediate image features can be the features output at the end of the encoding section, which retain the key structural and spatial distribution information in the degraded image samples.

[0053] The aforementioned prompt generator can be a module in the initial recovery model used to generate semantic features. It receives intermediate image features output from the encoding part as input and extracts semantically meaningful representations, i.e., initial semantic features, from the intermediate image features through operations such as feature mapping and semantic lookup. The initial semantic features can contain structured semantic information related to the image content, such as the category, location, or morphological information of anatomical structures.

[0054] In the application scenario, degraded image samples are input to the encoding part, which performs layer-by-layer downsampling and feature extraction on the degraded image samples, and outputs intermediate image features. The intermediate image features are then passed to the prompt generator, which processes the intermediate image features to generate initial semantic features. Further, the intermediate image features and the initial semantic features are input to the decoding part, which modulates and decodes the intermediate image features using the initial semantic features based on the dissected semantic codebook, to obtain the target semantic features and the restored image.

[0055] In one exemplary application scenario, such as Figure 4 One possible image restoration model training scheme is shown, such as Figure 4As shown, the initial restoration model comprises multiple parts and units between the degraded image samples and the restored image. The encoding part of the initial restoration model includes multiple Transformer blocks for encoding the degraded image samples to obtain intermediate image features. The decoding part of the initial restoration model includes multiple semantic decoders for decoding the intermediate image features and initial semantic features based on a anatomical semantic codebook to obtain the target semantic features and the restored image. Furthermore, the initial restoration model also includes a semantic cue word generation component (i.e., a cue generator) for generating initial semantic features based on the intermediate image features.

[0056] Through the above optional embodiments, this application decomposes the image restoration task into feature encoding, semantic generation, and decoding reconstruction by introducing an encoding part and a prompt generator. The encoding part extracts hierarchical features of the image, the prompt generator generates semantic information from the image features, and the decoding part uses the semantic information to guide image reconstruction, thereby improving the model's ability to understand the structure of medical images and enhancing the controllability of the image restoration process.

[0057] As an optional embodiment, the encoding part includes multiple encoding blocks corresponding to multiple levels, and the encoding blocks include self-attention units and frequency domain feedforward units; in step S321, the degraded image samples are encoded to obtain intermediate image features, including the following method steps: Step S3211, the initial image features corresponding to the degraded image samples are input into multiple encoding blocks for step-by-step encoding processing to obtain intermediate image features; wherein, in any encoding block, the self-attention unit is used to perform attention weighting processing on the encoded input image features to obtain a self-attention result, and the frequency domain feedforward unit is used to perform frequency domain modulation on the self-attention result to obtain encoded output image features; wherein, in the step-by-step encoding processing, the encoded input image features of the first encoding block are the initial image features obtained by pre-encoding processing of the degraded image samples, and the encoded output image features of the last encoding block are used as intermediate image features.

[0058] The aforementioned coding blocks can be processing modules corresponding to each level in the coding section. For example, a coding block can be a Transformer block. Multiple coding blocks are concatenated in hierarchical order, with the output of the previous coding block serving as the input of the next coding block. The aforementioned self-attention unit can be a module in the coding block used to calculate the internal correlation of the input features; for example, a self-attention unit can be a multi-head self-attention module. The aforementioned frequency domain feedforward unit can be a module in the coding block used to perform frequency domain transformation and modulation of the features. In a specific implementation, each coding block contains concatenated self-attention units and frequency domain feedforward units, with the output of the self-attention unit serving as the input of the frequency domain feedforward unit.

[0059] The initial image features mentioned above can be feature representations obtained after pre-encoding processing (e.g., convolution or linear mapping) of degraded image samples. The intermediate image features mentioned above can be features output from the final encoding block, which are passed to subsequent modules (e.g., the cue generator and the decoding part) for further processing.

[0060] In the application scenario, the initial image features are input into the first coding block. In this coding block, the self-attention unit performs attention-weighted processing on the input image features to obtain the self-attention result; the frequency domain feedforward unit performs frequency domain modulation on the self-attention result to obtain the coded output image features. The coded output image features are then input into the next coding block, and the above process is repeated. Through the step-by-step processing of multiple coding blocks, the final coding block will output the intermediate image features.

[0061] Through the above optional embodiments, this application introduces a coding block structure including self-attention units and frequency domain feedforward units, thereby combining spatial domain feature interaction and frequency domain feature modulation during the encoding process. The self-attention units can capture long-range dependencies in the image, while the frequency domain feedforward units can adaptively adjust features in the frequency domain, thus improving the coding part's ability to extract complex structural information from medical images and providing richer feature representations for subsequent semantic generation and decoding reconstruction.

[0062] As an optional embodiment, in any coding block, the self-attention result is frequency-domain modulated using a frequency-domain feedforward unit to obtain coded output image features, including the following steps: Step S3212, transforming the self-attention result to the frequency domain to obtain a first frequency domain feature; Step S3213, performing channel attention weighting processing on the first frequency domain feature to obtain a second frequency domain feature; Step S3214, transforming the second frequency domain feature to the spatial domain to obtain a transformation result; Step S3215, performing activation processing on the transformation result to obtain coded output image features.

[0063] The aforementioned frequency domain feedforward unit can be a module in the coding block used for frequency domain transformation and modulation of image features. For example, the frequency domain feedforward unit may include a frequency domain transformation module, a channel attention module, a spatial domain transformation module, and an activation module.

[0064] The self-attention result described above can be a feature output by the self-attention unit. The self-attention result is used as input to the frequency domain feedforward unit and is fed into the frequency domain feedforward unit for further frequency domain modulation processing.

[0065] The aforementioned first frequency domain feature can be a feature obtained after frequency domain transformation of the self-attention result. In a specific implementation, the frequency domain transformation can be a Discrete Fourier Transform (DFT) or a Fast Fourier Transform (FFT). Transforming the self-attention result from the spatial domain to the frequency domain yields the feature represented in the frequency domain, i.e., the first frequency domain feature.

[0066] The aforementioned channel attention weighting processing can be a weighted adjustment of the first frequency domain features along the channel dimension. For example, channel attention weighting processing can include the following operations: calculating attention weights for the first frequency domain features along the channel dimension, and using these attention weights to weight each channel of the first frequency domain features to obtain the second frequency domain features. Through channel attention weighting processing, the frequency domain feedforward unit can adaptively emphasize the information-rich frequency response and suppress the noise-dominated frequency response.

[0067] The aforementioned second frequency domain feature can be a feature obtained after channel attention weighting. The aforementioned transformation result can be a feature obtained by transforming the second frequency domain feature from the frequency domain back to the spatial domain. The spatial domain transformation can be an inverse Fourier transform. Transforming the second frequency domain feature back to the spatial domain yields the feature represented in the spatial domain, i.e., the transformation result.

[0068] The activation process described above can involve applying a nonlinear activation function to the transformation result. This activation process can include convolutional projection operations and activation function operations. For example, a convolutional layer can be used to project the transformation result, followed by a nonlinear transformation using a preset activation function to obtain coded output image features. These coded output image features are then passed to the next coding block or used as part of intermediate image features.

[0069] In one specific implementation, during the frequency domain modulation process described above, the frequency domain feedforward unit receives the self-attention result as input features; the input features are projected using a convolutional layer, and then the projection result is transformed to the frequency domain using a Fourier transform to obtain frequency domain features; a channel attention mechanism is applied to the frequency domain features to calculate the weight of each frequency domain channel, and the frequency domain features are weighted and adjusted to obtain weighted frequency domain features; the weighted frequency domain features are transformed back to the spatial domain using an inverse Fourier transform to obtain spatial domain features; the spatial domain features are further processed using convolutional projection and a nonlinear activation function to output encoded image features.

[0070] In one exemplary application scenario, such as Figure 5 This illustrates a frequency domain feedforward unit. For example... Figure 5As shown, the frequency domain feedforward unit is used to modulate the input image features to obtain modulated image features. In any coding block, the image features input to the frequency domain feedforward unit are the self-attention result of the current coding block, and the modulated image features of the frequency domain feedforward unit can be used as the coded output image features of the current coding block.

[0071] like Figure 5 As shown, in the frequency domain feedforward unit, the image features are first normalized by a normalization layer to obtain normalized image features; the normalized image features are then projected by a convolutional layer to obtain projected image features; the projected image features are then segmented by an image segmentation module to obtain multiple image blocks; these multiple image blocks enter a Fourier channel attention component, where frequency domain transformation, channel attention weighting, and inverse frequency domain transformation are performed on the multiple image blocks to obtain attention-weighted image blocks; the attention-weighted image blocks are then reconstructed by an image reconstruction module to obtain reconstructed image features; the reconstructed image features are then modulated and projected sequentially by a modulation layer and a projection layer to output modulated image features.

[0072] Through the above optional embodiments, this application introduces a frequency domain feedforward unit, adding a frequency domain modulation step to the encoding process. The frequency domain feedforward unit transforms features to the frequency domain, performs channel attention weighting in the frequency domain, and then transforms them back to the spatial domain, achieving adaptive adjustment of features at the frequency domain level. This processing method enhances the model's ability to perceive high-frequency structural information while suppressing noise and redundant information, thereby improving the accuracy of the encoding part in extracting anatomical structures from medical images.

[0073] As an optional embodiment, the initial recovery model includes a dynamic frequency domain mask attention part; the above model training method further includes the following steps: Step S324, in the dynamic frequency domain mask attention part, multiple frequency domain masks are used to perform frequency domain feature filtering on the degraded image samples to obtain multiple sets of frequency domain components, and the multiple sets of frequency domain components are cross-attention fused with intermediate image features to obtain updated intermediate image features.

[0074] The aforementioned Dynamic Frequency Domain Masked Attention (DFMA) part can be a module in the initial restoration model used for frequency domain adaptive processing of image features. The input to the DFMA part includes: degraded image samples and intermediate image features output from the coding part.

[0075] The aforementioned frequency domain mask can be used to selectively preserve or suppress specific frequency components in the frequency domain. Multiple frequency domain masks can correspond to different frequency ranges, such as high-frequency ranges and low-frequency ranges. The aforementioned frequency domain feature filtering can be performed by transforming the degraded image samples to the frequency domain and then using the frequency domain mask to filter the frequency domain features, obtaining multiple sets of frequency domain components. These multiple sets of frequency domain components can include feature components from different frequency ranges, such as high-frequency feature components and low-frequency feature components.

[0076] The aforementioned cross-attention fusion can be performed by cross-attention calculation of multiple sets of frequency domain components and intermediate image features, fusing the information in the frequency domain components into the intermediate image features to obtain updated intermediate image features.

[0077] In the application scenario, the dynamic frequency domain mask attention component performs frequency domain transformation on the degraded image samples, converting them to the frequency domain. Then, multiple frequency domain masks are used to filter the frequency domain features, resulting in multiple sets of frequency domain components. Further, these multiple sets of frequency domain components are cross-attention fused with intermediate image features, integrating the information from the frequency domain components into the intermediate image features, outputting updated intermediate image features. These updated intermediate image features can replace the intermediate image features output by the encoding component and be passed to subsequent modules (such as the cue generator and decoding components) for further processing.

[0078] Through the above optional embodiments, this application introduces a dynamic frequency domain mask attention component, adding a frequency domain adaptive processing step between the encoding and decoding parts. The dynamic frequency domain mask attention component can adaptively adjust the frequency domain mask according to image degradation characteristics, emphasizing frequency components useful for the recovery task while suppressing noise or irrelevant frequency components. Through cross-attention fusion, frequency domain information is integrated into intermediate image features, giving the intermediate image features a stronger frequency perception capability, thereby improving the model's adaptability to frequency distribution differences in different modalities of medical images.

[0079] As an optional embodiment, the multiple frequency domain masks include a first frequency domain mask and a second frequency domain mask, wherein the first frequency threshold of the first frequency domain mask is higher than the second frequency threshold of the second frequency domain mask; in step S324, the multiple frequency domain masks are used to perform frequency domain feature filtering on the degraded image samples respectively to obtain multiple sets of frequency domain components, including the following method steps: step S3241, performing frequency domain transformation processing on the degraded image samples to obtain a third frequency domain feature; step S3242, using the first frequency domain mask to filter the third frequency domain feature to obtain high-frequency feature components in the multiple sets of frequency domain components, wherein the high-frequency feature components include a portion of frequency domain features with a feature frequency higher than the first frequency threshold; step S3243, using the second frequency domain mask to filter the third frequency domain feature to obtain low-frequency feature components in the multiple sets of frequency domain components, wherein the low-frequency feature components include a portion of frequency domain features with a feature frequency lower than the second frequency threshold.

[0080] The first frequency domain mask mentioned above can be a filtering component used to preserve high-frequency components, such as a high-pass mask. The second frequency domain mask mentioned above can be a filtering component used to preserve low-frequency components, such as a low-pass mask. The first frequency threshold can be the cutoff frequency of the first frequency domain mask, and the second frequency threshold can be the cutoff frequency of the second frequency domain mask. A first frequency threshold higher than the second frequency threshold means that the frequency range allowed by the first frequency domain mask is above the frequency range allowed by the second frequency domain mask.

[0081] The aforementioned third frequency domain feature can be a frequency domain representation obtained after performing a frequency domain transformation on the degraded image sample. The frequency domain transformation process can include convolution operations and Fourier transform operations to transform the degraded image sample from the spatial domain to the frequency domain, thereby obtaining the third frequency domain feature.

[0082] The aforementioned high-frequency feature components can be the portions of the third frequency domain features whose frequencies are higher than the first frequency threshold. The aforementioned low-frequency feature components can be the portions of the third frequency domain features whose frequencies are lower than the second frequency threshold. By filtering the third frequency domain features using the first and second frequency domain masks respectively, the high-frequency and low-frequency information can be separated.

[0083] In the application scenario, frequency domain transformation is performed on degraded image samples to obtain third frequency domain features. The third frequency domain features are then filtered using a first frequency domain mask to extract high-frequency feature components. The low-frequency feature components are extracted by filtering the third-frequency-domain features using a second-frequency-domain mask. High-frequency characteristic components With low-frequency characteristic components Each with its own learnable weights and Element-wise multiplication is performed to obtain the weighted high-frequency and low-frequency components. Then, the weighted high-frequency and low-frequency components are multiplied by the overall weight. Combine and integrate with intermediate image features Perform cross attention The feature fusion result is obtained through calculation. The specific calculation method is shown in equation (1) below.

[0084] Equation (1)

[0085] Through the above optional embodiments, this application achieves the separation and processing of different frequency components in medical images by distinguishing between high-frequency and low-frequency feature components. High-frequency feature components typically contain detailed information such as image edges and textures, while low-frequency feature components typically contain overall structural information such as image contours and backgrounds. By extracting high-frequency and low-frequency feature components separately, the dynamic frequency domain mask attention component can adopt different processing strategies for different frequency components, thereby more flexibly integrating frequency information in subsequent cross-attention fusion and improving the model's adaptability to different degradation types (such as noise, blur, low resolution, etc.).

[0086] As an optional embodiment, in step S324, cross-attention fusion is performed on multiple sets of frequency domain components and intermediate image features to obtain updated intermediate image features, including the following method steps: Step S3244, cross-attention processing is performed on high-frequency feature components and intermediate image features to obtain a first processing result; Step S3245, cross-attention processing is performed on low-frequency feature components and intermediate image features to obtain a second processing result; Step S3246, the first processing result and the second processing result are fused to obtain updated intermediate image features.

[0087] The first processing result described above can be the feature obtained by performing cross-attention processing on the high-frequency feature components and the intermediate image features. The second processing result described above can be the feature obtained by performing cross-attention processing on the low-frequency feature components and the intermediate image features. Cross-attention processing allows information from the high-frequency and low-frequency feature components to interact with the intermediate image features respectively, achieving feature-level information fusion.

[0088] When fusing the first and second processing results, the first and second processing results can be added element by element, weighted summed, or concatenated to integrate high-frequency and low-frequency information into the updated intermediate image features.

[0089] In one exemplary application scenario, such as Figure 6 This illustrates a dynamic frequency domain mask attention component. For example... Figure 6As shown, this DFMA part receives degraded medical images and intermediate image features as input. The degraded medical image is first processed through a convolutional layer for feature extraction, and then transformed to the frequency domain using a Fast Fourier Transform (FFT) to obtain frequency domain features. The frequency domain features are then processed by filtering and thresholding to generate high-pass and low-pass masks, respectively. The frequency domain component corresponding to the high-pass mask and the intermediate image features are input to a cross-attention component for cross-attention processing to obtain a first processing result. The frequency domain component corresponding to the low-pass mask and the intermediate image features are input to another cross-attention component for cross-attention processing to obtain a second processing result. The first and second processing results are input to a fusion component for fusion to obtain fused features. The fused features and image features are added through a residual connection to output the updated intermediate image features.

[0090] Through the aforementioned optional embodiments, this application enables intermediate image features to simultaneously acquire supplementary information from both high-frequency and low-frequency information by performing cross-attention processing on high-frequency and low-frequency feature components separately. High-frequency feature components provide detailed information such as edges and textures to the intermediate image features, while low-frequency feature components provide overall information such as contours and structure. By fusing the two processing results, the intermediate image features gain a more comprehensive frequency perception capability while retaining the original information, thereby improving the model's adaptability to different frequency components and the fidelity of the restored image's details.

[0091] As an optional embodiment, in step S322, the intermediate image features are output to the prompt generator to generate initial semantic features, including the following method steps: step S3221, semantically map the intermediate image features to obtain intermediate semantic embedding; step S3222, query the target semantic code that matches the intermediate semantic embedding from the preset semantic dictionary; step S3223, generate initial semantic features based on the target semantic code.

[0092] The aforementioned prompt generator can be a module in the initial recovery model used to generate semantic features. The input to the prompt generator is the intermediate image features output from the encoding part. Through operations such as semantic mapping and semantic dictionary lookup, initial semantic features are generated. The intermediate semantic embedding can be a vector representation obtained by semantic mapping of the intermediate image features; this intermediate semantic embedding contains structured information about the image content at the semantic level.

[0093] The aforementioned pre-defined semantic dictionary can be a built-in dictionary of the prompt generator or an external dictionary that grants query access to the prompt generator. This pre-defined semantic dictionary contains multiple discrete semantic codes, each corresponding to a semantic pattern or anatomical structure category. The aforementioned target semantic code can be a code in the semantic dictionary that matches the intermediate semantic embedding.

[0094] In the application scenario, intermediate image features are input into the cue generator. The cue generator performs semantic mapping on the intermediate image features, transforming them into a semantic embedding space to obtain the intermediate semantic embedding. The generator then queries its built-in semantic dictionary for the target semantic code that best matches the intermediate semantic embedding. For example, it calculates the Euclidean distance between the intermediate semantic embedding and each code in the semantic dictionary, selecting the code with the closest Euclidean distance as the target semantic code. Further, initial semantic features are generated based on the target semantic code. These initial semantic features can be the target semantic code itself or a feature representation obtained after further transformation of the target semantic code. These initial semantic features are then passed to the decoding part.

[0095] Through the above optional embodiments, this application achieves the conversion from image features to semantic features through a cue generator. The cue generator projects image features onto a semantic space through semantic mapping, and then obtains the corresponding semantic encoding by querying a built-in semantic dictionary, thereby generating initial semantic features with clear semantic meaning. This initial semantic feature serves as one of the inputs to the decoding part, providing structured semantic prior information for subsequent decoding and reconstruction.

[0096] As an optional embodiment, the decoding part includes multiple decoders corresponding to multiple levels, and the multiple decoders are associated with the same anatomical semantic codebook; in step S323, the intermediate image features and initial semantic features are decoded based on the anatomical semantic codebook to obtain the target semantic features and the restored image, including the following method steps: Step S3231, the intermediate image features and initial semantic features are input to multiple decoders for step-by-step modulation decoding to obtain the target image features and the target semantic features; Step S3232, the target image features are convolutionally reconstructed to obtain the restored image; wherein, in any decoder, the anatomical semantic codebook is used to process the input... The input semantic features are modulated and decoded to obtain the output semantic features. The input image features are modulated and decoded using the dissected semantic codebook and the output semantic features to obtain the decoded output image features. In the step-by-step modulation and decoding, the decoded input image features of the first decoder are the intermediate image features. The decoded input image features of other decoders are obtained by fusing the encoded output image features of the corresponding level coding block with the decoded output image features of the previous level decoder. The input semantic features of the first decoder are the initial semantic features. The decoded output image features of the last decoder are used as the target image features. The output semantic features of the last decoder are used as the target semantic features.

[0097] The aforementioned decoder can be a processing module corresponding to each level in the decoding section. Multiple decoders are concatenated in hierarchical order, with the output of the previous decoder serving as the input of the next decoder. The aforementioned anatomical semantic codebook can be a learnable data structure shared by multiple decoders. This anatomical semantic codebook contains multiple code vectors used for vector quantization of semantic features and for modulating image features using the quantized semantic features.

[0098] The above-mentioned step-by-step modulation and decoding can be achieved by sequentially inputting intermediate image features and initial semantic features into multiple decoders, with each decoder modulating and decoding the input features to obtain the target image features and target semantic features through step-by-step processing.

[0099] The aforementioned decoded input image features can be the image feature input received by the current decoder. In the first decoder (whose level corresponds to the level of the last encoder), the decoded input image features are intermediate image features. In other decoders, the decoded input image features are obtained by residual fusion of the following two parts: the encoded output image features of the corresponding level's coding block and the decoded output image features of the previous level's decoder.

[0100] The aforementioned decoded output image features can be the image features output by the current decoder, and these decoded output image features serve as the decoded input image features for the next level decoder. Similarly, the aforementioned output semantic features can be the semantic features output by the current decoder, and these output semantic features serve as the input semantic features for the next level decoder.

[0101] In the application scenario, intermediate image features and initial semantic features are input to the first decoder. The first decoder modulates and decodes the input semantic features using the dissecting semantic codebook to obtain the output semantic features. It then modulates and decodes the decoded input image features using the dissecting semantic codebook and the output semantic features to obtain the decoded output image features. Next, the decoded output image features are fused with the encoded output image features of the corresponding level coding block, serving as the decoded input image features for the next decoder. The output semantic features are then used as the input semantic features for the next decoder. This process is repeated until the final decoder outputs the target image features and target semantic features. Further, convolutional reconstruction is performed on the target image features to obtain the restored image.

[0102] Through the aforementioned optional embodiments, this application achieves unified modulation and transmission of semantic features during the step-by-step decoding process by associating multiple decoders with the same anatomical semantic codebook. Multiple decoders sharing the same anatomical semantic codebook ensures consistency and coherence of semantic constraints throughout the decoding process. The decoded input image features are obtained by fusing encoded features with the output features of the previous layer. This approach combines multi-scale features from the encoding stage with step-by-step reconstruction information from the decoding stage, enabling the decoder to simultaneously utilize global context and local detail information, thereby improving the fidelity of the reconstructed image's anatomical structure and the quality of detail reconstruction.

[0103] As an optional embodiment, the dissecting semantic codebook includes a set of code vectors. In any decoder, the input semantic features are modulated and decoded using the dissecting semantic codebook to obtain output semantic features, including the following steps: Step S3234, querying the target code vector that matches the input semantic features from the set of code vectors; Step S3235, using the target code vector to perform vector quantization on the input semantic features to obtain quantized semantic features; Step S3236, determining the output semantic features based on the quantized semantic features and the input semantic features.

[0104] The aforementioned anatomical semantic codebook can be a learnable data structure containing multiple discrete code vectors. The code vector set can be used... It means that among them This represents the total number of code vectors. Indicates the first Each code vector.

[0105] The target code vector can be a code vector in the code vector set that matches the input semantic features. In a specific implementation, the target code vector can be queried using the following equation (2).

[0106] Equation (2)

[0107] in, Indicates the input semantic features, The first one in the code vector set Each code vector Represents Euclidean distance. This represents the index of the code vector that is closest to the input semantic features. This represents the target code vector found in the query.

[0108] The vector quantization described above can be an operation that replaces the input semantic features with the target code vector. In one specific implementation, the quantized semantic features obtained after vector quantization are the target code vector itself, i.e. .

[0109] The steps for determining the output semantic features described above can be based on quantized semantic features and input semantic features. In a specific implementation, the output semantic features can be determined by the following equation (3).

[0110] Equation (3)

[0111] in, This indicates the output semantic features. This represents the pass-through estimator operation. In the pass-through estimator operation, it is used during forward propagation. As output, the gradient during backpropagation is directly fed back to the input semantic features. This enables end-to-end learning of codebook gradients.

[0112] In one exemplary application scenario, such as Figure 7 This illustrates a semantic decoder. For example... Figure 7 As shown, the semantic decoder can receive input semantic features and use the dissecting semantic codebook to modulate and decode the input semantic features to obtain output semantic features.

[0113] Through the optional embodiments described above, this application achieves vector quantization and modulation of input semantic features using an anatomical semantic codebook. The code vectors in the anatomical semantic codebook represent typical semantic patterns of anatomical structures. By mapping input semantic features to target code vectors, the semantic features can be made closer to the semantic distribution of the real anatomical structure. The use of a pass-through estimator allows the code vectors to be optimized in end-to-end training, thereby learning a more effective anatomical semantic representation. This modulation method provides a basis for anatomical structure constraints during the decoding process, helping to restore the image while maintaining the rationality and consistency of the anatomical structure.

[0114] As an optional embodiment, in any decoder, the decoded input image features are modulated and decoded using the dissected semantic codebook and output semantic features to obtain decoded output image features, including the following method steps: Step S3237, the decoded input image features are modulated using the output semantic features to obtain preliminary modulated image features; Step S3238, the preliminary modulated image features are subjected to self-attention processing to obtain self-attention results; Step S3239, the self-attention results are subjected to frequency domain modulation to obtain decoded output image features.

[0115] The aforementioned feature modulation can be an operation that applies output semantic features to decoded input image features. For example, feature modulation can be a feature-wise linear modulation (FiLM) operation, which uses output semantic features to generate scaling and offset parameters, and then scales and offsets the decoded input image features channel by channel to obtain preliminary modulated image features.

[0116] The self-attention processing described above can be an operation that applies attention weighting to the features of the initially modulated image, used to calculate the correlation between different positions within the features, thus obtaining the self-attention result. The frequency domain modulation described above can be an operation that applies frequency domain transformation, channel attention weighting, and inverse frequency domain transformation to the self-attention result, used to adaptively adjust the features in the frequency domain, thus obtaining the decoded output image features.

[0117] Still as Figure 7 As shown, the semantic decoder can also receive decoded input image features, which are input to the anatomical semantic codebook. After processing by the anatomical semantic codebook, the features are output to the self-attention component. After self-attention processing by the self-attention component, the features are output to the frequency domain feedforward unit. After frequency domain modulation by the frequency domain feedforward unit, the decoded output image features are output.

[0118] Through the above optional embodiments, this application achieves the modulation and decoding of image features through a semantic decoder. This process sequentially includes anatomical semantic codebook modulation, self-attention processing, and frequency domain modulation, enabling image features to retain spatial structural information while receiving semantic guidance and frequency domain optimization, thereby improving the fidelity of the reconstructed image's anatomical structure and the quality of detail reconstruction.

[0119] As an optional embodiment, in step S303, the target loss is determined based on the degraded image sample, the real image sample, the restored image, and the target semantic features, including the following method steps: step S331, determining the reconstruction loss based on the real image sample and the restored image; step S332, determining the semantic loss based on the degraded image sample, the real image sample, and the target semantic features; step S333, determining the target loss based on the reconstruction loss and the semantic loss.

[0120] The reconstruction loss described above can be a loss function used to measure the pixel-level difference between the restored image and the real image sample. In one specific implementation, the reconstruction loss can be a smooth L1 loss or a mean squared error loss (MSE loss), which is obtained by calculating the pixel difference between the restored image and the real image sample.

[0121] The semantic loss described above can be a loss function used to measure the semantic plausibility of the restored image. The semantic loss is calculated based on degraded image samples, real image samples, and target semantic features, and is used to make the semantic features of the restored image closer to the semantic distribution of the real image samples. The target loss described above can be a combined result of the reconstruction loss and the semantic loss.

[0122] In the application scenario, the pixel-level difference between the real image sample and the restored image is calculated to obtain the reconstruction loss. A first distance between the target semantic features and the semantic embedding of the real image sample, and a second distance between the target semantic features and the semantic embedding of the degraded image sample are calculated. The semantic loss is then calculated based on the first and second distances. Furthermore, the reconstruction loss and the semantic loss are added or weighted to obtain the target loss.

[0123] Through the aforementioned optional embodiments, this application divides the target loss into two parts: reconstruction loss and semantic loss. This allows the model to simultaneously optimize pixel-level reconstruction quality and semantic-level structural rationality during training. The reconstruction loss ensures that the restored image is visually as consistent as possible with the real image sample, while the semantic loss ensures that the anatomical structure of the restored image remains semantically consistent with the real image sample. Therefore, the target loss enables the model to retain detailed information while maintaining the rationality and consistency of the anatomical structure during image restoration, thereby improving the overall quality of image restoration.

[0124] As an optional embodiment, in step S332, the semantic loss is determined based on the degraded image sample, the real image sample, and the target semantic features, including the following steps: Step S3321, using a preset model with parameter freezing, semantic features are extracted from the degraded image sample to obtain the first semantic feature, and semantic features are extracted from the real image sample to obtain the second semantic feature; Step S3322, the semantic loss is calculated based on the target semantic feature, the first semantic feature, and the second semantic feature.

[0125] The preset model for parameter freezing mentioned above can be a pre-trained medical baseline model whose parameters remain unchanged during training. In one specific implementation, the preset model can be an external medical model. This preset model is used to extract semantic embeddings from medical images, serving as a reference for semantic loss calculation.

[0126] The first semantic feature mentioned above can be a semantic embedding extracted from degraded image samples. The second semantic feature mentioned above can be a semantic embedding extracted from real image samples. The target semantic feature can be the semantic feature output by the initial restoration model during the decoding process.

[0127] In one specific implementation, semantic loss It can be calculated using the following formula (4).

[0128] Equation (4)

[0129] in, This represents the semantic embedding corresponding to the target semantic features. This represents the second semantic feature (i.e., the semantic embedding extracted from real image samples). This represents the first semantic feature (i.e., the semantic embedding extracted from the degraded image sample). Represents Euclidean distance. This represents the preset margin parameter. Equation (4) makes the semantic features of the recovered image closer to the semantic distribution of the real image by constraining the distance between the target semantic features and the semantic embedding of the real image sample to be smaller than the distance between the target semantic features and the semantic embedding of the degraded image sample.

[0130] In the application scenario, a preset model with parameter freezing is used to extract semantic features from degraded image samples and real image samples respectively, resulting in first semantic features and second semantic features. The target semantic features are obtained, and the first distance between the semantic embedding corresponding to the target semantic features and the second semantic features is calculated. The second distance between the semantic embedding corresponding to the target semantic features and the first semantic features is also calculated. Then, the semantic loss is calculated according to equation (4), ensuring that the first distance is less than the second distance. Further, the reconstruction loss and semantic loss are combined according to equation (5) to obtain the target loss. .

[0131] Equation (5)

[0132] in, This indicates the losses incurred during reconstruction.

[0133] Through the above optional embodiments, this application extracts semantic features by introducing a preset model with parameter freezing and calculates semantic loss, thereby constraining the semantic level of the restored image. This semantic loss compares the distance between the target semantic features and the semantic embeddings of the real image samples, as well as the distance between the target semantic features and the semantic embeddings of the degraded image samples, making the semantic features of the restored image closer to the semantic distribution of the real image samples, thus improving the rationality of the anatomical structure and the semantic consistency of the restored image.

[0134] In summary, the model training method and image restoration method provided in this application can achieve the following beneficial effects. First, improved anatomical structure consistency. By introducing an anatomical semantic codebook and semantic supervision mechanism, the structure of the restored image is constrained by high-level semantic information, reducing structural artifacts and enhancing anatomical rationality. Second, enhanced frequency domain adaptive capability. Through dynamic frequency domain masking attention and frequency domain feedforward modules, adaptive modeling of different frequency components is performed, preserving high-frequency detail information while suppressing noise. Third, improved multi-task generalization capability. Multi-task collaborative modeling is achieved through a unified framework, reducing system complexity and improving the model's generalization capability in different medical image restoration tasks. Thus, this application embodiment improves the structural rationality and visual quality of the restored images generated by the model while enhancing traditional evaluation metrics.

[0135] In the aforementioned operating environment, this application also provides, as follows: Figure 8 An image restoration method is shown. Figure 8 This is a flowchart of an image restoration method according to an embodiment of this application, such as... Figure 8 As shown, the image restoration method includes: step S801, obtaining the degraded image to be restored; step S802, using the target restoration model to perform restoration processing on the degraded image to obtain the restored image, wherein the target restoration model is generated according to any of the above model training methods.

[0136] The degraded image to be restored can be a medical image requiring restoration processing. This degraded image can be a medical image with image quality defects, such as a low-dose computed tomography (CT) image, a low-resolution magnetic resonance imaging (MRI) image, or a low-dose positron emission tomography (PET) image. The target restoration model can be a deep neural network model trained using the aforementioned model training method. This target restoration model possesses an anatomical semantic codebook, enabling it to constrain the anatomical structure of the restored image during the image restoration process, thereby improving the structural rationality and visual quality of the restored image.

[0137] In application scenarios, the process begins with acquiring the degraded image to be restored from medical imaging equipment or a medical image database. Next, the degraded image is input into a target restoration model, which performs restoration processing and outputs a restored image. This restored image is visually clearer than the degraded image and structurally closer to the anatomical distribution of a real medical image. This restored image can be used to assist doctors in diagnostic analysis or for target delineation and dose calculation in radiotherapy planning.

[0138] Based on steps S801 to S802 described above, this embodiment of the application utilizes a target restoration model generated according to the aforementioned model training method to restore degraded images. This ensures that the generation process of the restored image is constrained by the semantics of anatomical structures, thereby guiding the restored image to maintain anatomical rationality in its structural morphology. Through the semantic constraint mechanism established during the training process of the target restoration model, the restored image is optimized at both the pixel-level reconstruction and semantic-level structural matching levels. Therefore, this embodiment of the application can utilize a pre-trained model to perform high-quality restoration of degraded images, thereby improving the accuracy of image restoration and the fidelity of anatomical structures.

[0139] It should be noted that the preferred embodiments of steps S801 to S802 described above can be found in the foregoing description, and will not be repeated here.

[0140] In the aforementioned operating environment, this application also provides, as follows: Figure 9 This illustrates a computer-aided diagnostic method. Figure 9This is a flowchart of a computer-aided diagnostic method according to an embodiment of this application, such as... Figure 9 As shown, the computer-aided diagnostic method includes: step S901, acquiring the medical image to be restored of the target object, wherein the category of the medical image to be restored includes at least one of the following: computed tomography image, magnetic resonance image, positron emission tomography image; step S902, using the target restoration model to perform restoration processing on the medical image to be restored to obtain the restored medical image, wherein the target restoration model is generated according to the model training method of any of the above; step S903, performing diagnostic analysis on the target object based on the restored medical image to obtain auxiliary diagnostic suggestions.

[0141] The aforementioned medical images to be restored can be the original medical images of the target object, which may be degraded in quality due to insufficient dosage, resolution limitations, or equipment noise. Categories of medical images to be restored include, but are not limited to, Computed Tomography (CT) images, Magnetic Resonance Imaging (MRI) images, and Positron Emission Tomography (PET) images. The aforementioned target restoration model is a deep neural network model trained using the aforementioned model training method. It possesses an anatomical semantic codebook and can constrain the anatomical structures of the medical images during the image restoration process. The aforementioned auxiliary diagnostic suggestions can be the detection results of lesions in the target object, analytical opinions on structural abnormalities, or reference suggestions for treatment plans.

[0142] In this application scenario, the medical images of the target object to be restored are obtained from medical imaging equipment or medical image databases. These images are then input into a target restoration model, which performs restoration processing and outputs a restored medical image. This restored medical image is visually clearer than the original image and more closely approximates the actual physiological distribution in terms of anatomical structure. Furthermore, diagnostic analysis is performed based on the restored medical image, such as identifying suspicious lesions, measuring structural parameters, or comparing the morphology with normal tissue, outputting auxiliary diagnostic suggestions. These suggestions can be used to assist physicians in making clinical decisions.

[0143] Based on steps S901 to S903 described above, this embodiment of the application utilizes a target restoration model generated according to the aforementioned model training method to convert low-quality medical images to be restored into high-quality restored medical images, enabling diagnostic analysis to be performed based on images with clear structure and rich details. Through the anatomical constraint mechanism established during the training process of the target restoration model, the restored medical images maintain rationality and consistency in anatomical structure, providing a reliable image basis for diagnostic analysis. Therefore, this embodiment of the application utilizes high-quality restored medical images to assist in diagnostic analysis, achieving the technical effect of improving diagnostic accuracy and reliability.

[0144] It should be noted that the preferred embodiments of steps S901 to S903 described above can be found in the foregoing description, and will not be repeated here.

[0145] In the aforementioned operating environment, this application also provides, as follows: Figure 10 Another image restoration method is shown. Figure 10 This is a flowchart of another image restoration method according to an embodiment of this application, such as... Figure 10 As shown, the image restoration method includes: step S1001, obtaining an image restoration dialogue request through a first application programming interface, wherein the request data carried in the image restoration dialogue request includes: the degraded image to be restored; step S1002, returning an image restoration dialogue response through a second application programming interface, wherein the response data carried in the image restoration dialogue response includes: the restored image, which is obtained by restoring the degraded image using a target restoration model, and the target restoration model is generated according to any of the above model training methods.

[0146] The first application programming interface (API) described above can be an interface for receiving external requests, specifically image restoration dialog requests. The request data carried in the image restoration dialog request includes the degraded image to be restored. The second application programming interface described above can be an interface for returning processing results, specifically an image restoration dialog response, with the restored image carried in the response data. The restored image is obtained by restoring the degraded image using a target restoration model, which is generated according to the aforementioned model training method.

[0147] In this application scenario, the server deploying the image restoration service receives an image restoration dialog request from the client via a first API. This request carries the degraded image to be restored. The server calls the target restoration model to perform restoration processing on the degraded image, generating a restored image. The server then encapsulates the restored image in the image restoration dialog response via a second API and returns it to the client. Upon receiving the response, the client can display or further process the restored image. This API interface approach is suitable for cloud services, microservice architectures, or remote image processing platforms, facilitating the integration of image restoration capabilities into various medical imaging application systems.

[0148] Based on steps S1001 to S1002 described above, this embodiment of the application receives image restoration requests and returns restored images via an application programming interface (API), enabling users to remotely invoke a target restoration model over a network to restore degraded images. Through the anatomical constraint mechanism established during the training of the target restoration model, the returned restored images maintain anatomical rationality and consistency in their structural morphology. Therefore, this embodiment of the application can support the provision of high-quality image restoration services via an API interface, thereby achieving the technical effect of remote deployment and convenient integration of image restoration capabilities.

[0149] It should be noted that the preferred embodiments of steps S1001 to S1002 described above can be found in the foregoing description, and will not be repeated here.

[0150] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0151] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, they can also be implemented by hardware. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory (ROM), random access memory (RAM), magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0152] According to an embodiment of this application, an apparatus embodiment for implementing the above-described model training method is also provided. Figure 11 This is a schematic diagram of a model training device according to an embodiment of this application, as shown below. Figure 11 As shown, the device includes: an acquisition module 1101 for acquiring training data, wherein the training data includes degraded image samples and real image samples; a restoration module 1102 for using an initial restoration model to restore the degraded image samples to obtain target semantic features and restored images, wherein the decoding part of the initial restoration model includes an anatomical semantic codebook, which is used to constrain the anatomical structure of the restored images; a loss module 1103 for determining a target loss based on the degraded image samples, real image samples, restored images, and target semantic features; and an adjustment module 1104 for adjusting the parameters of the initial restoration model according to the target loss to obtain a target restoration model.

[0153] Optionally, the initial recovery model further includes an encoding part and a prompt generator; the recovery module 1102 is also used to: in the encoding part, encode the degraded image sample to obtain intermediate image features; input the intermediate image features to the prompt generator to generate initial semantic features; in the decoding part, decode the intermediate image features and the initial semantic features based on the anatomical semantic codebook to obtain the target semantic features and the recovered image.

[0154] Optionally, the encoding part includes multiple encoding blocks corresponding to multiple levels, and the encoding blocks include self-attention units and frequency domain feedforward units; the recovery module 1102 is further used to: input the initial image features corresponding to the degraded image samples into multiple encoding blocks for step-by-step encoding processing to obtain intermediate image features; wherein, in any encoding block, the self-attention unit is used to perform attention weighting processing on the encoded input image features to obtain a self-attention result, and the frequency domain feedforward unit is used to perform frequency domain modulation on the self-attention result to obtain encoded output image features; wherein, in the step-by-step encoding processing, the encoded input image features of the first encoding block are the initial image features obtained by pre-encoding processing of the degraded image samples, and the encoded output image features of the last encoding block are used as intermediate image features.

[0155] Optionally, the recovery module 1102 is further configured to: transform the self-attention result to the frequency domain to obtain a first frequency domain feature; perform channel attention weighting processing on the first frequency domain feature to obtain a second frequency domain feature; transform the second frequency domain feature to the spatial domain to obtain a transformation result; and perform activation processing on the transformation result to obtain coded output image features.

[0156] Optionally, the initial recovery model includes a dynamic frequency domain mask attention component; the recovery module 1102 is further configured to: in the dynamic frequency domain mask attention component, use multiple frequency domain masks to perform frequency domain feature filtering on the degraded image samples respectively to obtain multiple sets of frequency domain components, and perform cross-attention fusion of the multiple sets of frequency domain components with intermediate image features to obtain updated intermediate image features.

[0157] Optionally, the multiple frequency domain masks include a first frequency domain mask and a second frequency domain mask, wherein a first frequency threshold of the first frequency domain mask is higher than a second frequency threshold of the second frequency domain mask; the recovery module 1102 is further configured to: perform frequency domain transformation processing on the degraded image samples to obtain a third frequency domain feature; use the first frequency domain mask to filter the third frequency domain feature to obtain high-frequency feature components in multiple sets of frequency domain components, wherein the high-frequency feature components include some frequency domain features with feature frequencies higher than the first frequency threshold; use the second frequency domain mask to filter the third frequency domain feature to obtain low-frequency feature components in multiple sets of frequency domain components, wherein the low-frequency feature components include some frequency domain features with feature frequencies lower than the second frequency threshold.

[0158] Optionally, the recovery module 1102 is further configured to: perform cross-attention processing on the high-frequency feature components and the intermediate image features to obtain a first processing result; perform cross-attention processing on the low-frequency feature components and the intermediate image features to obtain a second processing result; and fuse the first processing result and the second processing result to obtain updated intermediate image features.

[0159] Optionally, the recovery module 1102 is further configured to: perform semantic mapping on the intermediate image features to obtain intermediate semantic embedding; query the target semantic code that matches the intermediate semantic embedding from the preset semantic dictionary; and generate initial semantic features based on the target semantic code.

[0160] Optionally, the decoding part includes multiple decoders corresponding to multiple levels, and the multiple decoders are associated with the same anatomical semantic codebook; the recovery module 1102 is further used to: input intermediate image features and initial semantic features into multiple decoders for step-by-step modulation and decoding to obtain target image features and target semantic features; perform convolutional reconstruction on the target image features to obtain the recovered image; wherein, in any decoder, the input semantic features are modulated and decoded using the anatomical semantic codebook to obtain output semantic features, and the decoded input image features are modulated and decoded using the anatomical semantic codebook and output semantic features to obtain decoded output image features; wherein, in the step-by-step modulation and decoding, the decoded input image features of the first decoder are intermediate image features, and the decoded input image features of other decoders are obtained by fusing the encoded output image features of the corresponding level coding block with the decoded output image features of the previous level decoder, the input semantic features of the first decoder are initial semantic features, the decoded output image features of the last decoder are used as target image features, and the output semantic features of the last decoder are used as target semantic features.

[0161] Optionally, the dissecting semantic codebook includes a set of code vectors; the recovery module 1102 is further configured to: query a target code vector from the set of code vectors that matches the input semantic features; use the target code vector to perform vector quantization on the input semantic features to obtain quantized semantic features; and determine the output semantic features based on the quantized semantic features and the input semantic features.

[0162] Optionally, the recovery module 1102 is further configured to: use the output semantic features to perform feature modulation on the decoded input image features to obtain preliminary modulated image features; perform self-attention processing on the preliminary modulated image features to obtain self-attention results; and perform frequency domain modulation on the self-attention results to obtain decoded output image features.

[0163] Optionally, the loss module 1103 is further configured to: determine the reconstruction loss based on the real image samples and the restored image; determine the semantic loss based on the degraded image samples, the real image samples and the target semantic features; and determine the target loss based on the reconstruction loss and the semantic loss.

[0164] Optionally, the loss module 1103 is further configured to: extract semantic features from degraded image samples using a preset model with parameter freezing to obtain a first semantic feature, and extract semantic features from real image samples to obtain a second semantic feature; and calculate semantic loss based on the target semantic feature, the first semantic feature, and the second semantic feature.

[0165] It should be noted that the above-mentioned acquisition module 1101, recovery module 1102, loss module 1103 and adjustment module 1104 correspond to steps S301 to S304 in the embodiment. The four modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the foregoing embodiment.

[0166] According to an embodiment of this application, an apparatus embodiment for implementing the above-described image restoration method is also provided. Figure 12 This is a schematic diagram of the structure of an image restoration device according to an embodiment of this application, as shown below. Figure 12 As shown, the device includes: an acquisition module 1201 for acquiring a degraded image to be restored; and a restoration module 1202 for performing restoration processing on the degraded image using a target restoration model to obtain a restored image, wherein the target restoration model is generated according to any of the above-mentioned model training methods.

[0167] It should be noted that the above-mentioned acquisition module 1201 and recovery module 1202 correspond to steps S801 to S802 in the embodiment. The two modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the foregoing embodiment.

[0168] According to an embodiment of this application, an apparatus embodiment for implementing the above-described computer-aided diagnostic method is also provided. Figure 13 This is a schematic diagram of the structure of a computer-aided diagnostic device according to an embodiment of this application, as shown below. Figure 13 As shown, the device includes: an acquisition module 1301 for acquiring a medical image of a target object to be restored, wherein the category of the medical image to be restored includes at least one of the following: computed tomography (CT) image, magnetic resonance imaging (MRI) image, and positron emission tomography (PET) image; a restoration module 1302 for using a target restoration model to perform restoration processing on the medical image to be restored to obtain a restored medical image, wherein the target restoration model is generated according to any of the above model training methods; and an analysis module 1303 for performing diagnostic analysis on the target object based on the restored medical image to obtain auxiliary diagnostic suggestions.

[0169] It should be noted that the above-mentioned acquisition module 1301, recovery module 1302 and analysis module 1303 correspond to steps S901 to S903 in the embodiment. The three modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the foregoing embodiment.

[0170] According to an embodiment of this application, another embodiment of an apparatus for implementing the above-described image restoration method is also provided. Figure 14 This is a schematic diagram of another image restoration device according to an embodiment of this application, such as... Figure 14As shown, the device includes: a request module 1401, used to obtain an image restoration dialogue request through a first application programming interface, wherein the request data carried in the image restoration dialogue request includes: a degraded image to be restored; and a response module 1402, used to return an image restoration dialogue response through a second application programming interface, wherein the response data carried in the image restoration dialogue response includes: a restored image, which is obtained by restoring the degraded image using a target restoration model, and the target restoration model is generated according to any of the above model training methods.

[0171] It should be noted that the above-mentioned request module 1401 and response module 1402 correspond to steps S1001 to S1002 in the embodiment. The two modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the foregoing embodiment.

[0172] It should be noted that the above-mentioned modules or units may be hardware or software components stored in memory and processed by one or more processors. The above-mentioned modules may also be part of a device and run in a computer terminal.

[0173] It should be noted that the preferred implementation methods of this embodiment can be found in the relevant descriptions of the foregoing embodiments, and will not be repeated here. The preferred implementation methods involved in the above embodiments of this application are the same as the solutions, application scenarios, and implementation processes provided in the above embodiments, but are not limited to the solutions provided in the above embodiments.

[0174] Embodiments of this application may provide an image restoration system, comprising: a client for sending a degraded image to be restored; a server connected to the client for performing restoration processing on the degraded image using a target restoration model to obtain a restored image, wherein the target restoration model is generated according to any of the model training methods described above; and the client is also used to output the restored image.

[0175] Embodiments of this application may provide an electronic device, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the method of any of the foregoing items during runtime.

[0176] Figure 15 This is a structural block diagram of an electronic device according to an embodiment of this application. Figure 15 As shown, the electronic device 150 may include: one or more (only one is shown in the figure) processors 152, memory 154, memory controller, and peripheral interfaces.

[0177] The aforementioned electronic device can be understood as an integrated smart terminal, including but not limited to servers, desktop computers, personal computers (PCs), all-in-one model machines, etc., and the electronic device may have the model described in the above embodiments of this application pre-installed.

[0178] Specifically, this electronic device can pre-install various types of models, including but not limited to models in natural language processing, visual processing, speech processing, code processing, and multimodal task processing, thus providing diverse model selection. In different product forms, this electronic device can support one or more model usage methods, including but not limited to model training, model invocation, model fine-tuning, model deployment, model inference, and application. In some product forms, this electronic device also supports model management, including but not limited to multi-type model management (supporting the management of discriminative, generative, and other types of models), model version control (supporting the control of different model versions), and model evaluation (evaluating model performance and effectiveness based on model evaluation tools). In other product forms, this electronic device can also create applications based on models, providing Application Programming Interface (API) invocation capabilities. Models can be invoked into the created applications through the API interface, and application management tools are provided to manage and monitor the applications.

[0179] Furthermore, the electronic device may also include data management (supporting the creation and management of model tuning datasets), a training center (providing rich training resources to help users learn and master artificial intelligence (AI) technology), and basic control capabilities (providing enterprise-level basic control capabilities to ensure the security and efficient operation of the system). Through the above functions, it provides a comprehensive and integrated device for AI development, training, deployment, and application.

[0180] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the methods and apparatus in the embodiments of this application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the methods in the above embodiments. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the computing terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0181] The processor can invoke an executable program stored in memory via a transmission device to execute any of the methods described in the above embodiments.

[0182] Those skilled in the art will understand that, Figure 15 The structure shown is for illustrative purposes only. Electronic devices can also be smart mobile devices (such as smartphones), tablets, PDAs, and mobile internet devices (MIDs) and other terminal devices. Figure 15 This does not limit the structure of the aforementioned electronic device. For example, electronic device 150 may also include components that are more... Figure 15 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 15 The different configurations shown.

[0183] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0184] Embodiments of this application also provide a computer-readable storage medium. Optionally, in this embodiment, the computer-readable storage medium includes a stored executable program, wherein, when the executable program runs, it controls the device where the computer-readable storage medium is located to execute any of the foregoing methods.

[0185] Optionally, in this embodiment, the storage medium may be located in an electronic device.

[0186] Optionally, in this embodiment, the computer-readable storage medium is configured to store an executable program, which, when the executable program is running, controls the device where the computer-readable storage medium is located to execute any of the methods described in the above embodiments.

[0187] Embodiments of this application also provide a computer program product. Optionally, in this embodiment, the computer program product may include a computer program that, when executed by a processor, implements the methods provided in the embodiments described above.

[0188] Embodiments of this application also provide a computer program product. Optionally, the computer program product may include a non-volatile computer-readable storage medium, which can be used to store a computer program that, when executed by a processor, implements the method provided in the above embodiments.

[0189] Embodiments of this application also provide a computer program. Optionally, in this embodiment, when the computer program is executed by a processor, it implements the method provided in the above embodiments.

[0190] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0191] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between units or modules, and may be electrical or other forms.

[0192] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0193] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0194] If the aforementioned integrated units are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, ROM, RAM, portable hard drives, magnetic disks, or optical disks.

[0195] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A model training method, characterized in that, include: Acquire training data, wherein the training data includes degraded image samples and real image samples; The degraded image samples are restored using an initial restoration model to obtain target semantic features and restored images. The decoding part of the initial restoration model includes an anatomical semantic codebook, which is used to constrain the anatomical structure of the restored image. The target loss is determined based on the degraded image sample, the real image sample, the restored image, and the target semantic features; Based on the target loss, the parameters of the initial recovery model are adjusted to obtain the target recovery model.

2. The model training method according to claim 1, characterized in that, The initial restoration model further includes an encoding part and a prompt generator; the restoration process of the degraded image sample using the initial restoration model to obtain the target semantic features and the restored image includes: In the encoding section, the degraded image samples are encoded to obtain intermediate image features; The intermediate image features are input into the cue generator to generate initial semantic features; In the decoding section, the intermediate image features and initial semantic features are decoded based on the anatomical semantic codebook to obtain the target semantic features and the restored image.

3. The model training method according to claim 2, characterized in that, The coding section includes multiple coding blocks corresponding to multiple levels, and each coding block includes a self-attention unit and a frequency domain feedforward unit. Encoding the degraded image samples to obtain the intermediate image features includes: The initial image features corresponding to the degraded image samples are input into the multiple coding blocks for step-by-step encoding processing to obtain the intermediate image features; In any of the coding blocks, the self-attention unit is used to perform attention weighting processing on the encoded input image features to obtain a self-attention result, and the frequency domain feedforward unit is used to perform frequency domain modulation on the self-attention result to obtain the encoded output image features. In the step-by-step encoding process, the encoded input image features of the first encoding block are the initial image features obtained by pre-encoding processing of the degraded image sample, and the encoded output image features of the last encoding block are the intermediate image features.

4. The model training method according to claim 2, characterized in that, The initial recovery model includes a dynamic frequency domain mask attention component; the method further includes: In the dynamic frequency domain mask attention part, multiple frequency domain masks are used to perform frequency domain feature filtering on the degraded image samples to obtain multiple sets of frequency domain components, and the multiple sets of frequency domain components are cross-attention fused with the intermediate image features to obtain updated intermediate image features.

5. The model training method according to claim 4, characterized in that, The plurality of frequency domain masks include a first frequency domain mask and a second frequency domain mask, wherein a first frequency threshold of the first frequency domain mask is higher than a second frequency threshold of the second frequency domain mask; The degraded image samples are subjected to frequency domain feature filtering using the multiple frequency domain masks to obtain the multiple sets of frequency domain components, including: The degraded image samples are subjected to frequency domain transformation to obtain the third frequency domain features; The first frequency domain mask is used to filter the third frequency domain feature to obtain the high-frequency feature component in the multiple sets of frequency domain components, wherein the high-frequency feature component includes a portion of frequency domain features with a feature frequency higher than the first frequency threshold. The third frequency domain feature is filtered using the second frequency domain mask to obtain the low-frequency feature component among the multiple sets of frequency domain components, wherein the low-frequency feature component includes a portion of frequency domain features with a feature frequency lower than the second frequency threshold.

6. The model training method according to claim 2, characterized in that, The intermediate image features are output to the cue generator to generate the initial semantic features, including: The intermediate image features are semantically mapped to obtain the intermediate semantic embedding; Search the preset semantic dictionary for the target semantic code that matches the intermediate semantic embedding; The initial semantic features are generated based on the target semantic encoding.

7. The model training method according to claim 2, characterized in that, The decoding part includes multiple decoders corresponding to multiple levels, and the multiple decoders are associated with the same anatomical semantic codebook; Based on the anatomical semantic codebook, decoding the intermediate image features and the initial semantic features yields the target semantic features and the reconstructed image, including: The intermediate image features and the initial semantic features are input into the multiple decoders for step-by-step modulation and decoding to obtain the target image features and the target semantic features. The target image features are convolutionally reconstructed to obtain the restored image; In any of the decoders, the input semantic features are modulated and decoded using the anatomical semantic codebook to obtain the output semantic features, and the decoded input image features are modulated and decoded using the anatomical semantic codebook and the output semantic features to obtain the decoded output image features. In the step-by-step modulation decoding, the decoding input image features of the first decoder are the intermediate image features, and the decoding input image features of other decoders are obtained by fusing the encoded output image features of the corresponding level coding block with the decoding output image features of the previous level decoder. The input semantic features of the first decoder are the initial semantic features, and the decoding output image features of the last decoder are the target image features. The output semantic features of the last decoder are the target semantic features.

8. The model training method according to claim 7, characterized in that, The decoded input image features are modulated and decoded using the anatomical semantic codebook and the output semantic features to obtain the decoded output image features, including: The output semantic features are used to modulate the decoded input image features to obtain preliminary modulated image features; The features of the preliminary modulated image are subjected to self-attention processing to obtain the self-attention result; The self-attention result is frequency-domain modulated to obtain the decoded output image features.

9. The model training method according to claim 1, characterized in that, Based on the degraded image sample, the real image sample, the restored image, and the target semantic features, the target loss is determined as follows: Based on the real image samples and the restored image, the reconstruction loss is determined; Based on the degraded image sample, the real image sample, and the target semantic features, determine the semantic loss; The target loss is determined based on the reconstruction loss and the semantic loss.

10. The model training method according to claim 9, characterized in that, Based on the degraded image sample, the real image sample, and the target semantic features, the semantic loss is determined to include: Using a preset model with parameter freezing, semantic features are extracted from the degraded image sample to obtain the first semantic feature, and semantic features are extracted from the real image sample to obtain the second semantic feature; The semantic loss is calculated based on the target semantic features, the first semantic features, and the second semantic features.

11. An image restoration method, characterized in that, include: Obtain the degraded image to be restored; The degraded image is restored using a target restoration model to obtain a restored image, wherein the target restoration model is generated according to the model training method of any one of claims 1 to 10.

12. A computer-aided diagnostic method, characterized in that, include: Acquire medical images of the target object to be restored, wherein the category of the medical images to be restored includes at least one of the following: computed tomography images, magnetic resonance images, and positron emission tomography images; The medical image to be restored is processed using a target restoration model to obtain a restored medical image, wherein the target restoration model is generated according to the model training method of any one of claims 1 to 10; Based on the restored medical images, diagnostic analysis is performed on the target object to obtain auxiliary diagnostic suggestions.

13. An electronic device, characterized in that, include: Memory, which stores executable programs; A processor for running the program, wherein the program, when running, performs the method according to any one of claims 1 to 12.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored executable program, wherein, when the executable program is executed, it controls the device on which the storage medium is located to perform the method according to any one of claims 1 to 12.

15. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 12.

Images