Medical image super-resolution reconstruction method based on multi-scale residual auto-regressive modeling

Abstract

Embodiments of the present disclosure disclose a medical image super-resolution reconstruction method based on multi-scale residual auto-regressive modeling. A specific embodiment of the method comprises: acquiring first resolution medical image data; performing standardization preprocessing on the first resolution medical image data to generate a first resolution medical image matrix; inputting the first resolution medical image matrix into an encoder of a multi-scale vector quantization variational autoencoder to obtain structural prior features; generating a multi-scale discrete label sequence based on a scale adaptive structural decoder and the structural prior features; inputting the multi-scale discrete label sequence into a decoder of the multi-scale vector quantization variational autoencoder, and generating a second resolution medical image matrix through parallel computing cores; and sending the second resolution medical image matrix to a target display device and controlling the target display device to display. The embodiment achieves the technical effects of avoiding image distortion, reducing computational delay, and reducing energy consumption.

Description

Technical Field

[0001] The embodiments disclosed herein relate to the field of computer technology, and more specifically to a method for medical image super-resolution reconstruction based on multi-scale residual autoregressive modeling. Background Technology

[0002] In primary healthcare, pre-hospital emergency care, or resource-constrained scenarios, deploying low-cost, portable, or low-field-strength magnetic resonance imaging (MRI) devices has become a significant development trend. However, constrained by physical design, power consumption, and cost, the hardware sensors of these devices (such as low-intensity magnets and limited-channel RF coils) have inherent performance limitations, resulting in lower spatial resolution and signal-to-noise ratio in the raw images they directly acquire. Improving image quality through computational post-processing methods without upgrading or replacing the aforementioned physical hardware has become a critical technical requirement. Medical image super-resolution reconstruction is an important technology in the fields of medical image digitization and computer vision. It aims to improve the spatial resolution of medical images acquired by imaging devices through algorithms, thereby providing richer visual details and information support for clinical diagnosis, treatment planning, and disease monitoring. Currently, common methods for improving medical image resolution include: traditional interpolation-based methods, such as bilinear or bicubic interpolation; regression methods based on convolutional neural networks, such as deep learning models like SwinIR; and generative model-based methods, such as generative adversarial networks and diffusion models.

[0003] However, when using the above methods to improve the resolution of medical images, the following technical problems often arise: Image distortion, long computational latency, and high energy consumption are significant challenges. Existing generative models based on autoregressive or iterative denoising suffer from significant structural distortions or anatomical errors in the final output image due to the amplification and propagation of small errors in early steps during the sequential generation process performed by the computer. Methods capable of producing high-quality results (such as diffusion models) require extremely intensive iterative computations, leading to long computational latency and high energy consumption. This contradicts the limited computing resources of portable devices or edge servers, making it difficult to complete high-quality reconstruction within a clinically acceptable timeframe.

[0004] The information disclosed in this background section is only intended to enhance the understanding of the background of the inventive concept, and therefore may contain information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The summary portion of this disclosure is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description portion. This summary portion is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0006] Some embodiments of this disclosure propose a method, apparatus, electronic device, and computer-readable medium for medical image super-resolution reconstruction based on multi-scale residual autoregressive modeling to solve one or more of the technical problems mentioned in the background section above.

[0007] In a first aspect, some embodiments of this disclosure provide a method for super-resolution reconstruction of medical images based on multi-scale residual autoregressive modeling. The method includes: acquiring first-resolution medical image data from a portable medical imaging device; performing standardized preprocessing on the first-resolution medical image data to generate a first-resolution medical image matrix; inputting the first-resolution medical image matrix into the encoder of a multi-scale vector quantization variational autoencoder to obtain structural prior features; generating a multi-scale discrete label sequence based on a scale-adaptive structural decoder and the structural prior features; inputting the multi-scale discrete label sequence into the decoder of the multi-scale vector quantization variational autoencoder, and performing parallel decoding operations through the parallel computing core of a graphics processing unit to generate a second-resolution medical image matrix, wherein the resolution of the second-resolution medical image matrix is ​​greater than the resolution of the first-resolution medical image matrix; sending the second-resolution medical image matrix to a target display device, and controlling the target display device to display the image.

[0008] Secondly, some embodiments of this disclosure provide a medical image super-resolution reconstruction apparatus based on multi-scale residual autoregressive modeling. The apparatus includes: an acquisition unit configured to acquire first-resolution medical image data from a portable medical imaging device; a preprocessing unit configured to perform standardized preprocessing on the first-resolution medical image data to generate a first-resolution medical image matrix; an input unit configured to input the first-resolution medical image matrix into the encoder of a multi-scale vector quantization variational autoencoder to obtain structural prior features; a generation unit configured to generate a multi-scale discrete label sequence based on a scale-adaptive structural decoder and the structural prior features; a decoding unit configured to input the multi-scale discrete label sequence into the decoder of the multi-scale vector quantization variational autoencoder and perform parallel decoding operations through the parallel computing core of a graphics processor to generate a second-resolution medical image matrix, wherein the resolution of the second-resolution medical image matrix is ​​greater than the resolution of the first-resolution medical image matrix; and a display unit configured to send the second-resolution medical image matrix to a target display device and control the target display device to display it.

[0009] Thirdly, some embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation of the first aspect above.

[0010] Fourthly, some embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method described in any of the implementations of the first aspect above.

[0011] The above embodiments of this disclosure have the following beneficial effects: The medical image super-resolution reconstruction method based on multi-scale residual autoregressive modeling, through a combination of multi-scale vector quantization, structural conditional autoregressive generation, and hierarchical bundle search optimization, achieves stable and efficient generation of high-resolution medical images with structural fidelity without upgrading physical hardware, overcoming the inherent performance limitations of portable medical imaging devices. Specifically, the reasons for image distortion, long computational latency, and high energy consumption are as follows: Existing generative models based on autoregression or iterative denoising, when executed by a computer in the serialization generation step, small errors in early steps are continuously amplified and propagated in subsequent steps, leading to significant, global structural distortion or anatomical errors in the final output image. Existing methods that can produce high-quality results (such as diffusion models) require extremely intensive iterative computation by the computer, resulting in long computational latency and high energy consumption. This contradicts the limited computing resources of portable devices or edge servers, making it difficult to complete high-quality reconstruction within a clinically acceptable timeframe. Based on this, some embodiments of the medical image super-resolution reconstruction method based on multi-scale residual autoregressive modeling disclosed herein first acquire first-resolution medical image data from a portable medical imaging device. This provides the entire reconstruction process with raw input data to be enhanced, directly acquired by the hardware. Second, the first-resolution medical image data undergoes standardization preprocessing to generate a first-resolution medical image matrix. This eliminates inconsistencies in numerical format, intensity distribution, and spatial orientation caused by different devices and scanning protocols. Then, the first-resolution medical image matrix is ​​input into the encoder of a multi-scale vector quantization variational autoencoder to obtain structural prior features. This extracts feature representations characterizing global anatomical structures, aligned with the latent space of the high-resolution image, from the low-quality input, providing structural constraints for subsequent generation processes and preventing anatomical structural distortions in the generated results. Finally, a multi-scale discrete label sequence is generated based on the scale-adaptive structural decoder and the aforementioned structural prior features. Therefore, under strong structural prior constraints, the discrete representation of the high-resolution image can be predicted step-by-step and position-by-position through autoregressive modeling. Furthermore, a hierarchical bundle search strategy is introduced to suppress error accumulation caused by local optima during the autoregressive process. Next, the aforementioned multi-scale discrete label sequence is input into the decoder of the multi-scale vector quantization variational autoencoder, and parallel decoding is performed through the parallel computing core of the graphics processing unit to generate a second-resolution medical image matrix.Therefore, discrete labeled sequences can be efficiently reconstructed into continuous image data. Furthermore, by leveraging the parallel computing capabilities of the graphics processing unit (GPU), the reconstruction tasks for different image regions can be distributed to multiple computing cores for simultaneous execution, accelerating the generation process of the high-resolution image matrix. Finally, the aforementioned second-resolution medical image matrix is ​​sent to the target display device, and the target display device is controlled to display it. This allows the high-quality digital image reconstructed by the algorithm to be output to the physical display interface, completing a full technical loop from data acquisition and algorithm processing to result visualization. This implementation achieves the technical effects of avoiding image distortion, reducing computational latency, and lowering energy consumption. Attached Figure Description

[0012] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0013] Figure 1 This is a flowchart of some embodiments of the medical image super-resolution reconstruction method based on multi-scale residual autoregressive modeling disclosed herein; Figure 2 These are schematic diagrams of some embodiments of the medical image super-resolution reconstruction device based on multi-scale residual autoregressive modeling according to this disclosure; Figure 3 This is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. Detailed Implementation

[0014] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0015] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0016] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0017] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0018] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0019] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0020] Figure 1 A flowchart 100 is shown illustrating some embodiments of a medical image super-resolution reconstruction method based on multi-scale residual autoregressive modeling according to this disclosure. This medical image super-resolution reconstruction method based on multi-scale residual autoregressive modeling includes the following steps: Step 101: Acquire first-resolution medical image data from a portable medical imaging device.

[0021] In some embodiments, the execution entity of the medical image super-resolution reconstruction method based on multi-scale residual autoregressive modeling (e.g., a server deployed in a stroke ambulance) can acquire first-resolution medical image data from a portable medical imaging device. In practice, the execution entity can acquire the first-resolution medical image data through a data interface connected to the portable medical imaging device. The portable medical imaging device can be a mobile medical device used to generate images of internal human structures. Typically, the portable medical imaging device is characterized by miniaturization and low power consumption in its hardware design, resulting in a lower physical resolution (i.e., spatial detail resolution) compared to large, fixed, similar devices. For example, the portable medical imaging device can be a portable computed tomography (CT) scanner or a portable magnetic resonance imaging (MRI) scanner. The data interface can be a DICOM (Digital Imaging and Communication in Medicine) network interface, a USB (Universal Serial Bus) interface, an Ethernet interface, or a wireless network interface (such as Wi-Fi). For example, the first-resolution medical image data can be a medical image with a resolution of 64×64 pixels.

[0022] Step 102: Standardize and preprocess the first-resolution medical image data to generate a first-resolution medical image matrix.

[0023] In some embodiments, the executing entity may perform standardized preprocessing on the first-resolution medical image data to generate a first-resolution medical image matrix. For example, the resolution of the first-resolution medical image matrix may be 64×64 pixels.

[0024] In some optional implementations of certain embodiments, the aforementioned execution entity may perform standardized preprocessing on the aforementioned first-resolution medical image data through the following steps to generate a first-resolution medical image matrix: The first step involves converting the aforementioned first-resolution medical image data into a format-uniformed image data based on a preset numerical format and a preset color channel format. In practice, the executing entity can convert the numerical format of the pixel value array of the first-resolution medical image data (e.g., 16-bit signed integer or 8-bit unsigned integer) into a preset numerical format to obtain format-uniformed first-resolution medical image data. Then, the color channel format of the format-uniformed first-resolution medical image data is converted into a preset color channel format to obtain format-uniformed image data. The preset numerical format can be a single-precision floating-point number. The preset color channel format can be RGB. For example, the executing entity can convert the numerical format of the pixel value array of the first-resolution medical image data from an 8-bit unsigned integer to a single-precision floating-point number. The color channel format of the format-uniformed first-resolution medical image data is then converted from BGR to RGB.

[0025] The second step is to normalize the intensity of the above-mentioned uniformly formatted image data to obtain normalized image data. In practice, the above-mentioned execution entity can use the Z-Score normalization algorithm to scale the pixel intensity of the above-mentioned uniformly formatted image data to the [0, 1] interval to obtain normalized image data.

[0026] The third step involves spatially aligning the normalized image data to obtain a first-resolution medical image matrix. In practice, the executing entity can read the medical image metadata attached to the normalized image data. Next, using the RAS coordinate system redirection method, the normalized image data is transformed from the image coordinate system to the standard anatomical coordinate system to obtain the first-resolution medical image matrix. The medical image metadata can be information describing the attributes of the normalized image data. For example, the medical image metadata can be the image orientation in a medical image conforming to the DICOM (Digital Imaging and Communications in Medicine) format. The RAS coordinate system redirection method can be the Reorient function or Reorientation to RAS function in medical image processing software (such as 3D Slicer, ITK-SNAP) and image processing libraries (such as SimpleITK, NiBabel).

[0027] Step 103: Input the first-resolution medical image matrix into the encoder of the multi-scale vector quantization variational autoencoder to obtain structural prior features.

[0028] In some embodiments, the execution entity may input the first-resolution medical image matrix into the encoder of a multi-scale vector quantization variational autoencoder to obtain structural prior features. These structural prior features may be embedded representations extracted from the first-resolution medical image matrix, representing geometric and topological constraints for characterizing target anatomical structures (such as brain blood vessels or tissue boundaries).

[0029] The aforementioned multi-scale vector quantization variational autoencoder is obtained through the following steps: The first step is to obtain the second-resolution training dataset. In practice, the aforementioned execution entity can obtain the second-resolution training dataset from a pre-defined medical image database through a database query interface. This database query interface can be an SQL interface or a DICOM network interface. The pre-defined medical image database can be a publicly available medical image dataset, such as the BraTS (brain tumor segmentation) or LIDC-IDRI (lung nodule) database. As an example, the second-resolution training data in the aforementioned second-resolution training dataset can be medical image data with a resolution of 512×512 pixels.

[0030] The second step involves inputting the aforementioned second-resolution training dataset into the encoder of the initial multi-scale vector quantization variational autoencoder to obtain a set of multi-scale latent feature maps. The initial multi-scale vector quantization variational autoencoder can be a neural network model comprising an encoder, a vector quantization codebook, and a decoder. The encoder can be a deep convolutional neural network that takes an image as input and outputs a discretized feature representation map. The encoder may include convolutional layers, downsampling layers (such as pooling layers), and non-linear activation functions (such as ReLU). The vector quantization codebook can be a learnable, discrete lookup table. The vector quantization codebook stores a fixed number of prototype feature vectors (also called embedding vectors). The decoder can be a deep convolutional neural network that takes a discretized feature representation map as input and outputs a reconstructed image. The decoder may include upsampling layers (such as transposed convolution), convolutional layers, and non-linear activation functions. The set of multi-scale latent feature maps can be a collection of feature maps organized according to a specific spatial scale (such as downsampling ratios of 1:8, 1:4, 1:2, etc.). The aforementioned multi-scale latent feature map set includes feature maps at multiple scales (such as 1:8, 1:4, 1:2, etc.) to hierarchically represent the semantic and structural information of the input image.

[0031] The third step involves inputting the aforementioned multi-scale latent feature map set into the decoder of the initial multi-scale vector quantization variational autoencoder to obtain the second resolution prediction dataset. This second resolution prediction dataset can be an image data set reconstructed from the multi-scale latent feature map set by the decoder of the initial model.

[0032] The fourth step is to determine the reconstruction loss between the second-resolution prediction dataset and the second-resolution training dataset. In practice, the execution entity can call the loss function calculation module to determine the reconstruction loss between the corresponding images in the second-resolution prediction dataset and the second-resolution training dataset pixel by pixel. The reconstruction loss can be L1 loss or L2 loss. The loss function calculation module can be a standard loss function provided by a deep learning framework. For example, the loss function calculation module can be the torch.nn.L1Loss (mean absolute error) module or the torch.nn.MSELoss (mean squared error) module in PyTorch.

[0033] Fifth, in response to the reconstruction loss being less than a first preset loss threshold, the initial multi-scale vector quantization variational autoencoder is determined as a multi-scale vector quantization variational autoencoder. The first preset loss threshold can be a preset numerical condition used to analyze whether the model training has converged. For example, the first preset loss threshold could be 1e-4.

[0034] In some optional implementations of certain embodiments, the aforementioned execution entity may obtain a set of multi-scale latent feature maps by inputting the second-resolution training dataset into the encoder of an initial multi-scale vector quantization variational autoencoder through the following steps: The first step involves performing a downsampling convolution operation on the second-resolution training dataset using the encoder of the aforementioned initial multi-scale vector quantization variational autoencoder to obtain initial latent features. In practice, the execution entity can input the second-resolution training dataset into the encoder of the aforementioned initial multi-scale vector quantization variational autoencoder to obtain the initial latent features. These initial latent features can be tensor data obtained by nonlinearly transforming the second-resolution training dataset using the encoder.

[0035] The second step involves dividing the initial latent features into latent features of different scales based on a preset multi-scale sequence, resulting in a multi-scale latent feature sequence. In practice, the execution entity can downsample or convolve the initial latent features based on the scale ratios in the preset multi-scale sequence to obtain feature maps with different spatial resolutions. Then, the feature maps with different spatial resolutions are arranged in descending order of scale to obtain the multi-scale latent feature sequence. The preset multi-scale sequence can be a predefined list of scale ratios. For example, the preset multi-scale sequence could be [1, 1 / 2, 1 / 4]. The multi-scale latent feature sequence can be a set of continuous feature maps arranged from coarse to fine. The downsampling can be pooling.

[0036] The third step is to determine the aforementioned initial latent features as the quantized reconstruction result. This quantized reconstruction result can be used to determine the residual between the current scale and the reconstructed portion.

[0037] Fourth step: Based on the above-mentioned preset multi-scale sequence, perform the following recursive multi-scale residual quantization operation on each multi-scale latent feature in the above multi-scale latent feature sequence: Step one: Based on the above quantization reconstruction results, determine the scale residual feature map of the above multi-scale latent features. In practice, the executing entity can subtract the above multi-scale latent features from the quantization reconstruction results element-wise to obtain the scale residual feature map.

[0038] Step two involves dividing the aforementioned scale residual feature map into a set of feature vectors. In practice, the executing entity can divide the scale residual feature map into multiple small blocks in the spatial dimension, and perform a flattening operation on the feature values ​​of all positions and all channels within each small block to generate feature vectors, thus obtaining a set of feature vectors.

[0039] Step 3: Perform a nearest neighbor search on each feature vector in the aforementioned feature vector set to obtain a discrete codebook index set. In practice, the execution entity can determine the Euclidean distance between each feature vector in the aforementioned feature vector set and each embedding vector in the aforementioned vector quantization codebook, and record the position number (index) of the embedding vector with the smallest Euclidean distance in the aforementioned vector quantization codebook, thus obtaining the discrete codebook index set. The aforementioned nearest neighbor search can be an algorithm that finds the sample closest to the query sample in a given dataset. The aforementioned discrete codebook index set can be a set of integer values, where each value represents the position number of the corresponding feature vector in the vector quantization codebook where it is matched.

[0040] Step four: Based on the aforementioned discrete codebook index set, generate an embedding vector set. In practice, the execution entity can search for the corresponding embedding vector in the aforementioned vector quantization codebook based on each discrete codebook index in the aforementioned discrete codebook index set, thus obtaining the embedding vector set.

[0041] Step 5: Assemble the above-mentioned set of embedded vectors to obtain a quantized feature map. In practice, the execution entity can adjust the shape of all vectors in the above-mentioned set of embedded vectors and arrange them on a two-dimensional spatial grid according to the correspondence when dividing them in Step 2, to obtain a quantized feature map.

[0042] Step six involves upsampling the quantized feature map to obtain a scale-quantized reconstruction result, and then summing the scale-quantized reconstruction result with the quantized reconstruction result to determine the final quantized reconstruction result. In practice, the execution entity can adjust the quantized feature map to the same spatial size as the initial latent features through an upsampling operation to obtain a scale-quantized reconstruction result. Subsequently, the scale-quantized reconstruction result is added element-wise with the quantized reconstruction result, and the sum is used to update the new quantized reconstruction result. The upsampling operation can be a transposed convolution or interpolation.

[0043] The fifth step is to determine each quantized feature map as a set of multi-scale latent feature maps. Each multi-scale latent feature map in the aforementioned set corresponds to a spatial scale in the preset multi-scale sequence.

[0044] Step 104: Generate a multi-scale discrete label sequence based on the scale-adaptive structure decoder and structural prior features.

[0045] In some embodiments, the execution entity described above can generate a multi-scale discrete label sequence based on the scale-adaptive structure decoder and the aforementioned structural prior features. The multi-scale discrete label sequence can be an ordered label sequence, where each label can be an index in the aforementioned vector quantization codebook.

[0046] The aforementioned scale-adaptive structure decoder is obtained through the following steps: The first step is to downsample the second-resolution training dataset to obtain the first-resolution training dataset. This downsampling reduces the image resolution by decreasing the spatial size (number of pixels). For example, the resolution of the first-resolution training data in the first-resolution training dataset could be 64×64 pixels or 128×128 pixels.

[0047] The second step involves inputting the first-resolution training dataset into the encoder of the multi-scale vector quantization variational autoencoder to obtain a conditional feature set. This conditional feature set can be a set of tensor data output from the first-resolution training dataset after processing by the encoder. This conditional feature set can represent prior information such as the global anatomical structure of the input image and can be used as conditional input to the scale-adaptive structural decoder.

[0048] The third step involves inputting the aforementioned conditional feature set and the aforementioned multi-scale latent feature map set into the initial scale adaptive structure decoder, and determining the prediction loss of the initial scale adaptive structure decoder. In practice, the executing entity can input the aforementioned conditional feature set and the aforementioned multi-scale latent feature map set into the initial scale adaptive structure decoder to obtain the predicted label sequence and the true label sequence. Then, the executing entity can determine the prediction loss between the predicted label sequence and the true label sequence using the cross-entropy loss function. The initial scale adaptive structure decoder can be an autoregressive generative neural network that takes structural prior features as input and multi-scale discrete label sequences as output. The initial scale adaptive structure decoder can include a label embedding layer, a structural conditional fusion module, a multi-scale attention mechanism layer, a feedforward network, and an output layer. The label embedding layer can be a lookup table used to assign a learnable feature vector to each discrete label index, thereby converting the symbol sequence into a numerical tensor that the neural network can process. The structural conditional fusion module can be a cross-attention mechanism layer that takes the aforementioned structural prior features as key and value inputs, the output of the aforementioned label embedding layer as query input, and a contextual feature representation that incorporates structural conditions as output. The aforementioned multi-scale attention mechanism layer can be a Transformer decoder layer that takes the contextual feature representation fused with structural conditions and positional encoding information as input, and outputs a new feature sequence after weighting and aggregating multi-scale contextual information. The aforementioned feedforward network includes fully connected layers, activation functions (such as GELU), and residual connected networks. The aforementioned output layer can include linear projection layers and a softmax function. The aforementioned predicted label sequence is the label sequence output when the aforementioned conditional feature set is used as input to train the aforementioned initial scale adaptive structure decoder. The aforementioned true label sequence can be the label sequence obtained after inputting the aforementioned multi-scale latent feature map set into the aforementioned initial scale adaptive structure decoder. Each label in the aforementioned label sequence is an integer value, corresponding to an index in the aforementioned vector quantization codebook.

[0049] Fourth, in response to the predicted loss being less than the second preset loss threshold, the initial scale-adaptive structure decoder is determined as a scale-adaptive structure decoder. The second preset loss threshold can be a preset value. For example, the second preset loss threshold can be 1e-4.

[0050] In addressing the aforementioned technical problems in the process of adopting technical solutions, the application scenario—real-time image analysis based on portable MRI / CT in pre-hospital emergency care (such as stroke ambulances)—often presents the following technical challenges: during the autoregressive generation of multi-scale discrete representation sequences of medical images, errors caused by the high local similarity of medical anatomical structures propagate and accumulate along the sequence, leading to structural distortion or artifacts in the final reconstructed image. Considering the following requirements for this application scenario—high fault tolerance, adaptability to low-quality input data, and low computational latency—we have decided to adopt the following solution: In some optional implementations of certain embodiments, the aforementioned execution entity can generate a multi-scale discrete label sequence based on the scale-adaptive structure decoder and the aforementioned structural prior features through the following steps: The first step involves creating a candidate label sequence set and a starting label, and storing the starting label into the candidate label sequence set. In practice, the executing entity can initialize an empty data structure (such as a list or queue) in memory to store the candidate label sequences, forming the candidate label sequence set. Then, the executing entity can define a starting label and store it into the candidate label sequence set. The candidate label sequence set can be a list or queue used to store the candidate label sequences. The starting label can be a predefined, special label used to initialize the autoregressive generation process. The starting label does not carry specific image semantics and can be used to identify the start of a sequence. For example, the starting label could be... <sos>Each candidate token in the above candidate token sequence can be an integer value, corresponding to an index in the above vector quantization codebook.

[0051] The second step is to perform the following operation for each candidate label sequence in the above candidate label sequence set: Step one: Based on the aforementioned structural prior features and the scale-adaptive structural decoder, determine the conditional probability distribution of the next candidate label in the aforementioned candidate label sequence. In practice, the executing entity can input the aforementioned candidate label sequence and the aforementioned structural prior features into the scale-adaptive structural decoder to obtain the conditional probability distribution of the next candidate label in the aforementioned candidate label sequence. The conditional probability distribution can be all possible candidate labels at the next position in the aforementioned candidate label sequence and their corresponding probabilities of occurrence.

[0052] Step two: Based on the aforementioned conditional probability distribution, generate at least one candidate label for the aforementioned candidate label sequence, and concatenate the at least one candidate label with the aforementioned candidate label sequence to obtain a set of concatenated candidate label sequences. In practice, the executing entity can select at least one candidate label with the highest probability in the aforementioned conditional probability distribution and add the at least one candidate label to the end of the aforementioned candidate label sequence to generate at least one new, longer concatenated candidate label sequence, thus obtaining a set of concatenated candidate label sequences.

[0053] Step 3: Based on a preset scoring function, determine the scoring result of each candidate marker sequence in the aforementioned candidate marker sequence set, obtaining a scoring result set. The preset scoring function can be expressed as the sum of the logarithms of the occurrence probabilities of all candidate markers in the candidate marker sequence. The scoring result can be the function value of the preset scoring function corresponding to the candidate marker sequence. The higher the scoring result, the greater the probability that the corresponding candidate marker sequence will occur under given conditions.

[0054] Step four: Based on the aforementioned scoring result set and the preset bundle width, at least one concatenation candidate marker sequence is selected to obtain a second concatenation candidate marker sequence set. Then, based on the second concatenation candidate marker sequence set, the aforementioned candidate marker sequence set is updated to obtain a candidate marker sequence set. In practice, the executing entity can sort the concatenation candidate marker sequence set in descending order based on the scoring results in the aforementioned scoring result set to obtain a sorted concatenation candidate marker sequence set. From this sorted set, a concatenation candidate marker sequence that satisfies the preset bundle width is selected as the second concatenation candidate marker sequence, resulting in the second concatenation candidate marker sequence set. Then, the executing entity can completely replace the contents of the first candidate marker sequence set with the second concatenation candidate marker sequence set to obtain the candidate marker sequence set. The preset bundle width can be a predefined positive integer value. For example, the preset bundle width can be 3.

[0055] The third step involves determining the multi-scale discrete label sequence as the candidate label sequence with the highest score in the candidate label sequence set, in response to the candidate label sequence in the aforementioned candidate label sequence set satisfying a preset termination condition. The preset termination condition can be a predefined condition used to determine when the autoregressive generation process ends. For example, the preset termination condition may include: the candidate label sequence contains a special label indicating the end (e.g., ...). <eos>(or the length of the candidate sequence has reached the maximum allowed value).

[0056] The first to third steps and related content described above, as an inventive point of this disclosure, combined with step "step 106," solve the technical problem that "in the process of generating a multi-scale discrete representation sequence of medical images through autoregression, errors caused by high local similarity of medical anatomical structures propagate and accumulate along the sequence, leading to structural distortion or artifacts in the final reconstructed image." The factors leading to structural distortion or artifacts in the reconstructed image are often as follows: in the process of generating a multi-scale discrete representation sequence of medical images through autoregression, errors caused by high local similarity of medical anatomical structures propagate and accumulate along the sequence, leading to structural distortion or artifacts in the final reconstructed image. Solving these factors can avoid structural distortion or artifacts in the reconstructed image. To achieve this effect, firstly, a candidate label sequence set and a starting label are created, and the starting label is stored in the candidate label sequence set. This initializes a search space for the hierarchical bundle search algorithm, containing a defined starting path, providing a basis for subsequent iterative path expansion, evaluation, and selection, ensuring the orderly start of the generation process. Second, for each candidate label sequence in the aforementioned candidate label sequence set, the following operations are performed: Step 1, based on the aforementioned structural prior features and the aforementioned scale-adaptive structural decoder, determine the conditional probability distribution of the next candidate label in the aforementioned candidate label sequence. Thus, using the aforementioned structural prior features as global constraints and all generated labels in the aforementioned candidate label sequence as historical context, the computer quantitatively analyzes the likelihood of all possible next labels appearing at the current position, providing a probabilistic basis for generation decisions. Step 2, based on the aforementioned conditional probability distribution, generate at least one candidate label for the aforementioned candidate label sequence, and concatenate the at least one candidate label with the aforementioned candidate label sequence to obtain a set of concatenated candidate label sequences. Thus, spatial expansion can be performed on each current candidate path to explore multiple possible subsequent generation directions, forming a new and longer set of candidate paths. Step 3, based on a preset scoring function, determine the scoring results of each concatenated candidate label sequence in the aforementioned set of concatenated candidate label sequences, obtaining a set of scoring results. Thus, all expanded new paths can be quantitatively evaluated and compared from the perspective of generation quality. The aforementioned scoring function is usually based on the cumulative log probability of the sequence, which can comprehensively reflect the overall generation quality of each path. Step four: Based on the above scoring result set and the preset bundle width, at least one candidate splicing label sequence is selected to obtain a second set of candidate splicing label sequences. Then, based on the second set of candidate splicing label sequences, the candidate label sequence set is updated to obtain a final set of candidate label sequences. Thus, at each step of the search, only the highest-scoring, i.e., optimal candidate paths (the number determined by the bundle width) are retained, while other suboptimal paths are discarded.The aforementioned "pruning" operation, while controlling computational complexity, effectively suppresses the continued expansion of erroneous paths caused by early suboptimal selections, thereby significantly reducing the risk of error accumulation in the autoregressive process and improving the robustness of the generation process. In the third step, in response to the candidate label sequences in the aforementioned candidate label sequence set meeting the preset termination condition, the candidate label sequence with the highest score in the aforementioned candidate label sequence set is determined as the multi-scale discrete label sequence. Finally, combined with step "106," the high-quality discrete representation obtained from the optimized and stable autoregressive generation process is efficiently reconstructed into high-resolution image pixel data through parallel decoding, and then driven by a physical display device for visualization. This forms a complete and reliable technical closed loop from algorithm generation to clinical visualization output, achieving the effect of avoiding structural distortion or artifacts in the reconstructed image.

[0057] Step 105: The multi-scale discrete label sequence is input into the decoder of the multi-scale vector quantization variational autoencoder, and parallel decoding is performed through the parallel computing core of the graphics processor to generate a second-resolution medical image matrix.

[0058] In some embodiments, the execution entity can input the multi-scale discrete label sequence into the decoder of the multi-scale vector quantization variational autoencoder, and perform parallel decoding operations through the parallel computing core of the graphics processing unit (GPU) to generate a second-resolution medical image matrix. The resolution of the second-resolution medical image matrix is ​​greater than that of the first-resolution medical image matrix. For example, the resolution of the second-resolution medical image matrix can be 512×512 pixels. The parallel computing core can be a basic physical computing unit within the GPU capable of independently executing instructions and performing arithmetic and logical operations.

[0059] In addressing the technical challenges mentioned above, the application scenario of bedside real-time image reconstruction in pre-hospital emergency mobile units (such as stroke / trauma ambulances) often presents the following technical issues: On computationally limited terminal hardware (such as vehicle-mounted computers), the significant processing latency caused by decoding and reconstructing high-resolution medical image matrices from high-dimensional, multi-scale discrete labeled sequences makes it difficult to meet the stringent real-time requirements of clinical real-time diagnosis. Considering the following requirements for this application scenario: real-time processing capability, adaptability to resource-constrained equipment, and compliance with clinical quality standards, we have decided to adopt the following solution: In some optional implementations of certain embodiments, the execution entity may, through the following steps, input the multi-scale discrete label sequence into the decoder of the multi-scale vector quantization variational autoencoder, and perform parallel decoding operations through the parallel computing core of the graphics processor to generate a second-resolution medical image matrix: The first step is to divide the aforementioned multi-scale discrete marker sequence into at least two marker subsequences in the spatial dimension, thus obtaining a set of marker subsequences. In practice, the executing entity can cut the aforementioned multi-scale discrete marker sequence into at least two consecutive, non-overlapping subsequence segments to obtain a set of marker subsequences.

[0060] The second step involves assigning each of the aforementioned tag subsequences in the set to at least two parallel computing cores of the graphics processing unit (GPU). In practice, the execution entity can treat each tag subsequence in the set as an independent data processing task and assign it to a separate parallel computing core of the GPU. This parallel computing core can be a CUDA core (Compute Unified Device Architecture Core) or a Streaming Multiprocessor (SM).

[0061] The third step is to drive at least two of the above parallel computing cores to synchronously execute the following decoding operations based on the above decoder: Step one: Query the vector quantization codebook for the embedding vectors at multiple scales corresponding to the labeled subsequence. In practice, the execution entity can use the integer values ​​of each label in the labeled subsequence as indexes to search the vector quantization codebook and generate the corresponding embedding vectors at multiple scales.

[0062] Step two involves element-wise summing of the embedding vectors from the multiple scales to obtain fused multi-scale information. This fused multi-scale information can be a single feature vector obtained by fusing (e.g., summing) embedding vectors from different scales at the same spatial location.

[0063] Step 3: Perform an upsampling operation on the above fused multi-scale information to obtain the fused feature map.

[0064] Step four involves mapping the fused feature map to the image space to generate a second-resolution medical image sub-block matrix. In practice, the executing entity can input the fused feature map into a convolutional layer to transform the fused feature map from a high-dimensional feature channel count to the target image channel count (e.g., 3 channels), thus obtaining the second-resolution medical image sub-block matrix. This second-resolution medical image sub-block matrix can be second-resolution image data of a local image region, or it can be a three-dimensional tensor (height × width × color channels).

[0065] The fourth step is to stitch together the second-resolution medical image sub-block matrices generated by the decoding of each parallel computing core to obtain the second-resolution medical image matrix.

[0066] The first to fourth steps and related content described above, as an inventive point of this disclosure, combined with step "step 106 (sending the second resolution medical image matrix to the target display device and controlling the target display device to display it)," solves the technical problem that "on terminal hardware with limited computing resources (such as an in-vehicle computer), the significant processing delay caused by the computationally intensive operations when decoding and reconstructing a high-resolution medical image matrix from a high-dimensional, multi-scale discrete label sequence makes it difficult to meet the stringent real-time requirements of clinical real-time diagnosis for image generation speed." The factors causing significant processing delays in image generation are often as follows: on terminal hardware with limited computing resources (such as an in-vehicle computer), the significant processing delay caused by the computationally intensive operations when decoding and reconstructing a high-resolution medical image matrix from a high-dimensional, multi-scale discrete label sequence makes it difficult to meet the stringent real-time requirements of clinical real-time diagnosis for image generation speed. If these factors are resolved, the image reconstruction processing delay can be reduced. To achieve this effect, firstly, the aforementioned multi-scale discrete label sequence is divided into at least two label sub-sequences in the spatial dimension to obtain a set of label sub-sequences. Therefore, the massive decoding computation task corresponding to the entire high-resolution image can be decomposed into multiple independent, smaller-granular subtasks, laying the foundation for subsequent simultaneous processing of multiple subtasks using parallel computing hardware, and realizing reasonable decomposition of computational load and parallel design of tasks. Second, each of the aforementioned marker subsequences in the set of marker subsequences is assigned to at least two parallel computing cores of the aforementioned graphics processor. Thus, the decomposed subtasks can be mapped to multiple physical computing units possessed by the graphics processor, enabling each computing core to independently and simultaneously process one subtask, thereby fully utilizing the large-scale parallel computing architecture of the graphics processor and transforming the originally serial decoding process into a highly parallel computational flow. Third, the aforementioned at least two parallel computing cores are driven to synchronously execute the following decoding operations based on the aforementioned decoder: Step 1, query the embedding vectors of multiple scales corresponding to the marker subsequences in the vector quantization codebook. Thus, each parallel computing core can independently convert the discrete marker subsequences it is responsible for back into continuous, numerically computable feature vector representations. This is the first step in restoring discrete symbol information to continuous image features, providing a data foundation for subsequent feature fusion and upsampling. Step two involves element-wise summing of the embedding vectors from the multiple scales to obtain fused multi-scale information. This allows for the effective integration of feature information from different scales at each spatial location. Coarse-scale features provide global structural context, while fine-scale features supplement local details, resulting in a more complete local feature representation. Step three involves upsampling the fused multi-scale information to obtain a fused feature map. This gradually improves the spatial resolution of the local feature map, restoring the proper size and spatial structure of image sub-blocks in stages.Step four involves mapping the fused feature map to the image space to generate a second-resolution medical image sub-block matrix. This allows for the conversion of high-dimensional features into standard image formats (such as RGB or grayscale pixel values) through final convolutional layers and other network transformations, thus completing the generation from abstract features to specific image sub-blocks and outputting local image data that can be directly displayed and analyzed. Step four involves stitching together the second-resolution medical image sub-block matrices generated by the parallel computing cores to obtain the second-resolution medical image matrix. This allows the parallel-generated sub-block matrices corresponding to different regions of the image to be reassembled according to their original spatial relationships, forming a complete and seamless high-resolution medical image, ensuring the correctness and global consistency of the parallel computing results. Finally, combined with step 106 (sending the second-resolution medical image matrix to the target display device and controlling the target display device to display it), the high-quality, high-resolution medical image generated through efficient parallel decoding is displayed promptly, reducing image reconstruction processing latency.

[0067] Step 106: Send the second-resolution medical image matrix to the target display device and control the target display device to display it.

[0068] In some embodiments, the execution entity can send the second-resolution medical image matrix to a target display device and control the target display device to display it. In practice, the execution entity can send the second-resolution medical image matrix to the target display device via a video output interface. Then, it drives the graphics processing unit of the target display device to call the frame buffer to load the second-resolution medical image matrix and render the second-resolution medical image onto the display screen. The target display device can be a diagnostic-grade medical monitor or a mobile device screen. For example, the target display device can be a tablet computer or a display screen.

[0069] The above embodiments of this disclosure have the following beneficial effects: The medical image super-resolution reconstruction method based on multi-scale residual autoregressive modeling, through a combination of multi-scale vector quantization, structural conditional autoregressive generation, and hierarchical bundle search optimization, achieves stable and efficient generation of high-resolution medical images with structural fidelity without upgrading physical hardware, overcoming the inherent performance limitations of portable medical imaging devices. Specifically, the reasons for image distortion, long computational latency, and high energy consumption are as follows: Existing generative models based on autoregression or iterative denoising, when executed by a computer in the serialization generation step, small errors in early steps are continuously amplified and propagated in subsequent steps, leading to significant, global structural distortion or anatomical errors in the final output image. Existing methods that can produce high-quality results (such as diffusion models) require extremely intensive iterative computation by the computer, resulting in long computational latency and high energy consumption. This contradicts the limited computing resources of portable devices or edge servers, making it difficult to complete high-quality reconstruction within a clinically acceptable timeframe. Based on this, some embodiments of the medical image super-resolution reconstruction method based on multi-scale residual autoregressive modeling disclosed herein first acquire first-resolution medical image data from a portable medical imaging device. This provides the entire reconstruction process with raw input data to be enhanced, directly acquired by the hardware. Second, the first-resolution medical image data undergoes standardization preprocessing to generate a first-resolution medical image matrix. This eliminates inconsistencies in numerical format, intensity distribution, and spatial orientation caused by different devices and scanning protocols. Then, the first-resolution medical image matrix is ​​input into the encoder of a multi-scale vector quantization variational autoencoder to obtain structural prior features. This extracts feature representations characterizing global anatomical structures, aligned with the latent space of the high-resolution image, from the low-quality input, providing structural constraints for subsequent generation processes and preventing anatomical structural distortions in the generated results. Finally, a multi-scale discrete label sequence is generated based on the scale-adaptive structural decoder and the aforementioned structural prior features. Therefore, under strong structural prior constraints, the discrete representation of the high-resolution image can be predicted step-by-step and position-by-position through autoregressive modeling. Furthermore, a hierarchical bundle search strategy is introduced to suppress error accumulation caused by local optima during the autoregressive process. Next, the aforementioned multi-scale discrete label sequence is input into the decoder of the multi-scale vector quantization variational autoencoder, and parallel decoding is performed through the parallel computing core of the graphics processing unit to generate a second-resolution medical image matrix.Therefore, discrete labeled sequences can be efficiently reconstructed into continuous image data. Furthermore, by leveraging the parallel computing capabilities of the graphics processing unit (GPU), the reconstruction tasks for different image regions can be distributed to multiple computing cores for simultaneous execution, accelerating the generation process of the high-resolution image matrix. Finally, the aforementioned second-resolution medical image matrix is ​​sent to the target display device, and the target display device is controlled to display it. This allows the high-quality digital image reconstructed by the algorithm to be output to the physical display interface, completing a full technical loop from data acquisition and algorithm processing to result visualization. This implementation achieves the technical effects of avoiding image distortion, reducing computational latency, and lowering energy consumption.

[0070] Continue to refer to Figure 2 As a response to the above Figure 1 The present disclosure provides some embodiments of a medical image super-resolution reconstruction device based on multi-scale residual autoregressive modeling, which are similar to the implementation of the method shown. Figure 1 Corresponding to the method embodiments shown, the device can be specifically applied to various electronic devices.

[0071] like Figure 2 As shown, a medical image super-resolution reconstruction device 200 based on multi-scale residual autoregressive modeling in some embodiments includes: an acquisition unit 201, a preprocessing unit 202, an input unit 203, a generation unit 204, a decoding unit 205, and a display unit 206. The acquisition unit 201 is configured to acquire first-resolution medical image data from a portable medical imaging device; the preprocessing unit 202 is configured to perform standardized preprocessing on the first-resolution medical image data to generate a first-resolution medical image matrix; the input unit 203 is configured to input the first-resolution medical image matrix into the encoder of a multi-scale vector quantization variational autoencoder to obtain structural prior features; the generation unit 204 is configured to generate a multi-scale discrete label sequence based on a scale-adaptive structural decoder and the structural prior features; the decoding unit 205 is configured to input the multi-scale discrete label sequence into the decoder of the multi-scale vector quantization variational autoencoder and perform parallel decoding operations through the parallel computing core of a graphics processor to generate a second-resolution medical image matrix, wherein the resolution of the second-resolution medical image matrix is ​​greater than the resolution of the first-resolution medical image matrix; and the display unit 206 is configured to send the second-resolution medical image matrix to a target display device and control the target display device to display it.

[0072] It is understandable that the units described in the device 200 are related to the reference. Figure 1 The steps in the described method correspond to each other. Therefore, the operations, features, and beneficial effects described above for the method also apply to the device 200 and the units contained therein, and will not be repeated here.

[0073] The following is for reference. Figure 3 It shows a schematic diagram of the structure of an electronic device 300 suitable for implementing some embodiments of the present disclosure. Figure 3 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0074] like Figure 3 As shown, the electronic device 300 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. The RAM 303 also stores various programs and data required for the operation of the electronic device 300. The processing unit 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0075] Typically, the following devices can be connected to I / O interface 305: input devices 306 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 307 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 308 including, for example, magnetic tapes, hard disks, etc.; and communication devices 309. Communication device 309 allows electronic device 300 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 3 An electronic device 300 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 3 Each box shown can represent a device or multiple devices as needed.

[0076] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 309, or installed from storage device 308, or installed from ROM 302. When the computer program is executed by processing device 301, it performs the functions defined in the methods of some embodiments of this disclosure.

[0077] It should be noted that, in some embodiments of this disclosure, the computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0078] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol), and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks ("LANs"), wide area networks ("WANs"), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0079] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: acquire first-resolution medical image data from a portable medical imaging device; perform standardized preprocessing on the first-resolution medical image data to generate a first-resolution medical image matrix; input the first-resolution medical image matrix into the encoder of a multi-scale vector quantization variational autoencoder to obtain structural prior features; generate a multi-scale discrete label sequence based on a scale-adaptive structural decoder and the aforementioned structural prior features; input the multi-scale discrete label sequence into the decoder of the multi-scale vector quantization variational autoencoder, and perform parallel decoding operations through the parallel computing core of a graphics processor to generate a second-resolution medical image matrix, wherein the resolution of the second-resolution medical image matrix is ​​greater than the resolution of the first-resolution medical image matrix; send the second-resolution medical image matrix to a target display device, and control the target display device to display it.

[0080] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0081] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0082] The units described in some embodiments of this disclosure can be implemented in software or hardware. The described units can also be housed in a processor; for example, a processor may be described as including an acquisition unit, a preprocessing unit, an input unit, a generation unit, a decoding unit, and a display unit. The names of these units do not necessarily limit the specific unit; for example, an acquisition unit may also be described as "a unit that acquires first-resolution medical image data from a portable medical imaging device."

[0083] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0084] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.< / eos> < / sos>

Claims

1. A medical image super-resolution reconstruction method based on multi-scale residual autoregressive modeling, applied to a stroke emergency system, the stroke emergency system comprising a portable medical imaging device, a graphics processor, and a target display device, the method comprising: Acquire first-resolution medical image data from a portable medical imaging device; The first-resolution medical image data is standardized and preprocessed to generate a first-resolution medical image matrix; The first-resolution medical image matrix is ​​input into the encoder of the multi-scale vector quantization variational autoencoder to obtain structural prior features; Based on the scale-adaptive structure decoder and the aforementioned structural prior features, a multi-scale discrete label sequence is generated; The multi-scale discrete label sequence is input to the decoder of the multi-scale vector quantization variational autoencoder, and parallel decoding is performed through the parallel computing core of the graphics processor to generate a second-resolution medical image matrix, wherein the resolution of the second-resolution medical image matrix is ​​greater than the resolution of the first-resolution medical image matrix. The second resolution medical image matrix is ​​sent to the target display device, and the target display device is controlled to display the image.

2. The method according to claim 1, wherein, The portable medical imaging device is a portable computed tomography (CT) scanner or a portable magnetic resonance imaging (MRI) device.

3. The method according to claim 1, wherein, The standardization preprocessing of the first-resolution medical image data to generate a first-resolution medical image matrix includes: Based on preset numerical format and preset color channel format, the first resolution medical image data is converted to obtain image data with unified format. Intensity normalization is performed on the uniformly formatted image data to obtain normalized image data; Spatially align the normalized image data to obtain a first-resolution medical image matrix.

4. The method according to claim 1, wherein, The multi-scale vector quantization variational autoencoder is obtained through the following steps: Obtain the second-resolution training dataset; The second resolution training dataset is input into the encoder of the initial multi-scale vector quantization variational autoencoder to obtain a set of multi-scale latent feature maps. The multi-scale latent feature map set is input into the decoder of the initial multi-scale vector quantization variational autoencoder to obtain the second resolution prediction dataset; Determine the reconstruction loss between the second resolution prediction dataset and the second resolution training dataset; In response to the reconstruction loss being less than a first preset loss threshold, the initial multi-scale vector quantization variational autoencoder is determined as a multi-scale vector quantization variational autoencoder.

5. The method according to claim 4, wherein, The scale-adaptive structure decoder is obtained through the following steps: The second resolution training dataset is downsampled to obtain the first resolution training dataset; The first resolution training dataset is input into the encoder of the multi-scale vector quantization variational autoencoder to obtain the conditional feature set; The conditional feature set and the multi-scale latent feature map set are input into the initial scale adaptive structure decoder, and the prediction loss of the initial scale adaptive structure decoder is determined. In response to the prediction loss being less than a second preset loss threshold, the initial scale-adaptive structure decoder is determined to be a scale-adaptive structure decoder.

6. The method according to claim 4, wherein, The step of inputting the second resolution training dataset into the encoder of the initial multi-scale vector quantization variational autoencoder to obtain a set of multi-scale latent feature maps includes: The encoder based on the initial multi-scale vector quantization variational autoencoder performs downsampling convolution operations on the second resolution training dataset to obtain initial latent features. Based on a preset multi-scale sequence, the initial latent features are divided into latent features of different scales to obtain a multi-scale latent feature sequence; The initial latent features are determined as the quantized reconstruction result; Based on the preset multi-scale sequence, the following recursive multi-scale residual quantization operation is performed on each multi-scale latent feature in the multi-scale latent feature sequence: Based on the quantization reconstruction results, the scale residual feature map of the multi-scale latent features is determined; The scale residual feature map is divided into a set of feature vectors; Perform a nearest neighbor search on each feature vector in the feature vector set to obtain a discrete codebook index set; Based on the discrete codebook index set, an embedding vector set is generated; The embedded vector set is assembled to obtain a quantized feature map; The quantized feature map is upsampled to obtain the scale quantization reconstruction result, and the sum of the scale quantization reconstruction result and the quantization reconstruction result is determined as the quantization reconstruction result; Each quantized feature map is defined as a set of multi-scale latent feature maps.

7. A medical image super-resolution reconstruction device based on multi-scale residual autoregressive modeling, comprising: The acquisition unit is configured to acquire first-resolution medical image data from a portable medical imaging device; The preprocessing unit is configured to perform normalization preprocessing on the first resolution medical image data to generate a first resolution medical image matrix; The input unit is configured to input the first-resolution medical image matrix into the encoder of a multi-scale vector quantization variational autoencoder to obtain structural prior features; The generation unit is configured to generate a multi-scale discrete label sequence based on the scale-adaptive structure decoder and the structure prior features; The decoding unit is configured to input the multi-scale discrete label sequence into the decoder of the multi-scale vector quantization variational autoencoder, and to perform parallel decoding operations through the parallel computing core of the graphics processor to generate a second-resolution medical image matrix, wherein the resolution of the second-resolution medical image matrix is ​​greater than the resolution of the first-resolution medical image matrix. The display unit is configured to send the second resolution medical image matrix to the target display device and control the target display device to display the image.

8. An electronic device, comprising: One or more processors; A storage device on which one or more programs are stored; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1 to 6.

9. A computer-readable medium having a computer program stored thereon, wherein, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 6.

Images