An image processing method, system, device and computer readable storage medium

Abstract

Embodiments of the present specification provide an image processing method, system and device of a medical image and a computer readable storage medium. The method comprises: acquiring a to-be-processed image, the to-be-processed image comprising a medical image obtained by a detector; determining an abnormal pixel point in the to-be-processed image; determining an image block from the to-be-processed image based on a preset window, the image block containing the abnormal pixel point; determining a pixel value of a region corresponding to the abnormal pixel point based on the image block by using a trained prediction model; and updating the to-be-processed image based on the pixel value.

Description

TECHNICAL FIELD

[0001] The present specification relates to the technical field of medical image processing, and in particular to a medical image image processing method and system. BACKGROUND

[0002] In medical imaging, such as Computed Tomography (CT) imaging, due to the manufacturing process of semiconductors, differences in electronic circuits, differences in materials, and damage during use, etc., the detector may have pixels that have zero or very weak response to X-ray signals, resulting in ring-shaped artifacts in the medical image and affecting the medical diagnosis result.

[0003] Therefore, it is desirable to provide a medical image image processing method that can complete abnormal pixel points in the image and improve the quality of the image. SUMMARY

[0004] One of the embodiments of the present specification provides an image processing method. The image processing method comprises: acquiring a to-be-processed image, the to-be-processed image comprising a medical image obtained by a detector; determining an abnormal pixel point in the to-be-processed image; determining an image block from the to-be-processed image based on a preset window, the image block containing the abnormal pixel point; determining a pixel value of a region corresponding to the abnormal pixel point based on the image block by using a trained prediction model; and updating the to-be-processed image based on the pixel value.

[0005] In some embodiments, the determining the image block from the to-be-processed image based on the preset window comprises: based on the preset window, cutting an image block containing the abnormal pixel point in a channel dimension of the to-be-processed image, the channel dimension being related to a channel of the detector.

[0006] In some embodiments, the channel value of the preset window is an odd number, and the region corresponding to the abnormal pixel point is in the middle channel of the image block.

[0007] In some embodiments, the prediction model comprises a self-attention layer, and the self-attention layer is used to determine the correlation between the region corresponding to the abnormal pixel point and other pixel point regions in the to-be-processed image.

[0008] In some embodiments, the prediction model comprises a bidirectional long short-term memory network model.

[0009] In some embodiments, the determining the pixel value of the region corresponding to the abnormal pixel point based on the image block by using the trained prediction model comprises: masking the pixel value of the region corresponding to the abnormal pixel point in the image block; and inputting the masked image block into the trained prediction model to predict the pixel value of the region corresponding to the abnormal pixel point.

[0010] In some embodiments, the trained prediction model is obtained by: obtaining a training set, the training set comprising a plurality of sample image blocks determined based on the preset window; performing mask processing on one of the pixel regions of the sample image blocks, and taking the original pixel value of the one of the pixel regions as a label after mask processing; training an initial model using the training set after mask processing to obtain the trained prediction model.

[0011] One of the embodiments of the present specification provides an image processing system, comprising: an acquisition module configured to acquire a to-be-processed image, the to-be-processed image comprising a medical image obtained by a detector; a determination module configured to determine an abnormal pixel point in the to-be-processed image; a division module configured to determine an image block from the to-be-processed image based on a preset window, the image block containing the abnormal pixel point; a prediction module configured to determine a pixel value of a region corresponding to the abnormal pixel point based on the image block by using a trained prediction model; and an update module configured to update the to-be-processed image based on the pixel value.

[0012] One of the embodiments of the present specification provides an image processing device, comprising at least one storage medium and at least one processor; the at least one storage medium is configured to store computer instructions, and the at least one processor is configured to execute the computer instructions to implement the image processing method as described above.

[0013] One of the embodiments of the present specification provides a computer-readable storage medium, which stores computer instructions, and when the computer instructions are executed by a processor, the image processing method as described above is implemented.

[0014] In some embodiments of the present specification, the pixel value of the region corresponding to the abnormal pixel point is predicted by calculating the correlation between the region corresponding to the abnormal pixel point and other pixel regions in the to-be-processed image, the to-be-processed image is updated based on the predicted pixel value, the pixel value of the region corresponding to the abnormal pixel point can be more accurately predicted, and the quality of the to-be-processed image is improved. BRIEF DESCRIPTION OF DRAWINGS

[0015] The present specification will be further illustrated in the form of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting, and in these embodiments, the same reference numbers represent the same structures, wherein:

[0016] Figure 1 is a schematic diagram of an application scenario of an image processing system according to some embodiments of the present specification;

[0017] Figure 2is an exemplary block diagram of an image processing system according to some embodiments of the present specification;

[0018] Figure 3 is an exemplary flow chart of an image processing method according to some embodiments of the present specification;

[0019] Figure 4 is an exemplary schematic diagram of a prediction model according to some embodiments of the present specification;

[0020] Figure 5 is an exemplary schematic diagram of obtaining a trained prediction model according to some embodiments of the present specification;

[0021] Figure 6 is an exemplary schematic diagram of a medical data image according to some embodiments of the present specification;

[0022] Figure 7 is an exemplary schematic diagram of an image channel dimension according to some embodiments of the present specification. DETAILED DESCRIPTION

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present specification, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some examples or embodiments of the present specification, and for those skilled in the art, without paying creative labor, the present specification can also be applied to other similar scenarios according to these drawings. Unless it is clear from the language context or otherwise stated, the same reference numbers in the drawings represent the same structure or operation.

[0024] It should be understood that the "system", "device", "unit" and / or "module" used herein is a method for distinguishing different components, elements, parts, sections or assemblies at different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions.

[0025] As shown in the specification and claims, unless the context clearly indicates otherwise, "one", "a", "an", and / or "the" do not refer to the singular, but can also include the plural. Generally speaking, the terms "comprise" and "include" only indicate the inclusion of the steps and elements explicitly identified, and these steps and elements do not constitute an exclusive list, and the method or device can also include other steps or elements.

[0026] Flow diagrams are used in the description of the specification to describe the operation of systems in accordance with embodiments of the specification. It will be understood that the acts need not necessarily be performed in the precise order shown. Rather, various steps can be handled in different order or simultaneously. Moreover, other acts can be provided in place of, or in addition to, those shown.

[0027] Figure 1 is a schematic diagram of an application scenario of an image processing system according to some embodiments of the specification.

[0028] As shown in Figure 1 , the image processing system 100 can include a processing device 110, an imaging device 120, and a storage device 150. In some embodiments, the processing device 110 can be part of the imaging device 120.

[0029] In some embodiments, the image processing system 100 can implement the processing of images by implementing the methods and / or processes disclosed in the specification. For example, in some application scenarios, the processing device 110 can acquire a to-be-processed image 130, determine an abnormal pixel point in the to-be-processed image 130, and determine an image block from the to-be-processed image 130 based on a preset window. Further, the processing device 110 can determine a pixel value of a region corresponding to the abnormal pixel point based on the image block by using a trained prediction model, update the to-be-processed image 130 based on the predicted pixel value, and obtain an updated image 140 in which the abnormal pixel point is eliminated.

[0030] In some embodiments, the processing device 110 can process data and / or information obtained from other devices or system components, execute program instructions based on the data, information, and / or processing results to complete one or more image processing processes described in the specification. For example, the processing device 110 can execute corresponding operation instructions based on image processing operations input by the imaging device 120. For another example, the processing device 110 can acquire the to-be-processed image 130 stored in the storage device 150 and obtain the updated image 140 by processing.

[0031] In some embodiments, the processing device 110 can include one or more sub-processing devices (e.g., single-core processing devices or multi-core multi-core processing devices). By way of example only, the processing device 110 can include a central processing unit (CPU), an application-specific integrated circuit (ASIC), an application-specific instruction-set processor (ASIP), a graphics processing unit (GPU), a physics processing unit (PPU), a digital signal processor (DSP), a field-programmable gate array (FPGA), a programmable logic device (PLD), a controller, a microcontroller unit, a reduced instruction set computer (RISC), a microprocessor, or the like, or any combination thereof.

[0032] In some embodiments, the processing device 110 may acquire an image 130 to be processed from the imaging device 120. For example, the processing device 110 may acquire medical images such as computed tomography (CT) images or magnetic resonance imaging (MRI) images generated by the imaging device 120 scanning a target object as the image to be processed. In some embodiments, the processing device 110 may send the updated image 140 to a terminal device or the imaging device 120 for output to a user, and / or send it to a storage device 150 for storage.

[0033] Imaging device 120 can be used to scan a target object or a portion of a target object located within its detection area and generate a medical image. In some embodiments, imaging device 120 may include a computed tomography (CT) device, a magnetic resonance imaging (MRI) device, an ultrasound imaging device, a digital imaging device, an X-ray computed tomography device, a magnetic resonance imaging device, a nuclear medicine imaging device, or any combination thereof.

[0034] Storage device 150 can store data, instructions, and / or any other information. In some embodiments, storage device 150 can store data obtained from imaging device 120 and / or processing device 110. For example, image 130 to be processed and updated image 140 can be stored in storage device 150.

[0035] In some embodiments, storage device 150 may include one or more storage components, each of which may be a separate device or part of another device. In some embodiments, storage device 150 may include random access memory (RAM), read-only memory (ROM), mass storage, removable memory, volatile read-write memory, and any combination thereof. Exemplarily, mass storage may include disks, optical disks, solid-state drives, etc. In some embodiments, storage device 150 may be implemented on a cloud platform.

[0036] Figure 2 This is an exemplary block diagram of an image processing system according to some embodiments of this specification.

[0037] like Figure 2 As shown, in some embodiments, the image processing system 200 may include an acquisition module 210, a determination module 220, a segmentation module 230, a prediction module 240, an update module 250, and a training module 260.

[0038] In some embodiments, the obtaining module 210 can be configured to obtain an image to be processed, such as the image to be processed 130. In some embodiments, the image to be processed can include a medical image obtained by a detector, such as a computed tomography image, a magnetic resonance imaging image, or the like.

[0039] In some embodiments, the determining module 220 can be configured to determine an abnormal pixel point in the image to be processed.

[0040] In some embodiments, the dividing module 230 can be configured to determine an image block from the image to be processed based on a preset window. In some embodiments, the image block can contain the abnormal pixel point. In some embodiments, the dividing module 230 can be configured to, based on the preset window, cut the image block containing the abnormal pixel point in a channel dimension of the image to be processed, the channel dimension being related to a channel of the detector. In some embodiments, the channel value of the preset window can be an odd number, and the abnormal pixel point corresponding region is in the middle channel of the image block.

[0041] In some embodiments, the predicting module 240 can be configured to determine a pixel value of the abnormal pixel point corresponding region based on the image block by using a trained prediction model. In some embodiments, the predicting module 240 can be configured to mask the pixel value of the abnormal pixel point corresponding region in the image block; input the masked image block into the trained prediction model to predict the pixel value of the abnormal pixel point corresponding region. In some embodiments, the prediction model can include a self-attention layer. In some embodiments, the self-attention layer can be configured to determine the correlation between the abnormal pixel point corresponding region and other pixel point regions in the image to be processed. In some embodiments, the prediction model can include a bidirectional long short-term memory network model.

[0042] In some embodiments, the updating module 250 can be configured to update the image to be processed based on the pixel value.

[0043] In some embodiments, the training module 260 can be configured to obtain a training set, the training set including a plurality of sample image blocks determined based on a preset window. In some embodiments, the training module 260 can also be configured to mask one of the pixel regions of the sample image block, and use the original pixel value of the one of the pixel regions as a label after the masking; use the training set after the masking to train an initial model to obtain a trained prediction model.

[0044] It should be understood that, Figure 2 The system and its modules shown can be implemented in various ways. For example, in some embodiments, the system and its modules can be implemented by hardware, software, or a combination of software and hardware.

[0045] It should be noted that the above description of the system and its modules is for the convenience of description only, and cannot limit the scope of the embodiments. It can be understood that, after understanding the principles of the system, those skilled in the art can combine the modules or connect the modules to form a subsystem without departing from the principles. For example, in some embodiments, Figure 2 The acquisition module 210, the determination module 220, the division module 230, the prediction module 240, the update module 250, and the training module 260 disclosed in the embodiments can be different modules in a system, or can be a module that implements the functions of two or more modules. For example, the modules can share a storage module, or each module can have its own storage module. Variations such as these are within the scope of the present disclosure.

[0046] Figure 3 is an exemplary flowchart of an image processing method according to some embodiments of the present disclosure. As shown in some embodiments, the flow 300 can include step 310, step 320, step 330, step 340, and step 350. Figure 3

[0047] Step 310, acquiring a to-be-processed image. In some embodiments, step 310 can be performed by the processing device 110 or the image processing system 200 (e.g., the acquisition module 210).

[0048] The to-be-processed image can refer to an image that has defects and needs to be processed, for example, an image with bright spots, a CT image with artifacts, an image with white spots, etc.

[0049] In some embodiments, the to-be-processed image can include a medical image obtained by a detector, such as a computed tomography image, a magnetic resonance imaging image, etc. In some embodiments, the to-be-processed image can include other types of medical images, such as positron emission tomography images, etc.

[0050] In some embodiments, the acquisition module 210 can acquire the to-be-processed image through an imaging device, for example, by detecting a generated scan image through the imaging device 120, a stored scan image through the imaging device 120, etc. In some embodiments, the acquisition module 210 can acquire the to-be-processed image (e.g., the to-be-processed image 130) from a storage device (e.g., the storage device 150).

[0051] Step 320, determining an abnormal pixel point in the to-be-processed image. In some embodiments, step 320 can be performed by the processing device 110 or the image processing system 200 (e.g., the determination module 220).

[0052] ​An abnormal pixel point can refer to a point in the to-be-processed image that is displayed differently from a normal pixel, for example, a pixel corresponding to a strip-shaped straight line, a bright spot, a ring-shaped artifact, a white spot, a black spot, and the like in the image. For example, in a CT imaging system, a pixel in which the detector has zero or very weak response to an X-ray signal can be referred to as an abnormal pixel, and one of the abnormal pixels can be referred to as an abnormal pixel point. For example, as shown in FIG. 13, there is a strip-shaped straight line with low response in the middle of the to-be-processed image 130, and the strip-shaped straight line is composed of a plurality of abnormal pixel points. The area corresponding to the strip-shaped straight line is an area corresponding to the abnormal pixel points. Figure 1

[0053] In some embodiments, an abnormal pixel point in the to-be-processed image can be determined based on a response value of a pixel point or an area in the to-be-processed image. In some embodiments, the response value can refer to a numerical value of the response of the detector to the X-ray signal. For example, in a CT imaging system, the response of the detector to the X-ray signal has differences in strength, and when the detector has zero or very weak response to the X-ray signal, a black or very low response strip-shaped straight line will be presented on a sinogram. In some embodiments, the determination module 220 can determine a pixel point with a response value less than a first preset threshold as an abnormal pixel point in the to-be-processed image. In some embodiments, an abnormal pixel point in the to-be-processed image can be determined based on a pixel value in the to-be-processed image. For example, when a pixel value of a point in the to-be-processed image is less than a second preset threshold, the determination module 220 can determine the corresponding point as an abnormal pixel point in the to-be-processed image. In some embodiments, the determination module 220 can determine a pixel point in which an abnormality such as a bright spot, a white spot, a black spot, and the like is displayed in the to-be-processed image as an abnormal pixel point in the to-be-processed image. In some embodiments, the determination of the abnormal pixel point in the to-be-processed image also includes other any feasible determination manner.

[0054] Step 330: determining an image block from the to-be-processed image based on a preset window, the image block containing the abnormal pixel point. In some embodiments, step 330 can be performed by the processing device 110 or the image processing system 200 (such as the division module 230).

[0055] The preset window can refer to a line frame with a fixed size that is set in advance. The size of the preset window can include width, length, diameter, and the like. For example, the preset window can be a rectangle with a length of 9 and a width of 500.

[0056] ​In some embodiments, the size of the preset window can be determined based on the horizontal and vertical coordinates of the to-be-processed image. For example, if the maximum horizontal coordinate of the to-be-processed image is 500 and the maximum vertical coordinate is 1000, the size of the preset window can be determined as 9*500. In some embodiments, the size of the preset window can be determined based on the shape of the to-be-processed image. For example, when the to-be-processed image is a rectangle of 1000*500, the preset window can be set as a rectangular frame of 7*200, or 7*300, or 7*500. In some embodiments, the preset window can be determined based on the distribution of the abnormal pixel points in the to-be-processed image. For example, when the abnormal pixel points occupy 1 unit length in the horizontal coordinate direction of the to-be-processed image, the length of the side corresponding to the horizontal coordinate of the preset window can be set to a value greater than 1.

[0057] In some embodiments, the value of the preset window can be an odd number, such as 5, 7, 9, 11, 13, etc., which is the length (or width) corresponding to the direction of the image block in the to-be-processed image. For example, when the image block is cut from the horizontal coordinate direction of the to-be-processed image, the value of the preset window in the horizontal coordinate direction can be an odd number. For another example, when the image block containing abnormal pixel points is cut from the channel dimension of the to-be-processed image, the channel value of the preset window can be an odd number.

[0058] The channel dimension can refer to the dimension corresponding to the detector channel of the imaging device in the medical data image, for example, Figure 6 the vertical coordinate of the data image shown in (a). For example only, as shown in (a), Figure 7 For example, as shown in (a), taking an image matrix as an example, if the detector size used by the imaging device is 1024*256 pixels, the channel dimension of the obtained data image can be represented as 1024 channels, and the slice dimension can be represented as 256 slices. If the data collected during the imaging process is 360 views, the corresponding data size can be represented as 1024*256*360.

[0059] In some embodiments, the size, value, etc. of the preset window can be determined according to different needs, for example, it can be set according to the input of the user, the default value of the system, historical records, etc. It should be noted that the above examples are only examples and are not limiting.

[0060] The image block can refer to a part of the image cut from the to-be-processed image. In some embodiments, the image block can contain abnormal pixel points.

[0061] In some embodiments, the region corresponding to the abnormal pixel points can be in one of the channels of the image block. For example, when the channel value of the preset window is 7, the region corresponding to the abnormal pixel points can be in any one of the 1st-7th channels of the image block.

[0062] In some embodiments, the abnormal pixel point corresponding region can be in the middle channel of the image block. For example, when the channel value of the preset window is 9, the abnormal pixel point corresponding region can be in the 5th channel of the image block.

[0063] In some embodiments, the division module 230 can determine the image block from the to-be-processed image based on the preset window. For example, if the maximum value of the horizontal coordinate of the to-be-processed image is 500 and the maximum value of the vertical coordinate is 1000, and the size of the preset window is 9*500, the division module 230 can extract a 9*500 image block containing abnormal pixel points from the to-be-processed image along the horizontal coordinate direction.

[0064] In some embodiments, the image block containing abnormal pixel points can be extracted from the channel dimension of the to-be-processed image (such as a computed tomography image, a magnetic resonance imaging image, etc.) based on the preset window.

[0065] In some embodiments, the division module 230 can divide the to-be-processed image (such as a computed tomography image) into multiple segments based on the preset window, and select the segment containing abnormal pixel points as the image block.

[0066] In some embodiments, the division module 230 can determine the channel of the image block in which the abnormal pixel point is located based on the preset window from the to-be-processed image. For example, based on the image block obtained by the preset window with a channel value of 11, the number of channels of the image block is 11, and if the abnormal pixel point is in the middle channel of the image block, the division module 230 can determine that the abnormal pixel point is in the 5th channel of the image block.

[0067] In some embodiments, the division module 230 can extract the image block containing abnormal pixel points from the to-be-processed image based on the preset window according to a pre-set manner. In some embodiments, the extraction manner of the image block also includes other any feasible extraction manner. For example, the division module 230 can extract the image block containing abnormal pixel points along a direction at a certain angle with the horizontal coordinate of the to-be-processed image based on the preset window. It should be noted that the above examples are only examples and not limitations, and the determination manner of the image block also includes other any feasible determination manner.

[0068] Step 340, based on the image block, determining the pixel value of the abnormal pixel point corresponding region by the trained prediction model. In some embodiments, step 340 can be performed by the processing device 110 or the image processing system 200 (such as the prediction module 240).

[0069] A pixel value of an image can refer to a value representing image information. A pixel value of a corresponding region can refer to a value representing image information of the corresponding region. For example, the pixel value of the corresponding region can include a resolution of the corresponding region, a brightness value of the corresponding region, a density value of the corresponding region, a color value of the corresponding region, or the like, or any combination thereof.

[0070] In some embodiments, the prediction module 240 can mask the pixel value of the corresponding region of the abnormal pixel point in the image block, and input the masked image block into the trained prediction model to predict the pixel value of the corresponding region of the abnormal pixel point. In some embodiments, the input of the trained prediction model can include an image block in which the pixel value of the corresponding region of the abnormal pixel point is masked, and the output can include the predicted pixel value of the corresponding region of the abnormal pixel point.

[0071] Masking can refer to performing a bitwise AND operation on a target field by a string of binary codes to mask the current input bit. In some embodiments, masking the pixel value of the corresponding region of the abnormal pixel point in the image block can refer to masking the pixel value of the corresponding region of the abnormal pixel point in the image block, for example, erasing or replacing the pixel value of one of the pixel regions in the image block with a specific value (such as 0).

[0072] In some embodiments, the prediction module 240 can mask the pixel value of the corresponding region of the abnormal pixel point in the image block in various ways. For example, the prediction module 240 can mask the corresponding region of the abnormal pixel point with the same numerical value, vector, symbol, or text, etc. The masking method can be set according to actual needs, which is not limited in this embodiment.

[0073] The masked image block can refer to an image block in which the corresponding region of the abnormal pixel point is masked, for example, an image block in which the pixel value of the corresponding region of the abnormal pixel point is all “0” or “1”, etc.

[0074] In some embodiments, the prediction model can include a bidirectional long short-term memory network model. In some embodiments, the bidirectional long short-term memory network model (Bi-LSTM) can be single-layer, multi-layer, etc.

[0075] In some embodiments of the present specification, the bidirectional long short-term memory network model is used to determine the pixel value of the corresponding region of the abnormal pixel point. The bidirectional long short-term memory network model can reflect the internal relationship between the corresponding region of the abnormal pixel point and the adjacent pixel point region in the image to be processed, as well as the internal relationship between other different pixel point regions, and can more accurately predict the pixel value of the corresponding region of the abnormal pixel point.

[0076] In some embodiments, the prediction model can include a self-attention layer. For example, Figure 4As shown in the self-attention layer, the input can include a vector representation c1, c2, …, c of N channels of pixels of the image block of the pixel value of the region corresponding to the abnormal pixel point of the mask N , where N corresponds to the channel value of the preset window, and the output can be a vector representation of the correlation of each channel of pixels with other channels of pixels

[0077] In some embodiments, the number of layers of the self-attention layer can include a single layer, multiple layers, etc., for example, 12 layers or 24 layers, etc. Figure 4 The self-attention layer is a single layer in the above embodiment, which is only an example. The number of layers and structure of the self-attention layer can be set according to actual needs or experience, which is not limited in the present embodiment.

[0078] In some embodiments, the self-attention layer can be used to determine the correlation of the region corresponding to the abnormal pixel point with other pixel point regions in the image to be processed.

[0079] As an example, if the channel value of the preset window is N, the image block containing the abnormal pixel point is cut in the channel dimension of the image to be processed, three data conversion matrices Q, K, and V can be defined, and the matrix size is V*V. For example, Figure 4 As shown in the trained prediction model, the input image block processed by the mask is converted into a corresponding vector representation, and the vector representation c1, c2, …, c of the pixels of different channels of the image block is obtained m , …, c N Input into the self-attention layer, each input vector c1, c2, …, c m , …, c N is multiplied by three data conversion matrices respectively to obtain corresponding q i , k i , v i vectors, for example: Further, the weight value of each channel corresponding vector is calculated to determine the similarity between the predicted vector c m and other vectors c1, c2, …, c N , specifically: wherein, represents the transpose of the vector c i , represents the transpose of the vector q m , i = [1, N]. Then, the softmax layer is normalized to obtain: After weighted summation, the following is obtained: Finally, a linear rectified function (Rectified Linear Unit, ReLU) of a fully connected layer (FC) is used to obtain a vector representation of the predicted pixel value of the channel of the region corresponding to the abnormal pixel wherein W is a coefficient of the linear rectified function, and c is a parameter of the linear rectified function.

[0080] In some embodiments of the present specification, the self-attention layer can reflect the correlation between the region corresponding to the abnormal pixel and other pixel regions in the image to be processed. The prediction model containing the self-attention layer can more accurately predict the pixel value of the region corresponding to the abnormal pixel.

[0081] For the training process of the prediction model, please refer to the related description of Figure 5 , which will not be repeated here.

[0082] At step 350, the image to be processed is updated based on the pixel value. In some embodiments, step 350 can be performed by processing device 110 or image processing system 200 (e.g., update module 250).

[0083] In some embodiments, update module 250 can replace the abnormal pixel in the image to be processed based on the pixel value of the region corresponding to the abnormal pixel determined by the prediction model, update the image to be processed, and obtain an updated image, such as updated image 140.

[0084] In some embodiments of the present specification, the pixel value of the region corresponding to the abnormal pixel is determined by locally weighted data, and the image to be processed is updated, which can more accurately predict the pixel value of the region corresponding to the abnormal pixel.

[0085] Figure 5 is an exemplary schematic diagram of obtaining a trained prediction model according to some embodiments of the present specification.

[0086] At step 510, a training set is obtained, and the training set includes a plurality of sample image blocks determined based on a preset window. In some embodiments, step 510 can be performed by processing device 110 or image processing system 200 (e.g., training module 260).

[0087] The training set can include a plurality of sample image blocks. The sample image block can refer to an image segment obtained by dividing a sample image.

[0088] In some embodiments, the training module 260 can determine the sample image blocks based on a preset window. For example, the preset window has a pixel size of 9*500, and the training module 260 can divide the sample image into multiple sample image blocks with a pixel size of 9*500. In some embodiments, there can be a partial overlap between the sample image blocks. For example, the training module 260 can divide the sample image from top to bottom to determine multiple sample image blocks, and there can be an overlap of 10%, 20%, etc. between the sample image blocks.

[0089] For example, as shown in FIG. 13, a medical data image with a horizontal coordinate of view and a vertical coordinate of channel can be divided into multiple sample image blocks based on a preset window. Figure 6 For example, as shown in FIG. 13, a medical data image with a horizontal coordinate of view and a vertical coordinate of channel can be divided into multiple sample image blocks based on a preset window. Figure 6 For example, as shown in FIG. 13, a medical data image with a horizontal coordinate of view and a vertical coordinate of channel can be divided into multiple sample image blocks based on a preset window. Figure 6 For example, as shown in FIG. 13, a medical data image with a horizontal coordinate of view and a vertical coordinate of channel can be divided into multiple sample image blocks based on a preset window. Figure 6 It can be understood that the sample image and the sample image block are only corresponding local illustrations, and in some embodiments, the length of the sample image and / or the sample image block can be greater than that shown in the figure.

[0090] The manner of obtaining the sample image block can be preset. For example, the direction (horizontal, vertical, rotated by a certain angle, etc.), size, number, etc. of the sample image block can be preset.

[0091] In some embodiments, multiple sample image blocks can be obtained based on multiple sample images. The multiple sample images can include images without abnormal pixel points, images without abnormal pixel points after preprocessing, etc. The image without abnormal pixel points after preprocessing can refer to an image without abnormal pixel points obtained by preprocessing an image with abnormal pixel points. For example, an image containing abnormal pixel points can be obtained by configuring different sampling parameters (tube voltage, tube current, and sampling frequency, etc.) of an X-ray tube and a detector, and the image with abnormal pixel points can be further preprocessed by an interpolation method to obtain an image without abnormal pixel points after preprocessing as a sample image. For another example, an image without abnormal pixel points, i.e., a normal image, can be directly obtained by configuring different sampling parameters (tube voltage, tube current, and sampling frequency, etc.) of an X-ray tube and a detector.

[0092] In some embodiments, the training module 260 can obtain the plurality of sample image blocks in various ways. In some embodiments, the training module 260 can obtain the plurality of sample image blocks by dividing the sample image in the horizontal direction based on a preset window. In some embodiments, the training module 260 can obtain the plurality of sample image blocks by dividing the sample image in the channel dimension based on a preset window. In some embodiments, sample image blocks of different lengths can be obtained according to actual needs. In some embodiments, the sample image can be rotated by a certain angle (for example, 90°) before being divided in the horizontal direction to obtain the plurality of sample image blocks.

[0093] In some embodiments, a plurality of sample image blocks can be obtained from a sample image. The training set of sample image blocks can be obtained by obtaining a plurality of sample image blocks from a plurality of sample images. In some embodiments, each sample image block can include a plurality of channels, for example Figure 5 As shown in FIG. 5, the sample image block includes N channels, c’1, c’2, c’ m , c’ N-1 , c’ N represent the vector representation of each channel, respectively.

[0094] At step 520, one of the pixel regions of the sample image block is masked, and the original pixel value of the one of the pixel regions is taken as the label after the masking. In some embodiments, step 520 can be performed by the processing device 110 or the image processing system 200 (for example, the training module 260).

[0095] The one of the pixel regions can refer to a partial region of the sample image block. For example, when the sample image block is obtained by dividing the sample image in the channel dimension based on a preset window, the sample image block includes a plurality of channels, and the one of the pixel regions can be one of the channels (for example, c’1) of the sample image block. In some embodiments, the one of the pixel regions can be a partial region of the sample image block, for example, a channel region of the sample image block. m The vector representation of the channel region image is c’1.

[0096] The original pixel value can refer to the initial pixel value of the pixel region, for example, the pixel value before masking. In some embodiments, the label of the pixel region after the masking is the original pixel value of the corresponding pixel region. As shown in FIG. 5, the vector representation c’1 Figure 5 corresponds to the pixel region that is masked, and the original pixel value c’1 m corresponds to the original pixel value of the pixel region that is masked. m The original pixel value c’1 m is the label of the mask.

[0097] For the masking of the one of the pixel regions of the sample image block, reference can be made to the related description of the masking in FIG. 5, which will not be repeated here. Figure 3

[0098] ​In some embodiments, the one pixel region of each sample image block can be masked respectively.

[0099] At step 530, the initial model is trained by using the masked training set to obtain a trained prediction model. In some embodiments, step 530 can be performed by the processing device 110 or the image processing system 200 (e.g., the training module 260).

[0100] In some embodiments, the masked training set can refer to a set of sample image blocks in which one pixel region of each sample image block is masked.

[0101] The initial model can refer to an untrained model, for example, an untrained bidirectional long short-term memory network model, etc.

[0102] In some embodiments, the initial model can be trained in an unsupervised manner, for example, the initial model can be input based on the masked training set and the label to obtain the prediction model.

[0103] In some embodiments, the parameters of the initial model can be optimized by a loss function. For example, the loss function can include a logarithmic loss function, an exponential loss function, a root mean square error loss function, etc.

[0104] In some embodiments, the closeness between the vector representation of the original pixel value of the masked pixel region and the vector representation of the predicted pixel value can be evaluated by a root mean square error (RMSE) loss function, for example, the calculation formula (1) can be:

[0105]

[0106] wherein, is the vector representation of the predicted pixel value of the masked pixel region, c' m is the vector representation of the original pixel value of the masked pixel region corresponding to c'; and V represents the vector dimension corresponding to the sample image block. is the vector representation of the predicted pixel value of the masked pixel region of different dimensions; is the vector representation of the original pixel value of the masked pixel region of different dimensions.

[0107] In some embodiments, when the error between the original pixel value and the predicted pixel value of the masked pixel region is less than a preset error threshold (for example, the preset error threshold is 0.05), the training can be stopped to obtain the trained prediction model.

[0108] In some embodiments of the present specification, the pixel value of the region corresponding to the abnormal pixel point is determined through the self-attention layer, the correlation between the region corresponding to the abnormal pixel point and other pixel point regions in the image to be processed is obtained, and the pixel value of the region corresponding to the abnormal pixel point can be more accurately predicted. In some embodiments of the present specification, by setting the channel value of the preset window to an odd number, the region corresponding to the abnormal pixel point is in the middle channel of the image block, which can make the two sides of the abnormal pixel point be symmetric data, and further improve the accuracy of predicting the pixel value of the region corresponding to the abnormal pixel point. In some embodiments of the present specification, the initial model is trained unsupervisedly to obtain the trained prediction model, and the labeling time and period of the labels of the training data are reduced, which can save manpower and time.

[0109] It should be noted that the above description of the processes 300 and 500 is merely for example and illustration, and does not limit the scope of the present specification. Those skilled in the art can make various modifications and changes to the processes 300 and 500 under the guidance of the present specification. However, these modifications and changes are still within the scope of the present specification. For example, in the process 300, while the image to be processed is obtained, the abnormal pixel point in the image to be processed can be determined. For another example, in the process 500, while the training set is obtained, one of the pixel regions of the sample image block is masked.

[0110] The above has described the basic concept, and it is obvious that the above detailed disclosure is only for example and does not limit the present specification. Although it is not explicitly stated here, those skilled in the art can make various modifications, improvements and corrections to the present specification. Such modifications, improvements and corrections are suggested in the present specification, so such modifications, improvements and corrections are still within the spirit and scope of the exemplary embodiments of the present specification.

[0111] At the same time, the present specification uses specific words to describe the embodiments of the present specification. As "one embodiment", "an embodiment", and / or "some embodiments" means a certain feature, structure or characteristic related to at least one embodiment of the present specification. Therefore, it should be emphasized and noted that the "an embodiment" or "one embodiment" or "one alternative embodiment" mentioned in different positions in the present specification does not necessarily refer to the same embodiment. In addition, some features, structures or characteristics in one or more embodiments of the present specification can be properly combined.

[0112] Furthermore, the order of the processing elements and sequences described in this specification are not intended to be construed as a limitation, unless specifically stated, but are included to provide a complete description of one or more embodiments of the present specification. Regardless of the particular sequence of processing elements and sequences, however, the description herein of a process should be understood to include any and all combinations of one or more elements, and sequences that can be perceived as either open-ended or specific.

[0113] Similarly, it is to be noticed that the term "comprising", used in the description, should not be interpreted as being restricted only to the means listed thereafter. It is to be understood that the term "comprising" means that any additional element, which is not specifically mentioned, is optionally present or can be added. In some embodiments, the description of an embodiment using the term "comprising" can also be interpreted as using the term "including" or "consisting of". Furthermore, the description herein of any particular embodiment of the present specification is intended to be illustrative only and is not intended to be limiting unless specifically stated. Thus, while the present specification has been described in terms of some embodiments, it is anticipated that alternatives, modifications, and equivalents will further occur to those skilled in the art. Accordingly, it is intended that the claims be interpreted as broadly as is reasonabl

[0114] Some embodiments use numerals to describe components, quantities of attributes. It should be understood that such numerals used in the description of the embodiments are, in some examples, modified by the adjectives "about", "approximately", or "generally". Unless otherwise stated, "about", "approximately", or "generally" indicates that the stated numerical value allows for a variation of ±20%. Accordingly, numerical parameters in the description and claims are approximations, and can vary depending upon the requirements of the particular embodiments. In some embodiments, numerical parameters are determined by the use of common rounding techniques. Although the numerical ranges and parameters setting forth the broadest scope of the embodiments of the specification are approximations, the numerical values set forth in the specific examples are reported as precisely as practicable. The numerical values set forth in the specific examples are provided to be as precise as reasonably possible. However, some variations may occur depending on the choice of input used to develop or derive the numerical values in the examples.

[0115] Each patent, patent application, publication, and other material cited in this specification is hereby incorporated by reference in its entirety for the purpose of describing and disclosing, for example, the compositions and methodologies described in such publications that might be used with the present specification. In the event of inconsistencies between the disclosure in such publications and the disclosure herein, the disclosure in such publications is hereby superseded by the disclosure herein. It is specifically noted that the description, definitions, and / or terminology used in the incorporated material is not intended to limit the description, definitions, and / or terminology used in the specification.

[0116] Finally, it should be understood that the embodiments described herein are only given by way of example and that other modifications can occur to persons skilled in the art. Therefore, the scope of the present description is not intended to be limited to the embodiments described herein but is only limited by the claims that follow.

Claims

1. An image processing method, characterized in that, The method includes: Acquire an image to be processed, the image to be processed including a medical image obtained by the detector; Abnormal pixels in the image to be processed are identified. Abnormal pixels are points in the image to be processed that are different from normal pixels. The abnormal pixels are identified based on the response values ​​of pixels or regions in the image to be processed, or based on the pixel values ​​in the image to be processed. An image block is determined from the image to be processed based on a preset window, wherein the channel value of the preset window is odd, the region corresponding to the abnormal pixel is in the middle channel of the image block, the image block is a portion of the image to be processed, and the image block contains the abnormal pixel. Based on the image patch, the pixel value of the region corresponding to the abnormal pixel is determined by the trained prediction model. The pixel value of the corresponding region is a value representing the image information of the corresponding region. The pixel value of the corresponding region includes the resolution of the corresponding region, the brightness value of the corresponding region, the density value of the corresponding region, the color value of the corresponding region, or any combination thereof. Updating the image to be processed based on the pixel values ​​includes: Based on the pixel values ​​of the corresponding regions of the abnormal pixels determined by the prediction model, the abnormal pixels in the image to be processed are replaced, and the image to be processed is updated to obtain the updated image.

2. The method as described in claim 1, characterized in that, The step of determining image blocks from the image to be processed based on a preset window includes: Based on the preset window, an image block containing the abnormal pixels is extracted along the channel dimension of the image to be processed.

3. The method as described in claim 1, characterized in that, The prediction model includes a self-attention layer, which is used to determine the correlation between the region corresponding to the abnormal pixel and other pixel regions in the image to be processed.

4. The method as described in claim 1, characterized in that, The prediction model includes a bidirectional long short-term memory network model.

5. The method as described in claim 1, characterized in that, The step of determining the pixel value of the region corresponding to the abnormal pixel point based on the image patch using a trained prediction model includes: Mask the pixel values ​​of the region corresponding to the abnormal pixel in the image block; The masked image block is input into the trained prediction model to predict the pixel value of the region corresponding to the abnormal pixel.

6. The method as described in claim 1, characterized in that, The trained prediction model is obtained through the following method: Obtain a training set, which includes multiple sample image patches determined based on the preset window; A pixel region of the sample image block is masked, and the original pixel value of the pixel region is used as the label after masking. The initial model is trained using the masked training set to obtain the trained prediction model.

7. An image processing system, characterized in that, The system includes: The acquisition module is used to acquire images to be processed, including medical images obtained by the detector; The determination module is used to determine abnormal pixels in the image to be processed. The abnormal pixels are points in the image to be processed that are different from normal pixels. The abnormal pixels are determined based on the response values ​​of pixels or regions in the image to be processed, or based on the pixel values ​​in the image to be processed. A segmentation module is used to determine an image block from the image to be processed based on a preset window. The channel value of the preset window is odd. The region corresponding to the abnormal pixel is in the middle channel of the image block. The image block is a portion of the image to be processed and contains the abnormal pixel. The prediction module is used to determine the pixel value of the region corresponding to the abnormal pixel point based on the image patch using a trained prediction model. The pixel value of the corresponding region is a value representing the image information of the corresponding region, including the resolution of the corresponding region, the brightness value of the corresponding region, the density value of the corresponding region, the color value of the corresponding region, or any combination thereof. An update module is used to update the image to be processed based on the pixel values, the update including: Based on the pixel values ​​of the corresponding regions of the abnormal pixels determined by the prediction model, the abnormal pixels in the image to be processed are replaced, and the image to be processed is updated to obtain the updated image.

8. An image processing apparatus, the apparatus comprising at least one storage medium and at least one processor; The at least one storage medium is used to store computer instructions; The at least one processor is used to execute the computer instructions to implement the image processing method according to any one of claims 1 to 6.

9. A computer-readable storage medium storing computer instructions that, when executed by a processor, implement the image processing method as described in any one of claims 1 to 6.

Images