Image Processing Method, Apparatus, Computer Device, and Storage Medium

Abstract

An embodiment of the present application discloses an image processing method, apparatus, computer device, and storage medium, belonging to the field of computer technology. The method includes: obtaining an image processing model, where the image processing model includes structural parameters, the structural parameters represent the structure of metal artifacts, and the structural parameters include first region structural parameters corresponding to a first region and second region structural parameters corresponding to a second region; when training the image processing model, adjusting the first region structural parameters based on training samples, and adjusting the first region structural parameters based on the angle corresponding to the first region and the angle corresponding to the second region to obtain the second region structural parameters corresponding to the second region, and the trained image processing model is used to remove metal artifacts in any medical image based on the adjusted structural parameters. This method fully considers the structural characteristic that metal artifacts are multiple rotationally symmetric regions, and can improve the effect of the image processing model in removing metal artifacts.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular, to an image processing method, apparatus, computer device, and storage medium. Background Art

[0002] Computed Tomography (CT) can non-destructively detect the tissue and organ structures in the human body, so it has a wide range of applications in the medical field. When collecting CT images using computed tomography, affected by metal implants in the human body, metal artifacts will appear in the collected CT images, affecting the quality of CT images. In the related art, an image processing model is used to remove metal artifacts in CT images, but the current image processing model has a poor effect on removing metal artifacts. Summary of the Invention

[0003] The embodiments of the present application provide an image processing method, apparatus, computer device, and storage medium, which improve the effect of removing metal artifacts. The technical solutions are as follows:

[0004] On the one hand, an image processing method is provided. The method includes:

[0005] Obtain an image processing model, where the image processing model includes structural parameters, the structural parameters represent the structure of metal artifacts, the structural parameters include first region structure parameters corresponding to a first region and second region structure parameters corresponding to a second region, the first region is any region in the metal artifacts, and the second region is the region in the metal artifacts except the first region;

[0006] When training the image processing model, adjust the first region structure parameters based on training samples, and adjust the first region structure parameters based on the angle corresponding to the first region and the angle corresponding to the second region to obtain the second region structure parameters corresponding to the second region;

[0007] Wherein, the trained image processing model is used to remove metal artifacts in any medical image based on the adjusted structural parameters.

[0008] On the other hand, an image processing apparatus is provided. The apparatus includes:

[0009] A model acquisition module, configured to obtain an image processing model, where the image processing model includes structural parameters, the structural parameters represent the structure of metal artifacts, the structural parameters include first region structure parameters corresponding to a first region and second region structure parameters corresponding to a second region, the first region is any region in the metal artifacts, and the second region is the region in the metal artifacts except the first region;

[0010] A model training module, which is used to adjust the first region structure parameters based on training samples when training the image processing model, and adjust the first region structure parameters based on the angle corresponding to the first region and the angle corresponding to the second region to obtain the second region structure parameters corresponding to the second region;

[0011] Wherein, the trained image processing model is used to remove metal artifacts in any medical image based on the adjusted structure parameters.

[0012] In a possible implementation manner, the structure parameters include an adjustment coefficient and multiple original region structure parameters, and the product of the adjustment coefficient and one of the original region structure parameters represents the region structure parameters corresponding to a region; the model training module is used to adjust the adjustment coefficient and the first original region structure parameters corresponding to the first region based on the training samples when training the image processing model.

[0013] In another possible implementation manner, each region in the metal artifacts includes at least one strip artifact, and the first original region structure parameters corresponding to the first region are matrices, and the matrices are used to represent the first region;

[0014] The model training module is used to:

[0015] Adjust the elements in the matrix that are not less than the reference value to the target value, and the target value indicates that the position does not correspond to any sub-region in the first region. The other positions in the adjusted matrix except the position where the target value is located correspond to multiple sub-regions in the first region respectively, and the elements in the other positions in the matrix except the position where the target value is located represent whether the corresponding sub-region in the first region contains a strip artifact and the shape of the strip artifact in the case that the sub-region contains the strip artifact.

[0016] In another possible implementation manner, the model training module is used to:

[0017] Determine the rotation parameters corresponding to the second region, and the rotation parameters represent the angle difference between the corresponding second region and the first region;

[0018] Based on the rotation parameters, adjust the first region structure parameters to obtain the second region structure parameters.

[0019] In another possible implementation manner, the first region structure parameter is a matrix, and the elements at other positions in the matrix except the position where the target value is located represent whether the corresponding sub-region in the first region contains a bar artifact, and the shape of the bar artifact in the case where the bar artifact is included in the sub-region. The model training module is configured to:

[0020] Based on the rotation parameter, adjust the positions of the elements in the first region structure parameter so that the elements at other positions in the obtained second region structure parameter except the position where the target value is located represent whether the corresponding sub-region in the second region contains a bar artifact, and the shape of the bar artifact in the case where the bar artifact is included in the sub-region.

[0021] In another possible implementation manner, the model training module is further configured to adjust the position extraction parameter based on the training sample when training the image processing model;

[0022] Wherein, the trained image processing model is used to remove metal artifacts in any medical image based on the adjusted structure parameter and the adjusted position extraction parameter.

[0023] In another possible implementation manner, the device further includes:

[0024] An image processing module, configured to call the trained image processing model, perform position extraction on the medical image based on the position extraction parameter to obtain a plurality of region position information, and each piece of region position information represents the position of each region in the metal artifacts included in the medical image; construct first artifact information based on the plurality of region position information, the first region structure parameter, and the second region structure parameter; and perform artifact removal on the medical image based on the first artifact information to obtain the target image.

[0025] In another possible implementation manner, the image processing module includes:

[0026] A position gradient determination unit, configured to respectively determine region position gradient information corresponding to the plurality of regions by comparing the medical image with the target image, and the region position gradient information indicates the change amplitude of the region position information;

[0027] A position information determination unit, configured to respectively adjust the plurality of region position information based on the plurality of region position gradient information to obtain the adjusted plurality of region position information;

[0028] An artifact removal unit, configured to determine adjusted first artifact information based on the adjusted multiple regional position information, the first regional structure parameter, and the second regional structure parameter, and remove artifacts from the medical image based on the adjusted first artifact information until a target number of target images are obtained, and determine the last obtained target image as the image of the medical image after removing the metal artifacts.

[0029] In another possible implementation, the position gradient determination unit is configured to:

[0030] Determine the difference information between the medical image and the target image as the second artifact information;

[0031] Based on the first artifact information and the second artifact information, respectively determine the regional position gradient information corresponding to the multiple regions.

[0032] In another possible implementation, the position gradient determination unit is configured to:

[0033] Determine the difference information between the first artifact information and the second artifact information as the artifact difference information;

[0034] Based on the artifact difference information, the first regional structure parameter, and the second regional structure parameter, respectively determine the regional position gradient information corresponding to the multiple regions.

[0035] In another possible implementation, the image processing model includes a position extraction network and an artifact removal network; the apparatus further includes:

[0036] An image processing module, configured to call the position extraction network to perform position extraction on the medical image to obtain the regional position information corresponding to multiple regions in the metal artifacts; call the artifact removal network to determine the first artifact information based on the multiple regional position information, the first regional structure parameter, and the second regional structure parameter, and remove artifacts from the medical image based on the first artifact information to obtain the target image.

[0037] On the other hand, a computer device is provided, the computer device includes a processor and a memory, and at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor to implement the operations performed by the image processing method as described in the above aspect.

[0038] On the other hand, a computer-readable storage medium is provided, and at least one computer program is stored in the computer-readable storage medium, and the at least one computer program is loaded and executed by a processor to implement the operations performed by the image processing method as described in the above aspect.

[0039] On the other hand, a computer program product is provided, including a computer program which, when executed by a processor, implements the operations performed by the image processing method described in the above aspect.

[0040] Based on the structural characteristic that metal artifacts include multiple rotationally symmetric regions, when adjusting the structural parameters of metal artifacts in the image processing model, by adjusting the first region structure parameters corresponding to the first region, and then adjusting the first region structure parameters, the second region structure parameters corresponding to the second region are determined. The structural characteristics of metal artifacts are used as prior knowledge when removing metal artifacts, fully considering the structural characteristic that metal artifacts are multiple rotationally symmetric regions, which can improve the effect of the image processing model in removing metal artifacts. At the same time, since only the first region structure parameters need to be adjusted based on the training samples and the second region structure parameters do not need to be adjusted, the training efficiency of the model can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0042] Figure 1 is a schematic diagram of an implementation environment provided by an embodiment of the present application;

[0043] Figure 2 is a flowchart of an image processing method provided by an embodiment of the present application;

[0044] Figure 3 is a flowchart of another image processing method provided by an embodiment of the present application;

[0045] Figure 4 is a schematic diagram of a medical image provided by an embodiment of the present application;

[0046] Figure 5 is a schematic diagram of a model structure provided by an embodiment of the present application;

[0047] Figure 6 is a schematic diagram of another model structure provided by an embodiment of the present application;

[0048] Figure 7 is a flowchart of yet another image processing method provided by an embodiment of the present application;

[0049] Figure 8It is a schematic diagram of an image processing process provided by an embodiment of the present application;

[0050] Figure 9 It is a flowchart of another image processing method provided by an embodiment of the present application;

[0051] Figure 10 It is a schematic structural diagram of an image processing device provided by an embodiment of the present application;

[0052] Figure 11 It is a schematic structural diagram of another image processing device provided by an embodiment of the present application;

[0053] Figure 12 It is a schematic structural diagram of a terminal provided by an embodiment of the present application;

[0054] Figure 13 It is a schematic structural diagram of a server provided by an embodiment of the present application. Detailed implementation manners

[0055] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the following will further describe the embodiments of the present application in detail with reference to the accompanying drawings.

[0056] It can be understood that the terms "first", "second", etc. used in the present application can be used herein to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the present application, the first arrangement order can be called the second arrangement order, and the second arrangement order can be called the first arrangement order.

[0057] The terms "at least one", "multiple", "each", "any one", etc. used in the present application, at least one includes one, two, or more than two, multiple includes two or more than two, each refers to each of the corresponding multiple, and any one refers to any one of the multiple. For example, multiple angles include 3 angles, and each angle refers to each of these 3 angles, and any one refers to any one of these 3 angles, which can be the first, the second, or the third.

[0058] Artificial Intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a way similar to human intelligence. Artificial intelligence also studies the design principles and implementation methods of various intelligent machines to enable machines to have the functions of perception, reasoning, and decision-making.

[0059] Artificial intelligence technology is an interdisciplinary subject with a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, and mechatronics. The software technologies of artificial intelligence mainly include several major directions such as computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning, autonomous driving, and intelligent transportation.

[0060] Computer Vision (CV) is a science that studies how to make machines "see". More specifically, it refers to using cameras and computers to replace human eyes for object recognition, measurement, and other machine vision, and further performing graphic processing to make the computer process the image into a form more suitable for human eyes to observe or transmit to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies and attempts to build artificial intelligence systems that can obtain information from images or multi-dimensional data. Computer vision technology usually includes technologies such as image processing, image recognition, image semantic understanding, image retrieval, OCR (Optical Character Recognition), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, autonomous driving, intelligent transportation, and also includes common biometric recognition technologies such as face recognition and fingerprint recognition.

[0061] Machine Learning (ML) is an interdisciplinary subject that involves multiple fields such as probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize the existing knowledge structure to continuously improve their own performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent, and its applications cover all fields of artificial intelligence. Machine learning and deep learning usually include technologies such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learning from demonstration.

[0062] With the research and progress of artificial intelligence technology, artificial intelligence technology has been studied and applied in multiple fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, driverless, autonomous driving, drones, robots, smart healthcare, smart customer service, vehicle networking, autonomous driving, intelligent transportation, etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0063] The image processing method provided by the embodiments of the present application can use computer vision technology and machine learning in artificial intelligence to perform artifact processing on medical images including metal artifacts and obtain images after removing the metal artifacts.

[0064] The image processing method provided by the embodiments of the present application can be used in a computer device. Optionally, the computer device is a terminal or a server. Optionally, the server is an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Optionally, the terminal is a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto.

[0065] In a possible implementation manner, the computer program involved in the embodiments of the present application can be deployed on a computer device for execution, or on multiple computer devices located at one place for execution, or on multiple computer devices distributed at multiple places and interconnected through a communication network. The multiple computer devices distributed at multiple places and interconnected through a communication network can form a blockchain system.

[0066] In a possible implementation manner, the computer device used to train the image processing model in the embodiments of the present application is a node in the blockchain system. The node can store the trained image processing model in the blockchain, and then the node or other device corresponding nodes in the blockchain can remove the metal artifacts in the image based on the image processing model.

[0067] Figure 1It is a schematic diagram of an implementation environment provided by an embodiment of the present application. The implementation environment includes a terminal 101 and a server 102. The terminal 101 and the server 102 are connected through a wireless or wired network. A target application provided by the server 102 is installed on the terminal 101, and the terminal 101 can implement functions such as data transmission and image processing through the target application. For example, the target application is an image processing application, and this image processing application can remove metal artifacts in CT images.

[0068] Among them, the server 102 trains an image processing model for removing metal artifacts in images. The server 102 sends the trained image processing model to the terminal 101. The terminal 101 stores the received image processing model and can subsequently process any medical image including metal artifacts based on this image processing model to obtain an image with metal artifacts removed.

[0069] The image processing method provided by the embodiments of the present application can be applied to multiple scenarios.

[0070] For example, in the medical field, scanning a patient can obtain the patient's CT image. Doctors can determine the patient's condition based on the patient's CT image and other relevant information of the patient. However, if the patient has a metal implant during the scan, metal artifacts will appear in the CT image. These metal artifacts will not only reduce the quality of the CT image but also have an adverse impact on the doctor's diagnosis process. Therefore, the image processing method provided by the embodiments of the present application can be used to remove metal artifacts in the CT image, improve the quality of the CT image, and thus provide accurate auxiliary information during the doctor's clinical diagnosis process.

[0071] Figure 2 It is a flowchart of an image processing method provided by an embodiment of the present application. The execution subject of the embodiments of the present application is a computer device. Refer to Figure 2 and the method includes the following steps:

[0072] 201. The computer device obtains an image processing model. The image processing model contains structural parameters, and these structural parameters represent the structure of metal artifacts. The structural parameters include the first region structure parameters corresponding to the first region and the second region structure parameters corresponding to the second region. The first region is any region in the metal artifact, and the second region is the region in the metal artifact except the first region.

[0073] Among them, the image processing model is used to remove metal artifacts in medical images. Metal artifacts refer to the noise information caused by metals during the generation of medical images. The medical image includes the metal artifacts and the metals that cause the metal artifacts. For example, the medical image is a CT image obtained by computer tomography of a target object. The metal artifacts in the CT image are caused by the absorption and reflection of X-rays by metals inside or on the surface of the target object, resulting in noise generated around the metals and throughout the CT image.

[0074] In the embodiments of the present application, the metal artifacts are rotationally symmetric strip structures. The metal artifacts include multiple rotationally symmetric regions, and each region contains at least one strip. The structure of the metal artifacts is a characteristic of the metal artifacts themselves. Therefore, for the image processing model, set structure parameters in the image processing model and train the structure parameters so that the structure parameters can accurately represent the structure of the metal artifacts. Among them, the first region structure parameter represents the structure of the first region in the metal artifacts, and the second region structure parameter represents the structure of the second region in the metal artifacts.

[0075] 202. When training the image processing model, the computer device adjusts the first region structure parameter based on the training samples, and adjusts the first region structure parameter based on the angle corresponding to the first region and the angle corresponding to the second region to obtain the second region structure parameter corresponding to the second region. The trained image processing model is used to remove metal artifacts in any medical image based on the adjusted structure parameters.

[0076] Since the multiple regions in the metal artifacts are rotationally symmetric, that is, the shapes of the multiple regions are the same. When the first region structure parameter corresponding to the first region is determined, the structural characteristic that the first region and the second region are rotationally symmetric can be utilized, that is, rotating the first region by a certain angle is the second region. Based on the angle corresponding to the first region and the angle corresponding to the second region, adjust the first region structure parameter to obtain the second region structure parameter corresponding to the second region. Thus, during the process of the computer device training the image processing model, the second region structure parameter can be determined by adjusting the first region structure parameter and then using the adjusted first region structure parameter, thereby improving the training efficiency.

[0077] The method provided by the embodiments of the present application, based on the structural characteristic that metal artifacts include multiple rotationally symmetric regions, when adjusting the structural parameters of metal artifacts in an image processing model, by adjusting the first region structure parameters corresponding to the first region, and then adjusting the first region structure parameters, to determine the second region structure parameters corresponding to the second region. Taking the structural characteristics of metal artifacts as prior knowledge when removing metal artifacts fully considers the structural characteristic that metal artifacts are multiple rotationally symmetric regions, which can improve the effect of the image processing model in removing metal artifacts. At the same time, since only the first region structure parameters need to be adjusted based on the training samples and the second region structure parameters do not need to be adjusted, the training efficiency of the model can be improved.

[0078] Figure 3 It is a flowchart of another image processing method provided by the embodiments of the present application. The execution subject of the embodiments of the present application is a computer device. Refer to Figure 3 and the method includes the following steps.

[0079] 301. The computer device obtains an image processing model, and the image processing model includes structural parameters and position extraction parameters.

[0080] Among them, the image processing model is used to remove metal artifacts in medical images. The image processing model is an untrained model or a model that has been trained one or more times. Metal artifacts refer to the noise information caused by metal during the generation of medical images. The structural parameters represent the structure of metal artifacts. The structure of metal artifacts belongs to the characteristics of the metal artifacts themselves. For different medical images, the structures of metal artifacts in different medical images are the same. Therefore, in the embodiments of the present application, structural parameters are set in the image processing model, and the structural parameters are obtained by training the image processing model.

[0081] Moreover, since metal artifacts are rotationally symmetric strip structures, the metal artifacts can be divided into multiple rotationally symmetric regions, and each region contains at least one strip. For these multiple regions, the shapes of any two regions are similar, and the difference lies in the angle in the metal artifacts. Therefore, for the image processing model, when setting the structural parameters in the image processing model, the first region structure parameters corresponding to the first region are set, and the second region structure parameters corresponding to the second region can be obtained by adjusting the first region structure parameters according to the angle corresponding to the first region and the angle corresponding to the second region. Among them, the first region structure parameters represent the structure of the first region in the metal artifacts, and the second region structure parameters represent the structure of the second region in the metal artifacts.

[0082] In a possible implementation, the metal artifacts are divided into multiple rotationally symmetric regions, and a reference bar in the metal artifacts is determined. The reference bar refers to any bar in the metal artifacts. Target bars are respectively determined in the multiple regions, and the target bars in each region correspond to each other. For example, the target bar in the first region is the rightmost bar in the first region. Similarly, the target bar in the second region is also the rightmost bar in the second region. Then, the angle between each target bar and the reference bar is respectively determined as the angle corresponding to each region.

[0083] In another possible implementation, based on the total number L of the regions divided in the metal artifacts, the following formula is used to determine the angle corresponding to each region:

[0084] θ l = 2π(l - 1) / L

[0085] where θ l represents the angle corresponding to the l-th region in the metal artifacts, and L is the total number of the regions divided in the metal artifacts. For example, L is 8.

[0086] Of course, the computer device can also use other methods to determine the angle corresponding to each region, and the embodiments of the present application do not limit this.

[0087] The position extraction parameter is used to extract the position information of the metal artifacts in the medical image. In different medical images, the positions where the metals are located may be different. Therefore, the position extraction parameter is set in the image processing model to extract the position information of the metal positions from the medical image.

[0088] In a possible implementation, the image processing model includes a position extraction network and an artifact removal network. The position extraction network is used to extract the position information in the medical image, and the artifact removal network is used to remove the metal artifacts in the medical image based on the position information and the structure parameters.

[0089] In another possible implementation, the image processing model includes multiple image processing sub-models, and each image processing sub-model includes a position extraction network and an artifact removal network.

[0090] 302. The computer device obtains training samples.

[0091] The training samples include sample medical images and corresponding sample target images. The sample medical images are images containing metal artifacts, and the sample target images are the images obtained by removing the metal artifacts from the sample medical images.

[0092] In a possible implementation, the computer device directly obtains a sample target image, which is an image without sample metal artifacts. The computer device obtains artifact information, which includes the position information of the metal and the structural information of the metal. The computer device uses a data simulation method to synthesize a sample medical image including metal artifacts according to the sample target image, the artifact information, and the imaging parameters of the CT device. Then, the computer device determines the sample medical image and the sample target image as training samples.

[0093] In another possible implementation, the computer device obtains a sample medical image, and then uses a method other than the image processing model in this application to remove artifacts from the sample medical image to obtain a sample target image.

[0094] Optionally, for CT images, the computer device adjusts the pixel values of the images in the training samples, controls the pixel value of each pixel point within the range of [0, 1], and then converts the pixel value of each pixel point to the range of [0, 255].

[0095] Optionally, the computer device crops the images in the training samples to the target size, and then randomly performs horizontal mirror flipping or vertical mirror flipping on each image to improve the diversity of the images in the training samples.

[0096] 303. When the computer device trains the image processing model, it adjusts the first region structure parameter and the position extraction parameter based on the training samples, and adjusts the first region structure parameter based on the angle corresponding to the first region and the angle corresponding to the second region to obtain the second region structure parameter corresponding to the second region.

[0097] The computer device calls the image processing model to process the sample medical image to obtain a predicted target image, and adjusts the first region structure parameter and the position extraction parameter based on the predicted target image and the sample target image.

[0098] In the embodiments of this application, since the shapes of the first region and the second region are the same, only the first region structure parameter corresponding to the first region needs to be adjusted based on the training samples, and then the second region structure parameter corresponding to the second region is determined based on the adjusted first region structure parameter.

[0099] In a possible implementation, the structural parameter includes an adjustment coefficient and a plurality of original region structural parameters. The product of the adjustment coefficient and one of the original region structural parameters represents the region structural parameter corresponding to a region. That is, for the first region structural parameter, the first region structural parameter is represented by the product of the adjustment coefficient and the first original region structural parameter corresponding to the first region. Then, when training the image processing model, the adjustment coefficient and the first original region structural parameter corresponding to the first region are adjusted based on the training sample, and then the adjusted first region structural parameter is determined based on the adjusted adjustment coefficient and the adjusted first original region structural parameter. For the second region structural parameter, the adjusted second region structural parameter is determined based on the adjusted adjustment coefficient, the angle corresponding to the first region, the angle corresponding to the second region, and the adjusted first original region structural parameter.

[0100] In a possible implementation, each region in the metal artifact includes at least one strip artifact. The first original region structural parameter corresponding to the first region is a matrix, and the matrix is used to represent the first region. After obtaining the matrix, the elements in the matrix that are not less than the reference value are adjusted to the target value. The target value indicates that the position does not correspond to any sub-region in the first region. The other positions in the adjusted matrix except the position where the target value is located correspond to a plurality of sub-regions in the first region respectively. Wherein, the element at each position in the matrix represents whether the corresponding sub-region in the first region contains a strip artifact, and the shape of the strip artifact in the case where the strip artifact is included in the sub-region. The reference value is determined based on the size of the convolution kernel representing the region structural parameter in the image processing model and a preset value. For example, if the size of the convolution kernel is p*p and the preset value is h, then the reference value is ((P + 1) / 2)h, where h is any value greater than 0 and p is an odd number. The target value is a preset value. For example, the target value is 0 or other values.

[0101] In a possible implementation, the computer device determines the rotation parameter corresponding to the second region. The rotation parameter represents the angle difference between the corresponding second region and the first region. For example, the rotation parameter is where θ l represents the angle difference between the second region and the first region. When the angle corresponding to the first region is 0, the angle difference is the angle corresponding to the second region. Then, based on the rotation parameter, the first region structural parameter is adjusted to obtain the second region structural parameter.

[0102] In a possible implementation, the first region structure parameter is a matrix, and the elements at other positions in the matrix except for the position where the target value is located represent whether the corresponding sub-region in the first region contains bar artifacts, and the shape of the bar artifact in the case where the bar artifact is included in the sub-region. Based on the rotation parameter, the positions of the elements in the first region structure parameter are adjusted so that the elements at other positions except for the position where the target value is located in the obtained second region structure parameter represent whether the corresponding sub-region in the second region contains bar artifacts, and the shape of the bar artifact in the case where the bar artifact is included in the sub-region. That is, by adjusting the positions of the respective elements in the first region structure parameter, the shape of the region represented by the adjusted region structure parameter remains unchanged, but the angle becomes the angle corresponding to the second region.

[0103] In a possible implementation, the smaller the error information between the predicted target image and the sample target image, the more accurate the image processing model. The computer device determines the error information between the predicted target image and the sample target image, and trains the image processing model according to the determined error information so that the error information becomes smaller and smaller, and the image processing model becomes more and more accurate.

[0104] In a possible implementation, the computer device determines the difference information between the sample medical image and the sample target image as the sample artifact information. The image processing model also outputs predicted artifact information. The predicted artifact information is the artifact information output by the image processing model, and the sample artifact information is the real artifact information corresponding to the sample medical image. Then, the smaller the error information between the predicted artifact information and the sample artifact information, the more accurate the image processing model. Therefore, the computer device respectively determines the error information between the predicted target image and the sample target image, and the error information between the predicted artifact information and the sample artifact information, and trains the image processing model according to the determined error information so that the error information becomes smaller and smaller, and the image processing model becomes more and more accurate.

[0105] For example, the computer device uses the following formula to determine the error information:

[0106]

[0107] where L represents the error information. μ n , λ1 and λ2 are compromise parameters used to balance the weights of various error information. X represents the sample target image, Y represents the sample medical image, and I represents the non-metal image corresponding to the sample medical image. X (n) represents the nth predicted target image, A (n) represents the nth predicted artifact information, N represents the total number of iterations in the image processing model, and n represents the nth iteration process. The operation of the 2-norm is denoted as ||·||₂, and the operation of the 1-norm is denoted as ||·||₁.

[0108] 304. The computer device invokes the trained image processing model to process the medical image and obtains the target image after removing the metal artifacts.

[0109] The process of invoking the image processing model to remove the metal artifacts from the medical image is described in the following Figure 7 illustrated embodiments and will not be elaborated herein.

[0110] Based on the structural characteristic that the metal artifacts contain multiple rotationally symmetric regions, when adjusting the structural parameters of the metal artifacts in the image processing model, the method provided by the embodiments of the present application determines the second region structure parameters corresponding to the second region by adjusting the first region structure parameters corresponding to the first region and then adjusting the first region structure parameters. Taking the structural characteristics of the metal artifacts as the prior knowledge when removing the metal artifacts, fully considering the structural characteristic that the metal artifacts are multiple rotationally symmetric regions, it can improve the effect of the image processing model in removing the metal artifacts. At the same time, since only the first region structure parameters need to be adjusted based on the training samples and the second region structure parameters do not need to be adjusted, the training efficiency of the model can be improved.

[0111] For the image processing model in the above embodiments, in one possible implementation, the creation process of the image processing model is as follows:

[0112] (I) Model principle:

[0113] The medical image containing metal artifacts can be represented by the following formula (1):

[0114] Formula (1): I⊙Y = I⊙X + I⊙A

[0115] Wherein, I is the medical image, Y is the target image after removing the metal artifacts, X is the non-metal image, used to represent the non-metal region in the medical image. H and W are the height and width of the image respectively. The pixel values in the non-metal image are 0 or 1, where 0 represents the metal region and 1 represents the non-metal region; A is the artifact information, representing the metal artifacts in the medical image, and ⊙ represents the point-by-point multiplication operation.

[0116] For example, the above Formula (1) can be represented as Figure 4 the image shown in FIG. The medical image 401 is determined by the medical image 402 and the metal artifact 403.

[0117] Among them, the artifact information corresponding to the metal artifact can be expressed as:

[0118] Formula (2):

[0119] Among them, represents the structural parameter of the metal artifact, and this structural parameter is represented by a convolution kernel. p×p is the size of the convolution kernel. represents the position information of the metal artifact. L represents the total number of multiple regions in the metal artifact, K represents the total number of convolution kernels corresponding to each region, k represents the k-th convolution kernel corresponding to each region, and θ l represents the angle corresponding to the l-th region in the metal artifact, and θ l = 2π(l - 1) / L, represents the two-dimensional plane convolution operation.

[0120] Among them, for the structural parameter, refer to the above Figure 4 As shown in the convolution kernel C, it can be seen that when using a convolution kernel to represent the structure of a certain region in the metal artifact, by rotating this convolution kernel, a convolution kernel representing the structure of another region in the metal artifact can be obtained. Based on this characteristic of the metal artifact in this embodiment of the present application, the structural parameter can be expressed as:

[0121] Formula Three:

[0122] Among them, a qtk and b qtk represent the adjustment parameters to be trained. represents the rotation parameter corresponding to the l-th region, and this rotation parameter can be expressed as:

[0123]

[0124] x ij represents the element in the i-th row and j-th column of the k-th convolution kernel corresponding to the l-th region:

[0125] x ij = [x i , x j T = [(i - (p + 1) / 2)h, (j - (p + 1) / 2)h] T

[0126] Among them, p represents the size of the convolution kernel, h is a preset parameter, for example, h is 1 / 4 or other values, x i represents the i-th row, and x j represents the j-th column.

[0127] and are both rotation Fourier basis functions. and are respectively expressed as:

[0128] ​

[0129]

[0130] Among them, Ω(x) represents a radial mask function, and Ω(x) ≥ 0. When ||x|| ≥ ((p + 1) / 2)h, Ω(x) = 0, and this ((p + 1) / 2)h is a reference value. When q ≤ p / 2, Otherwise, Similarly, When t ≤ p / 2, Otherwise,

[0131] Substituting Formula Two into Formula One, the following formula can be obtained to represent the medical image:

[0132] Formula Four:

[0133] Among them, and are respectively stacked by C k (θ l ) and M lk . Y is a medical image, and I is a non-metal image. Both the medical image and the non-metal image are known. The process of removing metal artifacts from the medical image is also the process of determining the position information M and the structure parameter C in Formula Five. Obtaining the position information M and the structure parameter C can determine the target image X.

[0134] Since the structure parameter C is a characteristic of the metal artifact itself and has nothing to do with the medical image, it can be assumed that the structure parameter C is known. Then, only the position information M and the target image X need to be determined. Among them, the way to determine the position information M and the target image X can be achieved by optimizing the following formula:

[0135] Formula Five:

[0136] Among them, α and β are compromise parameters, and f1(·) and f2(·) are regularization functions. The regularization function f1(·) represents a position feature, and this position feature represents the feature satisfied by the position information of the metal artifact, belonging to the prior knowledge corresponding to the position information of the metal artifact. The regularization function f2(·) represents an image feature, and this image feature represents the feature satisfied by the image without metal artifacts, belonging to the prior knowledge corresponding to the image without metal artifacts. The position information M and the target image X that can minimize the above Formula Six.

[0137] (2) Model Solving: In the embodiments of the present application, the proximal gradient technique is adopted to alternately update the position information M and the target image X to optimize Equation (5). Among them, in the nth iteration, the position information is determined by optimizing the following equation:

[0138] Equation (6):

[0139]

[0140] where M (n) represents the position information obtained in the nth iteration, X (n) represents the target image obtained in the nth iteration, and respectively represent the proximal operators corresponding to f1(·) and f2(·), M (n-1) represents the position information obtained in the (n - 1)th iteration, M (n-1) represents the target image obtained in the (n - 1)th iteration, η1 and η2 are update steps, and respectively represent the position gradient information and the image gradient information.

[0141] Among them, and are respectively expressed as:

[0142]

[0143]

[0144] (3) Model Creation: In order to use the image processing model to determine the position information and the target image, an image processing model can be constructed according to the above Equation (5). Since multiple iterations are required to determine the position information and the target image, the image processing model includes multiple position extraction networks and multiple artifact removal networks.

[0145] Among them, the position extraction network M-net and the artifact removal network X-net are respectively expressed as:

[0146]

[0147]

[0148] Among them, and are both residual networks, respectively representing the proximal operators in Equation (6) and M (n) represents the position information obtained in the nth iteration, X (n) represents the target image obtained in the nth iteration, M(n-1) represents the position information obtained in the (n - 1)-th iteration, X (n-1) represents the target image obtained in the (n - 1)-th iteration, and η1 and η2 are update steps, and represent the position gradient information and the image gradient information respectively. In the n-th iteration process, and the corresponding network parameters are respectively and η1 and η2 are update steps.

[0149] In a possible implementation manner, the proximal network is represented by a position residual network. As Figure 5 shown in the position extraction network 501 and the artifact removal network 502, input the M output by the previous position extraction network (n-1) into the position extraction network 501. In the position extraction network 501, fuse M (n-1) with and then input it into the residual network, and the residual network outputs M (n) . Similarly, input the X output by the previous displacement removal network (n-1) into the artifact removal network 502. In the artifact removal network 502, fuse X (n-1) with and then input it into the residual network, and the residual network outputs X (n) . Among them, the residual network successively includes: a convolutional layer, a Batch Normalization layer, a ReLU layer, a convolutional layer, a Batch Normalization layer, and a cross-linking layer. Optionally, the convolutional kernel size corresponding to the convolutional layer is 3x3, and the stride is 1. It should be noted that the proximal network can also adopt other types of network structures, and the embodiments of the present application do not limit this.

[0150] Based on the above creation process of the image processing model, an image processing model as Figure 6 shown can be created, where each image processing sub-model includes a position extraction network and an artifact removal network.

[0151] It should be noted that the image processing model provided by the embodiments of the present application is created based on the metal artifact removal task in the field of image processing. The network structure in the image processing model is determined by the structural characteristics of the medical image including metal artifacts and the structural characteristics of metal artifacts. Therefore, each operation in the image processing model has physical meaning, and the structure of the entire image processing model is equivalent to a white-box operation, having good model interpretability.

[0152] Figure 7It is a flowchart of another image processing method provided by an embodiment of the present application. In the embodiment of the present application, a computer device processes a medical image based on a target number of image processing sub-models in an image processing model to obtain a target image after removing metal artifacts. The image processing model includes multiple image processing sub-models, and each image processing sub-model includes a position extraction network and an artifact removal network. Then, the method includes the following steps.

[0153] 701. The computer device calls the position extraction network to determine the position information of multiple regions of metal artifacts in the medical image.

[0154] The computer device inputs the medical image into the position extraction network, performs position extraction on the medical image based on the position extraction parameters, and obtains the position information of multiple regions. Each region position information represents the position of each region in the metal artifacts included in the medical image. It should be noted that in the embodiment of the present application, the computer device will perform multiple iterative processes on the medical image based on multiple image processing sub-models. Each image processing sub-model will output the position information, artifact information, and target image corresponding to the medical image. The position information and target image output by the current image processing sub-model will be used as the input of the next image processing sub-model.

[0155] When the current position extraction network is the first position extraction network, the computer device obtains the stored position information of multiple reference regions and the third reference image. Optionally, the reference region position information is the position information preset by the computer device. For example, the position information is 0. Optionally, the third reference image is obtained by removing artifacts from the medical image, and the artifact removal method used to obtain the third reference image is different from the method provided by the embodiment of the present application. That is, the third reference image and the target image in the embodiment of the present application are obtained by removing artifacts from the medical image in different ways. For example, the third reference image is an image obtained by removing artifacts from the medical image using a linear interpolation algorithm, or the third reference image is an image obtained by removing artifacts from the medical image using Gaussian filtering or other filtering algorithms.

[0156] When the position extraction network is the position extraction network after the first position extraction network, the computer device obtains the position information and target image output by the previous image processing sub-model, and determines the input of the position extraction network based on the position information and target image output by the previous image processing sub-model.

[0157] When the position extraction network is the first position extraction network, the process by which the computer device determines the position information includes: The computer device inputs the medical image, the third reference image, and the reference position information into the position extraction network. The position extraction network determines the position gradient information by comparing the medical image and the third reference image, adjusts the reference position information according to the position gradient information, and outputs the position information. Among them, this process is the same as the process of outputting the adjusted position information in step 705 below, and will not be elaborated here for the time being.

[0158] 702. The computer device invokes the artifact removal network and constructs the first artifact information based on multiple regional position information, the first regional structure parameter, and the second regional structure parameter.

[0159] Among them, if the artifact removal network includes structure parameters, the computer device inputs the position information into the artifact removal network. The artifact removal network determines the regional artifact information corresponding to each region according to the structure parameters and the position information, that is, according to the regional position information and the corresponding regional structure parameter corresponding to each region respectively, and then forms the first artifact information from multiple regional artifact information. Optionally, the artifact removal network includes a convolution operation, and the structure parameter is a convolution kernel. Then the computer device performs convolution on the structure parameter and the position information in the artifact removal network to obtain the first artifact information.

[0160] In a possible implementation manner, the structure parameter is represented by a convolution kernel, and the computer device performs convolution processing on each regional structure parameter and the corresponding regional position information to obtain the regional artifact information.

[0161] 703. The computer device invokes the artifact removal network and removes the artifacts from the medical image according to the first artifact information to obtain the target image.

[0162] If the first artifact information obtained by the computer device is information representing the metal artifacts in the medical image, then removing the first artifact information from the medical image can obtain the target image corresponding to the medical image.

[0163] In a possible implementation manner, the medical image includes a metal region and a non-metal region. The metal region is the region where the metal is located in the medical image, and the non-metal region is the region in the medical image that does not include metal. The computer device determines the non-metal region in the medical image as the non-metal image, determines the first artifact information belonging to the non-metal region in the first artifact information, and removes the first artifact information belonging to the non-metal region from the non-metal image to obtain the target image.

[0164] In a possible implementation, the computer device determines the difference information between the medical image and the first artifact information as the second reference image, and weights the second reference image and the stored third reference image to obtain the fourth reference image. The third reference image and the target image are obtained by removing artifacts from the medical image in different ways.

[0165] 704. The computer device invokes the next location extraction network, and by comparing the medical image with the target image, respectively determines the regional position gradient information corresponding to multiple regions.

[0166] The next location extraction network is the location extraction network in the next image processing sub-model of the above artifact removal network. The computer device uses the target image as the input of the next location extraction network. Based on this location extraction network, by comparing the medical image with the target image, it respectively determines the regional position gradient information corresponding to multiple regions u.

[0167] In a possible implementation, the location extraction network includes a location extraction layer. The computer device invokes the location extraction layer, determines the difference information between the medical image and the target image as the second artifact information, and invokes the location extraction layer to respectively determine the regional position gradient information according to the first artifact information and the second artifact information.

[0168] Optionally, the computer device determines the difference information between the first artifact information and the second artifact information as the artifact difference information, and determines the position gradient information according to the artifact difference information, the first regional structure parameter, and the second regional structure parameter.

[0169] Optionally, the computer device determines the non-metal regions in the medical image. The non-metal regions refer to the regions in the medical image that do not include metal. The computer device determines the artifact difference information located in the non-metal regions in the artifact difference information, and determines the regional position gradient information of multiple regions according to the first regional structure parameter, the second regional structure parameter, and the artifact difference information located in the non-metal regions.

[0170] 705. The computer device invokes the next location extraction network, and respectively adjusts the multiple regional position information according to the multiple regional position gradient information to obtain the adjusted multiple regional position information.

[0171] 706. The computer device invokes the next artifact removal network, removes artifacts from the medical image according to the adjusted multiple regional position information to obtain the adjusted target image, until the target images output by the target number of image processing sub-models are obtained, and determines the last obtained target image as the image of the medical image after removing metal artifacts.

[0172] The image processing model includes a target number of image processing sub-models. After the artifact removal network outputs the target image, the computer device continues to use the position information, the target image, and the medical image output by the artifact removal network as the input of the next image processing sub-model of the artifact removal network. The next image processing sub-model outputs the next position information and the next target image until the target number of image processing sub-models have all executed the above process of removing metal artifacts. All the target number of image processing sub-models output target images, that is, until the target image output by the last image processing sub-model in the image processing model is obtained, the entire image processing process is completed. The computer device determines the last obtained target image as the image of the medical image after removing metal artifacts.

[0173] In a possible implementation, the artifact removal network includes an image reconstruction layer. The computer device invokes the image reconstruction layer to construct the adjusted artifact information according to the adjusted multiple region position information, the first region structure parameter, and the second region structure parameter; determines the difference information between the medical image and the adjusted artifact information as the first reference image; and weights the first reference image and the target image to obtain the adjusted target image.

[0174] For example, refer to Figure 8 , input the medical image into the image processing model. The image processing model processes the medical image based on the above steps 701 - 706 to obtain the target image.

[0175] It should be noted that the embodiments of the present application are only described by taking the image processing model including multiple image processing sub-models as an example. In another embodiment, when the image processing model includes one image processing sub-model, that is, when the image processing model includes one position extraction network and one artifact removal network, the target image output by the artifact removal network is used as the target image after removing metal artifacts.

[0176] In the method provided by the embodiments of the present application, when training the image processing model, based on the structural characteristic that metal artifacts include multiple rotationally symmetric regions, the structural characteristic of metal artifacts is used as prior knowledge when removing metal artifacts, fully considering the structural characteristic that metal artifacts are multiple rotationally symmetric regions. Therefore, when invoking the image processing model to remove metal artifacts from medical images, the effect of the image processing model in removing metal artifacts is improved.

[0177] Moreover, the embodiments of the present application perform multiple iterations, apply the result of the current iteration to the next iteration process to continuously optimize the determined target image, and determine the target image obtained in the last iteration as the image of the medical image after removing metal artifacts, which can further ensure the effect of removing metal artifacts.

[0178] Moreover, in the related art, to remove metal artifacts using a deep learning network, it is necessary to obtain the chord diagram corresponding to the image including metal artifacts and process the chord diagram. The solution provided by the embodiments of the present application is a processing method based on the image domain, which does not require collecting the chord diagram corresponding to the medical image, reducing the data acquisition cost.

[0179] Figure 9 FIG. 4 is a flowchart of another image processing method provided by the embodiments of the present application, including the training process and the testing process of the image processing model. In the training process, the computer device preprocesses the sample medical image, uses the image processing model to remove metal artifacts from the preprocessed sample medical image, and iteratively trains the image processing model according to the removal result until the number of iterations reaches the target number, and saves the trained image processing model. In the testing process, the computer device preprocesses the medical image, loads the trained image processing model, and based on this image processing model, removes metal artifacts from the preprocessed medical image and outputs the target image after removing the metal artifacts.

[0180] Figure 10 FIG. 5 is a schematic structural diagram of an image processing device provided by the embodiments of the present application. Referring to Figure 10 , the device includes:

[0181] A model acquisition module 1001, configured to acquire an image processing model, where the image processing model includes structural parameters, the structural parameters represent the structure of the metal artifacts, and the structural parameters include the first region structure parameters corresponding to the first region and the second region structure parameters corresponding to the second region. The first region is any region in the metal artifacts, and the second region is the region in the metal artifacts other than the first region;

[0182] A model training module 1002, configured to adjust the first region structure parameters based on the training samples and adjust the first region structure parameters based on the angle corresponding to the first region and the angle corresponding to the second region to obtain the second region structure parameters corresponding to the second region when training the image processing model;

[0183] Wherein, the trained image processing model is used to remove metal artifacts from any medical image based on the adjusted structural parameters.

[0184] The device provided by the embodiment of the present application, based on the structural characteristic that metal artifacts include multiple rotationally symmetric regions, when adjusting the structural parameters of metal artifacts in the image processing model, by adjusting the first region structure parameters corresponding to the first region, and then adjusting the first region structure parameters, to determine the second region structure parameters corresponding to the second region. Regarding the structural characteristics of metal artifacts as prior knowledge when removing metal artifacts, fully considering the structural characteristic that metal artifacts are multiple rotationally symmetric regions, can improve the effect of the image processing model in removing metal artifacts. At the same time, since only the first region structure parameters need to be adjusted based on the training samples and there is no need to adjust the second region structure parameters, the training efficiency of the model can be improved.

[0185] In a possible implementation manner, the structural parameters include an adjustment coefficient and multiple original region structure parameters, and the product of the adjustment coefficient and one of the original region structure parameters represents the region structure parameters corresponding to a region; the model training module 1002 is configured to, when training the image processing model, adjust the adjustment coefficient and the first original region structure parameters corresponding to the first region based on the training samples.

[0186] In another possible implementation manner, each of the regions in the metal artifacts includes at least one bar-shaped artifact, and the first original region structure parameters corresponding to the first region are matrices, and the matrices are used to represent the first region;

[0187] The model training module 1002 is configured to:

[0188] Adjust the elements in the matrix that are not less than the reference value to the target value, where the target value indicates that the position does not correspond to any sub-region in the first region. The other positions in the adjusted matrix except the position where the target value is located correspond to multiple sub-regions in the first region respectively, and the elements at the other positions in the matrix except the position where the target value is located represent whether a bar-shaped artifact is included in the corresponding sub-region in the first region, and the shape of the bar-shaped artifact in the case where the bar-shaped artifact is included in the sub-region.

[0189] In another possible implementation manner, the model training module 1002 is configured to:

[0190] Determine the rotation parameter corresponding to the second region, where the rotation parameter represents the angle difference between the corresponding second region and the first region;

[0191] Based on the rotation parameter, adjust the first region structure parameters to obtain the second region structure parameters.

[0192] In another possible implementation, the first region structure parameter is a matrix, and the elements at other positions in the matrix except for the position where the target value is located represent whether the corresponding sub-region in the first region contains bar artifacts and the shape of the bar artifact in the case where the bar artifact is included in the sub-region. The model training module 1002 is configured to:

[0193] Based on the rotation parameter, adjust the positions of the elements in the first region structure parameter so that the elements at other positions in the obtained second region structure parameter except for the position where the target value is located represent whether the corresponding sub-region in the second region contains bar artifacts and the shape of the bar artifact in the case where the bar artifact is included in the sub-region.

[0194] In another possible implementation, the model training module 1002 is further configured to adjust the position extraction parameter based on the training sample when training the image processing model;

[0195] Wherein, the trained image processing model is used to remove metal artifacts in any medical image based on the adjusted structure parameter and the adjusted position extraction parameter.

[0196] In another possible implementation, referring to Figure 11 , the apparatus further includes:

[0197] An image processing module 1003, configured to call the trained image processing model, extract positions of the medical image based on the position extraction parameter to obtain a plurality of region position information, where each piece of the region position information represents the position of each region in the metal artifact included in the medical image; construct first artifact information based on the plurality of region position information, the first region structure parameter, and the second region structure parameter; and remove artifacts from the medical image based on the first artifact information to obtain the target image.

[0198] In another possible implementation, referring to Figure 11 , the image processing module 1003 includes:

[0199] A position gradient determination unit, configured to respectively determine region position gradient information corresponding to the plurality of regions by comparing the medical image with the target image, where the region position gradient information indicates the change amplitude of the region position information;

[0200] A position information determination unit, configured to respectively adjust the plurality of region position information based on the plurality of region position gradient information to obtain the adjusted plurality of region position information;

[0201] An artifact removal unit, configured to determine adjusted first artifact information based on the adjusted multiple region position information, the first region structure parameter, and the second region structure parameter, and perform artifact removal on the medical image based on the adjusted first artifact information until a target number of target images are obtained, and determine the last obtained target image as the image of the medical image after removing the metal artifact.

[0202] In another possible implementation, the position gradient determination unit is configured to:

[0203] Determine the difference information between the medical image and the target image as the second artifact information;

[0204] Based on the first artifact information and the second artifact information, respectively determine the region position gradient information corresponding to the multiple regions.

[0205] In another possible implementation, the position gradient determination unit is configured to:

[0206] Determine the difference information between the first artifact information and the second artifact information as the artifact difference information;

[0207] Based on the artifact difference information, the first region structure parameter, and the second region structure parameter, respectively determine the region position gradient information corresponding to the multiple regions.

[0208] In another possible implementation, the image processing model includes a position extraction network and an artifact removal network; see Figure 11 , and the apparatus further includes:

[0209] An image processing module 1003, configured to call the position extraction network to perform position extraction on the medical image to obtain region position information corresponding to multiple regions in the metal artifact; call the artifact removal network to determine the first artifact information based on the multiple region position information, the first region structure parameter, and the second region structure parameter, and perform artifact removal on the medical image based on the first artifact information to obtain the target image.

[0210] All of the above optional technical solutions can be combined arbitrarily to form optional embodiments of the present application, which will not be elaborated one by one here.

[0211] It should be noted that: when the image processing apparatus provided in the above embodiments processes images, only the division of the above functional modules is used for illustration. In practical applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the computer device is divided into different functional modules to complete all or part of the functions described above. In addition, the image processing apparatus provided in the above embodiments and the embodiments of the image processing method belong to the same concept. For the specific implementation process, please refer to the method embodiments and will not be elaborated here.

[0212] An embodiment of the present application further provides a computer device, which includes a processor and a memory. At least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor to implement the operations performed by the image processing method in the above embodiments.

[0213] Optionally, the computer device is provided as a terminal. Figure 12 FIG. 1200 is a schematic structural diagram of a terminal 1200 provided by an embodiment of the present application. The terminal 1200 may be a portable mobile terminal, such as: a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV) player, a notebook computer or a desktop computer. The terminal 1200 may also be referred to by other names such as a user equipment, a portable terminal, a laptop terminal, a desktop terminal, etc.

[0214] The terminal 1200 includes: a processor 1201 and a memory 1202.

[0215] The processor 1201 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. The processor 1201 may be implemented in at least one of the following hardware forms: DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 1201 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the wake state, also known as the CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 1201 may be integrated with a GPU (Graphics Processing Unit), and the GPU is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1201 may further include an AI (Artificial Intelligence) processor, and the AI processor is used to process computational operations related to machine learning.

[0216] The memory 1202 may include one or more computer-readable storage media, and the computer-readable storage media may be non-transitory. The memory 1202 may further include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash storage devices. In some embodiments, the non-transitory computer-readable storage media in the memory 1202 is used to store at least one computer program, and the at least one computer program is used to be executed by the processor 1201 to implement the image processing method provided in the method embodiments of this application.

[0217] In some embodiments, the terminal 1200 may further optionally include: a peripheral device interface 1203 and at least one peripheral device. The processor 1201, the memory 1202, and the peripheral device interface 1203 may be connected through a bus or signal lines. Each peripheral device may be connected to the peripheral device interface 1203 through a bus, signal lines, or a circuit board. Specifically, the peripheral devices include at least one of the following: a radio frequency circuit 1204, a display screen 1205, a camera module 1206, an audio circuit 1207, and a power supply 1208.

[0218] The peripheral device interface 1203 can be used to connect at least one I / O (Input / Output) related peripheral device to the processor 1201 and the memory 1202. In some embodiments, the processor 1201, the memory 1202, and the peripheral device interface 1203 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 1201, the memory 1202, and the peripheral device interface 1203 can be implemented on a separate chip or circuit board, and this embodiment does not limit this.

[0219] The radio frequency circuit 1204 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 1204 communicates with a communication network and other communication devices through electromagnetic signals. The radio frequency circuit 1204 converts an electrical signal into an electromagnetic signal for transmission, or converts a received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 1204 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and so on. The radio frequency circuit 1204 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes but is not limited to: the World Wide Web, a metropolitan area network, an intranet, each generation of mobile communication networks (2G, 3G, 4G, and 5G), a wireless local area network, and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 1204 may further include a circuit related to NFC (Near Field Communication), and this application does not limit this.

[0220] The display screen 1205 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 1205 is a touch display screen, the display screen 1205 also has the ability to collect touch signals on or above the surface of the display screen 1205. The touch signals can be input as control signals to the processor 1201 for processing. At this time, the display screen 1205 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there may be one display screen 1205, which is provided on the front panel of the terminal 1200; in other embodiments, there may be at least two display screens 1205, which are respectively provided on different surfaces of the terminal 1200 or are in a folded design; in other embodiments, the display screen 1205 may be a flexible display screen, which is provided on the curved surface or the folding surface of the terminal 1200. Even further, the display screen 1205 can also be set to an irregular non-rectangular shape, that is, an irregular-shaped screen. The display screen 1205 can be prepared using materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0221] The camera module 1206 is used to capture images or videos. Optionally, the camera module 1206 includes a front camera and a rear camera. The front camera is provided on the front panel of the terminal, and the rear camera is provided on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth camera, a wide-angle camera, and a telephoto camera respectively, to achieve functions such as background blurring by fusing the main camera and the depth camera, panoramic shooting by fusing the main camera and the wide-angle camera, and VR (Virtual Reality) shooting functions or other fused shooting functions. In some embodiments, the camera module 1206 may further include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation under different color temperatures.

[0222] The audio circuit 1207 may include a microphone and a speaker. The microphone is used to collect sound waves of the user and the environment, and convert the sound waves into electrical signals for input to the processor 1201 for processing, or input to the radio frequency circuit 1204 to implement voice communication. For the purpose of stereo collection or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the terminal 1200. The microphone may also be an array microphone or an omnidirectional collection microphone. The speaker is used to convert the electrical signals from the processor 1201 or the radio frequency circuit 1204 into sound waves. The speaker may be a traditional thin film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert electrical signals into sound waves audible to humans, but also convert electrical signals into sound waves inaudible to humans for uses such as ranging. In some embodiments, the audio circuit 1207 may further include a headphone jack.

[0223] The power supply 1208 is used to supply power to each component in the terminal 1200. The power supply 1208 may be alternating current, direct current, a disposable battery or a rechargeable battery. When the power supply 1208 includes a rechargeable battery, the rechargeable battery may be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery charged through a wired line, and a wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0224] Those skilled in the art can understand that Figure 12 the structure shown in

[0225] does not limit the terminal 1200, and may include more or fewer components than shown in the figure, or combine some components, or adopt different component arrangements. Figure 13 is a schematic structural diagram of a server provided by an embodiment of the present application. The server 1300 may vary greatly due to different configurations or performances, and may include one or more processors (Central Processing Units, CPUs) 1301 and one or more memories 1302. Among them, at least one computer program is stored in the memory 1302, and the at least one computer program is loaded and executed by the processor 1301 to implement the methods provided by the above-mentioned various method embodiments. Of course, the server may also have components such as a wired or wireless network interface, a keyboard, and an input / output interface for input / output. The server may further include other components for implementing device functions, which will not be elaborated here.

[0226] The embodiment of the present application also provides a computer-readable storage medium, in which at least one computer program is stored, and the at least one computer program is loaded and executed by a processor to implement the operations performed by the image processing method in the above embodiment.

[0227] The embodiment of the present application also provides a computer program product, which includes a computer program that implements the operations performed by the image processing method in the above embodiment when executed by a processor.

[0228] In some embodiments, the computer program involved in the embodiment of the present application can be deployed to be executed on a single computer device, or on multiple computer devices located at one location. Alternatively, it can be executed on multiple computer devices distributed at multiple locations and interconnected through a communication network. The multiple computer devices distributed at multiple locations and interconnected through a communication network can form a blockchain system.

[0229] It can be understood that in the specific implementation of the present application, data related to user information, etc. is involved. When the above embodiments of the present application are applied to specific products or technologies, user permission or consent needs to be obtained, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions. For example, the medical images involved in the present application are obtained under full authorization.

[0230] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above embodiment can be completed by hardware, or can be completed by instructing relevant hardware through a program. The program can be stored in a computer-readable storage medium, and the above-mentioned storage medium can be a read-only memory, a magnetic disk, an optical disc, or the like.

[0231] The above are only optional embodiments of the embodiment of the present application, and are not intended to limit the embodiment of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the embodiment of the present application shall be included in the protection scope of the present application.

Claims

1. An image processing method, characterized in that, The method includes: Obtain an image processing model, where the image processing model includes structural parameters, the structural parameters represent the structure of metal artifacts, the structural parameters include first region structural parameters corresponding to a first region and second region structural parameters corresponding to a second region, the first region is any region in the metal artifacts, and the second region is the region in the metal artifacts other than the first region; When training the image processing model, adjust the first region structural parameters based on training samples, and adjust the first region structural parameters based on the angle corresponding to the first region and the angle corresponding to the second region to obtain the second region structural parameters corresponding to the second region; Among them, the trained image processing model is used to remove metal artifacts in any medical image based on the adjusted structural parameters.

2. The method according to claim 1, wherein The structural parameters include an adjustment coefficient and multiple original region structural parameters, and the product of the adjustment coefficient and one of the original region structural parameters represents the region structural parameters corresponding to a region; When training the image processing model, adjusting the first region structural parameters based on training samples includes: When training the image processing model, adjust the adjustment coefficient and the first original region structural parameters corresponding to the first region based on the training samples.

3. The method according to claim 2, characterized in that, Each region in the metal artifacts includes at least one strip artifact, and the first original region structural parameters corresponding to the first region are a matrix, and the matrix is used to represent the first region; After adjusting the adjustment coefficient and the first original region structural parameters corresponding to the first region based on the training samples when training the image processing model, the method further includes: Adjust the elements in the matrix that are not less than a reference value to a target value, where the target value indicates that the position does not correspond to any sub-region in the first region. The positions other than the position where the target value is located in the adjusted matrix correspond to multiple sub-regions in the first region respectively, and the elements at the positions other than the position where the target value is located in the matrix represent whether the corresponding sub-region in the first region contains a strip artifact and the shape of the strip artifact in the case where the strip artifact is included in the sub-region.

4. The method according to claim 1, characterized in that Adjusting the first region structural parameters based on the angle corresponding to the first region and the angle corresponding to the second region to obtain the second region structural parameters corresponding to the second region includes: Determine the rotation parameters corresponding to the second region, where the rotation parameters represent the angle difference between the corresponding second region and the first region; Based on the rotation parameters, adjust the first region structural parameters to obtain the second region structural parameters.

5. The method according to claim 4, characterized in that, The first region structure parameter is a matrix, and the elements at other positions in the matrix except the position where the target value is located represent whether the corresponding sub-region in the first region contains a bar artifact, and the shape of the bar artifact in the case where the bar artifact is included in the sub-region. Adjusting the first region structure parameter based on the rotation parameter to obtain the second region structure parameter includes: Based on the rotation parameter, adjusting the positions of the elements in the first region structure parameter so that the elements at other positions in the obtained second region structure parameter except the position where the target value is located represent whether the corresponding sub-region in the second region contains a bar artifact, and the shape of the bar artifact in the case where the bar artifact is included in the sub-region.

6. The method according to claim 1, characterized in that, The image processing model further includes a position extraction parameter, and the position extraction parameter is used to extract the position information of the metal artifact from the medical image. After obtaining the image processing model, the method further includes: When training the image processing model, adjusting the position extraction parameter based on the training sample; Wherein, the trained image processing model is used to remove the metal artifact in any medical image based on the adjusted structure parameter and the adjusted position extraction parameter.

7. The method according to claim 6, wherein The method further includes: Invoking the trained image processing model and performing the following steps: Based on the position extraction parameter, performing position extraction on the medical image to obtain a plurality of region position information, and each piece of the region position information represents the position of each region in the metal artifact included in the medical image; Based on the plurality of region position information, the first region structure parameter, and the second region structure parameter, constructing first artifact information; Based on the first artifact information, performing artifact removal on the medical image to obtain a target image.

8. The method according to claim 7, wherein After performing artifact removal on the medical image based on the first artifact information to obtain a target image, the method further includes: By comparing the medical image with the target image, respectively determining the region position gradient information corresponding to the plurality of regions, and the region position gradient information indicates the change amplitude of the region position information; Based on the plurality of region position gradient information, respectively adjusting the plurality of region position information to obtain the adjusted plurality of region position information; Based on the adjusted plurality of region position information, the first region structure parameter, and the second region structure parameter, determining the adjusted first artifact information, and based on the adjusted first artifact information, performing artifact removal on the medical image until a target number of target images are obtained, and determining the last obtained target image as the image of the medical image after removing the metal artifact.

9. The method according to claim 8, wherein The step of respectively determining the region position gradient information corresponding to the plurality of regions by comparing the medical image with the target image includes: Determining the difference information between the medical image and the target image as second artifact information; Based on the first artifact information and the second artifact information, respectively determining the region position gradient information corresponding to the plurality of regions.

10. The method according to claim 9, characterized in that, Determining the region position gradient information corresponding to the multiple regions respectively based on the first artifact information and the second artifact information includes: Determining the difference information between the first artifact information and the second artifact information as the artifact difference information; Based on the artifact difference information, the first region structure parameter, and the second region structure parameter, determining the region position gradient information corresponding to the multiple regions respectively.

11. The method according to claim 1, characterized in that, The image processing model includes a position extraction network and an artifact removal network; the method further includes: Invoking the position extraction network to perform position extraction on the medical image to obtain the region position information corresponding to multiple regions in the metal artifact; Invoking the artifact removal network, based on the multiple region position information, the first region structure parameter, and the second region structure parameter, determining the first artifact information, and based on the first artifact information, performing artifact removal on the medical image to obtain a target image.

12. An image processing apparatus, characterized in that, The apparatus includes: A model acquisition module, configured to acquire an image processing model, where the image processing model includes structure parameters, the structure parameters represent the structure of the metal artifact, the structure parameters include a first region structure parameter corresponding to a first region and a second region structure parameter corresponding to a second region, the first region is any region in the metal artifact, and the second region is the region in the metal artifact other than the first region; A model training module, configured to, when training the image processing model, adjust the first region structure parameter based on a training sample, and adjust the first region structure parameter based on the angle corresponding to the first region and the angle corresponding to the second region to obtain the second region structure parameter corresponding to the second region; Wherein, the trained image processing model is used to remove the metal artifact in any medical image based on the adjusted structure parameters.

13. The device according to claim 12, characterized in that, The structure parameters include an adjustment coefficient and multiple original region structure parameters, and the product of the adjustment coefficient and one of the original region structure parameters represents the region structure parameter corresponding to a region; the model training module is configured to: When training the image processing model, adjust the adjustment coefficient and the first original region structure parameter corresponding to the first region based on the training sample.

14. The device according to claim 13, wherein Each region in the metal artifact includes at least one strip artifact, and the first original region structure parameter corresponding to the first region is a matrix, and the matrix is used to represent the first region; The model training module is further configured to: Adjust the elements in the matrix that are not less than a reference value to a target value, where the target value indicates that the position does not correspond to any sub-region in the first region, and the other positions in the adjusted matrix except the position where the target value is located correspond to multiple sub-regions in the first region respectively, and the elements in the other positions in the matrix except the position where the target value is located represent whether the corresponding sub-region in the first region contains a strip artifact and the shape of the strip artifact in the case where the strip artifact is included in the sub-region.

15. The device according to claim 12, characterized in that, The model training module is configured to: Determine the rotation parameter corresponding to the second region, where the rotation parameter represents the angular difference between the corresponding second region and the first region; Based on the rotation parameter, adjust the structural parameters of the first region to obtain the structural parameters of the second region.

16. The device according to claim 15, characterized in that, The structural parameters of the first region are a matrix, and the elements at other positions in the matrix except the position where the target value is located represent whether the corresponding sub-region in the first region contains a bar artifact, and the shape of the bar artifact in the case where the bar artifact is included in the sub-region. The model training module is used for: Based on the rotation parameter, adjust the positions of the elements in the structural parameters of the first region so that the elements at other positions in the obtained structural parameters of the second region except the position where the target value is located represent whether the corresponding sub-region in the second region contains a bar artifact, and the shape of the bar artifact in the case where the bar artifact is included in the sub-region.

17. The device according to claim 12, characterized in that, The image processing model further includes a position extraction parameter, which is used to extract the position information of the metal artifact from the medical image. The model training module is further used for: When training the image processing model, adjust the position extraction parameter based on the training sample; Wherein, the trained image processing model is used to remove the metal artifact in any medical image based on the adjusted structural parameters and the adjusted position extraction parameter.

18. The device according to claim 17, characterized in that, The device further includes: An image processing module, configured to call the trained image processing model and perform the following steps: Based on the position extraction parameter, perform position extraction on the medical image to obtain a plurality of region position information, and each piece of region position information represents the position of each region in the metal artifact included in the medical image; Based on the plurality of region position information, the structural parameters of the first region, and the structural parameters of the second region, construct first artifact information; Based on the first artifact information, perform artifact removal on the medical image to obtain a target image.

19. The device according to claim 18, wherein The image processing module includes: A position gradient determination unit, configured to respectively determine the region position gradient information corresponding to the plurality of regions by comparing the medical image with the target image, and the region position gradient information indicates the change amplitude of the region position information; A position information determination unit, configured to respectively adjust the plurality of region position information based on the plurality of region position gradient information to obtain the adjusted plurality of region position information; An artifact removal unit, configured to determine the adjusted first artifact information based on the adjusted plurality of region position information, the structural parameters of the first region, and the structural parameters of the second region, and perform artifact removal on the medical image based on the adjusted first artifact information until a target number of target images are obtained, and determine the last obtained target image as the image of the medical image after removing the metal artifact.

20. The device according to claim 19, characterized in that, The position gradient determination unit is used for: Determine the difference information between the medical image and the target image as the second artifact information; Based on the first artifact information and the second artifact information, respectively determine the region position gradient information corresponding to the multiple regions.

21. The device according to claim 20, characterized in that, The position gradient determination unit is configured to: Determine the difference information between the first artifact information and the second artifact information as the artifact difference information; Based on the artifact difference information, the first region structure parameter, and the second region structure parameter, respectively determine the region position gradient information corresponding to the multiple regions.

22. The device according to claim 12, wherein The image processing model includes a position extraction network and an artifact removal network; the apparatus further includes: An image processing module, configured to call the position extraction network to perform position extraction on the medical image to obtain the region position information corresponding to multiple regions in the metal artifact; The image processing module is further configured to call the artifact removal network, based on the multiple region position information, the first region structure parameter, and the second region structure parameter, determine the first artifact information, and based on the first artifact information, perform artifact removal on the medical image to obtain a target image.

23. A computer device, characterized in that, The computer device includes a processor and a memory, and at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor to implement the operations performed by the image processing method according to any one of claims 1 to 11.

24. A computer-readable storage medium, characterized in that, At least one computer program is stored in the computer-readable storage medium, and the at least one computer program is loaded and executed by a processor to implement the operations performed by the image processing method according to any one of claims 1 to 11.

25. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the operations performed by the image processing method according to any one of claims 1 to 11.

Images