Image processing method and device, storage medium and electronic equipment
By calculating the marker points of unlabeled tomographic images using the grayscale values and deformation displacement parameters of labeled tomographic images in MRI images, and combining this with the watershed algorithm to automatically segment the target image region, the problem of low efficiency of manual annotation is solved, and efficient and accurate image processing is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NEUSOFT CORP
- Filing Date
- 2021-12-24
- Publication Date
- 2026-06-02
AI Technical Summary
In the medical field, multi-layer image analysis of MRI images requires manual annotation of each layer to determine the target image region, resulting in low efficiency.
By acquiring the grayscale values of labeled and unlabeled fault images, the corresponding second label point of each first label point in the unlabeled fault image is determined. The position of the label point is calculated using the target deformation and displacement parameters, and the target image region is automatically segmented using the watershed algorithm.
This reduces the workload of manually marking points and improves the efficiency and accuracy of image processing.
Smart Images
Figure CN114445338B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of artificial intelligence, and more specifically, to an image processing method, apparatus, storage medium, and electronic device. Background Technology
[0002] In the medical field, using deep learning algorithms to extract the target image region from MRI images for analysis has become a common image analysis method. For example, deep learning algorithms can be used to analyze MRI images of the spine to obtain the target image region containing the spine. To more accurately determine the target image region, manual annotation of the image is required to identify marker points (each marker point can represent the target image region containing the target object or the background region excluding the target object). The annotated image is then input into the deep learning algorithm model so that the algorithm can extract a more accurate target image region based on the marker points. In related technologies, MRI images consist of multiple layers, requiring manual annotation of each layer to determine the target image region for each layer. However, this method requires a large amount of manual image annotation, resulting in low efficiency. Summary of the Invention
[0003] The purpose of this disclosure is to provide an image processing method, apparatus, storage medium, and electronic device to solve the aforementioned problems existing in the related art.
[0004] To achieve the above objectives, a first aspect of this disclosure provides an image processing method, the method comprising:
[0005] Obtain labeled and unlabeled tomographic images of the target object; wherein, the labeled tomographic image includes one or more first marker points, the first marker points being used to mark the target image region where the target object is located or the background region excluding the target object in the labeled tomographic image;
[0006] Based on the grayscale values of the labeled fault image and the unlabeled fault image, determine the second marker point corresponding to each first marker point in the unlabeled fault image;
[0007] Based on the second marker point, the target image region where the target object is located in the unlabeled tomographic image is determined.
[0008] Optionally, determining the second marker point corresponding to each first marker point in the unlabeled fault image based on the grayscale values of the labeled fault image and the unlabeled fault image includes:
[0009] Based on the grayscale values of the labeled and unlabeled fault images, target deformation parameters and target displacement parameters are determined; wherein, the target deformation parameters are used to characterize the parameters for shape transformation of the first adjacent region corresponding to the first marker point, and the target displacement parameters are used to characterize the parameters for positional movement of the first marker point;
[0010] For each first marker point, the position information of the corresponding second marker point in the unlabeled fault image is calculated based on the target deformation parameter, the target displacement parameter, and the position information of the first marker point.
[0011] Optionally, determining the target image region where the target object is located in the unlabeled tomographic image based on the second marker point includes:
[0012] For each second marker point, the adjacent points of the second marker point are designated as similar marker points of the second marker point; wherein, both the similar marker points and the second marker point are used to mark the target area of the unlabeled tomographic image, or both are used to mark the background area of the unlabeled tomographic image;
[0013] Based on the second marker point and the similar marker points, the target image region where the target object is located in the unlabeled tomographic image is determined.
[0014] Optionally, the first marker point is obtained in the following way:
[0015] Obtain one or more candidate markers that the user has annotated on the labeled tomographic image;
[0016] The labeled tomographic image is used as the first target layer image. The marker point adjustment step is executed repeatedly until the candidate marker point is determined to meet the preset marking conditions. The candidate marker point is then used as the first marker point.
[0017] The marker adjustment step includes:
[0018] Based on the candidate marker points, a first candidate target region is determined in the first target layer image where the target object is located;
[0019] The first candidate target region is displayed so that the user can adjust the candidate markers based on the first candidate target region, or the user can determine the first candidate target region as the expected target region;
[0020] If the user adjusts the candidate markers according to the first candidate target region, the adjusted candidate markers will be used as new candidate markers.
[0021] If the user determines that the first candidate target area is the expected target area, the candidate marker point is determined to meet the preset marking conditions.
[0022] Optionally, before segmenting the unlabeled tomographic image based on the second marker points, the method further includes:
[0023] The second marker point is used as one or more candidate marker points on the unlabeled tomographic image;
[0024] The unlabeled tomographic image is used as the first target layer image, and the marker adjustment step is executed repeatedly until the candidate markers meet the preset marking conditions. The candidate markers are then used as new second markers.
[0025] Optionally, the method further includes:
[0026] Extract candidate images for each layer of the image to be processed in the target direction, wherein the image to be processed is a stereo image including the target object;
[0027] The candidate image is preprocessed to obtain the second target layer image, which includes the labeled fault image and the unlabeled fault image.
[0028] Optionally, the step of preprocessing the candidate image to obtain the second target layer image includes:
[0029] After processing the candidate image into grayscale, a grayscale image is obtained;
[0030] After noise processing and / or contrast processing of the grayscale image, the second target layer image is obtained.
[0031] Secondly, this disclosure provides an image processing apparatus, the apparatus comprising:
[0032] A tomographic image acquisition module is used to acquire labeled and unlabeled tomographic images of a target object; wherein, the labeled tomographic image includes one or more first marker points, the first marker points being used to mark the target image region where the target object is located or the background region excluding the target object in the labeled tomographic image;
[0033] The marker point determination module is used to determine, based on the grayscale values of the labeled fault image and the unlabeled fault image, a second marker point corresponding to each first marker point in the unlabeled fault image;
[0034] The target image processing module is used to determine the target image region where the target object is located in the unlabeled tomographic image based on the second marker point.
[0035] Optionally, the marker point determination module is used to determine target deformation parameters and target displacement parameters based on the grayscale values of the labeled fault image and the unlabeled fault image; wherein, the target deformation parameters are used to characterize the parameters for shape transformation of the first adjacent region corresponding to the first marker point, and the target displacement parameters are used to characterize the parameters for positional movement of the first marker point; for each first marker point, the position information of the second marker point corresponding to the first marker point in the unlabeled fault image is calculated based on the target deformation parameters, the target displacement parameters, and the position information of the first marker point.
[0036] Optionally, the target image processing module is configured to, for each second marker point, use adjacent points of the second marker point as similar marker points of the second marker point; wherein, both the similar marker points and the second marker point are used to mark the target region of the unlabeled tomographic image, or both are used to mark the background region of the unlabeled tomographic image; and determine the target image region where the target object is located in the unlabeled tomographic image based on the second marker point and the similar marker points.
[0037] Optionally, the device further includes:
[0038] The first marker acquisition module is used to acquire one or more candidate markers marked by the user on the labeled tomographic image; take the labeled tomographic image as the first target layer image, and repeatedly execute the marker adjustment step until it is determined that the candidate markers meet the preset marking conditions, and take the candidate markers as the first markers;
[0039] The marker adjustment step includes: determining a first candidate target region in the first target layer image where the target object is located based on the candidate markers; displaying the first candidate target region so that the user can adjust the candidate markers based on the first candidate target region, or the user determines that the first candidate target region is the expected target region; if the user adjusts the candidate markers based on the first candidate target region, using the adjusted candidate markers as new candidate markers; and if the user determines that the first candidate target region is the expected target region, determining that the candidate markers meet preset marking conditions.
[0040] Optionally, before segmenting the unlabeled tomographic image based on the second marker point, the apparatus further includes:
[0041] The second marker acquisition module is used to use the second marker as one or more candidate markers on the unlabeled tomographic image; and to use the unlabeled tomographic image as the first target layer image, and to repeatedly execute the marker adjustment step until it is determined that the candidate marker meets the preset marking condition, and then use the candidate marker as a new second marker.
[0042] Optionally, the device further includes:
[0043] An image preprocessing module is used to extract candidate images for each layer of the image to be processed in the target direction, wherein the image to be processed is a stereoscopic image including the target object; after preprocessing the candidate images, a second target layer image is obtained, wherein the second target layer image includes the labeled tomographic image and the unlabeled tomographic image.
[0044] Optionally, the image preprocessing module is used to process the candidate image into grayscale to obtain a grayscale image; and to process the grayscale image into noise and / or contrast to obtain the second target layer image.
[0045] Thirdly, this disclosure provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the method described in the first aspect of this disclosure.
[0046] Fourthly, this disclosure provides an electronic device, comprising: a memory having a computer program stored thereon; and a processor for executing the computer program in the memory to implement the steps of the method described in the first aspect of this disclosure.
[0047] The above technical solution acquires labeled and unlabeled tomographic images of the target object. The labeled tomographic image includes one or more first marker points, which are used to mark the target image region containing the target object or the background region excluding the target object within the labeled tomographic image. Based on the grayscale values of the labeled and unlabeled tomographic images, a second marker point corresponding to each first marker point in the unlabeled tomographic image is determined. Based on the second marker point, the target image region containing the target object in the unlabeled tomographic image is determined. In this way, the second marker point in the unlabeled tomographic image can be accurately determined based on the first marker point in the labeled tomographic image, thereby reducing the workload of manual marker point annotation and improving image processing efficiency.
[0048] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description
[0049] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the following detailed description to explain the present disclosure, but do not constitute a limitation thereof. In the drawings:
[0050] Figure 1 This is a flowchart of an image processing method provided in an embodiment of this disclosure.
[0051] Figure 2 This is a flowchart of a method for obtaining a first marker point provided in an embodiment of this disclosure.
[0052] Figure 3 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this disclosure.
[0053] Figure 4 This is a schematic diagram of the structure of a second image processing apparatus provided in the embodiments of this disclosure.
[0054] Figure 5 This is a schematic diagram of the structure of the third image processing device provided in the embodiments of this disclosure.
[0055] Figure 6 This is a schematic diagram of the structure of the fourth image processing device provided in the embodiments of this disclosure.
[0056] Figure 7 This is a block diagram of an electronic device provided in an embodiment of this disclosure. Detailed Implementation
[0057] The specific embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit this disclosure.
[0058] It should be noted that in this disclosure, terms such as "first" and "second" are used only for distinguishing purposes and should not be construed as indicating or implying relative importance or order; terms such as "S101," "S102," "S201," and "S202" are used to distinguish steps and should not be construed as performing method steps in a specific order or sequence; when the following description relates to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements.
[0059] First, the application scenarios of this disclosure will be described. This disclosure can be applied to image processing scenarios, particularly in the medical field for processing MRI images. In related technologies, MRI images consist of multiple layers, requiring manual labeling of each layer so that deep learning algorithms can be used to analyze each layer and determine the target image region. However, this method requires a large amount of manual image annotation, which is inefficient.
[0060] To address the aforementioned problems, this disclosure provides an image processing method, apparatus, storage medium, and electronic device. Based on first marker points in an annotated tomographic image and the grayscale values of the annotated and unannotated tomographic images, a second marker point corresponding to each first marker point in the unannotated tomographic image is determined. Then, based on the second marker point, the target image region containing the target object in the unannotated tomographic image is determined. This reduces the workload of manually annotating marker points and improves the efficiency of image processing.
[0061] The specific embodiments of this disclosure will be described in detail below with reference to the accompanying drawings.
[0062] Figure 1 This is an image processing method provided in an embodiment of the present disclosure, such as... Figure 1 As shown, the method may include:
[0063] S101. Obtain the labeled and unlabeled fault images of the target object.
[0064] The labeled tomographic image includes one or more first marker points, which are used to mark the target image region where the target object is located or the background region that does not include the target object in the labeled tomographic image.
[0065] For example, the labeled and unlabeled tomographic images mentioned above can be tomographic images obtained by slicing a target object. For instance, the target object can be the spine of a human or animal, steel bars or wood in a building wall, etc. Through MRI detection, an MRI image including multiple tomographic images can be obtained. A first marker point can be added to any one of these multiple tomographic images, and the tomographic image with the first marker point added is used as the labeled tomographic image mentioned above. Other tomographic images corresponding to the target object can be used as the unlabeled tomographic images mentioned above.
[0066] Furthermore, the MRI image can be a stereoscopic image, comprising multiple tomographic images along a target direction, which can be a sagittal, coronal, or transverse direction. Along this target direction, the multiple tomographic images can be ordered according to their positions. A first marker is added to the first tomographic image, and this first tomographic image with the added first marker is designated as the labeled tomographic image. The tomographic images adjacent to the labeled tomographic image are designated as the unlabeled tomographic images.
[0067] The aforementioned first marker point can be a marker point manually annotated by the user on the tomographic image. Further, the first marker point can be divided into two categories: a first target object marker point and a first background marker point. The first target object marker point is used to mark the target image region where the target object is located in the annotated tomographic image; the first target object marker point is used to mark the background region in the annotated tomographic image that does not include the target object.
[0068] In one example of this disclosure, the watershed algorithm can be used to segment an annotated fault image, thereby determining the target image region where the target object is located within the annotated fault image. The watershed algorithm is a mathematical morphology segmentation method based on topological theory. An image can be viewed as a topological landform in geodesy, where the gray value of each pixel represents its altitude, each local minimum and its affected area are called a catchment basin, and the boundary of the catchment basin forms a watershed. Due to the gray value distribution of the target object in the fault image, redundant edge regions may appear, leading to oversegmentation by the watershed algorithm—that is, segmenting a whole target object that should not be segmented. To eliminate oversegmentation caused by the watershed algorithm, a method of adding marker points is usually used to remove redundant edge regions, thereby avoiding oversegmentation. The watershed algorithm used in this embodiment can refer to existing algorithms in related technologies, and will not be described in detail here.
[0069] S102. Based on the grayscale values of the labeled fault images and the unlabeled fault images, determine the second label point corresponding to each first label point in the unlabeled fault image.
[0070] In this step, the second marker point can be determined using any of the following methods:
[0071] Method 1: First, determine the target displacement parameter based on the grayscale values of the labeled fault image and the unlabeled fault image; then, for each first marker point, calculate the position information of the corresponding second marker point in the unlabeled fault image based on the target displacement parameter and the position information of the first marker point.
[0072] For example, the target displacement parameters can be determined by a first preset optical flow algorithm, which may include the following formula (1):
[0073]
[0074] Wherein, (x11, y11) represents the position information of the first marker point in the labeled fault image (i.e., the pixel coordinates of the first marker point), (x12, y12) represents the position information of the second marker point in the unlabeled fault image corresponding to the first marker point (x11, y11) (i.e., the pixel coordinates of the second marker point), (Δx1, Δy1) represents the target displacement parameters, where Δx1 represents the displacement difference of the x-axis coordinate of the second marker point relative to the first marker point, i.e., x12 = x11 + Δx1, Δy1 represents the displacement difference of the y-axis coordinate of the second marker point relative to the first marker point, i.e., y12 = y11 + Δy1, I(x11, y11) represents the first gray value of the first marker point (x11, y11) in the labeled fault image, J(x12, y12) represents the second gray value of the second marker point (x12, y12) in the unlabeled fault image, [[-w x11 ,w x11 ],[-w y11 ,w y11 ]] represents the adjacent area of the preset rectangle corresponding to the first marker point (x11, y11), ε2 represents the difference in grayscale values between the first marker point and the second marker point, min (Δx1,Δy1) (ε1) represents the value of (Δx1, Δy1) that minimizes ε1 according to formula (1). For example, the value of (Δx2, Δy2) can be obtained by solving formula (1) using the least squares method. It should be noted that the method of obtaining the value of (Δx2, Δy2) using the least squares method can be found in the description in related technologies, and will not be repeated here.
[0075] Then, for each first marker point, the location information of the corresponding second marker point in the unlabeled tomographic image is calculated according to the following formula (2):
[0076]
[0077] in, This represents the pixel coordinates of the first marker point. This indicates the pixel coordinates of the second marker point corresponding to the first marker point in the aforementioned unlabeled tomographic image. This represents the target displacement parameters mentioned above.
[0078] Furthermore, the aforementioned first preset optical flow algorithm can be an LK optical flow algorithm based on pyramid layering. The window size corresponding to the first preset optical flow algorithm can be a preset window (e.g., 20 pixels). If the distance between the tracked second marker point and the corresponding first marker point is less than or equal to the preset window, the second marker point is retained; if the distance between the tracked second marker point and the corresponding first marker point is greater than the preset window, the second marker point is discarded.
[0079] It should be noted that, due to the inter-slice similarity of MRI images, when the difference in gray value distribution between two adjacent layers is small, using this method, the target displacement parameters are determined by the first preset optical flow algorithm, and the second marker point of the unlabeled tomographic image is calculated based on the target displacement parameters. When performing marker-based watershed segmentation on the unlabeled tomographic image, there is no need to reselect the marker point, which can reduce the workload of manual marking and improve the efficiency of image processing.
[0080] Method 2: First, determine the target deformation parameters and target displacement parameters based on the grayscale values of the labeled fault image and the unlabeled fault image; then, for each first marker point, calculate the position information of the corresponding second marker point based on the target deformation parameters, target displacement parameters, and the position information of the first marker point.
[0081] The target deformation parameter is used to characterize the parameters for shape transformation of the first adjacent region corresponding to the first marker point, and the target displacement parameter is used to characterize the parameters for positional movement of the first marker point.
[0082] For example, the target displacement parameters can be determined by a second preset optical flow algorithm, which may include the following formula (3):
[0083]
[0084] Wherein, (x21, y21) represents the position information of the first marker point in the labeled fault image (i.e., the pixel coordinates of the first marker point); (x22, y22) represents the position information of the second marker point in the unlabeled fault image corresponding to the first marker point (x21, y21) (i.e., the pixel coordinates of the second marker point); (Δx2, Δy2) represents the target displacement parameter; where Δx2 represents the x-axis coordinate of the second marker point relative to the first marker point. The displacement difference is x22 = x21 + Δx2; Δy2 represents the displacement difference of the y-axis coordinate of the second marker point relative to the first marker point, which is y22 = y21 + Δy2; a represents the target deformation parameter mentioned above; I(x21, y21) represents the first gray value of the first marker point (x21, y21) in the labeled fault image; J(x22, y22) represents the second gray value of the second marker point (x22, y22) in the unlabeled fault image; [[-w x21 ,w x21 ],[-w y21 ,w y21 ]] represents the adjacent area of the preset rectangle corresponding to the first marker point (x21, y21); ε2 represents the difference in grayscale values between the first marker point and the second marker point; min (Δx2,Δy2),a (ε2) represents the values of (Δx2,Δy2) and a that minimize ε2 by solving formula (3). For example, the values of (Δx2,Δy2) and a can be obtained by solving formula (3) using the least squares method.
[0085] Furthermore, the target deformation parameter 'a' in the above formula (3) can be a parameter matrix.
[0086] Then, for each first marker point, the location information of the corresponding second marker point in the unlabeled tomographic image is calculated according to the following formula (4):
[0087]
[0088] in, This represents the pixel coordinates of the first marker point. This indicates the pixel coordinates of the second marker point corresponding to the first marker point in the aforementioned unlabeled tomographic image. This represents the target deformation parameter mentioned above. This represents the target displacement parameters mentioned above.
[0089] Similarly, the second preset optical flow algorithm can be an LK optical flow algorithm based on pyramid layering. The window size corresponding to the second preset optical flow algorithm can be a preset window (e.g., 20 pixels). If the distance between the tracked second marker point and the corresponding first marker point is less than or equal to the preset window, the second marker point is retained; if the distance between the tracked second marker point and the corresponding first marker point is greater than the preset window, the second marker point is discarded.
[0090] Thus, by introducing target deformation parameters in Method 2, the adjacent regions of the marked points are transformed (e.g., rotated or scaled) between two tomographic images (annotated and unannotated tomographic images). This can adapt to tomographic images where the target object has certain shape differences between different layers, thereby improving the accuracy of the marked points. This can improve both image processing efficiency and the accuracy of the acquired target image region.
[0091] S103. Based on the second marker point, determine the target image region where the target object is located in the unmarked fault image.
[0092] For example, the second marker point can be used as a marker point for the watershed algorithm. The marker-based watershed algorithm can be used to segment the unlabeled tomographic image, thereby determining the target image region where the target object is located in the unlabeled tomographic image.
[0093] Using the above method, labeled and unlabeled tomographic images of the target object are obtained. The labeled tomographic image includes one or more first marker points, which are used to mark the target image region containing the target object or the background region excluding the target object in the labeled tomographic image. Based on the grayscale values of the labeled and unlabeled tomographic images, a second marker point corresponding to each first marker point in the unlabeled tomographic image is determined. Based on the second marker point, the target image region containing the target object in the unlabeled tomographic image is determined. In this way, the second marker point in the unlabeled tomographic image can be accurately determined based on the first marker point in the labeled tomographic image, thereby reducing the workload of manual marker point annotation and improving the efficiency of image processing.
[0094] In another embodiment of this disclosure, step S103 described above can also be implemented in the following manner:
[0095] First, for each second marker point, the adjacent points of the second marker point are taken as the same type of marker point of the second marker point.
[0096] Both the first and second markers are used to mark the target area of the unlabeled fault image, or both are used to mark the background area of the unlabeled fault image.
[0097] For example, the adjacent points of the second marker point can be all pixels within a region centered on the second marker point and with radius r, where r is a preset radius and can be any value greater than or equal to 1 pixel, for example, r can be 3 pixels or 5 pixels.
[0098] Then, based on the second marker point and the same type of marker point, the target image region where the target object is located in the unlabeled tomographic image is determined.
[0099] It should be noted that when determining the target image region where the target object is located in the labeled tomographic image based on the first marker point, a similar method described above can be used. For each first marker point, the adjacent points of that first marker point are treated as similar marker points. Both the similar marker points and the first marker point are used to mark the target region of the unlabeled tomographic image, or both are used to mark the background region of the unlabeled tomographic image. Then, based on the first marker point and the similar marker points, the target image region where the target object is located in the labeled tomographic image is determined.
[0100] In this way, by treating adjacent regions of the marked points as similar marked points, the workload of manual operations can be further reduced, and the reliability of image segmentation can be improved.
[0101] Figure 2 This disclosure provides a method for obtaining a first marker point, such as... Figure 2 As shown, the first marker point is obtained in the following way:
[0102] S201. Obtain one or more candidate marker points marked by the user on the labeled tomographic image.
[0103] The candidate marker can be a manually labeled marker.
[0104] S202. Using the labeled tomographic image as the first target layer image, repeatedly execute the marker adjustment step until it is determined that the candidate marker meets the preset marking conditions, and then use the candidate marker as the first marker.
[0105] The marker adjustment step may include:
[0106] S11. Based on the candidate marker point, determine the first candidate target region where the target object is located in the first target layer image.
[0107] S12. Display the first candidate target area so that the user can adjust the candidate marker point according to the first candidate target area, or the user can determine the first candidate target area as the expected target area.
[0108] S13. If the user adjusts the candidate marker point according to the first candidate target area, the adjusted candidate marker point is used as the new candidate marker point; or, if the user determines that the first candidate target area is the expected target area, the candidate marker point is determined to meet the preset marking conditions.
[0109] For example, a label-based watershed algorithm can be used to segment the first target layer image to determine the target image region where the target object is located in the first target layer image, and the first candidate target region can be displayed to the user through an electronic device.
[0110] In this way, users can determine whether the first candidate target area displayed is the expected target area they desire.
[0111] If the first candidate target region is determined to be inconsistent with the expected target region, the candidate markers can be adjusted (including adding, deleting, and modifying) via electronic devices. For example, if oversegmentation occurs due to a candidate marker, that candidate marker is deleted; if an area is undersegmented due to a lack of markers, a new candidate marker can be added to that area. After the user completes the adjustment of the candidate markers, they can send a marker adjustment command via electronic devices (for example, the user triggers the marker adjustment command by clicking a preset adjustment button on the electronic device). Upon receiving the marker adjustment command, the adjusted candidate marker can be used as the new candidate marker, and the above marker adjustment steps are repeated cyclically.
[0112] If the first candidate target area is determined to be the expected target area, a first preset command can be issued through an electronic device (e.g., by clicking a first preset button to trigger the first preset command). Upon receiving the first preset command, it can be determined that the candidate marker meets the preset marking conditions, the marker adjustment steps are stopped, and the candidate marker is adopted as the first marker.
[0113] This method enables semi-automatic, interactive control of marker points, thereby improving the efficiency and accuracy of manual annotation, as well as the efficiency and accuracy of image processing.
[0114] Furthermore, before segmenting the unlabeled tomographic image based on the second marker point, interactive marker point control can be performed on the second marker point of the unlabeled tomographic image to further improve the efficiency and accuracy of image processing. For example, the second marker point can first be used as one or more candidate marker points on the unlabeled tomographic image; then, the unlabeled tomographic image can be used as the first target layer image, and the above marker point adjustment steps can be executed cyclically until it is determined that the candidate marker point meets the preset marking conditions, and then the candidate marker point is used as the new second marker point.
[0115] In this way, the accuracy of the automatically acquired second marker point can be manually verified, thereby further improving the accuracy of the second marker point and also improving the accuracy of acquiring the target image region.
[0116] In another embodiment of this disclosure, the second target image includes an labeled fault image and an unlabeled fault image, which can be obtained by the following methods:
[0117] First, candidate images for each layer of the image to be processed in the target direction are extracted.
[0118] The image to be processed is a stereoscopic image including the target object, such as an MRI image.
[0119] Secondly, the candidate image is preprocessed to obtain the second target layer image.
[0120] The second target layer image includes labeled tomographic images and unlabeled tomographic images, and the preprocessing may include one or more of grayscale processing, noise processing, and contrast processing.
[0121] For example, the candidate image can first be processed into grayscale to obtain a grayscale image; then, the grayscale image can be processed into noise and / or contrast to obtain the second target layer image.
[0122] For example, the noise processing may include denoising the grayscale image using a Gaussian denoising algorithm. Since the watershed algorithm is sensitive to noise and prone to oversegmentation in noisy images, this noise processing can eliminate noise interference and prevent excessive small regions from appearing due to oversegmentation, thereby improving the watershed algorithm's image segmentation performance.
[0123] The contrast processing can include enhancing the contrast of the grayscale image using a contrast-limited adaptive histogram equalization method. This contrast processing improves the local contrast of the grayscale image, thereby further enhancing the watershed algorithm's performance in image segmentation.
[0124] It should be noted that the above-mentioned labeled tomographic images can be obtained by manually annotating the first marker point after preprocessing; or the first marker point can be manually annotated before preprocessing, and then preprocessed.
[0125] In another embodiment of this disclosure, when the image to be processed includes multiple candidate tomographic images of the target object in the target direction, the multiple tomographic images can be sorted according to their positions in the target direction. First, the tomographic image ranked first is marked with a first marker point and then used as the labeled tomographic image. The preset image processing steps are then executed cyclically according to the order of the multiple tomographic images until the last tomographic image is processed. The preset image processing steps may include:
[0126] S21. The next layer of fault image adjacent to the labeled fault image is taken as the aforementioned unlabeled fault image.
[0127] S22. Based on the grayscale values of the labeled fault images and the unlabeled fault images, determine the second label point corresponding to each first label point in the unlabeled fault image.
[0128] S23. Based on the second marker point, determine the target image region where the target object is located in the unmarked fault image.
[0129] S24. If the unlabeled fault image is not the last fault image in the sorting, the unlabeled fault image shall be taken as the new labeled fault image, and the second marker point shall be taken as the new first marker point.
[0130] In this way, after sorting by position, the similarity between adjacent fault images is the highest, and the proximity of the marker points is also higher. Therefore, through this preset image processing step, the marker points of each fault image are more accurate, and the target area image obtained after image processing is also more accurate.
[0131] Figure 3 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this disclosure, as shown below. Figure 3 As shown, the device 300 includes:
[0132] The fault image acquisition module 301 is used to acquire labeled fault images and unlabeled fault images of the target object; wherein, the labeled fault image includes one or more first marker points, which are used to mark the target image region where the target object is located or the background region excluding the target object in the labeled fault image;
[0133] The marker point determination module 302 is used to determine the second marker point corresponding to each first marker point in the unlabeled fault image based on the gray values of the labeled fault image and the unlabeled fault image;
[0134] The target image processing module 303 is used to determine the target image region where the target object is located in the unlabeled tomographic image based on the second marker point.
[0135] Optionally, the marker determination module 302 is used to determine target deformation parameters and target displacement parameters based on the grayscale values of the labeled fault image and the unlabeled fault image; wherein, the target deformation parameter is used to characterize the parameters for shape transformation of the first adjacent region corresponding to the first marker point, and the target displacement parameter is used to characterize the parameters for positional movement of the first marker point; for each first marker point, the position information of the second marker point corresponding to the first marker point in the unlabeled fault image is calculated based on the target deformation parameter, the target displacement parameter, and the position information of the first marker point.
[0136] Optionally, the target image processing module 303 is used to, for each second marker point, use the adjacent position points of the second marker point as the same type of marker point of the second marker point; wherein, both the same type of marker point and the second marker point are used to mark the target area of the unlabeled fault image, or both are used to mark the background area of the unlabeled fault image; and based on the second marker point and the same type of marker point, determine the target image area in the unlabeled fault image where the target object is located.
[0137] Figure 4 This is a schematic diagram of the structure of the second image processing apparatus provided in the embodiments of this disclosure, as shown below. Figure 4 As shown, the device also includes:
[0138] The first marker acquisition module 401 is used to acquire one or more candidate markers marked by the user on the labeled tomographic image; take the labeled tomographic image as the first target layer image, and repeatedly execute the marker adjustment steps until it is determined that the candidate marker meets the preset marking conditions, and then take the candidate marker as the first marker;
[0139] The marker adjustment step includes: determining a first candidate target region in the first target layer image where the target object is located based on the candidate marker; displaying the first candidate target region so that the user can adjust the candidate marker based on the first candidate target region, or the user determines that the first candidate target region is the expected target region; if the user adjusts the candidate marker based on the first candidate target region, the adjusted candidate marker is used as a new candidate marker; if the user determines that the first candidate target region is the expected target region, the candidate marker is determined to meet preset marking conditions.
[0140] Figure 5 This is a schematic diagram of the structure of the third image processing apparatus provided in the embodiments of this disclosure, as shown below. Figure 5 As shown, the device also includes:
[0141] The second marker acquisition module 501 is used to use the second marker as one or more candidate markers on the unlabeled fault image; to use the unlabeled fault image as the first target layer image, to repeatedly execute the marker adjustment step until it is determined that the candidate marker meets the preset marking condition, and to use the candidate marker as a new second marker.
[0142] Figure 6 This is a schematic diagram of the structure of the fourth image processing apparatus provided in the embodiments of this disclosure, as shown below. Figure 6 As shown, the device also includes:
[0143] The image preprocessing module 601 is used to extract candidate images of each layer of the image to be processed in the target direction, wherein the image to be processed is a stereo image including the target object; after preprocessing the candidate images, the second target layer image is obtained, wherein the second target layer image includes the labeled tomographic image and the unlabeled tomographic image.
[0144] Optionally, the image preprocessing module 601 is used to perform grayscale processing on the candidate image to obtain a grayscale image; and to perform noise processing and / or contrast processing on the grayscale image to obtain the second target layer image.
[0145] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.
[0146] Figure 7 This is a block diagram illustrating an electronic device 700 according to an exemplary embodiment. Figure 7 As shown, the electronic device 700 may include a processor 701 and a memory 702. The electronic device 700 may also include one or more of a multimedia component 703, an input / output (I / O) interface 704, and a communication component 705.
[0147] The processor 701 controls the overall operation of the electronic device 700 to complete all or part of the steps in the image processing method described above. The memory 702 stores various types of data to support the operation of the electronic device 700. This data may include, for example, instructions for any application or method operating on the electronic device 700, and application-related data such as contact data, sent and received messages, pictures, audio, and video. The memory 702 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The multimedia component 703 may include a screen and audio components. The screen may be, for example, a touchscreen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in memory 702 or transmitted via communication component 705. The audio component also includes at least one speaker for outputting audio signals. I / O interface 704 provides an interface between processor 701 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual or physical buttons. Communication component 705 is used for wired or wireless communication between the electronic device 700 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, 5G, NB-IoT, eMTC, or other 6G technologies, or combinations thereof, is not limited here. Therefore, the corresponding communication component 705 may include: a Wi-Fi module, a Bluetooth module, an NFC module, etc.
[0148] In an exemplary embodiment, the electronic device 700 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the image processing method described above.
[0149] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, which, when executed by a processor, implement the steps of the image processing method described above. For example, the computer-readable storage medium may be the memory 702 including program instructions described above, which may be executed by the processor 701 of the electronic device 700 to complete the image processing method described above.
[0150] In another exemplary embodiment, a computer program product is also provided, which includes a computer program executable by a programmable device, the computer program having a code portion for performing the above-described image processing method when executed by the programmable device.
[0151] The preferred embodiments of this disclosure have been described in detail above with reference to the accompanying drawings. However, this disclosure is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this disclosure, various simple modifications can be made to the technical solutions of this disclosure, and these simple modifications all fall within the protection scope of this disclosure.
[0152] It should also be noted that the various specific technical features described in the above embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, this disclosure will not describe the various possible combinations separately.
[0153] Furthermore, various different embodiments of this disclosure can be combined in any way, as long as they do not violate the spirit of this disclosure, they should also be regarded as the content disclosed in this disclosure.
Claims
1. An image processing method, characterized by, The method includes: Obtain labeled and unlabeled tomographic images of the target object; wherein, the labeled tomographic image includes one or more first marker points, the first marker points are used to mark the target image region where the target object is located or the background region excluding the target object in the labeled tomographic image, and the labeled and unlabeled tomographic images are tomographic images obtained by slice detection of the target object; Based on the grayscale values of the labeled fault image and the unlabeled fault image, determine the second marker point corresponding to each first marker point in the unlabeled fault image; Based on the second marker point, determine the target image region where the target object is located in the unlabeled tomographic image; The step of determining the second marker point corresponding to each first marker point in the unlabeled fault image based on the grayscale values of the labeled fault image and the unlabeled fault image includes: Based on the grayscale values of the labeled and unlabeled fault images, target deformation parameters and target displacement parameters are determined; wherein, the target deformation parameters are used to characterize the parameters for shape transformation of the first adjacent region corresponding to the first marker point, and the target displacement parameters are used to characterize the parameters for positional movement of the first marker point; for each first marker point, based on the target deformation parameters, target displacement parameters, and the position information of the first marker point, the position information of the second marker point corresponding to the first marker point in the unlabeled fault image is calculated; The step of determining the target image region where the target object is located in the unlabeled tomographic image based on the second marker point includes: For each second marker point, the adjacent points of the second marker point are designated as similar marker points of the second marker point; wherein, both the similar marker points and the second marker point are used to mark the target region of the unlabeled tomographic image, or both are used to mark the background region of the unlabeled tomographic image; based on the second marker point and the similar marker points, the target image region where the target object is located in the unlabeled tomographic image is determined.
2. The method of claim 1, wherein, The first marker point is obtained in the following way: Obtain one or more candidate markers that the user has annotated on the labeled tomographic image; The labeled tomographic image is used as the first target layer image. The marker point adjustment step is executed repeatedly until the candidate marker point is determined to meet the preset marking conditions. The candidate marker point is then used as the first marker point. The marker adjustment step includes: Based on the candidate marker points, a first candidate target region is determined in the first target layer image where the target object is located; The first candidate target region is displayed so that the user can adjust the candidate markers based on the first candidate target region, or the user can determine the first candidate target region as the expected target region; If the user adjusts the candidate markers according to the first candidate target region, the adjusted candidate markers will be used as new candidate markers. If the user determines that the first candidate target area is the expected target area, the candidate marker point is determined to meet the preset marking conditions.
3. The method of claim 2, wherein, Before segmenting the unlabeled tomographic image based on the second marker point, the method further includes: The second marker point is used as one or more candidate marker points on the unlabeled tomographic image; The unlabeled tomographic image is used as the first target layer image, and the marker adjustment step is executed repeatedly until the candidate markers meet the preset marking conditions. The candidate markers are then used as new second markers.
4. The method according to any one of claims 1 to 3, characterized in that, The method further includes: Extract candidate images for each layer of the image to be processed in the target direction, wherein the image to be processed is a stereo image including the target object; The candidate images are preprocessed to obtain the second target layer image, which includes the labeled fault images and the unlabeled fault images.
5. The method of claim 4, wherein, The step of preprocessing the candidate image to obtain the second target layer image includes: After processing the candidate image into grayscale, a grayscale image is obtained; After noise processing and / or contrast processing of the grayscale image, the second target layer image is obtained.
6. An image processing apparatus, characterized in that, The device includes: A tomographic image acquisition module is used to acquire labeled and unlabeled tomographic images of a target object; wherein, the labeled tomographic image includes one or more first marker points, the first marker points are used to mark the target image region where the target object is located or the background region excluding the target object in the labeled tomographic image, and the labeled and unlabeled tomographic images are tomographic images obtained by slice detection of the target object; The marker point determination module is used to determine, based on the grayscale values of the labeled fault image and the unlabeled fault image, a second marker point corresponding to each first marker point in the unlabeled fault image; The target image processing module is used to determine the target image region where the target object is located in the unlabeled tomographic image based on the second marker point; The marker point determination module is used to determine target deformation parameters and target displacement parameters based on the grayscale values of the labeled fault image and the unlabeled fault image; wherein, the target deformation parameters are used to characterize the parameters for shape transformation of the first adjacent region corresponding to the first marker point, and the target displacement parameters are used to characterize the parameters for positional movement of the first marker point; for each first marker point, the position information of the second marker point corresponding to the first marker point in the unlabeled fault image is calculated based on the target deformation parameters, the target displacement parameters, and the position information of the first marker point; The target image processing module is used to, for each second marker point, designate adjacent points of the second marker point as similar marker points of the second marker point; wherein, both the similar marker points and the second marker point are used to mark the target region of the unlabeled tomographic image, or both are used to mark the background region of the unlabeled tomographic image; and determine the target image region where the target object is located in the unlabeled tomographic image based on the second marker point and the similar marker points.
7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method according to any one of claims 1 to 5.
8. An electronic device, characterized in that, include: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of the method according to any one of claims 1 to 5.