A lymphocyte image segmentation method and system based on three-dimensional reconstruction technology

Abstract

A lymphocyte image segmentation method and system based on three-dimensional reconstruction technology, the method comprising: obtaining a slice image sequence and preprocessing the slice image sequence, wherein the slice image sequence is composed of multiple two-dimensional images and is divided into three slice image sequences whose perpendicular bisectors satisfy triangular symmetry. Image registration is performed on the preprocessed slice image sequence to determine the spatial position information of all pixels in the two-dimensional image and calculate the coordinate value of each pixel. Based on the coordinate value of each pixel, the voxel corresponding to each pixel in the three-dimensional space is determined by the voxel model method, and the two-dimensional image is segmented to extract the target area containing lymphocytes in each two-dimensional image. Edge detection is performed on the lymphocytes in the target area to extract the three-dimensional contour of the lymphocytes, and a three-dimensional model of the lymphocytes is constructed based on the three-dimensional contour of the lymphocytes, thereby improving the accuracy and stability of the three-dimensional model.

Description

Technical Field

[0001] The present application relates to the field of medical imaging technology, and in particular to a lymphocyte image segmentation method and system based on three-dimensional reconstruction technology. Background Art

[0002] Pathological diagnosis is the final diagnosis of the disease and is widely used in the diagnosis and medication guidance of tumors. In practice, pathologists observe pathological sections under a microscope, and identifying subtle lesions from complex tissue images is an essential step in diagnosis. This process is not only time-consuming, but also highly subjective. Digital pathology based on whole slide digital scanning technology (wholeslide imaging, WSI) enables us to obtain pathological images with histological characteristics captured at high resolution. In order to effectively obtain useful information in the image, tissue pathology image analysis algorithms have emerged. Since the algorithm's analysis is objective and stable, and its results can be verified by repeatability, the classic cell segmentation algorithm and cell classification algorithm have been widely used in digital pathology analysis platforms. Assisting pathologists in making pathological diagnoses through digital pathology technology has significantly improved the accuracy and efficiency of pathologists' diagnosis.

[0003] At present, the voxel model method has also been widely used in digital pathology technology. The core goal of the voxel model method is to build a high-precision three-dimensional model. However, due to the gaps between slices, traditional interpolation methods often lead to shape breaks or irregular structures in the voxel model, which to some extent affects the overall accuracy and structural integrity of the model. In addition, given that three-dimensional modeling is a continuous and complete process, in order to better meet the requirements of the voxel model method, it is necessary to optimize and improve the image segmentation stage. These improvements should target the gap problem between slices to ensure the continuity and regularity of the voxel model in space, thereby improving the accuracy and stability of the final three-dimensional model. Summary of the invention

[0004] The present application provides a lymphocyte image segmentation method and system based on three-dimensional reconstruction technology, which is used to solve the technical problem of low accuracy and stability of three-dimensional models in the prior art.

[0005] The present invention adopts the following technical solution.

[0006] The first aspect of the present invention discloses a lymphocyte image segmentation method based on three-dimensional reconstruction technology, the method comprising:

[0007] Acquire a slice image sequence and preprocess the slice image sequence, wherein the slice image sequence is composed of a plurality of two-dimensional images and is divided into three slice image sequences whose perpendicular bisectors satisfy triangular symmetry;

[0008] Performing image registration on the preprocessed slice image sequence to determine the spatial position information of all pixels in the two-dimensional image and calculate the coordinate value of each pixel;

[0009] Based on the coordinate value of each pixel point, determine the voxel corresponding to each pixel point in the three-dimensional space by a voxel model method, and perform image segmentation on the two-dimensional image to extract the target area containing lymphocytes in each two-dimensional image;

[0010] Edge detection is performed on the lymphocytes in the target area to extract the three-dimensional contour of the lymphocytes, and a three-dimensional model of the lymphocytes is constructed based on the three-dimensional contour of the lymphocytes.

[0011] Furthermore, the slice image sequence is acquired and preprocessed, and the slice image sequence is composed of a plurality of two-dimensional images, including:

[0012] Acquire the slice image sequence by electronic computer tomography or magnetic resonance imaging technology, and extract the coordinates of the two-dimensional images in the slice image sequence and the index of each two-dimensional image;

[0013] Preprocessing the two-dimensional pixel points corresponding to each two-dimensional image according to the coordinates and indexes of the two-dimensional image in the slice image sequence;

[0014] Among them, the algorithm formula corresponding to the preprocessing is:

[0015] ;

[0016] In the formula, For preprocessing operations, including Gaussian filtering, denoising and histogram equalization, are the coordinates of the two-dimensional image in the slice image sequence, is the index of the two-dimensional image, is the two-dimensional pixel before preprocessing, is the two-dimensional pixel point after preprocessing.

[0017] Furthermore, based on the coordinate value of each pixel point, determining the voxel corresponding to each pixel point in the three-dimensional space by a voxel model method, and performing image segmentation on the two-dimensional image to extract the target area containing lymphocytes in each two-dimensional image, includes:

[0018] Determine the grayscale interval of the two-dimensional pixel point where the lymphocyte is located by setting a threshold, and judge whether each two-dimensional pixel point in the two-dimensional image is within the grayscale interval;

[0019] When the two-dimensional pixel point is within the grayscale interval, determining the area within the grayscale interval as the target area, so as to perform threshold segmentation on the two-dimensional image;

[0020] Among them, the algorithm formula of threshold segmentation is:

[0021] ;

[0022] In the formula, is the grayscale interval, is the two-dimensional segmentation result, are the coordinates of the two-dimensional image in the slice image sequence, is the index of the two-dimensional image, is the preprocessed two-dimensional pixel point, when When it is equal to 1, the corresponding two-dimensional pixel point is in the grayscale interval corresponding to the lymphocyte.

[0023] Further, based on the coordinate value of each pixel point, the voxel corresponding to each pixel point in the three-dimensional space is determined by a voxel model method, and the two-dimensional image is segmented to extract the target area containing lymphocytes in each two-dimensional image, and further includes:

[0024] Selecting a plurality of adjacent pixel points adjacent to a target voxel in the slice image sequence, and taking the center point of the target voxel as the three-dimensional coordinate of the target voxel;

[0025] Based on the plurality of adjacent pixels, the target voxel is marked with three values, the target voxel whose adjacent pixels fall within a set interval is marked with a black value, the target voxel whose adjacent pixels fall outside the set interval is marked with a white value, and the target voxel whose adjacent pixels fall on the boundary of the set interval is marked with a gray value;

[0026] Based on each target voxel marked as a gray value, determine a plurality of target pixel points closest to the target voxel, and calculate the gray value corresponding to the target voxel according to the gray values ​​of the plurality of target pixel points and the maximum gradient value, so as to re-mark the target voxel as a black value or a white value;

[0027] The set interval is a pixel interval corresponding to lymphocytes, and the three-dimensional contour of the lymphocytes is within the range corresponding to the gray value.

[0028] Furthermore, the method further comprises:

[0029] Selecting a plurality of pixels closest to the target pixel, and selecting two pixels from the plurality of pixels to calculate the two-dimensional gradient directions of the two pixels, and then calculating the two-dimensional gradient direction corresponding to the target pixel;

[0030] The two-dimensional gradient direction calculation formula corresponding to the target pixel point is:

[0031] ;

[0032] In the formula, and are the two selected pixels, is the two-dimensional gradient direction corresponding to the target pixel point P, Respectively indicate points and The gray value of Indicate point and The distance value.

[0033] Further, the selecting of a plurality of pixels closest to the target pixel, and selecting two pixels from the plurality of pixels to calculate the two-dimensional gradient directions of the two pixels, and then calculating the two-dimensional gradient direction corresponding to the target pixel, then includes:

[0034] The maximum value of the two-dimensional gradient direction corresponding to the target pixel point is obtained as the gradient maximum value, and the direction of the gradient maximum value is taken as the direction from high to low along the grayscale value of the two pixel points corresponding to the gradient maximum value;

[0035] The three-dimensional gradient direction corresponding to the target voxel is calculated according to the direction of the maximum gradient value. The calculation formula of the three-dimensional gradient direction is:

[0036] ;

[0037] In the formula, is the three-dimensional gradient direction, Respectively represent the directions of the maximum gradient values ​​of the three slice image sequences.

[0038] The second aspect of the present invention discloses a lymphocyte image segmentation system based on three-dimensional reconstruction technology, the system comprising:

[0039] An image preprocessing module, used for acquiring and preprocessing a slice image sequence, wherein the slice image sequence is composed of a plurality of two-dimensional images and is divided into three slice image sequences whose perpendicular bisectors satisfy triangular symmetry;

[0040] A pixel coordinate calculation module is used to perform image registration on the preprocessed slice image sequence to determine the spatial position information of all pixels in the two-dimensional image and calculate the coordinate value of each pixel;

[0041] An image segmentation module, used to determine the voxel corresponding to each pixel point in the three-dimensional space by a voxel model method based on the coordinate value of each pixel point, and perform image segmentation on the two-dimensional image to extract a target area containing lymphocytes in each two-dimensional image;

[0042] The three-dimensional model building module is used to perform edge detection on the lymphocytes in the target area to extract the three-dimensional contours of the lymphocytes, and to build a three-dimensional model of the lymphocytes based on the three-dimensional contours of the lymphocytes.

[0043] The third aspect of the present invention discloses a terminal, including a processor and a storage medium; the characteristics are:

[0044] The storage medium is used to store instructions;

[0045] The processor is used to operate according to the instructions to execute the steps of the method described in the first aspect.

[0046] The fourth aspect of the present invention discloses a computer-readable storage medium having a computer program stored thereon, characterized in that when the program is executed by a processor, the steps of the method described in the first aspect are implemented.

[0047] Compared with the prior art, this application has the following beneficial effects:

[0048] By acquiring a slice image sequence and preprocessing the slice image sequence, the slice image sequence consists of multiple two-dimensional images, which are divided into a first slice image sequence, a second slice image sequence, and a third slice image sequence whose perpendicular bisectors satisfy the triangular symmetry. Then, the preprocessed slice image sequence is image registered to determine the spatial position information of all pixels in the two-dimensional image and calculate the coordinate value of each pixel. Subsequently, based on the coordinate value of each pixel, the voxel corresponding to each pixel in the three-dimensional space is determined by the voxel model method, and the two-dimensional image is segmented to extract the target area containing lymphocytes in each two-dimensional image. Finally, the lymphocytes in the target area are edge detected to extract the three-dimensional contour of the lymphocytes, and a three-dimensional model of the lymphocytes is constructed based on the three-dimensional contour of the lymphocytes. This method couples more accurate voxel units through multi-angle scanning to improve the gap problem between slices, so as to ensure the continuity and regularity of the voxel model in space, thereby improving the accuracy and stability of the final three-dimensional model. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0050] Figure 1It is a flow chart of a lymphocyte image segmentation method based on three-dimensional reconstruction technology;

[0051] Figure 2 is a schematic diagram of the perpendicular bisector of the three slice image sequences;

[0052] Figure 3 It is a schematic diagram of the distribution of voxels and selected pixels.

[0053] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0054] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0055] like Figure 1 As shown, in one embodiment, the lymphocyte image segmentation method based on three-dimensional reconstruction technology provided by the present invention comprises the following steps:

[0056] Step S110, obtaining a slice image sequence and preprocessing the slice image sequence, wherein the slice image sequence is composed of a plurality of two-dimensional images and is divided into three slice image sequences whose perpendicular bisectors satisfy triangular symmetry.

[0057] In some embodiments, the lymphocyte image segmentation method based on three-dimensional reconstruction technology provided by the present invention obtains a slice image sequence and preprocesses the slice image sequence, specifically comprising the following steps:

[0058] Step S111 , obtaining a slice image sequence by means of electronic computer tomography or magnetic resonance imaging technology, and extracting the coordinates of the two-dimensional images in the slice image sequence and the index of each two-dimensional image.

[0059] Step S112 , preprocessing the two-dimensional pixel points corresponding to each two-dimensional image according to the coordinates and indexes of the two-dimensional image in the slice image sequence.

[0060] Among them, the algorithm formula corresponding to the preprocessing is:

[0061] .

[0062] In the formula, For preprocessing operations, including Gaussian filtering, denoising and histogram equalization, are the coordinates of the two-dimensional image in the slice image sequence, is the index of the two-dimensional image, is the two-dimensional pixel before preprocessing, is the two-dimensional pixel point after preprocessing.

[0063] Step S120 , performing image registration on the preprocessed slice image sequence to determine the spatial position information of all pixels in the two-dimensional image and calculate the coordinate value of each pixel.

[0064] Step S130, based on the coordinate value of each pixel point, the voxel corresponding to each pixel point in the three-dimensional space is determined by the voxel model method, and the two-dimensional image is segmented to extract the target area containing lymphocytes in each two-dimensional image.

[0065] In some embodiments, the lymphocyte image segmentation method based on three-dimensional reconstruction technology provided by the present invention determines the voxel corresponding to each pixel in three-dimensional space by voxel model method based on the coordinate value of each pixel, and performs image segmentation on the two-dimensional image to extract the target area containing lymphocytes in each two-dimensional image, which specifically includes the following steps:

[0066] Step S131, determining the grayscale interval of the two-dimensional pixel where the lymphocytes are located by setting a threshold, and judging whether each two-dimensional pixel in the two-dimensional image is within the grayscale interval.

[0067] Step S132, when the two-dimensional pixel point is within the grayscale interval, determining the area within the grayscale interval as the target area, so as to perform threshold segmentation on the two-dimensional image.

[0068] Among them, the algorithm formula of threshold segmentation is:

[0069] .

[0070] In the formula, is the grayscale interval, is the two-dimensional segmentation result, are the coordinates of the two-dimensional image in the slice image sequence, is the index of the two-dimensional image, is the preprocessed two-dimensional pixel point, when When it is equal to 1, the corresponding two-dimensional pixel point is in the grayscale interval corresponding to the lymphocyte.

[0071] In some embodiments, the lymphocyte image segmentation method based on three-dimensional reconstruction technology provided by the present invention determines the voxel corresponding to each pixel in three-dimensional space by voxel model method based on the coordinate value of each pixel, and performs image segmentation on the two-dimensional image to extract the target area containing lymphocytes in each two-dimensional image, and specifically includes the following steps:

[0072] Step S133 : selecting a plurality of adjacent pixel points in the slice image sequence that are adjacent to the target voxel, and taking the center point of the target voxel as the three-dimensional coordinate of the target voxel.

[0073] Step S134, performing three-value marking on the target voxel based on multiple adjacent pixels, marking the target voxel whose adjacent pixels fall within the set interval as a black value, marking the target voxel whose adjacent pixels fall outside the set interval as a white value, and marking the target voxel whose adjacent pixels fall on the boundary of the set interval as a gray value.

[0074] Step S135, based on each target voxel marked as a gray value, determine multiple target pixel points closest to the target voxel, and calculate the gray value corresponding to the target voxel according to the gray values ​​of the multiple target pixel points and the maximum gradient value, so as to re-mark the target voxel as a black value or a white value.

[0075] The set interval is a pixel interval corresponding to lymphocytes, and the three-dimensional contour of lymphocytes is within the range corresponding to the gray value.

[0076] In some embodiments, the lymphocyte image segmentation method based on three-dimensional reconstruction technology provided by the present invention further includes the following steps:

[0077] Step S210, selecting multiple pixel points that are closest to the target pixel point, and selecting two pixel points from the multiple pixel points to calculate the two-dimensional gradient directions of the two pixel points, and then calculating the two-dimensional gradient direction corresponding to the target pixel point.

[0078] Among them, the calculation formula of the two-dimensional gradient direction corresponding to the target pixel point is:

[0079] .

[0080] In the formula, and are the two selected pixels, is the two-dimensional gradient direction corresponding to the target pixel point P, Respectively indicate points and The gray value of Indicate point and The distance value.

[0081] Step S220, obtaining the maximum value of the two-dimensional gradient direction corresponding to the target pixel as the gradient maximum value, and taking the two pixel points corresponding to the gradient maximum value along the grayscale value from high to low as the direction of the gradient maximum value.

[0082] Step S230, the three-dimensional gradient direction corresponding to the target voxel is calculated according to the direction of the maximum gradient value. The calculation formula of the three-dimensional gradient direction is:

[0083] .

[0084] In the formula, is the three-dimensional gradient direction, Represent the directions of the maximum gradient values ​​of the three slice image sequences respectively.

[0085] In this embodiment, the maximum gradient direction of one of the slice image sequences is taken as For example, the expression is:

[0086]

[0087] For all voxels marked as gray values, select the 3D gradient direction The maximum value is combined with the idea of ​​regional growth and gradual adjustment to determine the three-dimensional contour. Specifically, the three-dimensional gradient direction The maximum value is used as the initial point, and the three-dimensional gradient direction of its neighbors is continuously selected. The maximum value of is used as the connection point to form a three-dimensional contour.

[0088] It should be noted that after the initial three-dimensional contour is constructed, the stepwise adjustment method can be continuously used to discard a certain pixel point in the existing three-dimensional contour and select its adjacent pixel points for comparison to ensure the correctness of the maximum value of the three-dimensional gradient direction of the adjacent pixel points. Abandoning a certain pixel point in the existing three-dimensional contour and selecting its adjacent pixel points can be reselected (i.e., adjacent pixel points) based on the Sobel edge detection operator during the smoothing filtering process.

[0089] Step S140 , performing edge detection on the lymphocytes in the target area to extract the three-dimensional contour of the lymphocytes, and constructing a three-dimensional model of the lymphocytes based on the three-dimensional contour of the lymphocytes.

[0090] In some embodiments, the lymphocyte image segmentation method based on three-dimensional reconstruction technology provided by the present invention performs edge detection on lymphocytes in the target area to extract the three-dimensional contour of the lymphocytes, and constructs a three-dimensional model of the lymphocytes based on the three-dimensional contour of the lymphocytes, specifically comprising the following steps:

[0091] Step S141 , performing edge detection on lymphocytes in the target area by using an edge detection algorithm of a Sobel operator or a Canny operator to extract the three-dimensional contours of the lymphocytes.

[0092] Step S142, dividing the three-dimensional space into multiple volume units by voxel model method, each volume unit is used to represent a pixel in a two-dimensional image, multiple volume units are used to represent the three-dimensional structure of lymphocytes, and the three-dimensional pixel points corresponding to each volume unit are determined by an interpolation formula.

[0093] The interpolation formula is:

[0094] ;

[0095] In the formula, The spatial coordinates are The three-dimensional pixels, is the index of the two-dimensional image, is the grayscale interval, is the weight coefficient.

[0096] The formula for determining the weight coefficient is:

[0097] .

[0098] In the formula, and Respectively Two-dimensional image The coordinate values ​​and Two-dimensional image The numerical value of the coordinate.

[0099] The above-mentioned lymphocyte image segmentation method based on three-dimensional reconstruction technology obtains a slice image sequence and preprocesses the slice image sequence. The slice image sequence consists of multiple two-dimensional images and is divided into a first slice image sequence, a second slice image sequence and a third slice image sequence whose perpendicular bisectors satisfy triangular symmetry. Then, the preprocessed slice image sequence is image registered to determine the spatial position information of all pixels in the two-dimensional image and calculate the coordinate value of each pixel. Subsequently, based on the coordinate value of each pixel, the voxel corresponding to each pixel in the three-dimensional space is determined by the voxel model method, and the two-dimensional image is segmented to extract the target area containing lymphocytes in each two-dimensional image. Finally, the lymphocytes in the target area are edge detected to extract the three-dimensional contour of the lymphocytes, and a three-dimensional model of the lymphocytes is constructed based on the three-dimensional contour of the lymphocytes. The method couples more accurate voxel units by multi-angle scanning to improve the gap problem between slices, so as to ensure the continuity and regularity of the voxel model in space, thereby improving the accuracy and stability of the final three-dimensional model.

[0100] In a specific embodiment, the present invention provides a lymphocyte image segmentation method based on three-dimensional reconstruction technology.

[0101] Typically, the three-dimensional reconstruction method of lymphocyte images may include steps S1 to S4.

[0102] Step S1, obtaining a slice image sequence.

[0103] In some embodiments, the original slice image sequence can be acquired by electronic computed tomography (CT) or magnetic resonance imaging (MRI). , the slice image sequence is composed of multiple two-dimensional images. The slice image sequence of step S1 It can be described by the following formula:

[0104]

[0105] in, Represents preprocessing operations, including Gaussian filtering, denoising, histogram equalization, etc. and represents the coordinates of a two-dimensional image, Represents the index of multiple two-dimensional images, and They represent the original two-dimensional pixel points and the preprocessed two-dimensional pixel points respectively.

[0106] Step S2, performing image segmentation on the two-dimensional images in the slice image sequence.

[0107] Specifically, two-dimensional segmentation of lymphocytes is performed on each two-dimensional image. This step can use threshold segmentation, region growing or a deep learning-based segmentation algorithm to extract the region of interest (e.g., lymphocytes) in each slice.

[0108] In this embodiment, taking threshold segmentation as an example, the grayscale interval of the two-dimensional pixel point where the lymphocyte is located can be determined by a preset threshold. , and perform threshold segmentation by judging whether each two-dimensional pixel is in the grayscale interval. Two-dimensional segmentation result It can be shown as follows:

[0109]

[0110] It is understandable that when When it is equal to 1, it indicates that the two-dimensional pixel belongs to the lymphocyte area.

[0111] Step S3, extracting the three-dimensional contour.

[0112] It is not difficult to understand that the simple and crude method of threshold segmentation in step S2 may cause the final lymphocyte region to be fragmented and discrete. Therefore, in actual segmentation, it is necessary to combine other methods, such as region growing methods or edge detection algorithms such as Sobel operator and Canny operator to prevent discretization from occurring, thereby extracting a smoother three-dimensional contour.

[0113] In addition, before extracting the three-dimensional contour, image registration, interpolation processing and other operations are required. Among them, image registration refers to the positioning and registration of multiple two-dimensional images, that is, determining the position of all pixels in space. At present, it is mainly divided into hard positioning and soft positioning for positioning and registration. Hard positioning can be performed by laser or microelectrode perforation, while soft positioning can be performed by computer positioning programmed by language. The positioning accuracy is high, but the disadvantage is that the workload of human-computer interaction is very large. Interpolation processing is used to fill the gaps between two-dimensional images in order to generate smooth three-dimensional contours.

[0114] Step S4, realizing three-dimensional reconstruction based on the three-dimensional contour.

[0115] In some embodiments, three-dimensional reconstruction can be achieved based on methods such as wireframe modeling, stereogram pairing, and grayscale surface modeling. Three-dimensional models can also be generated by combining algorithms such as matching cubes.

[0116] In this embodiment, considering the special structure of lymphocyte images, the voxel model method is generally preferred. The voxel model method represents the three-dimensional structure of an object (two-dimensional pixel point) by dividing the three-dimensional space into a series of volume units (called voxels). Each voxel can be regarded as a small cube in the three-dimensional space, similar to a pixel in a two-dimensional image, but the voxel has a volume attribute.

[0117] It should be noted that voxel is the abbreviation of volume pixel, which is the smallest unit in three-dimensional space with a specific size and position. Each voxel stores an attribute value, such as grayscale value, density, color, etc., which represents the attribute parameters of the area covered by the voxel.

[0118] Due to the fine-grained representation of voxels, voxel models usually require a lot of storage space, and processing voxel models requires a lot of computing resources, especially in high-resolution data. For voxel models, the generation and interpolation of voxels are the most critical steps, and the interpolation formula is shown below:

[0119]

[0120] in, Represents the spatial coordinates as The three-dimensional pixels, is the weight coefficient.

[0121] In addition, the weight coefficient The value depends on and , which can be determined based on the following formula:

[0122]

[0123] in, and Respectively represent The value of the z coordinate of the two-dimensional image and the The z-coordinate value of the two-dimensional image.

[0124] The core goal of the voxel model method is to build a high-precision three-dimensional model. However, due to the gaps between slices, traditional interpolation methods often lead to shape breaks or irregular structures in the voxel model. This problem seriously affects the overall accuracy and structural integrity of the model. Given that three-dimensional modeling is a continuous and complete process, in order to better meet the requirements of the voxel model method, it is necessary to optimize and improve the image segmentation stage. These improvements should target the gap problem between slices to ensure the continuity and regularity of the voxel model in space, thereby improving the accuracy and stability of the final three-dimensional model.

[0125] Based on this, in this embodiment, a multi-angle scanning method can be adopted to couple to obtain a more accurate voxel unit, including steps 1 to 4.

[0126] Step 1: Acquire a first slice image sequence, a second slice image sequence, and a third slice image sequence.

[0127] In some embodiments, for computational convenience, the perpendicular bisectors of the three slice image sequences should ensure that the triangle symmetry is satisfied. Figure 2 As shown, straight lines a, b, and c can be perpendicular bisectors of the three slice image sequences, which intersect at the same point M on straight line o, and satisfy the angles between the three perpendicular bisectors and straight line o. The three points A, B and C are equal, and the equilateral triangle formed by the three points A, B and C is perpendicular to the line o. Among them, A, B and C are three points on the line a, line b and line c respectively.

[0128] In addition, in step 1, each slice image sequence needs to be preprocessed, which will not be described in detail here.

[0129] Step 2: perform image registration on each slice image sequence and calculate the coordinate value of each pixel point.

[0130] Specifically, the coordinate value of the pixel point It can be expressed as ,in, , The indices corresponding to the first slice image sequence, the second slice image sequence and the third slice image sequence respectively, Indicates that the pixel is at index The two-dimensional coordinate value on the two-dimensional image of Indicates the coordinate value of the pixel in three-dimensional space.

[0131] Step 3: Determine the voxel in the three-dimensional space based on the coordinate value of each pixel point.

[0132] In this embodiment, in order to facilitate the calculation of the adjacent pixel points corresponding to each voxel, the straight line o can be set as the z-axis in the three-dimensional space, and the x-axis and y-axis are not limited, thereby determining the coordinate system of the three-dimensional space. Step 3 Specifically for each voxel, execute steps 3.1 to 3.3:

[0133] Step 3.1, determine the adjacent pixels corresponding to the voxel.

[0134] Usually, for each voxel, one of the most adjacent pixels in three slice image sequences, a total of three pixels, can be selected as the adjacent pixels of the voxel, wherein the center point of the voxel is used as the three-dimensional coordinate of the voxel.

[0135] Step 3.2: Perform preliminary three-value labeling on voxels based on adjacent pixels.

[0136] If all the neighboring pixels corresponding to the voxel fall within the preset pixel interval, they are marked as black values; if all the neighboring pixels corresponding to the voxel fall outside the preset pixel interval, they are marked as white values; otherwise, they are marked as gray values.

[0137] The preset pixel interval is the pixel interval corresponding to the lymphocytes, and the three-dimensional contour of the lymphocytes is generally within the range of gray values.

[0138] Step 3.3, determine the three-dimensional contour based on the gray value voxels.

[0139] For each voxel marked as gray value, determine the 6 pixels closest to it. Figure 3 As shown, the center point of the voxel is point R, which is located between the two two-dimensional images of the first slice image sequence. Therefore, the two pixels closest to the center point R of the voxel and the two-dimensional image may be point P and point Q.

[0140] Secondly, it is necessary to determine the direction of the maximum value of the grayscale gradient between points P and Q, such as Figure 3 In order to make the direction of the maximum gradient smoother, in some embodiments, determining the direction of the maximum gradient of the grayscale of point P specifically includes steps A1 to A3.

[0141] Step A1, select the N (N is usually 16) points closest to point P.

[0142] Step A2: Randomly select 2 points from N points and , calculate its two-dimensional gradient direction.

[0143] The two-dimensional gradient direction corresponding to point P As shown below:

[0144]

[0145] in, Respectively indicate points and The gray value of Indicate point and The distance value.

[0146] Step A3, selecting the maximum value of the two-dimensional gradient direction as the gradient maximum value; taking the two points corresponding to the maximum value of the two-dimensional gradient direction, along the gray value from high to low, as the direction of the gradient maximum value.

[0147] It should be noted that the maximum direction of the gray gradient of point Q is determined Similar to this, I will not go into details here.

[0148] In this embodiment, similar points P and Q can also be found in the second slice image sequence and the third slice image sequence. Based on this, the three-dimensional gradient direction corresponding to the voxel can be calculated according to the direction of the maximum gradient value. , as shown below:

[0149]

[0150] in, Respectively represent the gradient maximum directions of the first slice image sequence, the second slice image sequence and the third slice image sequence.

[0151] Taking the first slice image sequence as an example, it can be expressed as follows:

[0152]

[0153] For all voxels marked as gray values, select the 3D gradient direction The maximum value is combined with the idea of ​​regional growth and gradual adjustment to determine the three-dimensional contour. Specifically, the three-dimensional gradient direction The maximum value is used as the initial point, and the three-dimensional gradient direction of its neighbors is continuously selected. The maximum value of is used as the connection point to form a 3D contour. After the preliminary 3D contour is constructed, the stepwise adjustment method can be continuously used to discard a certain point in the existing 3D contour and select its neighboring points for comparison to ensure that the 3D gradient direction of its neighbors is selected. The correctness of the maximum value of the "maximum value". Abandoning a certain point in the existing three-dimensional contour and selecting its neighboring points can be specifically based on the Sobel edge detection operator and reselected (i.e., neighboring points) during the smoothing filtering process.

[0154] It is not difficult to understand that, on the one hand, if the angle The smaller the value of , the higher the accuracy of the pixel value of each slice image sequence, but the cost is that the two-dimensional gradient direction of the grayscale is not obvious. On the other hand, if the angle The larger the value of , the more obvious the two-dimensional gradient direction of the grayscale will be, which will lead to a more accurate grayscale value corresponding to the voxel. Therefore, in the embodiment of the present invention, the angle The value range can be .

[0155] Step 4: Perform 3D reconstruction based on the 3D contour.

[0156] The lymphocyte image segmentation system based on three-dimensional reconstruction technology provided by the present invention is described below. The lymphocyte image segmentation system based on three-dimensional reconstruction technology described below and the lymphocyte image segmentation method based on three-dimensional reconstruction technology described above can be referred to each other.

[0157] In one embodiment, the lymphocyte image segmentation system based on three-dimensional reconstruction technology provided by the present invention includes an image preprocessing module, a pixel coordinate calculation module, an image segmentation module and a three-dimensional model establishment module.

[0158] The image preprocessing module is used to obtain and preprocess a slice image sequence, wherein the slice image sequence is composed of a plurality of two-dimensional images and is divided into three slice image sequences whose perpendicular bisectors satisfy triangular symmetry.

[0159] The pixel coordinate calculation module is used to perform image registration on the preprocessed slice image sequence to determine the spatial position information of all pixels in the two-dimensional image and calculate the coordinate value of each pixel.

[0160] The image segmentation module is used to determine the voxel corresponding to each pixel in three-dimensional space through the voxel model method based on the coordinate value of each pixel, and perform image segmentation on the two-dimensional image to extract the target area containing lymphocytes in each two-dimensional image.

[0161] The three-dimensional model building module is used to perform edge detection on lymphocytes in the target area to extract the three-dimensional contours of the lymphocytes, and to build a three-dimensional model of the lymphocytes based on the three-dimensional contours of the lymphocytes.

[0162] In this embodiment, the lymphocyte image segmentation system based on three-dimensional reconstruction technology provided by the present invention, the image preprocessing module is specifically used for:

[0163] A slice image sequence is acquired by electronic computer tomography or magnetic resonance imaging technology, and the coordinates of the two-dimensional images in the slice image sequence and the index of each two-dimensional image are extracted.

[0164] The two-dimensional pixel points corresponding to each two-dimensional image are preprocessed according to the coordinates and indexes of the two-dimensional image in the slice image sequence.

[0165] Among them, the algorithm formula corresponding to the preprocessing is:

[0166] .

[0167] In the formula, For preprocessing operations, including Gaussian filtering, denoising and histogram equalization, are the coordinates of the two-dimensional image in the slice image sequence, is the index of the two-dimensional image, is the two-dimensional pixel before preprocessing, is the two-dimensional pixel point after preprocessing.

[0168] In this embodiment, the lymphocyte image segmentation system based on three-dimensional reconstruction technology provided by the present invention, the image segmentation module is specifically used for:

[0169] The grayscale interval of the two-dimensional pixel point where the lymphocyte is located is determined by setting a threshold, and it is judged whether each two-dimensional pixel point in the two-dimensional image is within the grayscale interval.

[0170] When a two-dimensional pixel point is within a grayscale interval, the area within the grayscale interval is determined as a target area to perform threshold segmentation on the two-dimensional image.

[0171] Among them, the algorithm formula of threshold segmentation is:

[0172] .

[0173] In the formula, is the grayscale interval, is the two-dimensional segmentation result, are the coordinates of the two-dimensional image in the slice image sequence, is the index of the two-dimensional image, is the preprocessed two-dimensional pixel point, when When it is equal to 1, the corresponding two-dimensional pixel point is in the grayscale interval corresponding to the lymphocyte.

[0174] In this embodiment, the lymphocyte image segmentation system based on three-dimensional reconstruction technology provided by the present invention, the image segmentation module is specifically used for:

[0175] A plurality of adjacent pixel points adjacent to the target voxel in the slice image sequence are selected, and the center point of the target voxel is used as the three-dimensional coordinate of the target voxel.

[0176] The target voxels are marked with three values ​​based on multiple adjacent pixels. The target voxels whose adjacent pixels fall within the set interval are marked as black values, the target voxels whose adjacent pixels fall outside the set interval are marked as white values, and the target voxels whose adjacent pixels fall on the boundary of the set interval are marked as gray values.

[0177] Based on each target voxel marked as a gray value, multiple target pixels closest to the target voxel are determined, and the gray value corresponding to the target voxel is calculated according to the gray values ​​of the multiple target pixels and the maximum gradient value, so as to re-mark the target voxel as a black value or a white value.

[0178] The set interval is a pixel interval corresponding to lymphocytes, and the three-dimensional contour of lymphocytes is within the range corresponding to the gray value.

[0179] In this embodiment, the lymphocyte image segmentation system based on three-dimensional reconstruction technology provided by the present invention further includes a two-dimensional gradient direction calculation module for:

[0180] Select multiple pixels closest to the target pixel, and select two pixels from the multiple pixels to calculate the two-dimensional gradient directions of the two pixels, and then calculate the two-dimensional gradient direction corresponding to the target pixel.

[0181] Among them, the calculation formula of the two-dimensional gradient direction corresponding to the target pixel point is:

[0182] .

[0183] In the formula, and are the two selected pixels, is the two-dimensional gradient direction corresponding to the target pixel point P, Respectively indicate points and The gray value of Indicate point and The distance value.

[0184] In this embodiment, the lymphocyte image segmentation system based on the three-dimensional reconstruction technology provided by the present invention further includes a three-dimensional gradient direction calculation module for:

[0185] The maximum value of the two-dimensional gradient direction corresponding to the target pixel is obtained as the gradient maximum value, and the two pixel points corresponding to the gradient maximum value are taken as the direction of the gradient maximum value along the grayscale value from high to low.

[0186] The three-dimensional gradient direction corresponding to the target voxel is calculated according to the direction of the maximum gradient value. The calculation formula of the three-dimensional gradient direction is:

[0187] .

[0188] In the formula, is the three-dimensional gradient direction, Represent the directions of the maximum gradient values ​​of the three slice image sequences respectively.

[0189] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0190] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A lymphocyte image segmentation method based on three-dimensional reconstruction technology, characterized in that: The method comprises: Acquire a slice image sequence and preprocess the slice image sequence, wherein the slice image sequence is composed of a plurality of two-dimensional images and is divided into three slice image sequences whose perpendicular bisectors satisfy triangular symmetry; Performing image registration on the preprocessed slice image sequence to determine the spatial position information of all pixels in the two-dimensional image and calculate the coordinate value of each pixel; Based on the coordinate value of each pixel point, determine the voxel corresponding to each pixel point in the three-dimensional space by a voxel model method, and perform image segmentation on the two-dimensional image to extract the target area containing lymphocytes in each two-dimensional image; Edge detection is performed on the lymphocytes in the target area to extract the three-dimensional contour of the lymphocytes, and a three-dimensional model of the lymphocytes is constructed based on the three-dimensional contour of the lymphocytes.

2. The lymphocyte image segmentation method based on three-dimensional reconstruction technology according to claim 1, characterized in that: The step of acquiring a slice image sequence and preprocessing the slice image sequence, wherein the slice image sequence is composed of a plurality of two-dimensional images, includes: Acquire the slice image sequence by electronic computer tomography or magnetic resonance imaging technology, and extract the coordinates of the two-dimensional images in the slice image sequence and the index of each two-dimensional image; Preprocessing the two-dimensional pixel points corresponding to each two-dimensional image according to the coordinates and indexes of the two-dimensional image in the slice image sequence; Among them, the algorithm formula corresponding to the preprocessing is: ； In the formula, For preprocessing operations, including Gaussian filtering, denoising and histogram equalization, are the coordinates of the two-dimensional image in the slice image sequence, is the index of the two-dimensional image, is the two-dimensional pixel before preprocessing, is the two-dimensional pixel point after preprocessing.

3. The lymphocyte image segmentation method based on three-dimensional reconstruction technology according to claim 2, characterized in that: The method of determining the voxel corresponding to each pixel in the three-dimensional space by a voxel model method based on the coordinate value of each pixel, and performing image segmentation on the two-dimensional image to extract a target area containing lymphocytes in each two-dimensional image includes: Determine the grayscale interval of the two-dimensional pixel point where the lymphocyte is located by setting a threshold, and judge whether each two-dimensional pixel point in the two-dimensional image is within the grayscale interval; When the two-dimensional pixel point is within the grayscale interval, determining the area within the grayscale interval as the target area, so as to perform threshold segmentation on the two-dimensional image; Among them, the algorithm formula of threshold segmentation is: ； In the formula, is the grayscale interval, is the two-dimensional segmentation result, are the coordinates of the two-dimensional image in the slice image sequence, is the index of the two-dimensional image, is the preprocessed two-dimensional pixel point, when When it is equal to 1, the corresponding two-dimensional pixel point is in the grayscale interval corresponding to the lymphocyte.

4. The lymphocyte image segmentation method based on three-dimensional reconstruction technology according to claim 3, characterized in that: The method further comprises: determining the voxel corresponding to each pixel in the three-dimensional space by a voxel model method based on the coordinate value of each pixel, and performing image segmentation on the two-dimensional image to extract the target area containing lymphocytes in each two-dimensional image; Selecting a plurality of adjacent pixel points adjacent to a target voxel in the slice image sequence, and taking the center point of the target voxel as the three-dimensional coordinate of the target voxel; Based on the plurality of adjacent pixels, the target voxel is marked with three values, the target voxel whose adjacent pixels fall within a set interval is marked with a black value, the target voxel whose adjacent pixels fall outside the set interval is marked with a white value, and the target voxel whose adjacent pixels fall on the boundary of the set interval is marked with a gray value; Based on each target voxel marked as a gray value, determine a plurality of target pixel points closest to the target voxel, and calculate the gray value corresponding to the target voxel according to the gray values ​​of the plurality of target pixel points and the maximum gradient value, so as to re-mark the target voxel as a black value or a white value; The set interval is a pixel interval corresponding to lymphocytes, and the three-dimensional contour of the lymphocytes is within the range corresponding to the gray value.

5. The lymphocyte image segmentation method based on three-dimensional reconstruction technology according to claim 4, characterized in that: The method further comprises: Selecting a plurality of pixels closest to the target pixel, and selecting two pixels from the plurality of pixels to calculate the two-dimensional gradient directions of the two pixels, and then calculating the two-dimensional gradient direction corresponding to the target pixel; The two-dimensional gradient direction calculation formula corresponding to the target pixel point is: ； In the formula, and are the two selected pixels, is the two-dimensional gradient direction corresponding to the target pixel point P, Respectively indicate points and The gray value of Indicate point and The distance value.

6. The lymphocyte image segmentation method based on three-dimensional reconstruction technology according to claim 5, characterized in that: The step of selecting a plurality of pixels closest to the target pixel, selecting two pixels from the plurality of pixels to calculate the two-dimensional gradient directions of the two pixels, and then calculating the two-dimensional gradient direction corresponding to the target pixel, further comprises: The maximum value of the two-dimensional gradient direction corresponding to the target pixel point is obtained as the gradient maximum value, and the direction of the gradient maximum value is taken as the direction from high to low along the grayscale value of the two pixel points corresponding to the gradient maximum value; The three-dimensional gradient direction corresponding to the target voxel is calculated according to the direction of the maximum gradient value. The calculation formula of the three-dimensional gradient direction is: ； In the formula, is the three-dimensional gradient direction, Respectively represent the directions of the maximum gradient values ​​of the three slice image sequences.

7. A lymphocyte image segmentation system based on three-dimensional reconstruction technology, used to execute any of the methods of claims 1-6, characterized in that: The system comprises: An image preprocessing module, used for acquiring and preprocessing a slice image sequence, wherein the slice image sequence is composed of a plurality of two-dimensional images and is divided into three slice image sequences whose perpendicular bisectors satisfy triangular symmetry; A pixel coordinate calculation module is used to perform image registration on the preprocessed slice image sequence to determine the spatial position information of all pixels in the two-dimensional image and calculate the coordinate value of each pixel; An image segmentation module, used to determine the voxel corresponding to each pixel point in the three-dimensional space by a voxel model method based on the coordinate value of each pixel point, and perform image segmentation on the two-dimensional image to extract a target area containing lymphocytes in each two-dimensional image; The three-dimensional model building module is used to perform edge detection on the lymphocytes in the target area to extract the three-dimensional contours of the lymphocytes, and to build a three-dimensional model of the lymphocytes based on the three-dimensional contours of the lymphocytes.

8. A terminal comprising a processor and a storage medium; characterized in that: The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the method according to any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images