Image Processing Method and Apparatus, Electronic Device, and Storage Medium

Abstract

The present disclosure relates to an image processing method and apparatus, an electronic device, and a storage medium. The method includes: performing image segmentation on a vertebra image to be segmented to obtain a binary segmentation result corresponding to the vertebra image to be segmented; performing feature encoding on the vertebra image to be segmented to obtain a pixel embedding vector corresponding to each pixel point in the vertebra image to be segmented; determining a pixel embedding vector corresponding to the vertebra region in the vertebra image to be segmented according to the binary segmentation result and the pixel embedding vectors corresponding to the respective pixel points; and clustering the pixel embedding vectors corresponding to the vertebra region to obtain a vertebra segmentation result corresponding to the vertebra image to be segmented. The embodiments of the present disclosure can reduce the probability of adjacent vertebrae being misclassified and improve the accuracy of vertebra segmentation.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular, to an image processing method and apparatus, an electronic device, and a storage medium. Background Art

[0002] Vertebrae are an important part of the human skeletal system. While maintaining and supporting the body structure and organs, they also protect the central nervous system from mechanical shock damage. However, due to factors such as osteoporosis and external forces, vertebral fractures are likely to occur. In the diagnosis of vertebral fractures, doctors need to give the specific location of the fractured vertebra based on medical images of the chest (for example, chest CT images). In related technologies, neural networks are directly used for vertebral segmentation. However, due to the high similarity in appearance between adjacent vertebrae, missegmentation is likely to occur between adjacent vertebrae, resulting in low accuracy of vertebral segmentation. Summary of the Invention

[0003] The present disclosure provides a technical solution for an image processing method and apparatus, an electronic device, and a storage medium.

[0004] According to one aspect of the present disclosure, an image processing method is provided, including: performing image segmentation on a vertebra image to be segmented to obtain a binary segmentation result corresponding to the vertebra image to be segmented; performing feature encoding on the vertebra image to be segmented to obtain a pixel embedding vector corresponding to each pixel point in the vertebra image to be segmented; determining a pixel embedding vector corresponding to the vertebra region in the vertebra image to be segmented according to the binary segmentation result and the pixel embedding vectors corresponding to the respective pixel points; and performing clustering on the pixel embedding vectors corresponding to the vertebra region to obtain a vertebra segmentation result corresponding to the vertebra image to be segmented.

[0005] In a possible implementation, the method further includes: acquiring an original chest scan image; performing image preprocessing on the original chest scan image to obtain an initial vertebra image; performing convex hull segmentation on the initial vertebra image to obtain a vertebra convex hull region in the initial vertebra image; and cropping the initial vertebra image according to the vertebra convex hull region to obtain the vertebra image to be segmented.

[0006] In a possible implementation, the method further includes: performing position encoding on pixel points in the vertebra image to be segmented to obtain a position encoding image.

[0007] In a possible implementation, performing image segmentation on the vertebra image to be segmented to obtain a binary segmentation result corresponding to the vertebra image to be segmented includes: performing image segmentation on the vertebra image to be segmented based on the position encoding image to obtain the binary segmentation result.

[0008] In a possible implementation, the feature encoding of the vertebra image to be segmented to obtain the pixel embedding vectors corresponding to the pixel points in the vertebra image to be segmented includes: based on the position encoding image, performing feature encoding on the vertebra image to be segmented to obtain the pixel embedding vectors corresponding to the pixel points.

[0009] In a possible implementation, the method further includes: filtering the non-vertebra regions in the binary segmentation result according to the convex hull region of the vertebra.

[0010] In a possible implementation, the vertebra segmentation result includes multiple vertebrae in the vertebra image to be segmented; the method further includes: obtaining a rib marking result corresponding to the vertebra region in the vertebra image to be segmented, where the rib marking result includes multiple ribs and a rib identifier corresponding to each rib; determining a target rib-vertebra matching result according to the multiple vertebrae and the multiple ribs, where the target rib-vertebra matching result includes at least one rib-vertebra matching pair; and determining a vertebra identifier corresponding to each vertebra according to the rib identifier corresponding to each rib and the target rib-vertebra matching result.

[0011] In a possible implementation, the determining the target rib-vertebra matching result according to the multiple vertebrae and the multiple ribs includes: for any one vertebra, determining a candidate rib-vertebra matching result corresponding to the vertebra, where the candidate rib-vertebra matching result includes at least one rib-vertebra matching pair; determining a matching degree corresponding to each candidate rib-vertebra matching result according to the spatial geometric relationship of at least one rib-vertebra matching pair included in each candidate rib-vertebra matching result; and determining the candidate rib-vertebra matching result whose matching degree meets the preset matching condition as the target rib-vertebra matching result.

[0012] In a possible implementation, the determining the matching degree corresponding to each candidate rib-vertebra matching result according to the spatial geometric relationship of at least one rib-vertebra matching pair included in each candidate rib-vertebra matching result includes: determining the rib endpoint closest to the vertebra in each rib according to the rib marking result; determining the centroid of each vertebra corresponding to each vertebra according to the vertebra segmentation result, and the tangent vector corresponding to each vertebral centroid; determining the average cosine distance corresponding to each candidate rib-vertebra matching result according to the rib endpoint corresponding to each rib, the centroid of each vertebra corresponding to each vertebra, and the tangent vector corresponding to each vertebral centroid; and determining the average cosine distance corresponding to each candidate rib-vertebra matching result as the matching degree corresponding to each candidate rib-vertebra matching result.

[0013] In a possible implementation, determining the candidate rib-vertebra matching result whose matching degree meets the preset matching condition as the target rib-vertebra matching result includes: determining whether there is a set of target rib-vertebra matching results according to the average cosine distance corresponding to each candidate rib-vertebra matching result, where the average cosine distance corresponding to each candidate rib-vertebra matching result in the set of target rib-vertebra matching results is greater than 0; in the case where there is a set of target rib-vertebra matching results, determining the candidate rib-vertebra matching result corresponding to the smallest average cosine distance value in the set of target rib-vertebra matching results as the target rib-vertebra matching result.

[0014] In a possible implementation, the method further includes: in the case where there is no set of target rib-vertebra matching results, determining the candidate rib-vertebra matching result corresponding to the largest average cosine distance value as the target rib-vertebra matching result.

[0015] According to one aspect of the present disclosure, there is provided an image processing apparatus, including:

[0016] An image segmentation module, configured to perform image segmentation on the vertebra image to be segmented to obtain a binary segmentation result corresponding to the vertebra image to be segmented; a feature encoding module, configured to perform feature encoding on the vertebra image to be segmented to obtain a pixel embedding vector corresponding to each pixel point in the vertebra image to be segmented; a first determination module, configured to determine a pixel embedding vector corresponding to the vertebra region in the vertebra image to be segmented according to the binary segmentation result and the pixel embedding vector corresponding to each pixel point; a clustering module, configured to perform clustering on the pixel embedding vector corresponding to the vertebra region to obtain a vertebra segmentation result corresponding to the vertebra image to be segmented.

[0017] According to one aspect of the present disclosure, there is provided an electronic device, including: a processor; a memory for storing instructions executable by the processor; wherein, the processor is configured to call the instructions stored in the memory to execute the above method.

[0018] According to one aspect of the present disclosure, there is provided a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the above method is implemented.

[0019] In the embodiments of the present disclosure, an image segmentation is performed on the vertebra image to be segmented to obtain a binary segmentation result corresponding to the vertebra image to be segmented; a feature encoding is performed on the vertebra image to be segmented to obtain a pixel embedding vector corresponding to each pixel point in the vertebra image to be segmented; according to the binary segmentation result and the pixel embedding vector corresponding to each pixel point, a pixel embedding vector corresponding to the vertebra region in the vertebra image to be segmented is determined; since the pixel embedding vector has a higher semantic expression ability, therefore, clustering the pixel embedding vector corresponding to the vertebra region can reduce the probability of misclassification of adjacent vertebrae and obtain a vertebra segmentation result with a higher accuracy corresponding to the vertebra image to be segmented, effectively improving the vertebra segmentation accuracy.

[0020] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and do not limit the present disclosure. According to the following detailed description of exemplary embodiments with reference to the accompanying drawings, other features and aspects of the present disclosure will become clear. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to explain the technical solutions of the present disclosure.

[0022] Figure 1 A flowchart showing an image processing method according to an embodiment of the present disclosure;

[0023] Figure 2 A schematic diagram showing a convex hull region of a vertebra according to an embodiment of the present disclosure;

[0024] Figure 3 A schematic diagram showing a preset vertebra convex hull segmentation network according to an embodiment of the present disclosure

[0025] Figure 4 A schematic diagram showing a vertebra segmentation result according to an embodiment of the present disclosure;

[0026] Figure 5 A schematic diagram showing a preset vertebra instance segmentation network according to an embodiment of the present disclosure;

[0027] Figure 6 A schematic diagram showing the rib endpoints and the centroid of the vertebra according to an embodiment of the present disclosure;

[0028] Figure 7 A schematic diagram showing a target rib-vertebra matching result according to an embodiment of the present disclosure;

[0029] Figure 8 A schematic diagram showing a vertebra labeling result according to an embodiment of the present disclosure;

[0030] Figure 9A block diagram showing an image processing apparatus according to an embodiment of the present disclosure;

[0031] Figure 10 A block diagram showing an electronic device according to an embodiment of the present disclosure;

[0032] Figure 11 A block diagram showing an electronic device according to an embodiment of the present disclosure. Detailed implementation manners

[0033] Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the drawings denote elements having the same or similar functions. Although various aspects of the embodiments are shown in the drawings, the drawings do not have to be drawn to scale unless otherwise specified.

[0034] The term "exemplary" used herein means "serving as an example, embodiment, or illustration". Any embodiment described herein as "exemplary" is not necessarily to be construed as superior or better than other embodiments.

[0035] The term "and / or" herein is merely a description of an associated relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set composed of A, B, and C.

[0036] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following detailed implementation manners. Those skilled in the art should understand that the present disclosure can be implemented without some specific details. In some instances, methods, means, elements, and circuits well known to those skilled in the art are not described in detail so as to highlight the gist of the present disclosure.

[0037] Figure 1 A flowchart showing an image processing method according to an embodiment of the present disclosure. This image processing method can be executed by an electronic device such as a terminal device or a server. The terminal device can be a user equipment (UE), a mobile device, a user terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. This image processing method can be implemented by a processor calling computer-readable instructions stored in a memory. Alternatively, the server can execute this image processing method. As Figure 1 shown, this image processing method may include:

[0038] In step S11, image segmentation is performed on the vertebra image to be segmented, and a binary segmentation result corresponding to the vertebra image to be segmented is obtained.

[0039] In step S12, feature encoding is performed on the vertebra image to be segmented, and a pixel embedding vector corresponding to each pixel point in the vertebra image to be segmented is obtained.

[0040] In step S13, according to the binary segmentation result and the pixel embedding vector corresponding to each pixel point, the pixel embedding vector corresponding to the vertebra region in the vertebra image to be segmented is determined.

[0041] In step S14, clustering is performed on the pixel embedding vector corresponding to the vertebra region to obtain a vertebra segmentation result corresponding to the vertebra image to be segmented.

[0042] In the embodiment of the present disclosure, image segmentation is performed on the vertebra image to be segmented to obtain a binary segmentation result corresponding to the vertebra image to be segmented; feature encoding is performed on the vertebra image to be segmented to obtain a pixel embedding vector corresponding to each pixel point in the vertebra image to be segmented; according to the binary segmentation result and the pixel embedding vector corresponding to each pixel point, the pixel embedding vector corresponding to the vertebra region in the vertebra image to be segmented is determined; since the pixel embedding vector has higher semantic expression ability, therefore, clustering is performed on the pixel embedding vector corresponding to the vertebra region, which can reduce the probability of adjacent vertebrae being misclassified, and obtain a vertebra segmentation result with a higher accuracy rate corresponding to the vertebra image to be segmented, effectively improving the vertebra segmentation accuracy.

[0043] In a possible implementation manner, the image processing method further includes: acquiring an original chest scan image; performing image preprocessing on the original chest scan image to obtain an initial vertebra image; performing convex hull segmentation on the initial vertebra image to obtain a vertebra convex hull region in the initial vertebra image; and cropping the initial vertebra image according to the vertebra convex hull region to obtain a vertebra image to be segmented.

[0044] After performing image preprocessing on the original chest scan image to obtain the initial vertebra image, convex hull segmentation can be performed on the initial vertebra image to determine the vertebra convex hull region in the initial vertebra image, so that the vertebra image to be segmented can be cropped from the initial vertebra image according to the vertebra convex hull region, so that subsequent vertebra segmentation based on the cropped vertebra image to be segmented can effectively reduce the consumption of computing resources and improve the vertebra segmentation efficiency.

[0045] In a possible implementation manner, the original chest scan image may be a chest computed tomography (CT) image I. Since the CT image has good bone-soft tissue contrast, therefore, the chest CT image I is usually used as a medical image for diagnosing vertebral fractures.

[0046] Perform image preprocessing on the chest CT image I to obtain the initial vertebral image I n The image preprocessing may include one or more of redirection, cropping, normalization, etc. The present disclosure does not limit the specific manner of image preprocessing.

[0047] Due to different shooting angles, the vertebral directions in different chest CT images may be different. Perform the preprocessing of redirecting the chest CT image I according to the preset unit matrix to obtain the initial vertebral image I n so that the vertebral direction in the initial vertebral image I n is consistent with the preset coordinate axes (x / y / z axes) to improve the subsequent processing efficiency. The preset unit matrix can be set according to the actual situation. The present disclosure does not limit the specific form of the preset unit matrix.

[0048] In addition to the bone region, the chest CT image I also includes a large area of other background parts. In order to reduce the subsequent consumption of computing resources and improve the processing efficiency, use the preset gray-scale threshold to perform cropping processing on the chest CT image I. Specifically, it may include: performing binarization processing on the chest CT image I based on the preset gray-scale threshold to obtain a binary image. Among them, for the pixel points in the chest CT image I whose gray-scale values are greater than or equal to the preset gray-scale threshold, the corresponding pixel values in the binary image are 1; for the pixel points in the chest CT image I whose gray-scale values are less than the preset gray-scale threshold, the corresponding pixel values in the binary image are 0. Crop the chest CT image I according to the bounding box formed by the pixel points with pixel values of 1 in the binary image to obtain the initial vertebral image I n to reduce the image size, thereby effectively reducing the subsequent consumption of computing resources and improving the processing efficiency. The specific value of the preset gray-scale threshold can be set according to the actual situation. The present disclosure does not make specific limitations on this.

[0049] In practical applications, if the patient wearing the chest CT image I wears metal or other items, there will be pixel points with too high gray-scale values in the obtained chest CT image I, which will affect the subsequent processing accuracy. Therefore, for the initial vertebral image I obtained after cropping the above-mentioned chest CT image I n perform normalization processing using the preset gray-scale value normalization window so that the gray-scale value of the initial vertebral image I n is within a reasonable gray-scale value range to improve the vertebral segmentation accuracy. The preset gray-scale value normalization window can be set according to the actual situation. For example, the preset gray-scale value normalization window is [-1000, 2000]. The present disclosure does not make specific limitations on the actual value range of the preset gray-scale value normalization window.

[0050] Since the scanning area of the chest CT image I is very large, the initial vertebral image I nIn addition to vertebrae, it also includes other bones such as ribs, hip bones, and femurs. The vertebrae region only accounts for a very small part of them. Therefore, based on the convex hull algorithm, the initial vertebrae image I n is segmented to obtain the vertebrae convex hull region in the initial vertebrae image I n . Furthermore, based on the vertebrae convex hull region, the initial vertebrae image I n is cropped to obtain the vertebrae image to be segmented I v including the vertebrae region. Subsequently, based on the vertebrae image to be segmented I v for vertebrae segmentation, it is possible to reduce the waste of computing resources and improve the efficiency of vertebrae segmentation.

[0051] In one example, based on a preset vertebrae convex hull segmentation network, the initial vertebrae image I n is segmented to obtain the vertebrae convex hull region in the initial vertebrae image I n . To improve the segmentation efficiency, the initial vertebrae image I n is resampled to obtain the first vertebrae image I sp3 , where the resolution of the first vertebrae image I sp3 is the first resolution. The specific value of the first resolution can be determined according to the actual situation, and the present disclosure does not make specific limitations thereon. For example, the first resolution is 3mm×3mm×3mm, that is, the actual physical size corresponding to each pixel point in the first vertebrae image I sp3 is 3mm×3mm×3mm.

[0052] The first vertebrae image I sp3 is input into the preset vertebrae convex hull segmentation network. After the preset vertebrae convex hull segmentation network segments the first vertebrae image I sp3 , the vertebrae convex hull region H is obtained. Figure 2 Shows a schematic diagram of the vertebrae convex hull region according to an embodiment of the present disclosure.

[0053] According to the vertebrae convex hull region H, a detection frame including the overall vertebrae region can be determined. For example, the detection frame can be the smallest three-dimensional rectangular frame including the vertebrae convex hull region H. According to this detection frame, the initial vertebrae image I n is cropped to obtain the vertebrae image to be segmented I v including the overall vertebrae region. Based on the vertebrae image to be segmented I v , subsequent vertebrae segmentation is performed.

[0054] In one possible implementation, the preset vertebrae convex hull segmentation network can be a 3D-U-shaped network, including an encoder composed of several convolutional layers and downsampling layers, and a decoder composed of several convolutional layers and upsampling layers. A non-local module is embedded between the encoder and the decoder, and skip connections are introduced at corresponding stages of the encoder and the decoder.Figure 3 Schematic diagram showing a preset vertebral convex hull segmentation network according to an embodiment of the present disclosure.

[0055] When training the preset vertebral convex hull segmentation network, a learning rate setting strategy of warmup and cosine annealing can be adopted, and cross entropy (cross-entropy loss) and dice loss are used as loss functions to train the preset vertebral convex hull segmentation network for a number of training rounds.

[0056] For example, the training of the preset vertebral convex hull segmentation network can be achieved by minimizing the loss function L1 shown in the following formula (1).

[0057]

[0058] Where y in formula (1) i is the segmentation label corresponding to the training sample image i, is the segmentation prediction probability corresponding to the training sample image i determined according to the preset vertebral convex hull segmentation network, Y is the true segmentation result corresponding to the training sample image i, is the predicted segmentation result corresponding to the training sample image i determined according to the preset vertebral convex hull segmentation network.

[0059] Those skilled in the art should understand that the specific network structure and training process of the preset vertebral convex hull segmentation network can adopt other network structures and training methods in related technologies, and the present disclosure does not make specific limitations thereto.

[0060] In a possible implementation, the training samples of the preset vertebral convex hull segmentation network include vertebral sample images and vertebral convex hull region labels corresponding to the vertebral sample images. Among them, the vertebral convex hull region label can be a vertebral binary segmentation label obtained by performing binary segmentation on the vertebral sample image and undergoing Gaussian smoothing threshold dilation several times. The vertebral convex hull region label can also be determined by other methods in related technologies, and the present disclosure does not make specific limitations thereto.

[0061] In an example, the convex hull algorithm can be used to perform convex hull segmentation on the initial vertebral image I n to obtain the vertebral convex hull region in the initial vertebral image I n The specific algorithm form of the convex hull algorithm can be flexibly set according to the actual situation, and the present disclosure does not make specific limitations thereto.

[0062] In a possible implementation, the image processing method further includes: performing position encoding on pixel points in the vertebral image to be segmented to obtain a position encoding image.

[0063] In one example, the segmented vertebral image I can be treated using the following formula (2): v Perform position encoding to obtain the position encoded image I c .

[0064]

[0065] Among them, (i, j, k) is the vertebral image to be segmented I v and position encoded image I c The corresponding pixel point in (μ x ,μ y ,μ z ) is the vertebrae image to be segmented I v The center pixel of the image, W x , W y and W z is the preset hyperparameter. x , W y and W z The specific value of can be determined according to actual conditions, and the present disclosure does not make any specific limitation on this.

[0066] In one example, in order to improve the segmentation efficiency, the segmented vertebrae image I may also be treated as v Resample to obtain the second vertebra image I sp1.5 , where the second vertebra image I sp1.5 The resolution of the second vertebra image I is the second resolution. The specific value of the second resolution can be determined according to actual conditions, and the present disclosure does not make any specific restrictions on this. For example, the second resolution is 1.5 mm × 1.5 mm × 1.5 mm, that is, the second vertebra image I sp1.5 The actual physical size of each pixel in is 1.5mm×1.5mm×1.5mm. Using the above formula (1) for the second vertebra image I sp1.5 Perform position encoding to obtain the position encoded image I c Then, using the second vertebra image I sp1.5 and position encoded image I c , to achieve the segmentation of vertebral image I v Vertebral segmentation.

[0067] In a possible implementation, performing image segmentation on the vertebrae image to be segmented to obtain a binary segmentation result corresponding to the vertebrae image to be segmented includes: performing image segmentation on the vertebrae image to be segmented based on the position coding image to obtain a binary segmentation result.

[0068] In a possible implementation, feature encoding is performed on the vertebra image to be segmented to obtain pixel embedding vectors corresponding to each pixel point in the vertebra image to be segmented, including: based on the position encoding image, performing feature encoding on the vertebra image to be segmented to obtain pixel embedding vectors corresponding to each pixel point in the vertebra image to be segmented.

[0069] In a possible implementation, the image processing method further includes: filtering the non-vertebra regions in the binary segmentation result according to the convex hull region of the vertebra.

[0070] In one example, based on a preset vertebra instance segmentation network, the vertebra image I to be segmented v can be subjected to instance segmentation. The vertebra image I to be segmented v (or the second vertebra image I sp1.5 ) and the position encoding image I c are simultaneously input into the preset vertebra instance segmentation network.

[0071] The preset vertebra instance segmentation network includes two branches. One branch is used to perform binary segmentation on the vertebra image I to be segmented v (or the second vertebra image I sp1.5 ) to obtain the binary segmentation result A corresponding to the vertebra image I to be segmented v (or the second vertebra image I sp1.5 ). Using the convex hull region H of the vertebra, the binary segmentation result A b is filtered to eliminate the false positive part (non-vertebra region) that is mis-segmented as the vertebra region in the binary segmentation result A b , thereby improving the accuracy of the binary segmentation result A b . For example, the binary segmentation result A b = A b ∩ H. b

[0072] The other branch of the preset vertebra instance segmentation network is used to determine the pixel embedding vectors A c corresponding to each pixel point in the vertebra image I to be segmented v (or the second vertebra image I sp1.5 ) based on the position encoding image I e . The dimension of the embedding vector can be 8-dimensional, or can be set to other dimensions according to the actual situation, and the present disclosure does not make specific limitations on this.

[0073] Using the binary segmentation result A b and the pixel embedding vector A e , the pixel embedding vectors A v corresponding to the vertebra regions in the vertebra image I to be segmented sp1.5 (or the second vertebra image I re ) can be obtained = Ab ∩A e Then, the mean-shift clustering algorithm is used to cluster the pixel point embedding vectors A corresponding to the vertebral region re to obtain the vertebral image I to be segmented v and the corresponding vertebral segmentation result A ins Those skilled in the art should understand that, in addition to the mean-shift clustering algorithm, other clustering algorithms in related technologies can also be used for the clustering algorithm, and the present disclosure does not make specific limitations thereto

[0074] Figure 4 FIG. shows a schematic diagram of the vertebral segmentation result according to an embodiment of the present disclosure. As Figure 4 shown, the vertebral segmentation result A ins includes multiple vertebrae in the vertebral image I to be segmented v However, the vertebral segmentation result A ins cannot determine the vertebral identifier of each vertebra

[0075] In a possible implementation manner, the preset vertebral instance segmentation network may be a 3D-U-shaped network, and two encoder branches share a decoder, and a non-local module is embedded between the encoder and the decoder. Wherein, one encoder branch is used to perform binary segmentation on the vertebral image I to be segmented v (or the second vertebral image I sp1.5 ) to obtain a binary segmentation result A b , and the other encoder branch is used to determine the pixel point embedding vectors A corresponding to each pixel point in the vertebral image I to be segmented v (or the second vertebral image I sp1.5 ) e Figure 5 FIG. shows a schematic diagram of the preset vertebral instance segmentation network according to an embodiment of the present disclosure

[0076] When training the preset vertebral instance segmentation network, the encoder branch for binary segmentation can use cross entropy and dice loss as loss functions to perform training for several training rounds. The specific training formula can refer to the above formula (1), which will not be elaborated here

[0077] The encoder branch for embedding vector prediction can use discriminative loss as the loss function to perform training for several training rounds

[0078] For example, the training of the encoder branch for embedding vector prediction can be achieved by minimizing the loss function L shown in the following formula (3) d

[0079] ​​

[0080] Among them, C in formula (3) is the total number of categories in the training sample images, N c is the number of pixel points belonging to the same category in the training sample images, μ c is the mean vector of the category, μ c is the embedding vector of pixel point i in the training sample images.

[0081] Those skilled in the art should understand that the specific network structure and training process of the preset vertebral instance segmentation network can adopt other network structures and training methods in related technologies, and the present disclosure does not make specific limitations thereto.

[0082] In the diagnosis of vertebral fractures, doctors need to give the accurate vertebral identification (the anatomical label of the vertebra, e.g., vertebra No. 3) of the fractured vertebra based on the medical images of the chest (e.g., chest CT images). In related technologies, doctors usually need to count the vertebrae one by one from top to bottom (or from bottom to top) to determine the vertebral identification, resulting in a complex vertebral identification process and low accuracy. Therefore, after obtaining the vertebral segmentation result based on the method of the above embodiment, it is also necessary to provide a method that can automatically mark the vertebral identification to improve the vertebral marking efficiency and accuracy.

[0083] In a possible implementation manner, the vertebral segmentation result includes multiple vertebrae in the vertebra image to be segmented; the image processing method further includes: obtaining the rib marking result corresponding to the vertebral region in the vertebra image to be segmented, where the rib marking result includes multiple ribs and the rib identification corresponding to each rib; determining the target rib-vertebra matching result according to the multiple vertebrae and the multiple ribs, where the target rib-vertebra matching result includes at least one rib-vertebra matching pair; and determining the vertebral identification corresponding to each vertebra according to the rib identification corresponding to each rib and the target rib-vertebra matching result.

[0084] Based on the spatial geometric relationship between the vertebrae and the ribs, the multiple vertebrae in the vertebral segmentation result are matched with the multiple ribs in the rib marking result to obtain the target rib-vertebra matching result, and then the vertebral identification corresponding to each vertebra is determined by using the rib identification corresponding to each rib in the rib marking result, so as to effectively obtain a vertebral marking result with higher accuracy.

[0085] Among them, the rib marking result is obtained after segmenting and marking the rib image to be segmented, and the rib image to be segmented and the vertebra image to be segmented correspond to the same target object. For example, both the rib image to be segmented and the vertebra image to be segmented are chest CT images obtained from a chest computed tomography scan of the same target object.

[0086] In one example, performing segmentation marking on the rib image to be segmented, and determining the rib marking result corresponding to the rib image to be segmented, including: performing semantic category segmentation on the rib image to be segmented to obtain the semantic category segmentation result corresponding to the rib image to be segmented; performing instance segmentation on the rib image to be segmented to obtain the instance segmentation result corresponding to the rib image to be segmented; determining the rib marking result corresponding to the rib image to be segmented according to the semantic category segmentation result and the instance segmentation result, wherein the rib marking result includes multiple ribs in the rib image to be segmented and the rib identifier corresponding to each rib.

[0087] Based on the global semantic information of the image, performing semantic category segmentation on the rib image to be segmented to obtain the semantic category segmentation result corresponding to the rib image to be segmented; based on the local geometric information of the image, performing instance segmentation on the rib image to be segmented to obtain the instance segmentation result corresponding to the rib image to be segmented; since both the global semantic information of the image and the local geometric information of the image are comprehensively considered, therefore, the target marking result including multiple ribs in the rib image to be segmented and the rib identifier corresponding to each rib determined based on the semantic category segmentation result and the instance segmentation result has a high accuracy, thereby effectively improving the segmentation and marking accuracy of the ribs.

[0088] In one example, performing segmentation marking on the rib image to be segmented, and determining the rib marking result corresponding to the rib image to be segmented, further includes: in the case where the semantic category segmentation and the instance segmentation are performed at a first resolution, performing binary segmentation on the rib image to be segmented at a second resolution to obtain the fine binary segmentation result corresponding to the rib image to be segmented, wherein the second resolution is greater than the first resolution; updating the rib marking result according to the fine binary segmentation result.

[0089] When the semantic category segmentation and the instance segmentation of the rib image to be segmented are performed at a first resolution, the segmentation result of each rib in the rib marking result is relatively rough. Therefore, based on the fine binary segmentation result with a higher resolution, the rib marking result is updated to obtain a rib marking result with a higher resolution and a higher segmentation accuracy.

[0090] Those skilled in the art should understand that the method of performing segmentation marking on the rib image to be segmented, in addition to the above method, can also adopt any rib segmentation and marking method in the related art, and the present disclosure does not make specific limitations thereto.

[0091] In a possible implementation, determining a target rib-vertebra matching result based on multiple vertebrae and multiple ribs includes: for any one vertebra, determining a candidate rib-vertebra matching result corresponding to the vertebra, where the candidate rib-vertebra matching result includes at least one rib-vertebra matching pair; determining the matching degree corresponding to each candidate rib-vertebra matching result according to the spatial geometric relationship of at least one rib-vertebra matching pair included in each candidate rib-vertebra matching result; and determining the candidate rib-vertebra matching result whose matching degree meets a preset matching condition as the target rib-vertebra matching result.

[0092] In a possible implementation, determining the matching degree corresponding to each candidate rib-vertebra matching result according to the spatial geometric relationship of at least one rib-vertebra matching pair included in each candidate rib-vertebra matching result includes: determining the rib endpoint closest to the vertebra in each rib according to the rib marking result; determining the centroid of each vertebra corresponding to each vertebra according to the vertebra segmentation result, and the tangent vector corresponding to each vertebral centroid; determining the average cosine distance corresponding to each candidate rib-vertebra matching result according to the rib endpoint corresponding to each rib, the centroid of each vertebra corresponding to each vertebra, and the tangent vector corresponding to each vertebral centroid; and determining the average cosine distance corresponding to each candidate rib-vertebra matching result as the matching degree corresponding to each candidate rib-vertebra matching result.

[0093] The vertebra segmentation result includes a set of vertebrae {V i |V i ∈A ins} composed of multiple vertebrae, and determining the centroid m of each vertebra corresponding to each vertebra. Sorting the multiple vertebrae in the set of vertebrae according to the coordinates of the centroid corresponding to each vertebra, obtaining a vertebra sequence V=(V1, V2,..., V K ), and a vertebral centroid sequence M=(m1, m2,..., m N ). Curve fitting the N vertebral centroids to obtain the spinal centerline. Determining the tangent vector τ of each vertebral centroid on the spinal centerline, obtaining a tangent vector set T=(τ1, τ2,..., τ N ). The rib marking result includes a rib sequence R=(R1, R2,..., R K ) composed of multiple ribs, and determining the rib endpoint closest to the vertebra in each rib, that is, the rib endpoint closest to the spinal centerline in each rib, obtaining a rib endpoint set E=(e1, e2,..., e K ). Figure 6 A schematic diagram showing rib endpoints and vertebral centroids according to an embodiment of the present disclosure.

[0094] Enumerate all possible rib-vertebra matching ways to obtain the candidate rib-vertebra matching results corresponding to each vertebra For example, for vertebra V sv , assume that vertebra V sv and rib R sr form a rib-vertebra matching pair. Then, according to the sorting in rib sequence R and vertebra sequence V, it can be obtained that vertebra V sv+1 and rib R sr+1 form a rib-vertebra matching pair, and so on, to obtain the candidate rib-vertebra matching results corresponding to vertebra V sv Traverse each vertebra in the vertebra set to obtain a set of candidate rib-vertebra matching results

[0095] For any candidate rib-vertebra matching result Use the following formula (4) to determine the mean cosine distance corresponding to the candidate rib-vertebra matching result

[0096]

[0097] where p is the number of rib-vertebra matching pairs included in the candidate rib-vertebra matching result , (R i , V j ) is a rib-vertebra matching pair in the candidate rib-vertebra matching result , e i is the rib endpoint corresponding to rib R i , m j is the centroid of vertebra V j , τ j is the tangent vector corresponding to the centroid m of the vertebra j

[0098] Determine the mean cosine distance corresponding to each candidate rib-vertebra matching result as its corresponding matching degree. Furthermore, determine the candidate rib-vertebra matching results that meet the preset matching conditions as the target rib-vertebra matching results, that is, determine the candidate rib-vertebra matching results whose mean cosine distance meets the preset matching conditions as the target rib-vertebra matching results

[0099] ​​​In a possible implementation, determining the target rib-vertebra matching result from the candidate rib-vertebra matching results that meet the preset matching conditions includes: determining whether there is a set of target rib-vertebra matching results according to the mean cosine distance corresponding to each candidate rib-vertebra matching result, where the mean cosine distance corresponding to each candidate rib-vertebra matching result in the set of target rib-vertebra matching results is greater than 0; in the case where there is a set of target rib-vertebra matching results, determining the candidate rib-vertebra matching result corresponding to the smallest mean cosine distance value in the set of target rib-vertebra matching results as the target rib-vertebra matching result.

[0100] In a possible implementation, the image processing method further includes: in the case where there is no set of target rib-vertebra matching results, determining the candidate rib-vertebra matching result corresponding to the largest mean cosine distance value as the target rib-vertebra matching result.

[0101] In an example, according to the mean cosine distance corresponding to each candidate rib-vertebra matching result, the set Φ of candidate rib-vertebra matching results is divided into a positive cosine matching set Φ + , and a negative cosine matching set Φ - . Among them, the mean cosine distance corresponding to the candidate rib-vertebra matching results included in the positive cosine matching set Φ + is greater than 0, and the mean cosine distance corresponding to the candidate rib-vertebra matching results included in the negative cosine matching set Φ - is less than or equal to 0.

[0102] In the case where the positive cosine matching set Φ + is a non-empty set, that is, there is a set Φ of target rib-vertebra matching results + , at this time, since the mean cosine distance corresponding to the candidate rib-vertebra matching results included in the set Φ of target rib-vertebra matching results + is greater than 0, the smaller the value of the mean cosine distance of the candidate rib-vertebra matching result, the closer the distance between the rib and the vertebra in the rib-vertebra matching pair included in the candidate rib-vertebra matching result, and the higher the matching degree. Therefore, the candidate rib-vertebra matching result corresponding to the smallest mean cosine distance value in the set Φ of target rib-vertebra matching results + can be determined as the target rib-vertebra matching result

[0103] In the case where the positive cosine matching set Φ + is an empty set, that is, there is no set Φ of target rib-vertebra matching results + , only the negative cosine matching set Φ - exists. Since the negative cosine matching set Φ- The mean cosine distance corresponding to the candidate rib-vertebra matching results included is less than or equal to 0. The larger the value of the mean cosine distance of the candidate rib-vertebra matching results, the closer the distance between the rib and the vertebra in the rib-vertebra matching pairs included in the candidate rib-vertebra matching results, and the higher the matching degree. Therefore, the negative cosine matching set Φ - The candidate rib-vertebra matching result corresponding to the largest mean cosine distance value in it is determined as the target rib-vertebra matching result

[0104] In one example, the target rib-vertebra matching result can be determined according to the following formula (5)

[0105]

[0106] Figure 7 A schematic diagram showing the target rib-vertebra matching result according to an embodiment of the present disclosure. As Figure 7 shown, the distance between the rib endpoint and the vertebral centroid in each rib-vertebra matching pair in the target rib-vertebra matching result is the closest

[0107] After determining the target rib-vertebra matching result, the target rib-vertebra matching result includes at least one rib-vertebra matching pair. According to the rib identification of the rib in each rib-vertebra matching pair and the anatomical correspondence between the rib and the vertebra, the vertebra identification of the vertebra in each rib-vertebra matching pair is determined, and the vertebra identifications of at least one vertebra are obtained. Furthermore, according to the sorting of multiple vertebrae in the vertebra set, the vertebra identification of each vertebra can be obtained, and a vertebra marking result with a higher accuracy can be obtained, effectively realizing the vertebra marking of the vertebra image to be segmented and improving the vertebra marking accuracy

[0108] Figure 8 A schematic diagram showing the vertebra marking result according to an embodiment of the present disclosure. As Figure 8 shown, the vertebra marking result includes multiple vertebrae in the vertebra image I to be segmented v and the vertebra identification corresponding to each vertebra. In Figure 8 it, different colors can be used to indicate each vertebra and the vertebra identification corresponding to each vertebra. For example, gray is used to indicate the 1st vertebra, green is used to indicate the 2nd vertebra, and so on. Those skilled in the art should understand that other forms in the related art can be used in the vertebra marking result to indicate each vertebra and the vertebra identification corresponding to each vertebra, and the present disclosure does not make specific limitations thereto

[0109] It can be understood that, without violating the principle logic, the above-mentioned method embodiments mentioned in the present disclosure can be combined with each other to form combined embodiments. Due to space limitations, the present disclosure will not elaborate further. Those skilled in the art can understand that in the above method of the specific implementation manner, the specific execution order of each step should be determined according to its function and possible internal logic.

[0110] In addition, the present disclosure also provides an image processing device, an electronic device, a computer-readable storage medium, and a program, all of which can be used to implement any one of the image processing methods provided by the present disclosure. For the corresponding technical solutions and descriptions, please refer to the corresponding records in the method section and will not be elaborated further.

[0111] Figure 9 The block diagram of an image processing device according to an embodiment of the present disclosure is shown. As Figure 9 shown, the device 90 includes:

[0112] An image segmentation module 91, configured to perform image segmentation on the vertebra image to be segmented, and obtain a binary segmentation result corresponding to the vertebra image to be segmented;

[0113] A feature encoding module 92, configured to perform feature encoding on the vertebra image to be segmented, and obtain a pixel embedding vector corresponding to each pixel point in the vertebra image to be segmented;

[0114] A first determination module 93, configured to determine a pixel embedding vector corresponding to the vertebra region in the vertebra image to be segmented according to the binary segmentation result and the pixel embedding vector corresponding to each pixel point;

[0115] A clustering module 94, configured to perform clustering on the pixel embedding vector corresponding to the vertebra region, and obtain a vertebra segmentation result corresponding to the vertebra image to be segmented.

[0116] In a possible implementation manner, the device 90 further includes:

[0117] A first acquisition module, configured to acquire an original chest scan image;

[0118] An image preprocessing module, configured to perform image preprocessing on the original chest scan image, and obtain an initial vertebra image;

[0119] A convex hull segmentation module, configured to perform convex hull segmentation on the initial vertebra image, and obtain a vertebra convex hull region in the initial vertebra image;

[0120] A cropping module, configured to crop the initial vertebra image according to the vertebra convex hull region, and obtain the vertebra image to be segmented.

[0121] In a possible implementation manner, the device 90 further includes:

[0122] A position encoding module, configured to perform position encoding on pixel points in the vertebra image to be segmented, so as to obtain a position encoding image.

[0123] In a possible implementation manner, the image segmentation module 91 is specifically configured to:

[0124] Based on the position encoding image, perform image segmentation on the vertebra image to be segmented, so as to obtain a binary segmentation result.

[0125] In a possible implementation manner, the feature encoding module 92 is specifically configured to:

[0126] Based on the position encoding image, perform feature encoding on the vertebra image to be segmented, so as to obtain pixel point embedding vectors corresponding to each pixel point.

[0127] In a possible implementation manner, the device 90 further includes:

[0128] A filtering module, configured to filter non-vertebra regions in the binary segmentation result according to the convex hull region of the vertebra.

[0129] In a possible implementation manner, multiple vertebrae in the vertebra image to be segmented are included in the vertebra segmentation result;

[0130] The device 70 further includes:

[0131] A second obtaining module, configured to obtain a rib marking result corresponding to the vertebra region in the vertebra image to be segmented, where the rib marking result includes multiple ribs and a rib identifier corresponding to each rib;

[0132] A second determining module, configured to determine a target rib-vertebra matching result according to the multiple vertebrae and the multiple ribs, where the target rib-vertebra matching result includes at least one rib-vertebra matching pair;

[0133] A third determining module, configured to determine a vertebra identifier corresponding to each vertebra according to the rib identifier corresponding to each rib and the target rib-vertebra matching result.

[0134] In a possible implementation manner, the second determining module includes:

[0135] A first determining sub-module, configured to, for any one vertebra, determine a candidate rib-vertebra matching result corresponding to the vertebra, where the candidate rib-vertebra matching result includes at least one rib-vertebra matching pair;

[0136] A second determining sub-module, configured to determine a matching degree corresponding to each candidate rib-vertebra matching result according to the spatial geometric relationship of at least one rib-vertebra matching pair included in each candidate rib-vertebra matching result;

[0137] A third determination sub-module, configured to determine a candidate rib-vertebra matching result whose matching degree meets a preset matching condition as a target rib-vertebra matching result.

[0138] In a possible implementation, the second determination sub-module includes:

[0139] A first determination unit, configured to determine, according to a rib marking result, a rib end point in each rib that is closest to a vertebra;

[0140] A second determination unit, configured to determine, according to a vertebra segmentation result, a centroid of each vertebra and a tangent vector corresponding to each vertebra centroid;

[0141] A third determination unit, configured to determine an average cosine distance corresponding to each candidate rib-vertebra matching result according to the rib end point corresponding to each rib, the centroid of each vertebra corresponding to each vertebra, and the tangent vector corresponding to each vertebra centroid;

[0142] A fourth determination unit, configured to determine the average cosine distance corresponding to each candidate rib-vertebra matching result as the matching degree corresponding to each candidate rib-vertebra matching result.

[0143] In a possible implementation, the third determination sub-module includes:

[0144] A fifth determination unit, configured to determine whether there is a set of target rib-vertebra matching results according to the average cosine distance corresponding to each candidate rib-vertebra matching result, where the average cosine distance corresponding to each candidate rib-vertebra matching result in the set of target rib-vertebra matching results is greater than 0;

[0145] A sixth determination unit, configured to, when there is a set of target rib-vertebra matching results, determine a candidate rib-vertebra matching result corresponding to the minimum average cosine distance value in the set of target rib-vertebra matching results as the target rib-vertebra matching result.

[0146] In a possible implementation, the apparatus 90 further includes:

[0147] A fourth determination module, configured to, when there is no set of target rib-vertebra matching results, determine a candidate rib-vertebra matching result corresponding to the maximum average cosine distance value as the target rib-vertebra matching result.

[0148] In some embodiments, the functions or modules included in the apparatus provided in the embodiments of the present disclosure can be used to execute the methods described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0149] Embodiments of the present disclosure also provide a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the above method. The computer-readable storage medium may be a volatile or non-volatile computer-readable storage medium.

[0150] Embodiments of the present disclosure also provide an electronic device, including: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to call the instructions stored in the memory to execute the above method.

[0151] Embodiments of the present disclosure also provide a computer program product, including computer-readable code or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs on a processor of an electronic device, the processor in the electronic device executes the above method.

[0152] The electronic device may be provided as a terminal, a server, or other forms of devices.

[0153] Figure 10 A block diagram of an electronic device according to an embodiment of the present disclosure is shown. As Figure 10 shown, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or other terminals.

[0154] Referring to Figure 10 , the electronic device 800 may include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0155] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, telephone call, data communication, camera operation, and recording operation. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0156] The memory 804 is configured to store various types of data to support the operation of the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, and the like. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0157] The power supply component 806 provides power to various components of the electronic device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 800.

[0158] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can not only sense the boundaries of touch or swipe actions but also detect the duration and pressure associated with the touch or swipe operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front camera and the rear camera can be a fixed optical lens system or have a focal length and optical zoom capabilities.

[0159] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC) that is configured to receive external audio signals when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 further includes a speaker for outputting audio signals.

[0160] The I / O interface 812 provides an interface between the processing component 802 and a peripheral interface module, which can be a keyboard, a click wheel, buttons, etc. These buttons can include, but are not limited to: a home button, a volume button, a power button, and a lock button.

[0161] The sensor assembly 814 includes one or more sensors for providing an assessment of various aspects of the status of the electronic device 800. For example, the sensor assembly 814 can detect the on / off state of the electronic device 800, the relative positioning of components, such as the display and keypad of the electronic device 800. The sensor assembly 814 can also detect a change in the position of the electronic device 800 or a component of the electronic device 800, the presence or absence of user contact with the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and the temperature change of the electronic device 800. The sensor assembly 814 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 can also include a light sensor, such as a complementary metal oxide semiconductor (CMOS) or charge-coupled device (CCD) image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 can also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0162] The communication component 816 is configured to facilitate communication, either wired or wirelessly, between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on communication standards, such as a wireless local area network (WiFi), a second-generation mobile communication technology (2G), or a third-generation mobile communication technology (3G), or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0163] In an exemplary embodiment, the electronic device 800 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the above-described methods.

[0164] In an exemplary embodiment, a non-transitory computer-readable storage medium is also provided, such as a memory 804 including computer program instructions, and the above-described computer program instructions can be executed by a processor 820 of the electronic device 800 to complete the above-described methods.

[0165] Figure 11 A block diagram of an electronic device according to an embodiment of the present disclosure is shown. As Figure 11As shown, the electronic device 1900 can be provided as a server. Referring to Figure 11 , the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by a memory 1932 for storing instructions executable by the processing component 1922, such as application programs. The application programs stored in the memory 1932 may include one or more modules each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0166] The electronic device 1900 may further include a power component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output (I / O) interface 1958. The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as the Microsoft server operating system (Windows Server TM ), the graphical user interface-based operating system launched by Apple Inc. (Mac OS X TM ), the multi-user and multi-process computer operating system (Unix TM ), the free and open-source Unix-like operating system (Linux TM ), the open-source Unix-like operating system (FreeBSD TM ) or the like.

[0167] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as the memory 1932 including computer program instructions, and the above computer program instructions can be executed by the processing component 1922 of the electronic device 1900 to complete the above method.

[0168] The present disclosure may be a system, a method, and / or a computer program product. The computer program product may include a computer-readable storage medium having thereon computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0169] A computer-readable storage medium can be a tangible device that can hold and store instructions for use by an instruction execution device. A computer-readable storage medium may be, for example, (but is not limited to) an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), a memory stick, a floppy disk, a mechanically encoded device such as a punched card or raised structures in grooves having instructions stored thereon, and any suitable combination of the foregoing. The computer-readable storage medium used herein is not construed as an instantaneous signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated through a waveguide or other transmission medium (e.g., an optical pulse through an optical fiber cable), or an electrical signal transmitted through a wire.

[0170] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to various computing / processing devices, or downloaded to an external computer or external storage device through a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing / processing device.

[0171] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine - related instructions, microcode, firmware instructions, state - setting data, or source code or object code written in any combination of one or more programming languages, including object - oriented programming languages such as Smalltalk, C++, etc., and conventional procedural programming languages such as the "C" language or similar programming languages. The computer - readable program instructions may be executed entirely on the user's computer, partially on the user's computer, executed as a stand - alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or, alternatively, may be connected to an external computer (e.g., through the Internet using an Internet service provider). In some embodiments, by using the state information of the computer - readable program instructions to customize an electronic circuit, such as a programmable logic circuit, a field - programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit can execute the computer - readable program instructions to implement various aspects of the present disclosure.

[0172] Aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer - readable program instructions.

[0173] These computer - readable program instructions can be provided to a processor of a general - purpose computer, a special - purpose computer, or other programmable data - processing apparatus to produce a machine such that the instructions, when executed by the processor of the computer or other programmable data - processing apparatus, create a means for implementing the functions / acts specified in one or more blocks of the flowchart and / or block diagram. These computer - readable program instructions can also be stored in a computer - readable storage medium, which causes a computer, a programmable data - processing apparatus, and / or other devices to operate in a particular manner, so that the computer - readable medium storing the instructions includes a manufacture, which includes instructions for implementing various aspects of the functions / acts specified in one or more blocks of the flowchart and / or block diagram.

[0174] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device, causing a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, such that the instructions executed on the computer, other programmable data processing apparatus, or other device implement the functions / actions specified in one or more boxes of the flowchart and / or block diagram.

[0175] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram may represent a module, a segment of code, or a portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions noted in the boxes may occur in a different order than noted in the figures. For example, two consecutive boxes may in fact be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and combinations of boxes in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system that performs the specified functions or actions, or by a combination of dedicated hardware and computer instructions.

[0176] The computer program product may be implemented specifically in the form of hardware, software, or a combination thereof. In an alternative embodiment, the computer program product is specifically embodied as a computer storage medium. In another alternative embodiment, the computer program product is specifically embodied as a software product, such as a Software Development Kit (SDK), and so on.

[0177] The various embodiments of the present disclosure have been described above. The above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art in the technical field without departing from the scope and spirit of the described embodiments. The selection of the terms used herein is intended to best explain the principles of the embodiments, the practical application, or the improvement of the technology in the market, or to enable other ordinary skilled persons in the technical field to understand the embodiments disclosed herein.

Claims

1. An image processing method, characterized in that, Including: Performing image segmentation on the vertebra image to be segmented to obtain a binary segmentation result corresponding to the vertebra image to be segmented; Performing feature encoding on the vertebra image to be segmented to obtain pixel embedding vectors corresponding to each pixel point in the vertebra image to be segmented; Determining pixel embedding vectors corresponding to the vertebra region in the vertebra image to be segmented according to the binary segmentation result and the pixel embedding vectors corresponding to each pixel point; Performing clustering on the pixel embedding vectors corresponding to the vertebra region to obtain a vertebra segmentation result corresponding to the vertebra image to be segmented; Wherein, the vertebra segmentation result includes multiple vertebrae in the vertebra image to be segmented; The method further includes: Obtaining a rib marking result corresponding to the vertebra region in the vertebra image to be segmented, wherein the rib marking result includes multiple ribs and a rib identifier corresponding to each rib; Determining a target rib-vertebra matching result according to the multiple vertebrae and the multiple ribs, wherein the target rib-vertebra matching result includes at least one rib-vertebra matching pair; Determining a vertebra identifier corresponding to each vertebra according to the rib identifier corresponding to each rib and the target rib-vertebra matching result.

2. The method according to claim 1, characterized in that, The method further includes: Obtaining an original chest scan image; Performing image preprocessing on the original chest scan image to obtain an initial vertebra image; Performing convex hull segmentation on the initial vertebra image to obtain a vertebra convex hull region in the initial vertebra image; Cropping the initial vertebra image according to the vertebra convex hull region to obtain the vertebra image to be segmented.

3. The method according to claim 2, characterized in that, The method further includes: Performing position encoding on the pixel points in the vertebra image to be segmented to obtain a position encoding image.

4. The method according to claim 3, characterized in that, The performing image segmentation on the vertebra image to be segmented to obtain a binary segmentation result corresponding to the vertebra image to be segmented includes: Based on the position encoding image, performing image segmentation on the vertebra image to be segmented to obtain the binary segmentation result.

5. The method according to claim 3 or 4, characterized in that, The performing feature encoding on the vertebra image to be segmented to obtain pixel embedding vectors corresponding to each pixel point in the vertebra image to be segmented includes: Based on the position encoding image, performing feature encoding on the vertebra image to be segmented to obtain the pixel embedding vectors corresponding to each pixel point.

6. The method according to claim 5, wherein The method further includes: Filtering non-vertebra regions in the binary segmentation result according to the vertebra convex hull region.

7. The method according to claim 1, wherein The determining a target rib-vertebra matching result according to the multiple vertebrae and the multiple ribs includes: For any one vertebra, determining a candidate rib-vertebra matching result corresponding to the vertebra, wherein the candidate rib-vertebra matching result includes at least one rib-vertebra matching pair; Determining a matching degree corresponding to each candidate rib-vertebra matching result according to the spatial geometric relationship of at least one rib-vertebra matching pair included in each candidate rib-vertebra matching result; Determining the candidate rib-vertebra matching result whose matching degree meets a preset matching condition as the target rib-vertebra matching result.

8. The method according to claim 7, characterized in that, Determining the matching degree corresponding to each candidate rib-vertebra matching result according to the spatial geometric relationship of at least one rib-vertebra matching pair included in each candidate rib-vertebra matching result includes: Determining, according to the rib marking result, the rib end point closest to the vertebra in each rib; Determining, according to the vertebra segmentation result, the centroid of each vertebra and the tangent vector corresponding to each vertebra centroid; Determining the average cosine distance corresponding to each candidate rib-vertebra matching result according to the rib end point corresponding to each rib, the centroid of each vertebra, and the tangent vector corresponding to each vertebra centroid; Determining the average cosine distance corresponding to each candidate rib-vertebra matching result as the matching degree corresponding to each candidate rib-vertebra matching result.

9. The method according to claim 8, wherein Determining the candidate rib-vertebra matching result whose matching degree meets the preset matching condition as the target rib-vertebra matching result includes: Determining whether there is a set of target rib-vertebra matching results according to the average cosine distance corresponding to each candidate rib-vertebra matching result, where the average cosine distance corresponding to each candidate rib-vertebra matching result in the set of target rib-vertebra matching results is greater than 0; In the case where there is a set of target rib-vertebra matching results, determining the candidate rib-vertebra matching result corresponding to the minimum average cosine distance value in the set of target rib-vertebra matching results as the target rib-vertebra matching result.

10. The method according to claim 9, wherein The method further includes: In the case where there is no set of target rib-vertebra matching results, determining the candidate rib-vertebra matching result corresponding to the maximum average cosine distance value as the target rib-vertebra matching result.

11. An image processing apparatus, characterized in that, Including: An image segmentation module for performing image segmentation on the vertebra image to be segmented to obtain a binary segmentation result corresponding to the vertebra image to be segmented; A feature encoding module for performing feature encoding on the vertebra image to be segmented to obtain pixel embedding vectors corresponding to each pixel point in the vertebra image to be segmented; A first determination module for determining the pixel embedding vector corresponding to the vertebra region in the vertebra image to be segmented according to the binary segmentation result and the pixel embedding vectors corresponding to each pixel point; A clustering module for clustering the pixel embedding vectors corresponding to the vertebra region to obtain a vertebra segmentation result corresponding to the vertebra image to be segmented; Wherein, the vertebra segmentation result includes multiple vertebrae in the vertebra image to be segmented; The device further includes: A second acquisition module for acquiring a rib marking result corresponding to the vertebra region in the vertebra image to be segmented, where the rib marking result includes multiple ribs and rib identifiers corresponding to each rib; A second determination module for determining a target rib-vertebra matching result according to the multiple vertebrae and the multiple ribs, where the target rib-vertebra matching result includes at least one rib-vertebra matching pair; A third determination module, configured to determine a vertebra identifier corresponding to each vertebra according to the rib identifier corresponding to each rib and the target rib-vertebra matching result.

12. An electronic device, characterized in that, Comprising: A processor; A memory for storing instructions executable by the processor; Wherein, the processor is configured to call the instructions stored in the memory to execute the method according to any one of claims 1 to 10.

13. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, the method according to any one of claims 1 to 10 is implemented.

Images