Image Processing Method and Apparatus, Electronic Device, and Storage Medium

Abstract

The present disclosure relates to an image processing method, apparatus, electronic device, and storage medium. The method includes: performing semantic class segmentation on a rib image to be segmented to obtain a semantic class segmentation result corresponding to the rib image to be segmented; performing instance segmentation on the rib image to be segmented to obtain an instance segmentation result corresponding to the rib image to be segmented; and determining a target marking result corresponding to the rib image to be segmented according to the semantic class segmentation result and the instance segmentation result, where the target marking result includes multiple ribs in the rib image to be segmented and a rib identifier corresponding to each rib. Embodiments of the present disclosure can comprehensively consider the global semantic information of the image and the local geometric information of the image, and improve the segmentation and marking accuracy of the ribs.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technologies, and in particular, to an image processing method and apparatus, an electronic device, and a storage medium. Background Art

[0002] The rib is an important part of the human skeleton. It is located in the upper part of the human torso and can effectively protect the organs in the human chest cavity. In recent years, chest traumas (such as rib fractures) caused by accidents such as car crashes often occur. In the diagnosis of rib fractures, doctors need to give accurate rib identifications (anatomical labels of ribs, such as the 3rd rib on the left side) of the fractured ribs based on medical images of the chest (such as chest CT images). In related technologies, a neural network is used to segment the ribs to obtain the segmentation and labeling results of the ribs. However, when using a single neural network to complete both the segmentation and labeling tasks simultaneously, it is easy to cause misidentification between adjacent ribs, resulting in low accuracy of both rib segmentation and naming. Summary of the Invention

[0003] The present disclosure provides a technical solution for an image processing method and apparatus, an electronic device, and a storage medium.

[0004] According to one aspect of the present disclosure, there is provided an image processing method, including: performing semantic class segmentation on a rib image to be segmented to obtain a semantic class segmentation result corresponding to the rib image to be segmented; performing instance segmentation on the rib image to be segmented to obtain an instance segmentation result corresponding to the rib image to be segmented; determining a target labeling result corresponding to the rib image to be segmented according to the semantic class segmentation result and the instance segmentation result, where the target labeling result includes multiple ribs in the rib image to be segmented and a rib identification corresponding to each rib.

[0005] In a possible implementation, the method further includes: when the semantic class segmentation and the instance segmentation are performed at a first resolution, performing binary segmentation on the rib image to be segmented at a second resolution greater than the first resolution to obtain a fine binary segmentation result corresponding to the rib image to be segmented; updating the target labeling result according to the fine binary segmentation result.

[0006] In a possible implementation, the method further includes: acquiring an original chest scan image; performing image preprocessing on the original chest scan image to obtain an initial rib image; performing convex hull segmentation on the initial rib image to obtain a rib convex hull region in the initial rib image; cropping the initial rib image according to the rib convex hull region to obtain the rib image to be segmented.

[0007] In a possible implementation, the convex hull segmentation of the initial rib image to obtain the rib convex hull region in the initial rib image includes: determining whether the rib region in the initial rib image meets a preset integrity requirement; and performing convex hull segmentation on the initial rib image to obtain the rib convex hull region when the rib region in the initial rib image meets the preset integrity requirement.

[0008] In a possible implementation, the initial rib image includes a plurality of slice images; the determining whether the rib region in the initial rib image meets a preset integrity requirement includes: determining the rib category of each slice image in the plurality of slice images, where different rib categories are used to indicate different rib regions; determining a rib category sequence according to the rib category of each slice image; and determining that the rib region in the initial rib image meets the preset integrity requirement when the rib category sequence includes a preset rib category.

[0009] In a possible implementation, the instance segmentation of the rib image to be segmented to obtain the instance segmentation result corresponding to the rib image to be segmented includes: resampling the rib image to be segmented to obtain a first rib image, where the resolution of the first rib image is the first resolution; performing position encoding on the pixel points in the first rib image to obtain a position encoding image; and performing instance segmentation on the first rib image based on the position encoding image to obtain the instance segmentation result.

[0010] In a possible implementation, the performing instance segmentation on the first rib image based on the position encoding image to obtain the instance segmentation result includes: performing binary segmentation on the first rib image to obtain a target rough binary segmentation result corresponding to the first rib image; determining a pixel point embedding vector corresponding to the first rib image based on the position encoding image; determining a pixel point embedding vector corresponding to the rib region in the first rib image according to the target rough binary segmentation result and the pixel point embedding vector corresponding to the first rib image; and clustering the pixel point embedding vectors corresponding to the rib region in the first rib image to obtain the instance segmentation result.

[0011] In a possible implementation, the performing binary segmentation on the first rib image to obtain a target rough binary segmentation result corresponding to the first rib image includes: performing binary segmentation on the first rib image to obtain an initial rough binary segmentation result corresponding to the first rib image; and filtering the non-rib region in the initial rough binary segmentation result according to the rib convex hull region to obtain the target rough binary segmentation result.

[0012] In a possible implementation, determining the target labeling result corresponding to the rib image to be segmented according to the semantic category segmentation result and the instance segmentation result includes: determining a unilateral rib sequence according to the instance segmentation result and the rib convex hull area, wherein the unilateral rib sequence includes multiple unilateral ribs and their ordering, and the unilateral rib sequence includes a left rib sequence and a right rib sequence; determining multiple ribs in the rib image to be segmented and the rib identifier corresponding to each rib according to the semantic category segmentation result and the unilateral rib sequence.

[0013] In a possible implementation, determining a unilateral rib sequence according to the instance segmentation result and the rib convex hull area includes: determining a first unilateral rib set according to the instance segmentation result and the rib convex hull area, wherein the first unilateral rib sequence includes N unilateral ribs, and N is the number of unilateral ribs; sorting the N unilateral ribs in the z-axis direction, and taking the K ribs before the sorting to form a second unilateral rib set, wherein K is an integer greater than 1 and less than N; performing plane fitting on each rib in the second unilateral rib set to obtain K fitting planes; and sorting the K ribs according to the positional relationship between the K fitting planes to obtain the unilateral rib sequence.

[0014] In a possible implementation, the method further includes: determining the plane normal vector and the rib centroid corresponding to each rib according to the fitting plane corresponding to each rib among the K ribs; for any two ribs R among the K ribs, i and R j , according to the rib R i The corresponding plane normal vector and rib centroid, as well as the rib R j The corresponding plane normal vector and rib centroid determine the rib R i and the rib R j The inner product between the ribs R i and the rib R j The size of the inner product between the ribs R i The corresponding fitting plane and the rib R j The positional relationship between the corresponding fitting planes.

[0015] In a possible implementation, determining multiple ribs in the rib image to be segmented and rib identifiers corresponding to each rib according to the semantic category segmentation result and the unilateral rib sequence includes: determining the rib identifier corresponding to a target rib in the unilateral rib sequence according to the semantic category segmentation result, where the target rib is any rib in the unilateral rib sequence; and determining multiple ribs in the rib image to be segmented and rib identifiers corresponding to each rib according to the rib identifier corresponding to the target rib and the unilateral rib sequence.

[0016] In a possible implementation, updating the target marking result according to the fine binary segmentation result includes: determining multiple connected components in the fine binary segmentation result; and updating the target marking result according to the multiple connected components.

[0017] In a possible implementation, the method further includes: performing adhesion detection on the target marking result based on a preset rib adhesion detection condition to determine whether there are adhesion ribs in the target marking result, where the adhesion ribs are used to indicate at least two adjacent ribs corresponding to the same rib identifier; determining the pixel point embedding vectors corresponding to the adhesion ribs when it is determined that there are adhesion ribs in the target marking result; and clustering the pixel point embedding vectors corresponding to the adhesion ribs to obtain the rib identifiers corresponding to each rib after the adhesion ribs are de-adhered.

[0018] According to one aspect of the present disclosure, there is provided an image processing apparatus, including: a semantic category segmentation module configured to perform semantic category segmentation on a rib image to be segmented to obtain a semantic category segmentation result corresponding to the rib image to be segmented; an instance segmentation module configured to perform instance segmentation on the rib image to be segmented to obtain an instance segmentation result corresponding to the rib image to be segmented; and a marking module configured to determine a target marking result corresponding to the rib image to be segmented according to the semantic category segmentation result and the instance segmentation result, where the target marking result includes multiple ribs in the rib image to be segmented and rib identifiers corresponding to each rib.

[0019] According to one aspect of the present disclosure, there is provided an electronic device, including: a processor; and a memory for storing instructions executable by the processor; where the processor is configured to call the instructions stored in the memory to execute the above method.

[0020] According to one aspect of the present disclosure, there is provided a computer-readable storage medium having computer program instructions stored thereon, and when the computer program instructions are executed by a processor, the above method is implemented.

[0021] In the embodiments of the present disclosure, based on the global semantic information of the image, semantic class segmentation is performed on the rib image to be segmented to obtain the semantic class segmentation result corresponding to the rib image to be segmented; based on the local geometric information of the image, instance segmentation is performed on the rib image to be segmented to obtain the instance segmentation result corresponding to the rib image to be segmented; since both the global semantic information of the image and the local geometric information of the image are comprehensively considered, therefore, based on the semantic class segmentation result and the instance segmentation result, the target marking result including multiple ribs in the rib image to be segmented and the rib identifier corresponding to each rib has a high accuracy, thereby effectively improving the segmentation and marking accuracy of the ribs.

[0022] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and do not limit the present disclosure. According to the following detailed description of the exemplary embodiments with reference to the accompanying drawings, other features and aspects of the present disclosure will become clear. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification, which illustrate embodiments consistent with the present disclosure and are used together with the specification to explain the technical solutions of the present disclosure.

[0024] Figure 1 The flowchart showing an image processing method according to an embodiment of the present disclosure;

[0025] Figure 2 The schematic diagram showing the z-axis classification of the initial rib image according to an embodiment of the present disclosure;

[0026] Figure 3 The schematic diagram showing the preset z-axis classification network according to an embodiment of the present disclosure;

[0027] Figure 4 The schematic diagram showing the convex hull region of the rib according to an embodiment of the present disclosure;

[0028] Figure 5 The schematic diagram showing the preset rib convex hull segmentation network according to an embodiment of the present disclosure;

[0029] Figure 6 The schematic diagram showing the semantic class segmentation result corresponding to the rib image to be segmented according to an embodiment of the present disclosure;

[0030] Figure 7 The schematic diagram showing the preset semantic class segmentation network according to an embodiment of the present disclosure;

[0031] Figure 8 The schematic diagram showing the instance segmentation result corresponding to the rib image to be segmented according to an embodiment of the present disclosure;

[0032] Figure 9Schematic diagram showing a preset rib instance segmentation network according to an embodiment of the present disclosure;

[0033] Figure 10 Schematic diagram showing a target marking result corresponding to a rib image to be segmented according to an embodiment of the present disclosure

[0034] Figure 11 Schematic diagram showing a fine binary segmentation result corresponding to a rib image to be segmented according to an embodiment of the present disclosure;

[0035] Figure 12 Schematic diagram showing a preset rib binary segmentation network according to an embodiment of the present disclosure;

[0036] Figure 13 Schematic diagram showing that there are adhered ribs in the target marking result according to an embodiment of the present disclosure;

[0037] Figure 14 Show according to an embodiment of the present disclosure Figure 13 Schematic diagram after de - adhering the adhered ribs shown;

[0038] Figure 15 Block diagram showing an image processing device according to an embodiment of the present disclosure;

[0039] Figure 16 Block diagram showing an electronic device according to an embodiment of the present disclosure;

[0040] Figure 17 Block diagram showing an electronic device according to an embodiment of the present disclosure. Detailed implementation manners

[0041] The following will detail various exemplary embodiments, features, and aspects of the present disclosure with reference to the accompanying drawings. The same reference numerals in the drawings denote elements having the same or similar functions. Although various aspects of the embodiments are shown in the drawings, the drawings do not have to be drawn to scale unless otherwise specified.

[0042] The special word "exemplary" herein means "serving as an example, embodiment, or illustration". Any embodiment described as "exemplary" here does not have to be construed as superior or better than other embodiments.

[0043] The term "and / or" herein merely describes an association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set composed of A, B, and C.

[0044] In addition, to better illustrate the present disclosure, numerous specific details are given in the following detailed implementation manners. Those skilled in the art should understand that the present disclosure can also be implemented without certain specific details. In some instances, methods, means, elements, and circuits well-known to those skilled in the art are not described in detail to highlight the gist of the present disclosure.

[0045] Figure 1 The flowchart of an image processing method according to an embodiment of the present disclosure is shown. This image processing method can be executed by an electronic device such as a terminal device or a server. The terminal device can be a user equipment (UE), a mobile device, a user terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. This image processing method can be implemented by a processor invoking computer-readable instructions stored in a memory. Alternatively, the image processing method can be executed by a server. As Figure 1 shown, this image processing method may include:

[0046] In step S11, semantic category segmentation is performed on the rib image to be segmented to obtain a semantic category segmentation result corresponding to the rib image to be segmented.

[0047] In step S12, instance segmentation is performed on the rib image to be segmented to obtain an instance segmentation result corresponding to the rib image to be segmented.

[0048] In step S13, based on the semantic category segmentation result and the instance segmentation result, a target marking result corresponding to the rib image to be segmented is determined, where the target marking result includes multiple ribs in the rib image to be segmented and a rib identifier corresponding to each rib.

[0049] In the embodiment of the present disclosure, based on the global semantic information of the image, semantic category segmentation is performed on the rib image to be segmented to obtain a semantic category segmentation result corresponding to the rib image to be segmented; based on the local geometric information of the image, instance segmentation is performed on the rib image to be segmented to obtain an instance segmentation result corresponding to the rib image to be segmented; since both the global semantic information and the local geometric information of the image are comprehensively considered, the target marking result including multiple ribs in the rib image to be segmented and the rib identifier corresponding to each rib determined based on the semantic category segmentation result and the instance segmentation result has a high accuracy, thereby effectively improving the segmentation and marking accuracy of the ribs.

[0050] In a possible implementation, the image processing method further includes: obtaining an original chest scan image; performing image preprocessing on the original chest scan image to obtain an initial rib image; performing convex hull segmentation on the initial rib image to obtain a rib convex hull area in the initial rib image; and cropping the initial rib image according to the rib convex hull area to obtain a rib image to be segmented.

[0051] After performing image preprocessing on the original chest scan image to obtain an initial rib image, the rib convex hull area in the initial rib image can be determined by performing convex hull segmentation on the initial rib image. Thus, the rib image to be segmented can be cropped from the initial rib image according to the rib convex hull area, so that subsequent rib marking based on the cropped rib image to be segmented can effectively reduce the consumption of computing resources and improve the rib marking efficiency.

[0052] In a possible implementation, the original chest scan image can be a chest computed tomography (CT) image I. Since CT images have good bone-soft tissue contrast, chest CT image I is usually used as a medical image for rib fracture diagnosis.

[0053] Performing image preprocessing on chest CT image I to obtain an initial rib image I n . The image preprocessing can include one or more of redirection, cropping, normalization, etc. The present disclosure does not limit the specific manner of image preprocessing.

[0054] Due to different shooting angles, the rib directions in different chest CT images may be different. Performing preprocessing of redirecting chest CT image I according to a preset unit matrix to obtain an initial rib image I n so that the rib direction in the initial rib image I n is consistent with the preset coordinate axes (x / y / z axes) to improve the subsequent processing efficiency. The preset unit matrix can be set according to the actual situation. The present disclosure does not limit the specific form of the preset unit matrix.

[0055] In addition to ribs, chest CT image I also includes a large area of other background parts. To reduce the subsequent consumption of computing resources and improve the processing efficiency, chest CT image I is cropped using a preset gray threshold. Specifically, it can include: performing binary processing on chest CT image I based on the preset gray threshold to obtain a binary image. Among them, for the pixel points in chest CT image I whose gray values are greater than or equal to the preset gray threshold, the corresponding pixel values in the binary image are 1; for the pixel points in chest CT image I whose gray values are less than the preset gray threshold, the corresponding pixel values in the binary image are 0. Cropping chest CT image I according to the bounding box formed by the pixel points with pixel values of 1 in the binary image to obtain an initial rib image I n, to reduce the image size, thereby effectively reducing the subsequent consumption of computing resources and improving the processing efficiency. The specific value of the preset gray threshold can be set according to the actual situation, and the present disclosure does not make specific limitations thereon.

[0056] In practical applications, when a patient wearing metal or other items is photographed with a chest CT image I, there will be pixel points with too high gray values in the photographed chest CT image I, which affects the subsequent processing accuracy. Therefore, for the initial rib image I obtained by cropping the above-mentioned chest CT image I n , use the preset gray value normalization window for normalization processing so that the initial rib image I n has gray values within a reasonable gray value range, improving the rib marking accuracy. The preset gray value normalization window can be set according to the actual situation. For example, the preset gray value normalization window is [-1000, 2000], and the present disclosure does not make specific limitations on the actual value range of the preset gray value normalization window.

[0057] In a possible implementation manner, perform convex hull segmentation on the initial rib image to obtain the rib convex hull region in the initial rib image, including: determining whether the rib region in the initial rib image meets the preset integrity requirement; in the case where the rib region in the initial rib image meets the preset integrity requirement, perform convex hull segmentation on the initial rib image to obtain the rib convex hull region.

[0058] After performing image preprocessing on the chest CT image I to obtain the initial rib image I n , in order to improve the subsequent rib marking accuracy, the initial rib image I n can be first subjected to quality detection to detect whether the rib region in the initial rib image I n meets the preset integrity requirement. Since in the case where the rib region in the initial rib image I n does not meet the preset integrity requirement, the subsequent rib marking process cannot be completed. Therefore, only in the case where the rib region in the initial rib image I n meets the preset integrity requirement, the initial rib image I n is subjected to convex hull segmentation to obtain the rib convex hull region.

[0059] Among them, the rib region in the initial rib image I n meeting the preset integrity requirement may mean that the rib region in the initial rib image is relatively complete and there is no serious missing situation. The present disclosure does not make specific limitations thereon.

[0060] In a possible implementation, the initial rib image includes a plurality of slice images; determining whether the rib region in the initial rib image meets a preset integrity requirement includes: determining the rib category of each slice image in the plurality of slice images, where different rib categories are used to indicate different rib regions; determining a rib category sequence according to the rib category of each slice image; and determining that the rib region in the initial rib image meets the preset integrity requirement when the preset rib category is included in the rib category sequence.

[0061] According to the distribution of the complete rib regions, the rib regions can be divided into a plurality of rib regions, and each rib region can correspond to a rib category. Therefore, by determining the rib category corresponding to each slice image in the initial rib image I n and then determining the rib category sequence corresponding to the initial rib image I n when the preset rib category is included in the rib category sequence, it can be determined that the initial rib image I n includes most of the rib regions, that is, the rib region in the initial rib image I n is relatively complete and there is no serious missing situation, and the rib region in the initial rib image I n meets the preset integrity requirement.

[0062] Since the resolutions of different initial rib images may be different, in order to improve the subsequent processing efficiency, the initial rib image I n can be resampled to obtain a second rib image, where the resolution of the second rib image is the third resolution. The specific value of the third resolution can be determined according to the actual situation, and the present disclosure does not make a specific limitation thereto. For example, the third resolution is 1mm×1mm×3mm, that is, the actual physical size corresponding to each pixel point in the second rib image is 1mm×1mm×3mm.

[0063] Since the initial rib image I n is obtained by preprocessing the chest CT image I, and the chest CT image I includes a plurality of slice images, therefore, the initial rib image I n and the second rib image obtained by resampling the initial rib image I n also include a plurality of slice images.

[0064] In an example, the rib category of each slice image can be determined based on a preset z-axis classification network. Traverse the plurality of slice images included in the second rib image, and input each slice image into the preset z-axis classification network respectively, so that the rib category of each slice image can be obtained, and different rib categories are used to indicate different rib regions.

[0065] Figure 2Schematic diagram showing z - axis classification of an initial rib image according to an embodiment of the present disclosure. As Figure 2 shown, the complete rib region can include 5 rib categories (category 0, category 1, category 2, category 3, and category 4). Based on a preset z - axis classification network, the rib category of each slice image can be determined; according to the rib category of the slice image, it can be determined which rib region the slice image is located in. As Figure 2 shown, the rib category q i of slice image i j is category 1, and the rib category q

[0066] of slice image j 1 is category 3. 2 ,...,q d ), q i ∈{category 0, category 1, category 2, category 3, category 4}, i ∈ [0, d], where d is the total number of slice images. According to the number of each slice image, the rib category sequence Q is processed into a monotonically ordered rib category sequence

[0067] When the preset rib category is included in the rib category sequence , it can be determined that the rib region in the initial rib image I n meets the preset integrity requirements. For example, the preset rib categories include category 1, category 2, and category 3. As Figure 2 shown, when the rib categories include category 1, category 2, and category 3, it can be determined that the rib region in the initial rib image I n is relatively complete and there is no serious missing situation.

[0068] In a possible implementation, the preset z - axis classification network can include several convolutional layers, downsampling layers, global average pooling layers, and fully - connected layers. Figure 3 Schematic diagram showing the preset z - axis classification network according to an embodiment of the present disclosure.

[0069] When training the preset z - axis classification network, a learning rate setting strategy of warmup and cosine annealing can be adopted, and a cross - entropy loss function can be used to train the preset z - axis classification network for several training rounds.

[0070] For example, the training of the preset z - axis classification network can be achieved by minimizing the loss function L 1 shown in the following formula (1).

[0071]

[0072] Among them, y in formula (1) i is the classification label corresponding to the training sample image i, and is the classification prediction probability corresponding to the training sample image i determined according to the preset z-axis classification network.

[0073] Those skilled in the art should understand that the specific network structure and training process of the preset z-axis classification network can adopt other network structures and training methods in related technologies, and the present disclosure does not make specific limitations thereto.

[0074] Since the scanning area of the chest CT image I is very large, the chest CT image I also includes other bones such as vertebrae, hip bones, and femurs, and the rib area only accounts for a very small part of it. Therefore, when the rib area in the initial rib image I n meets the preset integrity requirements, the initial rib image I n can be subjected to convex hull segmentation to obtain the rib convex hull area in the initial rib image I n , and then based on the rib convex hull area, the initial rib image I n can be cropped to obtain the rib image I to be segmented that can include the rib area v . Subsequently, rib marking based on the rib image I to be segmented v can reduce the waste of computing resources and improve the rib marking efficiency.

[0075] In one example, based on a preset rib convex hull segmentation network, the initial rib image I n can be segmented to obtain the rib convex hull area in the initial rib image I n . To improve the segmentation efficiency, the initial rib image I n can be resampled to obtain the first rib image I sp3 , where the resolution of the first rib image I sp3 is the first resolution. The specific value of the first resolution can be determined according to the actual situation, and the present disclosure does not make specific limitations thereto. For example, the first resolution is 3mm×3mm×3mm, that is, the actual physical size corresponding to each pixel point in the first rib image I sp3 is 3mm×3mm×3mm.

[0076] Input the first rib image I sp3 into the preset rib convex hull segmentation network. After the preset rib convex hull segmentation network segments the first rib image I sp3 , the rib convex hull area is obtained. Figure 4 Shows a schematic diagram of the rib convex hull area according to an embodiment of the present disclosure. As Figure 4 shown, the rib convex hull area includes the rib convex hull area H corresponding to the left ribs l, and the rib convex hull area H corresponding to the right rib r .

[0077] According to the rib convex hull area, a detection frame including the overall rib area can be determined. For example, the detection frame can be the minimum rectangular frame including the rib convex hull area H l and the rib convex hull area H r . According to this detection frame, the initial rib image I n is cropped to obtain the rib image I to be segmented including the overall rib area v . Based on the rib image I to be segmented v , subsequent rib marking is performed.

[0078] In a possible implementation, the preset rib convex hull segmentation network can be a 3D-U-shaped network, including an encoder composed of several convolutional layers and downsampling layers, and a decoder composed of several convolutional layers and upsampling layers. A non-local module is embedded between the encoder and the decoder, and skip connections are introduced in the corresponding stages of the encoder and the decoder. Figure 5 The schematic diagram of the preset rib convex hull segmentation network according to the embodiment of the present disclosure is shown.

[0079] When training the preset rib convex hull segmentation network, a learning rate setting strategy of warmup and cosine annealing can be adopted, and cross entropy (cross-entropy loss) and dice loss are used as loss functions to train the preset rib convex hull segmentation network for several training rounds.

[0080] For example, the training of the preset rib convex hull segmentation network can be realized by minimizing the loss function L shown in the following formula (2) 2 .

[0081]

[0082] where y in formula (2) i is the segmentation label corresponding to the training sample image i, is the segmentation prediction probability corresponding to the training sample image i determined according to the preset rib convex hull segmentation network, Y is the true segmentation result corresponding to the training sample image i, is the predicted segmentation result corresponding to the training sample image i determined according to the preset rib convex hull segmentation network.

[0083] Those skilled in the art should understand that the specific network structure and training process of the preset rib convex hull segmentation network can adopt other network structures and training methods in related technologies, and the present disclosure does not make specific limitations thereto.

[0084] In one example, a convex hull algorithm can be used to perform convex hull segmentation on the initial rib image I n to obtain the convex hull region of the ribs in the initial rib image I n . The specific algorithm form of the convex hull algorithm can be flexibly set according to the actual situation, and the present disclosure does not make specific limitations thereon.

[0085] In a possible implementation, semantic class segmentation is performed on the rib image I to be segmented v to obtain the semantic class segmentation result A corresponding to the rib image I to be segmented v , including: resampling the rib image I to be segmented l to obtain the first rib image I v , where the resolution of the first rib image I sp3 is the first resolution; using a preset semantic class segmentation network to perform semantic class segmentation on the first rib image I sp3 to obtain the semantic class segmentation result A corresponding to the rib image I to be segmented sp3 . v l .

[0086] Performing semantic class segmentation on the rib image I to be segmented at the first resolution can effectively improve the semantic class segmentation efficiency. v Figure 6 FIG. shows a schematic diagram of the semantic class segmentation result corresponding to the rib image to be segmented according to an embodiment of the present disclosure. As Figure 6 shown, the semantic class segmentation result A l includes multiple ribs in the rib image I to be segmented v and the first predicted rib identifier corresponding to each rib. In Figure 6 , different colors can be used to indicate each rib and the first predicted rib identifier corresponding to each rib. For example, red is used to indicate the right rib No. 1, purple is used to indicate the left rib No. 2, and so on. Those skilled in the art should understand that other forms in related technologies can be used in the semantic class segmentation result to indicate each rib and the first predicted rib identifier corresponding to each rib, and the present disclosure does not make specific limitations thereon.

[0087] In a possible implementation, the network structure and training method of the preset semantic class segmentation network can be the same as those of the preset rib convex hull segmentation network. Figure 7 FIG. shows a schematic diagram of the preset semantic class segmentation network according to an embodiment of the present disclosure.

[0088] Those skilled in the art should understand that the specific network structure and training process of the preset semantic class segmentation network can adopt other network structures and training methods in related technologies, and the present disclosure does not make specific limitations thereon.​​

[0089] In a possible implementation, instance segmentation is performed on the rib image to be segmented, and an instance segmentation result corresponding to the rib image to be segmented is obtained, including: resampling the rib image to be segmented to obtain a first rib image, where the resolution of the first rib image is a first resolution; performing position encoding on the pixel points in the first rib image to obtain a position encoding image; and performing instance segmentation on the first rib image based on the position encoding image to obtain an instance segmentation result.

[0090] Segmenting the rib image I to be segmented at the first resolution v can effectively improve the instance segmentation efficiency. Performing position encoding on the first rib image I at the first resolution sp3 and, based on the position encoding image obtained after position encoding, performing instance segmentation on the first rib image I sp3 can effectively improve the instance segmentation accuracy.

[0091] In one example, the following formula (3) can be used to perform position encoding on the first rib image I sp3 to obtain a position encoding image I c ,

[0092]

[0093] where (i, j, k) are the corresponding pixel points in the first rib image I sp3 and the position encoding image I c , (μ x , μ y , μ z ) is the central pixel point of the first rib image I sp3 , and w x , w y , and w z are preset hyperparameters. The specific values of the preset hyperparameters w x , w y , and w z can be determined according to the actual situation, and the present disclosure does not make specific limitations thereon.

[0094] In a possible implementation, based on the position encoding image, instance segmentation is performed on the first rib image to obtain an instance segmentation result, including: performing binary segmentation on the first rib image to obtain a target rough binary segmentation result corresponding to the first rib image; determining a pixel embedding vector corresponding to the first rib image based on the position encoding image; determining a pixel embedding vector corresponding to the rib region in the first rib image according to the target rough binary segmentation result and the pixel embedding vector corresponding to the first rib image; and clustering the pixel embedding vectors corresponding to the rib region in the first rib image to obtain an instance segmentation result.

[0095] In a possible implementation, binary segmentation is performed on the first rib image to obtain a target rough binary segmentation result corresponding to the first rib image, including: performing binary segmentation on the first rib image to obtain an initial rough binary segmentation result corresponding to the first rib image; filtering the non-rib regions in the initial rough binary segmentation result according to the rib convex hull region to obtain the target rough binary segmentation result.

[0096] In one example, based on a preset rib instance segmentation network, instance segmentation can be performed on the first rib image I sp3 The first rib image I sp3 and the position encoding image I c are simultaneously input into the preset rib instance segmentation network.

[0097] The preset rib instance segmentation network includes two branches. One branch is used to perform binary segmentation on the first rib image I sp3 to obtain an initial rough binary segmentation result A sp3 corresponding to the first rib image I bc . Using the rib convex hull regions H l and H r , the initial rough binary segmentation result A bc is filtered to eliminate the false positive parts mis-segmented as rib regions in the initial rough binary segmentation result A bc to obtain a target rough binary segmentation result A bc with higher accuracy. For example, the target rough binary segmentation result A bc = A bc ∩(H l ∪H r ).

[0098] Another branch of the preset rib instance segmentation network is used to determine the pixel point embedding vector A c corresponding to the first rib image I sp3 based on the position encoding image I e . The pixel point embedding vector A e includes an embedding vector corresponding to each pixel point in the first rib image I sp3 . The dimension of the embedding vector can be 8-dimensional, or can be set to other dimensions according to the actual situation, and the present disclosure does not make specific limitations on this.

[0099] Using the target rough binary segmentation result A bc and the pixel point embedding vector A e , the pixel point embedding vector A sp3 corresponding to the rib region in the first rib image I re = A bc ∩A eThen, the mean-shift clustering algorithm is used to cluster the pixel point embedding vectors A corresponding to the rib region re to obtain the rib image I to be segmented v and the corresponding instance segmentation result A ins Those skilled in the art should understand that in addition to the mean-shift clustering algorithm, other clustering algorithms in related technologies can also be used for the clustering algorithm, and the present disclosure does not make specific limitations thereon

[0100] Figure 8 FIG. shows a schematic diagram of the instance segmentation result corresponding to the rib image to be segmented according to an embodiment of the present disclosure. As Figure 8 shown, the instance segmentation result A ins includes multiple ribs in the rib image I to be segmented v However, the instance segmentation result A ins cannot determine the rib identification of each rib

[0101] Figure 9 sp3 In a possible implementation manner, the preset rib instance segmentation network may be a 3D-U-shaped network, where two encoder branches share a decoder, and a non-local module is embedded between the encoder and the decoder. One of the encoder branches is used to perform binary segmentation on the first rib image I sp3 to obtain the initial rough binary segmentation result A corresponding to the first rib image I bc , and the other encoder branch is used to determine the pixel point embedding vector A corresponding to the first rib image I c based on the position-encoded image I sp3 e Figure 9 FIG. shows a schematic diagram of the preset rib instance segmentation network according to an embodiment of the present disclosure

[0102] When training the preset rib instance segmentation network, the encoder branch for binary segmentation can use cross entropy (cross-entropy loss) and dice loss as loss functions for training for several training rounds. The specific training formula can refer to the above formula (2), which will not be elaborated here

[0103] The encoder branch for embedding vector prediction can use discriminative loss as the loss function for training for several training rounds

[0104] For example, the training of the encoder branch for embedding vector prediction can be achieved by minimizing the loss function L shown in the following formula (4) d

[0105] ​​​

[0106] Among them, C in formula (4) is the total number of categories in the training sample images, N c is the number of pixel points belonging to the same category in the training sample images, μ c is the mean vector of the category, μ c is the embedding vector of pixel point i in the training sample images.

[0107] Those skilled in the art should understand that the specific network structure and training process of the preset rib instance segmentation network can adopt other network structures and training methods in related technologies, and the present disclosure does not make specific limitations thereto.

[0108] The semantic category segmentation result corresponding to the rib image to be segmented is obtained based on the global information of the image, and multiple ribs in the rib image to be segmented and the first predicted rib identifier corresponding to each rib can be obtained. However, due to possible missegmentation between adjacent ribs in semantic category segmentation, the accuracy of the first predicted rib identifier is relatively low. The instance segmentation result corresponding to the rib image to be segmented is obtained based on the local information of the image. Although the rib identifier of each rib cannot be obtained in the instance segmentation result, the instance segmentation result can well separate adjacent ribs. Therefore, by comprehensively considering the semantic category segmentation result and the instance segmentation result, the rib identifier of each rib can be obtained more accurately.

[0109] In a possible implementation manner, determining the target marking result corresponding to the rib image to be segmented according to the semantic category segmentation result and the instance segmentation result includes: determining a unilateral rib sequence according to the instance segmentation result and the rib convex hull region, where the unilateral rib sequence includes multiple unilateral ribs and their sorting, and the unilateral rib sequence includes a left rib sequence and a right rib sequence; determining multiple ribs in the rib image to be segmented and the rib identifier corresponding to each rib according to the semantic category segmentation result and the unilateral rib sequence.

[0110] In a possible implementation manner, determining the unilateral rib sequence according to the instance segmentation result and the rib convex hull region includes: determining a first unilateral rib set according to the instance segmentation result and the rib convex hull region, where the first unilateral rib sequence includes N unilateral ribs, and N is the number of unilateral ribs; sorting the N unilateral ribs in the z-axis direction, and taking the first K ribs before sorting to form a second unilateral rib set, where K is an integer greater than 1 and less than N; performing plane fitting on each rib in the second unilateral rib set to obtain K fitting planes; sorting the K ribs according to the positional relationship between the K fitting planes to obtain the unilateral rib sequence.

[0111] According to the rib convex hull region H l and H r , the instance segmentation result A insThe rib instances in the figure are divided into two first unilateral rib sets: the first unilateral rib set F corresponding to the left side l =A ins ∩H l , and the first unilateral rib set F corresponding to the right side r =A ins ∩H r .

[0112] Take the first unilateral rib set F on the left side l For example, the first unilateral rib set F l The first unilateral rib set F includes N unilateral ribs, where N is the number of unilateral ribs, for example, N=12. l The N unilateral ribs in the z-axis direction are sorted, and the first K ribs are selected to form the second unilateral rib set S l =(R 1 ,R 2 ,...,R K ). Wherein, K is a hyperparameter, K is an integer greater than 1 and less than N, and the specific value of K can be determined according to actual conditions, and the present disclosure does not make any specific limitation on this.

[0113] For the first unilateral rib set F l The ordering of the N unilateral ribs in the z-axis direction may be based on the z-axis 95th percentile, the z-axis 90th percentile, the z-axis 85th percentile, etc., and the present disclosure does not make any specific limitation on this.

[0114] Plane fitting is performed on each rib in the second unilateral rib set to determine a fitting plane corresponding to each rib.

[0115] In a possible implementation, the image processing method further includes: determining the plane normal vector and the rib centroid corresponding to each rib according to the fitting plane corresponding to each rib among the K ribs; for any two ribs R among the K ribs, i and R j , according to rib R i The corresponding plane normal vector and rib centroid, as well as the rib R j The corresponding plane normal vector and rib centroid determine the rib R i and rib R j The inner product between the ribs R i and rib R j The size of the inner product between is used to indicate the rib R i Corresponding fitting plane and rib R j The positional relationship between the corresponding fitting planes.

[0116] For any two ribs R i , R j∈S l Perform a plane fitting on the rib R i to obtain the plane normal vector n i corresponding to the rib R i and the centroid m of the rib bone i ; Perform a plane fitting on the rib R j to obtain the plane normal vector n j corresponding to the rib R j and the centroid m of the rib bone j . Taking the plane where the rib R i is located as a reference, that is, the direction of the plane normal vector of the rib R i points to the direction where the human head is located, determine the inner product p i between the rib R j and the rib R j,i according to the following formula (5).

[0117] p j,i =(m j -m i )·n i (5).

[0118] If the inner product p j,i is positive, the fitting plane of the rib R j is above the fitting plane of the rib R i , that is, the rib R j is above the rib R i . If the inner product p j,i is negative, the fitting plane of the rib R j is below the fitting plane of the rib R i , that is, the rib R j is above the rib R i .

[0119] In an example, taking the plane where the rib R i is located as a reference, if the inner product p i between the rib R x and the rib R x,i is positive, and the inner product p i between the rib R y and the rib R y,i is also positive, and the inner product p x,i is greater than the inner product p y,i , then the order of the rib R i , the rib R x and the rib R y from top to bottom is R x >R y >R i .

[0120] In an example, for any rib R i ∈Sl , taking the plane where the rib R i is located as a reference, determine the inner product of the rib R i and the remaining K - 1 ribs, and based on the inner product of the rib R i and the remaining K - 1 ribs, sort the K ribs to obtain the rib sequence S i corresponding to the rib R i . Traverse the K ribs in the second set of unilateral ribs S l to obtain K rib sequences. According to the sorting of each rib in the K rib sequences, vote to obtain the final unilateral rib sequence S. For example, for the rib R i , count the sorting in which it appears the most times in the K rib sequences as its final sorting in the final unilateral rib sequence S.

[0121] And so on, for the first set of unilateral ribs F r on the right, the corresponding unilateral rib sequence S can also be obtained in the above - mentioned manner.

[0122] In a possible implementation manner, based on the semantic category segmentation result and the unilateral rib sequence, determine multiple ribs in the rib image to be segmented and the rib identifier corresponding to each rib, including: according to the semantic category segmentation result, determine the rib identifier corresponding to the target rib in the unilateral rib sequence, where the target rib is any rib in the unilateral rib sequence; according to the rib identifier corresponding to the target rib and the unilateral rib sequence, determine multiple ribs in the rib image to be segmented and the rib identifier corresponding to each rib.

[0123] The unilateral rib sequence includes an ordered rib sequence. Therefore, after determining the rib identifier corresponding to any target rib in the unilateral rib sequence based on the semantic category segmentation result, based on the sorting of the unilateral rib sequence, the rib identifier corresponding to each rib in the rib image I v to be segmented can be obtained.

[0124] In an example, the unilateral rib sequence S=(R 1 , R 2 ,..., R K ), then its corresponding rib identifier is (q * , q * + 1,..., q * + k - 1), where

[0125] Perform the above - mentioned processing on the unilateral rib sequences on the left and right sides respectively to obtain the target marking result A v corresponding to the rib image I c . Figure 10Schematic diagram showing a target marking result corresponding to a rib image to be segmented according to an embodiment of the present disclosure. As Figure 10 shown, the target marking result A c includes multiple ribs in the rib image I to be segmented v and the rib identifier corresponding to each rib. In Figure 10 , different colors can be used to indicate each rib and the rib identifier corresponding to each rib. For example, red is used to indicate the right rib No. 1, purple is used to indicate the second rib on the left, and so on. Those skilled in the art should understand that other forms in the related art can be used in the target marking result A c to indicate each rib and the rib identifier corresponding to each rib, and the present disclosure does not make specific limitations on this. Figure 10 The target marking result A in c Compared with Figure 6 the semantic class segmentation result A in l , the rib identifier of each rib is more accurate.

[0126] The target marking result obtained by combining the global image information and the local image information can improve the rib marking accuracy in scenarios such as rib fractures, severe fractures, and rib dislocations.

[0127] In a possible implementation, the image processing method further includes: when the semantic class segmentation and the instance segmentation are performed at a first resolution, performing binary segmentation on the rib image to be segmented at a second resolution to obtain a fine binary segmentation result corresponding to the rib image to be segmented, where the second resolution is greater than the first resolution; updating the target marking result according to the fine binary segmentation result.

[0128] When the semantic class segmentation and the instance segmentation of the rib image to be segmented are performed at a first resolution, the segmentation result of each rib in the target marking result is relatively rough. Therefore, based on the fine binary segmentation result with a higher resolution, the target marking result is updated to obtain a target marking result with a higher resolution and a higher segmentation accuracy.

[0129] In a possible implementation, performing binary segmentation on the rib image I to be segmented at a second resolution v to obtain a fine binary segmentation result A corresponding to the rib image I to be segmented v , including: resampling the rib image I to be segmented bf to obtain a third rib image I v , where the resolution of the third rib image I sp1.5 is the second resolution; using a preset rib binary segmentation network to perform binary segmentation on the third rib image I sp1.5 to obtain the rib image I to be segmented sp1.5 ​v The corresponding fine binary segmentation result A bf . For example, the second resolution is 1.5 mm × 1.5 mm × 1.5 mm, that is, each pixel point in the third rib image I sp1.5 corresponds to an actual physical size of 1.5 mm × 1.5 mm × 1.5 mm.

[0130] The third rib image I sp1.5 includes multiple slice images. Using the preset rib binary segmentation network, traverse each slice image in the third rib image I sp1.5 to determine the predicted binary segmentation result corresponding to each slice image, and then stack the predicted binary segmentation results of all slice images to obtain the rib image I to be segmented v The corresponding fine binary segmentation result A bf . Figure 11 A schematic diagram showing the fine binary segmentation result corresponding to the rib image to be segmented according to an embodiment of the present disclosure. As Figure 11 shown, the fine binary segmentation result A bf includes multiple ribs in the rib image I to be segmented v . However, the fine binary segmentation result A bf cannot determine the rib identifier of each rib.

[0131] In a possible implementation manner, the preset rib binary segmentation network may be a 2.5D Unet network, including an encoder composed of several convolutional layers and downsampling layers, and a decoder composed of several convolutional layers and upsampling layers. A non-local module is embedded between the encoder and the decoder, and skip connections are introduced in the corresponding stages of the encoder and the decoder. Figure 12 A schematic diagram showing the preset rib binary segmentation network according to an embodiment of the present disclosure.

[0132] When training the preset rib binary segmentation network, a learning rate setting strategy of warmup and cosine annealing can be adopted, and cross entropy and dice loss are used as loss functions to train the preset rib binary segmentation network for several training rounds. The specific training formula can refer to the above formula (2), which will not be elaborated here.

[0133] Those skilled in the art should understand that the specific network structure and training process of the preset rib binary segmentation network can adopt other network structures and training methods in related technologies, and the present disclosure does not make specific limitations on this.

[0134] In a possible implementation manner, according to the fine binary segmentation result, update the target marking result, including: determining multiple connected components in the fine binary segmentation result; updating the target marking result according to the multiple connected components.

[0135] Determine the fine binary segmentation result A bf among multiple connected components {C i}, for any connected component C i , update its rib identification to Traverse the fine binary segmentation result A bf among each connected component to obtain the updated target marking result A f .

[0136] Figure 13 The figure shows a schematic diagram of the existence of adhered ribs in the target marking result according to an embodiment of the present disclosure. As Figure 13 shown, there may be adhesion between adjacent ribs in the target marking result, resulting in the adhered ribs including at least two adjacent ribs being indicated by the same color in the target marking result, that is, the adhered ribs including at least two adjacent ribs are determined to have the same rib identification, resulting in inaccurate rib marking.

[0137] In a possible implementation manner, the image processing method further includes: based on a preset rib adhesion detection condition, performing adhesion detection on the target marking result to determine whether there are adhered ribs in the target marking result, where the adhered ribs are used to indicate at least two adjacent ribs corresponding to the same rib identification; in the case of determining that there are adhered ribs in the target marking result, determining the pixel point embedding vector corresponding to the adhered ribs; clustering the pixel point embedding vectors corresponding to the adhered ribs to obtain the rib identification corresponding to each rib after de-adhesion of the adhered ribs.

[0138] The preset rib adhesion detection condition may at least include the following four conditions:

[0139] a), relative volume prior condition; for any rib R in the target marking result i , if then rib R i can be determined as a suspiciously adhered rib. Among them, is the average relative volume of rib R i , V r (R i ) can be obtained through big data statistics, |·| is used to indicate volume, N is the data statistic corresponding to the big data statistics process, K is the number of ribs in the rib image to be segmented, T 1 is a hyperparameter, and the specific values of N and T 1 can be determined according to the actual situation, and the present disclosure does not make specific limitations on this.

[0140] b), rib sequence discontinuity condition 1; for any two adjacent ribs in the target marking result and if Si +1 < S i+1 , it can be determined that and are suspected of being adhered ribs.

[0141] c), Rib sequence discontinuity condition 2; For any three adjacent ribs in the target marking result If then it can be determined that the rib and are suspected of being adhered ribs. Among them, T 2 is a hyperparameter, and the specific value of T 2 can be determined according to the actual situation, and the present disclosure does not make specific limitations on this.

[0142] d) Category condition; For any rib R in the target marking result i , if rib R i corresponds to multiple rib identifiers in the target marking result before being updated using the fine binary segmentation result, and the volume ratio of a certain rib identifier is less than 0.8, then it can be determined that rib R i is a rib with impure category.

[0143] For any rib R in the target marking result i , if it satisfies condition d) and satisfies any one of conditions a), b), and c), then it can be determined that rib R i is an adhered rib. Traverse each rib in the target marking result to obtain the adhered rib set M. Those skilled in the art should understand that the adhesion detection conditions can include other forms in addition to the above four, and the present disclosure does not make specific limitations on this.

[0144] For any rib R i ∈M, determine the pixel point embedding vector A corresponding to rib R i = A i ∩R e , and use the mean-shift clustering algorithm for the pixel point embedding vector A corresponding to rib R i , to obtain the clustering instance set {O i}. For any clustering instance O i , determine its corresponding rib identifier as j} Thus, the de-adhesion of the adhered ribs is realized, and the accuracy of rib marking is improved. j , and determine its corresponding rib identifier as Thereby realizing the de-adhesion of the adhered ribs and improving the accuracy of rib marking.

[0145] Figure 14 Shows a schematic diagram of the de-adhesion of the adhered ribs shown in Figure 13 according to an embodiment of the present disclosure. Figure 14The adhered ribs indicated by the same color in Figure 13 are indicated by different colors in

[0146] It can be understood that, for the above-mentioned various method embodiments mentioned in the present disclosure, without violating the principle logic, they can be combined with each other to form an embodiment after combination. Due to space limitations, the present disclosure will not elaborate further. Those skilled in the art can understand that in the above methods of the specific implementation manner, the specific execution order of each step should be determined according to its function and possible internal logic.

[0147] In addition, the present disclosure also provides an image processing device, an electronic device, a computer-readable storage medium, and a program, all of which can be used to implement any one of the image processing methods provided by the present disclosure. The corresponding technical solutions and descriptions are referred to the corresponding records in the method part and will not be elaborated further.

[0148] Figure 15 The block diagram of an image processing device according to an embodiment of the present disclosure is shown. As Figure 15 shown, the device 1500 includes:

[0149] A semantic class segmentation module 1501, configured to perform semantic class segmentation on the rib image to be segmented, and obtain a semantic class segmentation result corresponding to the rib image to be segmented;

[0150] An instance segmentation module 1502, configured to perform instance segmentation on the rib image to be segmented, and obtain an instance segmentation result corresponding to the rib image to be segmented;

[0151] A marking module 1503, configured to determine a target marking result corresponding to the rib image to be segmented according to the semantic class segmentation result and the instance segmentation result, where the target marking result includes multiple ribs in the rib image to be segmented and a rib identifier corresponding to each rib.

[0152] In a possible implementation manner, the device 1500 further includes:

[0153] A fine binary segmentation module, configured to perform binary segmentation on the rib image to be segmented at a second resolution to obtain a fine binary segmentation result corresponding to the rib image to be segmented when the semantic class segmentation and the instance segmentation are performed at a first resolution, where the second resolution is greater than the first resolution;

[0154] An update module, configured to update the target marking result according to the fine binary segmentation result.

[0155] In a possible implementation manner, the device 1500 further includes:

[0156] An acquisition module, configured to acquire an original chest scan image;

[0157] An image preprocessing module for preprocessing the original chest scan image to obtain an initial rib image;

[0158] A convex hull segmentation module for performing convex hull segmentation on the initial rib image to obtain the rib convex hull region in the initial rib image;

[0159] A cropping module for cropping the initial rib image according to the rib convex hull region to obtain the rib image to be segmented.

[0160] In a possible implementation, the convex hull segmentation module includes:

[0161] A first determination sub-module for determining whether the rib region in the initial rib image meets the preset integrity requirements;

[0162] A convex hull segmentation sub-module for performing convex hull segmentation on the initial rib image to obtain the rib convex hull region when the rib region in the initial rib image meets the preset integrity requirements.

[0163] In a possible implementation, the initial rib image includes multiple slice images;

[0164] The first determination sub-module includes:

[0165] A first determination unit for determining the rib category of each slice image in the multiple slice images, where different rib categories are used to indicate different rib regions;

[0166] A second determination unit for determining the rib category sequence according to the rib category of each slice image;

[0167] A third determination unit for determining that the rib region in the initial rib image meets the preset integrity requirements when the rib category sequence includes a preset rib category.

[0168] In a possible implementation, the instance segmentation module 1502 includes:

[0169] A resampling sub-module for resampling the rib image to be segmented to obtain a first rib image, where the resolution of the first rib image is the first resolution;

[0170] A position encoding sub-module for performing position encoding on the pixel points in the first rib image to obtain a position encoding image;

[0171] An instance segmentation sub-module for performing instance segmentation on the first rib image based on the position encoding image to obtain an instance segmentation result.

[0172] In a possible implementation, the instance segmentation sub-module includes:

[0173] A binary segmentation unit, configured to perform binary segmentation on the first rib image to obtain a target rough binary segmentation result corresponding to the first rib image;

[0174] A fourth determination unit, configured to determine a pixel embedding vector corresponding to the first rib image based on the position encoding image;

[0175] A fifth determination unit, configured to determine a pixel embedding vector corresponding to the rib region in the first rib image according to the target rough binary segmentation result and the pixel embedding vector corresponding to the first rib image;

[0176] A clustering unit, configured to cluster the pixel embedding vectors corresponding to the rib region in the first rib image to obtain an instance segmentation result.

[0177] In a possible implementation manner, the binary segmentation unit is specifically configured to:

[0178] Perform binary segmentation on the first rib image to obtain an initial rough binary segmentation result corresponding to the first rib image;

[0179] Filter the non-rib regions in the initial rough binary segmentation result according to the rib convex hull region to obtain the target rough binary segmentation result.

[0180] In a possible implementation manner, the marking module 1503 includes:

[0181] A second determination sub-module, configured to determine a unilateral rib sequence according to the instance segmentation result and the rib convex hull region, where the unilateral rib sequence includes multiple unilateral ribs and their sorting, and the unilateral rib sequence includes a left rib sequence and a right rib sequence;

[0182] A third determination sub-module, configured to determine multiple ribs in the rib image to be segmented and a rib identifier corresponding to each rib according to the semantic category segmentation result and the unilateral rib sequence.

[0183] In a possible implementation manner, the second determination sub-module includes:

[0184] A sixth determination unit, configured to determine a first set of unilateral ribs according to the instance segmentation result and the rib convex hull region, where the first unilateral rib sequence includes N unilateral ribs, and N is the number of unilateral ribs;

[0185] A first sorting unit, configured to sort the N unilateral ribs in the z-axis direction and take the first K ribs before sorting to form a second set of unilateral ribs, where K is an integer greater than 1 and less than N;

[0186] A plane fitting unit, configured to perform plane fitting on each rib in the second set of unilateral ribs to obtain K fitting planes;

[0187] A second sorting unit, configured to sort the K ribs according to the positional relationship between the K fitting planes to obtain a unilateral rib sequence.

[0188] In a possible implementation, the apparatus 100 further includes:

[0189] A first determination module, configured to determine a plane normal vector and a rib centroid corresponding to each rib according to the fitting plane corresponding to each rib among the K ribs;

[0190] A second determination module, configured to, for any two ribs R i and R j among the K ribs, determine an inner product between the rib R i corresponding plane normal vector and rib centroid, and the rib R j corresponding plane normal vector and rib centroid, where the magnitude of the inner product between the rib R i and the rib R j is used to indicate the positional relationship between the fitting plane corresponding to the rib R i and the fitting plane corresponding to the rib R j . i j

[0191] In a possible implementation, the third determination sub-module is specifically configured to:

[0192] Determine a rib identifier corresponding to a target rib in the unilateral rib sequence according to the semantic category segmentation result, where the target rib is any rib in the unilateral rib sequence;

[0193] Determine multiple ribs in the rib image to be segmented and a rib identifier corresponding to each rib according to the rib identifier corresponding to the target rib and the unilateral rib sequence.

[0194] In a possible implementation, the update module is specifically configured to:

[0195] Determine multiple connected regions in the fine binary segmentation result;

[0196] Update the target marking result according to the multiple connected regions.

[0197] In a possible implementation, the apparatus 1500 further includes:

[0198] A third determination module, configured to perform adhesion detection on the target marking result based on a preset rib adhesion detection condition to determine whether there are adhesion ribs in the target marking result, where the adhesion ribs are used to indicate at least two adjacent ribs corresponding to the same rib identifier;

[0199] ​​A fourth determination module, configured to determine a pixel point embedding vector corresponding to the adhered ribs when it is determined that there are adhered ribs in the target marking result.

[0200] A clustering module, configured to cluster the pixel point embedding vectors corresponding to the adhered ribs to obtain a rib identifier corresponding to each rib after the adhered ribs are de - adhered.

[0201] In some embodiments, the functions or modules included in the device provided in the embodiments of the present disclosure can be used to execute the methods described in the above - mentioned method embodiments. The specific implementation can refer to the description of the above - mentioned method embodiments. For the sake of brevity, it will not be repeated here.

[0202] The embodiments of the present disclosure also propose a computer - readable storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the above - mentioned methods are implemented. The computer - readable storage medium can be a non - volatile computer - readable storage medium.

[0203] The embodiments of the present disclosure also propose an electronic device, including: a processor; a memory for storing instructions executable by the processor; wherein, the processor is configured to call the instructions stored in the memory to execute the above - mentioned methods.

[0204] The embodiments of the present disclosure also provide a computer program product, including computer - readable code. When the computer - readable code runs on a device, the processor in the device executes instructions for implementing the image - processing method provided in any one of the above embodiments.

[0205] The embodiments of the present disclosure also provide another computer program product, for storing computer - readable instructions. When the instructions are executed, the computer executes the operations of the image - processing method provided in any one of the above embodiments.

[0206] The electronic device can be provided as a terminal, a server or other forms of devices.

[0207] Figure 16 A block diagram of an electronic device according to an embodiment of the present disclosure is shown. As Figure 16 shown, the electronic device 800 can be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, and other terminals.

[0208] Refer to Figure 16 , the electronic device 800 can include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0209] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, telephone calls, data communication, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-described methods. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0210] The memory 804 is configured to store various types of data to support the operation of the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, and the like. The memory 804 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, a magnetic disk, or an optical disk.

[0211] The power component 806 provides power to the various components of the electronic device 800. The power component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 800.

[0212] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe actions, but also detect the duration and pressure associated with the touch or swipe operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each of the front camera and the rear camera may be a fixed optical lens system or have a focal length and optical zoom capabilities.

[0213] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC) that is configured to receive external audio signals when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 further includes a speaker for outputting audio signals.

[0214] The I / O interface 812 provides an interface between the processing component 802 and peripheral interface modules, and the peripheral interface modules may be a keyboard, a click wheel, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0215] The sensor component 814 includes one or more sensors for providing status assessments of various aspects of the electronic device 800. For example, the sensor component 814 can detect the on / off state of the electronic device 800, the relative positioning of components, such as the display and keypad of the electronic device 800. The sensor component 814 can also detect a change in the position of the electronic device 800 or a component of the electronic device 800, the presence or absence of user contact with the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and the temperature change of the electronic device 800. The sensor component 814 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor component 814 can also include a light sensor, such as a complementary metal-oxide semiconductor (CMOS) or charge-coupled device (CCD) image sensor, for use in imaging applications. In some embodiments, the sensor component 814 can further include an acceleration sensor, a gyro sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0216] The communication component 816 is configured to facilitate communication between the electronic device 800 and other devices in a wired or wireless manner. The electronic device 800 can access a wireless network based on communication standards, such as Wireless Fidelity (WiFi), the second-generation mobile communication technology (2G), or the third-generation mobile communication technology (3G), or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0217] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the above method.

[0218] In an exemplary embodiment, a non - volatile computer - readable storage medium is also provided, such as a memory 804 including computer program instructions, and the above computer program instructions can be executed by a processor 820 of the electronic device 800 to complete the above method.

[0219] Figure 17 A block diagram of an electronic device according to an embodiment of the present disclosure is shown. As Figure 17 shown, the electronic device 1900 may be provided as a server. Referring Figure 17 to, the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by a memory 1932 for storing instructions executable by the processing component 1922, such as application programs. The application programs stored in the memory 1932 may include one or more modules each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0220] The electronic device 1900 may further include a power component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output (I / O) interface 1958. The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as Microsoft Server Operating System (Windows Server TM ), the graphical user interface - based operating system launched by Apple Inc. (Mac OS X TM ), the multi - user multi - process computer operating system (Unix TM ), the free and open - source Unix - like operating system (Linux TM ), the open - source Unix - like operating system (FreeBSD TM ) or the like.

[0221] In an exemplary embodiment, a non - volatile computer - readable storage medium is also provided, such as a memory 1932 including computer program instructions, and the above computer program instructions can be executed by the processing component 1922 of the electronic device 1900 to complete the above method.

[0222] The present disclosure may be a system, a method, and / or a computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions thereon for causing a processor to implement aspects of the present disclosure.

[0223] A computer-readable storage medium may be a tangible device that can retain and store instructions for use by an instruction execution device. A computer-readable storage medium may be, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as a punch card or raised structures in grooves having instructions stored thereon, and any suitable combination of the foregoing. The computer-readable storage medium as used herein is not to be construed as a transitory signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through an optical fiber cable), or an electrical signal transmitted through a wire.

[0224] The computer-readable program instructions described herein may be downloaded to respective computing / processing devices from a computer-readable storage medium or may be downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing / processing device.

[0225] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine - related instructions, microcode, firmware instructions, state - setting data, or source code or object code written in any combination of one or more programming languages, including object - oriented programming languages such as Smalltalk, C++, etc., and conventional procedural programming languages such as the "C" language or similar programming languages. The computer - readable program instructions may be executed entirely on the user's computer, partially on the user's computer, executed as a stand - alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider). In some embodiments, by using the state information of the computer - readable program instructions to customize an electronic circuit, such as a programmable logic circuit, a field - programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit can execute the computer - readable program instructions to implement various aspects of the present disclosure.

[0226] Aspects of the present disclosure are described herein with reference to the flowchart and / or block diagram of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowchart and / or block diagram, and the combinations of blocks in the flowchart and / or block diagram, can be implemented by computer - readable program instructions.

[0227] These computer - readable program instructions can be provided to a processor of a general - purpose computer, a special - purpose computer, or other programmable data - processing apparatus to produce a machine such that, when the instructions are executed by the processor of the computer or other programmable data - processing apparatus, a device is created that implements the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer - readable program instructions can also be stored in a computer - readable storage medium, which causes a computer, a programmable data - processing apparatus, and / or other devices to operate in a particular manner, so that the computer - readable medium storing the instructions includes a manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0228] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices, causing a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other devices to generate a computer-implemented process, such that the instructions executed on the computer, other programmable data processing apparatus, or other devices implement the functions / acts specified in one or more blocks of the flowchart and / or block diagram.

[0229] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of code, or a portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions noted in the blocks may occur in a different order than noted in the figures. For example, two consecutive blocks may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or acts, or by a combination of dedicated hardware and computer instructions.

[0230] The computer program product may be implemented specifically by hardware, software, or a combination thereof. In an alternative embodiment, the computer program product is embodied as a computer storage medium. In another alternative embodiment, the computer program product is embodied as a software product, such as a Software Development Kit (SDK), etc.

[0231] The embodiments of the present disclosure have been described above. The above description is exemplary and not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations are obvious to those of ordinary skill in the art in the technical field without departing from the scope and spirit of the described embodiments. The selection of the terms used herein is intended to best explain the principles of the embodiments, the practical application, or the improvement of the technology in the market, or to enable other ordinary skill in the art in the technical field to understand the embodiments disclosed herein.

Claims

1. An image processing method, characterized in that, comprising: Performing semantic category segmentation on the rib image to be segmented to obtain a semantic category segmentation result corresponding to the rib image to be segmented; Performing instance segmentation on the rib image to be segmented to obtain an instance segmentation result corresponding to the rib image to be segmented; Determining a target marking result corresponding to the rib image to be segmented according to the semantic category segmentation result and the instance segmentation result, wherein the target marking result includes multiple ribs in the rib image to be segmented and a rib identifier corresponding to each rib; Wherein, the determining a target marking result corresponding to the rib image to be segmented according to the semantic category segmentation result and the instance segmentation result includes: Determining a unilateral rib sequence according to the instance segmentation result and the rib convex hull region, wherein the unilateral rib sequence includes multiple unilateral ribs and their sorting, and the unilateral rib sequence includes a left rib sequence and a right rib sequence; Determining multiple ribs in the rib image to be segmented and a rib identifier corresponding to each rib according to the semantic category segmentation result and the unilateral rib sequence.

2. The method according to claim 1, characterized in that, The method further includes: In the case where the semantic category segmentation and the instance segmentation are performed at a first resolution, performing binary segmentation on the rib image to be segmented at a second resolution to obtain a fine binary segmentation result corresponding to the rib image to be segmented, wherein the second resolution is greater than the first resolution; Updating the target marking result according to the fine binary segmentation result.

3. The method according to claim 2, characterized in that, The method further includes: Obtaining an original chest scan image; Performing image preprocessing on the original chest scan image to obtain an initial rib image; Performing convex hull segmentation on the initial rib image to obtain a rib convex hull region in the initial rib image; Cropping the initial rib image according to the rib convex hull region to obtain the rib image to be segmented.

4. The method according to claim 3, characterized in that, The performing convex hull segmentation on the initial rib image to obtain a rib convex hull region in the initial rib image includes: Determining whether a rib region in the initial rib image meets a preset integrity requirement; In the case where the rib region in the initial rib image meets the preset integrity requirement, performing convex hull segmentation on the initial rib image to obtain the rib convex hull region.

5. The method according to claim 4, characterized in that, The initial rib image includes multiple slice images; The determining whether a rib region in the initial rib image meets a preset integrity requirement includes: Determining the rib category of each slice image in the multiple slice images, wherein different rib categories are used to indicate different rib regions; Determining a rib category sequence according to the rib category of each slice image; In the case where the rib category sequence includes a preset rib category, determining that the rib region in the initial rib image meets the preset integrity requirement.

6. The method according to any one of claims 3 to 5, wherein, the instance segmentation of the rib image to be segmented to obtain the instance segmentation result corresponding to the rib image to be segmented includes: resampling the rib image to be segmented to obtain a first rib image, wherein the resolution of the first rib image is the first resolution; performing position encoding on the pixel points in the first rib image to obtain a position encoding image; based on the position encoding image, performing instance segmentation on the first rib image to obtain the instance segmentation result.

7. The method according to claim 6, wherein, the performing instance segmentation on the first rib image based on the position encoding image to obtain the instance segmentation result includes: performing binary segmentation on the first rib image to obtain a target rough binary segmentation result corresponding to the first rib image; determining the pixel point embedding vector corresponding to the first rib image based on the position encoding image; determining the pixel point embedding vector corresponding to the rib region in the first rib image according to the target rough binary segmentation result and the pixel point embedding vector corresponding to the first rib image; clustering the pixel point embedding vectors corresponding to the rib regions in the first rib image to obtain the instance segmentation result.

8. The method according to claim 7, wherein, the performing binary segmentation on the first rib image to obtain the target rough binary segmentation result corresponding to the first rib image includes: performing binary segmentation on the first rib image to obtain an initial rough binary segmentation result corresponding to the first rib image; filtering the non-rib regions in the initial rough binary segmentation result according to the rib convex hull region to obtain the target rough binary segmentation result.

9. The method according to claim 1, wherein, the determining the unilateral rib sequence according to the instance segmentation result and the rib convex hull region includes: determining a first unilateral rib set according to the instance segmentation result and the rib convex hull region, wherein the first unilateral rib set includes root unilateral ribs, and is the number of unilateral ribs; sorting the root unilateral ribs in the axial direction and taking the first ribs before sorting to form a second unilateral rib set, wherein is an integer greater than 1 and less than ; performing plane fitting on each rib in the second unilateral rib set to obtain fitting planes; sorting the ribs according to the positional relationship between the fitting planes to obtain the unilateral rib sequence.

10. The method according to claim 9, wherein, the method further includes: determining the plane normal vector and rib centroid corresponding to each rib according to the fitting plane corresponding to each rib in the ribs; For any two ribs among the root ribs, determine the inner product between the rib and another rib according to the plane normal vector and rib centroid corresponding to the rib, and the plane normal vector and rib centroid corresponding to the other rib, wherein the magnitude of the inner product between the rib and the other rib is used to indicate the positional relationship between the fitting plane corresponding to the rib and the fitting plane corresponding to the other rib.

11. The method according to claim 1, wherein, the determining of multiple ribs in the rib image to be segmented and the rib identifier corresponding to each rib according to the semantic category segmentation result and the unilateral rib sequence includes: determining the rib identifier corresponding to the target rib in the unilateral rib sequence according to the semantic category segmentation result, where the target rib is any rib in the unilateral rib sequence; determining multiple ribs in the rib image to be segmented and the rib identifier corresponding to each rib according to the rib identifier corresponding to the target rib and the unilateral rib sequence.

12. The method according to any one of claims 2 to 5, wherein, the updating of the target marking result according to the fine binary segmentation result includes: determining multiple connected regions in the fine binary segmentation result; updating the target marking result according to the multiple connected regions.

13. The method according to any one of claims 1 to 5, wherein, the method further includes: performing adhesion detection on the target marking result based on a preset rib adhesion detection condition to determine whether there are adhesion ribs in the target marking result, where the adhesion ribs are used to indicate at least two adjacent ribs corresponding to the same rib identifier; determining the pixel point embedding vector corresponding to the adhesion ribs when it is determined that there are adhesion ribs in the target marking result; clustering the pixel point embedding vectors corresponding to the adhesion ribs to obtain the rib identifier corresponding to each rib after the adhesion ribs are de - adhered.

14. An image processing device, wherein, comprising: a semantic category segmentation module for performing semantic category segmentation on the rib image to be segmented to obtain the semantic category segmentation result corresponding to the rib image to be segmented; an instance segmentation module for performing instance segmentation on the rib image to be segmented to obtain the instance segmentation result corresponding to the rib image to be segmented; a marking module for determining the target marking result corresponding to the rib image to be segmented according to the semantic category segmentation result and the instance segmentation result, where the target marking result includes multiple ribs in the rib image to be segmented and the rib identifier corresponding to each rib; wherein, the marking module includes: a second determination sub - module for determining a unilateral rib sequence according to the instance segmentation result and the rib convex hull region, where the unilateral rib sequence includes multiple unilateral ribs and their sorting, and the unilateral rib sequence includes a left rib sequence and a right rib sequence; A third determination sub-module, configured to determine multiple ribs in the rib image to be segmented and a rib identifier corresponding to each rib according to the semantic category segmentation result and the unilateral rib sequence.

15. An electronic device, characterized in that it includes: a processor; a memory for storing instructions executable by the processor; wherein, the processor is configured to call the instructions stored in the memory to execute the method according to any one of claims 1 to 13.

16. A computer-readable storage medium, on which computer program instructions are stored, characterized in that when the computer program instructions are executed by a processor, the method according to any one of claims 1 to 13 is implemented.

Images