Bone image recognition optimization method, system, storage medium and electronic device

Abstract

The present invention relates to the field of image processing, and in particular to a method, system, storage medium and electronic device for optimizing the recognition of bone images. A method for optimizing the recognition of bone images comprises: stacking an abdominal CT image containing a detection frame of a target bone region to obtain a first image; inputting the abdominal CT image into a segmentation model to obtain a rib segmentation result; performing non-intersection filtering based on the first image and the rib segmentation result to obtain a filtering result, and using the filtering result as the recognition result of the target bone region. The present invention can optimize the existing imaging method to make it more intuitive, and can also accurately locate the bone region, thereby increasing the user experience and making it more convenient to know the part of the body where the bone region is located.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a method, system, storage medium and electronic device for optimizing the recognition of bone images. Background Art

[0002] Currently, the display processing of detection images is often two-dimensional, especially in the field of CT imaging. For example, in the display of bones, the generated images are usually displayed in the form of pictures. This presentation method is firstly too simple to be intuitive, and the user experience is low. Secondly, it is impossible to accurately locate the detection image. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a recognition optimization method, system, storage medium and electronic equipment for bone images.

[0004] The technical solution of the present invention to solve the above technical problem is as follows: a method for optimizing the recognition of bone images, comprising:

[0005] Performing stacking processing on the abdominal CT image of the detection frame including the target bone region to obtain a first image;

[0006] Inputting the abdominal CT image into a segmentation model to obtain a rib segmentation result;

[0007] Based on the first image and the rib segmentation result, non-intersection filtering processing is performed to obtain a filtering result, and the filtering result is used as the recognition result of the target bone region.

[0008] The beneficial effects of the present invention are: the present invention overcomes the traditional form of two-dimensional imaging and restores it to a three-dimensional image through steps such as stacking and filtering. The present invention can not only optimize the existing imaging method to make it more intuitive, but also accurately locate the bone area, thereby increasing the user experience and making it more convenient to know which part of the body the bone area is located.

[0009] Based on the above technical solution, the present invention can also be improved as follows.

[0010] Further, the stacking process of the abdominal CT image including the target bone region detection frame to obtain the first image specifically includes:

[0011] The abdominal CT image containing the target bone region detection frame is stacked in the order of chest and abdomen from top to bottom to obtain a three-dimensional image containing multiple fracture regions composed of the detection frames, and the three-dimensional image is the first image.

[0012] Further, the non-intersection filtering process is performed based on the first image and the rib segmentation result to obtain the filtering result, which specifically includes:

[0013] The background value of the detection frame area and the background value of the non-detection frame area in the first image are set to 1 and 0 respectively, and the background value of the rib position and the background value of the non-rib position in the rib segmentation result are set to 1 and 0 respectively. By multiplying the corresponding positions, the area where the product is 0 is eliminated, and the area where the product is 1 is used as the filtering result.

[0014] Another technical solution of the present invention to solve the above technical problem is as follows: a bone image recognition optimization system, comprising:

[0015] A stacking module, used for stacking the abdominal CT image including the detection frame of the target bone region to obtain a first image;

[0016] A segmentation module, used for inputting the abdominal CT image into a segmentation model to obtain a rib segmentation result;

[0017] A filtering module is used to perform non-intersection filtering processing based on the first image and the rib segmentation result to obtain a filtering result.

[0018] The beneficial effects of the present invention are: the present invention overcomes the traditional form of two-dimensional imaging and restores it to a three-dimensional image through steps such as stacking and filtering. The present invention can not only optimize the existing imaging method to make it more intuitive, but also accurately locate the bone area, thereby increasing the user experience and making it more convenient to know which part of the body the bone area is located.

[0019] Furthermore, the stacking module is specifically used for:

[0020] The abdominal CT image containing the target bone region detection frame is stacked in the order of chest and abdomen from top to bottom to obtain a three-dimensional image containing multiple fracture regions composed of the detection frames, and the three-dimensional image is the first image.

[0021] Furthermore, the filtering module is specifically used for:

[0022] The background value of the detection frame area and the background value of the non-detection frame area in the first image are set to 1 and 0 respectively, and the background value of the rib position and the background value of the non-rib position in the rib segmentation result are set to 1 and 0 respectively. By multiplying the corresponding positions, the area where the product is 0 is eliminated, and the area where the product is 1 is used as the filtering result.

[0023] Another technical solution of the present invention to solve the above technical problem is as follows: a storage medium, in which instructions are stored. When a computer reads the instructions, the computer executes an optimization method for determining a fracture area as described in any one of the above items.

[0024] The beneficial effects of the present invention are: the present invention overcomes the traditional form of two-dimensional imaging and restores it to a three-dimensional image through steps such as stacking and filtering. The present invention can not only optimize the existing imaging method to make it more intuitive, but also accurately locate the bone area, thereby increasing the user experience and making it more convenient to know which part of the body the bone area is located.

[0025] Another technical solution of the present invention to solve the above technical problem is as follows: an electronic device comprises the above storage medium and a processor for executing instructions in the above storage medium.

[0026] The beneficial effects of the present invention are: the present invention overcomes the traditional form of two-dimensional imaging and restores it to a three-dimensional image through steps such as stacking and filtering. The present invention can not only optimize the existing imaging method to make it more intuitive, but also accurately locate the bone area, thereby increasing the user experience and making it more convenient to know which part of the body the bone area is located. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A schematic flow chart of an embodiment of an optimization method for determining a fracture area according to the present invention;

[0028] Figure 2 A structural framework diagram provided for an embodiment of an optimization system for determining a fracture region of the present invention;

[0029] Figure 3 (a) is a schematic diagram before threshold segmentation provided by an embodiment of an optimization method for determining a fracture region of the present invention;

[0030] Figure 3 (b) is an example diagram after threshold segmentation provided by an embodiment of an optimization method for determining a fracture area of ​​the present invention;

[0031] Figure 4 A schematic diagram of stacking detection frames provided for an embodiment of an optimization method for determining a fracture region of the present invention;

[0032] Figure 5 A three-dimensional rib schematic diagram provided for an embodiment of an optimization method for determining a fracture region of the present invention;

[0033] Figure 6 A second image schematic diagram provided for an embodiment of an optimization method for determining a fracture region of the present invention;

[0034] Figure 7 A schematic diagram of a detection frame and a corresponding probability value in the prior art provided for an embodiment of an optimization method for determining a fracture area of ​​the present invention. DETAILED DESCRIPTION

[0035] The principles and features of the present invention are described below. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.

[0036] like Figure 1 As shown, a bone image recognition optimization method includes:

[0037] Performing stacking processing on the abdominal CT image of the detection frame including the target bone region to obtain a first image;

[0038] Inputting the abdominal CT image into a segmentation model to obtain a rib segmentation result;

[0039] Based on the first image and the rib segmentation result, non-intersection filtering processing is performed to obtain a filtering result, and the filtering result is used as the recognition result of the target bone region.

[0040] In some possible implementations, the present invention overcomes the traditional form of two-dimensional imaging and restores it to a three-dimensional image through steps such as stacking and filtering. The present invention can not only optimize the existing imaging method and make it more intuitive, but also accurately locate the bone area, thereby increasing the user experience and making it more convenient to know which part of the body the bone area is located.

[0041] It should be noted that the abdominal CT image containing the detection frame of the target bone area is the content that can be obtained by the prior art, such as: first generate a candidate frame (detection frame) of the target bone area through a deep learning model, and filter the generated detection frame through a very small threshold. The method of this step is consistent with the prior art, but the threshold selected in the present invention is very small (such as 0.1, 0.15), the purpose of which is to filter out obvious false positives, but not to delete the true positives in the detection frame. Figure 3 As shown, Figure 3 (a) is the state before threshold segmentation, where the dark gray box is the detection box, the area outlined by the white line is the bone, the medium gray and light gray parts are muscle tissue and fat, and the black part is the background. Figure 3 (b) is the state after threshold segmentation, and Figure 3 Same as in (a), the dark gray box is the detection box, the area outlined by the white line is the bone, the medium gray and light gray parts are muscle tissue and fat, and the black part is the background.

[0042] The generation process of the first image may be understood with reference to Example 1.

[0043] In Example 1, since the abdominal CT image is a tomographic scan of the human abdomen, the detection frame is stacked on the cross-sectional layer according to the layer number and position of the CT image where it is located, such as Figure 4 As shown, a three-dimensional image is generated, which is the first image. The significance of this is that some real information about the human body and bones in clinical practice can be used to perform more reasonable post-processing operations on the potential prediction frame. The gray area represents the three-dimensional detection frame area, and the mutually perpendicular dotted lines are the calibration lines in the software and have no specific meaning.

[0044] The process of inputting the abdominal CT image into the segmentation model to obtain the rib segmentation result can be understood by referring to Example 2.

[0045] In Example 2, since the detected area is the rib bone area, the generated detection frame must be on the ribs, so the rib area can be segmented by inputting the CT image, that is, the original data or the data image that can be understood as before the deep learning model, into the segmentation model, such as Figure 5 As shown, Figure 5 That is, the three-dimensional rib map of the human body presented through CT images.

[0046] The process of performing non-intersection filtering based on the first image and the rib segmentation result to obtain the second image may be understood by referring to Example 3.

[0047] Example 3, through the segmented rib area, filter out the bone area that does not intersect with the rib. The specific filtering method is: set the detection frame area to 1 and the non-detection area to 0. In the same way, set the segmented rib to 1 and the remaining area to 0. Multiply the two arrays, and the area with a value of 1 is the area where the two arrays intersect. Finally, generate the filtering result. The filtering result is as follows: Figure 6 As shown, the gray part is a three-dimensional image of the detection frame area that intersects with the ribs, and the vertically intersecting dotted lines are the calibration lines provided in the software and have no specific meaning.

[0048] So far, the positioning and optimized display of the bone region have been completed. The following examples illustrate the application of the present invention in real life.

[0049] Example 4, it should be noted that in this embodiment, the bone area is proposed as the fracture area. According to the obtained filtering results, the 2D detection frame is connected to the 3D detection frame on the 3D level by finding the connected domain, forming a three-dimensional fracture area in the 3D scene. Since each 2D detection frame has a different probability value, for the detection frame of the 3D level formed, the probabilities of the 2D detection frames of all levels are averaged to obtain the probability value of the 3D detection frame. There are 12 ribs in the human body, of which 1-3 ribs are shorter and are protected by the clavicle and scapula, so fractures are less likely to occur; 4-7 ribs are longer and fixed, and are most likely to fracture; 8-10 ribs are long, but they are indirectly connected to the sternum with the help of the seventh rib to form a costal arch, so they are more elastic and not easy to fracture; 11-12 ribs have free front edges, which are easier to avoid violent effects and have greater elasticity, so they are not easy to fracture. According to the characteristics of different fractures, the 12 ribs are divided into 4 parts. It should be noted that the 4 parts are used in this embodiment, and other numbers of divisions can be made according to the actual application. For different parts, a matching 3D detection frame threshold is set, and the remaining detection frames are filtered by the threshold. The probability value of the fracture area is determined according to the threshold at different positions, and the ribs corresponding to the fracture area greater than the threshold are determined to be fractured. Although the detection frame predicted by deep learning appears in the form of a single layer on CT, such as Figure 7 As shown, Figure 3 Similarly, the gray rectangle is the detection box, and each detection box is accompanied by a probability value, indicating the probability of the rectangle, and the rest are the same as Figure 3The meaning is the same. However, according to clinical experience, a fracture area usually appears continuously on multiple CT images, and the detection frame should be continuous, so the detection frame with only one isolated layer on the 3D level is filtered out. It should be noted that in this embodiment, one layer is used as an example for explanation, but in actual situations, two or three layers can be eliminated, and it can be handled flexibly according to actual conditions. The final judgment result obtained is input into the classification model, and the fracture area is divided into fresh and old. It should be noted here that the fracture area includes images that have been judged to be fractured and images that are judged to be unfractured. Because fresh fractures are more difficult to detect than other types of fractures, many times even if they are detected by the model, there is a problem of being blocked by the threshold due to inappropriate threshold setting. According to clinical analysis and Bayes formula calculation, if a rib has a fresh fracture, the probability of fresh fractures in the upper and lower ribs is significantly increased. Therefore, the last step of post-processing is to recall the fresh fractures that are blocked by the threshold in Example 6, that is, if there are fresh fractures with a lower probability within a certain range of the upper and lower ribs of the fresh fracture, the predicted probability of the detection frame of these fractures is multiplied by a coefficient (such as 1.1). It should be noted that this coefficient is an example value in this embodiment and can be appropriately adjusted in actual application. If the new probability value after calculation exceeds the threshold set for the rib, it is recalled, otherwise it remains unchanged. The specific formula is:

[0050] New probability = old probability * 1.1

[0051] The range calculation of fresh fractures and other ribs is achieved by calculating the Euclidean distance. The specific formula is as follows: Assume that the coordinates of the center point of the fracture frame (fracture area) A are X1, Y1, Z1, and the center point of the potential fracture frame (potential fracture area) B is X2, Y2, Z2. The calculation formula for the distance P between the two points is as follows:

[0052]

[0053] like Figure 2 As shown, a bone image recognition optimization system includes:

[0054] A stacking module 100 is used to stack the abdominal CT image including the detection frame of the target bone region to obtain a first image;

[0055] The segmentation module 200 is used to stack the abdominal CT image including the detection frame of the target bone region to obtain a first image;

[0056] The filtering module 300 is used to input the abdominal CT image into the segmentation model to obtain a rib segmentation result.

[0057] In some possible implementations, the present invention overcomes the traditional form of two-dimensional imaging and restores it to a three-dimensional image through steps such as stacking and filtering. The present invention can not only optimize the existing imaging method and make it more intuitive, but also accurately locate the bone area, thereby increasing the user experience and making it more convenient to know which part of the body the bone area is located.

[0058] Preferably, in any of the above embodiments, the stacking module is specifically used for:

[0059] The abdominal CT image containing the target bone region detection frame is stacked in the order of chest and abdomen from top to bottom to obtain a three-dimensional image containing multiple fracture regions composed of the detection frames, and the three-dimensional image is the first image.

[0060] Preferably, in any of the above embodiments, the filtering module is specifically used for:

[0061] The background value of the detection frame area and the background value of the non-detection frame area in the first image are set to 1 and 0 respectively, and the background value of the rib position and the background value of the non-rib position in the rib segmentation result are set to 1 and 0 respectively. By multiplying the corresponding positions, the area where the product is 0 is eliminated, and the area where the product is 1 is used as the filtering result.

[0062] Another technical solution of the present invention to solve the above technical problem is as follows: a storage medium, in which instructions are stored. When a computer reads the instructions, the computer executes an optimization method for determining a fracture area as described in any one of the above items.

[0063] In some possible implementations, the present invention overcomes the traditional form of two-dimensional imaging and restores it to a three-dimensional image through steps such as stacking and filtering. The present invention can not only optimize the existing imaging method and make it more intuitive, but also accurately locate the bone area, thereby increasing the user experience and making it more convenient to know which part of the body the bone area is located.

[0064] Another technical solution of the present invention to solve the above technical problem is as follows: an electronic device comprises the above storage medium and a processor for executing instructions in the above storage medium.

[0065] In some possible implementations, the present invention overcomes the traditional form of two-dimensional imaging and restores it to a three-dimensional image through steps such as stacking and filtering. The present invention can not only optimize the existing imaging method and make it more intuitive, but also accurately locate the bone area, thereby increasing the user experience and making it more convenient to know which part of the body the bone area is located.

[0066] The reader should understand that in the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples without contradiction.

[0067] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the method embodiments described above are only illustrative, for example, the division of steps is only a logical function division, and there may be other division methods in actual implementation, such as multiple steps can be combined or integrated into another step, or some features can be ignored or not executed.

[0068] If the above method is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0069] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for optimizing the recognition of bone images, characterized in that: include: Performing stacking processing on the abdominal CT image of the detection frame including the target bone region to obtain a first image; Inputting the abdominal CT image into a segmentation model to obtain a rib segmentation result; Based on the first image and the rib segmentation result, non-intersection filtering processing is performed to obtain a filtering result, and the filtering result is used as the recognition result of the target bone region; The bone region is proposed as the fracture region. According to the obtained filtering results, the 2D detection frame is connected into a 3D detection frame on the 3D level by finding the connected domain, thus forming a three-dimensional fracture region in the 3D scene. The stacking process of the abdominal CT image including the target bone region detection frame to obtain the first image specifically includes: The abdominal CT images containing the target bone region detection frame are stacked in the order of chest and abdomen from top to bottom to obtain a three-dimensional image containing multiple fracture regions composed of the detection frames, wherein the three-dimensional image is a first image; The detection frame of the target bone region is obtained specifically in the following manner: First, a candidate frame of the target bone area is generated by a deep learning model. The candidate frame is the detection frame, and the generated detection frame is filtered through a very small threshold. The process of inputting the abdominal CT image into the segmentation model to obtain the rib segmentation result is as follows: The rib area is segmented by inputting the CT image, i.e., the original data or the data image before being subjected to the deep learning model, into the segmentation model.

2. The method for optimizing bone image recognition according to claim 1, characterized in that: The performing non-intersection filtering processing based on the first image and the rib segmentation result to obtain the filtering result specifically includes: The background value of the detection frame area and the background value of the non-detection frame area in the first image are set to 1 and 0 respectively, and the background value of the rib position and the background value of the non-rib position in the rib segmentation result are set to 1 and 0 respectively. By multiplying the corresponding positions, the area where the product is 0 is eliminated, and the area where the product is 1 is used as the filtering result.

3. A bone image recognition optimization system, characterized in that: include: A stacking module, used for stacking the abdominal CT image including the detection frame of the target bone region to obtain a first image; A segmentation module, used for inputting the abdominal CT image into a segmentation model to obtain a rib segmentation result; A filtering module, configured to perform non-intersection filtering processing based on the first image and the rib segmentation result to obtain a filtering result, and use the filtering result as the recognition result of the target bone region; The bone region is proposed as the fracture region. According to the obtained filtering results, the 2D detection frame is connected into a 3D detection frame on the 3D level by finding the connected domain, thus forming a three-dimensional fracture region in the 3D scene. The stacking process of the abdominal CT image including the target bone region detection frame to obtain the first image specifically includes: The abdominal CT images containing the target bone region detection frame are stacked in the order of chest and abdomen from top to bottom to obtain a three-dimensional image containing multiple fracture regions composed of the detection frames, wherein the three-dimensional image is a first image; The detection frame of the target bone region is obtained specifically in the following manner: First, a candidate frame of the target bone area is generated by a deep learning model. The candidate frame is the detection frame, and the generated detection frame is filtered through a very small threshold. The process of inputting the abdominal CT image into the segmentation model to obtain the rib segmentation result is as follows: The rib area is segmented by inputting the CT image, i.e., the original data or the data image before being subjected to the deep learning model, into the segmentation model.

4. A bone image recognition optimization system according to claim 3, characterized in that: The filtering module is specifically used for: The background value of the detection frame area and the background value of the non-detection frame area in the first image are set to 1 and 0 respectively, and the background value of the rib position and the background value of the non-rib position in the rib segmentation result are set to 1 and 0 respectively. By multiplying the corresponding positions, the area where the product is 0 is eliminated, and the area where the product is 1 is used as the filtering result.

5. A storage medium, characterized in that: The storage medium stores instructions, and when a computer reads the instructions, the computer executes the bone image recognition optimization method as claimed in claim 1 or 2.

6. An electronic device, characterized in that: The invention comprises the storage medium as claimed in claim 5 and a processor for executing instructions in the storage medium.

Images