Image analysis method and related device, electronic device, and storage medium

Abstract

The present application discloses an image analysis method and related devices, electronic devices, and storage media, wherein the image analysis method includes: obtaining a medical image of an object to be tested; performing feature extraction on the medical image to obtain a feature image; performing a first test on the feature image to obtain a first test result, wherein the first test result indicates the possibility of the existence of a target area in the medical image, and based on the first test result, performing a second test on the feature image to obtain a second test result; wherein the second test result includes attribute information of the target area. The above scheme can improve the speed and accuracy of image analysis, and can be specifically applied to the first test result and the second test result of the fracture area in the medical image, which can improve the speed and accuracy of fracture area detection.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image analysis method and related devices, electronic equipment, and storage media. Background Art

[0002] Thanks to medical images such as CT (Computed Tomography), medical staff can analyze target areas in medical images, such as fracture areas, based on clinical knowledge and experience. For example, medical staff can analyze whether there is a fracture in a medical image, as well as the type of fracture. However, manual image analysis often has the problem of slow analysis speed and is easily affected by subjective factors, making it difficult to ensure the accuracy of the analysis. In view of this, how to improve the speed and accuracy of image analysis has become an urgent problem to be solved. Summary of the invention

[0003] The present application provides an image analysis method and related devices, electronic equipment, and storage media.

[0004] The first aspect of the present application provides an image analysis method, comprising: acquiring a medical image of an object to be tested; performing feature extraction on the medical image to obtain a feature image; performing a first detection on the feature image to obtain a first detection result, wherein the first detection result indicates the possibility of the existence of a target area in the medical image; based on the first detection result, performing a second detection on the feature image to obtain a second detection result; wherein the second detection result includes attribute information of the target area.

[0005] Therefore, by acquiring a medical image of the object to be tested and performing feature extraction on the medical image to obtain a feature image, a first test is performed on the feature image to obtain a first test result, and the first test result indicates the possibility of the existence of a target area in the medical image, and based on the first test result, a second test is performed on the feature image to obtain a second test result, and the second test result includes attribute information of the target area. Therefore, it is possible to avoid manual image analysis, which can be beneficial to improving the speed and accuracy of image analysis. In addition, since the first test on the possibility of the existence of the target area and the second test on the attribute information of the target area are both performed based on the feature image, that is, the first test and the second test share the feature image, the amount of calculation can be reduced compared to when the two tasks are performed completely independently. Therefore, the above method can improve the speed and accuracy of image analysis.

[0006] Wherein, the target area includes a fracture area. Before performing feature extraction on the medical image to obtain a feature image, it includes: performing a third detection on the medical image to obtain a bone area containing bone pixels; performing feature extraction on the medical image to obtain a feature image, including: based on the bone pixels in the bone area, performing region extraction on the medical image to obtain at least one region image; wherein the center of the region image is the bone pixel; performing feature extraction on at least one region image to obtain a feature image.

[0007] Therefore, the target area includes the fracture area. Before performing feature extraction on the medical image, a third detection is first performed on the medical image to obtain a bone area containing bone pixels, and then based on the bone pixels in the bone area, region extraction is performed on the medical image to obtain at least one regional image, and the center of the regional image is the bone pixel point, and then feature extraction is performed on at least one regional image to obtain a feature image. Therefore, feature extraction can be focused on the bone area, which can help reduce the interference of other areas on fracture detection, and can help reduce the missed detection of minor fractures, and further improve the accuracy of fracture detection.

[0008] Among them, based on the bone pixel points in the bone area, before performing region extraction on the medical image to obtain at least one regional image, the method also includes: obtaining bone pixel points on the skeleton line of the bone area as candidate pixel points, and selecting at least one candidate pixel point as a target pixel point; wherein the center of the regional image is the target pixel point.

[0009] Therefore, by obtaining bone pixel points on the skeleton line of the bone area as candidate pixel points, and selecting at least one candidate pixel point as the target pixel point, and the center of the regional image is the target pixel point, feature extraction can be focused on the bone area and the accuracy of fracture detection can be improved. At the same time, the number of regional images can be reduced, thereby reducing the amount of calculation and improving the speed of fracture detection.

[0010] Wherein, before performing feature extraction on at least one regional image respectively to obtain a feature image, the method also includes: performing a fourth detection on at least one regional image respectively to obtain a bone abnormality of the regional image; performing feature extraction on at least one regional image respectively to obtain a feature image, including: performing feature extraction on a regional image whose bone abnormality meets a preset condition to obtain a feature image.

[0011] Therefore, before performing feature extraction on at least one regional image to obtain a feature image, a fourth detection is performed on at least one regional image to obtain the bone abnormality of the regional image, and feature extraction is performed on the regional image whose bone abnormality meets the preset conditions to obtain the feature image. This enables "coarse to fine" fracture detection, that is, a preliminary screening of regional images whose bone abnormality meets the preset conditions is performed first, thereby avoiding the detection of regional images whose bone abnormality does not meet the preset conditions. On this basis, subsequent fine screening is performed on the screened regional images, which can help further improve the fracture detection speed.

[0012] Among them, the bone abnormality includes a bone abnormality score, and the preset conditions include: the bone abnormality score is greater than a preset score threshold.

[0013] Therefore, by setting the bone abnormality condition to include the bone abnormality score, and setting the preset condition to include: the bone abnormality score is greater than the preset score threshold, it can be helpful to avoid detecting images of areas with low bone abnormality levels, and screen images of areas with high bone abnormality levels for subsequent fine identification, which can help improve the speed of fracture detection.

[0014] Among them, feature extraction is performed on the medical image to obtain a feature image, including: using the feature extraction subnetwork of the image analysis model to extract features of the medical image to obtain a feature image; and / or, a first detection is performed on the feature image to obtain a first detection result, including: using the first detection subnetwork of the image analysis model to perform a first detection on the feature image to obtain the first detection result; and / or, a second detection is performed on the feature image to obtain a second detection result, including: using the second detection subnetwork of the image analysis model to perform a second detection on the feature image to obtain a second detection result.

[0015] Therefore, extracting features from medical images through the feature extraction subnetwork of the image analysis model can help improve the efficiency of feature extraction; performing a first detection on the feature image through the first detection subnetwork of the image analysis model can help improve the efficiency of the first detection; and performing a second detection on the feature image through the second detection subnetwork of the image analysis model can help improve the efficiency of the second detection.

[0016] Among them, the training steps of the image analysis model include: obtaining a sample medical image; wherein the sample medical image is annotated with a first label and a second label, the first label represents whether there is a sample target area in the sample medical image, and the second label represents sample attribute information of the sample target area; using the feature extraction subnetwork of the image analysis model to extract features of the sample medical image to obtain a sample feature image; using the first detection subnetwork of the image analysis model to perform a first detection on the sample feature image to obtain a first prediction result, and using the second detection subnetwork of the image analysis model to perform a second detection on the feature image to obtain a second prediction result; wherein the first prediction result indicates the possibility that there is a sample target area in the sample medical image, and the second prediction result includes the predicted attribute information of the sample target area; based on the difference between the first label and the first prediction result, and the difference between the second label and the second prediction result, adjusting the network parameters of the image analysis model.

[0017] Therefore, by acquiring a sample medical image, and the sample medical image is annotated with a first label and a second label, the first label represents whether there is a sample target area in the sample medical image, and the second label represents the sample attribute information of the sample target area, and the feature extraction subnetwork of the image analysis model is used to extract features of the sample medical image to obtain a sample feature image, and then the first detection subnetwork of the image analysis model is used to perform a first detection on the sample feature image to obtain a first prediction result, and the second detection subnetwork of the image analysis model is used to perform a second detection on the feature image to obtain a second prediction result, and the first prediction result indicates the possibility of the existence of the sample target area in the sample medical image, and the second prediction result includes the predicted attribute information of the sample target area, and then based on the difference between the first label and the first prediction result, and the difference between the second label and the second prediction result, the network parameters of the image analysis model are adjusted. Therefore, in the training process of the image analysis model, the first detection subnetwork and the second detection subnetwork can be jointly trained, which can facilitate the mutual supervision of the first detection task and the second detection task, and thus can help improve the accuracy of the image analysis model.

[0018] Among them, the medical image is a chest medical image of the object to be tested, the target area includes a fracture area, and the attribute information includes the fracture type of the fracture area; and / or, the first detection result includes a probability value of the existence of the target area in the medical image, and the larger the probability value, the higher the possibility of the existence of the target area.

[0019] Therefore, setting the medical image to a chest medical image of the object to be tested, setting the target area to include the fracture area, and setting the attribute information to include the fracture type of the fracture area can be beneficial to fracture detection; setting the first detection result to the probability value of the existence of the target area in the medical image, and the larger the probability value, the higher the possibility of the existence of the target area, which can be beneficial to quantifying the possibility of the target area, so that the user can easily judge whether there is a target area based on the probability value.

[0020] The second aspect of the present application provides an image analysis device, including: an image acquisition module, a feature extraction module, a first detection module, and a second detection module, the image acquisition module is used to acquire a medical image of an object to be tested; the feature extraction module is used to extract features from the medical image to obtain a feature image; the first detection module is used to perform a first detection on the feature image to obtain a first detection result, wherein the first detection result indicates the possibility of the existence of a target area in the medical image, and the second detection module is used to perform a second detection on the feature image based on the first detection result to obtain a second detection result; wherein the second detection result includes attribute information of the target area.

[0021] A third aspect of the present application provides an electronic device, comprising a memory and a processor coupled to each other, wherein the processor is used to execute program instructions stored in the memory to implement the image analysis method in the first aspect.

[0022] A fourth aspect of the present application provides a computer-readable storage medium having program instructions stored thereon, which implement the image analysis method in the first aspect when executed by a processor.

[0023] The above scheme, therefore, obtains a medical image of the object to be tested, and extracts features from the medical image to obtain a feature image, thereby performing a first test on the feature image to obtain a first test result, and the first test result indicates the possibility of the existence of a target area in the medical image, and based on the first test result, performs a second test on the feature image to obtain a second test result, and the second test result includes attribute information of the target area, so that it is possible to avoid manual image analysis, which can be beneficial to improving the speed and accuracy of image analysis. In addition, since the first test on the possibility of the existence of the target area and the second test on the attribute information of the target area are both performed based on the feature image, that is, the first test and the second test share the feature image, the amount of calculation can be reduced compared to when the two tasks are performed completely independently. Therefore, the above method can improve the speed and accuracy of image analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a flow chart of an embodiment of the image analysis method of the present application;

[0025] Figure 2 is a schematic diagram of a framework of an embodiment of an image analysis model;

[0026] Figure 3 is a flow chart of another embodiment of the image analysis method of the present application;

[0027] Figure 4 is a flowchart of an embodiment of a training process of an image analysis model;

[0028] Figure 5 It is a schematic diagram of the framework of an embodiment of the image analysis device of the present application;

[0029] Figure 6 It is a schematic diagram of the framework of an embodiment of the electronic device of the present application;

[0030] Figure 7 It is a schematic diagram of a framework of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0031] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0032] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0033] The terms "system" and "network" are often used interchangeably in this article. The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship. In addition, "many" in this article means two or more than two.

[0034] See also Figure 1 , Figure 1 This is a flow chart of an embodiment of the image analysis method of the present application. Specifically, it may include the following steps:

[0035] Step S11: Acquire a medical image of the object to be tested.

[0036] In an implementation scenario, the object to be tested may be different objects according to the actual application situation, and is not limited here. For example, in scenarios such as injury assessment, compensation, and sentencing, the object to be tested may include victims; or, in a physical examination scenario, the object to be tested may include the person undergoing the physical examination, and examples are not given one by one here. Furthermore, the object to be tested may not be limited to humans, but may also include zoo animals, pets, and the like.

[0037] In one implementation scenario, the target area to be detected in the medical image may include, but is not limited to, a fracture area, and may be specifically set according to actual application needs and is not limited here.

[0038] In one implementation scenario, taking the case where the target area includes a fracture area as an example, according to the location of the target area to be detected, the medical image may also be an image of different locations of the object to be detected. For example, the medical image may be a chest medical image of the object to be detected, in which case the fracture may be a rib fracture, that is, the chest medical image may be used to detect rib fractures; or the medical image may be a hand medical image of the object to be detected, in which case the fracture may be a hand bone fracture, that is, the hand bone fracture may be used to detect hand bone fractures, and other situations may be deduced by analogy, and examples are not given one by one here.

[0039] In one implementation scenario, the medical image may include, but is not limited to, a CT image and an MR (Magnetic Resonance) image.

[0040] In another implementation scenario, the medical image may be a three-dimensional image, or may be a two-dimensional image. For example, the two-dimensional image may be an axial slice of CT, which is not limited here.

[0041] In a specific implementation scenario, the medical image can be specifically set as a three-dimensional image, thereby avoiding the loss of spatial information due to using only axial slices, which is beneficial for obtaining changes on continuous slices during the detection process and improving detection accuracy.

[0042] Step S12: extracting features from the medical image to obtain a feature image.

[0043] In an implementation scenario, in order to improve detection efficiency, an image analysis model can be pre-trained, and the image analysis model can include a feature extraction subnetwork, so that the feature extraction subnetwork of the image analysis model can be used to extract features from medical images to obtain feature images.

[0044] In a specific implementation scenario, the feature extraction subnetwork may include a convolutional layer and a pooling layer.

[0045] In another specific implementation scenario, the image analysis model can be trained by sample medical images. For details, please refer to the following embodiment of the training process of the image analysis model, which will not be described here.

[0046] In another implementation scenario, taking the case where the target area includes a fracture area as an example, before performing feature extraction, the medical image can be detected to obtain a bone area containing bone pixels, and then based on the bone pixels in the bone area, regional extraction is performed on the medical image to obtain at least one regional image, and the center of the regional image is the bone pixel, and feature extraction is performed on at least one regional image to obtain a feature image. Therefore, feature extraction can be focused on the bone area, which can help reduce the interference of other areas on fracture detection, and can further help reduce missed detection of minor fractures, and further improve the accuracy of fracture detection.

[0047] In a specific implementation scenario, a medical image can be detected based on a threshold and shape to obtain a bone region containing bone pixels. Taking rib fracture detection as an example, a medical image can be converted into a binary image based on a preset threshold to extract bone structures (such as rib cage, sternum, clavicle, scapula), and since the sternum and scapula are shaped like "plates" and the ribs are shaped like "tubes", on this basis, the "plate degree" can be calculated based on Hessian analysis (the higher the "plate degree", the greater the possibility of plate-like bones), and the plate-like bones can be enhanced to obtain the sternum and scapula, and the detected sternum and scapula can be removed to finally obtain the rib structure.

[0048] In another specific implementation scenario, in order to improve the detection efficiency of the bone area, the above-mentioned image analysis model may further include a bone area detection subnetwork, so that the bone area detection subnetwork can be directly used to detect the medical image to obtain the bone area containing bone pixels. Specifically, the bone area detection subnetwork may include: U-Net, V-Net, etc., which are not limited here. Taking rib fracture detection as an example, a sample medical image marked with the actual rib area can be obtained in advance, and the sample medical image can be detected using the bone area detection subnetwork to obtain a predicted rib area containing rib pixels, so that the difference between the actual rib area and the predicted rib area can be used to adjust the network parameters of the bone area detection subnetwork. Other situations can be deduced by analogy, and examples are not given one by one here.

[0049] In another specific implementation scenario, the size of the regional image is a preset size, which can be set according to actual conditions and is not limited here. For example, when the medical image is a three-dimensional image, the regional image is an image of a preset length (e.g., 10), a preset width (e.g., 10), and a preset height (e.g., 10).

[0050] In another specific implementation scenario, before performing feature extraction on at least one regional image, bone pixel points on the skeleton line of the bone region can be obtained as candidate pixel points, and at least one candidate pixel point can be selected as the target pixel point, and the center of the regional image is the target pixel point. It should be noted that the skeleton line of the bone region is the line where the center of each cross section of the bone region is located. In addition, a candidate pixel point can be selected at a preset distance (e.g., 10) on the skeleton line as the target pixel point. Therefore, feature extraction can be focused on the bone region to improve the accuracy of fracture detection. At the same time, the number of regional images can be reduced, thereby reducing the amount of calculation and increasing the speed of fracture detection.

[0051] In another specific implementation scenario, before extracting features from at least one regional image, at least one regional image may be detected to obtain bone abnormalities of the regional image, so that only regional images whose bone abnormalities meet preset conditions can be subsequently extracted to obtain feature images. Therefore, "coarse to fine" fracture detection can be achieved, that is, regional images whose bone abnormalities meet preset conditions are preliminarily screened, so that regional images whose bone abnormalities do not meet preset conditions can be exempted from detection. On this basis, the screened regional images are subsequently finely identified, which can help further improve the fracture detection speed. Specifically, fractures, bone deformities, bone hyperplasia and other conditions that deviate from normal bones are all bone abnormalities. In practical applications, bone abnormalities can include bone abnormality scores, and the larger the bone abnormality score, the higher the degree of bone abnormality. The preset conditions can be set to include: the bone abnormality score is greater than the preset score threshold. In addition, in order to improve the efficiency of bone abnormality detection, the image analysis model can also include a bone abnormality detection subnetwork, so that the bone abnormality detection subnetwork can be used to detect at least one regional image respectively to obtain bone abnormalities of the regional image.

[0052] Step S13: Perform a first detection on the feature image to obtain a first detection result.

[0053] In the embodiment of the present disclosure, the first detection result indicates the possibility that a target region exists in the medical image. As mentioned above, the target region can be specifically set according to actual application needs, for example, it can include but is not limited to a fracture region.

[0054] In one implementation scenario, the first detection result may include a probability value of the target region existing in the medical image, and the larger the probability value, the higher the possibility of the target region existing. Still taking the target region including the fracture region as an example, the larger the probability value, the higher the possibility of the fracture region existing.

[0055] In one implementation scenario, in order to improve the efficiency of the first detection, the image analysis model may further include a first detection subnetwork, so that the first detection subnetwork may be used to perform a first detection on the feature image to obtain a first detection result. Specifically, the first detection subnetwork may include, but is not limited to: a convolutional layer, a pooling layer, and a fully connected layer, which are not limited here.

[0056] Step S14: Based on the first detection result, perform a second detection on the feature image to obtain a second detection result.

[0057] In the embodiment of the present disclosure, the second detection result includes attribute information of the target area. Still taking the example that the target area includes a fracture area, the attribute information of the fracture area may specifically include a fracture type.

[0058] In one implementation scenario, in order to improve the efficiency of the second detection, the image analysis model may further include a second detection subnetwork, so that the second detection subnetwork may be used to perform a second detection on the feature image to obtain a second detection result. Specifically, the second detection subnetwork may include, but is not limited to: a convolutional layer, a pooling layer, and a fully connected layer, which are not limited here.

[0059] In another implementation scenario, the fracture type can be set according to actual application needs. For example, the fracture type may include: complete fracture, incomplete fracture; in addition, the fracture type may also include: old fracture, new fracture, which is not limited here.

[0060] In another implementation scenario, still taking the target area including the fracture area as an example, the attribute information of the target area may further include the fracture location. The fracture location may specifically include, but is not limited to: the center of the fracture area, the size of the fracture area. For example, the center of the fracture area is the pixel point (48,64,100), and the size of the fracture area is 10*10*10. Then, it can be considered that the area centered on the pixel point (48,64,100) and with a size of 10*10*10 is the fracture area. Other situations can be deduced by analogy, and examples are not given one by one here. Therefore, the fracture area can be located according to the above center and size.

[0061] In one implementation scenario, as described above, the first detection result may include a probability value of the existence of the target area in the medical image. If the probability value is greater than a preset probability threshold, a second detection may be performed on the feature image to obtain a second detection result. In addition, if the probability value is not greater than the preset probability threshold, it may be directly indicated that the target area does not exist in the medical image.

[0062] In one implementation scenario, by combining the first test result and the second test result, a comprehensive test result can be obtained for reference by medical staff. For example, still taking the target area including the fracture area as an example, when the first test result shows that there is a high possibility that the fracture area exists in the medical image, the following comprehensive test result can be given based on the fracture type A included in the second test result: the object to be tested is more likely to have fracture type A; or, when the first test result shows that there is a low possibility that the fracture area exists in the medical image, the following comprehensive test result can be given based on the fracture type B included in the second test result: the possibility of fracture is low, and it is recommended to exclude whether there is fracture type B. The specific setting can be based on the actual application needs and is not limited here.

[0063] Please refer to Figure 2 , Figure 2 Schematic diagram of the framework of an embodiment of an image analysis model. Figure 2 As shown, the image analysis model may include a feature extraction subnetwork, a first detection subnetwork and a second detection subnetwork. The medical image is processed through the feature extraction subnetwork of the image analysis model to obtain a feature image. The feature image is sent to the first detection subnetwork and the second detection subnetwork respectively, and the first detection subnetwork outputs a first detection result, and the second detection subnetwork outputs a second detection result.

[0064] The above scheme obtains a medical image of the object to be tested, and extracts features from the medical image to obtain a feature image, thereby performing a first test on the feature image to obtain a first test result, and the first test result indicates the possibility of the existence of a target area in the medical image, and based on the first test result, performs a second test on the feature image to obtain a second test result, and the second test result includes attribute information of the target area. Therefore, it is possible to avoid manual image analysis, which can help improve the speed and accuracy of image analysis. In addition, since the first test on the possibility of the existence of the target area and the second test on the attribute information of the target area are both performed based on the feature image, that is, the first test and the second test share the feature image, the amount of calculation can be reduced compared to when the two tasks are performed completely independently. Therefore, the above method can improve the speed and accuracy of image analysis.

[0065] See also Figure 3 , Figure 3 1 is a flow chart of another embodiment of the image analysis method of the present application. In the embodiment of the present disclosure, the target area in the medical image may specifically include a fracture area. Specifically, the following steps may be included:

[0066] Step S31: Acquire a medical image of the object to be tested.

[0067] For details, please refer to the relevant description in the aforementioned disclosed embodiments, which will not be repeated here.

[0068] Step S32: Detect the medical image to obtain a bone region containing bone pixels.

[0069] The specific detection method of the bone area can refer to the relevant description in the aforementioned disclosed embodiment, which will not be repeated here.

[0070] In addition, as described in the aforementioned disclosed embodiment, in order to improve the efficiency of bone region detection, an image analysis model can be pre-trained, and the image analysis model includes a bone region detection subnetwork, so that the bone region detection subnetwork can be used to detect medical images to obtain bone regions containing bone pixels. For details about the bone region detection subnetwork, please refer to the relevant description in the aforementioned disclosed embodiment, which will not be repeated here.

[0071] Step S33: Obtain bone pixel points located on the skeleton line of the bone region as candidate pixel points, and select at least one candidate pixel point as a target pixel point.

[0072] For details, please refer to the relevant description in the aforementioned disclosed embodiments, which will not be repeated here.

[0073] Step S34: Based on the bone pixels in the bone region, region extraction is performed on the medical image to obtain at least one region image.

[0074] In the embodiment of the present disclosure, the center of the regional image is the target pixel. For details, please refer to the relevant description in the above-mentioned embodiment of the disclosure, which will not be repeated here.

[0075] Step S35: Detect at least one regional image respectively to obtain the bone abnormality of the regional image.

[0076] As described in the aforementioned disclosed embodiments, in order to improve the efficiency of bone abnormality detection, the image analysis model may further include a bone abnormality detection subnetwork, so that the bone abnormality detection subnetwork can be used to detect at least one regional image separately to obtain the bone abnormality of the regional image.

[0077] In a specific implementation scenario, the bone abnormality detection subnetwork may specifically include but is not limited to: a convolutional layer, a pooling layer, and a fully connected layer. That is, the bone abnormality detection subnetwork can be implemented through a lightweight network structure, which is conducive to reducing network parameters and reducing training difficulty and cost.

[0078] In another specific implementation scenario, sample medical images annotated with actual bone abnormalities (e.g., annotated with actual bone abnormality scores) can be obtained in advance, so that the sample medical images can be detected using a bone abnormality detection subnetwork to obtain predicted bone abnormalities (e.g., predicted bone abnormality scores), and then the difference between the actual bone abnormalities and the predicted bone abnormalities can be used to adjust the network parameters of the bone abnormality detection subnetwork.

[0079] In another specific implementation scenario, the bone abnormality detection subnetwork may also be independent of the image analysis model. In addition, the above-mentioned bone region detection subnetwork may also be independent of the image analysis model. Figure 2 As shown, the image analysis model may include only a feature extraction subnetwork, a first detection subnetwork, and a second detection subnetwork.

[0080] Step S36: extracting features from the regional images where the bone abnormalities meet the preset conditions to obtain feature images.

[0081] As described in the aforementioned disclosed embodiments, conditions that deviate from normal bones, such as fractures, bone deformities, and bone hyperplasia, are all considered bone abnormalities. In the disclosed embodiments, the bone abnormality condition includes a bone abnormality score, and the larger the bone abnormality score, the higher the degree of bone abnormality, and the preset conditions include: the bone abnormality score is greater than a preset score threshold.

[0082] In an implementation scenario, the preset score threshold can be set according to actual application needs. For example, in the case where the fracture detection accuracy requirement is relatively low, the preset score threshold can be set relatively large, so that only regional images with larger bone abnormality scores are screened, that is, no subsequent detection is performed on regional images with smaller bone abnormality scores; or, in the case where the fracture detection accuracy requirement is relatively high, the preset score threshold can be set relatively small, so that regional images with slightly larger bone abnormality scores are screened out for subsequent detection, which is not limited here.

[0083] Step S37: performing a first detection on the feature image to obtain a first detection result, and performing a second detection on the feature image based on the first detection result to obtain a second detection result.

[0084] In the disclosed embodiment, the first detection result indicates the possibility that a target region exists in the medical image, and the second detection result includes attribute information of the target region. For details, please refer to the relevant description in the aforementioned disclosed embodiment, which will not be repeated here.

[0085] Different from the above-mentioned embodiment, by acquiring a medical image of the object to be tested and detecting the medical image, a bone region containing bone pixels is obtained, thereby obtaining bone pixels on the skeleton line of the bone region as candidate pixels, and selecting at least one candidate pixel as a target pixel. Based on the bone pixels in the bone region, a region extraction is performed on the medical image to obtain at least one region image, and the center of the region image is the target pixel, which can make the subsequent feature extraction focus on the bone region, thereby being conducive to reducing the interference of other regions on fracture detection. On this basis, at least one region image is detected separately to obtain the bone abnormality of the region image, and feature extraction is performed on the region image whose bone abnormality meets the preset conditions to obtain a feature image, which can realize "coarse to fine" fracture detection, thereby improving the fracture detection speed, and finally performing a first detection on the feature image to obtain a first detection result, and based on the first detection result, performing a second detection on the feature image to obtain a second detection result, because the first detection and the second detection two detection tasks share the feature image, the amount of calculation can be reduced compared to the two tasks being completely independent. Therefore, the above method can improve the fracture detection speed and accuracy.

[0086] See also Figure 4 , Figure 4 This is a flow chart of an embodiment of the training process of an image analysis model. Specifically, the following steps may be included:

[0087] Step S41: Acquire a sample medical image.

[0088] In the disclosed embodiment, the sample medical image is annotated with a first mark and a second mark, the first mark represents whether there is a sample target area in the sample medical image, and the second mark represents sample attribute information of the sample target area. The meaning of the sample medical image can be specifically referred to the relevant description of the medical image in the aforementioned disclosed embodiment, which will not be repeated here. In addition, the sample target area can be specifically set according to the actual application needs, for example, it can include a sample fracture area. Taking the example that the sample target area includes a sample fracture area, the sample attribute information of the sample target area can include the sample fracture type of the sample fracture area, etc., which is not limited here.

[0089] Step S42: Use the feature extraction subnetwork of the image analysis model to extract features from the sample medical image to obtain a sample feature image.

[0090] For details, please refer to the related description of using the feature extraction subnetwork to extract features from medical images in the aforementioned disclosed embodiments, which will not be repeated here.

[0091] Step S43: Perform a first detection on the sample feature image using the first detection subnetwork of the image analysis model to obtain a first prediction result, and perform a second detection on the feature image using the second detection subnetwork of the image analysis model to obtain a second prediction result.

[0092] In the disclosed embodiment, the first prediction result indicates the possibility of the sample target region existing in the sample medical image, and the second prediction result includes the prediction attribute information of the sample target region. In one implementation scenario, the first prediction result may specifically include a first prediction probability value of the sample target region existing, and the higher the first prediction probability value, the greater the possibility of the sample target region existing; in another implementation scenario, taking the sample target region including the sample fracture region as an example, the prediction attribute information of the sample fracture region included in the second detection result may specifically include a second prediction probability value of at least one fracture type, for example, it may include a second prediction probability value of a new fracture and a second prediction probability value of an old fracture; or, it may include a second prediction probability value of a complete fracture and a second prediction probability value of an incomplete fracture, which is not limited here.

[0093] For details, please refer to the related description of using the first detection sub-network to perform the first detection and using the second detection sub-network to perform the second detection in the aforementioned disclosed embodiment, which will not be repeated here.

[0094] Step S44: adjusting the network parameters of the image analysis model based on the difference between the first label and the first prediction result, and the difference between the second label and the second prediction result.

[0095] In an implementation scenario, still taking the case where the sample target region includes a sample fracture region as an example, the first mark may indicate whether there is a sample fracture region in the sample medical image, and specifically, the first actual probability value of 100% may be used to indicate that there is a sample fracture region, and the first actual probability value of 0% may be used to indicate that there is no sample fracture region. In this case, a first loss value between the first actual probability value and the first predicted probability value may be calculated based on a cross entropy loss function.

[0096] In another implementation scenario, still taking the sample target area including the sample fracture area as an example, the second actual result may include the fracture type of the sample fracture area, and specifically, the second actual probability value of 100% may be used to indicate the presence of the corresponding fracture type, and the second actual probability value of 0% may be used to indicate the absence of the corresponding fracture type, for example, the second actual probability value of 0% for a new fracture and the second actual probability value of 100% for an old fracture, indicating that the fracture type of the sample fracture area is an old fracture; or, the second actual probability value of 0% for a complete fracture and the second actual probability value of 100% for an incomplete fracture, indicating that the fracture type of the sample fracture area is an incomplete fracture, which is not limited here. In this case, the second loss value between the second actual probability value and the second predicted probability value may be calculated based on the cross entropy loss function.

[0097] In another implementation scenario, the first loss value and the second loss value can be weighted using the first preset weight and the second preset weight respectively to obtain a weighted loss value, and based on the weighted loss value, the network parameters of the image analysis model are adjusted.

[0098] In a specific implementation scenario, the first preset weight and the second preset weight can be set according to actual application needs. For example, in the case of focusing on the prediction of fracture possibility, the first preset weight can be set to be greater than the second preset weight, such as the first preset weight is set to 0.6 and the second preset weight is set to 0.4; or, in the case of focusing on the prediction of fracture type, the first preset weight can be set to be less than the second preset weight, such as the first preset weight is set to 0.4 and the second preset weight is set to 0.6, which is not limited here.

[0099] In another specific implementation scenario, the parameters of the image analysis model can be adjusted using the above-mentioned weighted loss values ​​by using methods such as Stochastic Gradient Descent (SGD), Batch Gradient Descent (BGD), and Mini-Batch Gradient Descent (MBGD). Batch gradient descent refers to using all samples to update parameters at each iteration; Stochastic gradient descent refers to using one sample to update parameters at each iteration; Mini-batch gradient descent refers to using a batch of samples to update parameters at each iteration, which will not be repeated here.

[0100] In another specific implementation scenario, a training end condition may be set. When the training end condition is met, the training of the image analysis model may be terminated. Specifically, the training end condition may include any of the following: the loss value is less than a preset loss threshold, the current training times reaches a preset times threshold (e.g., 500 times, 1000 times, etc.), which is not limited here.

[0101] Different from the aforementioned embodiments, a sample medical image is obtained, and the sample medical image is annotated with a first mark and a second mark, the first mark represents whether a sample target area exists in the sample medical image, and the second mark represents sample attribute information of the sample target area, and the feature extraction subnetwork of the image analysis model is used to extract features of the sample medical image to obtain a sample feature image, and then the first detection subnetwork of the image analysis model is used to perform a first detection on the sample feature image to obtain a first prediction result, and the second detection subnetwork of the image analysis model is used to perform a second detection on the feature image to obtain a second prediction result, and the first prediction result indicates the possibility of the existence of the sample target area in the sample medical image, and the second prediction result includes the predicted attribute information of the sample target area, and then based on the difference between the first mark and the first prediction result, and the difference between the second mark and the second prediction result, the network parameters of the image analysis model are adjusted. Therefore, in the training process of the image analysis model, the first detection subnetwork and the second detection subnetwork can be jointly trained, so that the first detection task and the second detection task can be supervised by each other, and then the accuracy of the image analysis model can be improved.

[0102] See also Figure 5 , Figure 5 : is a schematic diagram of a framework of an embodiment of an image analysis device 50 of the present application. The image analysis device 50 includes: an image acquisition module 51, a feature extraction module 52, a first detection module 53, and a second detection module 54. The image acquisition module 51 is used to acquire a medical image of an object to be detected; the feature extraction module 52 is used to extract features from the medical image to obtain a feature image; the first detection module 53 is used to perform a first detection on the feature image to obtain a first detection result, wherein the first detection result indicates the possibility of the existence of a target area in the medical image; the second detection module 54 is used to perform a second detection on the feature image based on the first detection result to obtain a second detection result; wherein the second detection result includes attribute information of the target area.

[0103] The above scheme obtains a medical image of the object to be tested, and extracts features from the medical image to obtain a feature image, thereby performing a first test on the feature image to obtain a first test result, and the first test result indicates the possibility of the existence of a target area in the medical image, and based on the first test result, performs a second test on the feature image to obtain a second test result, and the second test result includes attribute information of the target area. Therefore, it is possible to avoid manual image analysis, which can help improve the speed and accuracy of image analysis. In addition, since the first test on the possibility of the existence of the target area and the second test on the attribute information of the target area are both performed based on the feature image, that is, the first test and the second test share the feature image, the amount of calculation can be reduced compared to when the two tasks are performed completely independently. Therefore, the above method can improve the speed and accuracy of image analysis.

[0104] In some disclosed embodiments, the target area includes a fracture area, and the image analysis device 50 also includes a third detection module for performing a third detection on the medical image to obtain a bone area containing bone pixels. The feature extraction module 52 includes a region extraction submodule for performing region extraction in the medical image based on the bone pixels in the bone area to obtain at least one regional image; wherein the center of the regional image is the bone pixel, and the feature extraction module 52 includes a feature extraction submodule for performing feature extraction on at least one regional image respectively to obtain a feature image.

[0105] Different from the aforementioned embodiment, the target area includes a fracture area. Before feature extraction is performed on the medical image, a third detection is first performed on the medical image to obtain a bone area containing bone pixels, and then based on the bone pixels in the bone area, region extraction is performed on the medical image to obtain at least one regional image, and the center of the regional image is the bone pixel, and then feature extraction is performed on at least one regional image to obtain a feature image. Therefore, feature extraction can be focused on the bone area, which can help reduce the interference of other areas on fracture detection, and can help reduce the missed detection of minor fractures, and further improve the accuracy of fracture detection.

[0106] In some disclosed embodiments, the feature extraction module 52 also includes a pixel extraction submodule for acquiring bone pixel points located on the skeleton line of the bone region as candidate pixel points, and selecting at least one candidate pixel point as a target pixel point; wherein the center of the regional image is the target pixel point.

[0107] Different from the aforementioned embodiment, by obtaining bone pixel points on the skeleton line of the bone area as candidate pixel points, and selecting at least one candidate pixel point as the target pixel point, and the center of the regional image is the target pixel point, feature extraction can be focused on the bone area, thereby improving the accuracy of fracture detection. At the same time, the number of regional images can be reduced, thereby reducing the amount of calculation and improving the speed of fracture detection.

[0108] In some disclosed embodiments, the feature extraction module 52 also includes an abnormality detection submodule, which is used to perform a fourth detection on at least one regional image to obtain bone abnormalities of the regional image. The feature extraction submodule is specifically used to perform feature extraction on regional images whose bone abnormalities meet preset conditions to obtain feature images.

[0109] Different from the aforementioned embodiment, before feature extraction is performed on at least one regional image to obtain a feature image, a fourth detection is performed on at least one regional image to obtain the bone abnormality of the regional image, and feature extraction is performed on the regional image whose bone abnormality meets the preset conditions to obtain the feature image. Therefore, "coarse to fine" fracture detection can be achieved, that is, the regional images whose bone abnormality meets the preset conditions are preliminarily screened, thereby avoiding the detection of regional images whose bone abnormality does not meet the preset conditions. On this basis, the screened regional images are subsequently finely identified, which can help to further improve the fracture detection speed.

[0110] In some disclosed embodiments, the bone abnormality condition includes a bone abnormality score, and the preset conditions include: the bone abnormality score is greater than a preset score threshold.

[0111] Different from the aforementioned embodiment, by setting the bone abnormality condition to include a bone abnormality score, the preset condition is set to include: the bone abnormality score is greater than the preset score threshold, which can help avoid detecting images of areas with low bone abnormalities and screen images of areas with high bone abnormalities for subsequent fine identification, which can help improve the speed of fracture detection.

[0112] In some disclosed embodiments, the feature extraction module 52 is specifically used to perform feature extraction on the medical image using the feature extraction subnetwork of the image analysis model to obtain a feature image, the first detection module 53 is specifically used to perform a first detection on the feature image using the first detection subnetwork of the image analysis model to obtain a first detection result, and the second detection module 54 is specifically used to perform a second detection on the feature image using the second detection subnetwork of the image analysis model to obtain a second detection result.

[0113] Different from the aforementioned embodiments, performing feature extraction on medical images through the feature extraction subnetwork of the image analysis model can help improve the feature extraction efficiency; performing a first detection on the feature image through the first detection subnetwork of the image analysis model can help improve the efficiency of the first detection; and performing a second detection on the feature image through the second detection subnetwork of the image analysis model can help improve the efficiency of the second detection.

[0114] In some disclosed embodiments, the image analysis device 50 also includes a sample acquisition module for acquiring a sample medical image; wherein the sample medical image is annotated with a first label and a second label, the first label represents whether there is a sample target area in the sample medical image, and the second label represents sample attribute information of the sample target area, the image analysis device 50 also includes a sample extraction module for extracting features from the sample medical image using a feature extraction subnetwork of the image analysis model to obtain a sample feature image, the image analysis device 50 also includes a sample detection module for performing a first detection on the sample feature image using a first detection subnetwork of the image analysis model to obtain a first prediction result, and performing a second detection on the feature image using a second detection subnetwork of the image analysis model to obtain a second prediction result; wherein the first prediction result indicates the possibility of the presence of the sample target area in the sample medical image, and the second prediction result includes the predicted attribute information of the sample target area, the image analysis device 50 also includes a parameter adjustment module for adjusting the network parameters of the image analysis model based on the difference between the first label and the first prediction result, and the difference between the second label and the second prediction result.

[0115] Different from the aforementioned embodiments, a sample medical image is obtained, and the sample medical image is annotated with a first mark and a second mark, the first mark represents whether a sample target area exists in the sample medical image, and the second mark represents sample attribute information of the sample target area, and the feature extraction subnetwork of the image analysis model is used to extract features of the sample medical image to obtain a sample feature image, and then the first detection subnetwork of the image analysis model is used to perform a first detection on the sample feature image to obtain a first prediction result, and the second detection subnetwork of the image analysis model is used to perform a second detection on the feature image to obtain a second prediction result, and the first prediction result indicates the possibility of the existence of the sample target area in the sample medical image, and the second prediction result includes the predicted attribute information of the sample target area, and then based on the difference between the first mark and the first prediction result, and the difference between the second mark and the second prediction result, the network parameters of the image analysis model are adjusted. Therefore, in the training process of the image analysis model, the first detection subnetwork and the second detection subnetwork can be jointly trained, so that the first detection task and the second detection task can be supervised by each other, and then the accuracy of the image analysis model can be improved.

[0116] In some disclosed embodiments, the medical image is a chest medical image of a subject to be tested, the target area includes a fracture area, and the attribute information includes the fracture type of the fracture area; and / or, the first detection result includes a probability value of the existence of the target area in the medical image, and the larger the probability value, the higher the possibility of the existence of the target area.

[0117] Different from the aforementioned embodiment, the medical image is set to a chest medical image of the object to be tested, the target area is set to include the fracture area, and the attribute information is set to include the fracture type of the fracture area, which can be beneficial to fracture detection; the first detection result is set to a probability value of the existence of the target area in the medical image, and the larger the probability value, the higher the possibility of the existence of the target area, which can be beneficial to quantifying the possibility of the existence of the target area, so that the user can judge whether there is a target area based on the probability value.

[0118] See also Figure 6 , Figure 6 : is a schematic diagram of a framework of an embodiment of an electronic device 60 of the present application. The electronic device 60 includes a memory 61 and a processor 62 coupled to each other, and the processor 62 is used to execute program instructions stored in the memory 61 to implement the steps of any of the above-mentioned image analysis method embodiments. In a specific implementation scenario, the electronic device 60 may include but is not limited to: a microcomputer, a server, and in addition, the electronic device 60 may also include a mobile device such as a laptop computer and a tablet computer, which is not limited here.

[0119] Specifically, the processor 62 is used to control itself and the memory 61 to implement the steps of any of the above-mentioned image analysis method embodiments. The processor 62 can also be called a CPU (Central Processing Unit). The processor 62 may be an integrated circuit chip with signal processing capabilities. The processor 62 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 62 can be implemented by an integrated circuit chip.

[0120] The above scheme can improve the speed and accuracy of image analysis.

[0121] See also Figure 7 , Figure 7The schematic diagram of the framework of an embodiment of a computer-readable storage medium 70 of the present application. The computer-readable storage medium 70 stores program instructions 701 that can be executed by a processor, and the program instructions 701 are used to implement the steps of any of the above-mentioned image analysis method embodiments.

[0122] The above scheme can improve the speed and accuracy of image analysis.

[0123] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0124] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0125] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0126] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0127] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0128] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

Claims

1. An image analysis method, characterized in that: include: Acquiring a medical image of the object to be tested; Extracting features from the medical image using a feature extraction subnetwork of an image analysis model to obtain a feature image; Performing a first detection on the feature image using a first detection subnetwork of the image analysis model to obtain a first detection result, wherein the first detection result includes a probability value of the existence of the target area in the medical image, and the greater the probability value, the higher the possibility of the existence of the target area; and Based on the first detection result, a second detection is performed on the feature image using a second detection subnetwork of the image analysis model to obtain a second detection result, wherein the second detection result includes attribute information of the target area.

2. The method according to claim 1, characterized in that The target area includes a fracture area; before the feature extraction subnetwork of the image analysis model is used to extract features from the medical image to obtain a feature image, the method includes: Performing a third detection on the medical image to obtain a bone region containing bone pixels; and The step of extracting features from the medical image to obtain a feature image comprises: Based on the bone pixel points in the bone region, extracting at least one regional image from the medical image, wherein the center of the regional image is the bone pixel point; Feature extraction is performed on the at least one regional image to obtain the feature image.

3. The method according to claim 2, characterized in that Before extracting at least one regional image from the medical image based on the bone pixels in the bone region, the method further includes: Acquire bone pixel points located on the skeleton line of the bone region as candidate pixel points, and select at least one of the candidate pixel points as a target pixel point; The center of the regional image is the target pixel point.

4. The method according to claim 2 or 3, characterized in that: Before extracting features from the at least one regional image to obtain the feature image, the method further includes: performing a fourth detection on the at least one regional image respectively to obtain a bone abnormality of the regional image; and The extracting features from the at least one regional image respectively to obtain the feature image comprises: Feature extraction is performed on the regional image where the bone abnormality meets the preset conditions to obtain the feature image.

5. The method according to claim 4, characterized in that The bone abnormality condition includes a bone abnormality score, and the preset condition includes: the bone abnormality score is greater than a preset score threshold.

6. The method according to claim 1, characterized in that The training steps of the image analysis model include: Acquire a sample medical image, wherein the sample medical image is annotated with a first mark and a second mark, the first mark represents whether a sample target area exists in the sample medical image, and the second mark represents sample attribute information of the sample target area; Using the feature extraction subnetwork of the image analysis model to extract features from the sample medical image to obtain a sample feature image; Performing a first detection on the sample feature image using the first detection subnetwork of the image analysis model to obtain a first prediction result, and performing a second detection on the feature image using the second detection subnetwork of the image analysis model to obtain a second prediction result; wherein the first prediction result indicates the possibility that a sample target region exists in the sample medical image, and the second prediction result includes prediction attribute information of the sample target region; Based on the difference between the first label and the first prediction result, and the difference between the second label and the second prediction result, the network parameters of the image analysis model are adjusted.

7. The method according to claim 1, characterized in that The medical image is a chest medical image of the subject to be tested, the target area includes a fracture area, and the attribute information includes a fracture type of the fracture area.

8. An image analysis device, characterized in that: include: An image acquisition module, used to acquire a medical image of the object to be tested; A feature extraction module, used to extract features from the medical image using a feature extraction subnetwork of an image analysis model to obtain a feature image; A first detection module, configured to perform a first detection on the feature image using a first detection subnetwork of the image analysis model to obtain a first detection result, wherein the first detection result includes a probability value of the existence of the target area in the medical image, and the larger the probability value, the higher the possibility of the existence of the target area; A second detection module is used to perform a second detection on the feature image based on the first detection result using a second detection subnetwork of the image analysis model to obtain a second detection result, wherein the second detection result includes attribute information of the target area.

9. An electronic device, characterized in that: It comprises a memory and a processor coupled to each other, wherein the processor is used to execute program instructions stored in the memory to implement the image analysis method according to any one of claims 1 to 7.

10. A computer-readable storage medium having program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the image analysis method according to any one of claims 1 to 7 is implemented.

Images