Method of processing an ultrasound image, ultrasound device and storage medium

Abstract

The application provides an ultrasound image processing method, an ultrasound device and a storage medium. The ultrasound image processing method comprises: inputting an ultrasound image of a target object into a trained target detection model to obtain a first attention region in the ultrasound image and a first quantity corresponding to the first attention region. Based on the first attention region and the first quantity, the bone erosion degree of the target object is determined. If the bone erosion degree meets a preset condition, at least the ultrasound image is displayed. The target detection model in the above method is trained based on an ultrasound training image, a second attention region corresponding to the ultrasound training image and a second quantity corresponding to the ultrasound training image. The above scheme avoids the influence of the clinical experience of the user on the accuracy of the determined bone erosion degree through an automatic means, and can also reduce the labor cost, especially in the case of an ultrasound image being an image in an ultrasound video.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and more specifically to a method for processing ultrasound images, an ultrasound device, a storage medium, and a computer program product. Background Technology

[0002] Rheumatic and immunological diseases are a group of chronic diseases that primarily affect connective tissues such as joints, bones, muscles, and blood vessels. They are typically characterized by a long course and recurrent flare-ups. Bone erosion is a common and serious complication in the progression of rheumatic and immunological diseases, usually manifesting as interruption of the bone cortex.

[0003] Ultrasound technology has significant advantages in the detection and assessment of bone erosion, as it can clearly display the morphology of periarticular soft tissues, synovium, cartilage, and cortical bone. Currently, the diagnosis of bone erosion typically involves the user combining ultrasound images of the subject's joint with clinical experience to determine the degree of erosion. However, the accuracy of the obtained degree of erosion is significantly influenced by the user's clinical experience; furthermore, in scenarios involving ultrasound scanning, users usually need to select ultrasound images from the obtained ultrasound video before determining the degree of erosion, resulting in high manual costs. Therefore, users urgently need a more efficient and accurate method for processing ultrasound images. Summary of the Invention

[0004] The present invention was proposed in view of the above-mentioned problems. The present invention provides a method for processing ultrasound images, an ultrasound device, an electronic device, a storage medium, and a computer program product.

[0005] According to one aspect of the present invention, a method for processing ultrasound images is provided. The method includes: inputting an ultrasound image of a target object into a trained target detection model to obtain a first region of interest in the ultrasound image and a first number corresponding to the first region of interest, wherein the first number represents the number of first bone erosion regions included in the first region of interest, the trained target detection model is trained based on an ultrasound training image, a second region of interest corresponding to the ultrasound training image, and a second number corresponding to the ultrasound training image, the second number representing the number of second bone erosion regions included in the ultrasound training image; determining the degree of bone erosion of the target object based on the first region of interest and the first number; and displaying at least the ultrasound image if the degree of bone erosion meets a preset condition.

[0006] For example, determining the degree of bone erosion of a target object based on a first region of interest and a first quantity includes:

[0007] If the first number is less than the number threshold, the first bone erosion area in the first area of ​​interest is determined at least based on the image corresponding to the first area of ​​interest in the ultrasound image;

[0008] The degree of bone erosion of the target object is determined based at least on the first bone erosion area in the first region of interest.

[0009] For example, determining a first bone erosion region within the first region of interest, at least based on the image corresponding to the first region of interest in the ultrasound image, includes:

[0010] Using the first quantity as a prompt word, the image corresponding to the first region of interest in the ultrasound image is input into the trained semantic segmentation model to obtain the first bone erosion region of the first region of interest with the first quantity.

[0011] For example, the degree of bone erosion of the target object is determined based at least on the first bone erosion area in the first region of interest, including any of the following:

[0012] The degree of bone erosion of the target object is determined based on the area size of the first bone erosion area in the first region of interest;

[0013] The degree of bone erosion of the target object is determined based on the area size and number of the first bone erosion area in the first region of interest.

[0014] For example, the ultrasound image whose degree of bone erosion meets the preset conditions is the target frame. The preset conditions include at least that the degree of bone erosion corresponding to the target frame is the highest among the multiple pre-selected video frames included in the ultrasound video.

[0015] For example, the preset condition also includes that the size of the first bone erosion region of the target frame is the largest among multiple pre-selected video frames.

[0016] For example, the ultrasound image showing a bone erosion degree that meets preset conditions is the target frame, and at least the ultrasound image shown includes:

[0017] Display the target frame and at least one of the following: the first bone erosion region marked in the target frame, the degree of bone erosion corresponding to the target frame, and the region size of the first bone erosion region in the target frame.

[0018] According to another aspect of the present invention, an ultrasound device is also provided, comprising a memory and a processor, wherein: the memory is used to store a computer program; and the processor is used to execute the computer program to implement the above-described ultrasound image processing method.

[0019] According to another aspect of the present invention, a storage medium is also provided. Program instructions are stored on this storage medium, which, when executed, are used to perform the aforementioned ultrasound image processing method.

[0020] According to another aspect of the present invention, a computer program product is also provided. This computer program product includes computer program instructions that, when executed by a processor, are used to perform the above-described ultrasound image processing method.

[0021] According to the above-described scheme of the present invention, an ultrasound image of the target object can be input into a trained target detection model to obtain a first region of interest and a first quantity corresponding to the first region of interest in the ultrasound image. Then, based on the first region of interest and the first quantity, the degree of bone erosion of the target object is determined. Finally, if the degree of bone erosion meets a preset condition, at least the ultrasound image is displayed. On the one hand, the above scheme avoids the influence of the user's clinical experience on the accuracy of the determined degree of bone erosion through automation. On the other hand, the target detection model in the above scheme is trained based on ultrasound training images, a second region of interest corresponding to the ultrasound training images, and a second quantity corresponding to the ultrasound training images. Compared to a scheme that directly determines the bone erosion area and then counts the number of bone erosion areas, the above-described scheme of the present invention is more accurate for cases with severe bone erosion. Specifically, because the degree of bone erosion is severe, it is difficult to calculate the size of the bone erosion area in the ultrasound image. If the quantity is counted based on this, the determined degree of bone erosion will not match the actual situation of the target object. However, the above-described scheme of the present invention can directly output the first quantity through the target detection model, thus its determination of the degree of bone erosion is more accurate. In addition, the above solution can also determine whether to display ultrasound images by setting preset conditions, which can reduce labor costs. Especially when the ultrasound image is an image in an ultrasound video, it can improve the representativeness of the displayed ultrasound image and provide better auxiliary effects. Attached Figure Description

[0022] The above and other objects, features, and advantages of the present invention will become more apparent from the more detailed description of the embodiments of the invention in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same parts or steps.

[0023] Figure 1 A schematic flowchart of a method for processing ultrasound images according to an embodiment of the present invention is shown;

[0024] Figure 2 A schematic diagram of a first region of interest according to an embodiment of the present invention is shown;

[0025] Figure 3 A schematic block diagram of an ultrasound image processing apparatus according to an embodiment of the present invention is shown;

[0026] Figure 4 A schematic block diagram of an electronic device according to an embodiment of the present invention is shown. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the present invention more apparent, exemplary embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely a part of the embodiments of the present invention, and not all of the embodiments of the present invention. It should be understood that the present invention is not limited to the exemplary embodiments described herein. Based on the embodiments of the present invention described herein, all other embodiments obtained by those skilled in the art without inventive effort should fall within the protection scope of the present invention.

[0028] In related technologies, users primarily rely on visually observing ultrasound images or videos of subjects to identify and manually count the areas of bone erosion (hereinafter referred to as bone erosion areas), thereby determining the severity of bone erosion (hereinafter referred to as bone erosion degree). However, this manual determination method is highly dependent on the user's professional experience and subjective judgment. Assessment results obtained by different users, and even by the same user at different times, may vary significantly, leading to a decrease in the accuracy and reliability of the assessment results. Furthermore, manual analysis is time-consuming, labor-intensive, and inefficient when dealing with a large number of ultrasound images or long-duration ultrasound videos.

[0029] To at least partially solve the above problems, embodiments of the present invention provide a method for processing ultrasound images. Figure 1 A schematic flowchart illustrating a method for processing ultrasound images according to an embodiment of the present invention is shown. Figure 1 As shown, the method may include the following steps S110 to S130.

[0030] In step S110, the ultrasound image of the target object is input into the trained target detection model to obtain the first region of interest in the ultrasound image and the first quantity corresponding to the first region of interest.

[0031] The target subject can be the aforementioned test subject. The test subject can be any object that requires in vivo observation by a user using an ultrasound device. The aforementioned ultrasound device can be a musculoskeletal ultrasound device, etc., used for observing the skeleton of the target subject.

[0032] The first region of interest in an ultrasound image can be a region encompassing all first bone erosion regions in the ultrasound image. The first bone erosion region is the area in the ultrasound image corresponding to the location where the bone of the target object is eroded. For example, the first region of interest can be the smallest rectangular region encompassing all first bone erosion regions in the ultrasound image. As another example, the first region of interest can be the smallest circular region encompassing all first bone erosion regions in the ultrasound image. It is understood that the shape of the first region of interest can be determined according to actual needs. Specifically, for example... Figure 2 A schematic diagram of a first region of interest according to an embodiment of the present invention is shown. Figure 2 As shown, a first region of interest can be marked in the ultrasound image. This first region of interest can be the smallest rectangular area containing all the first bone erosion areas in the ultrasound image. It is understood that the size of the first region of interest can be positively correlated with parameters such as the number of first bone erosion areas (equivalent to the first number below), the dispersion of the distribution of the first bone erosion areas, and the size of the first bone erosion areas.

[0033] The first quantity can be used to indicate the number of first bone erosion areas included in the first region of concern. For example, see [link to relevant documentation]. Figure 2 As shown, in an ultrasound image with 0 first bone erosion areas, the first quantity corresponding to the first region of interest can be used to represent 0. In an ultrasound image with 1 first bone erosion area, the first quantity corresponding to the first region of interest can be used to represent 1. In an ultrasound image with 2 first bone erosion areas, the first quantity corresponding to the first region of interest can be used to represent 2. In an ultrasound image with 3 or more first bone erosion areas, the first quantity corresponding to the first region of interest can be used to represent 3 or more.

[0034] The object detection model mentioned above can be one of the following: Faster R-CNN, RetinaNet, YOLO series models, etc.

[0035] The trained target detection model can be trained based on ultrasound training images, a second region of interest corresponding to the ultrasound training images, and a second number of ultrasound training images. For example, the ultrasound training images mentioned above are ultrasound images used for training. The second region of interest corresponding to the ultrasound training images is the smallest rectangular region containing all second bone erosion regions in the ultrasound image (here, a rectangular region is used as an example; the shape of the second region of interest is not limited in this invention). The second bone erosion region is the region in the ultrasound training image where the bone is eroded. The second number is used to represent the number of second bone erosion regions included in the ultrasound training image. Specifically, for example, this embodiment of the invention provides a training method for a target detection model for reference. A first training dataset is obtained. The target detection model is iteratively trained using the first training dataset, and the model parameters of the target detection model are adjusted using a loss function until training is complete. The first training data mentioned above may include multiple ultrasound training images, and the label of each ultrasound training image is the second region of interest in the ultrasound training image, and the second number of second bone erosion regions in the second region of interest. The conditions for the completion of the training may be that the loss value calculated by the loss function tends to stabilize, the number of iterations reaches a preset number, etc. Based on the above training process, a target detection model for ultrasound images can be obtained to determine the first region of interest (ROI) and the corresponding first number of ROIs in the ultrasound training image. It is understood that the second ROI and the corresponding second number of ROIs in the ultrasound training image can be manually defined. It is also understood that the location of each second ROI can be represented by a binary mask (where 1 represents the second ROI and 0 represents the area outside the second ROI) or by coordinates, for example, the coordinates of each pixel in the second ROI within the ultrasound training image.

[0036] In step S120, the degree of bone erosion of the target object is determined based on the first region of interest and the first quantity.

[0037] The aforementioned bone erosion levels can be used to indicate the severity of bone erosion in the region of first concern. For example, bone erosion levels can include multiple grades, each corresponding to a different degree of bone erosion. Specifically, bone erosion levels can include four grades: No bone erosion indicates no bone erosion in the region of first concern; Grade 1 indicates mild bone erosion in the region of first concern; Grade 2 indicates moderate bone erosion in the region of first concern; and Grade 3 indicates severe bone erosion in the region of first concern. It is understood that the number of grades corresponding to bone erosion levels, and the degree of bone erosion corresponding to each grade, can be determined based on the specific circumstances.

[0038] This invention provides an example for reference. The degree of bone erosion of a target object can be determined based on the area of ​​a first region of interest, the degree of image texture disorder and roughness of the ultrasound image in the first region of interest, and a first quantity. It is understood that the degree of image texture disorder and roughness of the ultrasound image in the first region of interest can be extracted as quantitative indicators using a texture analysis algorithm. Specifically, for example, since the area of ​​the first region of interest corresponds to the extent of bone erosion, the area of ​​the first region of interest can be positively correlated with the severity of bone erosion of the target object (equivalent to the level of bone erosion). Because uneroded bone has smooth, continuous echoes on ultrasound images, while bone erosion disrupts the continuity of the bone cortex, the image corresponding to the eroded bone will appear darker in the ultrasound image. If bone hyperplasia exists around the eroded site, its corresponding image will appear brighter in the ultrasound image, and the boundaries of the image will become rough. Therefore, the more disordered and rougher the image texture of the ultrasound image in the first region of interest, the more severe the bone erosion of the target object. That is, the degree of image texture disorder and roughness of the ultrasound image in the first region of interest can be positively correlated with the severity of bone erosion of the target object. Since the first quantity represents the number of first bone erosion regions included in the first region of interest, a larger value indicates a greater number of first bone erosion regions within the first region of interest, and thus a more severe degree of bone erosion in the target object. Therefore, the value of the first quantity can be positively correlated with the severity of bone erosion in the target object. Depending on the actual situation, weights can be assigned to the area of ​​the first region of interest, the degree of disorder and roughness of the ultrasound image texture within the first region of interest, and the first quantity. The product of each parameter and its corresponding weight can be summed to obtain a sum value used to determine the degree of bone erosion in the target object. The degree of bone erosion in the target object can be determined based on this sum value. For example, this sum value can be positively correlated with the degree of bone erosion in the target object.

[0039] In step S130, if the degree of bone erosion meets the preset conditions, at least an ultrasound image is displayed.

[0040] The aforementioned preset conditions can be used to filter several ultrasound images to identify those with higher representativeness. For example, the preset conditions may include a degree of bone erosion greater than a degree threshold, or an area of ​​the first region of interest greater than an area threshold. This embodiment of the invention is not limited thereto.

[0041] For displaying ultrasound images, the ultrasound image can be displayed when the display trigger conditions are met. For example, the ultrasound image can be displayed directly when the degree of bone erosion meets the preset conditions; the ultrasound image can be displayed when a display command is received from the user via physical buttons or interface controls; or, when it is determined that the report display state has been entered, the ultrasound image can be displayed in the image display area of ​​the report.

[0042] According to the above-described scheme of the present invention, an ultrasound image of the target object can be input into a trained target detection model to obtain a first region of interest and a first quantity corresponding to the first region of interest in the ultrasound image. Then, based on the first region of interest and the first quantity, the degree of bone erosion of the target object is determined. Finally, if the degree of bone erosion meets a preset condition, at least the ultrasound image is displayed. On the one hand, the above scheme can avoid the influence of the user's clinical experience on the accuracy of the determined degree of bone erosion through automation; on the other hand, the target detection model in the above scheme is trained based on an ultrasound training image, a second region of interest corresponding to the ultrasound training image, and a second quantity corresponding to the ultrasound training image. Compared to a scheme that directly determines the bone erosion area and then counts the number of bone erosion areas, the above-described scheme of the present invention is more accurate for cases with severe bone erosion. Specifically, because the degree of bone erosion is severe, it is difficult to calculate the size of the bone erosion area in the ultrasound image. If the quantity is counted based on this, the determined degree of bone erosion will not match the actual situation of the target object. The above-described scheme of the present invention can directly output the first quantity through the target detection model, thus its determination of the degree of bone erosion is more accurate. In addition, the above solution can also determine whether to display ultrasound images by setting preset conditions, which can reduce labor costs. Especially when the ultrasound image is an image in an ultrasound video, it can improve the representativeness of the displayed ultrasound image and provide better auxiliary effects.

[0043] For example, step S120 above, which determines the degree of bone erosion of the target object based on the first area of ​​interest and the first quantity, includes steps S121 and S122.

[0044] In step S121, if the first quantity is less than the quantity threshold, the first bone erosion region in the first region of interest is determined based at least on the image corresponding to the first region of interest in the ultrasound image.

[0045] This invention provides an example for reference. Taking a quantity threshold of 3 as an example, see [reference needed]. Figure 2 As shown, in an ultrasound image with one first bone erosion region, the first quantity corresponding to the first region of interest can be used to represent 1. The value represented by this first quantity is less than the aforementioned quantity threshold. This can be based on... Figure 2The first bone erosion region within the first interest area is determined using an ultrasound image containing one such region and a corresponding first interest area, along with traditional image processing algorithms (e.g., edge detection methods, thresholding methods, etc.). It is understood that the aforementioned quantity threshold can be determined based on actual circumstances. For example, the value represented by the highest corresponding bone erosion level can be used as the quantity threshold. Specifically, if the first quantity is greater than or equal to 3, indicating the bone erosion level is the third level (equivalent to severe bone erosion in the first interest area), then 3 can be used as the quantity threshold. It is also understood that the first bone erosion region within the first interest area can be determined not only using traditional image processing algorithms but also through manual judgment, feature point matching, template matching, etc. Furthermore, it is understood that when the value represented by the first quantity is greater than or equal to the quantity threshold, the bone erosion level of the target object can be directly determined as the highest level of bone erosion. Taking the aforementioned threshold of 3, and the first quantity representing a level of bone erosion greater than or equal to 3 (as described above, the third level), as an example, when the first quantity represents a level of bone erosion greater than or equal to 3 (the threshold in this example), the bone erosion level of the target object can be directly determined to be the third level, indicating severe bone erosion in the first region of interest. When the first quantity represents 0, the bone erosion level of the target object can be directly determined to be the level of no bone erosion, which indicates that no bone erosion has occurred in the first region of interest.

[0046] In step S122, the degree of bone erosion of the target object is determined based at least on the first bone erosion area in the first region of interest.

[0047] In one example, the degree of bone erosion of the target object can be determined based on at least one of the number (equivalent to the first quantity mentioned above) and size of the first bone erosion area in the first region of concern. Specifically, for example, when the number of the first bone erosion areas in the first region of concern is 0, the degree of bone erosion of the target object can be determined to be the no-bone-erosion level in step S120 above, meaning that no bone erosion has occurred in the first region of concern. When the number of the first bone erosion areas in the first region of concern is 1 or 2, the degree of bone erosion of the target object can be determined based on the size and the first quantity of the first bone erosion areas. When the number of the first bone erosion areas in the first region of concern is greater than or equal to 3, the degree of bone erosion of the target object can be determined to be the third level in step S120 above, meaning that severe bone erosion has occurred in the first region of concern.

[0048] In another example, the degree of bone erosion of the target object can be determined based on the location of the first bone erosion region within the first region of interest. For example, if there is no first bone erosion region within the first region of interest, the degree of bone erosion of the target object can be determined to be the "no bone erosion" level in step S120 above. If the erosion locations of the first bone erosion regions within the first region of interest are all located on non-weight-bearing surfaces or less stressed areas (e.g., joint edges rather than the center of the joint surface directly bearing pressure), the degree of bone erosion of the target object can be determined to be the first level in step S120 above. If the erosion location of at least one first bone erosion region within the first region of interest is a critical anatomical location (e.g., near the edge of the joint surface or tendon attachment point), and there is no first bone erosion region located on the main weight-bearing surface or multiple important anatomical structures, the degree of bone erosion of the target object can be determined to be the second level in step S120 above. If the erosion location of at least one first bone erosion region within the first region of interest is a main weight-bearing surface or multiple important anatomical structures (e.g., the distal ulna in the wrist joint, the femoral condyle in the knee joint, etc.), the degree of bone erosion of the target object can be determined to be the third level in step S120 above.

[0049] According to the above-described scheme of the present invention, when the first number is less than a number threshold, the first bone erosion region within the first region of interest can be determined at least based on the image corresponding to the first region of interest in the ultrasound image. Furthermore, the degree of bone erosion of the target object can be determined at least based on the first bone erosion region within the first region of interest. The above scheme determines the location of the first bone erosion region only when the first number is less than the number threshold, and performs a refined bone erosion region analysis of the target object's bone erosion degree based on the location of the first bone erosion region, effectively avoiding the waste of resources from high-cost refined processing of all samples. Furthermore, if the first number is greater than the number threshold, considering the actual scenario, there may be many dark areas in the ultrasound image, making it difficult to accurately delineate the bone erosion region. Delineating the bone erosion region may lead to a discrepancy between the degree of bone erosion and the actual situation. Therefore, the above scheme improves the adaptability of the bone erosion degree to the target object by setting a number threshold.

[0050] For example, in step S121 above, determining the first bone erosion region in the first region of interest based at least on the image corresponding to the first region of interest in the ultrasound image includes: using a first number as a prompt word, inputting the image corresponding to the first region of interest in the ultrasound image into the trained semantic segmentation model to obtain the first number of first bone erosion regions in the first region of interest.

[0051] The semantic segmentation model mentioned above can be one of the following: fully convolutional network, U-shaped network, segmentation network, pyramid pooling network, etc.

[0052] The trained semantic segmentation model can be trained based on the image corresponding to the second region of interest in the ultrasound training image, the second number of images corresponding to the second region of interest, and the second bone erosion region within the second region of interest. For example, this embodiment of the invention provides a training method for a semantic segmentation model for reference. A second training dataset is obtained. The semantic segmentation model is iteratively trained using the second training dataset, and the model parameters are adjusted using a loss function until training is complete. The second training dataset may include: multiple images corresponding to the second region of interest in the ultrasound training image (equivalent to images corresponding to regions containing all second bone erosion regions in the ultrasound training image), with the label of each image corresponding to the second region of interest being the second bone erosion region within that second region of interest. The training prompt is the second number of images corresponding to the second region of interest. It can be understood that the location of each second bone erosion region can be represented by a binary mask (where 1 represents the second bone erosion region and 0 represents the region outside the second bone erosion region) or by coordinates (equivalent to the coordinate values ​​of each pixel in the second bone erosion region in the ultrasound training image).

[0053] The conditions for completing the above training can include the loss value calculated by the loss function stabilizing and the number of iterations reaching a preset number. Based on the above training process, a semantic segmentation model can be obtained to segment the image corresponding to the first region of interest in the ultrasound training image, thereby determining the first number of first bone erosion regions in the image corresponding to the first region of interest in the ultrasound training image. It can be understood that the image corresponding to the second region of interest in the ultrasound training image can be obtained from the output of the target detection model described above, or it can be manually labeled. The second bone erosion region in the second region of interest can be manually labeled.

[0054] According to the above-described scheme of the present invention, a first quantity can be used as a prompt word to input the image corresponding to the first region of interest in the ultrasound image into the trained semantic segmentation model, so as to obtain a first quantity of first bone erosion regions within the first region of interest. The use of a first quantity as a prompt word in the above scheme ensures that the number of determined first bone erosion regions is related to the first quantity, which can improve the accuracy of the first bone erosion regions and thus improve the accuracy of the obtained degree of bone erosion.

[0055] For example, determining the degree of bone erosion of the target object in step S122 above, at least based on the first bone erosion region in the first region of interest, includes: determining the degree of bone erosion of the target object based on the region size of the first bone erosion region in the first region of interest.

[0056] In one example, a minimum bounding rectangle can be fitted to the first bone erosion region, and the major axis of the resulting rectangle can be used as the region size of the first bone erosion region. For example, the maximum cross-sectional diameter of the bone erosion can be used as the region size.

[0057] In another example, the first bone erosion region can be fitted with a minimum bounding rectangle, and the area of ​​the resulting rectangular region can be used as the region size of the first bone erosion region.

[0058] This invention provides an example for reference. Taking the region size of the first bone erosion region as the major axis of the rectangular region obtained by fitting the minimum bounding rectangle to the first bone erosion region, and based on the region size of the largest first bone erosion region in the first region of interest, the degree of bone erosion of the target object is determined as follows: If there is no first bone erosion region in the first region of interest, the degree of bone erosion of the target object can be determined to be the "no bone erosion" level in step S120 above. Or, if there is no first region of interest, it can be the "no bone erosion" level. If the region size of the largest first bone erosion region in the first region of interest is greater than 0 mm and less than or equal to 2 mm, the degree of bone erosion of the target object can be determined to be the first level in step S120 above. If the region size of the largest first bone erosion region in the first region of interest is greater than 2 mm and less than or equal to 6 mm, the degree of bone erosion of the target object can be determined to be the second level in step S120 above. If the region size of the largest first bone erosion region in the first region of interest is greater than 6 mm, the degree of bone erosion of the target object can be determined to be the third level in step S120 above. It is understandable that, in addition to determining the degree of bone erosion of the target object based on the size of the largest bone erosion region in the first region of interest, the degree of bone erosion of the target object can also be determined based on the total size of the first bone erosion regions in the first region of interest.

[0059] According to the above-described solution of the present invention, the degree of bone erosion of the target object can be determined based on the area size of the first bone erosion area in the first region of interest, which helps to improve the accuracy of the obtained degree of bone erosion.

[0060] For example, the above step S122, which determines the degree of bone erosion of the target object based at least on the first bone erosion area in the first region of interest, includes: determining the degree of bone erosion of the target object based on the area size and the first number of the first bone erosion area in the first region of interest.

[0061] This invention provides an example for reference. Similarly, taking the region size of the first bone erosion region as the major axis of the rectangular region obtained by fitting the minimum bounding rectangle of the first bone erosion region, and determining the degree of bone erosion of the target object based on the region size of the largest first bone erosion region in the first region of interest, as an example: when the region size of the largest first bone erosion region in the first region of interest is 0, the degree of bone erosion of the target object can be determined to be the no-bone-erosion level in step S120 above. When the first quantity is 1, if the region size of the largest first bone erosion region in the first region of interest is greater than 0 mm and less than or equal to 2 mm, the degree of bone erosion of the target object can be determined to be the first level in step S120 above. When the first quantity is 1, if the region size of the largest first bone erosion region in the first region of interest is greater than 2 mm and less than or equal to 3 mm, the degree of bone erosion of the target object can be determined to be the second level in step S120 above. When the first quantity is 1, if the region size of the largest first bone erosion region in the first region of interest is greater than 3 mm, the degree of bone erosion of the target object can be determined to be the third level in step S120 above. When the first quantity is 2, if the area size of the largest bone erosion area in the first region of interest is greater than 0 and less than or equal to 2 mm, then the degree of bone erosion of the target object can be determined to be the second level in step S120 above. When the first quantity is 2, if the area size of the largest bone erosion area in the first region of interest is greater than 2 mm, then the degree of bone erosion of the target object can be determined to be the third level in step S120 above.

[0062] According to the above-described solution of the present invention, the degree of bone erosion of a target object can be determined based on the area size and the number of first bone erosion areas within the first region of interest. Compared with solutions that determine the degree of bone erosion of a target object solely based on area size, the above solution yields a more representative and accurate degree of bone erosion, and better reflects the actual erosion condition of the target object.

[0063] For example, the ultrasound image is a video frame in the ultrasound video of the target object.

[0064] A video frame can be a single still image from an ultrasound video. The number of video frames in an ultrasound video can be determined based on the duration of the ultrasound video and its corresponding frame rate. For example, if the duration of an ultrasound video is 3 seconds and the corresponding frame rate is 30 frames per second, then the number of video frames in the ultrasound video can be determined by multiplying the duration of the ultrasound video by its corresponding frame rate, resulting in (30*3) video frames, or 90 video frames.

[0065] This invention provides an example for reference. Taking video frames A, B, and C from an ultrasound video of a target object as an example, the first region of interest and its corresponding first quantity can be determined sequentially based on the generation time order of video frames A, B, and C. Specifically, for example, video frame A is generated earlier than video frame B, and video frame B is generated earlier than video frame C. First, video frame A can be input into the trained target detection model to obtain the first region of interest and its corresponding first quantity in video frame A. Then, based on the first region of interest and its corresponding first quantity in video frame A, step S120 is executed to determine the degree of bone erosion determined based on video frame A. Second, video frame B can be input into the trained target detection model to obtain the first region of interest and its corresponding first quantity in video frame B. Then, based on the first region of interest and its corresponding first quantity in video frame B, step S120 is executed to determine the degree of bone erosion determined based on video frame B. Finally, video frame C can be input into the trained target detection model to obtain the first region of interest (ROI) and the first number corresponding to the ROI in video frame C. Then, based on the first ROI and the first number corresponding to the ROI in video frame C, step S120 is executed to determine the degree of bone erosion determined based on video frame C. It is understood that, according to actual needs, multiple video frames can also be selected from the ultrasound video as the ultrasound images input into the trained target detection model in step S110 above.

[0066] According to the above-described solution of the present invention, the ultrasound image is a video frame in the ultrasound video of the target object. In practical scenarios, users can perform ultrasound scanning on the target object to obtain an ultrasound video, and then determine the degree of bone erosion from the video frames in the ultrasound video. The above solution is adaptable to the aforementioned practical scenarios.

[0067] For example, the ultrasound image whose degree of bone erosion meets the preset conditions is the target frame. The preset conditions include at least that the degree of bone erosion corresponding to the target frame is the highest among the multiple pre-selected video frames included in the ultrasound video.

[0068] The aforementioned pre-selected video frames can be manually selected by the user or automatically selected from ultrasound videos based on algorithms (e.g., interval selection or selection based on resolution). The target frame can also be one of the pre-selected video frames. In other words, the target frame can also be determined from the pre-selected video frames. Furthermore, only a portion of the video frames can be filtered to reduce computational pressure. It can be understood that the pre-selected video frames can also include all video frames in the ultrasound video. In the above example, the pre-selected video frames in the ultrasound video of the target object include video frames A, B, and C. It can be determined that the degree of bone erosion corresponding to video frame A is the no-bone-erosion level in step S120 (i.e., no bone erosion has occurred in the first area of ​​interest), the degree of bone erosion corresponding to video frame B is the first level in step S120 (i.e., mild bone erosion has occurred in the first area of ​​interest), and the degree of bone erosion corresponding to video frame C is the second level in step S120 (i.e., moderate bone erosion has occurred in the first area of ​​interest). Then, video frame C is taken as the target frame in the ultrasound image. Understandably, if multiple pre-selected video frames correspond to the same degree of bone erosion, the pre-selected video frame with the largest number of pre-selected video frames or the largest size of the first bone erosion area can be used as the target frame. In one example, if the pre-selected video frame corresponds to the highest degree of bone erosion (here, the highest degree of bone erosion can be considered the most severe degree, such as level three mentioned above), this pre-selected video frame can also be directly used as the target frame and displayed. Understandably, in this case, the pre-selected video frames after the target frame in the ultrasound video do not need to be processed to improve overall efficiency. The first pre-selected video frame with the highest degree of bone erosion can be directly displayed for user reference.

[0069] According to the above-described scheme of the present invention, the target frame in the ultrasound video can be determined based on the degree of bone erosion. The target frame determined by the above scheme is more representative, which is beneficial for users to refer to the target frame to understand the actual bone erosion status of the target object.

[0070] For example, the preset condition also includes that the size of the first bone erosion region of the target frame is the largest among multiple pre-selected video frames.

[0071] The target frame can be a pre-selected video frame in the ultrasound video that corresponds to the highest degree of bone erosion and has the largest area size of the first bone erosion region. Taking the pre-selected video frames in the ultrasound video of the target object, including video frame A, video frame B, and video frame C, as an example, and the area size of the first bone erosion region being the major axis of the rectangle obtained by fitting the minimum bounding rectangle of the first bone erosion region, video frame A is determined to have the bone erosion level of no bone erosion in step S120 (i.e., no bone erosion has occurred in the first area of ​​interest), and the size of the first bone erosion region corresponding to video frame A is 0 mm. Video frame B is determined to have the bone erosion level of the first level in step S120 (i.e., mild bone erosion has occurred in the first area of ​​interest), and the size of the first bone erosion region corresponding to video frame B is 1 mm. Video frame C is determined to have the bone erosion level of the second level in step S120 (i.e., moderate bone erosion has occurred in the first area of ​​interest), and the size of the first bone erosion region corresponding to video frame C is 4 mm. Since video frame C exhibits the highest degree of bone erosion and has the largest corresponding first erosion region, it can be identified as the target frame of the ultrasound video. It is understood that if multiple pre-selected video frames have the same degree of bone erosion and the same size of the first bone erosion region, then the pre-selected video frame with the largest number of corresponding first erosion regions can be selected as the target frame.

[0072] According to the above-described scheme of the present invention, the pre-selected video frame with the highest degree of bone erosion and the largest area size of the first bone erosion region can be used as the target frame. This improves the representativeness of the target frame and allows users to refer to it to understand the actual bone erosion status of the target object. Compared to a scheme that determines the target frame solely based on the degree of bone erosion, this scheme further improves the accuracy of the degree of bone erosion.

[0073] For example, the ultrasound image whose degree of bone erosion meets the preset conditions is the target frame. The above step S130, which displays at least the ultrasound image, includes: displaying the target frame and at least one of the following: the first bone erosion area marked in the target frame, the degree of bone erosion corresponding to the target frame, and the area size of the first bone erosion area in the target frame.

[0074] This invention provides an example for reference. Following the example in step S210 above, a video frame C (the target frame in this example) marked with a first bone erosion region, the size of the first bone erosion region in video frame C (i.e., 4 mm as mentioned above), and the degree of bone erosion corresponding to video frame C (i.e., the second level mentioned above) can be displayed on the display component (e.g., a display screen) of an ultrasound device. It is understood that developers can add or delete the above display content according to actual needs, and this embodiment of the invention does not impose any limitations.

[0075] According to the above-described scheme of the present invention, a target frame and at least one of the following can be displayed: a first bone erosion region marked in the target frame, the degree of bone erosion corresponding to the target frame, and the area size of the first bone erosion region in the target frame, thereby providing the user with a more comprehensive reference.

[0076] This invention also provides an apparatus for processing ultrasound images. Figure 3 A schematic block diagram of an ultrasound image processing apparatus 300 according to an embodiment of the present invention is shown. (In conjunction with...) Figure 3 As shown, the processing device 300 may include: a first determination module 310, a bone erosion degree determination module 320, and an image display module 330.

[0077] The first determining module 310 can be configured to input the ultrasound image of the target object into the trained target detection model to obtain a first region of interest in the ultrasound image and a first number corresponding to the first region of interest. The first number is used to represent the number of first bone erosion regions included in the first region of interest. The trained target detection model is trained based on the ultrasound training image, the second region of interest corresponding to the ultrasound training image, and the second number corresponding to the ultrasound training image. The second number is used to represent the number of second bone erosion regions included in the ultrasound training image.

[0078] The bone erosion degree determination module 320 can be configured to determine the bone erosion degree of a target object based on a first region of interest and a first quantity.

[0079] The image display module 330 can be configured to display at least an ultrasound image when the degree of bone erosion meets preset conditions.

[0080] For example, the bone erosion degree determination module 320 may include: a first bone erosion area determination module and a first degree determination module.

[0081] The first bone erosion region determination module can be configured to determine the first bone erosion region in the first interest region based at least on the image corresponding to the first interest region in the ultrasound image when the first number is less than the number threshold.

[0082] The first degree determination module can be configured to determine the degree of bone erosion of a target object based at least on a first bone erosion region within a first region of interest.

[0083] For example, the first bone erosion region determination module may include: a first bone erosion region determination submodule.

[0084] The first bone erosion region determination submodule can be configured to use a first quantity as a prompt word, input the image corresponding to the first region of interest in the ultrasound image into the trained semantic segmentation model, so as to obtain the first quantity of the first bone erosion region in the first region of interest.

[0085] For example, the first bone erosion region determination module may be configured to perform any of the following: determine the degree of bone erosion of the target object based on the region size of the first bone erosion region in the first region of interest; determine the degree of bone erosion of the target object based on the region size and a first number of the first bone erosion region in the first region of interest.

[0086] For example, the ultrasound image is a video frame in the ultrasound video of the target object.

[0087] For example, the ultrasound image whose degree of bone erosion meets the preset conditions is the target frame. The preset conditions include at least that the degree of bone erosion corresponding to the target frame is the highest among the multiple pre-selected video frames included in the ultrasound video.

[0088] For example, the preset condition also includes that the size of the first bone erosion region of the target frame is the largest among a plurality of pre-selected video frames.

[0089] For example, the ultrasound image with a bone erosion degree that meets the preset conditions is the target frame, and the image display module 330 includes a display submodule.

[0090] The display submodule is configured to display the target frame and at least one of the following: the first bone erosion region marked in the target frame, the degree of bone erosion corresponding to the target frame, and the region size of the first bone erosion region in the target frame.

[0091] According to another aspect of the present invention, an electronic device is also provided. Figure 4 A schematic block diagram of an electronic device 400 according to an embodiment of the present invention is shown. Figure 4 As shown, the electronic device 400 includes a processor 410 and a memory 420. The memory 420 stores a computer program, and the computer program instructions are executed by the processor 410 to perform the above-described ultrasound image processing method.

[0092] Furthermore, according to another aspect of the present invention, a storage medium is provided, on which program instructions are stored. When the program instructions are executed by a computer or processor, the computer or processor performs corresponding steps of the ultrasound image processing method described in the embodiments of the present invention, and is used to implement corresponding modules in the ultrasound image processing apparatus or the electronic device described in the embodiments of the present invention. The storage medium may, for example, include a memory card of a smartphone, a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media.

[0093] According to another aspect of the present invention, a computer program product is also provided, comprising computer program instructions, which, when executed by a computer or processor, cause the computer or processor to perform corresponding steps of the ultrasound image processing method described above.

[0094] Those skilled in the art can understand the specific implementation scheme of the above-mentioned electronic device and storage medium by reading the relevant description of the ultrasound image processing method. For the sake of brevity, it will not be described in detail here.

[0095] Although exemplary embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above exemplary embodiments are merely illustrative and are not intended to limit the scope of the invention. Various changes and modifications can be made therein by those skilled in the art without departing from the scope and spirit of the invention. All such changes and modifications are intended to be included within the scope of the invention as claimed in the appended claims.

[0096] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0097] In the several embodiments provided by this invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed.

[0098] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0099] Similarly, it should be understood that, in order to streamline the invention and aid in understanding one or more of the various aspects of the invention, features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof in the description of exemplary embodiments of the invention. However, this approach should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the corresponding claims, its inventive point lies in solving the corresponding technical problem with fewer features than all of those in a single disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the invention.

[0100] Those skilled in the art will understand that, apart from the mutual exclusion of features, all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or apparatus so disclosed can be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0101] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the claims, any of the claimed embodiments can be used in any combination.

[0102] The various component embodiments of the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some modules in the ultrasound image processing apparatus according to embodiments of the present invention. The present invention can also be implemented as an apparatus program (e.g., a computer program and computer program product) for performing some or all of the methods described herein. Such programs implementing the present invention can be stored on a computer-readable medium or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0103] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0104] The above description is merely a specific embodiment of the present invention or an explanation of that embodiment. The scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this invention should be included within the scope of protection of this invention. The scope of protection of this invention should be determined by the scope of the claims.

Claims

1. A method for processing ultrasound images, characterized in that, The method includes: An ultrasound image of the target object is input into a trained target detection model to obtain a first region of interest in the ultrasound image and a first number corresponding to the first region of interest. The first number represents the number of first bone erosion regions included in the first region of interest. The trained target detection model is trained based on an ultrasound training image, a second region of interest corresponding to the ultrasound training image, and a second number corresponding to the ultrasound training image. The second number represents the number of second bone erosion regions included in the ultrasound training image. Based on the first region of interest and the first quantity, the degree of bone erosion of the target object is determined; If the degree of bone erosion meets the preset conditions, at least the ultrasound image shall be displayed; Determining the degree of bone erosion of the target object based on the first region of interest and the first quantity includes: If the first quantity is less than the quantity threshold, the first bone erosion region in the first region of interest is determined based at least on the image corresponding to the first region of interest in the ultrasound image; The degree of bone erosion of the target object is determined based at least on the first bone erosion area in the first region of interest.

2. The method as described in claim 1, characterized in that, Determining the first bone erosion region within the first region of interest based at least on the image corresponding to the first region of interest in the ultrasound image includes: Using the first quantity as a prompt word, the image corresponding to the first region of interest in the ultrasound image is input into the trained semantic segmentation model to obtain the first number of first bone erosion regions in the first region of interest.

3. The method as described in claim 1, characterized in that, Determining the degree of bone erosion of the target object based at least on a first bone erosion area within the first region of interest includes any one of the following: The degree of bone erosion of the target object is determined based on the area size of the first bone erosion area in the first region of interest; The degree of bone erosion of the target object is determined based on the area size of the first bone erosion area in the first region of interest and the first quantity.

4. The method according to any one of claims 1 to 3, characterized in that, The ultrasound image is a video frame from the ultrasound video of the target object.

5. The method as described in claim 4, characterized in that, The ultrasound image whose degree of bone erosion meets the preset conditions is the target frame. The preset conditions include at least that the degree of bone erosion corresponding to the target frame is the highest among the multiple pre-selected video frames included in the ultrasound video.

6. The method as described in claim 5, characterized in that, The preset conditions also include that the size of the first bone erosion region of the target frame is the largest among the plurality of pre-selected video frames.

7. The method as described in claim 1, characterized in that, An ultrasound image that meets the preset conditions for bone erosion is the target frame, and displaying at least the ultrasound image includes: Display the target frame and at least one of the following: a first bone erosion region marked in the target frame, the degree of bone erosion corresponding to the target frame, and the region size of the first bone erosion region in the target frame.

8. An ultrasonic device, characterized in that, The device includes a memory and a processor, wherein: the memory is used to store a computer program; and the processor is used to execute the computer program to implement the ultrasound image processing method as described in any one of claims 1-7.

9. A storage medium storing computer program instructions, characterized in that, The computer program instructions, when executed, are used to perform the ultrasound image processing method as described in any one of claims 1-7.

10. A computer program product comprising computer program instructions, characterized in that, The computer program instructions, when executed by a processor, are used to perform the ultrasound image processing method as described in any one of claims 1-7.

Images