Efficiency evaluation method and device for object positioning model

Abstract

The invention discloses an effectiveness evaluation method and device for an object positioning model. The method comprises the following steps: acquiring an image comprising a plurality of objects to be positioned; using a trained object positioning model to find out positioning points of each object in the images, wherein the object positioning model is trained by using a plurality of images and positions of a plurality of objects in the images; calculating the interest range of the object capable of being covered according to the position of the positioning point; and cutting the image within the range of interest as an object image, and calculating an overlap ratio between the object image and the corresponding live image to evaluate the efficiency of the object positioning model.

Description

Technical Field

[0001] The present disclosure generally relates to a performance evaluation method and apparatus, and more specifically, to a performance evaluation method and apparatus for an object positioning model. Background Art

[0002] Bone mineral density (BMD) testing uses dual energy X-ray absorptiometry (DXA) to take X-rays of the spine or hip joint to obtain bone images, and then calculates the BMD value by analyzing the bone images.

[0003] When predicting the BMD value of each joint, it is necessary to first locate the position of each joint in the image to crop the image of the joint, and then input the joint image into a pre-established prediction module to predict the BMD value of the joint. Among them, since the positioning of the joints in the image will affect the accuracy of the subsequent BMD value prediction, how to distinguish whether the position of each joint predicted by the object positioning model is correct to evaluate its performance is one of the issues for technicians in this field. Summary of the invention

[0004] In view of this, the present disclosure proposes a method and device for evaluating the performance of an object positioning model, which can accurately evaluate the performance of the object positioning model.

[0005] The disclosed embodiment provides a method for evaluating the performance of an object positioning model, which is applicable to an electronic device including a processor. The method includes the following steps: acquiring an image including multiple objects to be positioned; finding the positioning points of each object in the image using a trained object positioning model, wherein the object positioning model is trained using multiple images and the positions of multiple objects therein; calculating a region of interest (ROI) that can cover the object based on the positions of the positioning points; and cropping the image within the ROI as an object image, and calculating the overlap ratio between the object image and a ground truth image of the corresponding object, so as to evaluate the performance of the object positioning model.

[0006] The disclosed embodiment provides a performance evaluation device for an object positioning model, which includes a data acquisition device, a storage device, and a processor. The data acquisition device is used to acquire an image including multiple objects to be positioned. The storage device is used to store a trained object positioning model, which is trained using multiple images and the positions of multiple objects therein. The processor is coupled to the data acquisition device and the storage device, and is configured to use the object positioning model to find the positioning points of each object in the image, calculate the range of interest that can cover the object based on the position of the positioning point, and crop the image within the range of interest as the object image, and calculate the overlap ratio between the object image and the corresponding object's real-life image, so as to evaluate the performance of the object positioning model.

[0007] Based on the above, the embodiment of the present disclosure defines the range of interest that can cover the object by the positioning points output by the object positioning model, and by adjusting the proportion of the range of interest to filter the left and right translation, and filtering the up and down translation according to the distance of the positioning point offset, the failure cases can be identified, thereby accurately evaluating the effectiveness of the object positioning model. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 is a schematic diagram of predicting BMD values ​​according to an embodiment of the present disclosure;

[0009] Figure 2 is a block diagram of a performance evaluation device for an object positioning model according to an embodiment of the present disclosure;

[0010] Figure 3 is a flow chart of a method for evaluating the performance of an object positioning model according to an embodiment of the present disclosure;

[0011] Figure 4 is a schematic diagram of calculating the intersection-over-union ratio according to an embodiment of the present disclosure;

[0012] Figure 5A and Figure 5B is a schematic diagram of the translation of the interest range according to an embodiment of the present disclosure;

[0013] Fig. 6A and Figure 6B It is a schematic diagram of the translation of the interest range according to an embodiment of the present disclosure.

[0014] Explanation of symbols

[0015] 102: Chest X-ray Image

[0016] 104: Positioning module

[0017] 106: Lumbar Spine Image

[0018] 106a: Areas of Interest

[0019] 108: Object Image

[0020] 110: Prediction module

[0021] 112: BMD value

[0022] 20: Performance evaluation device

[0023] 22: Image acquisition device

[0024] 24: Storage device

[0025] 26: Processor

[0026] 40, 50, 60: Image

[0027] 52, 54, 56, 62, 64, 66: Frame

[0028] A, B: Object images

[0029] a, a1, a1', a1", a2, a2', a2", b, b1, b1', b1", b2, b2', b2": center points of condyles

[0030] S302~S308: Steps DETAILED DESCRIPTION

[0031] Reference will now be made in detail to exemplary embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings. Whenever possible, the same reference numerals are used in the drawings and the description to refer to the same or like parts.

[0032] The disclosed embodiment uses a complete chest X-ray image and an image obtained from the image and including two lumbar vertebrae as input images to predict the bone mineral density (BMD) value. Figure 1 is a schematic diagram of predicting BMD values ​​according to an embodiment of the present disclosure. Figure 1 In this embodiment, a dual energy X-ray absorptiometry (DXA) is used to perform X-ray photography of the spine to obtain a chest X-ray image 102, and the chest X-ray image 102 is sent to a pre-established positioning module 104 to predict the center points a and b of two of the joints, and a region of interest (ROI) 106a in the lumbar image 106 that can completely cover the center points a and b of the two joints is calculated, and finally the image within the region of interest 106a is cut from the lumbar image 106 as an object image 108. The chest X-ray image 102 and the object image 108 are input as input images to a prediction module 110 to predict the BMD value 112 of the spine.

[0033] Since the interest range 106a is calculated by the positioning module 104 according to the positions of the two joint center points a and b, the performance evaluation of the positioning module 104 in the disclosed embodiment is based on how to make the interest range 106a of each image completely cover the two joints under the same calculation standard.

[0034] In detail, Figure 2 is a block diagram of a performance evaluation device for an object positioning model according to an embodiment of the present disclosure. Figure 2 The performance evaluation device 20 of the embodiment of the present disclosure is, for example, an electronic device with computing capabilities such as a computer, a workstation, or a server, and the embodiment does not limit its type. The performance evaluation device 20 includes components such as an image acquisition device 22, a storage device 24, and a processor 26. The functions of these components are described as follows:

[0035] The image acquisition device 22 is, for example, a wired or wireless transmission interface such as Universal Serial Bus (USB), RS232, Bluetooth (BT), or Wireless fidelity (Wi-Fi), and is used to receive images provided by image source devices such as cameras, video cameras, and dual-energy X-ray absorptiometry (DXA). In one embodiment, the image acquisition device 22 may also include a network card that supports Ethernet or wireless network standards such as 802.11g, 802.11n, and 802.11ac, so that the performance evaluation device 20 can connect to a database or server on the network and acquire images via the network.

[0036] The storage device 24 is, for example, any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, hard disk or other similar devices, integrated circuits and combinations thereof, which are used to store computer programs and other data that can be executed by the processor 26. In some embodiments, the storage device 24 can store an object positioning model for positioning objects in an image and a prediction model for predicting bone density (BMD) values. The object positioning model and the prediction model are, for example, a convolutional neural network (CNN), a recurrent neural network (RNN) or a long short term memory (LSTM) network, but the present embodiment is not limited thereto.

[0037] The processor 26 is, for example, a central processing unit (CPU), a graphic processing unit (GPU), or other programmable general-purpose or special-purpose microprocessors, digital signal processors (DSP), image signal processors (ISP), programmable controllers, application specific integrated circuits (ASIC), programmable logic devices (PLD), or other similar devices or combinations of these devices. In this embodiment, the processor 26 can load a computer program from the storage device 24 to execute the performance evaluation method of the embodiment of the present disclosure.

[0038] Figure 3 is a flow chart of a method for evaluating the performance of an object positioning model according to an embodiment of the present disclosure. Figure 2 as well as Figure 3 The method of this embodiment is applicable to Figure 2 The performance evaluation device 20 in the embodiment is described below with reference to the various components in the performance evaluation device 20 to explain the detailed process of the method of this embodiment.

[0039] In step S302, the processor 26 of the performance evaluation device 20 uses the image acquisition device 22 to acquire an image including multiple objects to be located. The image is, for example, an image obtained by performing X-ray photography of the spine or hip joint using a dual-energy X-ray absorptiometry (DXA), and includes multiple joints or joints to be located, but the embodiment is not limited thereto.

[0040] In step S304, the processor 26 finds the location point of each object in the image using the object location model stored in the storage device 24. The object location model is trained using multiple images and the locations of multiple objects in these images, so the processor 26 can obtain the location point of each object in the image by inputting the currently acquired image into the object location model.

[0041] In step S306, the processor 26 calculates the region of interest (ROI) of the covered object based on the positions of the positioning points. In some embodiments, the processor 26 calculates the distance between adjacent positioning points and the position of the center point, multiplies the distance by a predetermined ratio as an extension length, and extends the extension length with the center point as the center to define the region of interest. The predetermined ratio is, for example, determined according to the shape of the object. In some embodiments, the predetermined ratio includes a horizontal ratio with a value between 1.1 and 1.5 and a vertical ratio with a value between 1.5 and 1.6.

[0042] For example, assuming that the distance between adjacent positioning points pt1 and pt2 is d, and the coordinates of the center point are (xc, yc), the width of the interest range can be defined by taking the x-coordinate xc of the center point as the center plus or minus 1.5d, and the height of the interest range can be defined by taking the y-coordinate yc of the center point as the center plus or minus 1.6d, thereby generating an interest range covering the upper and lower joints.

[0043] In step S308, the processor 26 cuts the image within the range of interest as the object image, and calculates the overlap ratio between the object image and the corresponding ground truth image to evaluate the performance of the object positioning model. In some embodiments, the processor 26 calculates the intersection over union (IoU) of the object image and the ground truth image as the overlap ratio, and determines the quality of the positioning point predicted by the object positioning model according to the size of the IoU. For example, when the IoU is greater than or equal to the default value, the performance of the object positioning model can be determined to be excellent, and when the IoU is less than the default value, the performance of the object positioning model is determined to be poor.

[0044] In some embodiments, the processor 26 may further calculate the Euclidean distance between each positioning point and the corresponding point in the live image, and determine whether the object image including the positioning point is a failure case by comparing the Euclidean distance with the predetermined distance. If the Euclidean distance is greater than the predetermined distance, the object image including the positioning point may be determined to be a failure case. Taking the spinal image as an example, for the positioning point found by the object positioning model, the Euclidean distance between the positioning point of the adjacent joint and the corresponding point in the live image may be calculated. If the calculated Euclidean distance is greater than half of the distance between the two positioning points, it may be determined that the object image including the two positioning points deviates from more than half of the joint and covers other adjacent joints, and therefore may be considered a failure case.

[0045] For example, Figure 4 FIG. 1 is a schematic diagram of calculating the intersection-over-union ratio according to an embodiment of the present disclosure. Figure 4, this embodiment is used to calculate the intersection-over-union ratio IoU(A, B) between the object image A in the image 40 and the corresponding object live image B. Figure 4 As shown in the figure, the intersection-over-union ratio IoU(A,B) is the intersection of the object image A and the live image B (A∩B) divided by the union of the object image A and the live image B (A∪B). This intersection-over-union ratio can be used to evaluate the performance of object detection models. For object detection, when the intersection-over-union ratio IoU>0.5, it means that the detection result is good.

[0046] It should be noted that if the intersection-and-union ratio is calculated for the interest range of the object image and the live image, when the intersection-and-union ratio is greater than 0.8, the quality of the prediction results of the object positioning model for the positioning point can be clearly distinguished, but when the intersection-and-union ratio is between 0.6 and 0.8, the prediction results will be mixed with many results that are shifted left or right or up and down. That is, when the intersection-and-union ratio is the same, when the interest range is shifted left or right or up or down, it will have different effects on the prediction results.

[0047] For example, Figure 5A and Figure 5B FIG. 1 is a schematic diagram of a translation of a range of interest according to an embodiment of the present disclosure. Figure 5A In the image 50, the frame 52 indicates the interest range including the joint center points a1 and b1 in the live image, and the frame 54 indicates the interest range calculated based on the positions of the joint center points a1' and b1' predicted by the object positioning model. Figure 5B In image 50, frame 52 indicates the range of interest including the joint center points a1 and b1 in the live image, and frame 56 indicates the range of interest calculated based on the positions of the joint center points a1” and b1” predicted by the object positioning model. Frames 52, 54, and 56 adopt an aspect ratio of 1.5:1.6 that is close to the original image, and the intersection-over-union ratio of the images of frame 52 and frame 54 and the intersection-over-union ratio of the images of frame 52 and frame 56 are both 0.61. Even if a left-right translation occurs as shown in frame 54, the range of interest still includes two correct joints, but when an up-down translation occurs as shown in frame 56, the range of interest will include the wrong joints.

[0048] In this regard, in some embodiments, by adjusting the ratio of the length to the width of the interest range, it is easier to distinguish between available and unavailable interest ranges.

[0049] For example, Fig. 6A and Figure 6B FIG. 1 is a schematic diagram of a translation of a range of interest according to an embodiment of the present disclosure. Fig. 6AIn the image 60, the frame 62 indicates the interest range including the joint center points a2 and b2 in the live image, and the frame 64 indicates the interest range calculated based on the positions of the joint center points a2' and b2' predicted by the object positioning model. Figure 6B In the image 60, the frame 62 indicates the interest range including the joint center points a2 and b2 in the live image, and the frame 66 indicates the interest range calculated based on the positions of the joint center points a2" and b2" predicted by the object positioning model. Among them, since the aspect ratio of the frames 62, 64, and 66 has been adjusted to 1.1:1.5, the intersection-over-union ratio of the images of the frames 62 and 64 has dropped to 0.5, and the intersection-over-union ratio of the images of the frames 62 and 66 has dropped to 0.59, so it is easier to distinguish that the interest range that has been translated left and right as shown in the frame 64 is unavailable.

[0050] For the range of interest where the up and down translation occurs as shown in the box, the Euclidean distance between the joint center points a2" and a2 and the Euclidean distance between the joint center points b2" and b2 can be calculated respectively. If the calculated Euclidean distances are both greater than half of the distance between the joint center points a2" and b2", the range of interest can be determined as a failure case. In this way, cases of up and down translation can be filtered out.

[0051] In summary, the performance evaluation method and device of the object positioning model of the embodiment of the present disclosure increases its sensitivity to left and right translation by adjusting the proportion of the range of interest. In actual application, the intersection and union ratio is used to screen out the correct samples without left and right translation. Then, the calculation of the Euclidean distance can be used to filter out samples with large up and down translation. As a result, failure cases can be accurately identified, thereby evaluating the performance of the object positioning model.

[0052] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present disclosure.

Claims

1. A method for evaluating the performance of an object positioning model, applicable to an electronic device including a processor, the method comprising the following steps: acquiring an image including a plurality of objects to be located; Finding the location point of each of the objects in the image using a trained object location model, wherein the object location model is trained using a plurality of images and the locations of a plurality of objects in the images; Calculate the interest range that can cover the object according to the position of the positioning point; as well as The image within the range of interest is cropped as an object image, and an overlap ratio between the object image and a live image of a corresponding object is calculated to evaluate the effectiveness of the object positioning model.

2. The method according to claim 1, wherein the step of calculating the interest range that can cover the object according to the position of the positioning point comprises: Calculating the distance between adjacent positioning points and the position of the center point; as well as The distance is multiplied by a predetermined ratio to obtain an extension length, and the extension length is extended from the center point to define the range of interest, wherein the predetermined ratio is determined according to the shape of the object. 3 . The method according to claim 2 , wherein the predetermined ratio comprises a horizontal ratio having a value between 1.1 and 1.5 and a vertical ratio having a value between 1.5 and 1.

6.

4. The method according to claim 1, wherein the step of calculating the overlap ratio between the object image and the live image of the corresponding object comprises: The intersection-over-union ratio between the object image and the live image is calculated as the overlap ratio.

5. The method according to claim 1, further comprising: Calculating the Euclidean distance between each positioning point and a corresponding point in the live image, and determining whether the Euclidean distance is greater than a predetermined distance; as well as If the Euclidean distance is greater than the predetermined distance, the object image including the positioning point is determined to be a failure case.

6. A performance evaluation device for an object positioning model, comprising: A data acquisition device, which acquires an image including a plurality of objects to be located; A storage device storing a trained object positioning model, wherein the object positioning model is trained using a plurality of images and positions of a plurality of objects in the images; as well as A processor, coupled to the data acquisition device and the storage device, is configured to: Using the object positioning model to find the positioning point of each object in the image; Calculate the interest range that can cover the object according to the position of the positioning point; as well as The image within the range of interest is cropped as an object image, and an overlap ratio between the object image and a live image of a corresponding object is calculated to evaluate the effectiveness of the object positioning model.

7. The performance evaluation device of the object positioning model according to claim 6, wherein the processor includes calculating the distance between adjacent positioning points and the position of the center point, multiplying the distance by a predetermined ratio as an extension length, and extending the extension length with the center point as the center to define the range of interest, wherein the predetermined ratio is determined according to the shape of the object. 8 . The performance evaluation device of the object positioning model according to claim 7 , wherein the predetermined ratio comprises a horizontal ratio with a value between 1.1 and 1.5 and a vertical ratio with a value between 1.5 and 1.

6. 9 . The performance evaluation device of the object positioning model according to claim 6 , wherein the processor calculates an intersection-over-union ratio of the object image and the live image as the overlap ratio.

10. According to the performance evaluation device of the object positioning model of claim 6, the processor further calculates the Euclidean distance between each of the positioning points and the corresponding point in the live image, and determines whether the Euclidean distance is greater than a predetermined distance, and if the Euclidean distance is greater than the predetermined distance, determines that the object image including the positioning point is a failure case.

Images