Fracture examination equipment and method based on images

Abstract

The invention belongs to the technical field of medical equipment. The image-based fracture examination equipment provided by the embodiment of the invention comprises the processor and the display, aiming at the same target part, on one hand, the X-ray image of the target part and the identified fracture position are displayed, and on the other hand, the appearance image of the target part and the identified trauma position are displayed; therefore, a doctor or a technician can conveniently judge the fracture position identified in the X-ray image according to the trauma position in the appearance image so as to diagnose whether missed diagnosis or misdiagnosis exists in the fracture position identified in the X-ray image or not, and the accuracy of a diagnosis result of the fracture position in the X-ray image is improved to a certain extent. In addition, the embodiment of the invention further provides an image-based fracture examination device and method.

Description

Technical Field

[0001] This application relates to the field of medical device technology, and more specifically, to image-based fracture examination and methods. Background Technology

[0002] Radiographic imaging techniques, such as digital X-ray imaging and computed tomography (CT), are widely used in clinical imaging examinations due to their advantages such as fast imaging speed, clear and detailed images, and the ability to examine the whole body, becoming one of the main auxiliary tools for doctors to diagnose diseases. Although X-ray images offer high resolution and clarity in diagnosis, there is still a possibility of missed or misdiagnosed fractures. Summary of the Invention

[0003] In view of this, this application provides an image-based fracture examination device and method, which provides users with richer diagnostic auxiliary information to help users diagnose the fracture location.

[0004] In a first aspect, embodiments of this application provide an image-based fracture examination device, including a processor and a display;

[0005] The processor is configured to: acquire X-ray images of the target site obtained by a radiographic imaging device, identify at least one fracture location in the X-ray images of the target site and generate a marker for each fracture location; and acquire an external image of the target site obtained by an image acquisition device, identify at least one injury location in the external image of the target site and generate a marker for each injury location; wherein: the field of view of the target site captured by the radiographic imaging device and the field of view of the target site captured by the image acquisition device at least partially overlap;

[0006] The display is used to display X-ray images of the target site and an appearance image of the target site, and to display a mark for each fracture location in the X-ray images and a mark for each injury location in the appearance image.

[0007] Secondly, embodiments of this application provide an image-based fracture examination device, including: an input device, a processor, and a display;

[0008] The display is used to: display an appearance image of the target area obtained by the image acquisition device, and display an X-ray image of the target area obtained by the radiation imaging device; wherein the field of view of the target area captured by the radiation imaging device and the field of view of the target area captured by the image acquisition device at least partially overlap;

[0009] The input device is used to acquire the user's operation of marking the fracture location in the X-ray image, and to acquire the user's input of the marking completion operation;

[0010] The processor is configured to: identify at least one injury location in the appearance image of the target site; whenever a user marks a fracture location in the X-ray image, control the display to display the marked fracture location in the X-ray image, and, in conjunction with the marked fracture location in the X-ray image and the at least one injury location identified in the appearance image, determine whether the marked fracture location in the X-ray image is misdiagnosed; if the marked fracture location is misdiagnosed, control the display to display a prompt message indicating that the marked fracture location is misdiagnosed; and, in response to a user-input marking completion operation, in conjunction with each marked fracture location in the X-ray image and the at least one injury location identified in the appearance image, determine whether there is a missed fracture location in the X-ray image; if there is a missed fracture location in the X-ray image, control the display to display a prompt message indicating that there is a missed fracture location in the X-ray image.

[0011] Thirdly, embodiments of this application provide an image-based fracture examination device, including a processor and a display;

[0012] The processor is configured to: acquire an X-ray image of the target body obtained by a radiographic imaging device; acquire an appearance image of the target body obtained by an image acquisition device of the same target body; wherein the field of view of the target body captured by the radiographic imaging device and the field of view of the target body captured by the image acquisition device at least partially overlap, and the target body has a fracture location or is suspected to have a fracture location;

[0013] The display is used to simultaneously display the X-ray image of the target area and the appearance image of the target area.

[0014] Fourthly, embodiments of this application provide an image-based fracture examination method, including:

[0015] Acquire X-ray images of the target area obtained by a radiation imaging device, identify at least one fracture location in the X-ray images of the target area, and generate a marker for each fracture location;

[0016] An image of the target body obtained by an image acquisition device is captured on the same target body. At least one injury location is identified in the image of the target body and a mark is generated for each injury location. The field of view of the target body captured by the radiographic imaging device at least partially overlaps with the field of view of the target body captured by the image acquisition device.

[0017] Display an X-ray image of the target site, and display a mark of each fracture location in the X-ray image; and display an external image of the target site, and display a mark of each injury location in the external image.

[0018] Fifthly, embodiments of this application provide an image-based fracture examination method, including:

[0019] Acquire an external image of the target body obtained by an image acquisition device, and identify at least one location of external injury in the external image of the target body;

[0020] The display shows X-ray images of the target area obtained by the radiographic imaging device from the same target area; wherein the field of view of the target area captured by the radiographic imaging device at least partially overlaps with the field of view of the target area captured by the image acquisition device;

[0021] Whenever a user marks a fracture location in the X-ray image, the marked fracture location is displayed in the X-ray image. The marked fracture location in the X-ray image is combined with at least one trauma location identified in the appearance image to determine whether the marked fracture location in the X-ray image is misdiagnosed. If the marked fracture location is misdiagnosed, a prompt message indicating that the marked fracture location is misdiagnosed is output.

[0022] The system responds to user input to complete the marking operation, and combines the marked fracture locations in the X-ray image with at least one trauma location identified in the appearance image to determine whether there are any missed fracture locations in the X-ray image; if there are any missed fracture locations in the X-ray image, it outputs a prompt message indicating that there are missed fracture locations in the X-ray image.

[0023] Sixthly, embodiments of this application provide an image-based fracture examination method, including:

[0024] The method involves acquiring an X-ray image of the target region obtained by a radiographic imaging device; acquiring an external image of the target region obtained by an image acquisition device of the same target region; wherein the field of view of the target region captured by the radiographic imaging device and the field of view of the target region captured by the image acquisition device at least partially overlap.

[0025] The X-ray image of the target area and the appearance image of the target area are displayed on the same screen.

[0026] The image-based fracture examination device and method provided in this application, for the same target area, displays both the X-ray image of the target area and the identified fracture location, and the external image of the target area and the identified injury location. This allows doctors or technicians to refer to the injury location in the external image to judge the fracture location identified in the X-ray image, so as to diagnose whether there is a missed diagnosis or misdiagnosis of the fracture location identified from the X-ray image, thereby improving the accuracy of the diagnosis results of the fracture location in the X-ray image to a certain extent. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 A schematic diagram of an image-based fracture examination device provided in an embodiment of this application;

[0029] Figure 2 A schematic flowchart of an image-based fracture examination method provided in an embodiment of this application;

[0030] Figure 3 An example diagram illustrating the identification of fracture location based on a diagnostic report, provided as an embodiment of this application;

[0031] Figure 4 Example diagrams of the location of external injuries in three types of marked appearance images provided in embodiments of this application;

[0032] Figure 5 An example interface diagram provided for embodiments of this application, including an appearance image and markings, as well as an X-ray image and markings;

[0033] Figure 6 An example diagram illustrating the location of a misdiagnosis in an X-ray image provided in an embodiment of this application;

[0034] Figure 7 Four example diagrams illustrating misdiagnosed locations in X-ray images provided for embodiments of this application;

[0035] Figure 8 An example interface diagram provided for embodiments of this application, including an appearance image and markings, as well as an X-ray image and markings;

[0036] Figure 9 An example diagram illustrating the location of a missed diagnosis in an X-ray image provided in an embodiment of this application;

[0037] Figure 10 An example diagram of an interface including an appearance image and an X-ray image provided for an embodiment of this application. Detailed Implementation

[0038] Radiographic imaging equipment uses X-rays to observe the internal condition of the body, and the images it captures are called X-ray images. In clinical practice, computer equipment can automatically identify fracture locations from X-ray images. However, in some cases, due to soft tissue obstruction, minor fractures, or inconspicuous fractures, misdiagnosis or missed diagnosis may occur. The inventors have discovered that using natural images of fracture locations as supplementary diagnostic information can provide richer information for determining fracture location, thereby improving the accuracy of fracture diagnosis.

[0039] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0040] See Figure 1 This application provides a computer device, which may be referred to as an image-based fracture examination device, including a memory, a processor, and a display connected via a system bus. Those skilled in the art will understand that... Figure 1 The computer structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0041] The computer device's memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. When the computer programs are loaded and executed by the processor, they implement the image-based fracture examination method. The internal memory provides the environment for the operation of the operating system and computer programs in the non-volatile storage medium. The computer device's processor provides computational and control capabilities, loading and executing the instructions of the computer programs in memory to implement the image-based fracture examination method. Specifically, the image-based fracture examination method includes, for example... Figure 2 As shown, it includes steps S201-S203.

[0042] S201: Acquire X-ray images of the target area obtained by a radiographic imaging device, identify at least one fracture location in the X-ray images of the target area, and generate a marker for each fracture location.

[0043] Specifically, radiographic imaging equipment includes, but is not limited to, DR (Digital Radiography) equipment, X-ray equipment, and PET (Positron Emission Tomography) equipment.

[0044] If a body part of the subject has a fracture or is suspected of having a fracture, that part can be called the target area, such as the wrist or leg. The radiographic imaging equipment then captures an X-ray image of the target area.

[0045] There are several ways to identify and mark fracture locations in X-ray images; three specific methods are listed here.

[0046] Method 1: Manual Identification. After reviewing the X-ray image of the target area displayed on the monitor, the doctor or technician can mark the fracture location using input devices such as a mouse or stylus. The processor responds to the user's marking operation on the X-ray image of the target area, and uses the marked fracture location as the identified fracture location in the X-ray image of the target area. This method is relatively simple to implement with computer equipment, relies on the experience of the doctor or other user, and requires manual operation. In some embodiments, in the case of manual marking, the computer device can display the marked fracture location each time it receives the user's marking operation, and can further judge the marked fracture location in conjunction with the identified injury location in the external image to determine whether it is a misdiagnosed fracture location. Furthermore, after the user completes the manual marking operation, it can make an overall judgment on whether the user-marked fracture locations have been missed. For detailed explanation, please refer to the relevant content below, which will not be repeated here.

[0047] Method 2: Automatic identification using image processing technology. Specifically, the X-ray image of the target area is input into a pre-trained machine learning model, which then identifies and outputs at least one fracture location in the X-ray image of the target area.

[0048] The machine learning model is trained using X-ray image samples containing the fracture location of the target site. During training, the machine learning model learns to distinguish features or patterns between the target region (fracture location) and non-target region (non-fracture location) in the X-ray image samples. Therefore, the pre-trained machine learning model has the ability to automatically distinguish between fracture and non-fracture locations, and can automatically identify the fracture location in the input X-ray image. In some embodiments, the machine learning model is a neural network model such as R-CNN, CenterNet, or FCN. Specifically, the fracture location can be obtained based on machine learning object detection, keypoint detection, or segmentation methods. Object detection methods can obtain a bounding box containing the fracture location, such as a rectangle or circle; keypoint detection methods can obtain key points in the image representing the fracture location; and segmentation methods can obtain the target contour of the fracture location.

[0049] Method 3: Automatic identification using language processing technology. Specifically, obtain the diagnostic report related to the target area, perform natural language processing on the report to identify at least one fracture location recorded in the report, and mark at least one fracture location in the X-ray image of the target area. The diagnostic report must contain at least some text describing the fracture location. Using natural language processing methods including text classification, entity recognition, and information extraction, semantic analysis is performed on the imaging diagnostic results to identify the fracture location, which can then be identified and marked in the X-ray image. See [link to relevant documentation]. Figure 3 The left side shows an example of a diagnostic report, which includes the imaging diagnosis (fracture of the distal radius of the left arm and possible greenstick fracture of the distal ulna of the left arm). Natural language processing can identify the fracture locations described in the text, including the distal radius and distal ulna of the left arm, and mark the fracture locations of the distal ulna and distal radius of the left arm with target boxes (rectangles) in the X-ray image on the right.

[0050] Of course, other image processing methods can also be used to mark the fracture location in X-ray images, which will not be elaborated here.

[0051] S202: Acquire an image of the appearance of the target part taken by an image acquisition device, identify at least one injury location in the image of the target part, and generate a mark for each injury location.

[0052] Specifically, image acquisition devices include, but are not limited to, cameras and camcorders. After capturing images of the same target area, the image acquisition device obtains an image of the target area's appearance, such as an RGB format image or video. It should be noted that the field of view of the target area captured by the radiographic imaging device and the field of view of the target area captured by the image acquisition device must at least partially overlap. For example, if the camera is installed anywhere on the head of the DR equipment that does not obstruct X-rays, this ensures that the imaging angles of the appearance image and the X-ray image partially overlap, guaranteeing that the image result is of the same location on the same part.

[0053] Typically, when a fracture occurs at the target site, there is often an external injury on the skin. There are various ways to identify and mark the location of the injury in the external image; three specific implementation methods are listed here.

[0054] Method 1: Manual Recognition. The location of the injury has the following characteristics: limb deformity, tissue swelling, or a skin wound. After viewing the image of the target area on the monitor, the doctor or technician can mark the injury location using input devices such as a mouse or stylus. The processor responds to the user's annotation of the injury location in the image of the target area, and uses the marked injury location as the identified injury location in the image of the target area. This method is relatively simple to implement with computer equipment, relies on the experience of the doctor or other user, and requires manual operation.

[0055] Method 2: Automatic identification using image processing technology. Specifically, the appearance image of the target area is input into a pre-trained machine learning model, which then identifies and outputs at least one injury location within the target area's appearance image. The machine learning model is trained using appearance image samples containing injury locations of the target area. During training, the model learns to distinguish features or patterns between target regions (injury locations) and non-target regions (non-injury locations) in the appearance image samples. Therefore, the pre-trained machine learning model has the ability to automatically distinguish between injury locations and non-injury locations, and can automatically identify the injury location within the input appearance image. Further explanation can be found in the specific implementation method for automatic fracture location identification described above, and will not be repeated here.

[0056] Of course, other image processing methods can also be used to mark the location of external injuries in appearance images, which will not be elaborated here.

[0057] It should be noted that the order of steps S201 and S202 is not limited to this. Alternatively, in step S201, X-ray images and appearance images of the target area can be obtained, and in step S202, the fracture location can be identified and marked on the X-ray images, and the location of the injury can be identified and marked on the appearance images.

[0058] S203: Display X-ray images of the target area and mark each fracture location in the X-ray images; display an external image of the target area and mark each injury location in the external image.

[0059] See Figure 4 One example provided shows an external image where the identified injury location can be marked using any of the following methods: points, bounding boxes, or outlines. Similarly, X-ray images can also be marked using any of these methods.

[0060] Specifically, the processor can control the display to simultaneously show X-ray images and external images, and display markers for each fracture location in the X-ray images and markers for each injury location in the external images. See Figure 5 One example provided is an external image of a wrist and hand, where a rectangle marks a location of injury, and an X-ray image of the same wrist and hand, where a rectangle marks a location of fracture.

[0061] Typically, when a fracture occurs at a target site, there is often an external injury on the skin. This application embodiment, for the same target site, not only displays the X-ray image of the target site and the fracture location identified therefrom, but also displays the external image of the target site and the external injury location identified therefrom. This allows doctors or technicians to accurately diagnose the fracture location by combining the markings from both aspects. Furthermore, this application embodiment also allows doctors or technicians to conveniently judge the fracture location markings in the X-ray image by referring to the external image of the target site for the external injury location, in order to diagnose whether there is a missed or misdiagnosis of the fracture location identified from the X-ray image. A missed diagnosis indicates that the computer equipment did not identify the fracture location corresponding to the external injury location in the X-ray image (the external injury location is considered to actually exist or suspected to exist as a fracture location), while a misdiagnosis indicates that the computer equipment identified a fracture location corresponding to a non-existent external injury location in the X-ray image (the non-existent external injury location is considered not to actually exist or suspected to exist as a fracture location), thereby improving the accuracy of the computer equipment's diagnostic results for fracture locations in X-ray images to a certain extent.

[0062] In one embodiment of this application, the computer device can also automatically indicate whether there are any missed or misdiagnosed fracture locations in the X-ray images, prompting doctors or technicians to pay close attention to these missed or misdiagnosed fracture locations in the X-ray images, thereby providing doctors or technicians with richer diagnostic information. Specifically, the processor is also configured to perform the following steps:

[0063] By combining at least one fracture location identified in the X-ray image and at least one injury location identified in the external image, it is determined whether the at least one fracture location identified in the X-ray image is missed or misdiagnosed; if the at least one fracture location identified in the X-ray image is missed, the control display is prompted to indicate the missed fracture location; if the at least one fracture location identified in the X-ray image is misdiagnosed, the control display is prompted to indicate the misdiagnosed fracture location.

[0064] In this case, the method of combining the fracture location in X-ray images and the location of the injury in external images to determine whether the fracture location in X-ray images has been missed or misdiagnosed indicates that both X-ray images and external images are involved in the judgment, rather than using only one type of data for judgment.

[0065] The joint judgment mainly achieves a one-to-one correspondence between fracture locations and injury locations through mutual mapping of image positions. Taking X-ray images as an example, if a fracture location in the X-ray image does not have a corresponding injury location in the external image, then this fracture location is considered a misdiagnosis. If the X-ray image lacks a fracture location corresponding to an injury location in the external image, then this missing fracture location is considered a missed diagnosis. In other words: for any fracture location identified in the X-ray image, mapping that fracture location to the external image; if the corresponding location of any fracture location in the external image does not match any injury location identified in the external image, then any fracture location identified in the X-ray image is considered a misdiagnosis. Similarly, for any injury location identified in the external image, mapping that injury location to the X-ray image; if the corresponding location of any injury location in the X-ray image does not match any fracture location identified in the X-ray image, then at least one fracture location identified in the X-ray image is considered a missed diagnosis.

[0066] The following illustrations explain how to identify the location of a misdiagnosed fracture.

[0067] See Figure 6 In Figures (a) and (b), the fracture location is identified as the epiphysis of the base of the 5th metatarsal bone in the X-ray image shown in Figure (b), but the foot appearance image shown in Figure (a) does not identify the location of the injury. It is determined that there is a misdiagnosis in the X-ray image. Further, prompt information can be displayed for the misdiagnosed fracture location to remind doctors or technicians to pay attention to the misdiagnosed location in the X-ray image.

[0068] In one specific implementation, controlling the display to indicate a misdiagnosed fracture location may include: controlling the display to indicate the misdiagnosed fracture location in at least one fracture location in the X-ray image with a first type of marking, and controlling the display to indicate the non-misdiagnosed fracture location in at least one fracture location in the X-ray image with a second type of marking.

[0069] Specifically, one type of marking (referred to as Type I marking for easy distinction) is used in X-ray images to indicate misdiagnosed fracture locations, while another type of marking (referred to as Type II marking for easy distinction) is used to indicate accurately identified, non-misdiagnosed fracture locations in the X-ray images. Furthermore, to ensure consistency with the markings in external images and to facilitate doctors' or technicians' understanding of non-misdiagnosed fracture locations in X-ray images, the markings for non-misdiagnosed fracture locations in X-ray images are of the same type as those for trauma locations in external images; that is, the markings for trauma locations in external images are also Type II markings.

[0070] The first type of marker differs from the second type of marker. For example, the first type of marker can be a rectangle formed by solid lines of a certain color. To distinguish it from the first type of marker, the second type of marker can use non-rectangular markers, such as key points or outline lines. Alternatively, the second type of marker can use a rectangle but with a different color and / or line style than the first type of marker. Or, the second type of marker can add missing text or symbol hints to the first type of marker, etc.

[0071] See Figure 7 Here are four examples of display methods. The first method uses dashed boxes to indicate possible misdiagnosed fracture locations in X-ray images. An external image of the hand identifies an injury location and marks it with a solid red rectangle. However, the X-ray image of the hand identifies two fracture locations. The wrist fracture corresponds to the injury location in the external image; this non-misdiagnosed fracture is also marked with a solid red rectangle. The thumb fracture, however, does not correspond to the injury location in the external image, therefore it is a misdiagnosed fracture and is indicated with a dashed red rectangle. The second method uses text to indicate possible misdiagnosed fracture locations in X-ray images; for example, the thumb fracture is indicated with the text "Not within the injury area." The third method uses simple symbols such as infection symbols to indicate possible misdiagnosed fracture locations in X-ray images; for example, the thumb fracture is indicated with an exclamation mark. The fourth method uses rectangles of different colors to indicate possible misdiagnosed fracture locations in X-ray images; for example, rectangles of different colors, such as yellow, are used.

[0072] The following illustrations explain how to identify the location of a missed fracture.

[0073] See Figure 8In Figures (a) and (b), the location marked by the outline in the forearm appearance image shown in Figure (a) indicates an injury. This injury location is matched externally to the marked fracture location in the X-ray image shown in Figure (b). Furthermore, the X-ray image does not show any additional marked fracture locations compared to the appearance image. Therefore, it is determined that the X-ray image does not indicate a missed or misdiagnosed fracture. However, if the X-ray image does not show the marked fracture location, it indicates a missed diagnosis. Further, a prompt message can be displayed for this missed fracture location to alert the doctor or technician to pay attention to it in the X-ray image.

[0074] In one specific implementation, controlling the display to indicate the location of a missed fracture may include: controlling the display to indicate at least one fracture location in the X-ray image with a first type of marking, and controlling the display to indicate the location of the missed fracture location in the X-ray image with a second type of marking.

[0075] Specifically, one type of marker (referred to as type I marker for easy distinction) is used in X-ray images to indicate the location of the identified fracture. For unidentified or missed fracture locations, another type of marker (referred to as type II marker for easy distinction) is used in the X-ray images to indicate the location. The types of markers are explained in the relevant descriptions above and will not be repeated here.

[0076] See Figure 9 Example of a prompting method provided. An external image of the forearm identifies an injury location and marks it with a solid red outline. However, the corresponding location in the X-ray image of the forearm does not show a fracture. This missed fracture location is marked with a dashed red outline to prompt the doctor or technician to focus on the missed fracture location in the X-ray image.

[0077] To simplify the work of doctors or technicians, in one embodiment, the computer device can output a judgment result on whether an X-ray image has been missed or misdiagnosed. Specifically:

[0078] The processor is also configured to: combine at least one fracture location identified in the X-ray image with at least one injury location identified in the appearance image to determine whether there is a missed diagnosis or misdiagnosis of at least one fracture location identified in the X-ray image, and obtain the determination result of the fracture location in the X-ray image; the display is also used to output the determination result of the fracture location in the X-ray image.

[0079] For specific instructions on judging missed diagnoses and misdiagnoses, please refer to the relevant content above, which will not be repeated here. If a missed diagnose or misdiagnosis is determined in the X-ray image, the judgment result can be output in the X-ray image report. The judgment result can be regarded as a conclusion on the identification of the fracture location in the X-ray image. Directly outputting the judgment result can make it convenient for doctors or technicians to quickly obtain the judgment conclusion.

[0080] In practical applications, after X-ray imaging equipment captures images, doctors or technicians can manually mark the fracture locations on the X-ray images. Computer equipment can then analyze the marked fracture locations in conjunction with the visual images and provide alerts for missed or misdiagnosed fractures. Specifically, this application also provides a computer device (which may be called an image-based fracture examination device), including: an input device, a processor, and a display.

[0081] The display is used to: display an external image of the target area obtained by an image acquisition device, and display an X-ray image of the same target area obtained by a radiographic imaging device; wherein the field of view of the target area captured by the radiographic imaging device and the field of view of the target area captured by the image acquisition device at least partially overlap. The display shows both the external image and the X-ray image of the same target area.

[0082] The input device is used to acquire the user's actions of marking fracture locations on X-ray images and to acquire the user's input of a completion mark. For example, marking fracture locations on X-ray images using tools such as a mouse, stylus, or touchpad, and inputting the completion mark input.

[0083] The processor is configured to: identify at least one injury location in the appearance image of the target area; whenever a user marks a fracture location in an X-ray image, control the display to display the marked fracture location in the X-ray image, and combine the marked fracture location in the X-ray image with the at least one injury location identified in the appearance image to determine whether the marked fracture location in the X-ray image is misdiagnosed; if the marked fracture location is misdiagnosed, control the display to display a prompt indicating that the marked fracture location is misdiagnosed; and, in response to a user-input marking completion operation, combine the marked fracture locations in the X-ray image with the at least one injury location identified in the appearance image to determine whether there is a missed fracture location in the X-ray image; if there is a missed fracture location in the X-ray image, control the display to display a prompt indicating that there is a missed fracture location in the X-ray image.

[0084] The methods for identifying the location of external injuries, the methods for joint judgment, and the methods for alerting to missed or misdiagnosed cases can all refer to the relevant content mentioned above, and will not be repeated here. In this embodiment, each time the user marks a fracture location on the X-ray image, the computer device automatically judges the marked fracture location to determine whether it is a misdiagnosis. If it is a misdiagnosis, a prompt message is output to promptly remind the user of the misdiagnosis. After the user completes the marking, the computer judges each marked fracture location in conjunction with the external injury location identified in the external image, and prompts the user of any missed fracture locations to alert the user to any missed diagnoses. This embodiment can assist doctors or technicians in diagnosing fracture locations and, to a certain extent, avoid missed or misdiagnosed cases.

[0085] Another embodiment of this application also provides a computer device, which may be referred to as an image-based fracture examination device, including a processor and a display.

[0086] The processor is configured to: acquire X-ray images of the target area obtained by the radiographic imaging device; acquire appearance images of the target area obtained by the image acquisition device; wherein the field of view of the target area captured by the radiographic imaging device and the field of view of the target area captured by the image acquisition device at least partially overlap, and the target area has a fracture location or is suspected to have a fracture location.

[0087] The monitor is used to display both the X-ray image of the target area and an external image of the target area on the same screen. For example... Figure 10 The provided display example shows an image of the hand's appearance taken by a camera on the left, and an X-ray image of the same hand taken by a radiographic imaging device on the right.

[0088] The computer equipment provided in this embodiment takes appearance images and X-ray images of the actual or suspected fracture sites, and displays the X-ray images and appearance images on the same screen, so that doctors or technicians can directly compare the two types of images to diagnose the fracture site, thereby improving the accuracy of doctors' or technicians' diagnosis of fracture site.

[0089] Based on the above embodiments, the processor is further configured to identify at least one fracture location in the X-ray image of the target site and generate a mark for each fracture location, and to identify at least one injury location in the appearance image of the target site and generate a mark for each injury location; the display is further configured to display the mark for each fracture location in the X-ray image and the mark for each injury location in the appearance image.

[0090] In this embodiment, the processor further automatically identifies and marks the fracture location in the X-ray image and identifies and marks the trauma location in the appearance image. The display shows the marked trauma location and fracture location to help doctors focus on the marked locations in the two types of images when viewing them. By combining the marked locations in the two types of images, doctors can diagnose the fracture location, thereby providing more auxiliary information for doctors or technicians in the fracture location diagnosis process.

[0091] Corresponding to the above-described computer device embodiments, the embodiments of this application also provide an image-based fracture examination method.

[0092] This application provides an image-based fracture examination method, including:

[0093] Acquire X-ray images of the target area obtained by a radiographic imaging device, identify at least one fracture location in the X-ray images of the target area, and generate a marker for each fracture location;

[0094] The method involves acquiring an external image of the target body obtained by an image acquisition device, identifying at least one injury location in the external image of the target body, and generating a mark for each injury location; wherein the field of view of the target body captured by the radiographic imaging device at least partially overlaps with the field of view of the target body captured by the image acquisition device.

[0095] Display X-ray images of the target area, with markings for each fracture location in the X-ray images; and display an external image of the target area, with markings for each injury location in the external image.

[0096] Another embodiment of this application provides an image-based fracture examination method, comprising:

[0097] Acquire an external image of the target area obtained by an image acquisition device, and identify at least one location of external injury in the external image of the target area;

[0098] Display X-ray images of the target area obtained by a radiographic imaging device; wherein the field of view of the target area captured by the radiographic imaging device and the field of view of the target area captured by the image acquisition device at least partially overlap;

[0099] Whenever a user marks a fracture location in an X-ray image, the marked fracture location is displayed in the X-ray image. The marked fracture location in the X-ray image is combined with at least one trauma location identified in the external image to determine whether the marked fracture location in the X-ray image is misdiagnosed. If the marked fracture location is misdiagnosed, a message indicating that the marked fracture location is misdiagnosed is output.

[0100] The system responds to user input to complete the operation by combining the marked fracture locations in the X-ray images with at least one trauma location identified in the external images to determine whether there are any missed fracture locations in the X-ray images. If there are any missed fracture locations in the X-ray images, the system outputs a prompt message indicating that there are missed fracture locations in the X-ray images.

[0101] Another embodiment of this application provides an image-based fracture examination method, comprising:

[0102] Acquire X-ray images of the target area obtained by a radiographic imaging device; acquire external images of the target area obtained by an image acquisition device from the same target area; wherein the field of view of the target area captured by the radiographic imaging device and the field of view of the target area captured by the image acquisition device at least partially overlap;

[0103] The X-ray image of the target area and the external image of the target area are displayed on the same screen.

[0104] The terms "first," "second," etc., used in this specification, claims, and the accompanying drawings are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, or apparatuses.

[0105] The features described above in the disclosed embodiments can be substituted or combined with each other to enable those skilled in the art to implement or use this application. This document has been described with reference to various exemplary embodiments. However, those skilled in the art will recognize that changes and modifications can be made to the exemplary embodiments without departing from the scope of this document. For example, various operational steps and components for performing operational steps can be implemented in different ways depending on the specific application or considering any number of cost functions associated with the operation of the system (e.g., one or more steps can be deleted, modified, or combined with other steps).

[0106] Furthermore, as those skilled in the art will understand, the principles herein can be reflected in a computer program product on a computer-readable storage medium pre-loaded with computer-readable program code. Any tangible, non-transitory computer-readable storage medium may be used, including magnetic storage devices (hard disks, floppy disks, etc.), optical storage devices (CD-ROMs, DVDs, Blu-ray discs, etc.), flash memory, and / or the like. These computer program instructions may be loaded onto a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to form a machine, such that instructions, which execute on the computer or other programmable data processing apparatus, can generate means for performing a specified function. These computer program instructions may also be stored in a computer-readable storage medium that can instruct the computer or other programmable data processing apparatus to operate in a particular manner, such that instructions stored in the computer-readable storage medium can form an article of manufacture, including means for implementing the specified function. The computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to perform a series of operational steps on the computer or other programmable apparatus to produce a computer-implemented process, such that instructions, which execute on the computer or other programmable apparatus, can provide steps for implementing the specified function.

[0107] The foregoing specific descriptions have been described with reference to various embodiments. However, those skilled in the art will recognize that various modifications and changes can be made without departing from the scope of this disclosure. Therefore, considerations for this disclosure are to be illustrative rather than restrictive, and all such modifications are to be included within its scope. Similarly, advantages, other advantages, and solutions to problems with respect to various embodiments have been described above. However, benefits, advantages, solutions to problems, and any elements that produce these, or make them more explicit, should not be construed as critical, essential, or necessary. The term “comprising” and any other variations thereof as used herein are non-exclusive inclusion, meaning that a process, method, product, or apparatus that includes a list of elements includes not only those elements but also other elements not expressly listed or not part of the process, method, system, product, or apparatus. Furthermore, the term “coupled” and any other variations thereof as used herein refer to physical connections, electrical connections, magnetic connections, optical connections, communication connections, functional connections, and / or any other connections.

[0108] The above embodiments are merely illustrative of several implementation methods, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this invention, and these all fall within the protection scope of this invention. Therefore, the protection scope of this invention should be determined by the appended claims.

Claims

1. An image-based fracture examination device, characterized in that, Including processor and display; The processor is configured to: acquire X-ray images of the target site obtained by a radiographic imaging device, identify at least one fracture location in the X-ray images of the target site and generate a marker for each fracture location; and acquire an external image of the target site obtained by an image acquisition device, identify at least one injury location in the external image of the target site and generate a marker for each injury location; wherein: the field of view of the target site captured by the radiographic imaging device and the field of view of the target site captured by the image acquisition device at least partially overlap; The display is used to display X-ray images of the target site and an appearance image of the target site, and to display a mark for each fracture location in the X-ray images and a mark for each injury location in the appearance image.

2. The image-based fracture examination device as described in claim 1, characterized in that, The processor is also configured to: By combining at least one fracture location identified in the X-ray images and at least one injury location identified in the external images, it is determined whether the at least one fracture location identified in the X-ray images was missed or misdiagnosed; and, If at least one fracture location identified in the X-ray image is missed, the display is controlled to indicate the missed fracture location; if at least one fracture location identified in the X-ray image is misdiagnosed, the display is controlled to indicate the misdiagnosed fracture location.

3. The image-based fracture examination device as described in claim 2, characterized in that, The control of the display to indicate the location of missed fractures includes: The display is controlled to indicate the location of at least one fracture in the X-ray image using a first type of marking, and the display is controlled to indicate the location of a missed fracture in the X-ray image using a second type of marking.

4. The image-based fracture examination device as described in claim 2, characterized in that, The control of the display to indicate the location of the misdiagnosed fracture includes: The display is controlled to indicate misdiagnosed fracture locations in the X-ray images using a first type of marking, and the display is controlled to indicate non-misdiagnosed fracture locations in the X-ray images using a second type of marking.

5. The image-based fracture examination device as described in claim 1, characterized in that, The processor is further configured to: combine at least one fracture location identified in the X-ray image with at least one injury location identified in the appearance image to determine whether the at least one fracture location identified in the X-ray image is missed or misdiagnosed, and obtain the determination result of the fracture location in the X-ray image; The display is also used to output the determination result of the fracture location in the X-ray image.

6. The image-based fracture examination device as described in claim 2 or 5, characterized in that, The determination of whether the at least one fracture location identified in the X-ray images and the at least one injury location identified in the external images are missed or misdiagnosed includes: For any fracture location identified in the X-ray image, the fracture location is mapped to the appearance image. If the corresponding location of the fracture location in the appearance image does not match any injury location identified in the appearance image, it is determined that the fracture location identified in the X-ray image is misdiagnosed. For any injury location identified in the external image, the injury location is mapped to the X-ray image. If the corresponding location of the injury location in the X-ray image does not match any fracture location identified in the X-ray image, it is determined that at least one fracture location identified in the X-ray image has been missed.

7. The image-based fracture examination device as described in claim 1, characterized in that, Identifying at least one fracture location in the X-ray image of the target site includes: In response to a user's marking operation on the fracture location in the X-ray image of the target area, the marked fracture location is used as the identified fracture location in the X-ray image of the target area; or, The X-ray image of the target site is input into a pre-trained machine learning model, which identifies and outputs at least one fracture location in the X-ray image of the target site, wherein the machine learning model is trained from X-ray image samples containing fracture locations of the target site; or, Obtain a diagnostic report related to the target site, perform natural language processing on the diagnostic report to identify at least one fracture location recorded in the diagnostic report, and mark the at least one fracture location in the X-ray image of the target site.

8. The image-based fracture examination device as described in claim 1, characterized in that, Identifying at least one location of injury in the appearance image of the target area includes: In response to a user's annotation of the injury location in the appearance image of the target area, the annotated injury location is used as the identified injury location in the appearance image of the target area; or, The appearance image of the target part is input into a pre-trained machine learning model, and the machine learning model identifies and outputs at least one injury location in the appearance image of the target part, wherein the machine learning model is trained from appearance image samples containing the injury locations of the target part.

9. The image-based fracture examination device as described in claim 1, characterized in that, It also includes: an input device; the input device is used to acquire the user's operation of marking the fracture location in the X-ray image and to acquire the user's input of the marking completion operation; The step of identifying at least one fracture location in the X-ray image of the target site includes: whenever the user marks a fracture location in the X-ray image, the marked fracture location is determined as the identified fracture location; The step of displaying the mark of each fracture location in the X-ray image includes: whenever the processor receives an operation from the user to mark the fracture location in the X-ray image, it controls the display to display the mark of the marked fracture location in the X-ray image; The processor is further configured to: whenever a user marks a fracture location in the X-ray image, combine the marked fracture location in the X-ray image with at least one injury location identified in the appearance image to determine whether the marked fracture location in the X-ray image is misdiagnosed; if the marked fracture location is misdiagnosed, control the display to display a prompt message indicating that the marked fracture location is misdiagnosed; and, in response to a user-input marking completion operation, combine the marked fracture locations in the X-ray image with at least one injury location identified in the appearance image to determine whether there is a missed fracture location in the X-ray image; if there is a missed fracture location in the X-ray image, control the display to display a prompt message indicating that there is a missed fracture location in the X-ray image.

10. An image-based fracture examination device, characterized in that, Including processor and display; The processor is configured to: acquire X-ray images of the target area obtained by a radiographic imaging device; and acquire external images of the target area obtained by an image acquisition device of the same target area. The field of view of the target area captured by the radiographic imaging device at least partially overlaps with the field of view of the target area captured by the image acquisition device, and the target area has a fracture location or is suspected to have a fracture location; The display is used to simultaneously display the X-ray image of the target area and the appearance image of the target area.

11. The image-based fracture examination device as described in claim 10, characterized in that, The processor is further configured to: identify at least one fracture location in the X-ray image of the target site and generate a marker for each fracture location, and identify at least one injury location in the appearance image of the target site and generate a marker for each injury location; The display is also used to show a marker for each fracture location in the X-ray image and a marker for each injury location in the appearance image.

12. An image-based fracture examination method, characterized in that, include: Acquire X-ray images of the target area obtained by a radiation imaging device, identify at least one fracture location in the X-ray images of the target area, and generate a marker for each fracture location; An image of the target body obtained by an image acquisition device is captured on the same target body. At least one injury location is identified in the image of the target body and a mark is generated for each injury location. The field of view of the target area captured by the radiographic imaging device at least partially overlaps with the field of view of the target area captured by the image acquisition device; Display an X-ray image of the target site, and display a mark of each fracture location in the X-ray image; and display an external image of the target site, and display a mark of each injury location in the external image.

13. An image-based fracture examination method, characterized in that, include: Acquire X-ray images of the target area obtained by a radiation imaging device; acquire external images of the target area obtained by an image acquisition device from the same target area; The field of view of the target area captured by the radiographic imaging device at least partially overlaps with the field of view of the target area captured by the image acquisition device; The X-ray image of the target area and the appearance image of the target area are displayed on the same screen.

Images