Method and system for detecting rib fracture in medical image and medium

Abstract

The invention relates to a method, a system and a medium for rib fracture detection in a medical image. The method comprises the following steps: for a received medical image, analyzing an anatomical structure in the received medical image by using a mark detection algorithm; and segmenting the rib and the rib fracture area in the rib area through the corresponding deep neural network. According to the method, various image processing and artificial intelligence technologies are combined, and the rib fracture existing in the medical image can be quickly, accurately and automatically detected.

Description

Technical Field

[0001] The present invention relates to medical image processing technology, and in particular to a method, system and computer-readable storage medium for detecting rib fractures in medical images. Background Art

[0002] Rib fractures are common in blunt and penetrating chest trauma. Compared with traditional X-ray films, three-dimensional medical images obtained through computed tomography (CT), magnetic resonance imaging (MRI), etc. can provide richer imaging information. In the current common method, clinicians directly read the film through the viewer of the medical image storage and transmission system. They can adjust the contrast of the image, scroll through the axial slices, and look for rib-related patterns to characterize the fracture, and report information such as the affected ribs, fracture type, and relative position. However, due to the large number of ribs and irregular shapes, physicians need to observe rib fractures from a large number of collected images, which is a large workload, and the diagnosis conclusion is based on personal experience, which is prone to the risk of misdiagnosis or missed diagnosis. Summary of the invention

[0003] The purpose of the present invention is to provide a method, system and computer-readable storage medium for rib fracture detection in medical images, which combines multiple image processing and artificial intelligence technologies to automatically detect rib fractures in medical images.

[0004] A technical solution of the present invention is to provide a method for detecting rib fractures in medical images, which comprises:

[0005] receiving medical images;

[0006] parsing the anatomical structure in the medical image using a marker detection algorithm to determine a rib region in the medical image;

[0007] Using a first deep neural network, segmenting ribs in a rib region of the medical image;

[0008] The rib fracture area in the medical image is segmented according to the result of the rib segmentation using a second deep neural network.

[0009] In this scheme, the anatomical structure is first analyzed through the marker detection algorithm, so that the computing resources can be concentrated on the determined rib area to improve the detection efficiency; the corresponding deep neural network is used to segment the ribs and fractures, which can automatically and accurately detect rib fractures.

[0010] Optionally, using the marker detection algorithm, the anatomical structure is parsed on the medical image to obtain the position information of the anatomical landmark;

[0011] When using the marker detection algorithm, including:

[0012] An artificial intelligence agent trained to execute a set of actions for moving a state space toward an anatomical landmark in a medical image and maximizing a cumulative future reward value based on an optimal action-value function approximator;

[0013] The state space is defined as candidate locations of anatomical landmarks and fixed regions around the candidate locations;

[0014] The set of actions performed is selected from a predetermined action set; the actions in the action set include: one or more parameter actions for changing the focus position of the state space, and one or more scale scaling actions for changing the resolution of the state space; a corresponding reward value is set for each action in the action set according to the proximity between the state space and the anatomical landmark after the action is performed;

[0015] The set of actions performed includes at least one scaling action.

[0016] In this solution, the modeling of the appearance of the target object and the search strategy is used as a unified behavioral task, which is automatically performed by the artificial intelligence agent. Feature extraction is performed by repeated interaction with the closed environment of the image. The accumulated reward changes quantify the behavior of the artificial intelligence agent, which facilitates its optimization of action selection. In the disclosed embodiment, both the optimization of the execution of behavioral learning through reinforcement learning and the advantages of effective hierarchical feature extraction through deep learning are achieved.

[0017] Optionally, a set of actions performed by a trained artificial intelligence agent includes multiple scale scaling actions, corresponding to multiple scales for gradually increasing the resolution of the state space; the convergence point when searching for anatomical landmarks at each scale is used as the starting position for searching at the next scale; at the scale with the lowest resolution, the search starts from an arbitrary position or a specified position.

[0018] In this scheme, a scale scaling action is performed to provide search windows with different sizes and resolutions; the artificial agent obtains different fields of view when searching for the target object, which effectively improves the speed and accuracy of detection.

[0019] Optionally, the first deep neural network is a first image-to-image neural network; the trained first image-to-image neural network generates a rib segmentation mask for segmenting ribs based on an image corresponding to a rib region input thereto.

[0020] In this scheme, the image can be directly input into the image-to-image neural network for segmentation processing without pre-specifying features, thereby improving detection efficiency; moreover, the same network can be used to perform multiple different segmentation tasks simultaneously (multi-label segmentation), or the network architecture or training method can be modified to adapt to different data sets and tasks.

[0021] Optionally, the training of the first image-to-image neural network comprises performing adversarial training on the first image-to-image neural network as a first generator together with a first discriminator; the first discriminator is used to distinguish between a predicted rib segmentation mask generated by the first generator according to the training image and a ground truth rib segmentation mask;

[0022] The first segmentation loss is reduced by learning loss weights of the first generator and the first discriminator to minimize a first loss function of the first image-to-image neural network, thereby increasing the probability that the predicted rib segmentation mask is negatively classified by the first discriminator and increasing the probability that the ground truth rib segmentation mask is positively classified by the first discriminator; the first segmentation loss is a binary cross entropy between the predicted rib segmentation mask and the ground truth rib segmentation mask.

[0023] During the training process, this scheme uses an adversarial network to distinguish between the predicted samples and the real samples output by the image-to-image neural network, achieving good segmentation quality and faster processing speed.

[0024] Optionally, the second deep neural network is a second image-to-image neural network; the trained second image-to-image neural network is input with the received medical image and a rib segmentation mask obtained by segmenting the ribs in the rib region of the medical image by the first deep neural network;

[0025] According to the rib segmentation mask, the loss weight of the second image-to-image neural network with a second loss function is adjusted so that the loss value outside the rib area is 0; and the image-to-image neural network after the loss weight adjustment is used to generate a fracture segmentation mask for segmenting rib fractures based on the received medical image.

[0026] Optionally, the training of the second image-to-image neural network comprises performing adversarial training on the second image-to-image neural network as a second generator together with a second discriminator; the second discriminator is used to distinguish between a predicted fracture segmentation mask generated by the second generator according to the training image and a ground truth fracture segmentation mask; during the training, the loss weight of the second loss function of the second image-to-image neural network is adjusted using the ground truth rib segmentation mask so that the loss value outside the rib region is 0;

[0027] By learning the loss weights of the second generator and the second discriminator to minimize the second loss function, the second segmentation loss is reduced, the probability of the predicted fracture segmentation mask being negatively classified by the second discriminator is increased, and the probability of the ground truth fracture segmentation mask being positively classified by the second discriminator is increased; the second segmentation loss is the binary cross entropy between the predicted fracture segmentation mask generated by the second image-to-image neural network based on the training image and the ground truth fracture segmentation mask.

[0028] In this scheme, the loss weight of the second image-to-image neural network is adjusted through the rib segmentation mask, so that we can focus on processing the features and details in the rib area, improve the accuracy of detection, and ignore the influence of factors such as noise and interference outside the rib area on training and segmentation tasks, avoid wasting computing resources to suppress false positives, and improve the efficiency of detection and training.

[0029] Optionally, the first image-to-image neural network and the second image-to-image neural network have the same backbone network for extracting features; the backbone network has a convolutional encoder-decoder architecture, including multiple encoders, multiple decoders, and multiple branches connecting at least a portion of the decoders to corresponding output layers;

[0030] The size of the output channel of each branch matches the size of the input image of the image-to-image neural network; the features of the output layers corresponding to all branches are concatenated and input into a final output layer; the segmentation loss of the image-to-image neural network corresponds to the weighted loss terms of the output layers of all branches and the final output layer. In this scheme, the performance of the image-to-image neural network can be improved based on the deep supervision network, and good boundary detection and segmentation results can be achieved.

[0031] Optionally, the method further comprises:

[0032] Using a rib segmentation mask on the medical image to obtain a rib image, using an algorithm for tracing the rib centerline to track the centerline of each rib in the rib image, and generating a centerline segmentation mask for extracting the rib centerline;

[0033] When using the algorithm for tracing the rib centerline, including:

[0034] Image noise is filtered out by anisotropic filtering, and voxels with values ​​greater than a set threshold in the rib image are determined as foreground;

[0035] The grayscale weighted distance transform method is used on the segmented foreground to obtain a weighted distance map; the point corresponding to the maximum value in the weighted distance map is used as the source point for the fast marching method to calculate the 3D distance estimate;

[0036] Use the RK4 gradient descent method to iteratively trace the current longest branch, each trace starts from the farthest position in the current remaining foreground and stops when it reaches the source or reaches the previously traced branch;

[0037] Calculate a confidence score for each tracked branch, wherein the confidence score represents the proportion of the current tracked branch encountering the foreground voxel; retain the tracked branches whose confidence scores are higher than a set threshold;

[0038] The centerline segmentation mask is generated according to the retained tracked branches, and the centerline segmentation mask is applied to the rib image to extract the centerline of each rib.

[0039] The algorithm used in this scheme to track the rib centerline combines fast marching and gradient descent methods to track discontinuous areas, and can find branches that are less affected by noise through backtracking iterations; a more accurate geodesic distance estimate with subvoxel accuracy can be obtained through the multi-template fast marching method, so that the tracked curve does not require post-smoothing; branches that are severely mistracked due to noise points or gaps can be eliminated through confidence scores. This scheme can quickly and accurately detect the rib centerline, with good robustness and low time complexity.

[0040] Optionally, the method further comprises:

[0041] Each connected domain of the fracture segmentation mask is defined as the fracture area;

[0042] For each fracture region, a third deep neural network was used to determine the fracture type;

[0043] Calculate the center point of each fracture area, determine the point on the rib where the fracture is located that is closest to the center point, and use the relative position of the rib of the closest point as the relative position of the fracture;

[0044] The ribs affected by the fracture are determined based on the intersection of each fracture region with the rib segmentation mask.

[0045] In this solution, based on the results of rib segmentation and rib fracture segmentation, the information required to generate a report related to rib fracture information can be easily obtained, and the visualization results related to the ribs and rib fractures (such as but not limited to advanced rendering of the ribs, rib fractures, rib centerline contours, corresponding fracture areas of interest, etc.) can be superimposed on the relevant images for display, so that clinical physicians can observe the site of the fracture or lesion.

[0046] Optionally, the medical image is a three-dimensional medical image obtained by computer tomography or magnetic resonance imaging. Three-dimensional medical images can provide richer imaging information; the present solution can be directly compatible with the workflow of a system that acquires and / or stores three-dimensional medical images, so as to receive images, or provide test results such as images superimposed with visualization results and reports filled with rib fracture information to these systems, without the need to set up a dedicated workstation for processing.

[0047] Another technical solution of the present invention is to provide a system for detecting rib fractures in medical images, the system comprising a module for implementing any one of the above methods for detecting rib fractures in medical images; the module comprises:

[0048] A module for receiving a medical image;

[0049] A module for parsing anatomical structures in medical images using a landmark detection algorithm to determine rib regions;

[0050] a module for segmenting ribs in the rib region of a medical image by means of a first deep neural network;

[0051] Module for segmenting rib fracture regions in medical images using a second deep neural network.

[0052] Another technical solution of the present invention is to provide a computer-readable storage medium storing computer program instructions; when the computer program instructions are executed by a processor, any one of the above methods for detecting rib fractures in medical images is implemented.

[0053] In summary, the method, system and computer-readable storage medium for detecting rib fractures in medical images described in the present invention combine a variety of image processing and artificial intelligence technologies, and can perform rapid and accurate automatic detection of rib fractures in medical images, helping clinical physicians make diagnostic decisions, effectively saving time and reducing workload. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 is a schematic flow chart of a rib fracture detection method provided by an embodiment of the present invention;

[0055] Figure 2 is a flow chart of a rib fracture detection method provided by another embodiment of the present invention;

[0056] Figure 3 is a schematic diagram of an uncropped medical image in an example of the present invention;

[0057] Figure 4 is Figure 3Schematic diagram of a medical image cropped around a rib region of interest;

[0058] Figure 5 is a schematic diagram of the training process of an artificial intelligence agent for parsing anatomical structures in an example of the present invention;

[0059] Figure 6 is a schematic diagram of the architecture of an image-to-image neural network for segmenting ribs in an example of the present invention;

[0060] Figure 7 yes Figure 6 Schematic diagram of adversarial training of image-to-image neural network in ;

[0061] Figure 8 is an image obtained after segmenting the ribs in the example of the present invention;

[0062] Fig. 9 is an image obtained after extracting the rib centerline in the example of the present invention;

[0063] Fig.10 is a schematic diagram of an implementation flow of an algorithm for tracing a rib centerline in an example of the present invention;

[0064] Fig.11 is a schematic diagram of the input and output channels of the image-to-image neural network for segmenting rib fractures in an example of the present invention;

[0065] Fig.12 is a flow chart of training two image-to-image neural networks and using them for detection in an example of the present invention;

[0066] Fig.13 is a schematic diagram of a rib expansion diagram superimposed with a visual inspection result in an example of the present invention;

[0067] Fig.14 is a schematic diagram of a chest three-dimensional reconstruction image superimposed with a visual detection result in an example of the present invention;

[0068] Fig.15 is a schematic diagram of a rib fracture detection system provided in an embodiment of the present invention;

[0069] Fig.16 is a schematic diagram of a computer upon which the present invention may be implemented.

[0070]

[0071]

[0072] DETAILED DESCRIPTION

[0073] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. The terms "first", "second", etc. in this article are used to distinguish different elements, and do not have a logical or temporal dependency, nor are they limitations on quantity and execution order. In the various embodiments of this article, the timing of the method steps is only used as an example, and the size of the sequence number does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment.

[0074] Rib fractures are common chest injuries. Three-dimensional (3D) medical images acquired in the context of trauma through computed tomography (CT), magnetic resonance imaging (MRI), etc. can provide rich imaging information and are widely used. Clinicians (such as doctors, medical professionals, or any other users) originally manually read the images and intuitively looked for patterns related to the ribs to characterize the fractures and report information related to rib fractures; however, there is a lack of effective algorithms or tools to assist in diagnosis, which is time-consuming and labor-intensive, and the accuracy is also affected by the experience of the reader.

[0075] The system and method provided by the present invention can combine a variety of artificial intelligence technologies and advanced image processing technologies to automatically detect, segment, characterize and render rib fractures in medical images in a modular workflow manner.

[0076] exist Figure 1 In one embodiment shown, a rib fracture detection method is provided, comprising the following steps:

[0077] Step 101, receiving a medical image;

[0078] Step 102: Analyze the anatomical structure in the medical image and determine the rib region in the medical image;

[0079] Step 103: segment the ribs in the rib region of the medical image using a first deep neural network;

[0080] Step 104: Use a second deep neural network to segment the rib fractures in the medical image according to the result of the rib segmentation.

[0081] For example, in step 102, the anatomical structure in the medical image is analyzed by a landmark detection algorithm to determine the rib region, so that in the subsequent process, computing resources can be concentrated to process the region, thereby improving detection efficiency.

[0082] For example, the first deep neural network in step 103 is a first image-to-image neural network, which is used to generate a rib segmentation mask for segmenting ribs based on an input medical image (in this example, based on an image of a corresponding rib region).

[0083] The second deep neural network in step 104 is a second image-to-image neural network, which has the same backbone network (backbone) for feature extraction as the first image-to-image neural network; and the input channels and loss weights of the second deep neural network can be adjusted accordingly according to the rib segmentation mask output by the first deep neural network, so that the second image-to-image neural network can generate a fracture segmentation mask for segmenting rib fractures according to the medical image and rib segmentation mask input thereto.

[0084] The exemplary rib fracture detection method can also superimpose visualization results related to ribs and rib fractures onto an image for presentation, and generate a report on rib fracture information. The report on rib fracture information includes, for example, affected ribs, fracture types, and relative positions, but is not limited thereto. When characterizing the fracture, a third deep neural network is also used to infer the fracture type based on the image area corresponding to each fracture in the medical image. For example, a residual neural network or a visual transformer can be used as the third deep neural network.

[0085] For example, visualization results related to ribs and rib fractures include rib contours obtained according to a rib segmentation mask and fracture contours obtained according to a fracture segmentation mask; the rib contours and / or fracture contours can be superimposed on related images for display. In some examples, the region of interest corresponding to the fracture obtained during fracture segmentation can also be superimposed on the related image for display. In some examples, the center lines of each rib can also be superimposed on the related image for display. For example, the center line of each rib can be extracted by the Rivulet tracking algorithm based on the segmented ribs. The above-mentioned related images include, for example, but not limited to, the original input medical image, or the rib unfolded image obtained after rendering the input medical image, or the three-dimensional reconstructed image of the rib area or chest, etc.

[0086] like Figure 2 In one embodiment shown, a rib fracture detection method is provided, comprising the following steps:

[0087] Step 201, receiving a medical image;

[0088] Step 202, analyzing the anatomical structure present in the medical image;

[0089] Step 203, defining a region of interest (ROI) corresponding to the ribs, and cropping the image around the region;

[0090] Step 204: segment and label the ribs in the image using the first deep neural network; perform multi-label binary segmentation, set labels for all ribs, and set a corresponding label for each rib;

[0091] Step 205: extract the center line of each rib based on the rib segmentation result;

[0092] Step 206: each rib is subdivided into three regions (i.e., the front region, the side region, and the back region), each region of each rib is provided with a corresponding label, which is associated with the corresponding rib and the label of the rib, and the result is stored in the multi-label segmentation data;

[0093] Step 207: segment rib fractures using a second deep neural network according to the result of rib segmentation;

[0094] Step 208, characterizing each fracture; for each connected domain in the fracture segmentation result, defining a region of interest (FxROI) corresponding to the fracture, and using a third deep neural network to infer the fracture type;

[0095] Step 209, calculating the relative position of each fracture in the relevant ribs;

[0096] Step 210, forming a report related to the rib fracture information, which can be viewed or stored;

[0097] Step 211: Overlay visualization results related to ribs and rib fractures onto the image for presentation.

[0098] The specific implementation of the above detection method is further described below.

[0099] In step 201, the medical image may be a three-dimensional medical image obtained by using any medical imaging modality, such as computed tomography (CT), magnetic resonance imaging (MRI), etc., on a patient (or other subject), at least covering the image of the chest (for example, some images also cover the image of the head and abdomen at the same time). The required medical image may be received directly from an image acquisition device, such as a CT scanner, an MRI scanner, etc.; or, the medical image may be received by loading a previously stored medical image from a memory or storage device of a computer system, or by electronic transmission from another computer system. For example, a system for implementing rib fracture detection (hereinafter referred to as the detection system) may obtain the required three-dimensional medical image from a medical data archiving system storing medical images, such as a PACS system (picture archiving and communication system), a RIS system (radiology information system), an EMR system (electronic medical record), etc.

[0100] In some examples, the received three-dimensional medical image is in DICOM format (i.e., Digital Imaging and Communications in Medicine), which needs to be converted into a data format applicable to the detection system. For example, data in DICOM format is converted into data of type mhd+raw or mhd+img; where raw or img formats store image information (such as three-dimensional volume data); and mhd format stores data header information, which stores non-image information of the data.

[0101] The detection system can extract a portion of the image data from the converted data and store it for use in subsequent detection processes. For example, the image data extracted from the header information of the converted data includes the original image volume data and image space metadata corresponding to the received three-dimensional medical image; wherein the original image volume data, for example, includes volume data, intercept and slope, data type (short or unsigned short), etc.; the image space metadata, for example, includes volume size (in voxel number), voxel spacing (in millimeters), image origin and direction, etc.

[0102] In step 202, the anatomical structure present in the three-dimensional medical image can be automatically parsed through the landmark detection algorithm, and the position information of the anatomical landmarks (such as the coordinates of the key points) can be output. The anatomical landmarks are key points that have specific characteristics that are generally present in the human body (or animal) in the anatomical sense, and have corresponding relationships in position and topology, and will present obvious grayscale features on the medical image; by detecting the anatomical structure, the corresponding body parts (such as various bones, blood vessels, organs, etc.) can be roughly determined to provide initialization conditions for subsequent segmentation tasks. In step 203, the region of interest (ROI) corresponding to the rib is defined according to the anatomical structure parsed in step 202 (if necessary, it can also be combined with the data that has been format-converted in step 201 or the image data extracted therefrom), and the image is cropped around it ( Figure 3 , Figure 4 In one example, a medical image 301 before cropping and a medical image 302 after cropping are respectively shown; Figure 3 and Figure 4 In the figure, P stands for the front, L stands for the left, R stands for the right, and A stands for the back).

[0103] In an example of the landmark detection algorithm described in step 202, the concept of deep reinforcement learning is combined with multi-scale image analysis, and a set of annotated training images are provided for training. The artificial intelligence agent not only learns how to identify the target object in the image (such as distinguishing the anatomical structure from other parts and giving the location of the anatomical landmark), but also learns how to find the best strategy for the target object to effectively optimize the method of identifying the target object. The training image can be a three-dimensional medical image, in which the anatomical structure, anatomical landmark, etc. are manually annotated, and the image comes from a medical image database, for example. The artificial intelligence agent (i.e., the artificial intelligence algorithm and the carrier of its implementation) is implemented on one or more computers or processors by executing computer program instructions (codes) loaded into the memory, for example, it can perceive the surrounding environment, combine the perceived information and prior knowledge to perform calculations, and perform operations.

[0104] like Figure 5 As shown, when training an artificial intelligence agent, the following steps are included:

[0105] Step 501, specifying the state space of the artificial intelligence agent for the discrete parts of the training image;

[0106] Step 502: Determine the actions that can be used in the state space;

[0107] Step 503: Establish a reward system; determine a reward value for each action in the action set based on at least one target position in the training image;

[0108] Step 504: The artificial intelligence agent learns an optimal action-value function approximator to specify that the artificial intelligence agent performs a set of actions that maximize the cumulative future reward value of the reward system.

[0109] In this example, the appearance of the target object (such as anatomical landmarks) and the modeling of the search strategy are taken as a unified behavioral task. During training, the artificial intelligence agent selects a set of actions in each state space according to the set state space, action set and reward system, and extracts features by repeatedly interacting with the closed environment of the image to maximize the accumulated future reward value, so as to optimize its action selection during the training phase.

[0110] In step 501, the state space is modeled as candidate locations of the target object and fixed regions around the candidate locations, specified by the parameter space and scale space of discrete portions of the training image. The length, width, and depth of the state space are represented by the number of voxels defined in the parameter space, and the focus is defined as the center coordinate of the set of voxels. The resolution of the state space is specified by the scale space. For example, at each resolution, the state space may include the same number of voxels. At higher resolutions, voxels are sampled from smaller volumes (higher density) of image data. Conversely, at lower resolutions, voxels are sampled from larger volumes (lower density) of image data.

[0111] The action set described in step 502 includes parameter actions and scale scaling actions. The parameter actions are used to specify possible changes in the parameter space relative to the image, that is, the parameter space of the state space is changed by sampling voxels from different positions in the image; based on the selected parameter actions, the focus position of the state space is changed by moving at the voxel level, for example, according to six optional parameter actions, the focus position of the state space can be changed in six directions (upward, downward, left, right, forward, and backward) relative to the current image, and each parameter action can move the focus position by one voxel in the corresponding direction.

[0112] The scaling action is used to specify possible changes in the scale space relative to the image; that is, the volume or density of the image data sampling the voxels of the state space is changed. Through the selected scaling action, such as the action of enlarging or reducing the scale space, the resolution of the state space relative to the image can be increased or decreased accordingly. The action set can also include the action of keeping the parameter space unchanged, keeping the focus position at the same position and not moving; and the action of keeping the scale space unchanged and not adjusting the resolution. That is, the action that can be selected also includes the action of keeping the state space unchanged. Once the target object is reached, no further actions are performed and the search can be ended.

[0113] In step 503, a reward system is established based on each action in the action set and on at least one target position of the training image (e.g., the coordinates of an anatomical landmark). For each action that the artificial intelligence agent may select from the action set, a reward value is determined based on the proximity between the current state space of the artificial intelligence agent and the target position; that is, the reward value of each action indicates the proximity between the state space and the target position after the action is performed; taking the aforementioned six directions of parameter actions as an example, a positive reward (increase in reward value) will be given for movement in the correct direction (indicating a decrease in the distance from the target position); and a negative reward (deduction in reward value) will be given for movement in the wrong direction (indicating an increase in the distance from the target position). A special case is that if an anatomical landmark cannot be detected in a medical image (e.g., because the patient itself lacks an organ, or because the anatomical landmark cannot be detected due to changes in the field of view), the navigation path during the search will attempt to cross the boundary of the image, and this operation will be subject to the highest penalty (deduction of the highest reward value), thereby also being able to identify the situation where the target object does not exist in the image.

[0114] The change in the accumulated reward quantifies the behavior of the artificial intelligence agent, so the behavior of the artificial intelligence agent can be optimized based on a mechanism that maximizes the accumulated future reward value. The behavior, for example, is a set of actions that move the artificial intelligence agent toward at least one target position in the image, which can be a parameter action or a scale-scaling action selected from the action set, and includes at least one scale-scaling action; wherein the parameter action moves the artificial intelligence agent to the target position within a specific scale, and the scale-scaling action improves the resolution of the artificial intelligence agent.

[0115] The scale space defines search windows of different sizes and resolutions; by performing a scale scaling action to change the scale space, the artificial intelligence agent can obtain different fields of view when searching for a target object (such as an anatomical landmark) to improve the speed and accuracy of detection. For example, in step 504, the artificial intelligence agent searches at different scale levels from coarse to fine, iteratively reducing the field of view (or density) to improve the resolution, so that attention is gradually focused and the search range of each scale is adjusted, and each transition to a finer scale can capture more details to guide the search. At each valid scale, the artificial intelligence agent analyzes applicable context and appearance factors to converge on the searched target object, and obtains a path for navigating from the starting position to the target position at the scale (the path can be presented in a visual manner on the image); determining the target position can include identifying the oscillation of the artificial intelligence agent between adjacent state space positions, and the artificial intelligence agent navigates in the image until the oscillation point of the trajectory between adjacent voxels converges.

[0116] The search starts from the image data of the coarsest scale (corresponding to the largest field of view), and the starting position can be any point in the image (or, in other examples, the starting position of the coarsest level is pre-specified, for example, the average position of the target object in the training data is used as the starting position of the coarsest level). After converging to the target position during the coarsest scale search, the artificial intelligence entity increases the resolution and enters a finer scale to search for the target object, and uses the convergence point at the coarsest scale of the previous level as the starting position for navigation at the current scale until it converges to the target position at the current scale; the above process is iterated at each scale, and the convergence point at one scale is used as the starting position at the next finer scale; the convergence point at the last finest scale is output as the detection result. Except for the coarsest scale, each subsequent level constrains the search range based on the convergence behavior of the previous level, so that the search range of each level decreases as the resolution increases. The search range at the coarsest scale is only constrained by the entire 3D volume, which can obtain a sufficiently large global context, increase the possibility of converging to the target location, and reduce the possibility of losing data in a smaller field of view; it can also avoid more detailed detection in areas far away from the target location, improving the speed and accuracy of detection. For example, using an independent search strategy at each scale level allows the search to adapt to the most discriminative image features visible at that scale.

[0117] In step 504, through the combination of deep learning (DL) and reinforcement learning (RL), the artificial intelligence agent learns the best action value function approximator that specifies the agent behavior; in this example, a deep Q network (DQN) is used as the best action value function approximator. DQN is a model-free temporal difference method based on the use of a deep convolutional neural network (CNN) to approximate the best action value function Q*; DQN is trained using an iterative method to minimize the mean square error based on the Bellman optimal equation; the supervised setting of deep learning combines stochastic gradient descent (SGD) with the back-propagation algorithm, periodically applying stochastic gradient descent steps to approximate the gradient through random sampling. The best action value function approximator is learned to specify that the artificial intelligence agent performs a set of actions that maximize the cumulative future reward value of the reward system. The behavior of the artificial intelligence agent is self-determined and can intelligently select the next action to change the position of the target object in the state space.

[0118] In addition, one way to ensure the stability of training is to decorrelate the training samples based on the concept of experience replay. That is, learning the best action value function approximator also includes generating an experience memory database that stores a limited number of previously executed states; by sampling the experience memory database, the parameters of the best action value function approximator are updated based on the experience memory. In an event, the artificial intelligence agent uses the ε-greedy algorithm to perform actions in a local environment (a given training image); when the target state is reached (such as finding the target object, or performing a predetermined maximum number of actions), the current event ends, thereby obtaining an event trajectory that instructs the artificial intelligence agent to perform a corresponding action sequence in a given training image by applying a search strategy. For example, some recently generated event trajectories are stored in the memory of the experience memory database; these event trajectories represent the experience accumulated by the artificial intelligence agent on different images, and sampling them and periodically updating the parameters of the best action value function approximation can more accurately approximate the optimal action value function Q* based on experience.

[0119] In summary, in this embodiment, the modeling of the appearance of the target object and the parameter / scale search strategy is used as a unified behavioral task, which not only optimizes the execution of behavioral learning through reinforcement learning, but also achieves the advantages of effective hierarchical feature extraction through deep learning; the artificial intelligence agent is not only trained to identify the target object, but also trained to learn the best navigation path to find the target object. Traditional medical image parsing methods based on machine learning only focus on the learning and training of appearance models, and often require exhaustive search for each image, which has low computational efficiency. In comparison, the method of this embodiment can locate the target object faster and more accurately; the trained artificial intelligence agent can be applied to the image to be tested (such as a three-dimensional medical image of a patient) received by the system, and the anatomical structure existing in the image to be tested can be parsed according to the method learned in the training stage, and the location information of the anatomical landmarks can be output (it can also identify the situation where a specific anatomical structure does not exist).

[0120] In step 204, when the ribs are segmented and labeled by the first deep neural network, the input is the cropped image in step 203 (if necessary, the intermediate information obtained by the other steps mentioned above can also be input); the rib segmentation task is defined as multi-label binary classification by voxel, and labels are set for all ribs together and for individual ribs respectively; the first deep neural network outputs a rib segmentation mask with 25 output channels (including one channel corresponding to the background and one channel corresponding to each rib), which is used to define the ribs existing in the medical image.

[0121] The first deep neural network of the example is an image-to-image neural network (DI2IN). The image-to-image neural network is a convolutional encoder-decoder architecture designed in a symmetrical manner, and combines a multi-level feature cascade and a deep supervision network. During training, the image-to-image neural network is used as a generator, and is trained together with the discriminator in an adversarial network to improve its performance. The trained image-to-image neural network can generate a probability map based on the input three-dimensional medical image to indicate the probability that the voxel belongs to the rib; for example, voxels with a probability greater than a threshold are marked as positive (indicating that they belong to the rib), and voxels with a probability less than a threshold are marked as negative (indicating that they do not belong to the rib); the probability map can be converted into a rib segmentation mask output. The mask can be superimposed on the input image to highlight the boundary contour of the segmented rib area.

[0122] like Figure 6 In an example shown, the input 601 of the first image-to-image neural network 600 is a training image (or a medical image to be detected), and the output 602 is a probability map or a corresponding rib segmentation mask. In the encoder of the first image-to-image neural network 600, all blocks use convolutional layers (some of which have a stride of 2); in the decoder, some blocks use convolutional layers, and some blocks include convolutional layers and bilinear upscaling layers; the upscaling layers are implemented in a bilinear interpolation manner; all convolution kernels are 3×3×3; for the x, y, z dimensions, the upscaling factor in the decoder is 2; leaky rectified linear units (Leaky ReLU) and batch normalization are used in all convolutional layers for proper gradient back propagation.

[0123] The encoder includes four blocks BLK1 to BLK4, two convolutional layers 6011 and 6012 of BLK1, two convolutional layers 6021 and 6022 of BLK2, two convolutional layers 6031 and 6032 of BLK3, and two convolutional layers 6041 and 6042 of BLK4 (s 6011 =s 6021 =s 6031 =s 6041 =1,s 6012 =s 6022 =s 6032 =s 6042 =2,f 6011 =f 6012 =16, f 6021 =f 6022 =32, f 6031 =f 6032 =64,f 6041 =f 6042 =128; where s represents the stride, f represents the number of filters, and the subscripts correspond to the numbers of the layers).

[0124] Block BLK5 in the decoder uses two convolutional layers 6051, 6052 (s 6051 =s 6052 =1,f 6051 =f 6052 =256). In the decoder blocks BLK6 to BLK8, BLK6 has a bilinear amplification layer 6061 and a convolution layer 6062, BLK7 has a bilinear amplification layer 6071 and a convolution layer 6072, and BLK8 has a bilinear amplification layer 6081 and a convolution layer 6082 (s 6061 =s 6071 =s 6081 =2,s 6062 =s 6072 =s 6082 =1,f 6062 =128, f 6072 =64,f 6082 =32); The decoder block BLK9 has four layers, the first bilinear amplification layer 6091 (s 6091 =2), the first convolutional layer 6092 (s 6092 =1,f 6092 =16), the second bilinear amplification layer 6093 (s 6093 =16), the second convolutional layer 6094 (s 6094 =1,f 6094 =8).

[0125] In some examples, the front end of the image-to-image neural network uses feature concatenation to build a fast bridge between some layers in the encoder and some layers in the decoder, concatenating the low-level features in the encoder with the high-level features in the decoder, and the combined features are used as the input of the next convolutional layer of the decoder. Figure 6 As shown, in an example of the first image-to-image neural network 600, the outputs of blocks BLK1 to BLK4 are connected to the feature connection points before the inputs of blocks BLK6 to BLK9 through their respective corresponding branches 6211, 6212, 6213, and 6214, and the features combined by BLK1+BLK5 are used as the input of BLK6, the features combined by BLK2+BLK6 are used as the input of BLK7, the features combined by BLK3+BLK7 are used as the input of BLK8, and the features combined by BLK4+BLK8 are used as the input of BLK9.

[0126] In some examples, the backend of the image-to-image neural network uses a multi-level deep supervision network. The outputs of some blocks at the decoder are led out through the corresponding branches; through appropriate amplification and convolution operations, the size of the output channels of all branches matches the size of the input image. Figure 6In the example, the outputs of blocks BLK5, BLK7, and BLK9 are connected to the inputs of blocks BLK10, BLK11, and BLK12 of the deep supervision network through corresponding branches 6311, 6312, and 6313, respectively (the amplification factors of these three blocks are 16, 4, and 1, respectively).

[0127] Among them, BLK10 has four convolutional layers 6101~6104 (s 6101 =s 6102 =s 6103 =s 6104 =1,f 6101 =f 6102 =f 6103 =8, f 6104 =1). BLK11 has a bilinear upscaling layer 6111 and three convolutional layers 6112-6114 (s 6111 =4,s 6112 =s 6113 =s 6114 =1,f 6112 =f 6113 =8, f 6114 =1). BLK12 has a bilinear amplifier layer 6121 (s 6121 = 1) and a convolutional layer of 6122 (s 6122 =1,f 6122 =1). The outputs of blocks BLK10 to BLK12 are concatenated and used as the input of block BLK13; BLK13 has a convolutional layer 6131 (s 6131 =1, f 6131 =1), whose output is the final output 602 of the first image-to-image neural network 600. The ground truth data is used to calculate the loss term, and supervision is performed at the end of each branch 6311, 6312, 6313; the outputs of these branches 6311, 6312, 6313 are convolved with Leaky ReLU to determine the final output 602. During training, a binary cross entropy loss is applied to each voxel of the output layer. The total loss l to be minimized during training total It is the weighted sum of the loss terms of all output layers (including the final output layer and the output layers of all branches).

[0128] like Figure 7 , Fig.12As shown, a training sample is received, and the first image-to-image neural network 600 is used as a generator and trained in an adversarial network together with a discriminator 604; the generator attempts to produce an output close to the real sample, while the discriminator 604 attempts to distinguish between the real sample and the generated sample. In some optional examples, the first image-to-image neural network 600 is pre-trained before training through the adversarial network to minimize the loss calculated between the ground truth label and the predicted output for a set of training samples.

[0129] exist Fig.12 In the example of FIG. 1 , the training phase 1201 of the first image-to-image neural network 600 includes the following steps:

[0130] Step 1211, receiving training samples;

[0131] Step 1212, the first image-to-image neural network 600 is pre-trained separately;

[0132] Step 1213: The first image-to-image neural network 600 is subjected to adversarial training together with the discriminator.

[0133] Then, in the reasoning stage 1203, the following steps are included:

[0134] Step 1231, receiving a three-dimensional medical image of a patient (for example, a cropped image);

[0135] Step 1232: segment the ribs in the three-dimensional medical image using the trained first image-to-image neural network 600;

[0136] Step 1232: Output the rib segmentation mask.

[0137] The received training samples include multiple training pairs, each training pair includes a three-dimensional medical image (for example, from a medical database) as a training image, and a ground truth rib segmentation mask corresponding to the training image (generated based on manual annotations of rib boundaries in the training image).

[0138] The discriminator 604 is a fourth deep neural network, which attempts to distinguish the ground truth labels (i.e., the ground truth rib segmentation mask 603) and the predicted labels generated by the generator (i.e., the predicted rib segmentation mask generated by the image-to-image neural network 600 based on the training image) respectively input thereto, and classifies these label maps as true (=1) or predicted (=0); for a set of input label maps, the output 605 of the discriminator 604 corresponds to the classification result (0 or 1). The exemplary discriminator 604 includes blocks BLK14 to BLK16, wherein BLK14 has two convolutional layers 6141, 6142 (s 6141 =1,s 6142=2,f 6141 =f 6142 =16), BLK15 has three convolutional layers 6151~6153 (s 6151 =s 6152 =1,s 6153 =2,f 6151 =f 6152 =f 6153 =32), BLK16 has four convolutional layers 6161~6164(s 6161 =s 6162 =s 6163 =s 164 =1,f 6161 =f 6162 =f 6163 =64,f 6164 =1).

[0139] The adversarial network provides an additional loss function, the purpose of which is to make the predicted label as close to the ground truth label as possible, which is used to update the parameters (weights) of the generator during training. The adversarial network is trained using binary cross entropy loss to improve the performance of the image-to-image neural network 600. D and G are used to represent the discriminator and generator respectively (the generator refers to the image-to-image neural network 600). For the discriminator D(Y; θ D ), the ground truth label Y gt Assigned 1, predicted label Y pred =G(X;θ G ) is assigned to 0, where X is the set of input training images; θ D and θ G Represent the parameters (weights) of the discriminator and the generator, respectively, which will be learned and adjusted during training. In order to train the adversarial network, the following loss function l is used for the discriminator D D :

[0140]

[0141] Where x represents the input training image, y represents the ground truth rib segmentation mask, y′=G(x; θ G ) represents the prediction generated by the generator G for a given input training image x; E represents the maximum likelihood estimate. The loss function l D The first component of is related to the positive classification of the ground truth samples by the discriminator D, and the loss function l D The second component of is related to the negative classification of the predictions made by the discriminator D to the generator G given the input training images. The parameters θ of the discriminator D are adjusted by learning D To minimize the loss function l D, maximizing the probability that the discriminator D will positively classify the ground truth samples and maximizing the probability that the discriminator D will negatively classify the predictions generated by the generator G (positive classification means being classified as real, and negative classification means being classified as prediction). During the training process of the discriminator network D, the loss l D The gradient of is propagated back to update the parameters θ of the discriminator network D D .

[0142] When adjusting the parameters θ of the discriminator D D After that, by adjusting the parameter θ G To train the generator G to minimize the following loss function l G :

[0143]

[0144] Among them, the loss function l G The first component is the segmentation loss l seg , represents the binary cross entropy between the predicted and ground truth labels (i.e., the predicted rib segmentation mask and the ground truth rib segmentation mask) associated with a given input training image. For example, the segmentation loss l seg The total loss l can be used as total (the weighted sum of the loss terms of all output layers). The loss function l G The second component of is related to the probability score calculated by the discriminator D based on the predictions produced by the generator G (the probability score represents the probability that the discriminator D judges that the image of the generator is real), so by minimizing the second component, the discriminator D is made to confuse the ground truth with the predictions produced by the generator G. Minimize the loss function l G This reduces the segmentation loss and increases the probability that the discriminator D will classify the prediction of the generator G as negative.

[0145] In another example, the following loss function l is provided G :

[0146]

[0147] Formula (3) replaces -log(1-D(G(x))) in formula (2) with log(D(G(x))); therefore, during training, the probability that the predicted label is negatively classified by the discriminator D is not minimized, but the probability that the ground truth label is positively classified by the discriminator D is maximized. This replacement provides a strong gradient in the process of training the generator G, which can effectively speed up the training process. In formulas (2,3), λ is the weight coefficient that balances the two parts of the loss.

[0148] In the adversarial training algorithm of the generator G and the discriminator D, the input has weight θ0G The generator (which can be a pre-trained generator) outputs the updated generator weight θ1 G The algorithm performs multiple training iterations on the training of the discriminator D and the training of the generator G. When training the discriminator D, a small batch of training images x~p data Sampling is performed using the current generator weight G(x;θ0 G ) Generates a prediction value y for each training image x pred , and learn the discriminator weights θ D , the stochastic gradient Propagate back to the discriminator weights θ D Update so that the loss function of the discriminator l D The above operation of training the discriminator D is repeated k times D times, and then the algorithm enters the training of the generator G. When the generator G is trained, a small batch of training images x'~p data is a sample, using the current generator weight G(x;θ0 G ) Generate a predicted value y' for each training image x' pred (x';θ0 G ), the discriminator D calculates a probability score D(G(x′)) for each prediction, and the stochastic gradient Propagate back to update the generator weight θ1 G , so that the generator loss function l G Minimize. The generator G is trained to repeat the above operation k times G times. The discriminator D training and the generator G training are alternately performed for multiple iterations (e.g., until the discriminator weights and the generator weights converge or a predetermined maximum number of iterations is reached), so that the discriminator D cannot easily distinguish between the ground truth label map (the ground truth rib segmentation mask) and the prediction of the image-to-image neural network output (the predicted rib segmentation mask). The algorithm outputs the final updated generator weights. After the training process, the adversarial network is no longer required, and for the newly input three-dimensional medical image to be detected, the trained generator (i.e., the image-to-image neural network) itself can be used for rib segmentation. Figure 8 An example image after the ribs are segmented in step 204 is shown. Each rib can be displayed in a different color. Figure 8 The point 802 where the middle dashed lines intersect schematically indicates that a rib 801 can be selected for marking.

[0149] In step 205, the center line of each rib is extracted according to the multi-label rib segmentation result obtained in step 204 (if necessary, the intermediate information obtained in the above-mentioned other steps can be used in combination). For example, the Rivulet algorithm can be used to achieve this. Fig. 9An example image after the center lines are extracted is shown, where each center line 902 is superimposed on a corresponding rib 901 .

[0150] In the Rivulet algorithm, based on the fast marching and iterative backtracking method, it is possible to track discontinuous areas without being interrupted by densely distributed noise points. Among them, the Hessian matrix-based measurement method is used to enhance the segmentation of linear or tubular structure targets (the target in this case refers to the rib centerline), and the fast marching method is performed based on the weighted distance map obtained by the gray-weighted distance transform (GWDT); in this case, the multi-stencils fast marching method (MSFM) is performed to obtain 3D distance estimates with subvoxel accuracy; subvoxel accuracy helps to generate a smooth centerline curve.

[0151] like Fig.10 As shown, the implementation process of the Rivulet algorithm includes the following steps:

[0152] Step 1001, filtering out image noise and determining a foreground in the filtered image;

[0153] Step 1002, performing GWDT transformation on the segmented foreground to obtain a weighted distance map;

[0154] Step 1003, obtaining a 3D distance estimation value based on the MSFM method;

[0155] Step 1004, based on the RK4 gradient descent method, track the current longest branch;

[0156] Step 1005, determine whether to stop tracing the current branch;

[0157] Step 1006, determining whether the tracking reaches the specified position, if it reaches the specified position, stop tracking, if not, continue tracking;

[0158] Step 1007, determine whether the stop condition is met, if the condition is met, stop tracking, if not, continue tracking;

[0159] Step 1008: Whether the confidence score of the currently tracked branch is higher than a threshold; branches with a confidence score greater than or equal to the threshold are retained, and branches with a confidence score less than the threshold are removed;

[0160] Step 1009, extracting the center line according to the retained branches.

[0161] During the preprocessing process of step 1001, nonlinear anisotropic filtering can be used to enhance the signal, filter out image noise, and mark all voxels with values ​​greater than a set threshold as foreground (including the center line of the ribs) in the filtered image.

[0162] In step 1002, the GWDT transformation originally applied to the APP2 method (All-path pruning 2.0) is applied to the segmented foreground. In the obtained weighted distance map D(u), the foreground voxels far from the foreground boundary are brighter than the foreground voxels close to the foreground boundary. The background voxels in the weighted distance map are set to 10 -10 Instead of 0, the fast marching method (FM) allows jumping between different foreground boundaries; the velocity image is F = (D(u) / D max ) 4 , where D max is the maximum value of D(u), which is used as the source point p in the multi-template fast marching method s and determine a specified position where the gradient descent stops (the speed image is a weighted image used in GWDT to define the distance factor between adjacent voxel points, thereby affecting the speed and effect of the distance transformation). For example, in step 1003, the multi-template fast marching process can be performed only once, and each voxel in the image is tracked at most once by excluding the voxels covered by the tracking branch from the starting point selection of subsequent iterations.

[0163] In step 1004, the RK4 (fourth-order Runge-Kutta) gradient descent method is used to trace back from the outermost region to the source point p s To track each branch, that is, each backtracking starts from the farthest position in the remaining foreground, and iteratively tracks the current longest branch, which provides a greater chance of finding long branches in early iterations, and these branches are less likely to be affected by noise points; when step 1005 determines whether to stop tracking, as long as the judgment result of step 1006 or step 1007 is satisfied, the tracking can be stopped; in other words, the gradient descent method stops when reaching the specified position (such as the source point p s ) or when the following set of conditions are met: (1) more than G1 consecutive points are tracked without encountering a foreground voxel (G1 is the gap threshold, which is determined by the maximum possible gap area allowed to jump and can be adjusted according to different data sets); (2) the Euclidean distance D1 (p n ,p s ) is smaller than the voxel size; (3) The point p tracked in the nth step n Outside the image boundary; (4) Tracking does not leave the current position within a set number of steps (such as 15 steps); (5) Corresponding to p n The arrival time T(p n)≤0. According to the above conditions, the backtracking of the new branch will stop when it is in the area covered by the existing branch.

[0164] In step 1008, a confidence score is set for each tracked branch to measure the proportion of foreground voxels encountered by the tracked branch; each branch {p1, p2, p3, ...., p n} has a length of l, defining a percentile To measure the overall credibility of the tracking process, where B i is the i-th node p i The voxel value of the segmented image is {0,1}, and the value of C can represent the proportion of tracking nodes generated on the foreground voxel; when the tracked branch has a high confidence score, it is retained in the tracking result to avoid the risk of over-tracking (thereby eliminating the tracking error branches caused by noise or gaps). In step 1009, all the retained branches are used as masks for segmenting the rib centerline to extract the rib centerline in the medical image.

[0165] In step 206, according to the multi-label rib segmentation result obtained in step 204 and the center line of each rib extracted in step 205 (if necessary, the intermediate information obtained from the above-mentioned other steps can be used in combination), each rib is subdivided into three regions (i.e., anterior region, lateral region, and posterior region), and the result is stored in the multi-label segmentation data; wherein each region of each rib has a corresponding label. According to the anatomical significance, the chest is divided into the anterior midline, clavicle midline, scapular line, and posterior midline: the "anterior region" of the unilateral rib corresponds to the part from the anterior midline to the clavicle midline, the "lateral region" corresponds to the part from the clavicle midline to the scapular line, and the "posterior region" corresponds to the part from the scapular line to the posterior midline.

[0166] In step 207, based on the multi-label rib segmentation result obtained in step 204 and the three-dimensional medical image received in step 201 and the converted data or image data extracted therefrom, the rib fracture is segmented by a second deep neural network. Fig.11 As shown, the second deep neural network of the example is a second image-to-image neural network 600 ′ (DI2IN), which has the same backbone network for feature extraction as the first image-to-image neural network 600 used in step 204 .

[0167] The difference between the two networks 600 and 600' is that the input channels and loss weights are adjusted in the second image-to-image neural network 600' described in step 207. The input channels of the second image-to-image neural network 600' are adjusted to support the reception of two sets of three-dimensional images, one set of inputs 601' is the three-dimensional medical image received in step 201, or the same image as the input 601 of the first image-to-image neural network 600 (such as a cropped image), and the other set of inputs is the rib segmentation mask obtained in step 204 (i.e., the output 602 of the first image-to-image neural network 600).

[0168] When adjusting the loss weight of the second image-to-image neural network 600', the rib segmentation mask of the binary classification in step 204 (wherein the value corresponding to the rib is 1 and the value corresponding to the background is 0) is used and multiplied pixel by pixel with the loss map (the value of each pixel in the loss map represents the size of the loss value at the corresponding position), so that the corresponding loss value of the area outside the rib is 0 (i.e., the loss of the area outside the rib is ignored), so that the deep learning model's learning process of fractures focuses on the area where the ribs are located, so that the residuals generated by false positives in the area outside the ribs will not affect the network, avoiding the waste of training resources for suppressing false positives, and improving the training efficiency.

[0169] exist Fig.12 In the example of FIG. 1 , the training phase 1202 of the second image-to-image neural network 600′ comprises the following steps:

[0170] Step 1221, receiving training samples;

[0171] Step 1222, the second image-to-image neural network 600' is pre-trained separately;

[0172] Step 1223: The second image-to-image neural network 600' is subjected to adversarial training together with the discriminator.

[0173] Afterwards, in the reasoning stage 1203, the following steps are further included:

[0174] Step 1234, receiving a three-dimensional medical image of the patient (either an original image or an image cropped around a rib region of interest);

[0175] Step 1235: receiving the rib segmentation mask outputted in step 1233 by the trained first image-to-image neural network 600, adjusting the loss weight of the trained second image-to-image neural network 600, and using the adjusted second image-to-image neural network 600' to segment the rib fracture in the three-dimensional medical image;

[0176] Step 1236: output the fracture segmentation mask.

[0177] The training samples of the second image-to-image neural network 600' include a set of training images, and their respective corresponding ground-truth fracture segmentation masks, and ground-truth rib segmentation masks (used to adjust the loss weight so that the loss value of the area outside the ribs is 0). During training, the second image-to-image neural network 600' is first pre-trained separately to minimize the loss between the ground-truth fracture segmentation mask and the predicted fracture segmentation mask generated by the second image-to-image neural network 600' (pre-training can be omitted in some examples).

[0178] When the second image-to-image neural network 600' is used as a generator and adversarially trained together with the discriminator, the discriminator is used to distinguish the ground truth fracture segmentation mask inputted thereto and the predicted fracture segmentation mask generated by the second image-to-image neural network 600'. When the discriminator is trained, the current generator weight is used to generate a prediction value for each corresponding image in the first batch of training images, the weight of the discriminator is learned, and the gradient of the loss function of the discriminator is propagated back to update the weight of the discriminator to minimize the loss function of the discriminator; after the discriminator is trained several times, the training of the generator is entered. When the generator is trained, the current generator weight is used to generate a prediction value for each corresponding image in the second batch of training images, the discriminator calculates a probability score for each prediction value, learns the weight of the generator, and propagates the gradient of the loss function of the generator back to update the weight of the generator to minimize the loss function of the generator; after the generator is trained several times, the discriminator is trained. The training of the generator and the training of the discriminator are alternately iterated for multiple times until the discriminator weight and the generator weight converge or reach the predetermined maximum number of iterations, and the final updated weight of the generator is output.

[0179] That is, through the training stage 1202, the loss weight of the second depth image to image 600' for the rib area is updated (during the training, the ground truth rib segmentation mask is multiplied pixel by pixel with the loss map, so that the loss value of the area outside the rib is 0). Afterwards, in the inference stage 1203, the trained second image to image neural network 600' adjusts the loss weight of the second image to image neural network 600' according to the rib segmentation mask output by the trained first image to image neural network in step 1233, and then segments the rib fractures existing in the three-dimensional medical image of the patient received by the second image to image neural network 600', and outputs a probability map (indicating the probability of whether a voxel belongs to a rib fracture), or outputs a fracture segmentation mask 606 converted from the probability map.

[0180] In step 208, each fracture is characterized according to the rib fracture segmentation result obtained in step 207 and the three-dimensional medical image received in step 201 (if necessary, other data obtained in step 201 or intermediate information obtained in the above-mentioned other steps can be used in conjunction). Each connected domain in the rib fracture segmentation result is defined as a region of interest (FxROI) corresponding to the fracture. In some examples, the detected FxROI area can be superimposed on the relevant image as a lesion site on the rib for visualization.

[0181] In step 208, the fracture type can be inferred using a third deep neural network based on the image corresponding to the FxROI region. The third deep neural network is exemplified as a residual neural network (ResNet) or a vision transformer (ViT), but is not limited thereto.

[0182] Among them, the residual neural network is a deep convolutional neural network. The residual block is the basic unit of the neural network. In addition to the normal convolution layer output on the main path, it also includes a shortcut connection branch to achieve simple identity mapping. For example, one or more convolution layers are set on the main path (the ReLU activation function can be set after the convolution layer). The input of the residual block can be directly connected to the output through the branch (the input and the convolution output are added and then output), so that the subsequent layers can directly learn the residual. After the introduction of residual learning, the deep learning network is easier to optimize, the accuracy can be improved by increasing the depth, and the gradient vanishing problem caused by the increase in depth can be alleviated.

[0183] In the visual transformer, the input image is divided into multiple slices (patches) of fixed size to obtain a sequence of slices (the length of the sequence corresponds to the number of slices); each slice in the sequence is flattened (Flatten) through a trainable linear mapping layer and converted into a vector (token), which can be implemented by a convolutional layer; the sequence obtained by concatenating these vectors with a classification vector (CLS, class token) is added to the position code (which has trainable parameters) and input into the encoder of the Transformer model for feature extraction (the Transformer model is an attention-based encoder-decoder architecture); the encoder includes multiple repeatedly stacked encoding blocks, which may include an LN layer (Layer Norm, normalization), a multi-head attention module (Multi-Head Attention), Dropout or DropPath layer (to prevent model overfitting), the multi-head attention module, in which multiple heads perform QKV processing under the attention mechanism in parallel (Q stands for query, which refers to the information to be queried; K stands for key, which refers to the queried vector; V stands for value, which refers to the value obtained by the query) to focus on different aspects of information, and then sum the weights according to the decision results of multiple heads; the residual passes through the LN layer, the multi-layer perceptron (MLP Block), the Dropout or DropPath layer and then the residual. The multi-layer perceptron is provided with a fully connected layer, a GELU activation function, a Dropout layer, a fully connected layer, and a Dropout layer. After the encoder, the MLP Head layer is used to extract the corresponding results generated by the CLS to classify the output (the MLP Head may include one LN layer, or two LN layers and a GELU activation function between them).

[0184] Therefore, in step 208, the fracture type is inferred by the third deep neural network based on the initial three-dimensional medical image (such as obtained in step 201), the rib segmentation result (obtained in step 204), and the corresponding fracture segmentation result in each FxROI (obtained in steps 207 and 208). The exemplary fracture types can be classified according to whether the fracture is connected to the outside world (such as open or closed fracture), according to the degree of fracture (such as complete or incomplete fracture), according to the stability of the fracture end (stable or unstable fracture), according to the fracture morphology (such as comminuted fracture, compression fracture, depressed fracture, crack fracture, etc.), according to whether the bone tissue is normal before fracture (such as traumatic fracture, pathological fracture); according to the time after fracture (such as fresh fracture, old fracture), and so on.

[0185] In step 209, the relative position of each fracture on the relevant rib is calculated based on the region of interest (FxROI) corresponding to the fracture obtained in step 208 and the subdivided regions of each rib obtained in step 206 (if necessary, the intermediate information obtained in the above-mentioned other steps can be used in combination). For example, the center point of the region of interest corresponding to each fracture is calculated, the point on the rib closest to the center point is determined, and the relative position of the rib where the closest point is located is recorded as the relative position of the fracture. The relative position of the rib is an anatomical description of the direction, position, etc. of the rib relative to the adjacent structure.

[0186] In step 209, for each finding (i.e., each connected domain in the fracture segmentation mask), the affected ribs can be determined according to the intersection of the fracture contour corresponding to the connected domain and the rib segmentation mask, including the rib side, rib number and subdivided area related to the fracture; wherein, "side" is used to distinguish the left or right side of the body, that is, whether it is the left rib or the right rib, which can be represented by L or R; "number" represents the order of the ribs, the left ribs are L1 to L12 from top to bottom, and the right ribs are R1 to R12 from top to bottom ( Fig.13 );“Subdivided area” refers to the front area, side area and back area of ​​the current rib.

[0187] In step 210, when generating a report related to rib fracture information, the rib number, side, subdivision area and fracture type associated with each fracture can be pre-filled into the radiology report and structured finding list according to the processing results of steps 208 to 209. The information obtained by step 210 can be sent to the storage or database of various medical data archiving systems for storage, such as RIS system, PACS system, EMR system, etc.; or, the relevant information can be sent to a viewer supporting DICOM format for immediate viewing, or sent to a radiology workstation for viewing or subsequent processing. For example, the detected fracture types and relative positions can be listed in the generated report; for example, the information recorded for the first fracture includes "displaced fracture, in the front area of ​​the fourth rib on the left"; the information recorded for the second fracture includes "old fracture, in the back area of ​​the fifth rib on the left".

[0188] In step 211, the segmentation mask obtained in steps 204, 205, and 207 is used for advanced rendering (such as by dyeing, increasing brightness, or other changes), and the contours corresponding to each rib, the center line of each rib, the contours or boundary lines of each rib fracture, etc. are superimposed on the relevant image for visualization, so that when viewing the image, the clinician can quickly identify and locate the part of the image where the fracture or lesion occurs. For example, Fig.13 An example is that on the rib unfolding diagram, the outlines of the fracture crack 1301 are superimposed on the ribs L3 to L9 respectively; for another example, Fig.14 An example is that in the three-dimensional reconstruction image of the chest cavity, two lesions 1401 and 1402 on the ribs are displayed (the lesions presented are, for example, obtained by rendering the region of interest corresponding to the fracture, and are presented in a different color from the ribs). The relevant image obtained in step 211 with the rendering effect of the rib fracture can be sent to the storage or database of various medical data archiving systems for storage, such as RIS systems, PACS systems, EMR systems, etc.; or, it can also be sent to the radiology workstation for viewing or subsequent processing, or sent to a viewer supporting the DICOM format for immediate viewing.

[0189] like Fig.15 As shown, a rib fracture detection system 1500 provided in one embodiment includes the following modules: an image receiving module 1504, a marker detection module 1505, a rib ROI definition module 1506, a rib detection module 1507, a rib centerline extraction module 1508, a fracture detection module 1509, a fracture characterization module 1510, a fracture positioning module 1511, a visualization presentation module 1512, and an information reporting module 1513. In one example, the detection system 1500 can be used to perform the above Figure 2 Method for detecting rib fractures is shown.

[0190] The image receiving module 1504 can receive three-dimensional medical images from an image acquisition device 1501 or a medical data archiving system 1502 to which the signal is connected (the example image acquisition device 1501 includes a CT scanner 15011 or an MRI scanner 15012; the example medical data archiving system 1502 includes a PACS system 15021, a RIS system 15022, and an EMR system 15023); the image receiving module 1504 is also used to convert the received three-dimensional medical image (for example, in DICOM format) into a data format suitable for the detection system 1500.

[0191] The marker detection module 1505 is signal-connected to the image receiving module 1504 to acquire a three-dimensional medical image and data related to the image, and is used to analyze the anatomical structure in the three-dimensional medical image through a marker detection algorithm.

[0192] The rib ROI definition module 1506 is signal-connected to the marker detection module 1505, and is used to define a region of interest (ROI) corresponding to the rib according to the analyzed anatomical structure, and to crop the image around the region.

[0193] The rib detection module 1507 is signal-connected to the rib ROI definition module 1506 to receive the cropped image, uses a first deep neural network to segment and mark the ribs in the image through multi-label binary classification, and outputs a rib segmentation mask.

[0194] The rib centerline extraction module 1508 is connected to the rib detection module 1507 by signal to receive the results of rib segmentation and labeling, and is used to extract the centerline of each rib. The rib detection module 1507 sets labels for all ribs and also sets labels for individual ribs; and in combination with the results of centerline extraction, the rib centerline extraction module 1508 can also set corresponding labels for the three subdivided areas (i.e., the front area, the side area, and the back area) of each rib, and associate them with the label of the rib, and store them together in the multi-label segmentation data.

[0195] The fracture detection module 1509 is signal-connected to the image receiving module 1504 to obtain the received three-dimensional medical image, and is signal-connected to the rib detection module 1507 to obtain the rib segmentation mask, segments the rib fracture through the second deep neural network, and outputs the fracture segmentation mask.

[0196] The fracture characterization module 1510 is signal-connected to the rib detection module 1507 and the fracture detection module 1509 respectively to obtain the results of rib and rib fracture segmentation for characterizing each fracture; wherein, for each connected domain in the fracture segmentation result, a region of interest (FxROI) corresponding to the fracture is defined, and a third deep neural network is used to infer the fracture type.

[0197] The fracture positioning module 1511 is signal-connected with the rib centerline extraction module 1508 and the fracture detection module 1509 to obtain the extracted rib centerline and fracture segmentation results, which are used to calculate the relative position of each fracture in the relevant ribs.

[0198] The visualization presentation module 1512 is connected to the rib detection module 1507, the rib centerline extraction module 1508, and the fracture detection module 1509 by signal to obtain the results of their respective outputs, which are used to form visualization results related to ribs and rib fractures (such as rendering the outline of each rib, the centerline of each rib, the outline or boundary line of each rib fracture, etc.), and superimposed on the relevant image (such as the original received three-dimensional medical image, the rib expansion diagram, the three-dimensional reconstruction diagram of the rib area or the chest, etc.). The visualization presentation module 1512 also sends the relevant image superimposed with the visualization result to the post-processing device 1503 connected to the signal, so that the clinician can view or post-process the relevant image, the visualization result related to the ribs and rib fractures, etc. (the example image processing device 1503 includes a radiology workstation 15031, a viewer 15032 supporting the DICOM format, etc.). The visualization presentation module 1512 also sends the relevant image superimposed with the visualization result to the memory or database of the medical data archiving system 1502 connected to the signal for storage.

[0199] The information reporting module 1513 is connected to the fracture characterization module 1510 and the fracture location module 1511 respectively by signals to obtain information related to rib fractures (for example, the rib number associated with each fracture, the side, the subdivided area and the fracture type, etc.), which is pre-filled into the radiology report and the structured search list by the information reporting module 1513. The information reporting module 1513 can send the report, etc. to the post-processing device 1503 connected to the signal so that the clinician can view or perform post-processing. The information reporting module 1513 can also send the relevant report to the memory or database of the medical data archiving system 1502 connected to the signal for storage.

[0200] The systems and / or methods described herein can be implemented using computers operating in a client-server relationship, wherein the client computer is remotely located from the server computer and interacts via a network. The client-server relationship can be defined and controlled by computer programs running on the respective client and server computers.

[0201] The systems and / or methods described herein may be implemented in a network-based cloud computing system. In such a network-based cloud computing system, a server connected to the network communicates with one or more client computers via the network. The client computer may communicate with the server via, for example, a web browser application that resides on and operates on the client computer. The client computer may store data on the server and access the data via the network. The client computer may transmit a request for data, or a request for an online service, to the server via the network. The server may perform the requested service and provide the data to (multiple) client computers. The server may also transmit data adapted to cause the client computer to perform a specified function, such as performing a calculation, displaying specified data on a screen, and the like. Certain steps of the methods and workflows described herein (including Figure 1 , Figure 2 , Figure 5 , Fig.10 , Fig.12 One or more of the steps or functions of the method may be performed by a server or by other computers / processors in a network-based cloud computing system. Certain steps of the above method and certain functions of the above system may be performed locally by a client computer in a network-based cloud computing system. The steps of the method for rib fracture detection and / or the method for training a deep neural network described herein may be implemented in a network-based cloud computing system and may be performed by the network-based cloud computing system or by a local client computer in any combination.

[0202] The systems and / or methods described herein may be implemented using a computer program product tangibly embodied in an information carrier (e.g., in a non-transitory machine-readable storage device) for execution by a programmable processor, and the methods and workflow steps described herein (including Figure 1 , Figure 2 , Figure 5 , Fig.10 , Fig.12 One or more of the steps or functions of a computer program may be implemented using one or more computer programs executable by such a processor. A computer program is a set of computer program instructions that can be used directly or indirectly in a computer to perform a certain activity or cause a certain result. A computer program may be written in any form of programming language, including compiled or interpreted languages, and it may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0203] The systems and / or methods described herein may be implemented using digital circuits, or using one or more computers using well-known computer processors, memory units, storage devices, computer software, and other components. Typically, a computer includes a processor for executing instructions and one or more memories for storing instructions and data. The computer may also include or be coupled to one or more mass storage devices, such as one or more magnetic disks, internal hard disks and removable disks, magneto-optical disks, optical disks, and the like.

[0204] Fig.16 A high-level block diagram of such a computer 1600 is provided in . The computer 1600 includes a processor 1601 operatively coupled to a storage device 1604 and a memory 1602. The processor 1601 controls the overall operation of the computer 1600 by executing computer program instructions that define such operations. The computer program instructions may be stored in the storage device 1604 or other computer-readable medium and loaded into the memory 1602 when execution of the computer program instructions is desired. Thus, for example Figure 1 , Figure 2 , Figure 5 , Fig.10 , Fig.12 The methods and workflow steps or functions may be defined by computer program instructions stored in the memory 1602 and / or storage device 1604 and controlled by the processor 1601 executing the computer program instructions.

[0205] In some examples, the image acquisition device 1501 or the medical data archiving system 1502 ( Fig.15), can be connected to the computer 1600 to input the three-dimensional medical image to the computer 1600. In some examples, the computer 1600 can send the output image or data to the medical data archiving system 1502 for archiving. In some examples, the computer 1600 can send the output image or data to the post-processing device 1503 connected thereto. In some examples, one or more components of the computer 1600 can be integrated into one device with the image acquisition device 1501 or the medical data archiving system 1502 or the post-processing device 1503. In some examples, the computer 1600 can be connected to the image acquisition device 1501 or the medical data archiving system 1502 or the post-processing device 1503 through a network or wireless communication protocol. In some examples, the computer 1600 can be remotely located relative to the image acquisition device 1501 or the medical data archiving system 1502 or the post-processing device 1503, and the methods and workflow steps described herein can be performed as part of a server or cloud-based service. In this case, the method steps can be performed on a single computer or distributed among multiple networked computers. The computer 1600 also includes one or more network interfaces 1605 for communicating with other devices via a network.

[0206] The processor 1601 may include both general and special purpose microprocessors, and may be a single processor or one of multiple processors of the computer 1600. The processor 1601 may include, for example, one or more central processing units (CPUs). The processor 1601, storage device 1604, and / or memory 1602 may include, be supplemented by, or be incorporated in one or more application specific integrated circuits (ASICs) and / or one or more field programmable gate arrays (FPGAs).

[0207] The storage device 1604 and the memory 1602 each include a tangible non-transitory computer-readable storage medium. The storage device 1604 and the memory 1602 may each include a high-speed random access memory, such as a dynamic random access memory (DRAM), a static random access memory (SRAM), a double data rate synchronous dynamic random access memory (DDR RAM), or other random access solid-state memory devices, and may include non-volatile memory, such as one or more magnetic disk storage devices, such as internal hard disks and removable disks, magneto-optical disk storage devices, optical disk storage devices, flash memory devices, semiconductor memory devices, such as erasable programmable read-only memory (EEPROM), electrically erasable programmable read-only memory (EEPROM), compact disk read-only memory (CD-ROM), digital versatile disk read-only memory (DVD-ROM) disks, or other non-volatile solid-state storage devices.

[0208] The computer 1600 may also include one or more input / output devices 1603 that enable user interaction with the computer 1600 (e.g., a display, keyboard, mouse, speakers, buttons, etc.). The input / output devices 1603 may include peripheral devices such as printers, scanners, display screens, etc. For example, the input / output devices 1603 may include a display device (such as a cathode ray tube (CRT) or liquid crystal display (LCD) monitor) for displaying information to a user, a keyboard, and a pointing device (such as a mouse or trackball) through which a user can provide input to the computer 1600. In some examples, the input / output devices 1603 may be used as an annotation tool for annotating a volume received by the computer 1600 in conjunction with a computer program set.

[0209] The system described in this article (including Fig.15 The detection system of the present invention can be implemented by using one or more computers (such as computer 1600). Those skilled in the art will recognize that the actual computer or computer system may have other structures and may include other components. Fig.16 There are some high-level representations of the components of such a computer, for illustrative purposes.

[0210] Although the content of the present invention has been described in detail through the above preferred embodiments, it should be appreciated that the above description should not be considered as a limitation of the present invention. After reading the above content, it will be apparent to those skilled in the art that various modifications and substitutions of the present invention will occur. Therefore, the protection scope of the present invention should be limited by the appended claims.

Claims

1. A method for detecting rib fractures in medical images, characterized in that: include: receiving medical images; parsing the anatomical structure in the medical image using a marker detection algorithm to determine a rib region in the medical image; Using a first deep neural network, segmenting ribs in a rib region of the medical image; The rib fracture area in the medical image is segmented according to the result of the rib segmentation using a second deep neural network.

2. The method according to claim 1, characterized in that Using the marker detection algorithm, analyzing the anatomical structure on the medical image to obtain the position information of the anatomical landmark; When using the marker detection algorithm, including: An artificial intelligence agent trained to execute a set of actions for moving a state space toward an anatomical landmark in a medical image and maximizing a cumulative future reward value based on an optimal action-value function approximator; The state space is defined as candidate locations of anatomical landmarks and fixed regions around the candidate locations; The set of actions performed is selected from a predetermined action set; the actions in the action set include: one or more parameter actions for changing the focus position of the state space, and one or more scale scaling actions for changing the resolution of the state space; a corresponding reward value is set for each action in the action set according to the proximity between the state space and the anatomical landmark after the action is performed; The set of actions performed includes at least one scaling action.

3. The method according to claim 2, characterized in that The set of actions performed by the trained artificial intelligence agent includes multiple scale scaling actions, corresponding to multiple scales that gradually increase the resolution of the state space; the convergence point when searching for anatomical landmarks at each scale is used as the starting position for searching at the next scale; at the scale with the lowest resolution, the search starts from an arbitrary position or a specified position.

4. The method according to claim 1, characterized in that The first deep neural network is a first image-to-image neural network; the trained first image-to-image neural network generates a rib segmentation mask for segmenting ribs based on an image corresponding to a rib region input thereto.

5. The method according to claim 4, characterized in that The training of the first image-to-image neural network comprises performing adversarial training on the first image-to-image neural network as a first generator together with a first discriminator; the first discriminator is used to distinguish between a predicted rib segmentation mask generated by the first generator based on the training image and a ground truth rib segmentation mask; The first segmentation loss is reduced by learning loss weights of the first generator and the first discriminator to minimize a first loss function of the first image-to-image neural network, thereby increasing the probability that the predicted rib segmentation mask is negatively classified by the first discriminator and increasing the probability that the ground truth rib segmentation mask is positively classified by the first discriminator; the first segmentation loss is a binary cross entropy between the predicted rib segmentation mask and the ground truth rib segmentation mask.

6. The method according to claim 4, characterized in that The second deep neural network is a second image-to-image neural network; the trained second image-to-image neural network inputs the received medical image and the rib segmentation mask obtained by segmenting the ribs in the rib region of the medical image by the first deep neural network; According to the rib segmentation mask, the loss weight of the second image-to-image neural network with a second loss function is adjusted so that the loss value outside the rib area is 0; and the image-to-image neural network after the loss weight adjustment is used to generate a fracture segmentation mask for segmenting rib fractures based on the received medical image.

7. The method according to claim 6, characterized in that The training of the second image-to-image neural network comprises adversarially training the second image-to-image neural network as a second generator together with a second discriminator; the second discriminator is used to distinguish between a predicted fracture segmentation mask generated by the second generator based on the training image and a ground truth fracture segmentation mask; During training, the loss weight of the second loss function of the second image-to-image neural network is adjusted using the ground truth rib segmentation mask so that the loss value outside the rib region is 0; By learning the loss weights of the second generator and the second discriminator to minimize the second loss function, the second segmentation loss is reduced, the probability of the predicted fracture segmentation mask being negatively classified by the second discriminator is increased, and the probability of the ground truth fracture segmentation mask being positively classified by the second discriminator is increased; the second segmentation loss is the binary cross entropy between the predicted fracture segmentation mask generated by the second image-to-image neural network based on the training image and the ground truth fracture segmentation mask.

8. The method according to claim 6, characterized in that The first image-to-image neural network and the second image-to-image neural network have the same backbone network for extracting features; the backbone network has a convolutional encoder-decoder architecture, including a plurality of encoders, a plurality of decoders, and a plurality of branches connecting at least a portion of the decoders to corresponding output layers; The size of the output channel of each branch matches the size of the input image of the image-to-image neural network; the features of the output layers corresponding to all branches are concatenated and input into a final output layer; the segmentation loss of the image-to-image neural network corresponds to the weighted loss terms of the output layers of all branches and the final output layer.

9. The method according to claim 1, characterized in that The method further comprises: Using a rib segmentation mask on the medical image to obtain a rib image, using an algorithm for tracing the rib centerline to track the centerline of each rib in the rib image, and generating a centerline segmentation mask for extracting the rib centerline; When using the algorithm for tracing the rib centerline, including: Image noise is filtered out by anisotropic filtering, and voxels with values ​​greater than a set threshold in the rib image are determined as foreground; The grayscale weighted distance transform method is used on the segmented foreground to obtain a weighted distance map; the point corresponding to the maximum value in the weighted distance map is used as the source point for the fast marching method to calculate the 3D distance estimate; Use the RK4 gradient descent method to iteratively trace the current longest branch. Each trace starts from the farthest position in the current remaining foreground and stops when it reaches the source or the previously traced branch. Calculate a confidence score for each tracked branch, wherein the confidence score represents the proportion of the current tracked branch encountering the foreground voxel; retain the tracked branches whose confidence scores are higher than a set threshold; The centerline segmentation mask is generated according to the retained tracked branches, and the centerline segmentation mask is applied to the rib image to extract the centerline of each rib.

10. The method according to claim 1, characterized in that The method further comprises: Each connected domain of the fracture segmentation mask is defined as the fracture area; For each fracture region, a third deep neural network was used to determine the fracture type; Calculate the center point of each fracture area, determine the point on the rib where the fracture is located that is closest to the center point, and use the relative position of the rib of the closest point as the relative position of the fracture; The ribs affected by the fracture are determined based on the intersection of each fracture region with the rib segmentation mask.

11. The method according to claim 1, characterized in that The medical image is a three-dimensional medical image obtained by computer tomography or magnetic resonance imaging.

12. A system for detecting rib fractures in medical images, characterized in that: The system comprises a module for implementing the method for detecting rib fractures in medical images as claimed in any one of claims 1 to 11; the module comprises: A module for receiving a medical image; A module for parsing anatomical structures in medical images using a landmark detection algorithm to determine rib regions; a module for segmenting ribs in the rib region of a medical image by means of a first deep neural network; Module for segmenting rib fracture regions in medical images using a second deep neural network.

13. A computer-readable storage medium storing computer program instructions, characterized in that: When the computer program instructions are executed by a processor, the method for detecting rib fractures in medical images as described in any one of claims 1 to 11 is implemented.

Images