Loss value determination method, model training method, device and electronic equipment

Abstract

The present application relates to the technical field of blood vessel segmentation, and particularly relates to a loss value determination method, a model training method, a device and electronic equipment.The loss value determination method comprises the following steps: obtaining a labeled image obtained by labeling a lumen and a tube wall based on a target blood vessel image, and extracting the inner and outer contours of the blood vessel in the labeled image to obtain corresponding labeled contours; obtaining a prediction image obtained by segmenting a lumen and a tube wall based on the target blood vessel image, and extracting the inner and outer contours of the blood vessel in the prediction image to obtain corresponding prediction contours; aligning the labeled contours and the prediction contours to obtain aligned labeled contours and prediction contours; extracting contour control points in the aligned labeled contours and the aligned prediction contours respectively; and determining a loss value between the labeled image and the prediction image based on the distance between the contour control points in the labeled contours and the prediction contours.

Description

Technical Field

[0001] This invention relates to the field of blood vessel segmentation technology, and in particular to a method for determining loss values, a model training method, an apparatus, and electronic equipment. Background Technology

[0002] Intracranial atherosclerosis is one of the high-risk factors for stroke. The severity of atherosclerosis can be assessed by quantitatively analyzing intracranial blood vessels using three-dimensional high-resolution black vessel wall imaging technology and measuring their morphological parameters (such as normalized wall index, remodeling index, and wall thickness).

[0003] Precise segmentation of the vessel wall is crucial to ensuring the accuracy of quantitative analysis results. Although manual delineation is more accurate, it is too labor-intensive and has low repeatability. To improve delineation efficiency, automated intracranial vessel wall segmentation based on deep learning has gradually become the mainstream, achieving Dice accuracies of 0.89 for the lumen and 0.77 for the vessel wall.

[0004] However, current deep learning network models mostly use Dice loss and cross-entropy loss as loss functions, giving the same weight to pixels in the lumen and walls, while ignoring the unique circular features of the lumen and walls. To better utilize this feature, researchers have proposed new approaches, such as converting the image to polar coordinates or applying the level set concept to the loss function. However, these methods are computationally complex; the former relies on accurate center point extraction, and the latter requires fine-tuning of regularization parameters, affecting the stability and versatility of model training. Summary of the Invention

[0005] In view of the above-mentioned problems in the prior art, the purpose of the present invention is to provide a method for determining loss values, a model training method, an apparatus and an electronic device, which can reduce the computational workload of determining loss values, improve the stability and versatility of model training, and improve the accuracy of the model for segmentation of lumen walls.

[0006] To address the above problems, this invention provides a method for determining the loss value in a pipe wall segmentation model, comprising:

[0007] Obtain an annotated image by annotating the lumen and wall of the target blood vessel image, and extract the inner and outer contours of the blood vessels in the annotated image to obtain the corresponding annotated contours;

[0008] A predicted image is obtained by segmenting the lumen and wall based on the target blood vessel image, and the inner and outer contours of the blood vessels in the predicted image are extracted to obtain the corresponding predicted contours.

[0009] Align the labeled contour and the predicted contour to obtain the aligned labeled contour and predicted contour;

[0010] Extract the contour control points from the aligned labeled contour and the predicted contour, respectively;

[0011] The loss value between the labeled image and the predicted image is determined based on the distance between the contour control points in the labeled contour and the predicted contour.

[0012] Further, the step of extracting the inner and outer contours of blood vessels in the predicted image to obtain the corresponding predicted contours includes:

[0013] The predicted image is preprocessed to obtain a preprocessed predicted image; wherein, the preprocessing is used to correct the segmentation error of the predicted image;

[0014] The inner and outer contours of blood vessels in the preprocessed predicted image are extracted to obtain the corresponding predicted contours.

[0015] Furthermore, the predicted image includes lumen segmentation results and wall segmentation results;

[0016] The step of preprocessing the predicted image to obtain a preprocessed predicted image includes:

[0017] The lumen segmentation result and the wall segmentation result in the predicted image are merged to obtain the merged first image;

[0018] The second image is obtained by taking the largest connected component from the merged first image;

[0019] The intersection of the second image and the predicted image is used to obtain the third image;

[0020] The third image is subjected to hole-filling processing, and the hole-filled image is used as the preprocessed predicted image.

[0021] Further, aligning the labeled contour and the predicted contour to obtain aligned labeled contour and predicted contour includes:

[0022] The predicted contour is corrected in the normal direction so that the normal direction of the predicted contour is consistent with the normal direction of the labeled contour.

[0023] Determine the first target contour point in the labeled contour, and the second target contour point in the corrected predicted contour that corresponds to the first target contour point;

[0024] Based on the first target contour point and the second target contour point, the labeled contour and the predicted contour are aligned to obtain the aligned labeled contour and predicted contour.

[0025] Furthermore, both the labeled contour and the predicted contour include multiple contour points;

[0026] The step of extracting contour control points from the aligned labeled contour and the predicted contour respectively includes:

[0027] Downsampling is performed on multiple contour points in the aligned labeled contour to obtain a preset number of first contour control points in the labeled contour.

[0028] Multiple contour points in the aligned predicted contour are downsampled to obtain a preset number of second contour control points in the predicted contour.

[0029] Furthermore, the labeled contour includes an inner labeled contour and an outer labeled contour, and the predicted contour includes a predicted inner contour and a predicted outer contour;

[0030] Determining the loss value between the labeled image and the predicted image based on the distance between the contour control points in the labeled contour and the predicted contour includes:

[0031] Calculate the sum of the distances between the first contour control point and the second contour control point corresponding to each position in the marked inner contour and the predicted inner contour to obtain the total inner contour distance;

[0032] Calculate the sum of the distances between the first contour control point and the second contour control point corresponding to each position in the marked outer contour and the predicted outer contour to obtain the total outer contour distance;

[0033] The weighted sum of the inner contour distance and the outer contour distance is calculated to obtain the loss value between the labeled image and the predicted image.

[0034] Another aspect of the present invention provides a model training method, comprising:

[0035] A labeled image is obtained by annotating the lumen and wall of a target blood vessel image;

[0036] The target blood vessel image is input into the vessel wall segmentation model for lumen and vessel wall segmentation to obtain the corresponding predicted image.

[0037] Based on the labeled image and the predicted image, the method according to any one of claims 1-6 determines the loss value between the labeled image and the predicted image;

[0038] The model parameters of the pipe wall segmentation model are updated based on the loss value to obtain the updated pipe wall segmentation model.

[0039] Another aspect of the present invention provides an apparatus for determining the loss value in a pipe wall segmentation model, comprising:

[0040] The first acquisition module is used to acquire an annotated image obtained by annotating the lumen and wall of the target blood vessel image, and to extract the inner and outer contours of the blood vessels in the annotated image to obtain the corresponding annotated contours.

[0041] The second acquisition module is used to acquire a predicted image obtained by segmenting the lumen and wall based on the target blood vessel image, and to extract the inner and outer contours of the blood vessels in the predicted image to obtain the corresponding predicted contours.

[0042] An alignment module is used to align the labeled contour and the predicted contour to obtain aligned labeled contour and predicted contour.

[0043] The extraction module is used to extract the contour control points from the aligned labeled contour and the predicted contour, respectively.

[0044] The first determining module is used to determine the loss value between the labeled image and the predicted image based on the distance between the contour control points in the labeled contour and the predicted contour.

[0045] Another aspect of the present invention provides a model training apparatus, comprising:

[0046] The third acquisition module is used to acquire the labeled image, which is obtained by labeling the lumen and wall of the target blood vessel image;

[0047] The prediction module is used to input the target blood vessel image into the vessel wall segmentation model for lumen and vessel wall segmentation processing to obtain the corresponding prediction image;

[0048] The second determining module is configured to determine a loss value between the labeled image and the predicted image based on the labeled image and the predicted image, according to the method described in any one of claims 1-6;

[0049] The update module is used to update the model parameters of the pipe wall segmentation model based on the loss value, so as to obtain the updated pipe wall segmentation model.

[0050] In another aspect, the present invention provides an electronic device, including a processor and a memory, wherein the memory stores at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the method for determining the loss value or the model training method in the pipe wall segmentation model as described above.

[0051] In another aspect, the present invention provides a computer-readable storage medium storing at least one instruction or at least one program, wherein the at least one instruction or the at least one program is loaded and executed by a processor to implement the method for determining the loss value or the model training method in the pipe wall segmentation model as described above.

[0052] Due to the above technical solution, the present invention has the following beneficial effects:

[0053] The method for determining the loss value according to embodiments of the present invention extracts contour control points of the lumen and pipe wall in the labeled image and the predicted image by simulating the approach of manual drawing, thereby determining the loss value between the labeled image and the predicted image. This method not only takes into account the unique annular features of the lumen and pipe wall, but also transforms the problem of segmenting the inner and outer boundaries of the lumen and pipe wall into an optimization problem of key contour control points, greatly reducing the computational workload of determining the loss value between the labeled image and the predicted image, and the calculation results are more stable. It can be applied to the training of various types of pipe wall segmentation models and has a wide range of applications.

[0054] Furthermore, applying the aforementioned method for determining the loss value to the training process of the pipe wall segmentation model can improve the consistency between the model's prediction results and the results of manual delineation, enhance the stability and versatility of the model training, and improve the accuracy of the trained pipe wall segmentation model in segmenting the lumen and pipe wall, thereby obtaining more accurate pipe wall segmentation results. Attached Figure Description

[0055] To more clearly illustrate the technical solutions of the present invention, the accompanying drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0056] Figure 1 This is a flowchart of a method for determining the loss value in a pipe wall segmentation model provided in an embodiment of the present invention;

[0057] Figure 2 This is a schematic diagram of an annotated image provided in one embodiment of the present invention;

[0058] Figure 3 This is a schematic diagram of a normal vector provided in one embodiment of the present invention;

[0059] Figure 4 This is a schematic diagram of a model training method provided in an embodiment of the present invention;

[0060] Figure 5 This is a schematic diagram of the structure of a device for determining the loss value in a pipe wall segmentation model provided in an embodiment of the present invention;

[0061] Figure 6 This is a schematic diagram of the structure of a model training device provided in one embodiment of the present invention;

[0062] Figure 7 This is a schematic diagram of the structure of an electronic device provided in one embodiment of the present invention. Detailed Implementation

[0063] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0064] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0065] The method provided in this invention can be applied to scenarios involving training a vessel wall segmentation model. During training, a labeled image is obtained by annotating the lumen and wall of a target blood vessel image. This labeled image is then input into the vessel wall segmentation model for lumen and wall segmentation, resulting in a corresponding predicted image. Using the method provided in this invention, a loss value is determined between the labeled image and the predicted image. Finally, the model parameters of the vessel wall segmentation model are updated based on the obtained loss value, resulting in an updated vessel wall segmentation model.

[0066] Reference manual attached Figure 1This document illustrates the flowchart of a method for determining the loss value in a pipe wall segmentation model provided by an embodiment of the present invention. This method can be applied to a server, which can be an independent server, a server cluster composed of multiple servers, or a distributed system. It can also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Specifically, as shown... Figure 1 As shown, the method may include the following steps:

[0067] S110: Obtain the labeled image obtained by annotating the lumen and wall of the target blood vessel image, and extract the inner and outer contours of the blood vessels in the labeled image to obtain the corresponding labeled contours.

[0068] In this embodiment of the invention, the target blood vessel image can be a cross-sectional image of various types of blood vessels, such as a cross-sectional image of a blood vessel acquired using magnetic resonance three-dimensional high-resolution black blood vessel wall imaging technology. This embodiment of the invention does not specifically limit the type of blood vessel or the method of acquiring the blood vessel image.

[0069] It should be noted that the source of the target blood vessel image can be directly imported relevant data, or it can be obtained from other resource libraries through real-time configuration and connection, or it can be obtained from a stored image database after searching based on information such as the user's name. This embodiment of the invention does not limit this.

[0070] In this embodiment of the invention, after obtaining the target blood vessel image, the lumen and wall of the blood vessel can be manually drawn to obtain an annotated image. Specifically, refer to the appendix to the specification. Figure 2 This illustrates a schematic diagram of an annotated image provided in an embodiment of the present invention. When manually delineating the lumen and wall of a blood vessel, a series of key control points can be selected sequentially, and then two closed curves, one inner and one outer, can be generated through interpolation. For example... Figure 2 The pixels enclosed by the inner curve shown in (a) are labeled as lumens, as... Figure 2 The pixels between the inner and outer curves shown in (b) are labeled as pipe walls.

[0071] In this embodiment of the invention, existing contour extraction methods can be used to extract the inner and outer contours of blood vessels in the labeled image to obtain the corresponding labeled contours. This embodiment of the invention does not impose specific limitations on the contour extraction method used. Specifically, the labeled contours may include an inner contour (denoted as L-Cin) and an outer contour (denoted as L-Cout). Both the inner and outer contours may include multiple contour points and can be represented by a vector formed by the corresponding multiple contour points. For example, the inner contour can be represented by its corresponding multiple contour points. The resulting vector To express.

[0072] S120: Obtain a predicted image obtained by segmenting the lumen and wall based on the target blood vessel image, and extract the inner and outer contours of the blood vessels in the predicted image to obtain the corresponding predicted contours.

[0073] In this embodiment of the invention, after obtaining the target blood vessel image, the lumen and wall of the blood vessel can be automatically segmented using a deep learning algorithm. For example, the target blood vessel image can be segmented using a wall segmentation model built based on a deep learning algorithm to obtain a corresponding predicted image. The predicted image may include both lumen segmentation results and wall segmentation results.

[0074] It should be noted that the embodiments of the present invention do not impose specific restrictions on the type of deep learning algorithm, nor on the specific structure of the tube wall segmentation model. In practical applications, those skilled in the art can construct the model according to actual needs. For example, the tube wall segmentation model can be a deep learning network model such as nnU-Net (no new-Net, an adaptive medical image segmentation framework based on U-Net).

[0075] In this embodiment of the invention, existing contour extraction methods can be used to extract the inner and outer contours of blood vessels in the predicted image to obtain the corresponding predicted contours. This embodiment of the invention does not impose specific limitations on the contour extraction method used. Specifically, the predicted contours may include a predicted inner contour (denoted as P-Cin) and a predicted outer contour (denoted as P-Cout). Both the predicted inner and outer contours may include multiple contour points and can be represented by a vector formed by the corresponding multiple contour points. For example, the predicted inner contour can be represented by its corresponding multiple contour points. The resulting vector To express.

[0076] In one possible embodiment, for the predicted image, the impact of segmentation error needs to be considered. Therefore, when extracting the inner and outer contours, the segmentation error of the predicted image can be corrected first, and then the inner and outer contours can be extracted from the corrected predicted image. Specifically, extracting the inner and outer contours of blood vessels in the predicted image to obtain the corresponding predicted contours may include: preprocessing the predicted image to obtain a preprocessed predicted image; wherein, the preprocessing is used to correct the segmentation error of the predicted image; and extracting the inner and outer contours of blood vessels in the preprocessed predicted image to obtain the corresponding predicted contours.

[0077] Specifically, the preprocessing of the predicted image to obtain a preprocessed predicted image may include: merging the lumen segmentation result and the wall segmentation result in the predicted image to obtain a merged first image; taking the largest connected component in the merged first image to obtain a second image; taking the intersection of the second image and the predicted image to obtain a third image; performing hole-filling processing on the third image, and using the hole-filled image as the preprocessed predicted image.

[0078] Specifically, the predicted image may include lumen segmentation results and wall segmentation results. Therefore, the lumen segmentation results and wall segmentation results can be merged first to obtain a first image that does not distinguish between the lumen and the wall.

[0079] Specifically, the maximum connected component can be taken from the first image that does not distinguish between the lumen and the wall, and then the intersection with the original predicted image can be taken. Finally, the holes are filled to obtain the predicted image after correcting the segmentation error, which can be used as the preprocessed predicted image. The specific implementation process of taking the maximum connected component, taking the intersection, and filling the holes can all refer to the existing technology, and will not be described in detail here.

[0080] It is understood that by correcting the segmentation error of the predicted image, the impact of the segmentation error can be reduced, the accuracy of the extracted predicted contour can be improved, and thus the stability of the loss value calculation results can be improved.

[0081] S130: Align the labeled contour and the predicted contour to obtain the aligned labeled contour and predicted contour.

[0082] In this embodiment of the invention, the inner contour of the annotation can be aligned with the predicted inner contour, and the outer contour of the annotation can be aligned with the predicted outer contour. That is, the starting contour point of the predicted inner contour / the predicted outer contour is adjusted to the contour point corresponding to the starting contour point position of the inner contour / the outer contour.

[0083] Specifically, aligning the labeled contour and the predicted contour to obtain aligned labeled contour and predicted contour may include: correcting the normal direction of the predicted contour so that the normal direction of the predicted contour is consistent with the normal direction of the labeled contour; determining a first target contour point in the labeled contour and a second target contour point in the corrected predicted contour corresponding to the first target contour point; and aligning the labeled contour and the predicted contour according to the first target contour point and the second target contour point to obtain aligned labeled contour and predicted contour.

[0084] Specifically, during the normal direction correction of the predicted inner contour, the normal vectors of the predicted inner contour P-Cin and the labeled inner contour L-Cin can be determined separately, and the angle between their normal vectors can be calculated. If the angle between their normal vectors is greater than 90°, the contour points of the predicted inner contour P-Cin can be reversed.

[0085] For example, in conjunction with the appendix to the reference specification Figure 3 Assuming the contour points of the predicted inner contour P-Cin are arranged counterclockwise (e.g., ... Figure 3 As shown in (a), if the normal vector is n1 and the angle between the normal vectors of the predicted inner contour P-Cin and the labeled inner contour L-Cin is greater than 90°, then the contour points of the predicted inner contour P-Cin can be reversed to a clockwise arrangement (e.g., ...). Figure 3 As shown in (b), the normal vector also becomes n2.

[0086] It should be noted that the process of correcting the normal direction of the predicted outer contour P-Cout can refer to the process of correcting the normal direction of the predicted inner contour P-Cin, and will not be described again in this embodiment of the invention.

[0087] It is understood that by correcting the normal direction of the predicted contour, the arrangement direction of the contour points of the predicted contour can be kept consistent with the arrangement direction of the contour points of the labeled contour, which facilitates subsequent alignment and calculation.

[0088] Specifically, after the normal direction correction is completed, the labeled inner contour can be aligned with the predicted inner contour, and the labeled outer contour can be aligned with the predicted outer contour. For the labeled inner contour L-Cin, the normal vector of the labeled inner contour L-Cin can be denoted as n, and the direction vector between the center point of the labeled inner contour L-Cin and any contour point can be denoted as n. Then the corresponding reference vector product can be calculated. Using the same method, for the predicted inner contour P-Cin, the reference vector product corresponding to any contour point of the predicted inner contour P-Cin can be calculated. .

[0089] Specifically, the first contour point in the labeled inner contour L-Cin can be... As the first target contour point, its corresponding reference vector product is It is possible to iterate through all contour points of the predicted inner contour P-Cin. ,calculate and dot product It can be confirmed. Contour point corresponding to the maximum value , which is the second target contour point in the predicted inner contour P-Cin that corresponds to the first target contour point.

[0090] Specifically, it can be As the first contour point of the predicted inner contour P-Cin, As the second contour point of the predicted inner contour P-Cin, and so on, and so on, As The next contour point is used to align P-Cin with L-Cin.

[0091] It should be noted that the process of aligning the labeled outer contour L-Cout and the predicted outer contour P-Cout can refer to the process of aligning the labeled inner contour L-Cin and the predicted inner contour P-Cin, and will not be repeated here in this embodiment of the invention.

[0092] S140: Extract the contour control points from the aligned labeled contour and the predicted contour, respectively.

[0093] In this embodiment of the invention, a first preset number of contour control points can be extracted from the labeled inner contour and the predicted inner contour, respectively, and a second preset number of contour control points can be extracted from the labeled outer contour and the predicted outer contour. The first and second preset numbers can be pre-set according to actual needs, and this embodiment of the invention does not specify a particular implementation. This embodiment of the invention also does not impose specific limitations on the method for extracting contour control points; those skilled in the art can choose according to actual needs. For example, the first and second preset number of contour control points can preferably be contour control points sampled at equal intervals.

[0094] In one possible embodiment, since both the labeled contour and the predicted contour include multiple contour points, the step of extracting contour control points from the aligned labeled contour and the predicted contour respectively may include: downsampling the multiple contour points in the aligned labeled contour to obtain a preset number of first contour control points in the labeled contour; and downsampling the multiple contour points in the aligned predicted contour to obtain a preset number of second contour control points in the predicted contour.

[0095] Specifically, the aligned inner contour L-Cin and the predicted inner contour P-Cin can be downsampled at equal intervals to obtain Nin contour points, which serve as the first contour control points for the inner contour L-Cin and the second contour control points for the predicted inner contour P-Cin. Similarly, the aligned outer contour L-Cout and the predicted outer contour P-Cout can be downsampled at equal intervals to obtain Nout contour points, which serve as the first contour control points for the outer contour L-Cout and the second contour control points for the predicted outer contour P-Cout. The specific process of equal-interval downsampling can be found in existing technologies, and will not be elaborated further in this embodiment of the invention.

[0096] The number of contour control points Nin and Nout for downsampling can be set according to actual needs. For example, analogous to the number of inner contour control points typically selected manually when sketching a tube, the number of contour control points Nin corresponding to the labeled inner contour L-Cin and the predicted inner contour P-Cin can be set to 6-8. The number of contour control points Nout corresponding to the labeled outer contour L-Cout and the predicted outer contour P-Cout can be determined by the following formula:

[0097] ;

[0098] Where lambda represents the ratio of the perimeter of the outer contour of the labeled image to the perimeter of the inner contour of the labeled image, and [*] indicates that the logarithmic value is rounded up.

[0099] S150: Determine the loss value between the labeled image and the predicted image based on the distance between the contour control points in the labeled contour and the predicted contour.

[0100] In this embodiment of the invention, the distance between the labeled contour and the predicted contour can be measured based on the distance between the contour control points corresponding to each position in the labeled inner contour and the predicted inner contour, and the distance between the contour control points corresponding to each position in the labeled outer contour and the predicted outer contour.

[0101] In one possible embodiment, determining the loss value between the labeled image and the predicted image based on the distance between contour control points in the labeled contour and the predicted contour may include: calculating the sum of the distances between first contour control points and second contour control points corresponding to each position in the labeled inner contour and the predicted inner contour to obtain the total inner contour distance; calculating the sum of the distances between first contour control points and second contour control points corresponding to each position in the labeled outer contour and the predicted outer contour to obtain the total outer contour distance; and performing a weighted summation of the total inner contour distance and the total outer contour distance to obtain the loss value between the labeled image and the predicted image.

[0102] Specifically, after extracting the first contour control point in the labeled inner contour and the second contour control point in the predicted inner contour, the total inner contour distance Lin can be calculated using the following formula:

[0103] ;

[0104] in, This represents the j-th first contour control point in the annotated inner contour. Nin represents the j-th second contour control point in the predicted inner contour, and Nin represents the number of contour control points.

[0105] It should be noted that the calculation method for the sum of outer contour distances Lout is the same as the calculation method for the sum of inner contour distances Lin, and will not be repeated here in this embodiment of the invention.

[0106] Specifically, after calculating the total contour distance Lin and the total outer contour distance Lout, the loss value ringLoss between the labeled image and the predicted image can be calculated using the following formula:

[0107] ;

[0108] Wherein, w1 and w2 are weighting coefficients. The values ​​of w1 and w2 can be preset according to actual needs, for example, they can be set to weighting coefficients w1=w2=0.5. This embodiment of the invention does not impose specific limitations on this. In one possible embodiment, since the prediction accuracy of the outer contour is generally lower than that of the inner contour, the outer contour can be given a higher weight, for example, it can be set to weighting coefficients w1=0.4, w2=0.6.

[0109] In summary, the loss value determination method according to embodiments of the present invention extracts contour control points of the lumen and pipe wall in the labeled image and the predicted image by simulating the approach of manual sketching, thereby determining the loss value between the labeled image and the predicted image. This method not only considers the unique annular features of the lumen and pipe wall, but also transforms the problem of segmenting the inner and outer boundaries of the lumen and pipe wall into an optimization problem of key contour control points, greatly reducing the computational workload of determining the loss value between the labeled image and the predicted image. Furthermore, the calculation results are more stable and applicable to the training of various types of pipe wall segmentation models, thus having a wide range of applications.

[0110] The following is a detailed description of the specific process of model training using the loss value determination method provided in the embodiments of the present invention.

[0111] Reference manual attached Figure 4 This illustrates the flow of a model training method provided by an embodiment of the present invention. This method can be applied to a server, which can be an independent server, a server cluster composed of multiple servers, or a distributed system. It can also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Specifically, as shown... Figure 4 As shown, the method may include the following steps:

[0112] S410: Obtain the labeled image, which is obtained by labeling the lumen and wall of the target blood vessel image.

[0113] In this embodiment of the invention, multiple different cross-sectional images of blood vessels can be pre-acquired as target blood vessel images to train the vessel wall segmentation model. Annotating the lumen and wall of the blood vessels in the multiple different cross-sectional images yields multiple annotated images.

[0114] S420: Input the target blood vessel image into the vessel wall segmentation model for lumen and vessel wall segmentation processing to obtain the corresponding predicted image.

[0115] In this embodiment of the invention, a vessel wall segmentation model can be pre-constructed based on a deep learning algorithm, and the constructed vessel wall segmentation model can be used to segment the target blood vessel image to obtain the corresponding predicted image.

[0116] S430: Based on the labeled image and the predicted image, determine the loss value between the labeled image and the predicted image according to the above-described method for determining the loss value.

[0117] In this embodiment of the invention, the specific content of step S430 can be found by referring to... Figures 1 to 3 The relevant content of the illustrated embodiments will not be repeated here.

[0118] S440: Update the model parameters of the pipe wall segmentation model based on the loss value to obtain the updated pipe wall segmentation model.

[0119] In this embodiment of the invention, the loss value ringLoss calculated according to the above-described method for determining the loss value can be used to evaluate the accuracy of the inner and outer boundaries of the pipe wall, preserving the annular structural features of the pipe wall, and therefore can be used for training deep learning models. Specifically, the model parameters of the pipe wall segmentation model can be updated based on the calculated loss function value to obtain an updated pipe wall segmentation model. By repeating the above steps to iteratively train the pipe wall segmentation model multiple times, a trained pipe wall segmentation model can be obtained. The specific details of the model training process can be found in existing technologies, and will not be elaborated further in this embodiment of the invention.

[0120] In one possible embodiment, when using the loss value ringLoss calculated by the above-described loss value determination method for deep learning model training, the accuracy of overall segmentation of the lumen and pipe wall can also be considered. That is, ringLoss can be combined with other loss functions for the final model training. For example, ringLoss can be combined with Dice loss and cross-entropy loss to train the pipe wall segmentation model.

[0121] Specifically, based on the labeled image and the predicted image, the Dice loss and cross-entropy loss can be calculated, and then the final loss value L can be calculated based on the following formula:

[0122]

[0123] Where Ldice represents Dice loss and Lce represents cross-entropy loss. This represents the corresponding weighting coefficient. The value can be preset according to actual needs; for example, it can be set as a weighting coefficient. , The embodiments of the present invention do not impose specific limitations in this regard.

[0124] It should be noted that the calculation process of Dice loss and cross-entropy loss can refer to existing technologies, and will not be repeated here in the embodiments of the present invention.

[0125] Specifically, the model parameters of the pipe wall segmentation model can be updated based on the calculated final loss value L, and then a trained pipe wall segmentation model can be obtained by iteratively training the pipe wall segmentation model multiple times.

[0126] It should be noted that the other contents of steps S410 to S440 above can be referred to Figures 1 to 3 The relevant content of the illustrated embodiments will not be repeated here.

[0127] In summary, the model training method provided by the embodiments of the present invention, by applying the above-mentioned method for determining the loss value to the training process of the pipe wall segmentation model, can improve the consistency between the model prediction results and the manual delineation results, the stability and universality of the model training, and improve the accuracy of the trained pipe wall segmentation model in segmenting the lumen and pipe wall, thereby obtaining more accurate pipe wall segmentation results.

[0128] Reference manual attached Figure 5 This illustrates the structure of a loss value determination device 500 in a pipe wall segmentation model provided by an embodiment of the present invention. For example... Figure 5 As shown, the device 500 may include:

[0129] The first acquisition module 510 is used to acquire an annotated image obtained by annotating the lumen and wall of the target blood vessel image, and to extract the inner and outer contours of the blood vessels in the annotated image to obtain the corresponding annotated contours.

[0130] The second acquisition module 520 is used to acquire a predicted image obtained by segmenting the lumen and wall based on the target blood vessel image, and to extract the inner and outer contours of the blood vessels in the predicted image to obtain the corresponding predicted contours.

[0131] Alignment module 530 is used to align the labeled contour and the predicted contour to obtain aligned labeled contour and predicted contour.

[0132] Extraction module 540 is used to extract contour control points from the aligned labeled contour and the predicted contour, respectively;

[0133] The first determining module 550 is used to determine the loss value between the labeled image and the predicted image based on the distance between the contour control points in the labeled contour and the predicted contour.

[0134] Reference manual attached Figure 6 This illustrates the structure of a model training apparatus 600 provided in one embodiment of the present invention. For example... Figure 6 As shown, the device 600 may include:

[0135] The third acquisition module 610 is used to acquire an annotated image, which is obtained by annotating the lumen and wall of the target blood vessel image.

[0136] The prediction module 620 is used to input the target blood vessel image into the vessel wall segmentation model for lumen and vessel wall segmentation processing to obtain the corresponding prediction image;

[0137] The second determining module 630 is used to determine the loss value between the labeled image and the predicted image based on the labeled image and the predicted image, according to the above-described loss value determination method;

[0138] The update module 640 is used to update the model parameters of the pipe wall segmentation model based on the loss value, so as to obtain the updated pipe wall segmentation model.

[0139] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when implementing its functions. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus provided in the above embodiments and the corresponding method embodiments belong to the same concept, and the specific implementation process can be found in the corresponding method embodiments, which will not be repeated here.

[0140] One embodiment of the present invention also provides an electronic device, which includes a processor and a memory. The memory stores at least one instruction or at least one program, which is loaded and executed by the processor to implement the method for determining the loss value or the model training method in the pipe wall segmentation model provided in the above method embodiments.

[0141] Memory can be used to store software programs and modules. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory. Memory can primarily include a program storage area and a data storage area. The program storage area can store the operating system, application programs required for the functions, etc.; the data storage area can store data created based on the use of the device, etc. Furthermore, memory can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, memory can also include a memory controller to provide the processor with access to the memory.

[0142] Refer to the attached reference manual Figure 7 The diagram shown is a block diagram of an electronic device 700 according to an embodiment of the present invention. The electronic device 700 may include one or more processors 702, system control logic 708 connected to at least one of the processors 702, system memory 704 connected to the system control logic 708, non-volatile memory (NVM) 706 connected to the system control logic 708, and network interface 710 connected to the system control logic 708.

[0143] Processor 702 may include one or more single-core or multi-core processors. Processor 702 may include any combination of general-purpose processors and special-purpose processors (e.g., graphics processors, application processors, baseband processors, etc.). In embodiments herein, processor 702 may be configured to perform operations according to... Figures 1 to 4 One or more embodiments of the various embodiments shown.

[0144] In some embodiments, system control logic 708 may include any suitable interface controller to provide any suitable interface to at least one of the processors 702 and / or any suitable device or component communicating with system control logic 708.

[0145] In some embodiments, system control logic 708 may include one or more memory controllers to provide an interface to system memory 704. System memory 704 may be used to load and store data and / or instructions. In some embodiments, memory 704 of device 700 may include any suitable volatile memory, such as suitable dynamic random access memory (DRAM).

[0146] NVM / Memory 706 may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. In some embodiments, NVM / Memory 706 may include any suitable non-volatile memory such as flash memory and / or any suitable non-volatile storage device, such as at least one of HDD (Hard Disk Drive), CD (Compact Disc) drive, and DVD (Digital Versatile Disc) drive.

[0147] NVM / Storage 706 may include a portion of storage resources mounted on the device 700, or it may be accessible by the device but is not necessarily part of the device. For example, NVM / Storage 706 may be accessed over a network via network interface 710.

[0148] Specifically, system memory 704 and NVM / memory 706 may each include a temporary copy and a permanent copy of instruction 720. Instruction 720 may include, when executed by at least one of processors 702, causing device 700 to perform, as Figures 1 to 4 The instructions for determining the loss value or training the model in the pipe wall segmentation model are shown. In some embodiments, the instructions 720, hardware, firmware, and / or their software components may be additionally / alternatively placed in the system control logic 708, the network interface 710, and / or the processor 702.

[0149] Network interface 710 may include a transceiver for providing a radio interface to device 700, thereby enabling communication with any other suitable device (such as a front-end module, antenna, etc.) via one or more networks. In some embodiments, network interface 710 may be integrated into other components of device 700. For example, network interface 710 may be integrated into at least one of the following: a communication module of processor 702, system memory 704, NVM / memory 706, and a firmware device (not shown) with instructions, which, when at least one of processor 702 executes the instructions, enable device 700 to implement... Figures 1 to 4 One or more embodiments of the various embodiments shown.

[0150] The network interface 710 may further include any suitable hardware and / or firmware to provide a multiple-input multiple-output radio interface. For example, the network interface 710 may be a network adapter, a wireless network adapter, a telephone modem, and / or a wireless modem.

[0151] In one embodiment, at least one of the processors 702 may be packaged together with the logic of one or more controllers for system control logic 708 to form a system package (SiP). In another embodiment, at least one of the processors 702 may be integrated on the same die with the logic of one or more controllers for system control logic 708 to form a system on chip (SoC).

[0152] Device 700 may further include an input / output (I / O) device 712. The I / O device 712 may include a user interface enabling a user to interact with device 700; the peripheral component interface is designed to allow peripheral components to also interact with device 700. In some embodiments, device 700 may also include sensors for determining at least one of environmental conditions and location information related to device 700.

[0153] In some embodiments, the user interface may include, but is not limited to, a display (e.g., a liquid crystal display, a touch screen display, etc.), a speaker, a microphone, one or more cameras (e.g., a still image camera and / or a video camera), a flashlight (e.g., a light-emitting diode flash), and a keyboard.

[0154] In some embodiments, the peripheral component interface may include, but is not limited to, a non-volatile memory port, an audio jack, and a power interface.

[0155] In some embodiments, the sensor may include, but is not limited to, a gyroscope sensor, an accelerometer, a proximity sensor, an ambient light sensor, and a positioning unit. The positioning unit may also be part of or interact with the network interface 710 to communicate with components of the positioning network, such as Global Positioning System (GPS) satellites.

[0156] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device 700. In other embodiments of the present invention, the electronic device 700 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0157] An embodiment of the present invention also provides a computer-readable storage medium, which can be disposed in an electronic device to store at least one instruction or at least one program related to implementing a method for determining the loss value or a model training method in a pipe wall segmentation model. The at least one instruction or the at least one program is loaded and executed by the processor to implement the method for determining the loss value or the model training method in the pipe wall segmentation model provided in the above method embodiment.

[0158] Optionally, in embodiments of the present invention, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0159] One embodiment of the present invention also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method for determining the loss value or the model training method in the pipe wall segmentation model provided in the various optional implementations described above.

[0160] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, specific embodiments have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0161] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the apparatus embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0162] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0163] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for determining the loss value in a pipe wall segmentation model, characterized in that, include: A labeled image is obtained by annotating the lumen and wall of the target blood vessel image, and the inner and outer contours of the blood vessels in the labeled image are extracted to obtain the corresponding labeled contours, wherein the labeled contours include the inner labeled contour and the outer labeled contour. A predicted image is obtained by segmenting the lumen and wall of the target blood vessel based on the target blood vessel image, and the inner and outer contours of the blood vessel in the predicted image are extracted to obtain the corresponding predicted contours, which include the predicted outer contour and the predicted inner contour. Align the inner contour of the annotation and the predicted inner contour, as well as the outer contour of the annotation and the predicted outer contour, to obtain the aligned inner contour of the annotation and the predicted inner contour, and the aligned outer contour of the annotation and the predicted outer contour. Both the labeled contour and the predicted contour include multiple contour points. The multiple contour points in the aligned labeled inner contour and the labeled outer contour are downsampled at equal intervals to obtain a preset number of L-Cin contour control points in the labeled inner contour and a preset number of L-Cout contour control points in the labeled outer contour. By performing equal-interval downsampling on multiple contour points in the aligned predicted inner contour and the predicted outer contour, a predetermined number of P-Cin contour control points in the predicted inner contour and a predetermined number of P-Cout contour control points in the predicted outer contour are obtained. Wherein, the number of contour control points of the downsampled labeled inner contour and predicted inner contour is Nin, and the number of contour control points of the downsampled labeled outer contour and predicted outer contour is Nout, wherein Nout and Nin satisfy the following relationship: ； Where lambda represents the ratio of the perimeter of the outer contour of the labeled image to the perimeter of the inner contour of the labeled image, and [*] indicates that the logarithmic value is rounded up; The sum of the distances between the L-Cin contour control points and the P-Cin contour control points corresponding to each position in the labeled inner contour and the predicted inner contour is calculated to obtain the total inner contour distance. Calculate the sum of the distances between the L-Cout contour control points and P-Cout contour control points corresponding to each position in the labeled outer contour and the predicted outer contour to obtain the total outer contour distance; The weighted sum of the inner contour distances and the outer contour distances is calculated to obtain the loss value between the labeled image and the predicted image, wherein the weight coefficient of the outer contour distance sum is higher than the weight coefficient of the inner contour distance sum.

2. The method according to claim 1, characterized in that, The step of extracting the inner and outer contours of blood vessels in the predicted image to obtain the corresponding predicted contours includes: The predicted image is preprocessed to obtain a preprocessed predicted image; wherein, the preprocessing is used to correct the segmentation error of the predicted image; The inner and outer contours of blood vessels in the preprocessed predicted image are extracted to obtain the corresponding predicted contours.

3. The method according to claim 2, characterized in that, The predicted image includes lumen segmentation results and tube wall segmentation results; The step of preprocessing the predicted image to obtain a preprocessed predicted image includes: The lumen segmentation result and the wall segmentation result in the predicted image are merged to obtain the merged first image; The second image is obtained by taking the largest connected component from the merged first image; The intersection of the second image and the predicted image is used to obtain the third image; The third image is subjected to hole-filling processing, and the hole-filled image is used as the preprocessed predicted image.

4. A device for determining the loss value in a pipe wall segmentation model, characterized in that, include: The first acquisition module is used to acquire an annotated image obtained by annotating the lumen and wall of the target blood vessel image, and to extract the inner and outer contours of the blood vessels in the annotated image to obtain the corresponding annotated contours, wherein the annotated contours include an inner annotated contour and an outer annotated contour. The second acquisition module is used to acquire a predicted image obtained by segmenting the lumen and wall based on the target blood vessel image, and to extract the inner and outer contours of the blood vessels in the predicted image to obtain the corresponding predicted contours, wherein the predicted contours include a predicted outer contour and a predicted inner contour. An alignment module is used to align the inner contour of the annotation with the predicted inner contour, and the outer contour of the annotation with the predicted outer contour, to obtain aligned inner contours of the annotation and the predicted inner contour, and outer contours of the annotation and the predicted outer contour. The extraction module is used to extract contour control points from the aligned labeled contour and the predicted contour, respectively, specifically for: Both the labeled contour and the predicted contour include multiple contour points. The multiple contour points in the aligned labeled inner contour and the labeled outer contour are downsampled at equal intervals to obtain a preset number of L-Cin contour control points in the labeled inner contour and a preset number of L-Cout contour control points in the labeled outer contour. By performing equal-interval downsampling on multiple contour points in the aligned predicted inner contour and the predicted outer contour, a predetermined number of P-Cin contour control points in the predicted inner contour and a predetermined number of P-Cout contour control points in the predicted outer contour are obtained. Wherein, the number of contour control points of the downsampled labeled inner contour and predicted inner contour is Nin, and the number of contour control points of the downsampled labeled outer contour and predicted outer contour is Nout, wherein Nout and Nin satisfy the following relationship: ； Where lambda represents the ratio of the perimeter of the outer contour of the labeled image to the perimeter of the inner contour of the labeled image, and [*] indicates that the logarithmic value is rounded up; The first determining module is used to determine the loss value between the labeled image and the predicted image based on the distance between the contour control points in the labeled contour and the predicted contour, specifically for: The sum of the distances between the L-Cin contour control points and the P-Cin contour control points corresponding to each position in the labeled inner contour and the predicted inner contour is calculated to obtain the total inner contour distance. Calculate the sum of the distances between the L-Cout contour control points and P-Cout contour control points corresponding to each position in the labeled outer contour and the predicted outer contour to obtain the total outer contour distance; The weighted sum of the inner contour distances and the outer contour distances is calculated to obtain the loss value between the labeled image and the predicted image, wherein the weight coefficient of the outer contour distance sum is higher than the weight coefficient of the inner contour distance sum.

5. An electronic device, characterized in that, The electronic device includes a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the method for determining the loss value in the pipe wall segmentation model as described in any one of claims 1-3.

Images