Loss value determination method and device, model training method and device and electronic equipment

Abstract

The invention relates to the technical field of blood vessel segmentation, in particular to a loss value determination method, a model training method, a model training device and electronic equipment, and the loss value determination method comprises the steps: obtaining an annotation image obtained through the annotation of a lumen and a tube wall based on a target blood vessel image, and extracting the inner and outer contours of a blood vessel in the annotation image, obtaining a corresponding marking contour; obtaining a prediction image obtained by performing lumen and tube wall segmentation based on the target blood vessel image, and extracting the inner and outer contours of the blood vessel in the prediction image to obtain a corresponding prediction contour; aligning the labeled contour with the predicted contour to obtain the aligned labeled contour and predicted contour; respectively extracting contour control points in the aligned marked contour and the predicted contour; and determining a loss value between the annotated image and the predicted image based on the distance between the outline control points in the annotated outline and the predicted outline.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of blood vessel segmentation, and in particular to a loss value determination method, a model training method and device, and an electronic device. BACKGROUND

[0002] Intracranial atherosclerosis is one of the high-risk factors of cerebral stroke. The severity of atherosclerosis can be evaluated by quantitatively analyzing intracranial blood vessels through magnetic resonance three-dimensional high-resolution black-blood vessel wall imaging technology and measuring morphological parameters (such as normalized wall index, remodeling index, and wall thickness) of the vessel wall.

[0003] In order to ensure the accuracy of the quantitative analysis results, the accurate segmentation of the vessel wall is crucial. Although manual delineation is more accurate, the workload is too large and the repeatability is low. In order to improve the efficiency of delineation, intracranial automatic vessel wall segmentation based on deep learning has gradually become the mainstream, and the Dice accuracy of the lumen and the vessel wall can reach 0.89 and 0.77, respectively.

[0004] However, at present, when training a deep learning network model, the loss function mostly uses Dice loss and cross entropy (Cross Entropy) loss, and gives the same weight coefficient to the pixels of the lumen and the vessel wall, while ignoring the unique circular ring feature of the lumen and the vessel wall. In order to better utilize this feature, researchers have successively proposed new ideas, such as converting the image to polar coordinates for processing, or applying the level set idea to the loss function. However, these methods have a large computational complexity, the former relies on accurate center point extraction, and the latter requires fine tuning of the regularization parameter, which affects the stability and universality of the model training. SUMMARY

[0005] In view of the above problems of the prior art, the purpose of the present application is to provide a loss value determination method, a model training method and device, and an electronic device, which can reduce the computational complexity of determining the loss value, improve the stability and universality of the model training, and the accuracy of the model for lumen and vessel wall segmentation.

[0006] In order to solve the above problems, the present application provides a method for determining the loss value in a vessel wall segmentation model, comprising: obtaining a labeled image obtained by labeling the lumen and the vessel wall based on a target blood vessel image, and extracting the inner and outer contours of the blood vessels in the labeled image to obtain corresponding labeled contours; obtaining a predicted image obtained by segmenting the lumen and the vessel wall based on the target blood vessel image, and extracting the inner and outer contours of the blood vessels in the predicted image to obtain corresponding predicted contours; aligning the labeled contours and the predicted contours to obtain aligned labeled contours and predicted contours; extract contour control points in the aligned labeled contour and the predicted contour respectively; determine a loss value between the labeled image and the predicted image based on distances between contour control points in the labeled contour and the predicted contour.

[0007] Further, the extracting the inner and outer contours of the blood vessels in the predicted image to obtain the corresponding predicted contour comprises: preprocessing the predicted image to obtain a preprocessed predicted image; wherein the preprocessing is used to correct segmentation errors of the predicted image; extracting the inner and outer contours of the blood vessels in the preprocessed predicted image to obtain the corresponding predicted contour.

[0008] Further, the predicted image comprises a lumen segmentation result and a vessel wall segmentation result; The preprocessing the predicted image to obtain a preprocessed predicted image comprises: merging the lumen segmentation result and the vessel wall segmentation result in the predicted image to obtain a merged first image; taking the maximum connected domain of the merged first image to obtain a second image; taking the intersection of the second image and the predicted image to obtain a third image; performing hole filling processing on the third image, and taking the image after hole filling as the preprocessed predicted image.

[0009] Further, the aligning the labeled contour and the predicted contour to obtain the aligned labeled contour and the predicted contour comprises: performing normal direction correction on the predicted contour to make the normal direction of the predicted contour consistent with the normal direction of the labeled contour; determining a first target contour point in the labeled contour, and a second target contour point in the corrected predicted contour corresponding to the first target contour point; aligning the labeled contour and the predicted contour according to the first target contour point and the second target contour point to obtain the aligned labeled contour and the predicted contour.

[0010] Further, the labeled contour and the predicted contour each comprise a plurality of contour points; The extracting contour control points in the aligned labeled contour and the predicted contour respectively comprises: down-sampling the plurality of contour points in the aligned labeled contour to obtain a preset number of first contour control points in the labeled contour; Downsampling is performed on the plurality of contour points in the aligned predicted contour to obtain a preset number of second contour control points in the predicted contour.

[0011] Furthermore, the marked contour includes a marked inner contour and a marked outer contour, and the predicted contour includes a predicted inner contour and a predicted outer contour; The determining, based on the distance between the marked contour and the contour control point in the predicted contour, a loss value between the marked image and the predicted image, comprises: Calculating the sum of the distances between the first contour control point and the second contour control point corresponding to each position in the marked inner contour and the predicted inner contour to obtain a total inner contour distance; Calculating the sum of the distances between the first contour control point and the second contour control point corresponding to each position in the marked outer contour and the predicted outer contour to obtain a total outer contour distance; A weighted sum is performed on the sum of the inner contour distances and the sum of the outer contour distances to obtain a loss value between the labeled image and the predicted image.

[0012] Another aspect of the present invention provides a model training method, comprising: Acquire a labeled image, where the labeled image is obtained by labeling the lumen and the wall of the target blood vessel image; Inputting the target blood vessel image into a vessel wall segmentation model to perform lumen and vessel wall segmentation processing to obtain a corresponding predicted image; determining, based on the annotated image and the predicted image, a loss value between the annotated image and the predicted image according to the method according to any one of claims 1 to 6; Model parameters of the pipe wall segmentation model are updated based on the loss value to obtain an updated pipe wall segmentation model.

[0013] Another aspect of the present invention provides a device for determining a loss value in a pipe wall segmentation model, comprising: A first acquisition module is configured to acquire an annotated image obtained by annotating the lumen and the wall of the target blood vessel image, and extract the inner and outer contours of the blood vessel in the annotated image to obtain a corresponding annotated contour; a second acquisition module, configured to acquire a predicted image obtained by segmenting the lumen and the wall of the target blood vessel image, and extract the inner and outer contours of the blood vessel in the predicted image to obtain a corresponding predicted contour; an alignment module, configured to align the marked contour and the predicted contour to obtain aligned marked contour and predicted contour; An extraction module, configured to extract contour control points from the aligned marked contour and the predicted contour respectively; The first determining module is configured to determine a loss value between the annotated image and the predicted image based on distances between the contour control points in the annotated contour and the predicted contour.

[0014] In another aspect, the present application provides a model training device, comprising: The third obtaining module is configured to obtain an annotated image, which is obtained by performing lumen and vessel wall annotation on a target vessel image. The predicting module is configured to input the target vessel image into a vessel wall segmentation model to perform lumen and vessel wall segmentation processing, and obtain a corresponding predicted image. The second determining module is configured to determine a loss value between the annotated image and the predicted image based on the annotated image and the predicted image according to the method of any one of claims 1-6. The updating module is configured to update model parameters of the vessel wall segmentation model based on the loss value to obtain an updated vessel wall segmentation model.

[0015] In another aspect, the present application provides an electronic device comprising a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the method for determining the loss value in the vessel wall segmentation model or the model training method as described above.

[0016] In another aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by a processor to implement the method for determining the loss value in the vessel wall segmentation model or the model training method as described above.

[0017] Due to the above technical solutions, the present application has the following advantages: According to the method for determining the loss value of the present application, the contour control points of the lumen and vessel wall in the annotated image and the predicted image are extracted to determine the loss value between the annotated image and the predicted image by simulating the idea of manual sketching. This method not only considers the unique circular ring feature of the lumen and vessel wall, but also converts the inner and outer boundary segmentation problem of the lumen and vessel wall into an optimization problem of key contour control points, greatly reducing the computational amount of determining the loss value between the annotated image and the predicted image, and the calculation result is more stable, which can be applied to various types of vessel wall segmentation model training, and has a wide range of applications.

[0018] And the determination method of the loss value is used in the training process of the pipe wall segmentation model, which can improve the consistency between the model prediction result and the manual delineation result, improve the stability and universality of the model training, and improve the accuracy of the pipe wall segmentation model obtained by training in the lumen pipe wall segmentation, so as to obtain more accurate pipe wall segmentation results. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0020] Figure 1 is a flow chart of the determination method of the loss value in the pipe wall segmentation model provided by an embodiment of the present application; Figure 2 is a schematic diagram of the labeled image provided by an embodiment of the present application; Figure 3 is a schematic diagram of the normal vector provided by an embodiment of the present application; Figure 4 is a schematic diagram of the model training method provided by an embodiment of the present application; Figure 5 is a structural schematic diagram of the determination device of the loss value in the pipe wall segmentation model provided by an embodiment of the present application; Figure 6 is a structural schematic diagram of the model training device provided by an embodiment of the present application; Figure 7 is a structural schematic diagram of the electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to make the person skilled in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0022] It should be noted that the terms "first", "second", and the like in the description and claims of the application and the above drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, device, product or apparatus including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or apparatuses.

[0023] The method provided by the embodiment of the application can be applied to a scene of training a pipe wall segmentation model. In the training process, a target blood vessel image is labeled for lumen and pipe wall to obtain a labeled image, the target blood vessel image is input into the pipe wall segmentation model for lumen and pipe wall segmentation processing to obtain a corresponding predicted image. Then, the method provided by the embodiment of the application is used to determine a loss value between the labeled image and the predicted image based on the labeled image and the predicted image. Finally, the model parameters of the pipe wall segmentation model are updated based on the obtained loss value to obtain an updated pipe wall segmentation model.

[0024] Reference is made to the accompanying drawings Figure 1 which shows a method for determining a loss value in a pipe wall segmentation model provided by an embodiment of the application. The method can be applied to a server, which can be a stand-alone server or a server cluster composed of multiple servers or a distributed system. The server can also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. As shown in Figure 1 The method can include the following steps: S110: obtaining a labeled image obtained by labeling a target blood vessel image for lumen and pipe wall, and extracting the inner and outer contours of the blood vessels in the labeled image to obtain corresponding labeled contours.

[0025] In the embodiment of the application, the target blood vessel image can be a blood vessel cross-sectional image of each type of blood vessel, for example, a blood vessel cross-sectional image collected by using a magnetic resonance three-dimensional high-resolution black blood vessel wall imaging technology, etc. The type of blood vessel and the acquisition method of the blood vessel image are not specifically limited in the embodiment of the application.

[0026] It should be noted that the source of the target blood vessel image can be directly imported related data, can be obtained from other resource library real-time configuration connection, can also be obtained from the stored image database according to the user's name and other information search, the embodiment of the application does not limit this.

[0027] In the embodiment of the application, after the target blood vessel image is obtained, the lumen and the wall of the blood vessel can be manually outlined to obtain a labeled image. Specifically, referring to the accompanying drawings Figure 2 , which shows a schematic diagram of a labeled image provided by an embodiment of the application. When manually outlining the lumen and the wall of the blood vessel, a series of key control points can be selected in sequence, and then two closed curves of the inside and the outside are generated by interpolation. Among them, as shown in (a) of Figure 2 , the pixel points surrounded by the inside curve are labeled as the lumen, as shown in (b) of Figure 2 , the pixel points between the inside and the outside curves are labeled as the wall.

[0028] In the embodiment of the application, the existing contour extraction method can be used to extract the inside and outside contours of the blood vessel in the labeled image to obtain the corresponding labeled contour, and the embodiment of the application does not specifically limit the method used for contour extraction. Specifically, the labeled contour can include a labeled inside contour (denoted as L-Cin) and a labeled outside contour (denoted as L-Cout). The labeled inside contour and the labeled outside contour can each include a plurality of contour points and can be represented by a vector formed by the plurality of contour points. For example, for the labeled inside contour, the vector formed by the plurality of contour points can be represented as L-Cin = (L-Cin 1, L-Cin 2, …, L-Cin n). For the labeled outside contour, the vector formed by the plurality of contour points can be represented as L-Cout = (L-Cout 1, L-Cout 2, …, L-Cout n).

[0029] S120: obtaining a prediction image obtained by segmenting the lumen and the wall based on the target blood vessel image, and extracting the inside and outside contours of the blood vessel in the prediction image to obtain the corresponding prediction contour.

[0030] In the embodiment of the application, after the target blood vessel image is obtained, the lumen and the wall of the blood vessel can be automatically segmented by a deep learning algorithm, for example, the target blood vessel image can be segmented by a wall segmentation model constructed based on a deep learning algorithm to obtain a corresponding prediction image. The prediction image can include a lumen segmentation result and a wall segmentation result.

[0031] It should be noted that the embodiments of the present application do not specifically limit the type of deep learning algorithm, nor the specific structure of the pipe wall segmentation model. In actual application, a person skilled in the art can construct according to actual needs. For example, the pipe wall segmentation model can be a deep learning network model such as nnU-Net (no new-Net, an adaptive medical image segmentation framework based on U-Net).

[0032] In the embodiments of the present application, an existing contour extraction method can be used to extract the inner and outer contours of the blood vessels in the prediction image to obtain the corresponding prediction contours. The embodiments of the present application do not specifically limit the method used for contour extraction. Specifically, the prediction contours can include a prediction inner contour (denoted as P-Cin) and a prediction outer contour (denoted as P-Cout). Both the prediction inner contour and the prediction outer contour can include a plurality of contour points and can be represented by a vector formed by the corresponding plurality of contour points. For example, for the prediction inner contour, the vector formed by the corresponding plurality of contour points can be represented as For the prediction outer contour, the vector formed by the corresponding plurality of contour points can be represented as

[0033] In one possible embodiment, for the prediction image, the influence of the segmentation error needs to be considered, so when extracting the inner and outer contours, the segmentation error of the prediction image can be corrected first, and the inner and outer contours are extracted from the corrected prediction image. Specifically, the extraction of the inner and outer contours of the blood vessels in the prediction image to obtain the corresponding prediction contours can include: pre-processing the prediction image to obtain a pre-processed prediction image; wherein the pre-processing is used to correct the segmentation error of the prediction image; extracting the inner and outer contours of the blood vessels in the pre-processed prediction image to obtain the corresponding prediction contours.

[0034] Specifically, the pre-processing of the prediction image to obtain the pre-processed prediction image can include: merging the lumen segmentation result and the pipe wall segmentation result in the prediction image to obtain a merged first image; taking the maximum connected domain of the merged first image to obtain a second image; taking the intersection of the second image and the prediction image to obtain a third image; performing hole filling processing on the third image, and taking the image after hole filling as the pre-processed prediction image.

[0035] Specifically, the prediction image can include a lumen segmentation result and a pipe wall segmentation result, so the lumen segmentation result and the pipe wall segmentation result can be merged first to obtain a first image that does not distinguish between lumen and pipe wall.

[0036] ​Specifically, the maximum connected domain of the first image, which does not distinguish between the lumen and the vessel wall, can be obtained, and then the intersection with the original predicted image can be taken. Finally, the holes can be filled. A predicted image with corrected segmentation errors can be obtained as the preprocessed predicted image. The specific implementation of the maximum connected domain, intersection, and hole filling can be referenced in the prior art and will not be further described in detail in this embodiment of the present invention.

[0037] It can be understood that by correcting the segmentation error of the predicted image, the influence of the segmentation error can be reduced, the accuracy of the extracted predicted contour can be improved, and the stability of the loss value calculation result can be improved.

[0038] S130: Align the marked contour and the predicted contour to obtain aligned marked contour and predicted contour.

[0039] In an embodiment of the present invention, the marked inner contour and the predicted inner contour, as well as the marked outer contour and the predicted outer contour can be aligned respectively, that is, the starting contour point of the predicted inner contour / the predicted outer contour is adjusted to a contour point corresponding to the position of the starting contour point of the marked inner contour / the marked outer contour.

[0040] Specifically, aligning the marked contour and the predicted contour to obtain the aligned marked contour and predicted contour may include: correcting the predicted contour in the normal direction so that the normal direction of the predicted contour is consistent with the normal direction of the marked contour; determining a first target contour point in the marked contour, and a second target contour point in the corrected predicted contour corresponding to the first target contour point; and aligning the marked contour and the predicted contour according to the first target contour point and the second target contour point to obtain the aligned marked contour and predicted contour.

[0041] Specifically, during the normal direction correction process for the predicted inner contour, the normal vectors of the predicted inner contour P-Cin and the annotated inner contour L-Cin can be determined separately, and the angle between the two normal vectors can be calculated. If the angle between the two normal vectors is greater than 90°, the contour points of the predicted inner contour P-Cin can be reversed.

[0042] For example, in conjunction with the reference to the specification Figure 3 , assuming that the contour points of the predicted inner contour P-Cin are arranged counterclockwise (such as Figure 3 As shown in (a), the normal vector is n1, and the angle between the normal vectors of the predicted inner contour P-Cin and the normal vector of the marked inner contour L-Cin is greater than 90°, then the contour points of the predicted inner contour P-Cin can be reversed to be arranged clockwise (as shown in Figure 3 As shown in (b) in the figure), the normal vector also becomes n2.

[0043] It should be noted that the process of normal direction correction of the predicted outer contour P-Cout can refer to the process of normal direction correction of the predicted inner contour P-Cin, and the embodiments of the present application will not be repeated here.

[0044] It can be understood that by normal direction correction of the predicted contour, the arrangement direction of the contour points of the predicted contour can be ensured to be consistent with the arrangement direction of the contour points of the labeled contour, facilitating subsequent alignment and calculation.

[0045] Specifically, after the normal direction correction is completed, the labeled inner contour and the predicted inner contour, and the labeled outer contour and the predicted outer contour can be aligned respectively. For the labeled inner contour L-Cin, the normal vector of the labeled inner contour L-Cin can be denoted as n, and the direction vector of the center point of the labeled inner contour L-Cin and any contour point can be denoted as Then the corresponding reference vector product can be calculated . Using the same method, for the predicted inner contour P-Cin, the reference vector product corresponding to any contour point of the predicted inner contour P-Cin can be calculated .

[0046] Specifically, the first contour point in the labeled inner contour L-Cin can be taken as the first target contour point, and the reference vector product corresponding thereto is All contour points of the predicted inner contour P-Cin can be traversed, and the dot product of and is calculated . The contour point corresponding to the maximum value of can be determined as the second target contour point in the predicted inner contour P-Cin corresponding to the first target contour point.

[0047] Specifically, the first contour point of the predicted inner contour P-Cin can be taken as the first contour point, the second contour point of the predicted inner contour P-Cin can be taken as the second contour point, and so on, and the contour point behind can be taken as a contour point, so as to realize the alignment of P-Cin and L-Cin.

[0048] It should be noted that the process of aligning the labeled outer contour L-Cout and the predicted outer contour P-Cout can refer to the process of aligning the labeled inner contour L-Cin and the predicted inner contour P-Cin, and the embodiments of the present application will not be repeated here.

[0049] S140: extracting contour control points from the aligned marked contour and the predicted contour respectively.

[0050] In an embodiment of the present invention, a first preset number of contour control points can be extracted from the marked inner contour and the predicted inner contour, respectively, and a second preset number of contour control points can be extracted from the marked outer contour and the predicted outer contour. The first preset number and the second preset number can be pre-set according to actual needs, and the embodiment of the present invention does not specifically implement this. The embodiment of the present invention does not specifically limit the method of extracting contour control points, and those skilled in the art can choose according to actual needs. For example, the first preset number of contour control points and the second preset number of contour control points can preferably be contour control points sampled at equal intervals.

[0051] In one possible embodiment, since both the marked contour and the predicted contour include multiple contour points, the extraction of contour control points in the aligned marked contour and the predicted contour respectively may include: downsampling the multiple contour points in the aligned marked contour to obtain a preset number of first contour control points in the marked contour; downsampling the multiple contour points in the aligned predicted contour to obtain a preset number of second contour control points in the predicted contour.

[0052] Specifically, the aligned marked inner contour L-Cin and the predicted inner contour P-Cin can be downsampled at equal intervals to obtain Nin contour points, which serve as the first contour control points of the marked inner contour L-Cin and the second contour control points of the predicted inner contour P-Cin. Similarly, the aligned marked outer contour L-Cout and the predicted outer contour P-Cout can be downsampled at equal intervals to obtain Nout contour points, which serve as the first contour control points of the marked outer contour L-Cout and the second contour control points of the predicted outer contour P-Cout. The specific process of equal-interval downsampling can be referred to the existing technology, and the embodiments of the present invention will not be repeated here.

[0053] The number of downsampled contour control points Nin and Nout can be set according to actual needs. For example, analogous to the number of inner contour control points generally selected manually when manually outlining a lumen, the number of contour control points Nin corresponding to the marked inner contour L-Cin and the predicted inner contour P-Cin can be set to 6-8. The number of contour control points Nout corresponding to the marked outer contour L-Cout and the predicted outer contour P-Cout can be determined by the following formula:

[0054] Wherein, lamda represents the ratio of the labeled outer contour perimeter of the labeled image to the labeled inner contour perimeter, and [*] represents rounding up the numerical value *.

[0055] S150: Determine the loss value between the labeled image and the predicted image based on the distance between the contour control points in the labeled contour and the predicted contour.

[0056] In the embodiment of the application, the distance between the labeled contour and the predicted contour can be measured according to the distance between the contour control points corresponding to each position in the labeled inner contour and the predicted inner contour, and the distance between the contour control points corresponding to each position in the labeled outer contour and the predicted outer contour.

[0057] In one possible embodiment, the determination of the loss value between the labeled image and the predicted image based on the distance between the contour control points in the labeled contour and the predicted contour can include: calculating the sum of distances between the first contour control point and the second contour control point corresponding to each position in the labeled inner contour and the predicted inner contour to obtain an inner contour distance sum; calculating the sum of distances between the first contour control point and the second contour control point corresponding to each position in the labeled outer contour and the predicted outer contour to obtain an outer contour distance sum; and performing weighted summation on the inner contour distance sum and the outer contour distance sum to obtain the loss value between the labeled image and the predicted image.

[0058] Specifically, after the first contour control point in the labeled inner contour and the second contour control point in the predicted inner contour are extracted, the inner contour distance sum Lin can be calculated by the following formula:

[0059] Wherein, represents the jth first contour control point in the labeled inner contour, represents the jth second contour control point in the predicted inner contour, and Nin represents the number of contour control points.

[0060] It should be noted that the calculation method of the outer contour distance sum Lout is the same as that of the inner contour distance sum Lin, which will not be described herein again.

[0061] Specifically, after the contour distance sum Lin and the outer contour distance sum Lout are calculated, the loss value ringLoss between the labeled image and the predicted image can be calculated by the following formula:

[0062] wherein w1 and w2 are weight coefficients. The values of w1 and w2 can be pre-set according to actual needs, for example, can be set as weight coefficients w1 = w2 = 0.5, and the embodiments of the present application do not make specific limitations. In one possible embodiment, in view of the fact that the prediction accuracy of the outer contour is generally lower than that of the inner contour, the outer contour can be given a higher weight, for example, can be set as weight coefficients w1 = 0.4, w2 = 0.6.

[0063] In summary, according to the loss value determination method of the embodiments of the present application, the loss value between the labeled image and the predicted image is determined by simulating the idea of manual sketching and extracting the contour control points of the lumen and the wall in the labeled image and the predicted image. This method not only takes into account the unique circular ring feature of the lumen and the wall, but also converts the inner and outer boundary segmentation problem of the lumen and the wall into an optimization problem of key contour control points, greatly reducing the computational amount of determining the loss value between the labeled image and the predicted image, and the calculation result is more stable, which can be applied to various types of wall segmentation model training, and has a wide range of applications.

[0064] The specific process of model training using the loss value determination method provided by the embodiments of the present application is described in detail below.

[0065] Reference is made to the accompanying drawings Figure 4 which shows the flow of the model training method provided by an embodiment of the present application. The method can be applied in a server, which can be an independent server or a server cluster composed of multiple servers or a distributed system, and can also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content distribution networks (CDN) and big data and artificial intelligence platforms. Specifically, as shown in Figure 4 The method can include the following steps: S410: obtaining a labeled image, wherein the labeled image is obtained by performing lumen and wall labeling on a target blood vessel image.

[0066] In the embodiments of the present application, a plurality of different blood vessel cross-sectional images can be pre-acquired as target blood vessel images to train the wall segmentation model. By performing lumen and wall labeling on a plurality of different blood vessel cross-sectional images, a plurality of labeled labeled images can be obtained.

[0067] S420: inputting the target blood vessel image into the wall segmentation model for lumen and wall segmentation processing to obtain a corresponding predicted image.

[0068] In an embodiment of the present invention, a vessel wall segmentation model may be pre-constructed based on a deep learning algorithm, and the constructed vessel wall segmentation model may be used to perform segmentation processing on the target blood vessel image to obtain a corresponding predicted image.

[0069] S430: Based on the annotated image and the predicted image, determine a loss value between the annotated image and the predicted image according to the above-mentioned method for determining a loss value.

[0070] In the embodiment of the present invention, the specific content of step S430 can be referred to Figures 1 to 3 The relevant contents of the illustrated embodiment will not be repeated here in the embodiment of the present invention.

[0071] S440: updating the model parameters of the pipe wall segmentation model based on the loss value to obtain an updated pipe wall segmentation model.

[0072] In an embodiment of the present invention, the loss value ringLoss calculated according to the above-mentioned loss value determination method can be used to evaluate the accuracy of the inner and outer boundaries of the pipe wall, retain the annular structural characteristics of the pipe wall, and can therefore be used to train a deep learning model. Specifically, the model parameters of the pipe wall segmentation model can be updated based on the value of the calculated loss function to obtain an updated pipe wall segmentation model. By repeating the above steps and iteratively training the pipe wall segmentation model multiple times, a trained pipe wall segmentation model can be obtained. The specific content of the model training process can be referred to the existing technology, and the embodiment of the present invention will not be repeated here.

[0073] In one possible embodiment, when the loss value ringLoss calculated using the above loss value determination method is used for deep learning model training, the accuracy of the overall lumen and wall segmentation can also be considered. That is, the ringLoss can be combined with other loss functions for the final model training. For example, the ringLoss can be combined with Dice loss and cross entropy loss to train the vessel wall segmentation model.

[0074] Specifically, the Dice loss and the cross entropy loss may be calculated based on the labeled image and the predicted image, and then the final loss value L may be calculated based on the following formula:

[0075] Where Ldice represents Dice loss, Lce represents cross entropy loss, Indicates the corresponding weight coefficient. The value can be preset according to actual needs, for example, it can be set as a weight coefficient , , the embodiment of the present invention does not impose any specific limitation on this.

[0076] It should be noted that the calculation process of Dice loss and cross entropy loss can refer to the existing technology and will not be described in detail in the embodiment of the present invention.

[0077] Specifically, the model parameters of the pipe wall segmentation model may be updated according to the calculated final loss value L, and then the pipe wall segmentation model may be iteratively trained multiple times to obtain a trained pipe wall segmentation model.

[0078] It should be noted that the other contents of the above steps S410 to S440 can be referred to Figures 1 to 3 The relevant contents of the illustrated embodiment will not be repeated here in the embodiment of the present invention.

[0079] In summary, according to the model training method provided by the embodiment of the present invention, by applying the above-mentioned loss value determination method to the training process of the tube wall segmentation model, it is possible to improve the consistency between the model prediction results and the manual outlining results, improve the stability and versatility of the model training, and improve the accuracy of the trained tube wall segmentation model in performing lumen and tube wall segmentation, thereby obtaining more accurate tube wall segmentation results.

[0080] Reference Manual Figure 5 , which shows the structure of a device 500 for determining loss values ​​in a pipe wall segmentation model provided by one embodiment of the present invention. Figure 5 As shown, the apparatus 500 may include: A first acquisition module 510 is configured to acquire an annotated image obtained by annotating the lumen and the wall of the target blood vessel image, and extract the inner and outer contours of the blood vessel in the annotated image to obtain a corresponding annotated contour; A second acquisition module 520 is configured to acquire a predicted image obtained by segmenting the lumen and the wall of the target blood vessel image, and extract the inner and outer contours of the blood vessel in the predicted image to obtain a corresponding predicted contour; an alignment module 530, configured to align the annotated contour and the predicted contour to obtain an aligned annotated contour and a predicted contour; An extraction module 540 is configured to extract contour control points from the aligned marked contour and the predicted contour respectively; The first determining module 550 is configured to determine a loss value between the annotated image and the predicted image based on a distance between contour control points in the annotated contour and the predicted contour.

[0081] Reference Manual Figure 6 , which shows the structure of a model training device 600 provided by an embodiment of the present invention. Figure 6 As shown, the apparatus 600 may include: The third obtaining module 610 is configured to obtain a labeled image, which is obtained by performing lumen and vessel wall labeling on a target vessel image. The prediction module 620 is configured to input the target vessel image into a vessel wall segmentation model to perform lumen and vessel wall segmentation processing, and obtain a corresponding predicted image. The second determination module 630 is configured to determine a loss value between the labeled image and the predicted image according to the above loss value determination method based on the labeled image and the predicted image. The updating module 640 is configured to update model parameters of the vessel wall segmentation model based on the loss value, and obtain an updated vessel wall segmentation model.

[0082] It should be noted that the apparatus provided in the above embodiments, in realizing its functions, is only exemplified by the above division of functional modules, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above described functions. In addition, the apparatus provided in the above embodiments and the corresponding method embodiments belong to the same concept, and the specific implementation process is described in detail in the corresponding method embodiments, which will not be repeated here.

[0083] One embodiment of the present application also provides an electronic device, which comprises a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the determination method of the loss value in the vessel wall segmentation model or the model training method provided in the above method embodiments.

[0084] The memory can be used to store software programs and modules, and the processor can execute various functional applications and data processing by running the software programs and modules stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs required by functions, etc.; the data storage area can store data created according to the use of the device, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device. Accordingly, the memory can also include a memory controller to provide access of the processor to the memory.

[0085] The above description is made in conjunction with the accompanying drawings Figure 7, shown is a block diagram of an electronic device 700 according to one embodiment of the application. Electronic device 700 can include one or more processors 702, system control logic 708 connected to at least one of processors 702, system memory 704 connected to system control logic 708, non-volatile memory (NVM) 706 connected to system control logic 708, and network interface 710 connected to system control logic 708.

[0086] Processor 702 can include one or more single-core or multi-core processors. Processor 702 can include any combination of general-purpose processors and dedicated processors (e.g., graphics processors, application processors, baseband processors, etc.). In embodiments herein, processor 702 can be configured to perform one or more embodiments of various embodiments as shown in Figures 1 to 4

[0087] In some embodiments, system control logic 708 can include any suitable interface controllers to provide for any suitable interface to at least one of processors 702 and / or any suitable device or component in communication with system control logic 708.

[0088] In some embodiments, system control logic 708 can include one or more memory controllers to provide an interface to system memory 704. System memory 704 can be used to load and store data and / or instructions. In some embodiments, memory 704 of device 700 can include any suitable volatile memory, such as suitable dynamic random access memory (DRAM).

[0089] NVM / memory 706 can include one or more tangible, non-transitory computer-readable media for storage of data and / or instructions. In some embodiments, NVM / memory 706 can include any suitable non-volatile storage, such as flash memory, and / or any suitable non-volatile storage device, such as at least one of a Hard Disk Drive (HDD), a Compact Disc (CD) drive, a Digital Versatile Disc (DVD) drive.

[0090] NVM / memory 706 can include a portion of storage resident on a device of device 700, or it can be accessible by the device but not necessarily a part of the device. For example, NVM / memory 706 can be accessed over a network via network interface 710.

[0091] ​In particular, system memory 704 and NVM / memory 706 can include, respectively, a temporary copy of instructions 720 and a permanent copy of instructions 720. Instructions 720 can include instructions that, when executed by at least one of processors 702, cause device 700 to implement various embodiments as described herein, such as Figures 1 to 4 the determination of loss values in the pipe wall segmentation model or the model training method as illustrated. In some embodiments, instructions 720, hardware, firmware, and / or software components thereof can additionally / alternatively be placed in system control logic 708, network interface 710, and / or processors 702.

[0092] Network interface 710 can include a transceiver to provide a radio interface for device 700 to communicate with any other suitable device (e.g., front-end modules, antennas, etc.) over one or more networks. In some embodiments, network interface 710 can be integrated with other components of device 700. For example, network interface 710 can be integrated with a communication module of processors 702, system memory 704, NVM / memory 706, and firmware devices (not shown) having instructions that, when executed by at least one of processors 702, cause device 700 to implement various embodiments as described herein. Figures 1 to 4

[0093] Network interface 710 can further include any suitable hardware and / or firmware to provide a multiple-input multiple-output radio interface. For example, network interface 710 can be a network adapter, a wireless network adapter, a telephone modem, and / or a wireless modem.

[0094] In one embodiment, at least one of processors 702 can be packaged with logic for system control logic 708 in a system-in-a-package (SiP). In one embodiment, at least one of processors 702 can be integrated on the same die with logic for system control logic 708 to form a system-on-a-chip (SoC).

[0095] Device 700 can further include input / output (I / O) devices 712. I / O devices 712 can include a user interface to enable a user to interact with device 700; a peripheral component interface to enable peripheral components to interact with device 700. In some embodiments, device 700 further includes sensors to determine at least one of environmental conditions and location information related to device 700.

[0096] In some embodiments, the user interface can include, without limitation, a display (e.g., a liquid crystal display, a touch screen display, etc.), a speaker, a microphone, one or more cameras (e.g., a still image camera and / or a video camera), a flashlight (e.g., a light emitting diode flash), and a keypad.​

[0097] In some embodiments, the peripheral component interface can include, but is not limited to, a non-volatile memory port, an audio jack, and a power supply interface.

[0098] In some embodiments, the sensors can include, but are not limited to, a gyroscope sensor, an accelerometer, a proximity sensor, an ambient light sensor, and a positioning unit. The positioning unit can also be part of or interact with the network interface 710 to communicate with components of a positioning network (e.g., Global Positioning System (GPS) satellites).

[0099] It can be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 700. In other embodiments of the present application, the electronic device 700 can include more or fewer components than illustrated, or combine certain components, or split certain components, or different arrangement of components. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.

[0100] An embodiment of the present application further provides a computer readable storage medium, which can be arranged in an electronic device to store at least one instruction or at least one program related to a loss value determination method in a pipe wall segmentation model or a model training method, the at least one instruction or the at least one program being loaded and executed by the processor to implement the loss value determination method in the pipe wall segmentation model or the model training method provided by the method embodiments.

[0101] Optionally, in the embodiments of the present application, the storage medium can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0102] An embodiment of the present application further provides a computer program product or a computer program, which includes computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to enable the computer device to perform the loss value determination method in the pipe wall segmentation model or the model training method provided in the various optional implementation examples.

[0103] It should be noted that the above-mentioned embodiments of the present application are merely intended to describe the present application and are not intended to limit the present application. The above-mentioned embodiments of the present application are described in a progressive manner, and the same or similar parts among the embodiments can be mutually referred to. Each embodiment focuses on the difference from other embodiments. In particular, the device embodiments are described simply because they are basically similar to the method embodiments, and the relevant parts can be referred to the description of the method embodiments.

[0104] The embodiments in the present specification are described in a progressive manner, and the same or similar parts among the embodiments can be mutually referred to. Each embodiment focuses on the difference from other embodiments. In particular, the device embodiments are described simply because they are basically similar to the method embodiments, and the relevant parts can be referred to the description of the method embodiments.

[0105] A person of ordinary skill in the art can understand that all or part of the above-mentioned embodiments can be completed by hardware, or a program instructing relevant hardware, and the program can be stored in a computer readable storage medium, such as a read-only memory, a magnetic disk or an optical disk.

[0106] The above-mentioned embodiments are merely intended to describe the present application and are not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for determining loss value in a pipe wall segmentation model, characterized in that: include: Acquire a labeled image obtained by labeling the lumen and the wall of the target blood vessel image, and extract the inner and outer contours of the blood vessel in the labeled image to obtain a corresponding labeled contour; Acquire a predicted image obtained by performing lumen and wall segmentation based on the target blood vessel image, and extract the inner and outer contours of the blood vessel in the predicted image to obtain a corresponding predicted contour; Aligning the marked contour and the predicted contour to obtain aligned marked contour and predicted contour; extracting contour control points from the aligned marked contour and the predicted contour respectively; A loss value between the annotated image and the predicted image is determined based on distances between contour control points in the annotated contour and the predicted contour.

2. The method according to claim 1, characterized in that The extracting the inner and outer contours of the blood vessels in the predicted image to obtain the corresponding predicted contours includes: Preprocessing the predicted image to obtain a preprocessed predicted image; wherein the preprocessing is used to correct a segmentation error of the predicted image; The inner and outer contours of the blood vessels in the preprocessed predicted image are extracted to obtain the corresponding predicted contours.

3. The method according to claim 2, characterized in that The predicted image includes a lumen segmentation result and a vessel wall segmentation result; The preprocessing of the predicted image to obtain a preprocessed predicted image includes: Merging the lumen segmentation result and the tube wall segmentation result in the predicted image to obtain a merged first image; Taking the maximum connected domain of the merged first image to obtain a second image; Taking the intersection of the second image and the predicted image to obtain a third image; Perform hole filling processing on the third image, and use the hole-filled image as the pre-processed prediction image.

4. The method according to claim 1, wherein The step of aligning the marked contour and the predicted contour to obtain the aligned marked contour and the predicted contour comprises: Performing normal direction correction on the predicted contour so that the normal direction of the predicted contour is consistent with the normal direction of the marked contour; determining a first target contour point in the annotated contour, and a second target contour point in the corrected predicted contour corresponding to the first target contour point; The marked contour and the predicted contour are aligned according to the first target contour point and the second target contour point to obtain aligned marked contour and predicted contour.

5. The method according to claim 1, wherein The marked contour and the predicted contour both include a plurality of contour points; The extracting contour control points from the aligned marked contour and the predicted contour respectively includes: Downsampling the plurality of contour points in the aligned marked contour to obtain a preset number of first contour control points in the marked contour; Downsampling is performed on the plurality of contour points in the aligned predicted contour to obtain a preset number of second contour control points in the predicted contour.

6. The method according to claim 5, characterized in that The marked contour includes a marked inner contour and a marked outer contour, and the predicted contour includes a predicted inner contour and a predicted outer contour; The determining, based on the distance between the marked contour and the contour control point in the predicted contour, a loss value between the marked image and the predicted image, comprises: Calculating the sum of the distances between the first contour control point and the second contour control point corresponding to each position in the marked inner contour and the predicted inner contour to obtain a total inner contour distance; Calculating the sum of the distances between the first contour control point and the second contour control point corresponding to each position in the marked outer contour and the predicted outer contour to obtain a total outer contour distance; A weighted sum is performed on the sum of the inner contour distances and the sum of the outer contour distances to obtain a loss value between the labeled image and the predicted image.

7. A model training method, characterized in that: include: Acquire a labeled image, where the labeled image is obtained by labeling the lumen and the wall of the target blood vessel image; Inputting the target blood vessel image into a vessel wall segmentation model to perform lumen and vessel wall segmentation processing to obtain a corresponding predicted image; determining, based on the annotated image and the predicted image, a loss value between the annotated image and the predicted image according to the method according to any one of claims 1 to 6; Model parameters of the pipe wall segmentation model are updated based on the loss value to obtain an updated pipe wall segmentation model.

8. A device for determining loss values ​​in a pipe wall segmentation model, characterized in that: include: A first acquisition module is configured to acquire an annotated image obtained by annotating the lumen and the wall of the target blood vessel image, and extract the inner and outer contours of the blood vessel in the annotated image to obtain a corresponding annotated contour; a second acquisition module, configured to acquire a predicted image obtained by segmenting the lumen and the wall of the target blood vessel image, and extract the inner and outer contours of the blood vessel in the predicted image to obtain a corresponding predicted contour; an alignment module, configured to align the marked contour and the predicted contour to obtain aligned marked contour and predicted contour; An extraction module, configured to extract contour control points from the aligned marked contour and the predicted contour respectively; The first determining module is configured to determine a loss value between the annotated image and the predicted image based on a distance between contour control points in the annotated contour and the predicted contour.

9. A model training device, characterized in that: include: a third acquisition module, configured to acquire a labeled image, wherein the labeled image is obtained by labeling the lumen and the wall of the target blood vessel image; A prediction module, configured to input the target blood vessel image into a vessel wall segmentation model to perform lumen and vessel wall segmentation processing to obtain a corresponding predicted image; a second determining module, configured to determine, based on the labeled image and the predicted image, a loss value between the labeled image and the predicted image according to the method according to any one of claims 1 to 6; An updating module is configured to update model parameters of the pipe wall segmentation model based on the loss value to obtain an updated pipe wall segmentation model.

10. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the method for determining the loss value in the pipe wall segmentation model as described in any one of claims 1 to 6 or the model training method as described in claim 7.

Images