Image segmentation model training method, electronic device and storage medium

Abstract

The present disclosure provides a training method for an image segmentation model, an image processing method, an electronic device and a storage medium, and relates to the field of image processing. The method comprises: obtaining a training sample set including an image to be segmented of an object to be segmented and a real contour of the object to be segmented; processing the image to be segmented of the training sample set by a convolutional neural network model to obtain a predicted contour of the object to be segmented; obtaining a level set function of the predicted contour and a level set function of the real contour; determining an angle fidelity loss function according to the level set function of the predicted contour and the level set function of the real contour, the angle fidelity loss function being used to control the degree of fit of the curve angle between the predicted contour and the real contour; determining a target loss function according to the angle fidelity loss function; and training the training sample set according to the target loss function to obtain an image segmentation model. The image segmentation model obtained by training the method can obtain a predicted contour with a more fitted curve angle.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, and in particular to a training method for an image segmentation model, an image processing method, an electronic device, and a storage medium. Background Art

[0002] The accuracy of image segmentation boundary contours is one of the most important and challenging research topics in the field of image processing.

[0003] However, due to the complexity of the image segmentation problem (such as the different sizes of objects, the diversity of image resolution, the presence of contamination and occlusion of the segmented targets, and different lighting conditions, such as strong light, cloudy and sunny, etc.), the accuracy of segmentation has always been a great challenge.

[0004] In the related art, deep learning technology is used to segment the target object. However, due to the limitations of the mask size and secondary feature pooling, the accuracy of the segmentation results obtained is greatly lost, resulting in inaccurate segmentation results, especially the poor fit between the curve angle of the segmentation result and the real contour.

[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0006] The purpose of the present disclosure is to provide a training method for an image segmentation model, an electronic device and a storage medium. The image segmentation model trained by the method can obtain a predicted contour with a curve angle that fits better.

[0007] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0008] The present disclosure provides a method for training an image segmentation model, comprising: obtaining a training sample set including an image to be segmented of an object to be segmented and a real contour of the object to be segmented; processing the image to be segmented of the training sample set by a convolutional neural network model to obtain a predicted contour of the object to be segmented; obtaining a level set function of the predicted contour and a level set function of the real contour; determining an angle fidelity loss function according to the level set function of the predicted contour and the level set function of the real contour, the angle fidelity loss function being used to control the degree of curve angle fit between the predicted contour and the real contour; determining a target loss function according to the angle fidelity loss function; and training the training sample set according to the target loss function to obtain an image segmentation model.

[0009] In some exemplary embodiments of the present disclosure, the step of determining the angular fidelity loss function based on the level set function of the predicted contour and the level set function of the true contour comprises: obtaining the energy gradient descent direction of the predicted contour based on the level set function of the predicted contour; obtaining the energy gradient descent direction of the true contour based on the level set function of the true contour; determining the angular fidelity loss function based on the energy gradient descent direction of the level set function of the predicted contour and the energy gradient descent direction of the level set function of the true contour.

[0010] In some exemplary embodiments of the present disclosure, the angle fidelity loss function is determined according to the following formula:

[0011]

[0012] Among them, l direction (θ) is the angle fidelity loss function, is the energy drop gradient direction of the pixel points on the real contour of the object to be segmented, It is the energy drop gradient direction of the pixel points on the predicted contour of the object to be segmented.

[0013] In some exemplary embodiments of the present disclosure, the following formula is used to determine

[0014]

[0015] Among them, φ gt (p) is the level set function of the pixel point p on the real contour of the object to be segmented; it is determined according to the following formula

[0016]

[0017] Among them, φ pred (p) is the level set function of pixel point p on the predicted contour of the object to be segmented.

[0018] In some exemplary embodiments of the present disclosure, the step of determining the target loss function based on the angle fidelity loss function includes: determining a curve initialization loss function based on the difference between the level set function of the predicted contour and the level set function of the true contour; obtaining the level set evolution function of the predicted contour based on the level set function of the predicted contour, and obtaining a curve evolution loss function based on the level set evolution function of the predicted contour and the true contour; determining the target loss function based on the angle fidelity loss function, the curve initialization loss function and the curve evolution loss function.

[0019] In some exemplary embodiments of the present disclosure, the step of obtaining a training sample set including an image to be segmented of an object to be segmented and a true contour of the object to be segmented further includes: obtaining multiple extreme points of the true contour, and generating a Gaussian heat map with the multiple extreme points of the true contour as the center; merging the Gaussian heat map and the image to be segmented in layers to update the image to be segmented in the training sample set.

[0020] In some exemplary embodiments of the present disclosure, the step of obtaining multiple extreme points of the real contour includes: obtaining the coordinates of each point of the real contour to obtain the top, bottom, leftmost and rightmost four extreme points of the real contour.

[0021] The disclosed embodiment provides a training method for an image segmentation model, comprising: obtaining a training sample set including an image to be segmented of an object to be segmented and a true contour of the object to be segmented; obtaining multiple extreme points of the true contour, and generating a Gaussian heat map with the multiple extreme points of the true contour as the center; merging the Gaussian heat map and the image to be segmented in layers to update the image to be segmented in the training sample set; processing the updated image to be segmented in the training sample set through a convolutional neural network model to obtain a predicted contour of the object to be segmented; and obtaining an image segmentation model through training according to a target loss function obtained based on the predicted contour and the true contour.

[0022] In some exemplary embodiments of the present disclosure, the step of processing the updated image to be segmented of the training sample set through a convolutional neural network model to obtain a predicted contour of the object to be segmented includes: performing feature extraction on the updated image to be segmented through the convolutional neural network model to obtain feature maps of multiple pooling layers of the convolutional neural network model; and upsampling the feature maps of the multiple pooling layers to obtain the predicted contour of the object to be segmented.

[0023] The embodiment of the present disclosure provides an image processing method, comprising: obtaining a test set including images to be segmented, wherein the images to be segmented include objects to be segmented; and segmenting the images to be segmented by using an image segmentation model trained by the above method.

[0024] The present disclosure provides an image processing method, comprising: obtaining a test set including an image to be segmented, wherein the image to be segmented includes an object to be segmented; processing the image to be segmented in the test set to obtain a mask area of ​​the object to be segmented; obtaining multiple extreme points in the mask area, and generating a Gaussian heat map with the multiple extreme points in the mask area as the center; merging the Gaussian heat map and the image to be segmented in layers to update the image to be segmented in the test set; and segmenting the updated image to be segmented by using an image segmentation model.

[0025] In some exemplary embodiments of the present disclosure, the step of obtaining multiple extreme points of the mask area includes: obtaining the coordinates of each point in the mask area to obtain the top, bottom, leftmost and rightmost four extreme points of the mask area.

[0026] An embodiment of the present disclosure provides an electronic device, comprising: at least one processor; a storage device for storing at least one program, wherein when the at least one program is executed by the at least one processor, the at least one processor implements any one of the above-mentioned image segmentation model training methods or any one of the above-mentioned image processing methods.

[0027] An embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned image segmentation model training methods or any of the above-mentioned image processing methods.

[0028] Some embodiments of the present disclosure provide a training method for an image segmentation model, which determines a target loss function by determining an angle fidelity loss function based on a level set function of a predicted contour and a level set function of a true contour, thereby ensuring the fit of the curve angle between the predicted contour and the true contour. The image segmentation model trained according to the target loss function can obtain a predicted contour with a more fitted curve angle when processing an image to be segmented, thereby making the image processing result more accurate.

[0029] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0031] Figure 1 A schematic diagram of an exemplary system architecture to which the image segmentation model training method or image processing method according to an embodiment of the present disclosure can be applied is shown.

[0032] Figure 2 is a flowchart of a method for training an image segmentation model according to an exemplary embodiment.

[0033] Figure 3 is a flowchart of another method for training an image segmentation model according to an exemplary embodiment.

[0034] Figure 4 is a flowchart of another method for training an image segmentation model according to an exemplary embodiment.

[0035] Figure 5 is a flowchart of a method for training an image segmentation model according to an exemplary embodiment.

[0036] Figure 6 is a flowchart of another method for training an image segmentation model according to an exemplary embodiment.

[0037] Figure 7 is a flowchart showing an image processing method according to an exemplary embodiment.

[0038] Figure 8 is a schematic diagram showing an image to be segmented and an image processing result according to an exemplary embodiment.

[0039] Fig. 9 is a schematic diagram showing another image processing result according to an exemplary embodiment.

[0040] Fig.10 is a flowchart showing an image processing method according to an exemplary embodiment.

[0041] Fig.11 is a schematic structural diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0042] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0043] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0044] Figure 1A schematic diagram of an exemplary system architecture to which the image segmentation model training method or image processing method according to an embodiment of the present disclosure can be applied is shown.

[0045] like Figure 1 As shown, the system architecture may include a server 101, a network 102 and a terminal device 103. The network 102 is used to provide a medium for a communication link between the terminal device 103 and the server 101. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0046] The server 101 may be a server that provides various services, such as a background management server that provides support for devices operated by users using the terminal device 103. The background management server may analyze and process the received request data, and feed back the processing results to the terminal device.

[0047] The server 101 may, for example, obtain a training sample set including an image to be segmented of an object to be segmented and a true contour of the object to be segmented; the server 101 may, for example, process the image to be segmented of the training sample set through a convolutional neural network model to obtain a predicted contour of the object to be segmented; the server 101 may, for example, determine an angle fidelity loss function based on a level set function of the predicted contour and a level set function of the true contour; determine a target loss function based on the angle fidelity loss function; the server 101 may, for example, train the training sample set based on the target loss function to obtain an image segmentation model.

[0048] The terminal device 103 may, for example, capture an image to be segmented including an object to be segmented. The terminal device 103 may, for example, send the image to be segmented captured by the terminal device 103 to the server 101. The server 101 may, for example, process the image to be segmented using an image segmentation model to obtain the contour of the object to be segmented in the image to be segmented. The server 101 may, for example, send the image including the contour of the object to be segmented to the terminal device 103. The terminal device 103 may display the image including the contour of the object to be segmented.

[0049] It should be understood that Figure 1 The number of terminal devices, networks and servers is merely illustrative. Server 101 may be a physical server, a server cluster consisting of multiple servers, or a cloud server. It may have any number of terminal devices, networks and servers according to actual needs.

[0050] The image segmentation model training method and image processing method provided by the embodiments of the present disclosure include but are not limited to applications in unmanned driving, high-precision map production, nuclear industry planting, medical imaging, building smart cities and other fields.

[0051] Below, each step of the training method of the image segmentation model in the exemplary embodiment of the present disclosure will be described in more detail with reference to the accompanying drawings and embodiments.

[0052] Figure 2 is a flowchart of a method for training an image segmentation model according to an exemplary embodiment. The method provided in the embodiment of the present disclosure can be performed as follows: Figure 1 The process is executed in the server shown in the figure, but the present disclosure is not limited to this.

[0053] like Figure 2 As shown, the training method of the image segmentation model provided by the embodiment of the present disclosure may include the following steps.

[0054] In step S201, a training sample set including an image to be segmented of an object to be segmented and a real contour of the object to be segmented is obtained.

[0055] The actual contour of the object to be segmented may be a manually labeled contour of the object to be segmented, which may be used as a label in the convolutional neural network model.

[0056] In the disclosed embodiment, the image to be segmented may be an image taken by a user, a road image captured by a camera, etc., and the object to be segmented may be a person, an animal, or an object in the image taken by a user, or an independent ground print in the road image captured by a camera. The independent ground print may be a ground print with an independent geometric boundary structure on the road different from the lane line, such as a directional arrow, a diamond-shaped deceleration sign, etc.

[0057] The training sample set may include, for example, road images and true contours of independent ground prints in the road images.

[0058] In step S202, the image to be segmented in the training sample set is processed by a convolutional neural network model to obtain a predicted contour of the object to be segmented.

[0059] Among them, Convolutional Neural Networks (CNN) can be a feed-forward neural network that includes convolution calculations and has a deep structure, has representation learning capabilities, and can perform translation invariant analysis on input information according to its hierarchical structure.

[0060] In the disclosed embodiment, the image to be segmented in the training sample set is processed by a convolutional neural network model, which may include operations such as feature encoding and feature extraction of the image to be segmented.

[0061] The predicted contour of the object to be segmented may be a mask region of the object to be segmented.

[0062] For example, the road images in the training sample set can be processed by a convolutional neural network model to obtain the predicted contours of independent ground printed objects in the road images.

[0063] In step S203, the level set function of the predicted contour and the level set function of the true contour are obtained.

[0064] Among them, the level set function is an implicit function that represents the curve, which can easily track the changes in the topological structure of the object.

[0065] In the embodiment of the present disclosure, φ gt (p) represents the level set function of pixel point p on the true contour of the object to be segmented, φ pred (p) is the level set function of pixel point p on the predicted contour of the object to be segmented.

[0066] In step S204, an angle fidelity loss function is determined according to the level set function of the predicted contour and the level set function of the true contour.

[0067] Among them, the angle fidelity loss function can be used to control the curve angle fit between the predicted contour and the true contour.

[0068] In an exemplary embodiment, the profile can be predicted based on the level set function φ pred (p), obtain the energy gradient descent direction of the predicted contour

[0069] For example, the energy gradient direction of the pixel points on the predicted contour of the object to be segmented is It can be determined according to the following formula:

[0070]

[0071] For example, the energy gradient direction of the pixel points on the predicted contour of the object to be segmented is It can also be determined according to the following formula:

[0072]

[0073] in, The mean of the level set function representing the pixel point p on the predicted contour of the object to be segmented can be determined according to the following formula:

[0074]

[0075] Wherein, n may represent the number of pixel points p, and i is an integer greater than or equal to 1.

[0076] In an exemplary embodiment, the level set function φ of the true contour can be used to calculate the gt(p) The energy gradient descent direction to obtain the true contour.

[0077] For example, the energy gradient direction of the pixel points on the real contour of the object to be segmented It can be determined according to the following formula:

[0078]

[0079] For example, the energy gradient direction of the pixel points on the real contour of the object to be segmented It can also be determined according to the following formula:

[0080]

[0081] in, The mean value of the level set function representing the pixel point p on the true contour of the object to be segmented can be determined according to the following formula:

[0082]

[0083] Wherein, n may represent the number of pixel points p, and i is an integer greater than or equal to 1.

[0084] In an exemplary embodiment, the angle-fidelity loss function may be determined according to an energy gradient descent direction of a level set function of a predicted contour and an energy gradient descent direction of a level set function of a true contour.

[0085] For example, the angle-fidelity loss function can be determined according to the following formula:

[0086]

[0087] Among them, l direction (θ) is the angle-fidelity loss function.

[0088] In step S205, a target loss function is determined according to the angle fidelity loss function.

[0089] In the embodiments of the present disclosure, for example, the angle fidelity loss function can be determined as the target loss function, or the angle fidelity loss function and other loss functions can be combined to determine the target loss function, for example, the angle fidelity loss function and other loss functions are added together to determine the target loss function.

[0090] Determining the target loss function based on the angle fidelity loss function can make the angle of the predicted contour and the real contour more consistent. Therefore, when the trained image segmentation model segments the image, the contour of the object to be segmented is more consistent with the real contour of the object to be segmented.

[0091] In step S206, the training sample set is trained according to the target loss function to obtain an image segmentation model.

[0092] For example, the training sample set can be trained according to the target loss function. When the target loss function converges, the training of the convolutional neural network model stops, and the trained convolutional neural network model is used as the image segmentation model. The signed distance function representing the level set can be output according to the trained convolutional neural network model, and the signed distance function can be used for subsequent testing and application. The coordinate information of the contour line can be extracted according to the signed distance function to obtain the geometric structure information of the image segmentation boundary.

[0093] In the disclosed embodiment, the training sample set may include, for example, a road image and the actual contours of independent ground prints in the road image. The image segmentation model trained based on the training sample set may identify the road image and obtain the contours of the independent ground prints in the road image.

[0094] The training method of the image segmentation model provided by the embodiment of the present disclosure determines the target loss function by determining the angle fidelity loss function based on the level set function of the predicted contour and the level set function of the true contour, which can ensure the fit of the curve angle between the predicted contour and the true contour. The image segmentation model trained according to the target loss function can obtain a predicted contour with a more fitting curve angle when processing the image to be segmented, making the image processing result more accurate.

[0095] Figure 3 is a flowchart of another method for training an image segmentation model according to an exemplary embodiment.

[0096] In the present disclosure, Figure 2 The training method of the image segmentation model shown is different in that Figure 3 The training method of the image segmentation model shown further provides how to determine the target loss function based on the angle fidelity loss function, that is, provides an embodiment of the above step S205.

[0097] like Figure 3 As shown, step S205 may include the following steps.

[0098] In step S2051, a curve initialization loss function is determined according to the difference between the level set function of the predicted contour and the level set function of the true contour.

[0099] For example, the initialization loss function can be determined according to the following formula:

[0100]

[0101] Among them, l initial (θ) represents the initialization loss function.

[0102] In step S2052, a level set evolution function of the predicted contour is obtained according to the level set function of the predicted contour, and a curve evolution loss function is obtained according to the level set evolution function of the predicted contour and the true contour.

[0103] For example, the level set evolution function of the predicted contour can be determined according to the following formula:

[0104]

[0105] in, represents the level set evolution function of the predicted contour, It can be determined according to the following formula:

[0106]

[0107] Among them, λ, m θ , μ are parameters in model training, which can be determined according to actual conditions. For example, m θ ∈[0,1], λ=1, μ=0.04. i is an integer from 0 to T, and T can be set according to actual conditions.

[0108] For example, the curve evolution loss function can be determined according to the following formula:

[0109]

[0110] Among them, l T (θ) represents the curve evolution loss function, M gt (p) can represent the true contour.

[0111] In step S2053, a target loss function is determined according to the angle fidelity loss function, the curve initialization loss function and the curve evolution loss function.

[0112] In the embodiment of the present disclosure, for example, the sum of the angle fidelity loss function, the curve initialization loss function and the curve evolution loss function can be used as the target loss function, and the target loss function can be determined according to the following formula:

[0113] l pre = l direction (θ)+l initial (θ)+l T (θ) (12)

[0114] Among them, l pre represents the target loss function.

[0115] In the disclosed embodiment, the angle fidelity loss function in the objective loss function can ensure the fit of the curve angle between the predicted contour and the true contour, the curve initialization loss function can ensure the accuracy of the predicted contour as a whole, and the curve evolution loss function can ensure the robustness of the level set evolution curve segmentation effect. The combination of these three loss functions makes the visual effect of the image segmentation model segmentation and the quantitative evaluation of the segmentation results competitive.

[0116] Figure 4 is a flowchart of another method for training an image segmentation model according to an exemplary embodiment.

[0117] In the present disclosure, Figure 2 The training method of the image segmentation model shown is different in that Figure 4 The training method of the image segmentation model shown further provides how to obtain a training sample set including an image to be segmented of an object to be segmented and a real contour of the object to be segmented, that is, provides an embodiment of the above step S201.

[0118] like Figure 4 As shown, step S201 may include the following steps.

[0119] In step S2011, multiple extreme points of the true contour are obtained, and a Gaussian heat map is generated with the multiple extreme points of the true contour as the center.

[0120] The multiple extreme value points may be two or more extreme value points. The embodiment of the present disclosure is described by taking four extreme value points as an example, but the present disclosure is not limited thereto.

[0121] For example, the top, bottom, leftmost and rightmost extreme points of the true contour can be obtained, and a Gaussian heat map can be generated with the top, bottom, leftmost and rightmost extreme points as the center.

[0122] In an exemplary embodiment, the coordinates of each point of the real contour can be obtained, and the top, bottom, leftmost and rightmost extreme points of the real contour can be obtained according to the coordinates of each point of the real contour. Among them, the coordinates of each point can be two-dimensional coordinates, the top extreme point is the point with the largest ordinate, the bottom extreme point is the point with the smallest ordinate, the leftmost extreme point is the point with the smallest abscissa, and the rightmost extreme point is the point with the largest abscissa.

[0123] In the disclosed embodiment, the real contour may also be processed by a neural network model to obtain multiple extreme points of the real contour.

[0124] In step S2012, the Gaussian heat map and the image to be segmented are layer-merged to update the image to be segmented in the training sample set.

[0125] For example, the Gaussian heat map and the R (red), G (green), and B (blue) layers of the image to be segmented can be merged to generate a four-channel image, and the generated four-channel image can be used to update the image to be segmented in the training sample set.

[0126] In the disclosed embodiment, the extreme points of the true contour can be automatically obtained without manual addition, so that the image segmentation method is completely transformed into a fully automatic end-to-end method. A Gaussian heat map is generated based on multiple extreme points of the true contour and the layers are merged, which can make the subsequent training of the convolutional neural network model more accurate.

[0127] Figure 5 is a flowchart of a method for training an image segmentation model according to an exemplary embodiment. The method provided in the embodiment of the present disclosure can be performed as follows: Figure 1 The process is executed in the server shown in the figure, but the present disclosure is not limited to this.

[0128] like Figure 5 As shown, the training method of the image segmentation model provided by the embodiment of the present disclosure may include the following steps.

[0129] In step S301, a training sample set including an image to be segmented of an object to be segmented and a real contour of the object to be segmented is obtained.

[0130] The actual contour of the object to be segmented may be a manually labeled contour of the object to be segmented, which may be used as a label in the convolutional neural network model.

[0131] In the disclosed embodiment, the image to be segmented may be an image taken by a user, a road image captured by a camera, etc., and the object to be segmented may be a person, animal, object in the image taken by the user, or an independent ground print in the road image captured by a camera.

[0132] The training sample set may include, for example, road images and true contours of independent ground prints in the road images.

[0133] In step S302, multiple extreme points of the true contour are obtained, and a Gaussian heat map is generated with the multiple extreme points of the true contour as the center.

[0134] The multiple extreme value points may be two or more extreme value points. The embodiment of the present disclosure is described by taking four extreme value points as an example, but the present disclosure is not limited thereto.

[0135] For example, the top, bottom, leftmost and rightmost extreme points of the true contour can be obtained, and a Gaussian heat map can be generated with the top, bottom, leftmost and rightmost extreme points as the center.

[0136] In an exemplary embodiment, the coordinates of each point of the real contour can be obtained, and the top, bottom, leftmost and rightmost extreme points of the real contour can be obtained according to the coordinates of each point of the real contour. Among them, the coordinates of each point can be two-dimensional coordinates, the top extreme point is the point with the largest ordinate, the bottom extreme point is the point with the smallest ordinate, the leftmost extreme point is the point with the smallest abscissa, and the rightmost extreme point is the point with the largest abscissa.

[0137] In the disclosed embodiment, the real contour may also be processed by a neural network model to obtain multiple extreme points of the real contour.

[0138] In step S303, the Gaussian heat map and the image to be segmented are layer-merged to update the image to be segmented in the training sample set.

[0139] For example, the Gaussian heat map and the R (red), G (green), and B (blue) layers of the image to be segmented can be merged to generate a four-channel image, and the generated four-channel image can be used to update the image to be segmented in the training sample set.

[0140] In step S304, the images to be segmented in the updated training sample set are processed by a convolutional neural network model to obtain predicted contours of the objects to be segmented.

[0141] In the disclosed embodiment, the images to be segmented in the updated training sample set are processed by a convolutional neural network model, which may include operations such as feature encoding and feature extraction of the images to be segmented.

[0142] For example, the road images in the training sample set can be processed by a convolutional neural network model to obtain the predicted contours of independent ground printed objects in the road images.

[0143] In step S305, an image segmentation model is obtained by training according to a target loss function obtained based on the predicted contour and the true contour.

[0144] For example, a target loss function can be determined based on the predicted contour and the true contour, and a convolutional neural network model can be trained according to the determined target loss function. When the target loss function converges, the training of the convolutional neural network model can be stopped, and the trained convolutional neural network model can be used as an image segmentation model.

[0145] In the disclosed embodiment, the extreme points of the true contour can be automatically obtained without manual addition, and a Gaussian heat map is generated based on multiple extreme points of the true contour. After layer merging with the image to be segmented, the image to be segmented in the training sample set is updated, and the convolutional neural network model is trained using the updated training sample set, so that the obtained image segmentation model is more accurate.

[0146] Figure 6is a flowchart of another method for training an image segmentation model according to an exemplary embodiment.

[0147] In the present disclosure, Figure 5 The training method of the image segmentation model shown is different in that Figure 6 The training method of the image segmentation model shown further provides how to process the images to be segmented in the updated training sample set through a convolutional neural network model to obtain the predicted contours of the objects to be segmented, that is, provides an embodiment of the above-mentioned step S304.

[0148] like Figure 6 As shown, step S304 may include the following steps.

[0149] In step S3041, feature extraction is performed on the updated image to be segmented through a convolutional neural network model to obtain feature maps of multiple pooling layers of the convolutional neural network model.

[0150] Among them, the convolutional neural network model can include multiple convolutional layers, multiple pooling layers, fully connected layers and logistic regression layers, and the convolutional layers and pooling layers can be used for feature extraction.

[0151] For example, the updated image to be segmented can be encoded through a convolutional neural network model to obtain feature maps of multiple pooling layers of the convolutional neural network model.

[0152] In step S3042, the feature maps of multiple pooling layers are upsampled to obtain the predicted contour of the object to be segmented.

[0153] For example, an operation similar to FCN-8s (Fully Convolutional Networks) can be used to upsample the feature maps of multiple pooling layers of the convolutional neural network model to obtain the predicted contour of the object to be segmented.

[0154] Among them, FCN-8s is the first algorithm framework used in deep learning for semantic segmentation. The X in FCN-Xs represents the X-fold upsampling of the feature map, and X is a positive integer greater than 1.

[0155] Figure 7 is a flowchart of an image processing method according to an exemplary embodiment. The method provided by the embodiment of the present disclosure can be performed as follows Figure 1 The process is executed in the server shown in the figure, but the present disclosure is not limited to this.

[0156] like Figure 7 As shown, the image processing method provided by the embodiment of the present disclosure may include the following steps.

[0157] In step S401, a test set including images to be segmented is obtained, where the images to be segmented include objects to be segmented.

[0158] In the disclosed embodiment, the image to be segmented may be an image taken by a user, a road image captured by a camera, etc., and the object to be segmented may be a person, animal, object in the image taken by the user, or an independent ground print in the road image captured by a camera.

[0159] The test set of images to be segmented may be, for example, road images including independent map prints.

[0160] Figure 8 is a schematic diagram showing an image to be segmented and an image processing result according to an exemplary embodiment.

[0161] like Figure 8 As shown in (a) and (b), the image to be segmented is a road image, and the image to be segmented includes independent ground prints: arrows.

[0162] In step S402, the image to be segmented is segmented using an image segmentation model trained by using an image segmentation model training method.

[0163] The training method of the image segmentation model may be any of the above-mentioned training methods of the image segmentation model, or may be a training method in related technologies.

[0164] In the disclosed embodiment, the image to be segmented is segmented by a trained image segmentation model, so that the contour of the object to be segmented in the image to be segmented can be obtained.

[0165] For example, by segmenting a road image using a trained image segmentation model, the outlines of independent ground printed objects in the road image can be obtained.

[0166] After obtaining the outline of the object to be segmented, the method provided by the embodiment of the present disclosure can also mark the outline on the original image to be segmented. According to the spatial geographic coordinates corresponding to the two-dimensional coordinates of the original image to be segmented, the outline can be projected into the spatial geographic coordinate system to meet the production requirements of high-precision maps and obtain independent ground printed elements of the high-precision map.

[0167] like Figure 8 (c) and (d), the thin solid line in the figure is the image processing method provided by the embodiment of the present disclosure. Figure 8 Outlines of independent ground prints after segmentation of the images to be segmented in (a) and (b).

[0168] Fig. 9 is a schematic diagram showing another image processing result according to an exemplary embodiment.

[0169] like Fig. 9 As shown in (a) and (b), the straight right turn arrow 901 and the right turn arrow 905 are the objects to be segmented in the image to be segmented, and the lane line 901 and the lane line 908 are not the objects to be segmented. The predicted contours 904 and 905 of the objects to be segmented are obtained by the image processing method in the related art.

[0170] like Fig. 9 As shown in (a), the straight right turn arrow 901 is an object to be segmented in the image to be segmented, and the lane line 901 does not belong to the object to be segmented. Compared with the predicted contour 903 of the straight right turn arrow obtained by the embodiment of the present disclosure and the predicted contour 904 of the straight right turn arrow obtained by the image processing method in the related art, the predicted contour 903 of the straight right turn arrow obtained by the embodiment of the present disclosure is more accurately fitted in the corner points.

[0171] Similarly, if Fig. 9 As shown in (b), the right-turn arrow 905 is an object to be segmented in the image to be segmented, and the lane line 908 does not belong to the object to be segmented. Compared with the predicted contour 907 of the right-turn arrow obtained by the embodiment of the present disclosure and the predicted contour 906 of the straight-ahead right-turn arrow obtained by the image processing method in the related art, the predicted contour 906 of the straight-ahead right-turn arrow obtained by the embodiment of the present disclosure is more accurately fitted in the corner points.

[0172] The image processing method provided by the embodiment of the present disclosure has achieved the best mIOU (Mean Intersection over Union) index in the world on two public datasets, the Cityscapes dataset and the PASCALVOC (Pattern Analysis Statistical Modelling and Computational Learning VisualObject Classes) dataset. The mIOU index can represent the ratio of the intersection and union of the two sets of true value and predicted value, and can be used as an evaluation index for semantic segmentation.

[0173] The image processing method provided in the embodiments of the present disclosure uses deep learning combined with the level set method to achieve SOTA (State of the Art, indicating the level reached by the algorithm technology) results on public data sets, and is one of the methods with the best target known segmentation effect.

[0174] Fig.10 is a flowchart of an image processing method according to an exemplary embodiment. The method provided by the embodiment of the present disclosure can be performed as follows Figure 1 The process is executed in the server shown in the figure, but the present disclosure is not limited to this.

[0175] like Fig.10 As shown, the image processing method provided by the embodiment of the present disclosure may include the following steps.

[0176] In step S501, a test set including images to be segmented is obtained, where the images to be segmented include objects to be segmented.

[0177] In the disclosed embodiment, the image to be segmented may be an image taken by a user, a road image captured by a camera, etc., and the object to be segmented may be a person, animal, object in the image taken by the user, or an independent ground print in the road image captured by a camera.

[0178] The test set of images to be segmented may be, for example, road images including independent map prints.

[0179] In step S502, the image to be segmented in the test set is processed to obtain a mask region of the object to be segmented.

[0180] In the disclosed embodiment, the image to be segmented in the test set may be processed based on the segmentation network to obtain the mask region of the object to be segmented.

[0181] In step S503, a plurality of extreme value points of the mask region are obtained, and a Gaussian heat map is generated with the plurality of extreme value points of the mask region as the center.

[0182] The multiple extreme value points may be two or more extreme value points. The embodiment of the present disclosure is described by taking four extreme value points as an example, but the present disclosure is not limited thereto.

[0183] For example, the top, bottom, leftmost and rightmost extreme points of the mask area may be obtained, and a Gaussian heat map may be generated with the top, bottom, leftmost and rightmost extreme points as the center.

[0184] In an exemplary embodiment, the coordinates of each point in the mask area can be obtained, and the four extreme value points of the uppermost, lowermost, leftmost and rightmost of the mask area can be obtained according to the coordinates of each point in the mask area. Among them, the coordinates of each point can be two-dimensional coordinates, the uppermost extreme value point is the point with the largest ordinate, the lowermost extreme value point is the point with the smallest ordinate, the leftmost extreme value point is the point with the smallest abscissa, and the rightmost extreme value point is the point with the largest abscissa.

[0185] In the disclosed embodiment, the mask area may also be processed by a neural network model to obtain multiple extreme value points of the mask area.

[0186] In step S504, the Gaussian heat map and the image to be segmented are layer-merged to update the image to be segmented in the test set.

[0187] For example, the Gaussian heat map and the R (red), G (green), and B (blue) layers of the image to be segmented can be merged to generate a four-channel image, and the generated four-channel image can be used to update the image to be segmented in the training sample set.

[0188] In step S505, the updated image to be segmented is segmented using the image segmentation model.

[0189] In an exemplary embodiment, the image segmentation model may be the above Figure 5 The image segmentation model is trained using the training method shown.

[0190] In an exemplary embodiment, the image segmentation model may also be a model with segmentation function that has been trained in the related art.

[0191] In the disclosed embodiment, the updated image to be segmented is segmented by using an image segmentation model, so that the contour of the object to be segmented in the image to be segmented can be obtained.

[0192] For example, by segmenting a road image using an image segmentation model, the outlines of independent ground printed objects in the road image can be obtained.

[0193] After obtaining the contour of the object to be segmented, the method provided in the embodiment of the present disclosure can also mark the contour on the original image to be segmented.

[0194] In the disclosed embodiment, during the test process, before the image to be segmented is processed by the image segmentation model, the extreme points of the mask area can be automatically obtained without manual addition. A Gaussian heat map is generated based on multiple extreme points of the mask area, and after layer merging with the image to be segmented, the image to be segmented in the training sample set is updated, so that the contour of the object to be segmented can be obtained more accurately.

[0195] It should be noted that the above figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0196] The following are embodiments of the device disclosed herein, which can be used to execute the method embodiments disclosed herein. For details not disclosed in the device embodiments disclosed herein, please refer to the method embodiments disclosed herein.

[0197] The embodiment of the present disclosure provides a training device for an image segmentation model, which may include: a training sample set acquisition module, a predicted contour acquisition module, a level set function acquisition module, an angle fidelity loss function determination module, a target loss function determination module and an image segmentation model acquisition module.

[0198] Among them, the training sample set acquisition module can be used to acquire a training sample set including an image to be segmented of an object to be segmented and a real contour of the object to be segmented; the predicted contour acquisition module can be used to process the image to be segmented of the training sample set through a convolutional neural network model to obtain a predicted contour of the object to be segmented; the level set function acquisition module can be used to obtain the level set function of the predicted contour and the level set function of the real contour; the angle fidelity loss function determination module can be used to determine the angle fidelity loss function according to the level set function of the predicted contour and the level set function of the real contour, and the angle fidelity loss function is used to control the curve angle fit between the predicted contour and the real contour; the target loss function determination module can be used to determine the target loss function according to the angle fidelity loss function; the image segmentation model acquisition module can be used to train the training sample set according to the target loss function to obtain an image segmentation model.

[0199] In an exemplary embodiment, the angle fidelity loss function determination module may include: a first energy gradient descent direction acquisition module, which can be used to obtain the energy gradient descent direction of the predicted contour according to the level set function of the predicted contour; a second energy descent direction acquisition module, which can be used to obtain the energy gradient descent direction of the real contour according to the level set function of the real contour; an angle fidelity loss function determination module, which can be used to determine the angle fidelity loss function according to the energy gradient descent direction of the level set function of the predicted contour and the energy gradient descent direction of the level set function of the real contour.

[0200] In an exemplary embodiment, the angle-fidelity loss function is determined according to the following formula:

[0201]

[0202] Among them, l direction (θ) is the angle fidelity loss function, is the energy drop gradient direction of the pixel points on the real contour of the object to be segmented, It is the energy drop gradient direction of the pixel points on the predicted contour of the object to be segmented.

[0203] In an exemplary embodiment, the following formula is used to determine

[0204]

[0205] Among them, φ gt (p) is the level set function of pixel point p on the true contour of the object to be segmented;

[0206] Determine according to the following formula

[0207]

[0208] Among them, φ pred (p) is the level set function of pixel point p on the predicted contour of the object to be segmented.

[0209] In some embodiments, the target loss function determination module may include: a curve initialization loss function determination module, which can be used to determine the curve initialization loss function based on the difference between the level set function of the predicted contour and the level set function of the true contour; a curve evolution loss function determination module, which can be used to obtain the level set evolution function of the predicted contour based on the level set function of the predicted contour, and obtain the curve evolution loss function based on the level set evolution function of the predicted contour and the true contour; a target loss function acquisition module, which can be used to determine the target loss function based on the angle fidelity loss function, the curve initialization loss function and the curve evolution loss function.

[0210] In an exemplary embodiment, the training sample set acquisition module may include: an extreme point acquisition module, which can be used to obtain multiple extreme points of the true contour and generate a Gaussian heat map with the multiple extreme points of the true contour as the center; an image to be segmented update module, which can be used to merge the Gaussian heat map and the image to be segmented in layers to update the image to be segmented of the training sample set.

[0211] In an exemplary embodiment, the extreme point acquisition module may include: a coordinate acquisition module, which can be used to obtain the coordinates of each point of the real contour to obtain the top, bottom, leftmost and rightmost four extreme points of the real contour.

[0212] The disclosed embodiment provides another image segmentation model training device, which may include: a training sample set acquisition module, a heat map generation module, a to-be-segmented image update module, a predicted contour acquisition module and an image segmentation model acquisition module.

[0213] Among them, the training sample set acquisition module can be used to obtain a training sample set including an image to be segmented of an object to be segmented and a real contour of the object to be segmented; the heat map generation module can be used to obtain multiple extreme points of the real contour, and generate a Gaussian heat map with the multiple extreme points of the real contour as the center; the image to be segmented update module can be used to merge the Gaussian heat map and the image to be segmented into layers to update the image to be segmented of the training sample set; the predicted contour acquisition module can be used to process the updated image to be segmented of the training sample set through a convolutional neural network model to obtain a predicted contour of the object to be segmented; the image segmentation model acquisition module can be used to obtain an image segmentation model through training based on a target loss function obtained based on the predicted contour and the real contour.

[0214] In an exemplary embodiment, the predicted contour acquisition module may include: a feature map acquisition module, which can be used to extract features of the updated image to be segmented through a convolutional neural network model to obtain feature maps of multiple pooling layers of the convolutional neural network model; a predicted contour acquisition module, which can be used to upsample the feature maps of multiple pooling layers to obtain the predicted contour of the object to be segmented.

[0215] The embodiment of the present disclosure provides an image processing device, which may include: a test set acquisition module, which may be used to acquire a test set including images to be segmented, where the images to be segmented include objects to be segmented; and a segmentation module, which may be used to segment the images to be segmented using an image segmentation model trained using the above-mentioned image segmentation model training method.

[0216] The disclosed embodiment provides another image processing device, which may include: a test set acquisition module, which may be used to acquire a test set including an image to be segmented, wherein the image to be segmented includes an object to be segmented; a mask area acquisition module, which may be used to process the image to be segmented of the test set to obtain a mask area of ​​the object to be segmented; an extreme point acquisition module, which may be used to obtain multiple extreme points of the mask area, and generate a Gaussian heat map with the multiple extreme points of the mask area as the center; a test set update module, which may be used to merge the Gaussian heat map and the image to be segmented into layers to update the image to be segmented of the test set; and a segmentation module, which may be used to segment the updated image to be segmented through an image segmentation model.

[0217] In an exemplary embodiment, the extreme point acquisition module may include: a coordinate acquisition module, which may be used to obtain the coordinates of each point of the mask area to obtain the top, bottom, leftmost and rightmost four extreme points of the mask area.

[0218] In an exemplary embodiment, the image segmentation model is trained according to the above-mentioned image segmentation model training method.

[0219] It should be noted that the block diagrams shown in the above figures are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0220] Fig.11 is a schematic diagram of a structure of an electronic device according to an exemplary embodiment. It should be noted that: Fig.11 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0221] like Fig.11As shown, the electronic device 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the system 600 are also stored. The CPU 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0222] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage section 608 as needed.

[0223] In particular, according to an embodiment of the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 609, and / or installed from a removable medium 611. When the computer program is executed by a central processing unit (CPU) 601, the above-mentioned functions defined in the system of the present invention are executed.

[0224] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0225] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0226] The units involved in the embodiments of the present invention may be implemented in software or hardware. The units described may also be provided in a processor. For example, they may be described as follows: a processor includes a sending unit, an acquiring unit, a determining unit, and a first processing unit. The names of these units do not, in some cases, limit the units themselves. For example, the sending unit may also be described as a "unit that sends a request for obtaining an image to a connected server."

[0227] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes: obtaining a training sample set including an image to be segmented of an object to be segmented and a real contour of the object to be segmented; processing the image to be segmented of the training sample set through a convolutional neural network model to obtain a predicted contour of the object to be segmented; obtaining a level set function of the predicted contour and a level set function of the real contour; determining an angle fidelity loss function according to the level set function of the predicted contour and the level set function of the real contour, the angle fidelity loss function is used to control the curve angle fit between the predicted contour and the real contour; determining a target loss function according to the angle fidelity loss function; training the training sample set according to the target loss function to obtain an image segmentation model.

[0228] The exemplary embodiments of the present invention are specifically shown and described above. It should be understood that the present invention is not limited to the detailed structure, configuration or implementation method described herein; on the contrary, the present invention is intended to cover various modifications and equivalent configurations included in the spirit and scope of the appended claims.

Claims

1. A training method for an image segmentation model, characterized in that: include: Acquire a training sample set including an image to be segmented of an object to be segmented and a real contour of the object to be segmented; Processing the image to be segmented of the training sample set by a convolutional neural network model to obtain a predicted contour of the object to be segmented; Obtaining the level set function of the predicted contour and the level set function of the true contour; wherein the level set function is an implicit function representing a curve, which is used to track the topological structure change of the object Determine an angle fidelity loss function according to the level set function of the predicted contour and the level set function of the true contour, wherein the angle fidelity loss function is used to control the curve angle fit between the predicted contour and the true contour; Determine a target loss function according to the angle fidelity loss function; Training the training sample set according to the target loss function to obtain an image segmentation model; Wherein, the step of determining the target loss function according to the angle fidelity loss function comprises: Determine a curve initialization loss function according to the difference between the level set function of the predicted contour and the level set function of the true contour; Obtaining a level set evolution function of the predicted contour according to the level set function of the predicted contour, and obtaining a curve evolution loss function according to the level set evolution function of the predicted contour and the true contour; The target loss function is determined according to the angle fidelity loss function, the curve initialization loss function and the curve evolution loss function.

2. The method according to claim 1, characterized in that The step of determining the angle fidelity loss function according to the level set function of the predicted contour and the level set function of the true contour comprises: Obtaining an energy gradient descent direction of the predicted contour according to a level set function of the predicted contour; Obtaining the energy gradient descent direction of the real contour according to the level set function of the real contour; The angle fidelity loss function is determined according to an energy gradient descent direction of a level set function of the predicted contour and an energy gradient descent direction of a level set function of the true contour.

3. The method according to claim 1 or 2, characterized in that: The angle fidelity loss function is determined according to the following formula: Among them, l direction (θ) is the angle fidelity loss function, is the energy drop gradient direction of the pixel points on the real contour of the object to be segmented, It is the energy drop gradient direction of the pixel points on the predicted contour of the object to be segmented.

4. The method according to claim 3, characterized in that Determine according to the following formula Among them, φ gt (p) is the level set function of pixel point p on the real contour of the object to be segmented; Determine according to the following formula Among them, φ pred (p) is the level set function of pixel point p on the predicted contour of the object to be segmented.

5. The method according to claim 1, characterized in that The step of obtaining a training sample set including an image to be segmented of the object to be segmented and a real contour of the object to be segmented further comprises: Obtaining multiple extreme points of the true contour, and generating a Gaussian heat map with the multiple extreme points of the true contour as the center; The Gaussian heat map and the image to be segmented are layer-merged to update the image to be segmented in the training sample set.

6. The method according to claim 5, characterized in that The step of obtaining a plurality of extreme value points of the real contour comprises: The coordinates of each point of the real contour are obtained to obtain the top, bottom, leftmost and rightmost four extreme points of the real contour.

7. A training method for an image segmentation model, characterized in that: include: Acquire a training sample set including an image to be segmented of an object to be segmented and a real contour of the object to be segmented; Obtaining multiple extreme points of the true contour, and generating a Gaussian heat map with the multiple extreme points of the true contour as the center; Merging the Gaussian heat map and the image to be segmented in layers to update the image to be segmented in the training sample set; Processing the updated image to be segmented of the training sample set through a convolutional neural network model to obtain a predicted contour of the object to be segmented; An image segmentation model is obtained by training according to a target loss function obtained based on the predicted contour and the true contour; The image segmentation model is obtained by training according to the target loss function obtained based on the predicted contour and the true contour, including: Obtaining the level set function of the predicted contour and the level set function of the true contour; wherein the level set function is an implicit function representing a curve, which is used to track the topological structure change of the object Determine an angle fidelity loss function according to the level set function of the predicted contour and the level set function of the true contour, wherein the angle fidelity loss function is used to control the curve angle fit between the predicted contour and the true contour; Determine a target loss function according to the angle fidelity loss function; Training the training sample set according to the target loss function to obtain an image segmentation model; Wherein, the step of determining the target loss function according to the angle fidelity loss function comprises: Determine a curve initialization loss function according to the difference between the level set function of the predicted contour and the level set function of the true contour; Obtaining a level set evolution function of the predicted contour according to the level set function of the predicted contour, and obtaining a curve evolution loss function according to the level set evolution function of the predicted contour and the true contour; The target loss function is determined according to the angle fidelity loss function, the curve initialization loss function and the curve evolution loss function.

8. The method according to claim 7, characterized in that The step of processing the image to be segmented of the training sample set by using a convolutional neural network model to obtain the predicted contour of the object to be segmented includes: Performing feature extraction on the updated image to be segmented by using the convolutional neural network model to obtain feature maps of multiple pooling layers of the convolutional neural network model; The feature maps of the multiple pooling layers are upsampled to obtain a predicted contour of the object to be segmented.

9. An image processing method, characterized in that: include: Acquire a test set including an image to be segmented, wherein the image to be segmented includes an object to be segmented; The image to be segmented is segmented by using an image segmentation model trained by the method according to claim 1 or 7.

10. An image processing method, characterized in that: include: Acquire a test set including an image to be segmented, wherein the image to be segmented includes an object to be segmented; Processing the image to be segmented in the test set to obtain a mask area of ​​the object to be segmented; Obtaining multiple extreme value points of the mask area, and generating a Gaussian heat map with the multiple extreme value points of the mask area as the center; Merging the Gaussian heat map and the image to be segmented into layers to update the image to be segmented in the test set; Segmenting the updated image to be segmented using an image segmentation model; Wherein, the image segmentation model is trained according to the method according to claim 1 or 7.

11. The method according to claim 10, characterized in that The step of obtaining a plurality of extreme points of the mask area comprises: The coordinates of each point of the mask area are obtained to obtain the four extreme value points of the uppermost, lowermost, leftmost and rightmost of the mask area.

12. An electronic device, characterized in that: include: at least one processor; A storage device for storing at least one program, which, when executed by the at least one processor, enables the at least one processor to implement the image segmentation model training method as described in any one of claims 1 to 6, or the image segmentation model training method as described in claim 7 or 8, or the image processing method as described in claim 9, or the image processing method as described in any one of claims 10-11.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the image segmentation model training method as described in any one of claims 1 to 6, or the image segmentation model training method as described in claim 7 or 8, or the image processing method as described in claim 9, or the image processing method as described in any one of claims 10 to 11.

Images