Image segmentation model training method, image processing method and related devices

Abstract

The present application discloses an image segmentation model training method, an image processing method and related devices, the training method comprising: obtaining a training image and a corresponding training label; wherein the training image is a cross-sectional image of the levator ani muscle, the training label comprises a first label and a second label, the first label is used to characterize the range of the levator ani muscle hiatus in the training image, and the second label is used to characterize the range of the urethra region in the training image; using an initial segmentation model to segment the training image to obtain a first training result and a second training result; generating a loss value based on the training label, the first training result and the second training result, and adjusting the parameters of the initial segmentation model based on the loss value; if it is detected that the training completion condition is met, determining the initial segmentation model after parameter adjustment as the image segmentation model; the method associates the levator ani muscle hiatus region and the urethra region for training, so that the image segmentation model can more accurately identify the position of the levator ani muscle hiatus and the position of the urethra region.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image segmentation model training method, an image processing method and related devices. Background Art

[0002] Pelvic floor ultrasound, due to its noninvasive, repeatable, radiation-free, and low-cost advantages, is widely used in the diagnosis of pelvic floor dysfunction and the evaluation of treatment efficacy. Three-dimensional pelvic floor imaging can be used to assess the status of the levator ani hiatus and assist in the diagnosis of pelvic floor dysfunction. Typically, pelvic floor measurements are performed based on cross-sectional images of the levator ani muscles. However, this accuracy is low, resulting in inaccurate measurements. Summary of the Invention

[0003] In view of this, the purpose of this application is to provide an image segmentation model training method, an image processing method and related devices, which can more accurately identify the location of the levator ani hiatus.

[0004] To solve the above technical problems, this application provides an image segmentation model training method, including:

[0005] Obtaining a training image and a corresponding training label; wherein the training image is a cross-sectional image of the levator ani muscle, and the training label includes a first label and a second label, the first label being used to characterize the range of the levator ani hiatus in the training image, and the second label being used to characterize the range of the urethra region in the training image;

[0006] Segmenting the training image using the initial segmentation model to obtain a first training result and a second training result;

[0007] generating a loss value based on the training label, the first training result, and the second training result, and adjusting parameters of the initial segmentation model based on the loss value;

[0008] If it is detected that the training completion condition is met, the initial segmentation model after parameter adjustment is determined as the image segmentation model.

[0009] Optionally, generating a loss value based on the training label, the first training result, and the second training result includes:

[0010] Generating a first loss value using the first label and the first training result;

[0011] A second loss value is generated using the second label and the second training result.

[0012] Optionally, the initial segmentation model includes an encoder and a decoder;

[0013] The adjusting parameters of the initial segmentation model based on the loss value includes:

[0014] The first loss value and the second loss value are used to adjust parameters of the encoder and the decoder respectively.

[0015] Optionally, the initial segmentation model includes an encoder and two decoders;

[0016] The first decoder is used to output a first training result, and the second decoder is used to output a second training result. The output of the encoder is the input of the two decoders.

[0017] Optionally, adjusting parameters of the initial segmentation model based on the loss value includes:

[0018] performing parameter adjustment on the encoder using the first loss value and the second loss value respectively;

[0019] Using the first loss value to adjust parameters of the first decoder;

[0020] The second loss value is used to adjust parameters of the second decoder.

[0021] Optionally, the initial segmentation model includes two encoders and two decoders;

[0022] The input of the first encoder is the training image, the output of the first encoder is the input of the first decoder, the input of the second encoder is the training image and the output of the first decoder, and the output of the second encoder is the input of the second decoder;

[0023] The first decoder is used to output the first training result, and the second decoder is used to output the second training result.

[0024] Optionally, adjusting parameters of the initial segmentation model based on the loss value includes:

[0025] Using the first loss value, adjusting parameters of the first encoder and the first decoder;

[0026] The second loss value is used to adjust parameters of the second encoder and the second decoder.

[0027] Optionally, acquiring a training image includes:

[0028] Get the initial training image;

[0029] Performing data augmentation processing or image conversion processing on the initial training image to obtain the training image.

[0030] This application also provides an image processing method, comprising:

[0031] Acquire an image to be processed; wherein the image to be processed is a cross-sectional image of the levator ani muscle;

[0032] The image to be processed is input into an image segmentation model to obtain a levator ani hiatus segmentation image; the image segmentation model is obtained based on the above-mentioned image segmentation model training method.

[0033] Optionally, it also includes:

[0034] Obtaining key point positions in the image to be processed using a target image; wherein the target image includes the image to be processed and the levator ani hiatus segmented image, or includes the image to be processed, the levator ani hiatus segmented image, and a urethra region segmented image output by the image segmentation model;

[0035] Generate measurement values ​​corresponding to preset measurement items according to the key point positions.

[0036] This application also provides an image segmentation model training device, comprising:

[0037] a training acquisition module, configured to acquire a training image and corresponding training labels; wherein the training image is a cross-sectional image of the levator ani muscle, and the training labels include a first label and a second label, wherein the first label is used to characterize the range of the levator ani hiatus in the training image, and the second label is used to characterize the range of the urethra region in the training image;

[0038] A training segmentation module, configured to segment the training image using an initial segmentation model to obtain a first training result and a second training result;

[0039] a parameter adjustment module, configured to generate a loss value based on the training label, the first training result, and the second training result, and to adjust parameters of the initial segmentation model based on the loss value;

[0040] The model determination module is used to determine the initial segmentation model after parameter adjustment as the image segmentation model if it is detected that the training completion condition is met.

[0041] The present application also provides an image processing device, comprising:

[0042] The test acquisition module is used to acquire an image to be processed; wherein the image to be processed is a cross-sectional image of the levator ani muscle;

[0043] The image segmentation module is used to input the image to be processed into an image segmentation model to obtain a segmented image of the levator ani hiatus; the image segmentation model is obtained based on the above-mentioned image segmentation model training method.

[0044] The present application also provides an electronic device, including a memory and a processor, wherein:

[0045] The memory is used to store computer programs;

[0046] The processor is used to execute the computer program to implement the above-mentioned image segmentation model training method and / or the above-mentioned image processing method.

[0047] The present application also provides a computer-readable storage medium for storing a computer program, wherein when the computer program is executed by a processor, the image segmentation model training method and / or the image processing method are implemented.

[0048] It can be seen that this method marks the levator ani hiatus area and the urethra area in the training image during model training. Since the relative positions of the levator ani hiatus and the urethra area have a certain regularity, the first label and the second label are used to jointly train the initial segmentation model, so that the initial segmentation model can learn the positional relationship information between the two, so that the image segmentation model obtained after training is not only able to directly identify the levator ani hiatus area, but also can assist in the identification of the levator ani hiatus area based on the recognition ability of the urethra area. At the same time, it can also give the recognition of the urethra area based on the recognition ability of the levator ani hiatus area, that is, the two promote and assist each other, so that the image segmentation model has a stronger recognition ability of the levator ani hiatus area and the urethra area. When applied, it can more accurately identify the position of the levator ani hiatus and the urethra area in the image to be processed.

[0049] In addition, the present application also provides an image processing method, an image segmentation model training device, an image processing device, an electronic device and a computer-readable storage medium, which also have the above-mentioned beneficial effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.

[0051] Figure 1 A flow chart of an image segmentation model training method provided in an embodiment of the present application;

[0052] Figure 2 A model structure diagram provided for an embodiment of the present application;

[0053] Figure 3Another model structure diagram provided for an embodiment of the present application;

[0054] Figure 4 Another model structure diagram provided for an embodiment of the present application;

[0055] Figure 5 This is a diagram showing the effect of an image to be processed after processing provided in an embodiment of the present application;

[0056] Figure 6 A schematic diagram of the structure of an image segmentation model training device provided in an embodiment of the present application;

[0057] Figure 7 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application;

[0058] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0060] It should be noted that the image segmentation model training method and / or image processing method in the present application can be executed by a specified electronic device, and the number and form of the specified electronic device are not limited. For example, in one embodiment, the number of electronic devices is one, such as a server, which can independently execute each step of the image segmentation model training method, or an ultrasound device, which can independently execute each step of the image processing method. In another embodiment, the number of electronic devices is multiple, and each electronic device cooperates with each other to respectively execute some steps in the image segmentation model training method and / or image processing method, and finally complete the model training and / or image processing. When the number of electronic devices is multiple, the form of the electronic device can be multiple, for example, it can include a computer, an ultrasound device, and a server.

[0061] Please refer to Figure 1 , Figure 1 A flowchart of an image segmentation model training method provided in an embodiment of the present application. The method includes:

[0062] S101: Obtain training images and corresponding training labels.

[0063] A training image refers to an image with a corresponding identifier (i.e., a training label) used to train the initial segmentation model. In this application, the training image is a cross-sectional image of the levator ani muscle. This embodiment does not limit the specific method of obtaining the training image. In one embodiment, the initial image can be obtained and visualized for manual identification. During the manual identification process, the training label is obtained according to the identification instruction input by the user. After the training label is obtained, it is bound to the initial image in a corresponding relationship to complete the acquisition of the training image and the training label. In another embodiment, the training image and the corresponding training label can be pre-set, and the acquisition is completed by reading the data from the object (such as a storage path, a hardware device) where the training image and the label are stored.

[0064] In this application, the training labels for each training image include a first label and a second label. The first label is used to represent the extent of the levator ani hiatus in the training image, while the second label is used to represent the extent of the urethra region in the training image. The urethra region refers to the imaging area of ​​the human urethra on an ultrasound image during an ultrasound pelvic floor examination. Its specific shape and other factors vary depending on factors such as the angle of the ultrasound pelvic floor examination.

[0065] It is understood that in practical applications, different ultrasound images can vary significantly in terms of color style and other aspects, depending on factors such as the ultrasound device model, imaging mode, and imaging environment used to acquire the ultrasound images. To improve the versatility of the image segmentation model while preventing these factors from interfering with image processing accuracy, data augmentation or image conversion can be employed when generating training images. Specifically, initial training images are obtained; the specific number and color data distribution of the initial training images are not limited. Data augmentation or image conversion is performed on the initial training images to obtain training images. It is understood that if data augmentation is performed, the model can learn how to perform image segmentation under various color styles and data distributions during subsequent training, thereby ensuring that the image segmentation model is not affected by image color and different data distributions. If image conversion is employed, the images to be processed also need to be converted during application. This allows the image segmentation model to operate under a uniform color scale and data distribution, thus avoiding interference and being able to process images of various color scales and data distributions.

[0066] The specific methods of data augmentation processing and image conversion processing are not limited. For example, in one embodiment, all initial training images can be converted into grayscale images, and normalized after conversion to make the data distribution uniform, thereby completing the image conversion processing. In another embodiment, a generative adversarial network (GAN) can be set up. Using the generative adversarial network, levator ani muscle cross-section images with different color styles and data distributions can be generated based on the initial training images as training images to complete the data augmentation processing. Alternatively, using GAN, the initial training images with different color styles and data distributions are mapped to the same color style and data distribution to obtain training images, thereby completing the image conversion processing.

[0067] S102: Segment the training image using the initial segmentation model to obtain a first training result and a second training result.

[0068] The initial segmentation model refers to an untrained image segmentation model that, after training, can produce an image segmentation model for identifying and segmenting the levator ani hiatus region. By processing the training image using the initial segmentation model, the range of the levator ani hiatus and the range of the urethra region in the training image can be predicted based on the current model parameters, resulting in a first training result and a second training result. The first training result reflects the initial segmentation model's prediction of the levator ani hiatus, and the second training result reflects the initial segmentation model's prediction of the urethra region. Both training results are in the form of images.

[0069] S103: Generate a loss value based on the training label, the first training result, and the second training result, and adjust parameters of the initial segmentation model based on the loss value.

[0070] The training labels can represent the exact positions of the levator ani hiatus and the urethra region in the training image. Therefore, using the training labels, the first training results and the second training results, the gap between the initial segmentation model and the truly accurate prediction of the levator ani hiatus and the urethra region can be determined, and the gap can be reflected by the loss value. This embodiment does not limit the specific calculation method of the loss value. In one embodiment, since the first training result and the second training result respectively represent the positions of different parts, the loss value can be calculated using the first training result and the training labels, and the second training result and the training labels. In another embodiment, since the levator ani hiatus and the urethra region have a certain correlation, the first training result and the second training result can directly or indirectly reflect the model's prediction ability for the levator ani hiatus. Therefore, the training labels, the first training result and the second training result can be used in combination to calculate the loss value. In addition, the calculation method of the loss value can also vary depending on the model structure.

[0071] This embodiment does not limit the specific type of the initial segmentation model; any segmentation model may be used, such as an Fcn (Fully Convolutional Network) model, a U-Net model, a SegNet model, a PSPNet model, etc. In one embodiment, the initial segmentation model may include an encoder and a decoder, wherein the encoder is used to extract deep features of the input image, and the decoder is used to restore the features and perform prediction. The number and connection relationship of the encoder and decoder are not limited, and various feasible configuration schemes are possible according to needs.

[0072] The loss value generation method should match the model structure. In one embodiment, the loss value includes a first loss value and a second loss value, wherein the first loss value is generated using the first label and the first training result, and the second loss value is generated using the second label and the second training result. Depending on the model structure, the method for adjusting parameters using the first loss value and the second loss value may vary.

[0073] Specifically, considering that the area of ​​the levator ani hiatus is larger than that of the urethra, and the levator ani hiatus includes the urethra, an encoder can simultaneously extract features of the levator ani hiatus and the urethra. Therefore, in the first embodiment, the initial segmentation model includes an encoder and a decoder. Figure 2 , Figure 2 A model structure diagram provided for an embodiment of the present application, wherein the encoder m and the decoder p constitute an initial segmentation model. During the training process, the encoder m and the decoder p are not only responsible for predicting the levator ani hiatus region and thereby obtaining the first training result, but are also used to predict the urethra region and thereby obtain the second training result, so the model prediction is a dual-channel output. The encoder m can simultaneously extract the features of the levator ani hiatus region and the urethra region, and the decoder can simultaneously predict the levator ani hiatus region and the urethra region, so that the initial segmentation model can learn the two regions in association. During the parameter adjustment process, the first loss value and the second loss value are used to adjust the parameters of the encoder and the decoder, respectively. Specifically, the first loss value is used to adjust the parameters of the encoder and the decoder, and the second loss value is used to adjust the parameters of the encoder and the decoder, because the encoder m and the decoder p need to have the ability to recognize the two regions at the same time. There is no limitation on the execution order of the adjustment process using the first loss value and the adjustment process using the second loss value.

[0074] In the second embodiment, the initial segmentation model includes an encoder and two decoders, wherein the first decoder is used to output a first training result, and the second decoder is used to output a second training result, and the output of the encoder is the input of the two decoders. Figure 3 , Figure 3This is another model structure diagram provided by an embodiment of the present application. After extracting features, encoder m sends them to decoder p and decoder q, respectively. Decoder p uses the features to generate a first training result, and decoder q generates a second training result. During parameter adjustment, the encoder parameters are adjusted using the first loss value and the second loss value, respectively. In addition, the first loss value is used alone to adjust the parameters of the first decoder, and the second loss value is used to adjust the parameters of the second decoder.

[0075] In the third embodiment, considering that the shape and position relationship between the levator ani hiatus region and the urethra region is relatively complex, the prediction result of the levator ani hiatus region can be used to assist in the prediction of the urethra region and assist in the segmentation of the urethra region. During the parameter backpropagation, the urethra region will promote the segmentation of the levator ani hiatus region. Specifically, the initial segmentation model includes two encoders and two decoders, wherein the input of the first encoder is the training image, the output of the first encoder is the input of the first decoder, the input of the second encoder is the training image and the output of the first decoder, the output of the second encoder is the input of the second decoder, the first decoder is used to output the first training result, and the second decoder is used to output the second training result. Please refer to Figure 4 , Figure 4 Another model structure diagram provided for an embodiment of the present application. Among them, the cross-sectional image of the levator ani muscle is input into two encoders, encoder m is the first encoder, which extracts features and outputs them to decoder p (first decoder), and decoder p generates a first training result, i.e., a segmentation map of the levator ani muscle hiatus region. Encoder n (second encoder) not only obtains the cross-sectional image of the levator ani muscle, but also obtains a segmentation map of the levator ani muscle hiatus region as the basic data for urethra region segmentation, which extracts features and outputs them to decoder q (second decoder), which generates a urethra region segmentation map, i.e., a second training result. When adjusting the parameters, the first loss value is used to adjust the parameters of the first encoder and the first decoder, and the second loss value is used to adjust the parameters of the second encoder and the second decoder.

[0076] It should be noted that the above three model structures are only three feasible structures, and other optional structures are not described here in detail.

[0077] S104: If it is detected that the training completion condition is met, the initial segmentation model after parameter adjustment is determined as the image segmentation model.

[0078] The training completion conditions refer to conditions indicating that the initial segmentation model has been trained. The number and specific content of the training completion conditions are not limited. For example, the training completion conditions may be conditions that limit the accuracy of the initial segmentation model, or conditions that limit the number of training rounds, or conditions that limit the training duration. When one, multiple, or all of the training completion conditions are detected, the initial segmentation model can be determined to have been trained and can be used. Therefore, the initial segmentation model with adjusted parameters is determined as the image segmentation model.

[0079] During application, an image segmentation model can be used to identify the levator ani hiatus region in an image to be processed. Specifically, the image to be processed is first acquired. It should be noted that the image to be processed is a cross-sectional image of the levator ani muscle. The image to be processed is input into the image segmentation model to obtain a segmented image of the levator ani hiatus. The image segmentation model is obtained based on the aforementioned image segmentation model training method. The specific process for generating the segmented image of the levator ani hiatus can vary depending on the model structure.

[0080] After obtaining the levator ani hiatus segmentation image, subsequent operations can be further performed. For example, in one embodiment, a key point heat map prediction network can be pre-trained to obtain the key points in the levator ani hiatus segmentation image. The network type is not limited. For example, it can be a key point heat map prediction network based on deep learning technology. Any network architecture that can complete target detection or prediction can be selected, such as HRnet (high-resolution network), LAB network or CenterNet network. The key point heat map prediction network can predict the key point position based on the input image, and the input image can be called the target image. There is no limit on the number and specific practical significance of the preset key points. For example, there can be four key points, namely, the attachment point of the left levator ani muscle, the attachment point of the right levator ani muscle, the posterior edge of the pubic symphysis and the posterior edge of the levator ani anal canal.

[0081] It is understood that, in addition to being able to identify the levator ani hiatus region, the image segmentation model also has the ability to identify the urethra region. Therefore, the image segmentation model can also output a segmented image of the urethra region. The relative positional relationship between the urethra region and the levator ani hiatus region enables the urethra region segmented image to assist in locating key points within the levator ani hiatus. Therefore, in one embodiment, the locations of key points in the image to be processed are obtained using a target image. The target image may include the image to be processed and a segmented image of the levator ani hiatus, or the image to be processed, the segmented image of the levator ani hiatus, and a segmented image of the urethra region output by the image segmentation model. That is, there are two feasible implementations for selecting the target image. It is understood that the greater the number of target images, the more information can be obtained when locating key points, and the higher the accuracy of the positioning. Introducing the urethra region segmented image can further improve the accuracy of key point positioning. Generally, introducing the image to be processed for key point detection can also improve positioning accuracy to a certain extent. After determining the target images, each target image can be input into the key point heat map prediction network described above to obtain the key point locations.

[0082] After locating the key points, the measurement values ​​corresponding to the preset measurement items can be generated according to the key point positions. The key point positions can be in coordinate form or other forms, such as image form marked on the image. Based on the key point positions, measurement values ​​that are helpful for doctors' diagnosis can be calculated, such as the area and circumference of the levator ani hiatus area, the distance from the left levator ani muscle attachment point to the right levator ani muscle attachment point, the distance from the posterior edge of the pubic symphysis to the posterior edge of the levator ani and anal canal, the distance from the center of mass of the urethra area to the left levator ani muscle attachment point and the right levator ani muscle attachment point, etc. After obtaining the above measurement values, they can also be visualized to assist doctors in diagnosis. The specific form of visualization is not limited. In addition to the visualization of the above measurement values, other content can also be output, such as the image to be processed, the levator ani hiatus segmentation image, or the urethra area segmentation image. Please refer to Figure 5 , Figure 5 This is a rendering of the processed image provided in an embodiment of the present application, which shows the range of the levator ani hiatus area, the range of the urethra area, and the positions and names of various key points.

[0083] By applying the model training method and image processing method provided in the embodiments of the present application, the levator ani hiatus area and the urethra area in the training image are marked during model training. Since the relative positions of the levator ani hiatus and the urethra area have a certain regularity, the initial segmentation model is trained together with the first label and the second label, so that the initial segmentation model can learn the positional relationship information between the two, and thus the image segmentation model obtained after the training is completed can not only directly identify the levator ani hiatus area, but also assist in the identification of the levator ani hiatus area based on the recognition ability of the urethra area. At the same time, it can also give recognition to the urethra area based on the recognition ability of the levator ani hiatus area, that is, the two promote and assist each other, so that the image segmentation model has a stronger recognition ability of the levator ani hiatus area and the urethra area. When applied, it can more accurately identify the position of the levator ani hiatus and the urethra area in the image to be processed.

[0084] The image segmentation model training device provided in an embodiment of the present application is introduced below. The image segmentation model training device described below and the image segmentation model training method described above can be referenced to each other.

[0085] Please refer to Figure 6 , Figure 6 A schematic diagram of the structure of an image segmentation model training device provided in an embodiment of the present application includes:

[0086] A training acquisition module 110 is configured to acquire a training image and corresponding training labels, wherein the training image is a cross-sectional image of the levator ani muscle, and the training labels include a first label and a second label, wherein the first label is used to represent the range of the levator ani hiatus in the training image, and the second label is used to represent the range of the urethra region in the training image;

[0087] A training segmentation module 120 is configured to segment the training image using the initial segmentation model to obtain a first training result and a second training result;

[0088] a parameter adjustment module 130 for generating a loss value based on the training label, the first training result, and the second training result, and adjusting parameters of the initial segmentation model based on the loss value;

[0089] The model determination module 140 is configured to determine the initial segmentation model after parameter adjustment as the image segmentation model if it is detected that the training completion condition is met.

[0090] Optionally, the parameter adjustment module 130 includes:

[0091] A first generating unit, configured to generate a first loss value using the first label and the first training result;

[0092] The second generating unit is configured to generate a second loss value using the second label and the second training result.

[0093] Optionally, the initial segmentation model includes an encoder and a decoder;

[0094] The parameter adjustment module 130 includes:

[0095] The first adjustment unit is used to adjust parameters of the encoder and the decoder using the first loss value and the second loss value respectively.

[0096] Optionally, the initial segmentation model includes an encoder and two decoders;

[0097] The first decoder is used to output a first training result, the second decoder is used to output a second training result, and the encoder output is the input of the two decoders.

[0098] Optionally, the parameter adjustment module 130 includes:

[0099] a second adjustment unit, configured to adjust parameters of the encoder using the first loss value and the second loss value respectively;

[0100] a third adjustment unit, configured to adjust parameters of the first decoder using the first loss value;

[0101] The fourth adjustment unit is used to adjust parameters of the second decoder using the second loss value.

[0102] Optionally, the initial segmentation model includes two encoders and two decoders;

[0103] The input of the first encoder is the training image, the output of the first encoder is the input of the first decoder, the input of the second encoder is the training image and the output of the first decoder, and the output of the second encoder is the input of the second decoder;

[0104] The first decoder is used to output a first training result, and the second decoder is used to output a second training result.

[0105] Optionally, the parameter adjustment module 130 includes:

[0106] a fifth adjustment unit, configured to adjust parameters of the first encoder and the first decoder using the first loss value;

[0107] The sixth adjustment unit is configured to adjust parameters of the second encoder and the second decoder using the second loss value.

[0108] Optionally, the training acquisition module 110 includes:

[0109] An initial acquisition unit, used for acquiring an initial training image;

[0110] The augmentation or conversion processing unit is used to perform data augmentation processing or image conversion processing on the initial training image to obtain a training image.

[0111] The image processing device provided in an embodiment of the present application is introduced below. The image processing device described below and the image processing method described above can be referenced to each other.

[0112] Please refer to Figure 7 , Figure 7 A schematic structural diagram of an image processing device provided in an embodiment of the present application includes:

[0113] The test acquisition module 210 is used to acquire an image to be processed; wherein the image to be processed is a cross-sectional image of the levator ani muscle;

[0114] The image segmentation module 220 is used to input the image to be processed into an image segmentation model to obtain a segmented image of the levator ani hiatus; the image segmentation model is obtained based on the above-mentioned image segmentation model training method.

[0115] Optionally, it also includes:

[0116] a positioning module, configured to obtain key point positions in the image to be processed using a target image; wherein the target image includes the image to be processed and the levator ani hiatus segmented image, or includes the image to be processed, the levator ani hiatus segmented image, and a urethra region segmented image output by the image segmentation model;

[0117] The measurement module is used to generate measurement values ​​corresponding to preset measurement items according to the key point positions.

[0118] The electronic device provided in the embodiments of the present application is introduced below. The electronic device described below and the image segmentation model training method and / or the image processing method described above can be referenced to each other.

[0119] Please refer to Figure 8 , Figure 8 The electronic device 100 may include a processor 101 and a memory 102, and may further include one or more of a multimedia component 103, an information input / output (I / O) interface 104, and a communication component 105.

[0120] The processor 101 is used to control the overall operation of the electronic device 100 to complete the above-mentioned image segmentation model training method and / or all or part of the steps in the image processing method; the memory 102 is used to store various types of data to support the operation of the electronic device 100. These data may include, for example, instructions for any application or method operating on the electronic device 100, as well as application-related data. The memory 102 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0121] The multimedia component 103 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may be further stored in the memory 102 or sent through the communication component 105. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 104 provides an interface between the processor 101 and other interface modules. The above-mentioned other interface modules may be a keyboard, a mouse, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 105 is used for wired or wireless communication between the electronic device 100 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G or 4G, or a combination of one or more of them, so the corresponding communication component 105 may include: a Wi-Fi component, a Bluetooth component, an NFC component.

[0122] The electronic device 100 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to execute the image segmentation model training method and / or the image processing method given in the above embodiments.

[0123] The computer-readable storage medium provided in the embodiments of the present application is introduced below. The computer-readable storage medium described below and the image segmentation model training method and / or image processing method described above can be referenced to each other.

[0124] The present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned image segmentation model training method and / or the steps of the image processing method are implemented.

[0125] The computer-readable storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc., which can store program codes.

[0126] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from the other embodiments. Reference can be made to the descriptions of the identical or similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and the relevant parts can be referred to the descriptions of the methods.

[0127] Those skilled in the art may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0128] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0129] Finally, it should be noted that, in this document, relationships such as first and second, etc., are used solely to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms include, comprise, or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0130] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for training an image segmentation model, characterized in that: include: Acquire a training image and a corresponding training label; wherein the training image is a cross-sectional image of the levator ani muscle, and the training label includes a first label and a second label, the first label being used to characterize the range of the levator ani hiatus in the training image, and the second label being used to characterize the range of the urethra region in the training image, wherein the levator ani hiatus includes the urethra region; Segmenting the training image using the initial segmentation model to obtain a first training result and a second training result; the first training result is used to reflect the initial segmentation model's prediction of the levator ani hiatus, and the second training result is used to reflect the initial segmentation model's prediction of the urethra region; generating a loss value based on the training label, the first training result, and the second training result, and adjusting parameters of the initial segmentation model based on the loss value; If it is detected that the training completion condition is met, the initial segmentation model after parameter adjustment is determined as the image segmentation model; The initial segmentation model includes an encoder and a decoder; Alternatively, the initial segmentation model includes an encoder and two decoders, the first decoder is used to output a first training result, the second decoder is used to output a second training result, and the output of the encoder is the input of the two decoders; Alternatively, the initial segmentation model includes two encoders and two decoders, the input of the first encoder is the training image, the output of the first encoder is the input of the first decoder, the input of the second encoder is the training image and the output of the first decoder, the output of the second encoder is the input of the second decoder, the first decoder is used to output the first training result, and the second decoder is used to output the second training result.

2. The image segmentation model training method according to claim 1, characterized in that Generating a loss value based on the training label, the first training result, and the second training result includes: Generating a first loss value using the first label and the first training result; A second loss value is generated using the second label and the second training result.

3. The image segmentation model training method according to claim 2, characterized in that: The initial segmentation model includes an encoder and a decoder; The adjusting parameters of the initial segmentation model based on the loss value includes: The first loss value and the second loss value are used to adjust parameters of the encoder and the decoder respectively.

4. The image segmentation model training method according to claim 2, characterized in that: The initial segmentation model includes an encoder and two decoders; The adjusting parameters of the initial segmentation model based on the loss value includes: performing parameter adjustment on the encoder using the first loss value and the second loss value respectively; Using the first loss value to adjust parameters of the first decoder; The second loss value is used to adjust parameters of the second decoder.

5. The image segmentation model training method according to claim 2, characterized in that: The initial segmentation model includes two encoders and two decoders; The adjusting parameters of the initial segmentation model based on the loss value includes: Using the first loss value, adjusting parameters of the first encoder and the first decoder; The second loss value is used to adjust parameters of the second encoder and the second decoder.

6. The image segmentation model training method according to claim 1, characterized in that: The acquiring of the training image comprises: Get the initial training image; Performing data augmentation processing or image conversion processing on the initial training image to obtain the training image.

7. An image processing method, characterized in that: include: Acquire an image to be processed; wherein the image to be processed is a cross-sectional image of the levator ani muscle; The image to be processed is input into an image segmentation model to obtain a segmented image of the levator ani hiatus; the image segmentation model is obtained based on the image segmentation model training method according to any one of claims 1 to 6.

8. The image processing method according to claim 7, wherein: Also includes: Obtaining key point positions in the image to be processed using a target image; wherein the target image includes the image to be processed and the levator ani hiatus segmented image, or includes the image to be processed, the levator ani hiatus segmented image, and a urethra region segmented image output by the image segmentation model; Generate measurement values ​​corresponding to preset measurement items according to the key point positions.

9. An image segmentation model training device, characterized in that: include: a training acquisition module, configured to acquire a training image and corresponding training labels; wherein the training image is a cross-sectional image of the levator ani muscle, and the training labels include a first label and a second label, wherein the first label is used to characterize the range of the levator ani hiatus in the training image, and the second label is used to characterize the range of the urethra region in the training image, wherein the levator ani hiatus includes the urethra region; A training segmentation module is configured to segment the training image using an initial segmentation model to obtain a first training result and a second training result; the first training result is configured to reflect the initial segmentation model's prediction of the levator ani hiatus, and the second training result is configured to reflect the initial segmentation model's prediction of the urethra region; the initial segmentation model includes an encoder and a decoder; or, the initial segmentation model includes an encoder and two decoders, the first decoder is configured to output the first training result, the second decoder is configured to output the second training result, and the output of the encoder is input to the two decoders; or, the initial segmentation model includes two encoders and two decoders, the input of the first encoder is the training image, the output of the first encoder is input to the first decoder, the input of the second encoder is the training image and the output of the first decoder, the output of the second encoder is input to the second decoder, the first decoder is configured to output the first training result, and the second decoder is configured to output the second training result; a parameter adjustment module, configured to generate a loss value based on the training label, the first training result, and the second training result, and to adjust parameters of the initial segmentation model based on the loss value; The model determination module is used to determine the initial segmentation model after parameter adjustment as the image segmentation model if it is detected that the training completion condition is met.

10. An image processing device, characterized in that: include: The test acquisition module is used to acquire an image to be processed; wherein the image to be processed is a cross-sectional image of the levator ani muscle; An image segmentation module is used to input the image to be processed into an image segmentation model to obtain a segmented image of the levator ani hiatus; the image segmentation model is obtained based on the image segmentation model training method according to any one of claims 1 to 6.

11. An electronic device, characterized in that: comprising a memory and a processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program to implement the image segmentation model training method according to any one of claims 1 to 6, and / or the image processing method according to any one of claims 7 to 8.

12. A computer-readable storage medium, characterized in that Used to store a computer program, wherein when the computer program is executed by a processor, it implements the image segmentation model training method according to any one of claims 1 to 6, and / or the image processing method according to any one of claims 7 to 8.

Images