Artificial Intelligence-Based Semi-Supervised Image Segmentation Methods and Related Equipment

Abstract

This application relates to the fields of artificial intelligence and digital healthcare technology, proposing a semi-supervised image segmentation method, apparatus, electronic device, and storage medium based on artificial intelligence. The method includes: acquiring labeled and unlabeled images; constructing a student model and a teacher model; inputting the unlabeled image into the teacher model and student model to obtain a first teacher output and a first student output, and training the student model according to a first cost function; performing data augmentation on the unlabeled image and inputting it into the teacher model to obtain a second teacher output, and calculating a second cost function based on the first teacher output to train the teacher model; inputting the labeled image into the student model to obtain a second student output, and counting the number of incorrectly segmented pixels to obtain a segmentation error coefficient; constructing a third cost function based on the segmentation error coefficient to train the teacher model; when all cost functions are less than a preset value, the student model is used as the semi-supervised image segmentation model. This application can improve the accuracy of semi-supervised image segmentation.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and digital medical technology, and in particular to a semi-supervised image segmentation method and related equipment based on artificial intelligence. The related equipment includes a semi-supervised image segmentation device, electronic equipment and storage medium based on artificial intelligence. Background Technology

[0002] Image segmentation is one of the fundamental tasks in the field of artificial intelligence, with wide applications in many areas such as digital healthcare and smart cities. For example, in digital healthcare scenarios, images can be medical images. Semi-supervised learning-based image segmentation only requires a small number of manually labeled images and a large number of unlabeled images to achieve image segmentation, which can significantly reduce the manpower and financial costs associated with image annotation in practical applications.

[0003] Currently, semi-supervised learning often uses a teacher-student model, where the teacher model labels unlabeled data to guide the training of the student model. There are typically two ways to train the teacher model: 1) After training on labeled data, the model parameters are fixed, and labels are generated for unlabeled data during the student model's training; 2) The teacher model is trained together with the student model, and is used to calculate the consistency of unlabeled data segmentation with the student model. The first method is simpler and more convenient, but its drawback is that the teacher model is prone to overfitting on labeled datasets, resulting in an inability to generate accurate labels for unlabeled data. The second method is more widely used; however, simply constraining the consistency of the two models' outputs means that incorrect segmentation results from the student model cannot be fed back to the teacher model. Instead, it can easily mislead the teacher model, causing the performance of both models to gradually deteriorate during training, resulting in lower accuracy in semi-supervised image segmentation. Summary of the Invention

[0004] In view of the above, it is necessary to propose a semi-supervised image segmentation method and related equipment based on artificial intelligence to solve the technical problem of how to improve the accuracy of semi-supervised image segmentation. The related equipment includes an artificial intelligence-based semi-supervised image segmentation device, electronic equipment and storage medium.

[0005] This application provides a semi-supervised image segmentation method based on artificial intelligence, the method comprising:

[0006] S10, acquire multiple labeled images with labeled data and multiple unlabeled images without labeled data, wherein the labeled data includes the pixel type of each pixel in the labeled image, and the pixel type includes foreground and background;

[0007] S11, build student and teacher models;

[0008] S12, input any unlabeled image into the teacher model and the student model respectively to obtain the first teacher output and the first student output, and calculate the first cost function based on the first teacher output and the first student output to train the student model;

[0009] S13, perform data augmentation on the unlabeled image to obtain an augmented image, input the augmented image into the teacher model to obtain a second teacher output, and calculate a second cost function based on the first teacher output and the second teacher output to train the teacher model;

[0010] S14, input any label image into the student model to obtain the second student output, and count the number of incorrectly segmented pixels in the second student output to obtain the segmentation error coefficient, which includes the false negative coefficient and the false positive coefficient;

[0011] S15, Calculate the third cost function based on the segmentation error coefficient of the labeled image and the labeled data to train the teacher model;

[0012] S16, return to step S12, stop training when the values ​​of the first cost function, the second cost function and the third cost function are all less than the preset value, and use the student model as a semi-supervised model. The input of the semi-supervised model is the image to be segmented, and the output is the segmentation result of the image to be segmented.

[0013] In some embodiments, building the student model and teacher model includes:

[0014] The input to both the student model and the teacher model is an arbitrary image, and the expected output is a segmentation probability map of the image, which includes the probability that each pixel in the image belongs to the foreground.

[0015] The student model and the teacher model may have the same or different model structures.

[0016] In some embodiments, the first teacher output includes a first teacher probability for each pixel in the unlabeled image, and the first student output includes a first student probability for each pixel in the unlabeled image. The step of calculating a first cost function based on the first teacher output and the first student output to train the student model includes:

[0017] In the first teacher output, the pixel value of the pixel point where the probability of the first teacher is greater than the preset probability is set to 1, and the pixel value of the pixel point where the probability of the first teacher is not greater than the preset probability is set to 0, so as to obtain the first teacher label;

[0018] Based on the first teacher label and the first student output, a first cost function is calculated, which satisfies the following relationship:

[0019]

[0020] Among them, P w This represents all pixels in the unlabeled image. T represents the first student probability of pixel i in the unlabeled image. i 1 This represents the pixel value of pixel i in the first teacher label, and Loss1 is the value of the first cost function;

[0021] The student model is updated using stochastic gradient descent to reduce the value of the first cost function.

[0022] In some embodiments, the data enhancement is any one of flipping, brightness change, contrast change, saturation change, or adding Gaussian noise, and the enhanced image is the same size as the unlabeled image.

[0023] In some embodiments, the second teacher output includes the second teacher probability of each pixel in the unlabeled image, and the step of calculating the second cost function based on the first teacher output and the second teacher output to train the teacher model includes:

[0024] In the second teacher output, the pixel values ​​of pixels whose second teacher probability is greater than the preset probability are set to 1, and the pixel values ​​of pixels whose second teacher probability is not greater than the preset probability are set to 0, thus obtaining the second teacher label;

[0025] The second cost function is calculated based on the first teacher output and the second teacher label, and the second cost function satisfies the following relationship:

[0026]

[0027] Among them, P w This represents all pixels in the unlabeled image. T represents the probability of the first teacher for pixel i in the output of the first teacher. i 2 ε represents the pixel value of pixel i in the second teacher label, ε is the preset adjustment coefficient, and Loss2 is the value of the second cost function;

[0028] The teacher model is updated using stochastic gradient descent to reduce the value of the second cost function.

[0029] In some embodiments, the second student output includes the second student probability of each pixel in the label image, and the step of calculating the number of incorrectly segmented pixels in the second student output to obtain the segmentation error coefficient includes:

[0030] In the second student output, the pixel values ​​of pixels whose probability of the second student is greater than the preset probability are set to 1, and the pixel values ​​of pixels whose probability of the second student is not greater than the preset probability are set to 0, thus obtaining the student label;

[0031] The segmentation error coefficient is calculated based on the student labels and the label data of the label images. The segmentation error coefficient includes the false positive coefficient and the false negative coefficient.

[0032] The false positive coefficient is the number of pixels in the student label that misclassify the background as foreground, and the false positive coefficient satisfies the following relationship:

[0033]

[0034] Among them, P b For all pixels in the label image, S j Let B be the pixel value of pixel j in the student label. j FP is the pixel value of pixel j in the label data of the label image, and FP is the false positive coefficient;

[0035] The false negative coefficient is the number of pixels in the student label that misclassify the foreground as background, and the false negative coefficient satisfies the following relationship:

[0036]

[0037] Among them, P b For all pixels in the label image, S j Let B be the pixel value of pixel j in the student label. j Let be the pixel value of pixel j in the label data of the label image, and FN be the false negative coefficient.

[0038] In some embodiments, calculating the third cost function based on the segmentation error coefficient of the labeled image and the labeled data to train the teacher model includes:

[0039] The label image is input into the teacher model to obtain the third teacher output, which includes the third teacher probability of each pixel in the label image;

[0040] A first weighting factor and a second weighting factor are calculated based on the segmentation error coefficient, and the first weighting factor and the second weighting factor satisfy the following relationship:

[0041]

[0042] Wherein, FP and FN are the false positive coefficient and the false negative coefficient, respectively, and α and β are the first weighting factor and the second weighting factor, respectively;

[0043] A third cost function is calculated based on the third teacher output, the first weighting factor, the second weighting factor, and the label data. The third cost function satisfies the following relationship:

[0044]

[0045]

[0046] Among them, P b This represents all pixels in the label image. B represents the third teacher probability of pixel j in the labeled image. j The pixel value of pixel j in the label data of the label image is represented by Loss3, which is the value of the third cost function.

[0047] The teacher model is updated using stochastic gradient descent to reduce the value of the third cost function. This application also provides a semi-supervised image segmentation apparatus based on artificial intelligence, the apparatus comprising:

[0048] The acquisition unit is used to acquire multiple labeled images with labeled data and multiple unlabeled images without labeled data. The labeled data includes the pixel types of each pixel in the labeled image, and the pixel types include foreground and background.

[0049] Building units are used to build student and teacher models;

[0050] The first training unit is used to input any unlabeled image into the teacher model and the student model respectively to obtain the first teacher output and the first student output, and to calculate the first cost function based on the first teacher output and the first student output to train the student model.

[0051] The second training unit is used to perform data augmentation on the unlabeled image to obtain an augmented image, input the augmented image into the teacher model to obtain a second teacher output, and calculate a second cost function based on the first teacher output and the second teacher output to train the teacher model.

[0052] The statistical unit is used to input any labeled image into the student model to obtain the second student output, and to count the number of incorrectly segmented pixels in the second student output to obtain the segmentation error coefficient, which includes the false negative coefficient and the false positive coefficient.

[0053] The third training unit is used to calculate a third cost function based on the segmentation error coefficient of the labeled image and the labeled data to train the teacher model;

[0054] An iterative unit is used to return to the first training unit until the values ​​of the first cost function, the second cost function, and the third cost function are all less than a preset value, at which point training stops and the student model is used as a semi-supervised model. The input of the semi-supervised model is the image to be segmented, and the output is the segmentation result of the image to be segmented.

[0055] This application embodiment also provides an electronic device, the electronic device comprising:

[0056] Memory, storing at least one instruction;

[0057] The processor executes the instructions stored in the memory to implement the artificial intelligence-based semi-supervised image segmentation method.

[0058] This application also provides a computer-readable storage medium storing at least one instruction, which is executed by a processor in an electronic device to implement the artificial intelligence-based semi-supervised image segmentation method.

[0059] In summary, this application achieves semi-supervised image segmentation through a teacher model and a student model. The teacher model generates first teacher labels for unlabeled images to train the student model, and the labeled images are input into the student model to obtain the number of misclassified pixels as feedback information. This feedback information is then introduced into the training of the teacher model to control its sensitivity to misclassified pixels, enabling the teacher model to generate more accurate first teacher labels, thereby training a better-performing student model and improving its segmentation accuracy. This, in turn, improves the accuracy of semi-supervised image segmentation. This application can be applied to various functional modules of smart cities, such as smart government, smart transportation, and smart healthcare, including the semi-supervised image segmentation module in smart healthcare, thus promoting the rapid development of smart cities. Attached Figure Description

[0060] Figure 1 This is a flowchart of a preferred embodiment of the semi-supervised image segmentation method based on artificial intelligence involved in this application.

[0061] Figure 2 This is a functional block diagram of a preferred embodiment of the semi-supervised image segmentation apparatus based on artificial intelligence involved in this application.

[0062] Figure 3 This is a schematic diagram of the structure of an electronic device that is a preferred embodiment of the semi-supervised image segmentation method based on artificial intelligence involved in this application. Detailed Implementation

[0063] To better understand the purpose, features, and advantages of this application, a detailed description of the application is provided below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of this application can be combined with each other. Numerous specific details are set forth in the following description to provide a thorough understanding of this application; the described embodiments are only a part of the embodiments of this application, and not all of them.

[0064] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0065] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0066] This application provides a semi-supervised image segmentation method based on artificial intelligence, which can be applied to one or more electronic devices. An electronic device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0067] Electronic devices can be any electronic product that allows human-computer interaction with a customer, such as personal computers, tablets, smartphones, personal digital assistants (PDAs), game consoles, interactive network television (IPTV), smart wearable devices, etc.

[0068] Electronic devices may also include network devices and / or client devices. The network devices include, but are not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of hosts or network servers.

[0069] The networks in which electronic devices are located include, but are not limited to, the Internet, wide area networks, metropolitan area networks, local area networks, and virtual private networks (VPNs).

[0070] like Figure 1 The diagram shown is a flowchart of a preferred embodiment of the semi-supervised image segmentation method based on artificial intelligence according to this application. The order of steps in this flowchart can be changed, and some steps can be omitted, depending on different needs. The semi-supervised image segmentation method based on artificial intelligence provided in this application can be applied to any scenario requiring image segmentation, and thus can be applied to products in these scenarios, such as tumor segmentation in the medical field, lane line segmentation in the smart city field, etc.

[0071] S10: Collect multiple labeled images with labeled data and multiple unlabeled images without labeled data. The labeled data includes the pixel type of each pixel in the labeled image, and the pixel type includes foreground and background.

[0072] In an optional embodiment, the label data includes the pixel type of each pixel in the label image, and the number of pixel types is 2, that is, all pixels in the label image are divided into two different pixel types: foreground and background. The label data of the label image is obtained through manual annotation.

[0073] In this optional embodiment, for any labeled image, the pixel values ​​of the foreground pixels in the labeled image are set to 1, and the pixel values ​​of the background pixels in the labeled image are set to 0, thus obtaining the label data of the labeled image. In the digital healthcare scenario, the image can be a medical image, and the object contained in the image belongs to the type of lesion, that is, the part of the body where a disease has occurred. Medical images refer to internal tissues obtained non-invasively for medical treatment or medical research, such as images of the stomach, abdomen, heart, knee, and brain, such as CT (Computed Tomography), MRI (Magnetic Resonance Imaging), US (ultrasonic), X-ray images, electroencephalograms, and images generated by medical instruments such as optical imaging lamps. For example, in the scenario of tumor segmentation in the medical field, the pixels of the tumor area can be used as the foreground, and the pixels outside the tumor area can be used as the background, thereby segmenting the tumor area from the image.

[0074] In this way, multiple labeled images and multiple unlabeled images are obtained. The label data of the labeled images includes the pixel type of each pixel in the labeled image, which provides a data foundation for semi-supervised image segmentation.

[0075] S11, build student and teacher models.

[0076] In an optional embodiment, building the student model and teacher model includes:

[0077] The input to both the student model and the teacher model is an arbitrary image, and the expected output is a segmentation probability map of the image, which includes the probability that each pixel in the image belongs to the foreground.

[0078] The student model and the teacher model may have the same or different model structures.

[0079] Preferably, both the student model and the teacher model are parametric models. The teacher model can be an existing segmentation model such as Unet or DeepLab, while the student model can be a segmentation model built based on lightweight feature extraction models such as MobileNet or ShuffleNet. The number of parameters in the student model is much smaller than that in the teacher model. Therefore, the student model can obtain the segmentation probability map of the image more quickly. Furthermore, due to the smaller number of parameters in the student model, it can be embedded into mobile smart terminals such as mobile phones and tablets.

[0080] In this way, teacher and student models with the same input and output are obtained, providing a model foundation for semi-supervised image segmentation.

[0081] S12, input any unlabeled image into the teacher model and the student model respectively to obtain the first teacher output and the first student output, and calculate the first cost function based on the first teacher output and the first student output to train the student model.

[0082] In an optional embodiment, the first teacher output includes a first teacher probability for each pixel in the unlabeled image, and the first student output includes a first student probability for each pixel in the unlabeled image. The step of calculating a first cost function based on the first teacher output and the first student output to train the student model includes:

[0083] In the first teacher output, the pixel value of the pixel point where the probability of the first teacher is greater than the preset probability is set to 1, and the pixel value of the pixel point where the probability of the first teacher is not greater than the preset probability is set to 0, so as to obtain the first teacher label;

[0084] Based on the first teacher label and the first student output, a first cost function is calculated, which satisfies the following relationship:

[0085]

[0086] Among them, P w This represents all pixels in the unlabeled image. T represents the first student probability of pixel i in the unlabeled image. i 1 This represents the pixel value of pixel i in the first teacher label, and Loss1 is the value of the first cost function;

[0087] The student model is updated using stochastic gradient descent to reduce the value of the first cost function.

[0088] The preset probability is set to 0.5.

[0089] In this optional embodiment, The Dice loss function can be used to constrain the regional similarity between the output of the student model and the output of the teacher model. When the distribution of the foreground region in the first teacher output and the first student output is the same, the value of the Dice loss function is 0. The cross-entropy loss function can be used to constrain the consistency of pixel types between the output of the student model and the output of the teacher model. When the same pixel type is the same in the first teacher output and the first student output, the cross-entropy loss function takes the value of 0. Therefore, the first cost function can constrain the student model to learn the feature extraction ability of the teacher model, which can be regarded as the process of the teacher model guiding the training of the student model.

[0090] In this way, the student model is trained based on unlabeled images, and the student model is trained by the teacher model, so that the student model learns the feature extraction capabilities of the teacher model.

[0091] S13, perform data augmentation on the unlabeled image to obtain an augmented image, input the augmented image into the teacher model to obtain a second teacher output, and calculate a second cost function based on the first teacher output and the second teacher output to train the teacher model.

[0092] In an optional embodiment, the data enhancement is any one of flipping, brightness variation, contrast variation, saturation variation, or adding Gaussian noise, and the enhanced image is the same size as the unlabeled image.

[0093] In an optional embodiment, the second teacher output includes the second teacher probability for each pixel in the unlabeled image, and the step of calculating the second cost function based on the first teacher output and the second teacher output to train the teacher model includes:

[0094] In the second teacher output, the pixel values ​​of pixels whose second teacher probability is greater than the preset probability are set to 1, and the pixel values ​​of pixels whose second teacher probability is not greater than the preset probability are set to 0, thus obtaining the second teacher label;

[0095] The second cost function is calculated based on the first teacher output and the second teacher label, and the second cost function satisfies the following relationship:

[0096]

[0097] Among them, P w This represents all pixels in the unlabeled image. T represents the probability of the first teacher for pixel i in the output of the first teacher. i 2 ε represents the pixel value of pixel i in the second teacher label, ε is the preset adjustment coefficient, and Loss2 is the value of the second cost function;

[0098] The teacher model is updated using stochastic gradient descent to reduce the value of the second cost function.

[0099] The preset probability is 0.5, and the preset adjustment coefficient ε is a very small number set to prevent the denominator from being 0. The preset adjustment coefficient ε is 0.001.

[0100] In this optional embodiment, for the same unlabeled image, the unlabeled image is directly input into the teacher model to obtain the first teacher output; and the augmented image obtained after data augmentation is input into the teacher model to obtain the second teacher output; the second cost function is used to constrain that the pixel types of each pixel in the second teacher output and the first teacher output are the same. The data augmentation in this embodiment can be regarded as adding interference factors of the external environment. Through the second cost function, the teacher model can be unaffected by the interference of the external environment, ensuring the accuracy of the segmentation result and improving the robustness of the teacher model.

[0101] In this way, training the teacher model using unlabeled images ensures that the teacher model is not affected by the external environment, guarantees the accuracy of the segmentation results, and improves the robustness of the teacher model.

[0102] S14, input any label image into the student model to obtain the second student output, and count the number of incorrectly segmented pixels in the second student output to obtain the segmentation error coefficient, which includes the false negative coefficient and the false positive coefficient.

[0103] In an optional embodiment, the second student output includes the second student probability of each pixel in the label image, and the step of calculating the number of mis-segmented pixels in the second student output to obtain the segmentation error coefficient includes:

[0104] In the second student output, the pixel values ​​of pixels whose probability of the second student is greater than the preset probability are set to 1, and the pixel values ​​of pixels whose probability of the second student is not greater than the preset probability are set to 0, thus obtaining the student label;

[0105] The segmentation error coefficient is calculated based on the student labels and the label data of the label images. The segmentation error coefficient includes the false positive coefficient and the false negative coefficient.

[0106] The false positive coefficient is the number of pixels in the student label that misclassify the background as foreground, and the false positive coefficient satisfies the following relationship:

[0107]

[0108] Among them, P b For all pixels in the label image, S j Let B be the pixel value of pixel j in the student label. jFP is the pixel value of pixel j in the label data of the label image, and FP is the false positive coefficient;

[0109] The false negative coefficient is the number of pixels in the student label that misclassify the foreground as background, and the false negative coefficient satisfies the following relationship:

[0110]

[0111] Among them, P b For all pixels in the label image, S j Let B be the pixel value of pixel j in the student label. j Let be the pixel value of pixel j in the label data of the label image, and FN be the false negative coefficient.

[0112] In this optional embodiment, in the student labels and the label data of the label image, all pixels have a value of 0 or 1, where a pixel value of 1 indicates that the corresponding pixel is foreground and a pixel value of 0 indicates that the corresponding pixel is background. The label data of the label image is an accurate segmentation result annotated by humans, the student labels are the segmentation results predicted by the student model, and the false positive coefficient and the false negative coefficient can characterize the segmentation effect of the student model.

[0113] In this way, the label image is input into the student model to obtain the second student output, and the false negative coefficient and false positive coefficient are calculated by comparing the label data of the second student output and the label image, so as to accurately measure the segmentation effect of the student model.

[0114] S15, calculate the third cost function based on the segmentation error coefficient of the labeled image and the labeled data to train the teacher model.

[0115] In an optional embodiment, calculating the third cost function based on the segmentation error coefficients of the labeled images and the labeled data to train the teacher model includes:

[0116] The label image is input into the teacher model to obtain the third teacher output, which includes the third teacher probability of each pixel in the label image;

[0117] A first weighting factor and a second weighting factor are calculated based on the segmentation error coefficient, and the first weighting factor and the second weighting factor satisfy the following relationship:

[0118]

[0119] Wherein, FP and FN are the false positive coefficient and the false negative coefficient, respectively, and α and β are the first weighting factor and the second weighting factor, respectively;

[0120] A third cost function is calculated based on the third teacher output, the first weighting factor, the second weighting factor, and the label data. The third cost function satisfies the following relationship:

[0121]

[0122] Among them, P b This represents all pixels in the label image. B represents the third teacher probability of pixel j in the labeled image. j The pixel value of pixel j in the label data of the label image is represented by Loss3, which is the value of the third cost function.

[0123] The teacher model is updated using stochastic gradient descent to reduce the value of the third cost function.

[0124] In this optional embodiment, the false positive coefficient FP and false negative coefficient FN are used as feedback signals in the construction of the third cost function. The training process of the teacher model is guided by the segmentation performance of the student model. During the training process, the teacher model's sensitivity to false positive and false negative pixels is balanced, allowing the teacher model to better learn the features of false positive and false negative pixels, thus improving the segmentation accuracy of the teacher model. This enables the teacher model to generate more accurate first teacher labels in step S12, resulting in a student model with better segmentation performance. Here, the false positive pixel is a pixel in the student label that misclassifies the background as foreground; the false negative pixel is a pixel in the student label that misclassifies the foreground as background.

[0125] It should be noted that, especially in image segmentation scenarios involving small targets, when the student model misses small targets, the false negative coefficient becomes larger. This leads to an increase in the second weight factor corresponding to the false negative coefficient in the teacher model's loss function, causing the teacher model to pay more attention to the missed small targets during training and improve the segmentation accuracy of the pixels corresponding to the false negatives.

[0126] Thus, the segmentation error coefficient is introduced as feedback information from the student model into the training of the teacher model. During training, the sensitivity of the teacher model to false positive and false negative pixels is balanced, thereby improving the segmentation accuracy of the teacher model for false positive and false negative pixels. This can improve the accuracy of the first teacher label in step S12, thereby improving the segmentation accuracy of the student model.

[0127] S16, return to step S12, stop training when the values ​​of the first cost function, the second cost function and the third cost function are all less than the preset value, and use the student model as a semi-supervised model. The input of the semi-supervised model is the image to be segmented, and the output is the segmentation result of the image to be segmented.

[0128] In an optional embodiment, the student model and the teacher model are trained using arbitrary unlabeled images and arbitrary labeled images. The process then returns to step S12 to continuously train the student model and the teacher model until the values ​​of the first cost function, the second cost function, and the third cost function are all less than a preset value. Training then stops, and the student model is used as a semi-supervised model. The preset value is 0.01.

[0129] In an optional embodiment, the semi-supervised model takes an image to be segmented as input and outputs an accurate segmentation probability map of the image to be segmented. The segmentation probability map includes the probability that each pixel in the image belongs to the foreground. Pixels with a probability greater than a preset probability are designated as foreground, and pixels with a probability not greater than the preset probability are designated as background, thereby obtaining the segmentation result of the image to be segmented.

[0130] Thus, the training process of the student model and the teacher model is completed. The trained student model is used as a semi-supervised model, which can obtain accurate segmentation results of the image to be segmented, thereby realizing semi-supervised image segmentation.

[0131] As can be seen from the above technical solution, this application achieves semi-supervised image segmentation through a teacher model and a student model. The teacher model generates a first teacher label for unlabeled images to train the student model, and the labeled images are input into the student model to obtain the number of misclassified pixels as feedback information. This feedback information is introduced into the training of the teacher model to control its sensitivity to misclassified pixels, enabling the teacher model to generate more accurate first teacher labels, thereby training a better-performing student model and improving the segmentation accuracy of the student model, thus improving the accuracy of semi-supervised image segmentation. This application can be applied to various functional modules of smart cities, such as smart government, smart transportation, and smart healthcare, including the semi-supervised image segmentation module in smart healthcare, and can promote the rapid development of smart cities.

[0132] Please see Figure 2 , Figure 2This is a functional block diagram of a preferred embodiment of the semi-supervised image segmentation device based on artificial intelligence according to this application. The semi-supervised image segmentation device 11 based on artificial intelligence includes an acquisition unit 110, a construction unit 111, a first training unit 112, a second training unit 113, a statistics unit 114, a third training unit 115, and an iteration unit 116. The module / unit referred to in this application refers to a series of computer-readable instruction segments that can be executed by the processor 13 and perform a fixed function, and are stored in the memory 12. In this embodiment, the functions of each module / unit will be described in detail in subsequent embodiments.

[0133] In an optional embodiment, the acquisition unit 110 is used to acquire multiple labeled images with labeled data and multiple unlabeled images without labeled data. The labeled data includes the pixel type of each pixel in the labeled image, and the pixel type includes foreground and background.

[0134] Unit 111 is used to build student and teacher models.

[0135] In some embodiments, building the student model and teacher model includes:

[0136] The input to both the student model and the teacher model is an arbitrary image, and the expected output is a segmentation probability map of the image, which includes the probability that each pixel in the image belongs to the foreground.

[0137] The student model and the teacher model may have the same or different model structures.

[0138] The first training unit 112 is used to input any unlabeled image into the teacher model and the student model respectively to obtain a first teacher output and a first student output, and to calculate a first cost function based on the first teacher output and the first student output to train the student model.

[0139] In some embodiments, the first teacher output includes a first teacher probability for each pixel in the unlabeled image, and the first student output includes a first student probability for each pixel in the unlabeled image. The step of calculating a first cost function based on the first teacher output and the first student output to train the student model includes:

[0140] In the first teacher output, the pixel value of the pixel point where the probability of the first teacher is greater than the preset probability is set to 1, and the pixel value of the pixel point where the probability of the first teacher is not greater than the preset probability is set to 0, so as to obtain the first teacher label;

[0141] Based on the first teacher label and the first student output, a first cost function is calculated, which satisfies the following relationship:

[0142]

[0143] Among them, P w This represents all pixels in the unlabeled image. T represents the first student probability of pixel i in the unlabeled image. i 1 This represents the pixel value of pixel i in the first teacher label, and Loss1 is the value of the first cost function;

[0144] The student model is updated using stochastic gradient descent to reduce the value of the first cost function.

[0145] The second training unit 113 is used to perform data augmentation on the unlabeled image to obtain an augmented image, input the augmented image into the teacher model to obtain a second teacher output, and calculate a second cost function based on the first teacher output and the second teacher output to train the teacher model.

[0146] In some embodiments, the data enhancement is any one of flipping, brightness change, contrast change, saturation change, or adding Gaussian noise, and the enhanced image is the same size as the unlabeled image.

[0147] In some embodiments, the second teacher output includes the second teacher probability of each pixel in the unlabeled image, and the step of calculating the second cost function based on the first teacher output and the second teacher output to train the teacher model includes:

[0148] In the second teacher output, the pixel values ​​of pixels whose second teacher probability is greater than the preset probability are set to 1, and the pixel values ​​of pixels whose second teacher probability is not greater than the preset probability are set to 0, thus obtaining the second teacher label;

[0149] The second cost function is calculated based on the first teacher output and the second teacher label, and the second cost function satisfies the following relationship:

[0150]

[0151] Among them, P w This represents all pixels in the unlabeled image. T represents the probability of the first teacher for pixel i in the output of the first teacher. i 2 ε represents the pixel value of pixel i in the second teacher label, ε is the preset adjustment coefficient, and Loss2 is the value of the second cost function;

[0152] The teacher model is updated using stochastic gradient descent to reduce the value of the second cost function.

[0153] The statistics unit 114 is used to input any label image into the student model to obtain the second student output, and to count the number of pixels with segmentation errors in the second student output to obtain the segmentation error coefficient, which includes the false negative coefficient and the false positive coefficient.

[0154] In some embodiments, the second student output includes the second student probability of each pixel in the label image, and the step of calculating the number of incorrectly segmented pixels in the second student output to obtain the segmentation error coefficient includes:

[0155] In the second student output, the pixel values ​​of pixels whose probability of the second student is greater than the preset probability are set to 1, and the pixel values ​​of pixels whose probability of the second student is not greater than the preset probability are set to 0, thus obtaining the student label;

[0156] The segmentation error coefficient is calculated based on the student labels and the label data of the label images. The segmentation error coefficient includes the false positive coefficient and the false negative coefficient.

[0157] The false positive coefficient is the number of pixels in the student label that misclassify the background as foreground, and the false positive coefficient satisfies the following relationship:

[0158]

[0159] Among them, P b For all pixels in the label image, S j Let B be the pixel value of pixel j in the student label. j FP is the pixel value of pixel j in the label data of the label image, and FP is the false positive coefficient;

[0160] The false negative coefficient is the number of pixels in the student label that misclassify the foreground as background, and the false negative coefficient satisfies the following relationship:

[0161]

[0162] Among them, P b For all pixels in the label image, S j Let B be the pixel value of pixel j in the student label. j Let be the pixel value of pixel j in the label data of the label image, and FN be the false negative coefficient.

[0163] The third training unit 115 is used to calculate a third cost function based on the segmentation error coefficient of the labeled image and the labeled data to train the teacher model.

[0164] In some embodiments, calculating the third cost function based on the segmentation error coefficient of the labeled image and the labeled data to train the teacher model includes:

[0165] The label image is input into the teacher model to obtain the third teacher output, which includes the third teacher probability of each pixel in the label image;

[0166] A first weighting factor and a second weighting factor are calculated based on the segmentation error coefficient, and the first weighting factor and the second weighting factor satisfy the following relationship:

[0167]

[0168] Wherein, FP and FN are the false positive coefficient and the false negative coefficient, respectively, and α and β are the first weighting factor and the second weighting factor, respectively;

[0169] A third cost function is calculated based on the third teacher output, the first weighting factor, the second weighting factor, and the label data. The third cost function satisfies the following relationship:

[0170]

[0171]

[0172] Among them, P b This represents all pixels in the label image. B represents the third teacher probability of pixel j in the labeled image. j The pixel value of pixel j in the label data of the label image is represented by Loss3, which is the value of the third cost function.

[0173] The teacher model is updated using stochastic gradient descent to reduce the value of the third cost function.

[0174] The iteration unit 116 is used to return to the first training unit until the values ​​of the first cost function, the second cost function and the third cost function are all less than a preset value, and then stop training. The student model is used as a semi-supervised model, the input of which is the image to be segmented and the output is the segmentation result of the image to be segmented.

[0175] As can be seen from the above technical solutions, this application achieves semi-supervised image segmentation through a teacher model and a student model. The teacher model generates the first teacher label for the unlabeled image to train the student model, and the labeled image is input into the student model to obtain the number of misclassified pixels as feedback information. The feedback information is introduced into the training of the teacher model to control the sensitivity of the teacher model to misclassified pixels, so that the teacher model generates more accurate first teacher labels, thereby training a better-performing student model, improving the segmentation accuracy of the student model, and thus improving the accuracy of semi-supervised image segmentation.

[0176] Please see Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 1 includes a memory 12 and a processor 13. The memory 12 is used to store computer-readable instructions, and the processor 13 executes the computer-readable instructions stored in the memory to implement the semi-supervised image segmentation method based on artificial intelligence as described in any of the above embodiments.

[0177] In an alternative embodiment, the electronic device 1 further includes a bus and a computer program stored in the memory 12 and executable on the processor 13, such as an AI-based semi-supervised image segmentation program.

[0178] Figure 3 Only electronic device 1 with memory 12 and processor 13 is shown. It will be understood by those skilled in the art that... Figure 3 The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0179] Combination Figure 1 The memory 12 in the electronic device 1 stores a plurality of computer-readable instructions to implement a semi-supervised image segmentation method based on artificial intelligence, and the processor 13 can execute the plurality of instructions to achieve:

[0180] S10, acquire multiple labeled images with labeled data and multiple unlabeled images without labeled data, wherein the labeled data includes the pixel type of each pixel in the labeled image, and the pixel type includes foreground and background;

[0181] S11, build student and teacher models;

[0182] S12, input any unlabeled image into the teacher model and the student model respectively to obtain the first teacher output and the first student output, and calculate the first cost function based on the first teacher output and the first student output to train the student model;

[0183] S13, perform data augmentation on the unlabeled image to obtain an augmented image, input the augmented image into the teacher model to obtain a second teacher output, and calculate a second cost function based on the first teacher output and the second teacher output to train the teacher model;

[0184] S14, input any label image into the student model to obtain the second student output, and count the number of incorrectly segmented pixels in the second student output to obtain the segmentation error coefficient, which includes the false negative coefficient and the false positive coefficient;

[0185] S15, Calculate the third cost function based on the segmentation error coefficient of the labeled image and the labeled data to train the teacher model;

[0186] S16, return to step S12, stop training when the values ​​of the first cost function, the second cost function and the third cost function are all less than the preset value, and use the student model as a semi-supervised model. The input of the semi-supervised model is the image to be segmented, and the output is the segmentation result of the image to be segmented.

[0187] Specifically, the processor 13's implementation method for the above instructions can be found in [reference needed]. Figure 1 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.

[0188] Those skilled in the art will understand that the schematic diagram is merely an example of electronic device 1 and does not constitute a limitation on electronic device 1. Electronic device 1 can be a bus-type structure or a star-type structure. Electronic device 1 may also include more or fewer other hardware or software than shown in the diagram, or different component arrangements. For example, electronic device 1 may also include input / output devices, network access devices, etc.

[0189] It should be noted that electronic device 1 is only an example. Other existing or future electronic products that are suitable for this application should also be included within the scope of protection of this application and are incorporated herein by reference.

[0190] The memory 12 includes at least one type of readable storage medium, which can be non-volatile or volatile. The readable storage medium includes flash memory, portable hard drives, multimedia cards, card-type memory (e.g., SD or DX memory), magnetic storage, magnetic disks, optical disks, etc. In some embodiments, the memory 12 can be an internal storage unit of the electronic device 1, such as a portable hard drive of the electronic device 1. In other embodiments, the memory 12 can also be an external storage device of the electronic device 1, such as a plug-in portable hard drive, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc., equipped on the electronic device 1. The memory 12 can be used not only to store application software and various types of data installed on the electronic device 1, such as the code of a semi-supervised image segmentation program based on artificial intelligence, but also to temporarily store data that has been output or will be output.

[0191] In some embodiments, the processor 13 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits packaged with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 13 is the control unit of the electronic device 1, connecting various components of the electronic device 1 via various interfaces and lines. It executes programs or modules stored in the memory 12 (e.g., executing semi-supervised image segmentation programs based on artificial intelligence) and calls data stored in the memory 12 to perform various functions and process data in the electronic device 1.

[0192] The processor 13 executes the operating system of the electronic device 1 and various installed applications. The processor 13 executes these applications to implement the steps in the various embodiments of the artificial intelligence-based semi-supervised image segmentation method described above, for example... Figure 1 The steps are shown.

[0193] For example, the computer program may be divided into one or more modules / units, which are stored in the memory 12 and executed by the processor 13 to complete this application. The one or more modules / units may be a series of computer-readable instruction segments capable of performing specific functions, which describe the execution process of the computer program in the electronic device 1. For example, the computer program may be divided into a data acquisition unit 110, a data construction unit 111, a first training unit 112, a second training unit 113, a statistics unit 114, a third training unit 115, and an iteration unit 116.

[0194] The integrated unit implemented as a software functional module described above can be stored in a computer-readable storage medium. This software functional module, stored in a storage medium, includes several instructions to cause a computer device (which may be a personal computer, computer equipment, or network device, etc.) or processor to execute portions of the artificial intelligence-based semi-supervised image segmentation method described in the various embodiments of this application.

[0195] If the modules / units integrated in electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware devices. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above.

[0196] The computer program includes computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory, and other memory.

[0197] Furthermore, the computer-readable storage medium may primarily include a stored program area and a stored data area, wherein the stored program area may store the operating system, an application program required for at least one function, etc.; and the stored data area may store data created based on the use of blockchain nodes, etc.

[0198] The blockchain referred to in this application is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0199] The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, in... Figure 3 The symbol is represented by only one arrow, but this does not indicate that there is only one bus or one type of bus. The bus is configured to enable communication between the memory 12 and at least one processor 13, etc.

[0200] This application also provides a computer-readable storage medium (not shown) storing computer-readable instructions, which are executed by a processor in an electronic device to implement the semi-supervised image segmentation method based on artificial intelligence as described in any of the above embodiments.

[0201] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0202] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0203] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0204] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices described in the specification may also be implemented by a single unit or device through software or hardware. Terms such as "first," "second," etc., are used to indicate names and do not indicate any specific order.

[0205] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application and are not intended to limit it. Although this application has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of this application without departing from the spirit and scope of the technical solutions of this application.

Claims

1. A semi-supervised image segmentation method based on artificial intelligence, characterized in that, The method includes: S10, acquire multiple labeled images with labeled data and multiple unlabeled images without labeled data, wherein the labeled data includes the pixel type of each pixel in the labeled image, and the pixel type includes foreground and background; S11, build student and teacher models; S12, input any unlabeled image into the teacher model and the student model respectively to obtain the first teacher output and the first student output, and calculate the first cost function based on the first teacher output and the first student output to train the student model; S13, perform data augmentation on the unlabeled image to obtain an augmented image, input the augmented image into the teacher model to obtain a second teacher output, and calculate a second cost function based on the first teacher output and the second teacher output to train the teacher model; S14, input any label image into the student model to obtain the second student output, and count the number of incorrectly segmented pixels in the second student output to obtain the segmentation error coefficient, which includes the false negative coefficient and the false positive coefficient; S15, Calculate the third cost function based on the segmentation error coefficient of the labeled image and the labeled data to train the teacher model; S16, return to step S12, stop training when the values ​​of the first cost function, the second cost function and the third cost function are all less than the preset value, and use the student model as a semi-supervised model. The input of the semi-supervised model is the image to be segmented, and the output is the segmentation result of the image to be segmented.

2. The semi-supervised image segmentation method based on artificial intelligence as described in claim 1, characterized in that, The construction of the student and teacher models includes: The input to both the student model and the teacher model is an arbitrary image, and the expected output is a segmentation probability map of the image, which includes the probability that each pixel in the image belongs to the foreground. The student model and the teacher model may have the same or different model structures.

3. The semi-supervised image segmentation method based on artificial intelligence as described in claim 1, characterized in that, The first teacher output includes the first teacher probability for each pixel in the unlabeled image, and the first student output includes the first student probability for each pixel in the unlabeled image. The step of calculating the first cost function based on the first teacher output and the first student output to train the student model includes: In the first teacher output, the pixel value of the pixel point where the probability of the first teacher is greater than the preset probability is set to 1, and the pixel value of the pixel point where the probability of the first teacher is not greater than the preset probability is set to 0, so as to obtain the first teacher label; Based on the first teacher label and the first student output, a first cost function is calculated, which satisfies the following relationship: Among them, P w This represents all pixels in the unlabeled image. T represents the first student probability of pixel i in the unlabeled image. i 1 This represents the pixel value of pixel i in the first teacher label, and Loss1 is the value of the first cost function; The student model is updated using stochastic gradient descent to reduce the value of the first cost function.

4. The semi-supervised image segmentation method based on artificial intelligence as described in claim 1, characterized in that, The data enhancement is any one of flipping, brightness change, contrast change, saturation change, or adding Gaussian noise, and the enhanced image is the same size as the unlabeled image.

5. The semi-supervised image segmentation method based on artificial intelligence as described in claim 1, characterized in that, The second teacher output includes the second teacher probability for each pixel in the unlabeled image, and the step of calculating the second cost function based on the first teacher output and the second teacher output to train the teacher model includes: In the second teacher output, the pixel values ​​of pixels whose second teacher probability is greater than the preset probability are set to 1, and the pixel values ​​of pixels whose second teacher probability is not greater than the preset probability are set to 0, thus obtaining the second teacher label; The second cost function is calculated based on the first teacher output and the second teacher label, and the second cost function satisfies the following relationship: Among them, P w This represents all pixels in the unlabeled image. T represents the probability of the first teacher for pixel i in the output of the first teacher. i 2 ε represents the pixel value of pixel i in the second teacher label, ε is the preset adjustment coefficient, and Loss2 is the value of the second cost function; The teacher model is updated using stochastic gradient descent to reduce the value of the second cost function.

6. The semi-supervised image segmentation method based on artificial intelligence as described in claim 1, characterized in that, The second student output includes the second student probability of each pixel in the label image. The step of calculating the number of incorrectly segmented pixels in the second student output to obtain the segmentation error coefficient includes: In the second student output, the pixel values ​​of pixels whose probability of the second student is greater than the preset probability are set to 1, and the pixel values ​​of pixels whose probability of the second student is not greater than the preset probability are set to 0, thus obtaining the student label; The segmentation error coefficient is calculated based on the student labels and the label data of the label images. The segmentation error coefficient includes the false positive coefficient and the false negative coefficient. The false positive coefficient is the number of pixels in the student label that misclassify the background as foreground, and the false positive coefficient satisfies the following relationship: Among them, P b For all pixels in the label image, S j Let B be the pixel value of pixel j in the student label. j FP is the pixel value of pixel j in the label data of the label image, and FP is the false positive coefficient; The false negative coefficient is the number of pixels in the student label that misclassify the foreground as background, and the false negative coefficient satisfies the following relationship: Among them, P b For all pixels in the label image, S j Let B be the pixel value of pixel j in the student label. j Let be the pixel value of pixel j in the label data of the label image, and FN be the false negative coefficient.

7. The semi-supervised image segmentation method based on artificial intelligence as described in claim 1, characterized in that, The step of calculating the third cost function based on the segmentation error coefficient of the labeled image and the labeled data to train the teacher model includes: The label image is input into the teacher model to obtain the third teacher output, which includes the third teacher probability of each pixel in the label image; A first weighting factor and a second weighting factor are calculated based on the segmentation error coefficient, and the first weighting factor and the second weighting factor satisfy the following relationship: Wherein, FP and FN are the false positive coefficient and the false negative coefficient, respectively, and α and β are the first weighting factor and the second weighting factor, respectively; A third cost function is calculated based on the third teacher output, the first weighting factor, the second weighting factor, and the label data. The third cost function satisfies the following relationship: Among them, P b This represents all pixels in the label image. B represents the third teacher probability of pixel j in the labeled image. j The pixel value of pixel j in the label data of the label image is represented by Loss3, which is the value of the third cost function. The teacher model is updated using stochastic gradient descent to reduce the value of the third cost function.

8. A semi-supervised image segmentation device based on artificial intelligence, characterized in that, The device includes: The acquisition unit is used to acquire multiple labeled images with labeled data and multiple unlabeled images without labeled data. The labeled data includes the pixel types of each pixel in the labeled image, and the pixel types include foreground and background. Building units are used to build student and teacher models; The first training unit is used to input any unlabeled image into the teacher model and the student model respectively to obtain the first teacher output and the first student output, and to calculate the first cost function based on the first teacher output and the first student output to train the student model. The second training unit is used to perform data augmentation on the unlabeled image to obtain an augmented image, input the augmented image into the teacher model to obtain a second teacher output, and calculate a second cost function based on the first teacher output and the second teacher output to train the teacher model. The statistical unit is used to input any labeled image into the student model to obtain the second student output, and to count the number of incorrectly segmented pixels in the second student output to obtain the segmentation error coefficient, which includes the false negative coefficient and the false positive coefficient. The third training unit is used to calculate a third cost function based on the segmentation error coefficient of the labeled image and the labeled data to train the teacher model; An iterative unit is used to return to the first training unit until the values ​​of the first cost function, the second cost function, and the third cost function are all less than a preset value, at which point training stops and the student model is used as a semi-supervised model. The input of the semi-supervised model is the image to be segmented, and the output is the segmentation result of the image to be segmented.

9. An electronic device, characterized in that, The electronic device includes: Memory, which stores computer-readable instructions; and The processor executes computer-readable instructions stored in the memory to implement the AI-based semi-supervised image segmentation method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the semi-supervised image segmentation method based on artificial intelligence as described in any one of claims 1 to 7.

Images