Semantic segmentation model training method and device, equipment, storage medium and product

Abstract

The invention relates to a semantic segmentation model training method and device, equipment, a storage medium and a product. The method comprises the following steps: acquiring a source domain image set and an enhanced image set; wherein the source domain image set comprises at least two images in the first color space; the enhanced image set comprises at least two images in a second color space; the color gamut of the second color space is wider than that of the first color space; processing the source domain image set and the enhanced image set based on a teacher model to obtain a first processing result; and based on a student model, processing the source domain image set and the enhanced image set to obtain a second processing result, and adjusting model parameters of the student model according to the first processing result and the second processing result. By adopting the method, the model precision of the semantic segmentation model can be improved.

Description

Technical Field

[0001] The present application relates to the field of semantic segmentation technology, and in particular to a semantic segmentation model training method, device, equipment, storage medium and product. Background Art

[0002] Semantic segmentation models are designed to classify each pixel in an image and are widely used in autonomous driving, medical imaging, image editing, etc. In order to train an advanced semantic segmentation model, a large amount of pixel-level annotation data is required as support, but the acquisition of such data is very expensive and time-consuming.

[0003] In this case, domain adaptation semantic segmentation technology, which can effectively improve cross-domain segmentation performance, has gradually become an important research topic. However, due to the large differences in style and statistics between the source domain and the target domain data, it is easy to cause a serious decline in model performance. Summary of the invention

[0004] Based on this, it is necessary to provide a semantic segmentation model training method, device, equipment, storage medium and product that can improve the model accuracy of the semantic segmentation model in response to the above technical problems.

[0005] In a first aspect, the present application provides a semantic segmentation model training method, comprising:

[0006] Acquire a source domain image set and an enhanced image set; wherein the source domain image set includes at least two images in a first color space; the enhanced image set includes at least two images in a second color space; and the color gamut of the second color space is wider than the color gamut of the first color space;

[0007] Based on the teacher model, the source domain image set and the enhanced image set are processed to obtain a first processing result; and

[0008] Based on the student model, the source domain image set and the enhanced image set are processed to obtain a second processing result;

[0009] According to the first processing result and the second processing result, the model parameters of the student model are adjusted.

[0010] In one embodiment, based on a teacher model, a source domain image set and an enhanced image set are processed to obtain a first processing result, including: inputting the source domain image set into the teacher model to obtain a first output result; and, inputting the enhanced image set into the teacher model to obtain a second output result; wherein the first processing result includes a first output result and a second output result.

[0011] In one embodiment, based on the student model, the source domain image set and the enhanced image set are processed to obtain a second processing result, including: inputting the source domain image set into the student model to obtain a third output result; and, inputting the enhanced image set into the student model to obtain a fourth output result; wherein the second processing result includes the third output result and the fourth output result.

[0012] In one embodiment, the model parameters of the student model are adjusted according to the first processing result and the second processing result, including: determining a first cross entropy loss according to the first output result and the fourth output result; and determining a second cross entropy loss according to the second output result and the third output result; determining a first loss value according to the first cross entropy loss and the second cross entropy loss; and adjusting the model parameters of the student model according to the first loss value.

[0013] In one of the embodiments, the method also includes: obtaining a target domain image set; wherein the target domain image set includes at least two images in a first color space; the images in the target domain image set and the images in the source domain image set and the enhanced image set belong to different types of images; constructing a cross-domain mixed image set based on the target domain image set and the source domain image set; processing the cross-domain mixed image set based on the teacher model to obtain a third processing result; and, processing the cross-domain mixed image set based on the student model to obtain a fourth processing result; determining a second loss value based on the third processing result and the fourth processing result; and adjusting the model parameters of the student model based on the second loss value.

[0014] In one of the embodiments, the method further includes: obtaining an image processing result corresponding to each source domain image in the first processing result; determining a third loss value based on a segmentation label map corresponding to the image processing result and the corresponding source domain image; and adjusting the model parameters of the student model based on the third loss value.

[0015] In a second aspect, the present application also provides a semantic segmentation model training device, comprising:

[0016] An image acquisition module is used to acquire a source domain image set and an enhanced image set; wherein the source domain image set includes at least two images in a first color space; the enhanced image set includes at least two images in a second color space; and the color gamut of the second color space is wider than the color gamut of the first color space;

[0017] A first image processing module is used to process the source domain image set and the enhanced image set based on the teacher model to obtain a first processing result; and

[0018] A second image processing module is used to process the source domain image set and the enhanced image set based on the student model to obtain a second processing result;

[0019] The first model training module is used to adjust the model parameters of the student model according to the first processing result and the second processing result.

[0020] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0021] Acquire a source domain image set and an enhanced image set; wherein the source domain image set includes at least two images in a first color space; the enhanced image set includes at least two images in a second color space; and the color gamut of the second color space is wider than the color gamut of the first color space;

[0022] Based on the teacher model, the source domain image set and the enhanced image set are processed to obtain a first processing result; and

[0023] Based on the student model, the source domain image set and the enhanced image set are processed to obtain a second processing result;

[0024] According to the first processing result and the second processing result, the model parameters of the student model are adjusted.

[0025] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the following steps are implemented:

[0026] Acquire a source domain image set and an enhanced image set; wherein the source domain image set includes at least two images in a first color space; the enhanced image set includes at least two images in a second color space; and the color gamut of the second color space is wider than the color gamut of the first color space;

[0027] Based on the teacher model, the source domain image set and the enhanced image set are processed to obtain a first processing result; and

[0028] Based on the student model, the source domain image set and the enhanced image set are processed to obtain a second processing result;

[0029] According to the first processing result and the second processing result, the model parameters of the student model are adjusted.

[0030] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0031] Acquire a source domain image set and an enhanced image set; wherein the source domain image set includes at least two images in a first color space; the enhanced image set includes at least two images in a second color space; and the color gamut of the second color space is wider than the color gamut of the first color space;

[0032] Based on the teacher model, the source domain image set and the enhanced image set are processed to obtain a first processing result; and

[0033] Based on the student model, the source domain image set and the enhanced image set are processed to obtain a second processing result;

[0034] According to the first processing result and the second processing result, the model parameters of the student model are adjusted.

[0035] The above-mentioned semantic segmentation model training method, device, equipment, storage medium and product, in the process of adjusting the model parameters of the student model, not only processes the source domain image set based on the teacher model and the student model, but also processes the enhanced image set based on the teacher model and the student model. Furthermore, the model parameters of the student model are adjusted according to the first processing result obtained after the teacher model processes the source domain image set and the enhanced image set, and according to the second processing result obtained after the student model processes the source domain image set and the enhanced image set. In this process, the introduction of the enhanced image set to train the student model instead of the traditional adversarial training method can increase the model's understanding of category information, thereby improving the model accuracy of the semantic segmentation model. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1 A flowchart of a semantic segmentation model training method in one embodiment;

[0038] Figure 2 A flowchart of a semantic segmentation model training step in one embodiment;

[0039] Figure 3 A flowchart of a semantic segmentation model training step in another embodiment;

[0040] Figure 4 A flowchart of a semantic segmentation model training method in another embodiment;

[0041] Figure 5 is a structural block diagram of a semantic segmentation model training device in one embodiment;

[0042] Figure 6 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0044] Before introducing the semantic segmentation model training method provided in this application, it should be noted that semantic segmentation aims to classify each pixel of the image and is widely used in autonomous driving, medical imaging, image editing and other fields. In order to train an advanced semantic segmentation model, a large amount of pixel-level annotation data is required as support, but the acquisition of such data is very expensive and time-consuming. For example, if you want to carefully annotate an image with a pixel size of 1024*2048, it takes about 90 minutes. Therefore, domain adaptation semantic segmentation technology, which can effectively improve cross-domain segmentation performance, has gradually become an important research topic. However, due to the large differences in style and statistics between source domain images and target domain images, the performance of the semantic segmentation model will be seriously reduced. Current research in the field of semantic segmentation technology has made remarkable progress, but there is still much room for improvement.

[0045] In one embodiment, Figure 1 As shown, a semantic segmentation model training method is provided. This embodiment uses the method applied to a terminal as an example. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0046] S110, obtaining a source domain image set and an enhanced image set.

[0047] The source domain image set includes at least two images in a first color space; the enhanced image set includes at least two images in a second color space; the color gamut of the second color space is wider than the color gamut of the first color space. For example, the first color space may be a red, green, blue (RGB) color space, and the second color space may be Color space, this application does not impose any limitation on this.

[0048] Exemplarily, in this embodiment, each image in the enhanced image set can be obtained by performing color gamut expansion processing on the corresponding image in the source domain image set. That is to say, for each image in the enhanced image set, a corresponding image can be found in the source domain image set. The difference between the two is that the image in the enhanced image set has a wider color gamut than the image in the source domain image set.

[0049] Optionally, the manner of performing the color gamut expansion processing on each image in the source domain image set may be a common image manner, which will not be described in detail here.

[0050] S120, based on the teacher model, processing the source domain image set and the enhanced image set to obtain a first processing result.

[0051] The teacher model may be a trained model that can perform semantic segmentation on each image in the source domain image set. Moreover, after the teacher model performs semantic segmentation processing on each image in the source domain image set, the semantic segmentation result obtained is relatively accurate. The first processing result includes a first output result and a second output result.

[0052] Exemplarily, in this embodiment, the source domain image set can be input into the teacher model to obtain a first output result, and at the same time, the enhanced image set can be input into the teacher model to obtain a second output result.

[0053] S130, processing the source domain image set and the enhanced image set based on the student model to obtain a second processing result.

[0054] The student model may be a model that is intended to be used for semantic segmentation of each image in the target domain image set. Before the student model is trained, the semantic segmentation effect of the student model is poor. The second processing result includes a third output result and a fourth output result.

[0055] Exemplarily, in this embodiment, the source domain image set can be input into the student model to obtain the third output result, and at the same time, the enhanced image set can be input into the student model to obtain the fourth output result.

[0056] S140: Adjust the model parameters of the student model according to the first processing result and the second processing result.

[0057] In an optional implementation, a cross entropy loss may be determined according to the first processing result and the second processing result, and based on the cross entropy loss, model parameters of the student model may be adjusted.

[0058] In an optional implementation, a first cross entropy loss can be determined based on the first output result and the fourth output result; and a second cross entropy loss can be determined based on the second output result and the third output result; a first loss value can be determined based on the first cross entropy loss and the second cross entropy loss; and model parameters of the student model can be adjusted based on the first loss value.

[0059] It should be noted that the process of determining the first cross entropy loss according to the first output result and the fourth output result, and the process of determining the second cross entropy loss according to the second output result and the third output result are both conventional processes for determining the cross entropy loss, which will not be described in detail here. Further, in this embodiment, the sum of the first cross entropy loss and the second cross entropy loss can be used as the first loss value, and the model parameters of the student model can be adjusted according to the first loss value.

[0060] In the above-mentioned semantic segmentation model method, in the process of adjusting the model parameters of the student model, not only the source domain image set is processed based on the teacher model and the student model, but also the enhanced image set is processed based on the teacher model and the student model. Furthermore, the model parameters of the student model are adjusted according to the first processing result obtained after the teacher model processes the source domain image set and the enhanced image set, and according to the second processing result obtained after the student model processes the source domain image set and the enhanced image set. In this process, the enhanced image set is introduced to train the student model instead of the traditional adversarial training method, which can increase the model's understanding of category information, thereby improving the model accuracy of the semantic segmentation model.

[0061] Based on the technical solutions of the above embodiments, the present application also provides an optional embodiment. In this optional embodiment, part of the target domain knowledge is added during the training process, but the use of complete target domain images may cause the training of the student model to be offset. Therefore, in this optional embodiment, cross-domain mixed samples are used to train the teacher model and the student model. The following is a further introduction to the semantic segmentation model training method provided by the present application.

[0062] See also Figure 2 The steps for training the semantic segmentation model shown include:

[0063] S210, obtaining a target domain image set.

[0064] The target domain image set includes at least two images in the first color space; the images in the target domain image set and the images in the source domain image set and the enhanced image set are of different types. For example, the images in the source domain image set may be road images, and the images in the target domain image set may be person images, which is not limited in this application.

[0065] S220, constructing a cross-domain mixed image set according to the target domain image set and the source domain image set.

[0066] Exemplarily, in this embodiment, half of the pixels of each image in the source domain image set may be pasted to the corresponding position of the target domain image, thereby constructing a cross-domain mixed image set.

[0067] S230, based on the teacher model, processing the cross-domain mixed image set to obtain a third processing result. And, based on the student model, processing the cross-domain mixed image set to obtain a fourth processing result.

[0068] S240: Determine a second loss value according to the third processing result and the fourth processing result.

[0069] For example, in this embodiment, the sum of the third processing result and the fourth processing result, or the weighted sum, can be used as the second loss value. The weight ratio corresponding to the third processing result and the fourth processing result can be determined based on manual experience or through a large number of experiments, and this application does not impose any limitation on this.

[0070] S250: Adjust the model parameters of the student model according to the second loss value.

[0071] In the above embodiment, based on the teacher model and the student model, the cross-domain mixed image set constructed by the images in the target domain image set and the source domain image set is processed, and the second loss is determined according to the processing result, and then the model parameters of the student model are adjusted according to the second loss.

[0072] Furthermore, in this embodiment, the model parameters of the student model may also be adjusted according to the first loss value and the second loss value.

[0073] Based on the technical solutions of the above embodiments, the present application also provides an optional embodiment. In this optional embodiment, the semantic segmentation model training method provided by the present application is further introduced.

[0074] See also Figure 3 The steps for training the semantic segmentation model shown include:

[0075] S310: Obtain an image processing result corresponding to each source domain image in the first processing result.

[0076] S320, determining a third loss value according to the image processing result and the segmentation label map corresponding to the corresponding source domain image.

[0077] Exemplarily, in this embodiment, for any image in the source domain image set, the image processing result corresponding to the image in the first processing result can be compared with the segmentation label map corresponding to the image, and the difference between the two can be used as the loss value corresponding to the image. Further, the third loss value is determined according to the loss value corresponding to each image in the source domain image set.

[0078] S330: Adjust the model parameters of the student model according to the third loss value.

[0079] In the above embodiment, the model parameters of the student model are adjusted according to the processing results corresponding to each image in the source domain image set and the corresponding segmentation label map, so that the student model can better learn the segmentation ability of the teacher model.

[0080] Furthermore, in this embodiment, the model parameters of the student model may also be adjusted according to at least two of the first loss value, the second loss value, and the third loss value.

[0081] Based on the technical solutions of the above embodiments, the present application also provides an optional embodiment. In this optional embodiment, the semantic segmentation model training method provided by the present application is introduced in detail.

[0082] See also Figure 4 The semantic segmentation model training method shown includes:

[0083] S401, obtaining a source domain image set and an enhanced image set.

[0084] S402: construct a cross-domain mixed image set according to the source domain image set and the enhanced image set.

[0085] S403, inputting the source domain image set into the teacher model to obtain a first output result; and inputting the enhanced image set into the teacher model to obtain a second output result.

[0086] S404, inputting the source domain image set into the student model to obtain a third output result; and inputting the enhanced image set into the student model to obtain a fourth output result.

[0087] S405 , determining a first cross entropy loss according to the first output result and the fourth output result; and determining a second cross entropy loss according to the second output result and the third output result.

[0088] Exemplarily, in this embodiment, the knowledge distillation method can be used to alternate inputs between the teacher model and the student model, and different views can be used to perform pixel-by-pixel distillation losses to enhance the generalization ability of the model. Specifically, in a batch, each image in the source domain image set and each image in the enhanced image set are used as inputs to the teacher model and the student model, respectively, to obtain the segmentation mapping of their probability space. Exemplarily, the segmentation mapping of its probability space can be as follows:

[0089] ;

[0090] ;

[0091] In the formula, represents the first output result; represents the second output result; represents the third output result; represents the fourth output result; is the softmax function; represents the source domain image set; represents the enhanced image set; F represents the teacher model; f represents the student model.

[0092] S406: Determine a first loss value according to the first cross entropy loss and the second cross entropy loss.

[0093] ;

[0094] In the formula, represents the first cross entropy loss; Represents the second cross entropy loss; the overline indicates the calculation of the pixel-by-pixel mean; Represents the first loss value.

[0095] S407, based on the teacher model, processing the cross-domain mixed image set to obtain a third processing result; and, based on the student model, processing the cross-domain mixed image set to obtain a fourth processing result.

[0096] ;

[0097] ;

[0098] In the formula, represents a set of cross-domain mixed images; Indicates the third processing result; Indicates the fourth processing result; is the softmax function.

[0099] S408: Determine a second loss value according to the third processing result and the fourth processing result.

[0100] ;

[0101] In the formula, represents the second loss value; Indicates the third processing result; Indicates the fourth processing result; Represents the cross entropy loss between the third processing result and the fourth processing result; the overline indicates the calculation of the pixel-by-pixel mean.

[0102] S409: Determine a third loss value according to the image processing result and the segmentation label map corresponding to the corresponding source domain image.

[0103] ;

[0104] In the formula, represents the third loss value; Represents the pixel label corresponding to the cth pixel in the i-th image in the source domain image set; It represents the output result obtained after the student model processes the cth pixel of the ith image in the source domain image set.

[0105] S410, adjusting the model parameters of the student model according to the first loss, the second loss and the third loss values.

[0106] ;

[0107] Where L is the target loss value; represents the first loss value; represents the second loss value; represents the third loss value; is the preset weighting factor.

[0108] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0109] Based on the same inventive concept, the embodiment of the present application also provides a semantic segmentation model training device for implementing the semantic segmentation model training method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more semantic segmentation model training device embodiments provided below can refer to the limitations of the semantic segmentation model training method above, and will not be repeated here.

[0110] In an exemplary embodiment, Figure 5 As shown, a semantic segmentation model training device is provided, comprising: an image acquisition module 510, a first image processing module 520, a second image processing module 530 and a first model training module 540, wherein:

[0111] The image acquisition module 510 is used to acquire a source domain image set and an enhanced image set.

[0112] The source domain image set includes at least two images in a first color space; the enhanced image set includes at least two images in a second color space; and the color gamut of the second color space is wider than the color gamut of the first color space.

[0113] The first image processing module 520 is used to process the source domain image set and the enhanced image set based on the teacher model to obtain a first processing result.

[0114] The second image processing module 530 is used to process the source domain image set and the enhanced image set based on the student model to obtain a second processing result.

[0115] The first model training module 540 is used to adjust the model parameters of the student model according to the first processing result and the second processing result.

[0116] In one embodiment, the first image processing module includes a first processing unit, which is used to input the source domain image set into the teacher model to obtain a first output result; a second processing unit, which is used to input the enhanced image set into the teacher model to obtain a second output result; wherein the first processing result includes a first output result and a second output result.

[0117] In one embodiment, the second image processing module includes a third processing unit, which is used to input the source domain image set into the student model to obtain a third output result; and a fourth processing unit, which is used to input the enhanced image set into the student model to obtain a fourth output result; wherein the second processing result includes the third output result and the fourth output result.

[0118] In one embodiment, the first model training module includes a first determination unit, used to determine a first cross entropy loss based on the first output result and the fourth output result; a second determination unit, used to determine a second cross entropy loss based on the second output result and the third output result; a first loss determination unit, used to determine a first loss value based on the first cross entropy loss and the second cross entropy loss; and a first training unit, used to adjust the model parameters of the student model based on the first loss value.

[0119] In one embodiment, the semantic segmentation model training device also includes a second model training module, including an image set acquisition unit, used to acquire a target domain image set; wherein the target domain image set includes at least two images in a first color space; the images in the target domain image set and the images in the source domain image set and the enhanced image set belong to different types of images; a cross-domain construction unit, used to construct a cross-domain mixed image set based on the target domain image set and the source domain image set; a cross-domain image processing unit, used to process the cross-domain mixed image set based on a teacher model to obtain a third processing result; and, based on a student model, processing the cross-domain mixed image set to obtain a fourth processing result; a second loss determination unit, used to determine a second loss value based on the third processing result and the fourth processing result; and a second training unit, used to adjust the model parameters of the student model based on the second loss value.

[0120] In one embodiment, the semantic segmentation model training device also includes a third model training module, including an image processing result acquisition unit, which is used to obtain the image processing results corresponding to each source domain image in the first processing result; a third loss determination unit, which is used to determine a third loss value based on the image processing result and the segmentation label map corresponding to the corresponding source domain image; and a third training unit, which is used to adjust the model parameters of the student model according to the third loss value.

[0121] Each module in the above semantic segmentation model training device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.

[0122] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 6As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (Near Field Communication, NFC) or other technologies. When the computer program is executed by the processor, a semantic segmentation model training method is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0123] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0124] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0125] Acquire a source domain image set and an enhanced image set; wherein the source domain image set includes at least two images in a first color space; the enhanced image set includes at least two images in a second color space; and the color gamut of the second color space is wider than the color gamut of the first color space;

[0126] Based on the teacher model, the source domain image set and the enhanced image set are processed to obtain a first processing result; and

[0127] Based on the student model, the source domain image set and the enhanced image set are processed to obtain a second processing result;

[0128] According to the first processing result and the second processing result, the model parameters of the student model are adjusted.

[0129] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0130] Inputting the source domain image set into the teacher model to obtain a first output result; and,

[0131] Input the enhanced image set into the teacher model to obtain a second output result;

[0132] The first processing result includes a first output result and a second output result.

[0133] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0134] Inputting the source domain image set into the student model to obtain a third output result; and,

[0135] Inputting the enhanced image set into the student model to obtain a fourth output result;

[0136] The second processing result includes a third output result and a fourth output result.

[0137] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0138] Determining a first cross entropy loss according to the first output result and the fourth output result; and,

[0139] Determine a second cross entropy loss according to the second output result and the third output result;

[0140] Determine a first loss value according to the first cross entropy loss and the second cross entropy loss;

[0141] According to the first loss value, the model parameters of the student model are adjusted.

[0142] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0143] Acquire a target domain image set; wherein the target domain image set includes at least two images in a first color space; the images in the target domain image set and the images in the source domain image set and the enhanced image set are images of different types;

[0144] According to the target domain image set and the source domain image set, a cross-domain mixed image set is constructed;

[0145] Based on the teacher model, the cross-domain mixed image set is processed to obtain a third processing result; and based on the student model, the cross-domain mixed image set is processed to obtain a fourth processing result;

[0146] Determining a second loss value according to the third processing result and the fourth processing result;

[0147] According to the second loss value, the model parameters of the student model are adjusted.

[0148] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0149] Obtaining image processing results corresponding to each source domain image in the first processing result;

[0150] Determine a third loss value according to the image processing result and the segmentation label map corresponding to the corresponding source domain image;

[0151] According to the third loss value, the model parameters of the student model are adjusted.

[0152] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0153] Acquire a source domain image set and an enhanced image set; wherein the source domain image set includes at least two images in a first color space; the enhanced image set includes at least two images in a second color space; and the color gamut of the second color space is wider than the color gamut of the first color space;

[0154] Based on the teacher model, the source domain image set and the enhanced image set are processed to obtain a first processing result; and

[0155] Based on the student model, the source domain image set and the enhanced image set are processed to obtain a second processing result;

[0156] According to the first processing result and the second processing result, the model parameters of the student model are adjusted.

[0157] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0158] Inputting the source domain image set into the teacher model to obtain a first output result; and,

[0159] Input the enhanced image set into the teacher model to obtain a second output result;

[0160] The first processing result includes a first output result and a second output result.

[0161] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0162] Inputting the source domain image set into the student model to obtain a third output result; and,

[0163] Inputting the enhanced image set into the student model to obtain a fourth output result;

[0164] The second processing result includes a third output result and a fourth output result.

[0165] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0166] Determining a first cross entropy loss according to the first output result and the fourth output result; and,

[0167] Determine a second cross entropy loss according to the second output result and the third output result;

[0168] Determine a first loss value according to the first cross entropy loss and the second cross entropy loss;

[0169] According to the first loss value, the model parameters of the student model are adjusted.

[0170] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0171] Acquire a target domain image set; wherein the target domain image set includes at least two images in a first color space; the images in the target domain image set and the images in the source domain image set and the enhanced image set are images of different types;

[0172] According to the target domain image set and the source domain image set, a cross-domain mixed image set is constructed;

[0173] Based on the teacher model, the cross-domain mixed image set is processed to obtain a third processing result; and based on the student model, the cross-domain mixed image set is processed to obtain a fourth processing result;

[0174] Determining a second loss value according to the third processing result and the fourth processing result;

[0175] According to the second loss value, the model parameters of the student model are adjusted.

[0176] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0177] Obtaining image processing results corresponding to each source domain image in the first processing result;

[0178] Determine a third loss value according to the image processing result and the segmentation label map corresponding to the corresponding source domain image;

[0179] According to the third loss value, the model parameters of the student model are adjusted.

[0180] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0181] Acquire a source domain image set and an enhanced image set; wherein the source domain image set includes at least two images in a first color space; the enhanced image set includes at least two images in a second color space; and the color gamut of the second color space is wider than the color gamut of the first color space;

[0182] Based on the teacher model, the source domain image set and the enhanced image set are processed to obtain a first processing result; and

[0183] Based on the student model, the source domain image set and the enhanced image set are processed to obtain a second processing result;

[0184] According to the first processing result and the second processing result, the model parameters of the student model are adjusted.

[0185] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0186] Inputting the source domain image set into the teacher model to obtain a first output result; and,

[0187] Input the enhanced image set into the teacher model to obtain a second output result;

[0188] The first processing result includes a first output result and a second output result.

[0189] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0190] Inputting the source domain image set into the student model to obtain a third output result; and,

[0191] Inputting the enhanced image set into the student model to obtain a fourth output result;

[0192] The second processing result includes a third output result and a fourth output result.

[0193] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0194] Determining a first cross entropy loss according to the first output result and the fourth output result; and,

[0195] Determine a second cross entropy loss according to the second output result and the third output result;

[0196] Determine a first loss value according to the first cross entropy loss and the second cross entropy loss;

[0197] According to the first loss value, the model parameters of the student model are adjusted.

[0198] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0199] Acquire a target domain image set; wherein the target domain image set includes at least two images in a first color space; the images in the target domain image set and the images in the source domain image set and the enhanced image set are images of different types;

[0200] According to the target domain image set and the source domain image set, a cross-domain mixed image set is constructed;

[0201] Based on the teacher model, the cross-domain mixed image set is processed to obtain a third processing result; and based on the student model, the cross-domain mixed image set is processed to obtain a fourth processing result;

[0202] Determining a second loss value according to the third processing result and the fourth processing result;

[0203] According to the second loss value, the model parameters of the student model are adjusted.

[0204] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0205] Obtaining image processing results corresponding to each source domain image in the first processing result;

[0206] Determine a third loss value according to the image processing result and the segmentation label map corresponding to the corresponding source domain image;

[0207] According to the third loss value, the model parameters of the student model are adjusted.

[0208] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0209] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0210] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0211] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A semantic segmentation model training method, characterized in that: The method comprises: Acquire a source domain image set and an enhanced image set; wherein the source domain image set includes at least two images in a first color space; the enhanced image set includes at least two images in a second color space; the color gamut of the second color space is wider than the color gamut of the first color space; Based on the teacher model, the source domain image set and the enhanced image set are processed to obtain a first processing result; and Based on the student model, the source domain image set and the enhanced image set are processed to obtain a second processing result; According to the first processing result and the second processing result, the model parameters of the student model are adjusted.

2. The method according to claim 1, characterized in that: The processing of the source domain image set and the enhanced image set based on the teacher model to obtain a first processing result includes: Inputting the source domain image set into the teacher model to obtain a first output result; and, Inputting the enhanced image set into the teacher model to obtain a second output result; The first processing result includes the first output result and the second output result.

3. The method according to claim 2, characterized in that The processing of the source domain image set and the enhanced image set based on the student model to obtain a second processing result includes: Inputting the source domain image set into the student model to obtain a third output result; and, Inputting the enhanced image set into the student model to obtain a fourth output result; The second processing result includes the third output result and the fourth output result.

4. The method according to claim 3, characterized in that The adjusting the model parameters of the student model according to the first processing result and the second processing result includes: Determining a first cross entropy loss according to the first output result and the fourth output result; and Determine a second cross entropy loss according to the second output result and the third output result; Determining a first loss value according to the first cross entropy loss and the second cross entropy loss; According to the first loss value, the model parameters of the student model are adjusted.

5. The method according to any one of claims 1 to 4, characterized in that The method further comprises: Acquire a target domain image set; wherein the target domain image set includes at least two images in the first color space; the images in the target domain image set and the images in the source domain image set and the enhanced image set are of different types; Constructing a cross-domain mixed image set according to the target domain image set and the source domain image set; Based on the teacher model, the cross-domain mixed image set is processed to obtain a third processing result; and based on the student model, the cross-domain mixed image set is processed to obtain a fourth processing result; Determining a second loss value according to the third processing result and the fourth processing result; According to the second loss value, the model parameters of the student model are adjusted.

6. The method according to any one of claims 1 to 4, characterized in that The method further comprises: Obtaining an image processing result corresponding to each source domain image in the first processing result; Determining a third loss value according to the image processing result and a segmentation label map corresponding to the corresponding source domain image; According to the third loss value, the model parameters of the student model are adjusted.

7. A semantic segmentation model training device, characterized in that: The device comprises: An image acquisition module, configured to acquire a source domain image set and an enhanced image set; wherein the source domain image set includes at least two images in a first color space; the enhanced image set includes at least two images in a second color space; and the color gamut of the second color space is wider than the color gamut of the first color space; A first image processing module is used to process the source domain image set and the enhanced image set based on a teacher model to obtain a first processing result; and A second image processing module, used for processing the source domain image set and the enhanced image set based on the student model to obtain a second processing result; The first model training module is used to adjust the model parameters of the student model according to the first processing result and the second processing result.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images