Training method, device and medium of power transmission line smoke semantic segmentation model
By training a semantic segmentation model for smoke in power transmission lines and adjusting the feature extraction network and classifier, the difficulty of smoke detection in power transmission lines in the field is solved, and efficient smoke detection in power transmission line scenarios is achieved.
Patent Information
- Application Number
- CN202211344966.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-10-31
- Publication Date
- 2025-10-21
- Estimated Expiration
- 2042-10-31
AI Technical Summary
In field environments, traditional smoke detection methods rely on smoke concentration and temperature to determine the fire situation in power transmission lines, which is difficult to accurately monitor smoke conditions over a wide area.
A semantic segmentation model for smoke in power transmission lines is adopted. By combining a feature extraction network, first and second classifiers, and a discriminator, the model is trained to enhance the smoke detection capability in power transmission line scenarios. The model is adjusted using image features and classification results to reduce inter-domain differences.
This technology enables smoke detection based on smoke images in power transmission line scenarios without requiring smoke concentration and temperature monitoring, thus improving the accuracy and applicability of smoke detection.
Smart Images
Figure CN115564958B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical fields of computer vision and electrical engineering, and in particular to a training method, device and medium for a power transmission line smoke semantic segmentation model. Background Art
[0002] Fires are often accompanied by rising smoke, so smoke detection plays a crucial role in early fire detection. Traditional smoke detection methods rely primarily on smoke concentration and temperature, triggering smoke alarms only when the smoke reaches a certain concentration and temperature. However, this is impractical for outdoor transmission line fire monitoring. This is because the monitoring area is extremely wide, and outdoor environments are not closed, making it difficult to accurately measure smoke concentration and temperature near the fire source. Summary of the Invention
[0003] Based on this, it is necessary to provide a training method, equipment and medium for the power transmission line smoke semantic segmentation model to solve the problem of difficult power transmission line smoke detection in field environments.
[0004] A method for training a power transmission line smoke semantic segmentation model, the power transmission line smoke semantic segmentation model comprising a feature extraction network, a first classifier, a second classifier, and a discriminator, wherein the first classifier and the second classifier have the same structure but different initial parameters, the method comprising:
[0005] Acquire a training image set; wherein the training image set includes a target domain image set and a source domain image set, the target domain image set includes a plurality of first smoke images in a power transmission line scenario, the source domain image set includes a plurality of second smoke images in a non-power transmission line scenario, and each pixel in the smoke image carries a smoke annotation;
[0006] Inputting the first smoke image and the second smoke image into the feature extraction network respectively, and obtaining a first feature map corresponding to the first smoke image and a second feature map corresponding to the second smoke image as output;
[0007] Inputting the second feature map into the first classifier, obtaining a first smoke classification result for each pixel in the output second smoke image, calculating a first classification loss based on the first smoke classification result and smoke annotation for the same pixel, and adjusting parameters of the feature extraction network and the first classifier based on the first classification loss;
[0008] Inputting the second feature map into the second classifier, obtaining a second smoke classification result for each pixel in the output second smoke image, calculating a second classification loss based on the second smoke classification result and smoke annotation for the same pixel, and adjusting parameters of the feature extraction network and the second classifier based on the second classification loss;
[0009] Input the first feature map and the second feature map into the discriminator respectively, obtain a first identification result corresponding to the first feature map and a second identification result corresponding to the second feature map, calculate the migration loss based on the first identification result and the second identification result, adjust the parameters of the feature extraction network and the parameters of the discriminator based on the migration loss to enhance the ability of the feature extraction network to extract domain-invariant features and increase the probability of correct identification by the discriminator, and return to the step of inputting the first smoke image and the second smoke image into the feature extraction network respectively and subsequent steps until the power transmission line smoke semantic segmentation model converges; wherein the identification result is used to indicate whether the pixel belongs to the target domain image set or the source domain image set.
[0010] In one embodiment, the formula for calculating the first classification loss and the second classification loss is:
[0011]
[0012]
[0013] In the above formula, L seg (E, C1) indicates the first classification loss; L seg (E, C2) indicates the second classification loss; E indicates the feature extraction network; C1 indicates the first classifier; C2 indicates the second classifier; n×m indicates the total number of pixels; C indicates the total number of categories; y ik Indicates whether pixel i belongs to the kth category smoke label. When pixel i belongs to the kth category, y ik =1, otherwise y ik =0;p' ik Indicates the first smoke classification result of pixel i; p″ ik Indicates the second smoke classification result of pixel i.
[0014] In one embodiment, the formula for calculating migration loss is:
[0015]
[0016]
[0017] In the above formula, E indicates the feature extraction network; D indicates the discriminator; n×m indicates the total number of pixels; y jz Indicates the image domain to which pixel j belongs, y jz =1 indicates belonging to the source domain, y jz =0 indicates belonging to the target domain; p' jz Indicates the image domain to which pixel j belongs when adjusting the parameters of the feature extraction network and the parameters of the discriminator; p″ jz Indicates the image domain to which pixel j belongs when adjusting the parameters of the feature extraction network.
[0018] In one embodiment, the method further includes:
[0019] Acquire a test image set; wherein the test image set includes a plurality of third smoke images, each pixel in the third smoke image carries a smoke annotation, and the third smoke annotation image is in a power transmission line scene;
[0020] Inputting the third smoke image into the feature extraction network, and obtaining a third feature map corresponding to the output third smoke image;
[0021] Inputting the third feature map into the first classifier to obtain a third smoke classification result for each pixel in the output third smoke image; inputting the third feature map into the second classifier to obtain a fourth smoke classification result for each pixel in the output third smoke image;
[0022] A test classification result of each pixel in the third smoke image is determined according to the third smoke classification result and the fourth smoke classification result, and a predicted segmented image of the third smoke image is output based on the test classification results of all pixels.
[0023] In one embodiment, the method further includes:
[0024] A mean square error is calculated based on a test classification result of a first pixel in the predicted segmented image and a smoke annotation carried by the first pixel in the third smoke image, and is used as a first evaluation indicator; wherein the first pixel is a pixel at the same position in the predicted segmented image and the third smoke image.
[0025] In one embodiment, the method further includes:
[0026] Determine a first pixel, a second pixel, a third pixel, and a fourth pixel among all pixels according to a test classification result of each pixel in the predicted segmented image and a smoke annotation carried by each pixel in the third smoke image; wherein the first pixel is a positive sample predicted by the model as a positive class, the second pixel is a negative sample predicted by the model as a negative class, the third pixel is a negative sample predicted by the model as a positive class, and the fourth pixel is a positive sample predicted by the model as a negative class;
[0027] A second evaluation index and a third evaluation index are calculated based on the first pixel, the second pixel, the third pixel and the fourth pixel; wherein the second evaluation index indicates the ratio of the total number of pixels where the test classification result intersects with the smoke annotation and the total number of pixels where the test classification result and the smoke annotation intersect, and the third evaluation index indicates the percentage between correctly classified pixels and the total number of pixels.
[0028] In one embodiment, the method further includes:
[0029] Clustering is performed based on all eigenvalues in the first feature map and all eigenvalues in the second feature map to obtain a plurality of clusters; wherein the distance between the plurality of eigenvalues in each cluster is less than a preset value;
[0030] Determining an original pixel based on a feature value within a first cluster, and obtaining a smoke annotation carried by the original pixel; wherein the first cluster is any one of the plurality of clusters, the original pixel belongs to the second smoke image, and the feature value within the first cluster is formed by feature extraction of the original pixel;
[0031] Unlabeled pixels are determined based on the eigenvalues within the first cluster, and the smoke annotations carried by the original pixels are used as pseudo labels for the unlabeled pixels; wherein the unlabeled pixels belong to the first smoke image, and the unlabeled pixels are subjected to feature extraction to form eigenvalues within the first cluster.
[0032] After all pixels in the first smoke image are assigned pseudo labels, the first smoke image carrying the pseudo labels is added to the training image set.
[0033] In one embodiment, after obtaining the training image set, the method further includes:
[0034] A preprocessing operation is performed on the first smoke image and the second smoke image; wherein the preprocessing operation includes random rotation, random erasing, random horizontal flipping, random padding, mirror flipping, and random cropping.
[0035] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the processor performs the steps of the training method for the power transmission line smoke semantic segmentation model.
[0036] A power transmission line smoke semantic segmentation device includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the above-mentioned training method for the power transmission line smoke semantic segmentation model.
[0037] The present invention provides a training method, device, and medium for a power transmission line smoke semantic segmentation model. First, a training image set is obtained, comprising a target domain image set of multiple first smoke images and a source domain image set of multiple second smoke images. A feature extraction network is then used to extract a first feature map corresponding to the first smoke image and a second feature map corresponding to the second smoke image. The second feature map is then input into a first classifier and a second classifier. The first classifier, the second classifier, and the feature extraction network are then trained based on the difference between the actual annotation results of the second smoke image and the predicted classification results output by the classifiers. This allows the first classifier, the second classifier, and the feature extraction network to segment smoke in non-power transmission line scenarios. The first and second feature maps are then input into a discriminator. The parameters of the feature extraction network and the discriminator are then adjusted based on the first and second identification results output by the discriminator to enhance the feature extraction network's ability to extract domain-invariant features. This reduces the difference between the target and source domains, allowing the trained feature extraction network to also segment smoke in power transmission line scenarios. Using this trained model, smoke detection can be completed based solely on smoke images in power transmission line scenarios without obtaining smoke concentration and smoke temperature. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0039] in:
[0040] Figure 1 1 is a flow chart of a method for training a power transmission line smoke semantic segmentation model in one embodiment;
[0041] Figure 2 Schematic diagram of the transmission line smoke semantic segmentation model;
[0042] Figure 3Schematic diagram of the image before annotation;
[0043] Figure 4 is a schematic diagram of the annotated image;
[0044] Figure 5 Schematic diagram of adding pseudo labels;
[0045] Figure 6 Schematic diagram of the process of adding pseudo labels;
[0046] Figure 7 A schematic diagram of a process for testing a power line smoke semantic segmentation model in one embodiment;
[0047] Figure 8 This is a structural block diagram of a power transmission line smoke semantic segmentation device in one embodiment. DETAILED DESCRIPTION
[0048] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.
[0049] like Figure 1 As shown, Figure 1 FIG. 1 is a flow chart of a method for training a power line smoke semantic segmentation model in one embodiment. Figure 2 As shown, the power line smoke semantic segmentation model includes a feature extraction network, a first classifier, a second classifier, and a discriminator. The feature extraction network is composed of a multi-layer fully convolutional neural network and is capable of extracting smoke features. The first and second classifiers have the same structure but different initial parameters. Both classifiers are composed of convolutional layers, ReLU activation functions, and fully connected layers. During training, the parameters within the model are not shared. Combining the outputs of these two classifiers allows for more accurate semantic classification of pixels. In addition, to eliminate differences between different data domains, a discriminator is designed. It consists of a convolutional layer, a BN layer, a ReLU activation function, and a fully connected layer. Through adversarial learning between the discriminator and the feature extraction network, the domain differences between power line smoke images and smoke images from other scenes can be narrowed.
[0050] The training method of the power transmission line smoke semantic segmentation model in this embodiment includes the following steps:
[0051] S101, obtain a training image set.
[0052] The training image set includes a target domain image set and a source domain image set. The target domain image set includes multiple first smoke images in the transmission line scene, and the source domain image set includes multiple second smoke images in the non-transmission line scene. Each pixel in the smoke image carries a smoke label, that is, a label of whether it belongs to smoke or not. Here, the first smoke image is defined as (f=1,2,…,M, M represents the number of samples in the target domain image set), the second smoke image is defined as (e=1, 2,…, N, N represents the number of samples in the source domain image set).
[0053] For example, the smoke in the second smoke image is caused by a wildfire, and the captured image does not include power transmission lines. Alternatively, the smoke in the second smoke image is caused by a building fire, and the captured image does not include power transmission lines. Alternatively, other forms are possible, and are not specifically limited here. To obtain sufficient first smoke images, the first smoke images can be of the same transmission line scene, captured at different time periods, angles, and weather conditions. To achieve optimal semantic segmentation, the first smoke image should be as free of obstructions as possible, and ideally capture the overall appearance of the smoke. The image resolution should be no less than 1920*1080, 200dpi. If there is a small amount of occlusion, it should not exceed 10% of the target object. The image should be clear, with uniform brightness, moderate contrast, and no noise. It should not contain spots, snowflakes, ripples, overexposure, underexposure, or blur.
[0054] Then labelme can be used to annotate the image. The annotation example is as follows Figure 3 and Figure 4 As shown. Among them, Figure 3 is the image before annotation, Figure 4 This is the labeled image. It can be seen that after labeling, it is clear which part of the image belongs to smoke and which part does not belong to smoke.
[0055] Optionally, to increase the subsequent training speed, the first smoke image and the second smoke image may be preprocessed, wherein the preprocessing operations include random rotation, random erasing, random horizontal flipping, random padding, mirror flipping, and random cropping. Of course, other preprocessing operations may also be included.
[0056] S103: Input the first smoke image and the second smoke image into a feature extraction network respectively, and obtain a first feature map corresponding to the first smoke image and a second feature map corresponding to the second smoke image.
[0057] The process can be expressed as:
[0058]
[0059]
[0060] In the above formula, is the first feature map, is the second feature map, and E is the feature extraction network.
[0061] Optional, see Figure 5 , to expand the training image set, such as Figure 6 As shown, you can also perform the following steps:
[0062] S1041: Clustering is performed based on all eigenvalues in the first feature map and all eigenvalues in the second feature map, and a plurality of clusters are obtained by clustering.
[0063] The distances between multiple eigenvalues within each cluster are less than a preset value, which means that the eigenvalues within the same cluster are basically consistent.
[0064] S1042: Determine the original pixel according to the eigenvalue in the first cluster, and obtain the smoke annotation carried by the original pixel.
[0065] The first cluster is any one of the multiple clusters, the original pixel belongs to the second smoke image, and the original pixel is subjected to feature extraction to form a feature value within the first cluster. In other words, the original pixel in the second smoke image is mapped by the feature extraction network and classified into the first cluster. Therefore, the original pixel carries the correct smoke label.
[0066] S1043: Determine unlabeled pixels according to the eigenvalues in the first cluster, and use the smoke annotations carried by the original pixels as pseudo labels for the unlabeled pixels.
[0067] The unlabeled pixels belong to the first smoke image, and feature extraction on these unlabeled pixels forms eigenvalues within the first cluster. Here, the first smoke image is considered the unlabeled image, and the unlabeled pixels within the first smoke image are also classified into the first cluster after being mapped by the feature extraction network. Since the mapped eigenvalues of the original pixels and the mapped eigenvalues of the unlabeled pixels belong to the same cluster, the original pixels and unlabeled pixels can be considered close, and the smoke annotation carried by the original pixels is used as the pseudo-label for the unlabeled pixels.
[0068] S1044: After all pixels in the first smoke image are assigned pseudo labels, the first smoke image carrying the pseudo labels is added to the training image set.
[0069] Steps S1041-S103 are repeated until all pixels in the first smoke image have been assigned pseudo-labels. This results in a pseudo-labeled first smoke image, which is then added to the training image set for continued training. Pseudo-labels are derived based on the similarity between the source and target domains. Continuing to train the network with these pseudo-labels enhances the network's ability to transfer from the source domain to the target domain.
[0070] S105: Input the second feature map into the first classifier, obtain the first smoke classification result for each pixel in the output second smoke image, calculate the first classification loss based on the first smoke classification result and smoke annotation of the same pixel, and adjust the parameters of the feature extraction network and the parameters of the first classifier based on the first classification loss.
[0071] S107: Input the second feature map into the second classifier, obtain the second smoke classification result for each pixel in the output second smoke image, calculate the second classification loss based on the second smoke classification result and smoke annotation of the same pixel, and adjust the parameters of the feature extraction network and the parameters of the second classifier based on the second classification loss.
[0072] In S105-S107, the formula for calculating the first classification loss and the second classification loss is:
[0073]
[0074]
[0075] In the above formula, L seg (E, C1) indicates the first classification loss; L seg (E, C2) indicates the second classification loss; E indicates the feature extraction network; C1 indicates the first classifier; C2 indicates the second classifier; n×m indicates the total number of pixels; C indicates the total number of classifications; y ik Indicates whether pixel i belongs to the kth category smoke label. When pixel i belongs to the kth category, y ik =1, otherwise y ik = 0 (for example, k = 1 for smoke, y i1 =1 means pixel i belongs to smoke); p′ ik Indicates the first smoke classification result of pixel i; p″ ik Indicates the second smoke classification result of pixel i.
[0076] Then, the Adam algorithm can be used to adjust the parameters of the feature extraction network and the classifier. The number of iterations of the Adam algorithm is set to 500 times, the initialization learning rate is set to 0.001, and the weight decay is set to 0.0005. Every 50 iterations, the learning rate decays to 1 / 10 of the original.
[0077] S109: Input the first feature map and the second feature map into the discriminator, obtain the first identification result corresponding to the first feature map, and the second identification result corresponding to the second feature map. Calculate the transfer loss based on the first and second identification results. Adjust the parameters of the feature extraction network and the discriminator based on the transfer loss to improve the feature extraction network's ability to extract domain-invariant features and increase the probability of the discriminator's correct identification. Return to S103 and continue until the power line smoke semantic segmentation model converges.
[0078] The identification result is used to indicate whether a pixel belongs to the target domain image set or the source domain image set. For example, when the identification result = 1, it indicates that the pixel belongs to the source domain. When the identification result = 0, it indicates that the pixel belongs to the target domain.
[0079] Optionally, the formula for calculating migration loss is:
[0080]
[0081]
[0082] In the above formula, E indicates the feature extraction network; D indicates the discriminator; n×m indicates the total number of pixels; y jz Indicates the image domain to which pixel j belongs, y jz =1 indicates belonging to the source domain, y jz =0 indicates belonging to the target domain; p' jz Indicates the image domain to which pixel j belongs when adjusting the parameters of the feature extraction network and the parameters of the discriminator; p″ jz Indicates the image domain to which pixel j belongs when tuning the parameters of the feature extraction network.
[0083] When adjusting the parameters of the feature extraction network and the discriminator, first adv1 The formula (E, D) jointly optimizes E and D, allowing D to distinguish whether the pixel comes from the source domain or the target domain. Then fix the parameters of D and based on L adv2 The formula of (E) optimizes E, and D can extract more domain-invariant features. Then alternately optimize L adv1 (E,D) and L adv2 The formula for (E) is sufficient, and the optimized model parameter file is saved. This cyclical interaction between E and D, through adversarial learning, continuously improves D's ability to extract domain-invariant features, thereby reducing the difference between the target and source domains.
[0084] As can be seen, the training method for the power transmission line smoke semantic segmentation model described above first obtains a training image set, comprising a target domain image set of multiple first smoke images and a source domain image set of multiple second smoke images. A feature extraction network is then used to extract a first feature map corresponding to the first smoke image and a second feature map corresponding to the second smoke image. The second feature map is then input into the first and second classifiers. Based on the difference between the actual annotation results of the second smoke image and the predicted classification results output by the classifiers, the first and second classifiers, and the feature extraction network are trained, enabling them to segment smoke in non-power transmission line scenarios. The first and second feature maps are then input into the discriminator. Based on the first and second identification results output by the discriminator, the parameters of the feature extraction network and the discriminator are adjusted to enhance the feature extraction network's ability to extract domain-invariant features. This, in turn, reduces the difference between the target and source domains, enabling the trained feature extraction network to also segment smoke in power transmission line scenarios. Using this trained model, smoke detection can be completed based solely on smoke images in power transmission line scenarios without obtaining smoke concentration and smoke temperature.
[0085] Further, such as Figure 7 As shown in the figure, after the network training is completed, it is necessary to evaluate the training effect of the model through testing. The specific steps include:
[0086] S111, obtain a test image set.
[0087] The test image set includes multiple third smoke images, each pixel of which carries a smoke annotation, and the third smoke annotation images are in the power transmission line scene. To ensure test accuracy, the number of third smoke images can be set to twice the number of first smoke images.
[0088] S113: Input the third smoke image into a feature extraction network to obtain a third feature map corresponding to the output third smoke image.
[0089] S115: Input the third feature map into the first classifier to obtain a third smoke classification result for each pixel in the output third smoke image; input the third feature map into the second classifier to obtain a fourth smoke classification result for each pixel in the output third smoke image.
[0090] S113-S115 are basically the same as S103-S107 above, except that the third smoke image is input and only the third smoke classification result and the fourth smoke classification result need to be obtained as output, without adjusting the model parameters.
[0091] S117: Determine a test classification result for each pixel in the third smoke image according to the third smoke classification result and the fourth smoke classification result, and output a predicted segmented image of the third smoke image based on the test classification results of all pixels.
[0092] Alternatively, the classification results can be averaged as the final test classification result, or weighted or other methods can be used. Finally, a predicted segmented image of the third smoke image is output based on the test classification results of all pixels.
[0093] Furthermore, in order to understand the effectiveness of training, we can further organize and analyze the results to obtain a series of evaluation indicators:
[0094] Optionally, a mean square error is calculated based on a test classification result of a first pixel in the predicted segmented image and a smoke annotation carried by the first pixel in the third smoke image, and used as the first evaluation metric, wherein the first pixel is a pixel at the same position in the predicted segmented image and the third smoke image.
[0095] For example, for the i-th test image, the first evaluation index is expressed as:
[0096]
[0097] For L third smoke images, the first evaluation index is expressed as:
[0098]
[0099] It can be understood that the smaller the value of the first evaluation indicator here, the better the model training.
[0100] Optionally, the first pixel, the second pixel, the third pixel, and the fourth pixel among all pixels may be determined according to the test classification result of each pixel in the predicted segmented image and the smoke annotation carried by each pixel in the third smoke image; wherein,
[0101] The first pixel is a positive sample predicted by the model as positive, denoted as TP;
[0102] The second pixel is a negative sample predicted by the model as a negative class, denoted as TN;
[0103] The third pixel is a negative sample predicted by the model as positive, denoted as FP;
[0104] The fourth pixel is a positive sample predicted by the model as a negative class, denoted as FN.
[0105] Then, a second evaluation index and a third evaluation index are calculated based on the first pixel, the second pixel, the third pixel, and the fourth pixel. The second evaluation index indicates the ratio of the total number of pixels where the test classification result intersects with the smoke annotation to the total number of pixels where the test classification result and the smoke annotation intersect.
[0106] For the i-th test image, the second evaluation index is expressed as:
[0107]
[0108] For L third smoke images, the second evaluation index is expressed as:
[0109]
[0110] Here, the larger the value of the second evaluation indicator is, the more accurate the model training is.
[0111] The third evaluation metric indicates the percentage of correctly classified pixels to the total number of pixels, expressed as:
[0112]
[0113] Here, the larger the value of the third evaluation indicator is, the more accurate the model training is.
[0114] Optionally, you can also evaluate model complexity. Evaluation metrics include parameter count, FLOPs, training time, testing time, and model size. Generally speaking, lower model complexity indicates a more practical model.
[0115] Figure 8 FIG. 1 shows an internal structure diagram of a power line smoke semantic segmentation device in one embodiment. Figure 8 As shown, the power line smoke semantic segmentation device includes a processor, a memory, and a network interface connected via a system bus. Among them, the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the power line smoke semantic segmentation device stores an operating system and may also store a computer program. When the computer program is executed by the processor, the processor can implement a training method for the power line smoke semantic segmentation model. The internal memory may also store a computer program. When the computer program is executed by the processor, the processor can implement a training method for the power line smoke semantic segmentation model. Those skilled in the art will understand that Figure 8The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the power transmission line smoke semantic segmentation device to which the solution of the present application is applied. The specific power transmission line smoke semantic segmentation device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.
[0116] A device for semantic segmentation of smoke on a power transmission line includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are implemented: obtaining a training image set; inputting a first smoke image and a second smoke image into a feature extraction network respectively, obtaining a first feature map corresponding to the output first smoke image and a second feature map corresponding to the output second smoke image; inputting the second feature map into a first classifier, obtaining a first smoke classification result for each pixel in the output second smoke image, calculating a first classification loss based on the first smoke classification result and smoke annotation of the same pixel, and adjusting parameters of the feature extraction network and parameters of the first classifier based on the first classification loss; inputting the second feature map into a second classifier, obtaining a first smoke classification result for each pixel in the output second smoke image The second smoke classification result of a pixel is obtained, and the second classification loss is calculated based on the second smoke classification result and smoke label of the same pixel, and the parameters of the feature extraction network and the second classifier are adjusted according to the second classification loss; the first feature map and the second feature map are respectively input into the discriminator, and the first identification result corresponding to the output first feature map and the second identification result corresponding to the second feature map are obtained, and the migration loss is calculated based on the first identification result and the second identification result. The parameters of the feature extraction network and the parameters of the discriminator are adjusted according to the migration loss to increase the ability of the feature extraction network to extract domain-invariant features and increase the probability of correct identification by the discriminator, and return to execute the step of inputting the first smoke image and the second smoke image into the feature extraction network respectively and subsequent steps until the power transmission line smoke semantic segmentation model converges.
[0117] A computer-readable storage medium stores a computer program that, when executed by a processor, implements the following steps: obtaining a training image set; inputting a first smoke image and a second smoke image into a feature extraction network, respectively, to obtain a first feature map corresponding to the first smoke image and a second feature map corresponding to the second smoke image; inputting the second feature map into a first classifier, obtaining a first smoke classification result for each pixel in the output second smoke image, calculating a first classification loss based on the first smoke classification result and smoke annotation of the same pixel, and adjusting parameters of the feature extraction network and the first classifier based on the first classification loss; inputting the second feature map into a second classifier, obtaining a second smoke classification result for each pixel in the output second smoke image As a result, the second classification loss is calculated based on the second smoke classification result and smoke labeling of the same pixel, and the parameters of the feature extraction network and the second classifier are adjusted according to the second classification loss; the first feature map and the second feature map are respectively input into the discriminator, and the first identification result corresponding to the output first feature map and the second identification result corresponding to the second feature map are obtained, and the migration loss is calculated based on the first identification result and the second identification result. The parameters of the feature extraction network and the parameters of the discriminator are adjusted according to the migration loss to increase the ability of the feature extraction network to extract domain invariant features and increase the probability of correct identification of the discriminator, and return to the step of inputting the first smoke image and the second smoke image into the feature extraction network and subsequent steps until the transmission line smoke semantic segmentation model converges.
[0118] It should be noted that the above-mentioned training method, device and computer-readable storage medium for the transmission line smoke semantic segmentation model belong to a general inventive concept, and the contents of the training method, device and computer-readable storage medium embodiments of the transmission line smoke semantic segmentation model are applicable to each other.
[0119] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, which can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
[0120] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0121] The above embodiments merely illustrate several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.
Claims
1. A training method for a power transmission line smoke semantic segmentation model, characterized in that: The power transmission line smoke semantic segmentation model includes a feature extraction network, a first classifier, a second classifier, and a discriminator. The first classifier and the second classifier have the same structure but different initial parameters. The method includes: Acquire a training image set; wherein the training image set includes a target domain image set and a source domain image set, the target domain image set includes a plurality of first smoke images in a power transmission line scenario, the source domain image set includes a plurality of second smoke images in a non-power transmission line scenario, and each pixel in the smoke image carries a smoke annotation; Inputting the first smoke image and the second smoke image into the feature extraction network respectively, and obtaining a first feature map corresponding to the first smoke image and a second feature map corresponding to the second smoke image as output; Inputting the second feature map into the first classifier, obtaining a first smoke classification result for each pixel in the output second smoke image, calculating a first classification loss based on the first smoke classification result and smoke annotation for the same pixel, and adjusting parameters of the feature extraction network and the first classifier based on the first classification loss; Inputting the second feature map into the second classifier, obtaining a second smoke classification result for each pixel in the output second smoke image, calculating a second classification loss based on the second smoke classification result and smoke annotation for the same pixel, and adjusting parameters of the feature extraction network and the second classifier based on the second classification loss; Input the first feature map and the second feature map into the discriminator respectively, obtain a first identification result corresponding to the first feature map and a second identification result corresponding to the second feature map, calculate the migration loss based on the first identification result and the second identification result, adjust the parameters of the feature extraction network and the parameters of the discriminator based on the migration loss to enhance the ability of the feature extraction network to extract domain-invariant features and increase the probability of correct identification by the discriminator, and return to the step of inputting the first smoke image and the second smoke image into the feature extraction network respectively and subsequent steps until the power transmission line smoke semantic segmentation model converges; wherein the identification result is used to indicate whether the pixel belongs to the target domain image set or the source domain image set.
2. The method according to claim 1, characterized in that The formula for calculating the first classification loss and the second classification loss is: In the above formula, L seg (E, C1) indicates the first classification loss; L seg (E, C2) indicates the second classification loss; E indicates the feature extraction network; C1 indicates the first classifier; C2 indicates the second classifier; n×m indicates the total number of pixels; C indicates the total number of categories; y ik Indicates whether pixel i belongs to the kth category smoke label. When pixel i belongs to the kth category, y ik =1, otherwise y ik =0; p' ik Indicates the first smoke classification result of pixel i; p" ik Indicates the second smoke classification result of pixel i.
3. The method according to claim 1, characterized in that The formula for calculating migration loss is: In the above formula, E indicates the feature extraction network; D indicates the discriminator; n×m indicates the total number of pixels; y jz Indicates the image domain to which pixel j belongs, y jz =1 indicates belonging to the source domain, y jz =0 indicates belonging to the target domain; p' jz Indicates the image domain to which pixel j belongs when adjusting the parameters of the feature extraction network and the parameters of the discriminator; p" jz Indicates the image domain to which pixel j belongs when adjusting the parameters of the feature extraction network.
4. The method according to claim 1, wherein The method further comprises: Acquire a test image set; wherein the test image set includes a plurality of third smoke images, each pixel in the third smoke image carries a smoke annotation, and the third smoke annotation image is in a power transmission line scene; Inputting the third smoke image into the feature extraction network, and obtaining a third feature map corresponding to the output third smoke image; Inputting the third feature map into the first classifier to obtain a third smoke classification result for each pixel in the output third smoke image; inputting the third feature map into the second classifier to obtain a fourth smoke classification result for each pixel in the output third smoke image; A test classification result of each pixel in the third smoke image is determined according to the third smoke classification result and the fourth smoke classification result, and a predicted segmented image of the third smoke image is output based on the test classification results of all pixels.
5. The method according to claim 4, characterized in that The method further comprises: A mean square error is calculated based on a test classification result of a first pixel in the predicted segmented image and a smoke annotation carried by the first pixel in the third smoke image, and is used as a first evaluation indicator; wherein the first pixel is a pixel at the same position in the predicted segmented image and the third smoke image.
6. The method according to claim 4, characterized in that The method further comprises: Determine a first pixel, a second pixel, a third pixel, and a fourth pixel among all pixels according to a test classification result of each pixel in the predicted segmented image and a smoke annotation carried by each pixel in the third smoke image; wherein the first pixel is a positive sample predicted by the model as a positive class, the second pixel is a negative sample predicted by the model as a negative class, the third pixel is a negative sample predicted by the model as a positive class, and the fourth pixel is a positive sample predicted by the model as a negative class; A second evaluation index and a third evaluation index are calculated based on the first pixel, the second pixel, the third pixel and the fourth pixel; wherein the second evaluation index indicates the ratio of the total number of pixels where the test classification result intersects with the smoke annotation and the total number of pixels where the test classification result and the smoke annotation intersect, and the third evaluation index indicates the percentage between correctly classified pixels and the total number of pixels.
7. The method according to claim 1, characterized in that The method further comprises: Clustering is performed based on all eigenvalues in the first feature map and all eigenvalues in the second feature map to obtain a plurality of clusters; wherein the distance between the plurality of eigenvalues in each cluster is less than a preset value; Determining an original pixel based on a feature value within a first cluster, and obtaining a smoke annotation carried by the original pixel; wherein the first cluster is any one of the plurality of clusters, the original pixel belongs to the second smoke image, and the feature value within the first cluster is formed by feature extraction of the original pixel; Determining unlabeled pixels based on the eigenvalues within the first cluster, and using the smoke annotations carried by the original pixels as pseudo labels for the unlabeled pixels; wherein the unlabeled pixels belong to the first smoke image, and the eigenvalues within the first cluster are formed by feature extraction of the unlabeled pixels; After all pixels in the first smoke image are assigned pseudo labels, the first smoke image carrying the pseudo labels is added to the training image set.
8. The method according to claim 1, characterized in that After obtaining the training image set, the method further includes: A preprocessing operation is performed on the first smoke image and the second smoke image; wherein the preprocessing operation includes random rotation, random erasing, random horizontal flipping, random padding, mirror flipping, and random cropping.
9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 8.
10. A power transmission line smoke semantic segmentation device, comprising a memory and a processor, characterized in that: The memory stores a computer program, and when the computer program is executed by the processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 8.