Image detection method, system and equipment based on ground cover change and medium

Abstract

The invention discloses an image detection method, system and device based on ground cover change and a medium, and the method comprises the following steps: constructing a change detection network which comprises a first branch network, a second branch network and a third branch network; training the change detection network by using the training data set; obtaining a to-be-detected image pair, wherein the to-be-detected image pair comprises a to-be-detected initial image before the ground cover changes and a to-be-detected change image after the ground cover changes; and detecting the change of the ground cover from the to-be-detected initial image to the to-be-detected change image by using the change detection network. According to the method provided by the invention, only a small number of annotated images are needed, natural disasters which cause the change of the ground cover in a large range can be automatically detected, and the occurrence area and the influence range of the natural disasters can be positioned, so that rescue work can be quickly and effectively carried out, and the life and property safety of people is guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of environmental image processing, and in particular to an image detection method, system, equipment and medium based on changes in ground cover. Background Art

[0002] In the field of natural disaster detection, existing methods are mainly based on prevention and supplemented by detection. The current mainstream method relies on manual and expert modeling detection.

[0003] For flood prevention and control, most existing methods rely on manual determination of the impact range of floods. This solution requires dozens of experts to discuss and define based on a lot of experience. However, flood damage is continuous and rapid, and there is often a lag in the quantitative determination of the impact range by manpower. Traditional modeling methods require a large amount of data to model a certain basin, and the features used for identification need to be carefully screened by experts in professional fields, which is a very labor-intensive and resource-intensive task. At the same time, there is little data on floods, and the accuracy of the corresponding modeling is still unknown. For other scenarios where the natural cover of the surface changes due to natural factors, the above-mentioned manual or modeling methods can also be used to detect changes in the natural cover of the surface.

[0004] In general, the current detection of changes in natural surface cover through manual methods and traditional modeling has problems such as long communication time, slow detection, slow response time, lack of versatility, and lack of timeliness and accuracy. Summary of the invention

[0005] The purpose of the present invention is to provide an image detection method, system, equipment and medium based on surface cover changes, so as to solve the problems existing in the change detection of natural surface cover by artificial means and traditional modeling, the specific problems are reflected in long communication time, slow detection, slow response time, lack of versatility, and lack of timeliness and accuracy.

[0006] To solve the above technical problems, a technical solution adopted by the present invention is to provide an image detection method based on changes in ground cover, the method comprising the following steps: constructing a change detection network, the change detection network comprising a first branch network, a second branch network and a third branch network; using a training data set to train the change detection network; obtaining a pair of images to be detected, the pair of images to be detected comprising an initial image to be detected before the ground cover changes and a changed image to be detected after the change; using the change detection network to detect changes in ground cover from the initial image to be detected to the changed image to be detected.

[0007] In some embodiments, before using the training data set to train the change detection network, the method further includes the step of constructing an initial data set, wherein the initial data set includes a plurality of labeled image pairs and a plurality of unlabeled image pairs.

[0008] In some embodiments, the change detection network is trained using a training data set, comprising the steps of: establishing each training data set required for each training round based on the initial data set; performing image enhancement processing on each training data set to obtain each pre-processed image set; and inputting each pre-processed image set into the change detection network in sequence to perform training for a corresponding number of training rounds.

[0009] In some embodiments, the training data set includes only all the labeled image pairs in the initial data set, or includes all the labeled image pairs in the initial data set and some or all of the unlabeled image pairs in the initial data set; wherein, except for the first round of training, as the number of training rounds increases, the ratio of the number of labeled image pairs in the training data set used to the number of unlabeled image pairs decreases dynamically.

[0010] In some embodiments, the image enhancement processing of each of the training data sets includes: performing weak enhancement processing on the labeled image pairs to obtain a first weakly enhanced image pair; performing weak enhancement processing and strong enhancement processing on the unlabeled image pairs, respectively, to obtain a second weakly enhanced image pair and a strongly enhanced image pair.

[0011] In some embodiments, each of the preprocessed image sets is sequentially input into the change detection network for a corresponding number of training rounds, including: in a first training stage, only the difference decoder and the difference classifier in the third branch network are trained, and the number of labeled image pairs in the corresponding training data set is greater than or equal to the number of unlabeled image pairs; in a second training stage, only the first encoder in the first branch network and the second encoder in the second branch network are trained, and the number of labeled image pairs in the corresponding training data set is less than the number of unlabeled image pairs.

[0012] In some embodiments, the change detection network uses a cross entropy loss function: ; in, p represents the classification probability; if the input of the change detection network is the first weakly enhanced image pair, then y represents the true label; if the input of the change detection network is the second weakly enhanced image pair and the strongly enhanced image pair, then y Represents a pseudo label.

[0013] The present invention also provides an image detection system based on changes in ground cover, the system comprising: a network construction unit, used to construct a change detection network, the change detection network comprising a first branch network, a second branch network and a third branch network; a network training unit, used to train the change detection network using a training data set; an image acquisition unit, used to acquire a pair of images to be detected, the pair of images to be detected comprising an initial image to be detected before a change in ground cover occurs and a changed image to be detected after the change; an image detection unit, used to detect changes in ground cover from the initial image to be detected to the changed image to be detected using the change detection network.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0015] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0016] The beneficial effects of the present invention are as follows: the present invention discloses an image detection method, system, device and medium based on changes in ground cover, the method comprising the following steps: constructing a change detection network, the change detection network comprising a first branch network, a second branch network and a third branch network; using a training data set to train the change detection network; obtaining a pair of images to be detected, the pair of images to be detected comprising an initial image to be detected before the ground cover changes and a changed image to be detected after the change; using the change detection network to detect changes in ground cover from the initial image to be detected to the changed image to be detected. The method of the present invention only requires a small amount of annotated images, and can automatically detect natural disasters that cause changes in ground cover in a large range, and locate the area where they occur and the scope of impact, so that rescue work can be carried out quickly and effectively to protect people's lives and property. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flow chart of an image detection method based on ground cover change of the present invention; Figure 2 It is a structural block diagram of a change detection network in an image detection method based on ground cover change of the present invention; Figure 3 It is a training principle diagram of a change detection network in an image detection method based on ground cover change of the present invention; Figure 4 It is a specific flow chart of step S2 in an image detection method based on changes in ground cover of the present invention; Figure 5 It is a structural block diagram of an image detection system based on ground cover change of the present invention; Figure 6 It is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention; Figure 7 It is a schematic block diagram of an embodiment of a computer-readable storage medium of the present invention. DETAILED DESCRIPTION

[0018] In order to facilitate the understanding of the present invention, the present invention is described in more detail below in conjunction with the accompanying drawings and specific embodiments. Preferred embodiments of the present invention are provided in the accompanying drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described in this specification. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present invention more thorough and comprehensive.

[0019] It should be noted that, unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used in this specification includes any and all combinations of one or more related listed items.

[0020] like Figure 1-4 As shown, an image detection method based on ground cover change of the present invention is shown, and the method comprises the following steps: Step S1: Construct a change detection network.

[0021] It should be noted that each time the change detection network receives an image pair, it outputs the classification probabilities of the ground cover in the three corresponding images. Each image pair includes an initial image and a change image. The initial image corresponds to the image before the ground cover on the ground changes, and the change image corresponds to the image after the ground cover on the ground changes. Ground cover generally refers to natural cover whose coverage on the ground changes significantly due to extreme weather such as heavy rain and heavy snow, generally including rain, snow and other natural covers. The three-way classification probabilities outputted correspond to the probability that the corresponding parts in the initial image and the change image belong to the ground cover, and the probability that the changed area of ​​the change image relative to the initial image belongs to the ground cover.

[0022] Combination Figure 2, the change detection network 1 includes a first branch network, a second branch network and a third branch network. The first branch network includes a first encoder 11, a first decoder 12 and a first classifier 13, the second branch network includes a second encoder 14, a second decoder 15 and a second classifier 16, and the third branch network includes a difference decoder 17 and a difference classifier 18.

[0023] Among them, for the first branch network, the first encoder 11 is used to receive an initial image in an image pair, perform image encoding on the initial image, and obtain initial encoding features; the first decoder 12 is used to receive the initial encoding features, perform image decoding on the initial encoding features, and obtain initial decoding features; the first classifier 13 is used to classify the initial decoding features and output the probability that the corresponding part in one initial image belongs to the surface cover.

[0024] For the second branch network, the second encoder 14 is used to receive a change image in an image pair, perform image encoding on the change image, and obtain a change encoding feature; the second decoder 15 is used to receive the change encoding feature, perform image decoding on the change encoding feature, and obtain a change decoding feature; the second classifier 16 is used to classify the change decoding feature and output the probability that the corresponding part in one change image belongs to the surface cover.

[0025] For the third branch network, the difference decoder 17 is used to receive the initial coding features and the change coding features, perform image difference decoding on the difference features between the two, and obtain the difference decoding features; the difference classifier 18 is used to classify the difference decoding features and output the probability that the changed area of ​​the changed image relative to the initial image belongs to the surface cover.

[0026] In this embodiment, the first encoder 11 and the second encoder 14 mentioned in the above content both use resnet50. The first decoder 12, the second decoder 15 and the difference decoder 17 are all foreground decoders, which can directly predict the foreground (surface cover), and all use deeplabv3. The first classifier 13, the second classifier 16 and the difference classifier 18 are all fully connected layers.

[0027] In some other embodiments, other types of encoders and decoders may also be used to implement corresponding functions.

[0028] Combination Figure 3, before step S2, it also includes the step of: constructing an initial data set, the initial data set includes a plurality of labeled image pairs 10 and a plurality of unlabeled image pairs 20. The number of labeled image pairs 10 is much less than the number of unlabeled image pairs 20. The labeled image pairs 10 include labeled initial images and labeled change images with manually labeled areas of land cover changes, and the unlabeled image pairs 20 include unlabeled initial images and unlabeled change images that are not manually labeled.

[0029] Step S2: Use the training data set to train the change detection network.

[0030] It should be noted that the training data set only includes all labeled image pairs 10 in the initial data set, or includes all labeled image pairs 10 in the initial data set and some or all unlabeled image pairs 20 in the initial data set. The training of the change detection network 1 includes multiple rounds of training, each round of training uses a different training data set (including a different number of labeled image pairs 10 and unlabeled image pairs 20) to input the change detection network 1, and the multiple rounds of training of the change detection network 1 can be divided into two training stages, namely the first training stage and the second training stage.

[0031] Among them, for the first training stage, including multiple training rounds, the first round of training only uses all the labeled image pairs 10 in the initial data set, and as the number of training rounds increases, the ratio of the number of labeled image pairs 10 to the number of unlabeled image pairs 20 in the training data set used in each subsequent round of training gradually decreases to 1. Among them, except for the first round of training, the ratio of the number of labeled image pairs 10 to the number of unlabeled image pairs 20 in the training data set used in each subsequent round of training is greater than or equal to 1, that is, the number of labeled image pairs 10 used in each round of training is greater than or equal to the number of unlabeled image pairs 20 used.

[0032] Furthermore, the second training stage also includes multiple training rounds. As the number of training rounds increases, the ratio of the number of labeled image pairs 10 to the number of unlabeled image pairs 20 in the training data set used gradually decreases, until the last round of training, the training data set used includes all labeled image pairs 10 and all unlabeled image pairs 20 in the initial data set. Among them, the ratio of the number of labeled image pairs 10 to the number of unlabeled image pairs 20 in the training data set used corresponding to each round of training is less than 1, that is, the number of labeled image pairs 10 used in each round of training is less than the number of unlabeled image pairs 20 used.

[0033] like Figure 3 and Figure 4 As shown, step S2 includes the following sub-steps: Step S21: Establishing each training data set required for each training round according to the initial data set.

[0034] Among them, the training of the change detection network 1 includes N training rounds. The first training data set used in the first round of training only includes all the labeled image pairs 10 in the initial data set. The second training data set used in the second round of training includes all the labeled image pairs 10 in the initial data set and the individual unlabeled image pairs 20 in the initial data set. The third training data set used in the third round of training includes all the image pairs in the second training data set and the individual unlabeled image pairs 20 that are not used in the initial data set. And so on. The Nth training data set used in the Nth round of training includes all the image pairs in the (N-1)th training data set and the remaining individual unused unlabeled image pairs 20 in the initial data set, that is, the Nth training data set includes all the labeled image pairs 10 and all the unlabeled image pairs 20 in the initial data set.

[0035] Step S22: performing image enhancement processing on each training data set to obtain each pre-processed image set.

[0036] Combination Figure 3 , for the labeled image pair 10, only weak enhancement processing is performed to obtain the first weakly enhanced image pair. For the unlabeled image pair 20, weak enhancement processing is performed to obtain the second weakly enhanced image pair, and strong enhancement processing is also performed to obtain the strongly enhanced image pair. The first pre-processed image set includes the first weakly enhanced image pairs corresponding to all the labeled image pairs 10 in the first training data set used in the first round of training, and the i-th pre-processed image set includes the first weakly enhanced image pairs corresponding to all the labeled image pairs 10 in the i-th training data set used in the i-th round of training, as well as the second weakly enhanced image pairs and strongly enhanced image pairs corresponding to the unlabeled image pairs 20. Wherein, i=2, 3, ..., N. Therefore, the i-th training data set, the i-th pre-processed image set and the i-th round of training are associated.

[0037] Step S23: input each pre-processed image set into the change detection network in sequence and perform training for a corresponding number of training rounds.

[0038] Among them, the first preprocessed image set is input into the change detection network 1 for the first round of training; the second preprocessed image set is input into the change detection network 1 for the second round of training; and so on, the Nth preprocessed image set is input into the change detection network 1 for the Nth round of training.

[0039] In this embodiment, N=80, the 1st round of training to the 20th round of training is the first training stage, and the 21st round of training to the 80th round of training is the second training stage.

[0040] Combination Figure 2In the first training stage, the weights of the first encoder 11 and the second encoder 14, as well as the two auxiliary trained first decoders 12 and the second decoders 15 are frozen (i.e., fixed weights are used, and the corresponding weights do not participate in training updates), and only the difference decoder 17 and the difference classifier 18 are trained.

[0041] Among them, for the first round of training, the corresponding first training data set includes all the labeled image pairs 10 in the initial training set. For the second to 20th rounds of training, the corresponding second training data sets to the 20th training data sets all include the labeled image pairs 10 in the first round of training, and also include the second to 20th training data sets gradually adding unlabeled image pairs 20 that were not used in the previous round of training. In this process, the ratio of the number of labeled image pairs 10 to the number of unlabeled image pairs 20 gradually decreases, and in the 20th training data set corresponding to the 20th round of training, the number of labeled image pairs 10: the number of unlabeled image pairs 20 = 1:1.

[0042] Furthermore, for the 10th training data set corresponding to the 10th round of training, the number of labeled image pairs 10: the number of unlabeled image pairs 20 = 3:1.

[0043] Further, for the first preprocessed image set corresponding to the first training data set, the labeled image pair 10 in the first training data set is weakly enhanced to obtain the corresponding first preprocessed image set, and then the first weakly enhanced image pair in the first preprocessed image set can be input into the change detection network 1 for the first round of training, which can ensure that the change detection network 1 can learn relatively accurate parameters. For the second preprocessed image set to the 20th preprocessed image set corresponding to the second training data set to the 20th training data set, the labeled image pair 10 in the second training data set to the 20th training data set is weakly enhanced to obtain the first weakly enhanced image pair, and the unlabeled image pair 20 is weakly enhanced and strongly enhanced to obtain the corresponding second weakly enhanced image pair and strongly enhanced image pair, and the corresponding second preprocessed image set to the 20th preprocessed image set can be obtained.

[0044] Therefore, in the first round of training, only the first weakly enhanced image pair in the first preprocessed image set is input into the change detection network 1 for the first round of training; in the second to 20th rounds of training, the first weakly enhanced image pair, the second weakly enhanced image pair and the strongly enhanced image pair in the second preprocessed image set to the 20th preprocessed image set are respectively input into the change detection network 1 for the second to 20th rounds of training. In this process, the ratio of the first weakly enhanced image pair to the second weakly enhanced image pair / strongly enhanced image pair used correspondingly gradually decreases, until in the 20th round of training, the first weakly enhanced image pair: the second weakly enhanced image pair / strongly enhanced image pair = 1:1.

[0045] Combined with Figure 3 , in the first training stage, pseudo-label screening is also required to screen out the target pseudo-labels that best match the surface coverages to be detected . The pseudo-label screening adopts an adaptive screening strategy. The weights corresponding to each module in the change detection network 1 during the last K rounds of training are saved. The K classification probabilities output by the difference classifier 18 are the corresponding K pseudo-labels. A voting strategy is adopted to select the pseudo-label with the highest number of votes as the final target pseudo-label , 1 < K ≤ 20. Among them, the classification probability represents the probability that the corresponding pixel area is classified as a surface coverage. The larger the value corresponding to the classification probability, the more it indicates that the corresponding pixel area belongs to the surface coverage; the smaller the value corresponding to the classification probability, the more it indicates that the corresponding pixel area belongs to non-surface coverage / surface. For example, when K = 5, the five obtained classification probabilities are 0.8, 0.2, 0.8, 0.8, 0.2 in sequence. Among them, there are three 0.8s and two 0.2s. Obviously, the number of votes (confidence) corresponding to 0.8 is the highest, and the corresponding pseudo-label is the target pseudo-label of the surface coverage to be detected .

[0046] Combined with Figure 2 , in the second training stage, the weights of the two auxiliary training first decoder 12 and second decoder 15 are still frozen, but the first encoder 11 and second encoder 14 participate in the training. At this time, as the training progresses, the ratio of the number of labeled image pairs 10 to the number of unlabeled image pairs 20 in the training dataset used gradually decreases, and the proportion of unlabeled image pairs 20 gradually increases. That is, as the number of training rounds increases, in the preprocessed image set input to the change detection network 1, the ratio of the number of first weakly enhanced image pairs to the number of second weakly enhanced image pairs / strongly enhanced image pairs gradually decreases, and the proportion of second weakly enhanced image pairs / strongly enhanced image pairs gradually increases

[0047] Among them, for the 21st training dataset corresponding to the 21st round of training, the 20th training dataset corresponding to the 20th round of training is retained, and individual unused unlabeled image pairs 20 in the initial dataset are added to form the 21st training dataset; for the 22nd training dataset corresponding to the 22nd round of training, the 21st training dataset corresponding to the 21st round of training is retained, and individual unused unlabeled image pairs 20 in the initial dataset are added to form the 22nd training dataset; and so on, until the 80th training dataset corresponding to the 80th round of training. The 80th training dataset includes all the labeled image pairs 10 and all the unlabeled image pairs 20 in the initial dataset

[0048] Furthermore, for the 60th training dataset corresponding to the 60th round of training, the ratio of the number of labeled image pairs 10 to the number of unlabeled image pairs 20 = 1:3

[0049] Furthermore, for the 21st pre-processed image set - the 80th pre-processed image set corresponding to the 21st training data set - the 80th training data set respectively, the labeled image pair 10 in the 21st training data set - the 80th training data set is weakly enhanced to obtain a first weakly enhanced image pair, and the unlabeled image pair 20 is weakly enhanced and strongly enhanced to obtain the corresponding second weakly enhanced image pair and strongly enhanced image pair, and the corresponding 21st pre-processed image set - the 80th pre-processed image set can be obtained.

[0050] Therefore, in the 21st round of training, the first weakly enhanced image pair, the second weakly enhanced image pair and the strongly enhanced image pair in the 21st pre-processed image set are input into the change detection network 1 for the 21st round of training; in the 22nd round of training, the first weakly enhanced image pair, the second weakly enhanced image pair and the strongly enhanced image pair in the 22nd pre-processed image set are input into the change detection network 1 for the 22nd round of training; and so on, until the 80th round of training, the first weakly enhanced image pair, the second weakly enhanced image pair and the strongly enhanced image pair in the 80th pre-processed image set are input into the change detection network 1 for the 80th round of training.

[0051] In this embodiment, all labeled image pairs 10 are used at the beginning of the first round of training. In each subsequent round of training, the training data set used in the previous round is retained, and new unlabeled image pairs 20 are gradually added until all data in the initial data set are used in the last round of training, thereby ensuring the accuracy of the training results.

[0052] It should be noted that the main goal of contrastive learning is to map similar pairs of samples to close positions and dissimilar pairs of samples to far positions, and contrastive loss helps achieve this goal. Contrastive loss is a loss function that is often used to train contrastive learning models, which aim to learn similarities and differences in data. Consistency Regularization Loss can help the model utilize a large amount of unlabeled data, and through consistency constraints, the model's predictions on unlabeled data are consistent with those on labeled data, thereby improving the performance of the model.

[0053] In the present application, the change detection network 1 is a contrastive learning model, and the change detection network 1 can be trained using contrastive loss and consistency regularization loss.

[0054] Combination Figure 2 and Figure 3 In each round of training, the first classification probability corresponding to the labeled image pair 10 output by the first classifier 13 is calculated. The weakly enhanced difference probability corresponding to the labeled image pair 10 output by the difference classifier 18 Do the first contrast loss ; And the second classification probability output by the second classifier 16 Weakly enhanced difference probability with the output of difference classifier 18 Do the first contrast loss , so that the background is moved away and the pixel area corresponding to the surface cover is moved closer, so that the identified change area is closer to the pixel area corresponding to the surface cover, and the change of the surface cover can be better detected.

[0055] Furthermore, for other training rounds except the first training round, the first classification probability corresponding to the unlabeled image pair 20 output by the first classifier 13 is further included: The second classification probability corresponding to the unlabeled image pair 20 output by the second classifier 16 , respectively, and the strongly enhanced difference probabilities corresponding to the unlabeled image pair 20 output by the difference classifier 18 Do the second contrast loss However, it should be emphasized that the second contrast loss The weight of is very low. For example, the first contrast loss The weight is 0.4, the second contrast loss The weight of is 0.1.

[0056] In some other embodiments, since the strong enhancement processing has a greater impact on the image, the first classification probability corresponding to the unlabeled image pair 20 is and the second classification probability It is also possible not to respectively compare the strongly enhanced difference probabilities corresponding to the unlabeled image pair 20 Do the second contrast loss .

[0057] Among them, for the training of the second training stage, it is also necessary to convert the target pseudo labels obtained in the first training stage Strongly enhanced difference probability with the output of difference classifier 18 Do consistency regularization loss , so that the prediction of the change detection network 1 on the unlabeled image pair 20 is consistent with the prediction on the labeled image pair 10, thereby improving the detection performance of the change detection network 1.

[0058] It should be noted that, during the training of the change detection network 1, the first branch network and the second branch network share weights.

[0059] In this embodiment, the cross entropy loss function is used in the change detection network 1: ; in, p represents the classification probability; if the input of the change detection network 1 is the first weakly enhanced image pair corresponding to the labeled image pair 10, then y represents the true label; if the input of the change detection network 1 is the second weakly enhanced image pair and the strongly enhanced image pair corresponding to the unlabeled image pair 20, then y represents a pseudo label.

[0060] Step S3: Acquire a pair of images to be detected, wherein the pair of images to be detected includes an initial image to be detected before the ground cover changes and a changed image to be detected after the change.

[0061] It should be noted that the pair of images to be detected includes an initial image to be detected and a changed image to be detected corresponding to before and after the ground cover changes.

[0062] Among them, a sensor can be used to obtain remote sensing images, or a fixed-focus camera can be used to obtain real-time videos or images, so as to obtain the initial image to be detected and the change image to be detected before and after the surface cover changes corresponding to different time points. After that, the change detection network 1 trained in step S2 can be used to detect the changes in the surface cover in the initial image to be detected and the change image to be detected, so as to achieve accurate and rapid prediction and identification of natural disasters that cause changes in surface cover.

[0063] Step S4: Using the change detection network, detect the change of the ground cover from the initial image to be detected to the change image to be detected.

[0064] Combination Figure 2 and Figure 3 , the initial image to be detected and the change image to be detected are weakly enhanced respectively to obtain the weakly enhanced initial image and the weakly enhanced change image respectively. Then the obtained weakly enhanced initial image and the weakly enhanced change image are respectively input into the first encoder 11 and the second encoder 14 in the change detection network 1 for image encoding to obtain the first encoding feature and the second encoding feature respectively. Then, the first decoder 12 decodes the first encoding feature to obtain the first decoding feature; the second decoder 15 decodes the second encoding feature to obtain the second decoding feature; the difference decoder 17 decodes the difference feature between the first encoding feature and the second encoding feature to obtain the third decoding feature. Finally, the first classifier 13, the second classifier 16 and the difference classifier 18 respectively perform pixel-level classification prediction on the first decoding feature, the second decoding feature and the third decoding feature, and output three-way classification probabilities. Among them, the difference classifier 18 directly outputs the corresponding classification probability.

[0065] Furthermore, according to the size of the classification probability, a mask corresponding to the changed area of ​​the change image to be detected relative to the initial image to be detected is obtained, and the mask is a binary image, the white part represents the surface cover, and the black part represents the non-surface cover / surface. It is judged whether the obtained classification probability exceeds the classification threshold. If it exceeds, the changed area of ​​the corresponding change image to be detected relative to the initial image to be detected represents the surface cover; if it does not exceed, the changed area of ​​the corresponding change image to be detected relative to the initial image to be detected represents the non-surface cover / surface.

[0066] It should be noted that since strong enhancement processing has a great impact on the original image, when detecting the change of the ground cover in the initial image to be detected and the change image to be detected, the initial image to be detected and the change image to be detected are weakly enhanced and input into the change detection network 1 to obtain the corresponding target classification probability. No consistency regularization loss is required, and the final target mask can be obtained according to the size of the target classification probability. The target mask shows the change of the ground cover between the initial image to be detected and the change image to be detected.

[0067] Based on the same inventive concept, Figure 5 As shown, the present invention also provides an image detection system based on changes in ground cover, the system comprising: The network construction unit 101 is used to construct a change detection network, and the change detection network includes a first branch network, a second branch network and a third branch network.

[0068] The network training unit 102 is used to train the change detection network using the training data set.

[0069] The image acquisition unit 103 is used to acquire a pair of images to be detected, wherein the pair of images to be detected includes an initial image to be detected before the ground cover changes and a changed image to be detected after the change.

[0070] The image detection unit 104 is used to detect the change of the ground cover from the initial image to be detected to the change image to be detected by using the change detection network.

[0071] In the present application, other technical features of the above-mentioned image detection system based on changes in ground cover are the same as the features disclosed in the above-mentioned method embodiment and will not be repeated here.

[0072] Based on the same inventive concept, the present application also provides an electronic device, which includes a processor, a memory and a communication circuit, and the processor is connected to the memory and the communication circuit respectively; wherein the communication circuit is used for communication connection, the memory is used to store a computer program, and the processor is used to execute the computer program to implement the above method.

[0073] See also Figure 6The electronic device described in the embodiment of the present application may specifically include a processor 210 and a memory 220. The memory 220 is coupled to the processor 210.

[0074] The processor 210 is used to control the operation of the electronic device. The processor 210 may also be referred to as a CPU (Central Processing Unit). The processor 210 may be an integrated circuit chip with signal processing capabilities. The processor 210 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor 210 may also be any conventional processor, etc.

[0075] The memory 220 is used to store computer programs, and may be a RAM, a ROM, or other types of storage terminals. Specifically, the memory 220 may include one or more computer-readable storage media, which may be non-transitory or transient. The memory 220 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage terminals, flash memory storage terminals. In some embodiments, the non-transitory computer-readable storage medium in the memory 220 is used to store at least one program code.

[0076] The processor 210 is used to execute the computer program stored in the memory 220 to implement the methods described in the various method embodiments of the present application.

[0077] In some embodiments, the electronic device may further include: a peripheral terminal interface 230 and at least one peripheral terminal. The processor 210, the memory 220 and the peripheral terminal interface 230 may be connected via a bus or a signal line. Each peripheral terminal may be connected to the peripheral terminal interface 230 via a bus, a signal line or a circuit board. Specifically, the peripheral terminal includes: at least one of a radio frequency circuit 240, a display screen 250, an audio circuit 260 and a power supply 270.

[0078] The peripheral terminal interface 230 may be used to connect at least one peripheral terminal related to I / O (Input / output) to the processor 210 and the memory 220. In some embodiments, the processor 210, the memory 220, and the peripheral terminal interface 230 are integrated on the same chip or circuit board; in some other implementations, any one or two of the processor 210, the memory 220, and the peripheral terminal interface 230 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0079] The radio frequency circuit 240 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 240 communicates with the communication network and other IoT devices through electromagnetic signals, and the radio frequency circuit 240 is the communication circuit of the electronic device. The radio frequency circuit 240 converts the electrical signal into an electromagnetic signal for transmission, or converts the received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 240 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, an operator identity module card, and the like. The radio frequency circuit 240 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: the World Wide Web, a metropolitan area network, an intranet, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 240 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0080] The display screen 250 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 250 is a touch display screen, the display screen 250 also has the ability to collect touch signals on the surface or above the surface of the display screen 250. The touch signal can be input to the processor 210 as a control signal for processing. At this time, the display screen 250 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 250 can be one, set on the front panel of the electronic device; in other embodiments, the display screen 250 can be at least two, respectively set on different surfaces of the electronic device or in a folding design; in other embodiments, the display screen 250 can be a flexible display screen, set on a curved surface or a folding surface of the electronic device. Even, the display screen 250 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 250 can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode, organic light-emitting diode).

[0081] The audio circuit 260 may include a microphone and a speaker. The microphone is used to collect sound waves from the operator and the environment, and convert the sound waves into electrical signals and input them into the processor 210 for processing, or input them into the radio frequency circuit 240 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged in different parts of the electronic device. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 210 or the radio frequency circuit 240 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 260 may also include a headphone jack.

[0082] The power supply 270 is used to power various components in the electronic device. The power supply 270 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 270 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged through a wired line, and a wireless rechargeable battery is a battery that is charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0083] For a detailed description of the functions and execution processes of each functional module or component in the electronic device embodiment of the present application, reference can be made to the description in the above-mentioned method embodiments of the present application, which will not be repeated here.

[0084] In the several embodiments provided in the present application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the various embodiments of the electronic devices described above are only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0085] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0086] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0087] Based on the same inventive concept, the present application also provides a computer-readable storage medium storing a computer program, which can be executed by a processor to implement the above method.

[0088] See also Figure 7 , if the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium 300. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions / computer programs to enable an IoT device (which can be a personal computer, server, or network terminal, etc.) or a processor (processor) to execute all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks or optical disks, and electronic terminals such as computers, mobile phones, laptops, tablet computers, cameras, etc. having the above-mentioned storage media.

[0089] The description of the execution process of the program data in the computer-readable storage medium can refer to the description in the above-mentioned method embodiments of the present application, and will not be repeated here.

[0090] It can be seen that the present invention discloses an image detection method, system, device and medium based on changes in ground cover, the method comprising the following steps: constructing a change detection network, the change detection network comprising a first branch network, a second branch network and a third branch network; using a training data set to train the change detection network; obtaining a pair of images to be detected, the pair of images to be detected comprising an initial image to be detected before the ground cover changes and a changed image to be detected after the change; using the change detection network to detect changes in ground cover from the initial image to be detected to the changed image to be detected. The method of the present invention only requires a small amount of annotated images, and can automatically detect natural disasters that cause changes in ground cover in a large range, and locate the area where they occur and the scope of influence, so that rescue work can be carried out quickly and effectively to protect people's lives and property.

[0091] The above are only embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Any equivalent structural transformations made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are included in the protection scope of the present invention.

Claims

1. An image detection method based on changes in ground cover, characterized in that: The method comprises the following steps: Constructing a change detection network, wherein the change detection network includes a first branch network, a second branch network, and a third branch network; Using a training data set, training the change detection network; Acquire a pair of images to be detected, wherein the pair of images to be detected includes an initial image to be detected before a ground cover changes and a changed image to be detected after the change; The change detection network is used to detect the change of the ground cover from the initial image to be detected to the change image to be detected.

2. The image detection method according to claim 1, characterized in that: Before the change detection network is trained using the training data set, the following steps are also included: An initial dataset is constructed, where the initial dataset includes a plurality of labeled image pairs and a plurality of unlabeled image pairs.

3. The image detection method according to claim 2, characterized in that: The method of training the change detection network using the training data set comprises the following steps: According to the initial data set, establish each training data set required for each training round; Performing image enhancement processing on each of the training data sets to obtain each preprocessed image set; Each of the preprocessed image sets is sequentially input into the change detection network for training for a corresponding number of training rounds.

4. The image detection method according to claim 3, characterized in that: The training data set only includes all the labeled image pairs in the initial data set, or includes all the labeled image pairs in the initial data set and some or all of the unlabeled image pairs in the initial data set; wherein, except for the first round of training, as the number of training rounds increases, the ratio of the number of labeled image pairs in the training data set used to the number of unlabeled image pairs decreases dynamically.

5. The image detection method according to claim 4, characterized in that: The image enhancement processing of each training data set comprises the steps of: Performing weak enhancement processing on the labeled image pair to obtain a first weakly enhanced image pair; The unlabeled image pair is subjected to weak enhancement processing and strong enhancement processing respectively to obtain a second weakly enhanced image pair and a strongly enhanced image pair.

6. The image detection method according to claim 5, characterized in that: The step of sequentially inputting each of the pre-processed image sets into the change detection network and performing a corresponding number of training rounds of training includes: In the first training stage, only the difference decoder and the difference classifier in the third branch network are trained, and the number of the labeled image pairs in the corresponding training data set is greater than or equal to the number of the unlabeled image pairs; In the second training stage, only the first encoder in the first branch network and the second encoder in the second branch network are trained, and the number of the labeled image pairs in the corresponding training data set is less than the number of the unlabeled image pairs.

7. The image detection method according to claim 5, characterized in that: The change detection network adopts the cross entropy loss function: ； in, p represents the classification probability; if the input of the change detection network is the first weakly enhanced image pair, then y represents the true label; if the input of the change detection network is the second weakly enhanced image pair and the strongly enhanced image pair, then y Represents a pseudo label.

8. An image detection system based on changes in ground cover, characterized in that: The system comprises: A network construction unit, used to construct a change detection network, wherein the change detection network includes a first branch network, a second branch network and a third branch network; A network training unit, used to train the change detection network using a training data set; An image acquisition unit, used for acquiring a pair of images to be detected, wherein the pair of images to be detected includes an initial image to be detected before a change occurs in a ground cover and a changed image to be detected after the change occurs; The image detection unit is used to detect the change of the ground cover from the initial image to be detected to the change image to be detected by using the change detection network.

9. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images