Battlefield environment change detection method and device and electronic equipment

Abstract

The invention provides a battlefield environment change detection method and device and electronic equipment, and relates to the technical field of image processing. Wherein the battlefield environment image comprises a battlefield environment front time phase image and a battlefield environment rear time phase image; and inputting the battlefield environment pre-time-phase image and the battlefield environment post-time-phase image into a battlefield environment change detection model to obtain a battlefield environment change detection result output by the battlefield environment change detection model. According to the technical scheme of the invention, battlefield environment element extraction and rough multi-class change detection are carried out on the battlefield environment pre-time-phase image and the battlefield environment post-time-phase image based on the battlefield environment change detection model; and carrying out fusion processing on battlefield environment element extraction and rough multi-class change detection to obtain a battlefield environment change detection result of accurate multi-class change detection, thereby improving accurate detection of battlefield environment change.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a battlefield environment change detection method, device and electronic equipment. Background Art

[0002] In modern warfare, real-time monitoring of battlefield environment and rapid detection of changes play a vital role in command decision-making, tactical deployment, battlefield situation, and damage assessment. With the rapid development of remote sensing technology, computer vision, and artificial intelligence, battlefield environment change detection has become one of the hot topics in military research. However, the battlefield environment is complex and changeable, with numerous and diverse elements, coupled with interference from natural factors such as weather, lighting, and obstructions, making battlefield environment change detection face many challenges.

[0003] At present, in the existing technology, most traditional battlefield environment change detection methods rely on single sensor data or simple image processing technology. These methods often have problems such as low detection efficiency, low accuracy, and poor robustness when processing large-scale, high-resolution battlefield environment remote sensing images. Especially when the battlefield environment elements have large morphological changes, complex backgrounds, or a lot of noise, traditional methods are difficult to effectively extract and accurately identify battlefield environment change information. Summary of the invention

[0004] The present invention provides a battlefield environment change detection method, device and electronic equipment to solve the defect in the prior art that it is difficult to effectively extract and accurately identify the change information of the battlefield environment, realize battlefield environment element extraction and rough multi-class change detection of the battlefield environment pre-phase image and the battlefield environment post-phase image based on the battlefield environment change detection model, and fuse the battlefield environment element extraction and rough multi-class change detection to obtain a battlefield environment change detection result of accurate multi-class change detection, thereby improving the accurate detection of battlefield environment changes.

[0005] The present invention provides a battlefield environment change detection method, comprising the following steps.

[0006] Obtain battlefield environment images; wherein the battlefield environment images include battlefield environment pre-phase images and battlefield environment post-phase images.

[0007] The battlefield environment before-phase images and the battlefield environment after-phase images are input into the battlefield environment change detection model to obtain the battlefield environment change detection result output by the battlefield environment change detection model; wherein, the battlefield environment change detection model is trained based on a sample data set of battlefield environment impact changes, and the battlefield environment change detection model is a model that extracts battlefield environment elements and performs rough multi-class change detection on the battlefield environment before-phase images and the battlefield environment after-phase images, and performs fusion processing on the battlefield environment element extraction and the rough multi-class change detection; the battlefield environment change detection result is an accurate multi-class change detection result obtained by the battlefield environment change detection model through the fusion processing of the battlefield environment element extraction and the rough multi-class change detection.

[0008] According to a battlefield environment change detection method provided by the present invention, a battlefield environment pre-phase image and a battlefield environment post-phase image are input into a battlefield environment change detection model to obtain a battlefield environment change detection result output by the battlefield environment change detection model, including: inputting the battlefield environment pre-phase image and the battlefield environment post-phase image into an adaptive twin change detection sub-network of the battlefield environment change detection model to obtain a rough multi-class change detection result output by the adaptive twin change detection sub-network; wherein the adaptive twin change detection sub-network refers to a network that performs rough detection on the battlefield environment pre-phase image and the battlefield environment post-phase image; respectively inputting the battlefield environment pre-phase image and the battlefield environment post-phase image into the semantic segmentation sub-network of the battlefield environment change detection model. , obtain the battlefield environment pre-phase element extraction results corresponding to the battlefield environment pre-phase image output by the semantic segmentation subnetwork and the battlefield environment post-phase element extraction results corresponding to the battlefield environment post-phase image; wherein, the semantic segmentation subnetwork refers to the network that extracts environmental elements from the battlefield environment pre-phase image and the battlefield environment post-phase image; the rough multi-class change detection results, the battlefield environment pre-phase element extraction results and the battlefield environment post-phase element extraction results are input into the result fusion device of the battlefield environment change detection model to obtain the battlefield environment change detection results output by the result fusion device; wherein, the result fusion device refers to the fusion device that fuses the rough multi-class change detection results, the battlefield environment pre-phase element extraction results and the battlefield environment post-phase element extraction results.

[0009] According to a battlefield environment change detection method provided by the present invention, the battlefield environment before-phase image and the battlefield environment after-phase image are input into the adaptive twin change detection subnetwork of the battlefield environment change detection model to obtain a rough multi-class change detection result output by the adaptive twin change detection subnetwork, including: inputting the battlefield environment before-phase image and the battlefield environment after-phase image into the encoder of the adaptive twin change detection subnetwork to obtain a fused feature map output by the encoder; wherein the encoder refers to a tool for extracting multi-scale and multi-level feature maps of the battlefield environment before-phase image and the battlefield environment after-phase image; inputting the fused feature map into the decoder of the adaptive twin change detection subnetwork to obtain a rough multi-class change detection result output by the decoder; wherein the decoder refers to a tool for performing multi-class change information detection on the fused feature map through the joint action of the self-attention mechanism and the cross-attention mechanism.

[0010] According to a battlefield environment change detection method provided by the present invention, a front-phase image and a back-phase image of the battlefield environment are input into the encoder of an adaptive twin change detection subnetwork to obtain a fused feature map output by the encoder, including: inputting the front-phase image and the back-phase image of the battlefield environment into the feature extractor of the encoder of the adaptive twin change detection subnetwork to obtain a front-phase feature map corresponding to the front-phase image of the battlefield environment and a back-phase feature map corresponding to the back-phase image of the battlefield environment output by the feature extractor; wherein the feature extractor refers to an extractor that extracts feature maps from the front-phase image and the back-phase image of the battlefield environment; inputting the front-phase feature map and the back-phase feature map into the feature fusion device of the encoder of the adaptive twin change detection subnetwork to obtain a fused feature map output by the feature fusion device; wherein the feature fusion device refers to a fusion device that fuses feature maps from the front-phase feature map and the back-phase feature map.

[0011] According to a battlefield environment change detection method provided by the present invention, the training process of a battlefield environment change detection model includes the following: obtaining a battlefield environment impact change sample data set; wherein the battlefield environment impact change sample data set includes a battlefield environment change training sample set, a battlefield environment change verification sample set and a battlefield environment change test sample set; iteratively training an adaptive twin network model according to the battlefield environment change training sample set to obtain an initial battlefield environment change detection model; wherein the adaptive twin network model is a pre-acquired basic model for battlefield environment detection; iteratively tuning the initial battlefield environment change detection model according to the battlefield environment change verification sample set until the change detection loss of the initial battlefield environment change detection model is not reduced, thereby obtaining a candidate battlefield environment change detection model; testing the candidate battlefield environment change detection model according to the battlefield environment change test sample set until the model accuracy of the candidate battlefield environment change detection model reaches a preset accuracy, thereby obtaining a battlefield environment change detection model.

[0012] A battlefield environment change detection method provided by the present invention obtains a sample data set of battlefield environment impact change, including: obtaining sample images of a battlefield environment before the phase, sample images of a battlefield environment after the phase, and typical battlefield environment elements; wherein the typical battlefield environment elements are predefined battlefield environment elements; constructing a battlefield environment change matrix according to the typical battlefield environment elements; labeling change sample vector data according to the battlefield environment change matrix; and determining the sample data set of battlefield environment impact change according to the sample images of the battlefield environment before the phase, sample images of the battlefield environment after the phase, and change sample vector data.

[0013] According to a battlefield environment change detection method provided by the present invention, a battlefield environment impact change sample data set is determined based on a battlefield environment pre-phase sample image, a battlefield environment post-phase sample image and change sample vector data, including: when the change sample vector data quality check passes, vector rasterization processing is performed on the change sample vector data to generate change sample raster label data; the battlefield environment pre-phase sample image, the battlefield environment post-phase sample image and the change sample raster label data are cropped respectively to generate pre-phase image tiles corresponding to the battlefield environment pre-phase sample image, post-phase image tiles corresponding to the battlefield environment post-phase sample image and change sample label tiles corresponding to the change sample raster label data; image enhancement processing is performed on the pre-phase image tiles and the post-phase image tiles to generate target image tiles; pseudo-color processing is performed on the change sample label tiles to generate color label tiles; and the battlefield environment impact change sample data set is determined based on the target image tiles and the color label tiles.

[0014] A battlefield environment change detection method provided by the present invention determines a battlefield environment impact change sample data set based on target image tiles and color label tiles, including: performing validity checks on the target image tiles and the color label tiles respectively, generating a valid image tile data set corresponding to the target image tiles and a sample label tile data set corresponding to the color label tiles; performing augmentation processing on the valid image tile data set and the sample label tile data set respectively, generating an augmented image tile data set corresponding to the valid image tile data set and an augmented sample label tile data set corresponding to the sample label tile data set; determining a battlefield environment impact change sample data set based on the augmented image tile data set and the augmented sample label tile data set; wherein the battlefield environment impact change sample data set is a data set that divides the augmented image tile data set and the augmented sample label tile data set into a battlefield environment change training sample set, a battlefield environment change verification sample set and a battlefield environment change test sample set.

[0015] The present invention also provides a battlefield environment change detection device, comprising the following modules.

[0016] The environmental image acquisition module is used to acquire battlefield environmental images; wherein the battlefield environmental images include battlefield environment pre-phase images and battlefield environment post-phase images.

[0017] The detection result determination module is used to input the battlefield environment before-phase images and the battlefield environment after-phase images into the battlefield environment change detection model to obtain the battlefield environment change detection result output by the battlefield environment change detection model; wherein, the battlefield environment change detection model is trained based on a battlefield environment impact change sample data set, and the battlefield environment change detection model is a model that extracts battlefield environment elements and performs rough multi-class change detection on the battlefield environment before-phase images and the battlefield environment after-phase images, and performs fusion processing on the battlefield environment element extraction and the rough multi-class change detection; the battlefield environment change detection result is an accurate multi-class change detection result obtained by the battlefield environment change detection model through the fusion processing of the battlefield environment element extraction and the rough multi-class change detection.

[0018] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, any of the above-mentioned battlefield environment change detection methods is implemented.

[0019] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, it implements any of the battlefield environment change detection methods described above.

[0020] The present invention also provides a computer program product, including a computer program, which implements any of the above-mentioned battlefield environment change detection methods when executed by a processor.

[0021] The present invention provides a battlefield environment change detection method, device and electronic equipment, which obtain battlefield environment images; wherein the battlefield environment images include battlefield environment pre-phase images and battlefield environment post-phase images; the battlefield environment pre-phase images and battlefield environment post-phase images are input into a battlefield environment change detection model to obtain a battlefield environment change detection result output by the battlefield environment change detection model; wherein the battlefield environment change detection model is trained based on a battlefield environment impact change sample data set, the battlefield environment change detection model is a model that performs battlefield environment element extraction and rough multi-class change detection on the battlefield environment pre-phase images and the battlefield environment post-phase images, and performs fusion processing on the battlefield environment element extraction and the rough multi-class change detection; the battlefield environment change detection result is an accurate multi-class change detection result obtained by the battlefield environment change detection model by fusion processing the battlefield environment element extraction and the rough multi-class change detection. The technical solution of the present invention is used to solve the defect in the prior art that it is difficult to effectively extract and accurately identify the change information of the battlefield environment, and to realize battlefield environment element extraction and rough multi-class change detection of the battlefield environment pre-phase images and battlefield environment post-phase images based on the battlefield environment change detection model, and to fuse the battlefield environment element extraction and rough multi-class change detection to obtain a battlefield environment change detection result of accurate multi-class change detection, thereby improving the accurate detection of battlefield environment changes. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 It is a flow chart of the battlefield environment change detection method provided by the present invention.

[0024] Figure 2 It is a schematic diagram of the battlefield environment image provided by the present invention.

[0025] Figure 3 It is a schematic diagram of the battlefield environment change detection result provided by the present invention.

[0026] Figure 4 It is a schematic diagram of a local area battlefield environment image provided by the present invention.

[0027] Figure 5 It is a schematic diagram of the battlefield environment change detection result of a local area provided by the present invention.

[0028] Figure 6It is a schematic diagram of a battlefield environment change detection model provided by the present invention.

[0029] Figure 7 It is a schematic diagram of obtaining a sample data set of battlefield environment impact changes provided by the present invention.

[0030] Figure 8 It is a schematic diagram of determining battlefield environment change detection results provided by the present invention.

[0031] Fig. 9 It is a schematic diagram of the processing process of the battlefield environment change detection model provided by the present invention.

[0032] Fig.10 It is a structural schematic diagram of the battlefield environment change detection device provided by the present invention.

[0033] Fig.11 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] Combine the following Figure 1-Figure 8 The battlefield environment change detection method provided by the present invention is described. The battlefield environment change detection method provided by the present invention can be applicable to battlefield environment change detection situations. The executor of the method can be an electronic device or a battlefield environment change detection device arranged in the electronic device. The battlefield environment change detection device can be implemented by software, hardware or a combination of both. Figure 1 FIG. 1 is a flow chart of a battlefield environment change detection method provided by the present invention, such as Figure 1 As shown, the method includes the following steps 101 and 102.

[0036] Step 101: Acquire battlefield environment images.

[0037] In this step, the battlefield environment image includes the battlefield environment pre-phase image and the battlefield environment post-phase image. Phase refers to the time point or time period that affects the shooting in the remote sensing image; the pre-phase and post-phase refer to the two time points or time periods before and after the change respectively. The battlefield environment pre-phase image refers to the image collected at the time point before the battlefield environment changes, and the battlefield environment post-phase image refers to the image collected at the time point after the battlefield environment changes.

[0038] The battlefield environment pre-phase image and the battlefield environment post-phase image of the battlefield environment image may be, for example, a plurality of images divided according to a grid of 512 pixels×512 pixels, which is not limited in this embodiment.

[0039] Specifically, a pre-phase image of the battlefield environment collected at a time point before the battlefield environment changes and a post-phase image of the battlefield environment collected at a time point after the battlefield environment changes are obtained.

[0040] Step 102: input the battlefield environment pre-phase image and the battlefield environment post-phase image into a battlefield environment change detection model to obtain a battlefield environment change detection result output by the battlefield environment change detection model.

[0041] In this step, the battlefield environment change detection model is trained based on the battlefield environment impact change sample data set, the battlefield environment impact change sample data set is the sample data of battlefield environment changes acquired in advance, the battlefield environment change detection model is a model that extracts battlefield environment elements and performs rough multi-class change detection on the battlefield environment pre-phase images and the battlefield environment post-phase images, and fuses the battlefield environment element extraction and rough multi-class change detection; the battlefield environment change detection result is the accurate multi-class change detection result obtained by the battlefield environment change detection model through the fusion processing of the battlefield environment element extraction and rough multi-class change detection.

[0042] Specifically, the pre-phase images and post-phase images of the battlefield environment are input into the battlefield environment change detection model, and the battlefield environment change detection model is used to extract battlefield environment elements and perform rough multi-class change detection on the pre-phase images and post-phase images of the battlefield environment. The battlefield environment element extraction and the rough multi-class change detection are then fused to obtain an accurate multi-class change detection result, i.e., a battlefield environment change detection result.

[0043] For example, Figure 2 is a schematic diagram of a battlefield environment image provided by the present invention, such as Figure 2 As shown, Figure 2 The middle (a) figure shows the front phase image of the battlefield environment. Figure 2 The middle (b) figure shows the after-phase image of the battlefield environment. Figure 3 is a schematic diagram of the battlefield environment change detection result provided by the present invention, such as Figure 3 As shown, Figure 3 The part with a black background color indicates that the battlefield environment has not changed, and the part with a blue background color indicates that the battlefield environment has changed.

[0044] If the battlefield environment image is cropped in order to improve the accuracy of the battlefield environment change detection result, that is, the battlefield environment before phase image and the battlefield environment after phase image are cropped respectively, and the local area before phase image corresponding to the battlefield environment before phase image and the local area after phase image corresponding to the battlefield environment after phase image are obtained, Figure 4 is a schematic diagram of a local area battlefield environment image provided by the present invention, such as Figure 4 As shown, Figure 4 The middle (a) figure shows the previous phase image of the battlefield environment in a local area. Figure 4 The middle (b) figure shows the post-phase image of the battlefield environment in a local area. Figure 5 FIG. 1 is a schematic diagram of the battlefield environment change detection result of a local area provided by the present invention, such as Figure 5 As shown, Figure 5 The part with a black background color indicates that the battlefield environment in the local area has not changed, and the part with a blue background color indicates that the battlefield environment in the local area has changed.

[0045] The battlefield environment change detection results for the local area are specifically shown in Table 1. As shown in Table 1, the battlefield environment change detection results include the changed file identifier descriptor (FID), shape, front phase, back phase and changed area.

[0046] Table 1

[0047] In a specific embodiment, the training process of the battlefield environment change detection model includes the following: obtaining a battlefield environment impact change sample data set; wherein the battlefield environment impact change sample data set includes a battlefield environment change training sample set, a battlefield environment change verification sample set and a battlefield environment change test sample set; iteratively training the adaptive twin network model according to the battlefield environment change training sample set to obtain an initial battlefield environment change detection model; wherein the adaptive twin network model is a pre-acquired basic model for battlefield environment detection; iteratively tuning the initial battlefield environment change detection model according to the battlefield environment change verification sample set until the change detection loss of the initial battlefield environment change detection model is not reduced, thereby obtaining a candidate battlefield environment change detection model; testing the candidate battlefield environment change detection model according to the battlefield environment change test sample set until the model accuracy of the candidate battlefield environment change detection model reaches a preset accuracy, thereby obtaining a battlefield environment change detection model.

[0048] Figure 6 Schematic diagram of a battlefield environment change detection model provided by the present invention, such as Figure 6 As shown, the training process of the battlefield environment change detection model includes the following steps.

[0049] Step 601: Obtain a sample data set of battlefield environment impact changes.

[0050] Specifically, a battlefield environment impact change sample data set is obtained, and the battlefield environment impact change sample data set includes a battlefield environment change training sample set, a battlefield environment change verification sample set, and a battlefield environment change test sample set.

[0051] Step 602: Determine an initial battlefield environment change detection model based on the battlefield environment change training sample set and the adaptive twin network model.

[0052] In this step, the adaptive twin network model is a pre-acquired basic model for battlefield environment detection.

[0053] Specifically, the adaptive twin network model is iteratively trained according to the battlefield environment change training sample set to obtain an initial battlefield environment change detection model.

[0054] Step 603: Determine a candidate battlefield environment change detection model based on the battlefield environment change verification sample set and the initial battlefield environment change detection model.

[0055] Specifically, the initial battlefield environment change detection model is iteratively tuned according to the battlefield environment change verification sample set until the change detection loss of the initial battlefield environment change detection model is not reduced, and a candidate battlefield environment change detection model is obtained.

[0056] Step 604: Determine a battlefield environment change detection model based on the battlefield environment change test sample set and the candidate battlefield environment change detection models.

[0057] Specifically, the candidate battlefield environment change detection model is tested according to the battlefield environment change test sample set until the model accuracy of the candidate battlefield environment change detection model reaches a preset accuracy, thereby obtaining the battlefield environment change detection model.

[0058] In a specific implementation, for the training of the battlefield environment change detection model, during the training process, in order to improve the model robustness of the battlefield environment change detection model, the target image tiles and color label tiles are further enhanced during the training of the battlefield environment change detection model: random cropping, random color shift, random scale scaling, random up and down and horizontal inversion, random rotation, mean variance normalization and other data enhancements, and the Stochastic Gradient Descent (SGD) optimization algorithm is used to perform the gradient back propagation operation, set the iterative cycle (epoch), the number of iterations (batchsize), the number of graphics cards, the initial learning rate and other hyperparameters, and use the poly (an exponential transformation strategy) learning rate strategy for iterative training. The iterative learning rate formula is: ,in, is the current iteration number, is the initial learning rate, is the attenuation coefficient, set to 0.9, Represents the maximum number of iterations in the entire training process. The total loss used for the battlefield environment change detection model includes a multi-class change detection loss and two semantic segmentation losses, using Dice Loss ( ) and cross entropy loss CrossEntropyLoss ( , ) Multi-class change detection loss and semantic segmentation loss are evaluated, and the overall loss function is defined as: ,in, The semantic segmentation loss function representing the extraction result of the front-phase elements of the battlefield environment, The semantic segmentation loss function represents the results of post-temporal element extraction of the battlefield environment. Multi-class change detection loss function: , , ,in, is the true value label, is the prediction result obtained by the change detection model, represents the sum of the elements of the multi-class change detection matrix, express and The number of spots shared between is the number of training sample patches, is the number of change types, , To adjust the weight parameters of the multi-class change detection loss function and the semantic segmentation loss function, , . The accuracy of the battlefield environment change detection model is evaluated using instance-level evaluation indicators. The instance-level evaluation indicators refer to treating each image spot instance in the label as a sample and calculating the Recall, Precision, IoU, F1 indicators, etc. of the image spots on all images. This embodiment does not limit this.

[0059] In a specific implementation, obtaining a battlefield environment impact change sample data set includes: Obtain sample images of the battlefield environment before the phase, sample images of the battlefield environment after the phase, and typical battlefield environment elements; wherein the typical battlefield environment elements are pre-defined battlefield environment elements; construct a battlefield environment change matrix according to the typical battlefield environment elements; annotate the change sample vector data according to the battlefield environment change matrix; determine the battlefield environment impact change sample data set according to the sample images of the battlefield environment before the phase, sample images of the battlefield environment after the phase, and the change sample vector data.

[0060] In this step, the sample images of the battlefield environment before and after are sample data previously acquired for training the battlefield environment change detection model. Typical battlefield environment elements can be, for example, buildings, roads, vegetation, water bodies, structures, and others, denoted by AF respectively; the change type of the battlefield environment is determined by the typical battlefield environment elements, and the change type is the change between the above-mentioned A-F6 typical battlefield environment elements, and the change type has a total of The battlefield environment change matrix is ​​a matrix constructed based on six types of typical battlefield environment elements. The matrix type of the battlefield environment change matrix is ​​determined by the changes between the six types of elements. The matrix type is coded and represented by "1-36", which is not limited in this embodiment.

[0061] Specifically, obtain the sample images of the battlefield environment before the time phase, the sample images of the battlefield environment after the time phase and the typical battlefield environment elements; construct the battlefield environment change matrix according to the typical battlefield environment elements; annotate the change sample vector data according to the battlefield environment change matrix, and determine the data field of the change sample vector data, the data field includes the before time phase feature type code, the after time phase feature type code and the change type code annotated from the battlefield environment change matrix. The annotation accuracy of the change sample vector data is controlled within 2 pixels, and after the annotation is completed, it can be stored in the shp (a vector data format for storing data) format. After determining the change sample vector data, determine the battlefield environment impact change sample data set according to the sample images of the battlefield environment before the time phase, the sample images of the battlefield environment after the time phase and the change sample vector data.

[0062] Exemplarily, as shown in Table 2, Table 2 represents the battlefield environment change matrix, the horizontal axis in Table 2 represents the battlefield environment elements of the sample images in the post-phase of the battlefield environment, and the vertical axis in Table 2 represents the battlefield environment elements of the sample images in the pre-phase of the battlefield environment. The battlefield environment elements may be, for example, buildings A, roads B, vegetation C, water bodies D, structures E and others F. The middle table 1-36 in Table 2 represents the matrix type of changes between battlefield environment elements, for example, "1" represents "the sample images in the pre-phase of the battlefield environment are buildings, the sample images in the post-phase of the battlefield environment are buildings, and the matrix type is unchanged", "2" represents "the sample images in the pre-phase of the battlefield environment are buildings, the sample images in the post-phase of the battlefield environment are roads, and the matrix type is changed, from buildings to roads".

[0063] Table 2

[0064] In a specific embodiment, a battlefield environment impact change sample data set is determined based on a battlefield environment pre-phase sample image, a battlefield environment post-phase sample image, and change sample vector data, including: when the change sample vector data quality check passes, vector rasterization processing is performed on the change sample vector data to generate change sample raster label data; the battlefield environment pre-phase sample image, the battlefield environment post-phase sample image, and the change sample raster label data are cropped respectively to generate pre-phase image tiles corresponding to the battlefield environment pre-phase sample image, post-phase image tiles corresponding to the battlefield environment post-phase sample image, and change sample label tiles corresponding to the change sample raster label data; image enhancement processing is performed on the pre-phase image tiles and the post-phase image tiles to generate target image tiles; pseudo-color processing is performed on the change sample label tiles to generate color label tiles; and the battlefield environment impact change sample data set is determined based on the target image tiles and the color label tiles.

[0065] In a specific embodiment, a battlefield environment impact change sample data set is determined based on target image tiles and color label tiles, including: performing validity checks on the target image tiles and color label tiles respectively to generate a valid image tile data set corresponding to the target image tiles and a sample label tile data set corresponding to the color label tiles; performing augmentation processing on the valid image tile data set and the sample label tile data set respectively to generate an augmented image tile data set corresponding to the valid image tile data set and an augmented sample label tile data set corresponding to the sample label tile data set; determining a battlefield environment impact change sample data set based on the augmented image tile data set and the augmented sample label tile data set; wherein the battlefield environment impact change sample data set is a data set that divides the augmented image tile data set and the augmented sample label tile data set into a battlefield environment change training sample set, a battlefield environment change verification sample set, and a battlefield environment change test sample set.

[0066] Figure 7 is a schematic diagram of obtaining a sample data set of battlefield environment impact changes provided by the present invention. Specifically, Figure 7 As shown, obtaining a battlefield environment impact change sample data set includes the following steps.

[0067] Step 701: Determine whether the changed sample vector data passes the quality check.

[0068] Specifically, after determining the change sample vector data, a quality check is performed on the change sample vector data to check whether the change sample vector data is consistent with the sample image of the pre-phase of the battlefield environment and the sample image of the post-phase of the battlefield environment, to check whether the boundary range marking accuracy is within 2 pixels, to check whether there are null values ​​in the data field and whether the field value is correct. When it is determined that the boundary range marking accuracy of the change sample vector data is within 2 pixels, the data field of the change sample vector data has no null values ​​and the field value is correct, it is determined that the change sample vector data quality check has passed. When the change sample vector data quality check has passed, continue to execute step 702.

[0069] Step 702: Perform vector rasterization processing on the change sample vector data to generate change sample raster label data.

[0070] Specifically, when the quality check of the change sample vector data is passed, vector rasterization is performed on the change sample vector data to generate change sample raster label data.

[0071] Step 703: determine the front phase image tile, the rear phase image tile and the change sample label tile according to the front phase sample image of the battlefield environment, the rear phase sample image of the battlefield environment and the change sample grid label data.

[0072] Specifically, the sample images of the front phase of the battlefield environment, the sample images of the back phase of the battlefield environment and the change sample grid label data are cropped respectively to generate the front phase image tiles corresponding to the sample images of the front phase of the battlefield environment, the back phase image tiles corresponding to the sample images of the back phase of the battlefield environment and the change sample label tiles corresponding to the change sample grid label data.

[0073] Exemplarily, for example, the sample images before the battlefield environment, the sample images after the battlefield environment, and the change sample grid label data may be cropped according to tiles of 1024 pixels×1024 pixels in size, respectively, and this embodiment does not limit this.

[0074] Step 704: Generate target image tiles according to the previous phase image tiles and the next phase image tiles, and generate color label tiles according to the changed sample label tiles.

[0075] Specifically, the image tiles of the previous phase and the next phase are subjected to image enhancement processing such as histogram equalization and Gaussian filtering to generate target image tiles, so that the target image tiles maintain the same spectral characteristics as the image tiles of the previous phase and the next phase. At the same time, pseudo-color processing is performed on the change sample label tiles to generate color label tiles, which facilitates viewing of changes through color label tiles.

[0076] Step 705: Generate a valid image tile dataset and a sample label tile dataset according to the target image tile and the color label tile.

[0077] Specifically, the validity of the target image tiles and color label tiles is checked respectively, tiles with change samples are screened and retained, and tiles without change samples are retained according to a certain ratio, so as to generate a valid image tile dataset corresponding to the target image tiles and a sample label tile dataset corresponding to the color label tiles.

[0078] Step 706: Determine an augmented image tile dataset and an augmented sample label tile dataset according to the valid image tile dataset and the sample label tile dataset.

[0079] Specifically, the effective image tile dataset and the sample label tile dataset are respectively augmented by rotation, mirroring, filtering, etc. to generate an augmented image tile dataset corresponding to the effective image tile dataset and an augmented sample label tile dataset corresponding to the sample label tile dataset.

[0080] Step 707: Determine a battlefield environment impact change sample dataset based on the augmented image tile dataset and the augmented sample label tile dataset.

[0081] In this step, the battlefield environment impact change sample dataset is a dataset that divides the augmented image tile dataset and the augmented sample label tile dataset into a battlefield environment change training sample set, a battlefield environment change verification sample set, and a battlefield environment change test sample set.

[0082] Specifically, the augmented image tile dataset and the augmented sample label tile dataset are divided into a battlefield environment impact change sample dataset of a battlefield environment change training sample set, a battlefield environment change verification sample set and a battlefield environment change test sample set according to a preset ratio.

[0083] In a specific implementation, the pre-phase image and the post-phase image of the battlefield environment are input into a battlefield environment change detection model to obtain a battlefield environment change detection result output by the battlefield environment change detection model, including: inputting the pre-phase image and the post-phase image of the battlefield environment into an adaptive twin change detection subnetwork of the battlefield environment change detection model to obtain a rough multi-class change detection result output by the adaptive twin change detection subnetwork; wherein the adaptive twin change detection subnetwork refers to a network that performs rough detection on the pre-phase image and the post-phase image of the battlefield environment; respectively inputting the pre-phase image and the post-phase image of the battlefield environment into the semantic segmentation subnetwork of the battlefield environment change detection model to obtain a semantic segmentation result. The segmentation subnetwork outputs the battlefield environment pre-phase element extraction results corresponding to the battlefield environment pre-phase image and the battlefield environment post-phase element extraction results corresponding to the battlefield environment post-phase image; wherein, the semantic segmentation subnetwork refers to the network that extracts environmental elements from the battlefield environment pre-phase image and the battlefield environment post-phase image; the rough multi-class change detection results, the battlefield environment pre-phase element extraction results and the battlefield environment post-phase element extraction results are input into the result fusion device of the battlefield environment change detection model to obtain the battlefield environment change detection results output by the result fusion device; wherein, the result fusion device refers to the fusion device that fuses the rough multi-class change detection results, the battlefield environment pre-phase element extraction results and the battlefield environment post-phase element extraction results.

[0084] In a specific embodiment, the pre-phase image and the post-phase image of the battlefield environment are input into the adaptive twin change detection subnetwork of the battlefield environment change detection model to obtain a rough multi-class change detection result output by the adaptive twin change detection subnetwork, including: inputting the pre-phase image and the post-phase image of the battlefield environment into the encoder of the adaptive twin change detection subnetwork to obtain a fused feature map output by the encoder; wherein the encoder refers to a tool for extracting multi-scale and multi-level feature maps of the pre-phase image and the post-phase image of the battlefield environment; inputting the fused feature map into the decoder of the adaptive twin change detection subnetwork to obtain a rough multi-class change detection result output by the decoder; wherein the decoder refers to a tool for performing multi-class change information detection on the fused feature map through the joint action of the self-attention mechanism and the cross-attention mechanism.

[0085] In a specific embodiment, the front-phase image and the back-phase image of the battlefield environment are input into the encoder of the adaptive twin change detection subnetwork to obtain a fused feature map output by the encoder, including: inputting the front-phase image and the back-phase image of the battlefield environment into the feature extractor of the encoder of the adaptive twin change detection subnetwork to obtain a front-phase feature map corresponding to the front-phase image of the battlefield environment and a back-phase feature map corresponding to the back-phase image of the battlefield environment output by the feature extractor; wherein, the feature extractor refers to an extractor that extracts feature maps from the front-phase image and the back-phase image of the battlefield environment; inputting the front-phase feature map and the back-phase feature map into the feature fusion device of the encoder of the adaptive twin change detection subnetwork to obtain a fused feature map output by the feature fusion device; wherein, the feature fusion device refers to a fusion device that fuses feature maps from the front-phase feature map and the back-phase feature map.

[0086] Figure 8 is a schematic diagram of determining the battlefield environment change detection result provided by the present invention, such as Figure 8 As shown, determining the battlefield environment change detection result includes the following steps.

[0087] Step 801: Input the front-phase image and the back-phase image of the battlefield environment into the feature extractor of the encoder of the adaptive twin change detection subnetwork to obtain the front-phase feature map and the back-phase feature map output by the feature extractor.

[0088] In this step, the encoder of the adaptive twin change detection subnetwork refers to a tool for extracting multi-scale and multi-level feature maps of the pre-phase image and the post-phase image of the battlefield environment, and the feature extractor of the encoder of the adaptive twin change detection subnetwork refers to an extractor for extracting feature maps of the pre-phase image and the post-phase image of the battlefield environment. The feature extractor can be, for example, a Swin Transformer (a hierarchical visual transformer) network, which is not limited in this embodiment.

[0089] Specifically, the front-phase image and the back-phase image of the battlefield environment are input into the feature extractor of the encoder of the adaptive twin change detection subnetwork, and the front-phase feature map corresponding to the front-phase image of the battlefield environment and the back-phase feature map corresponding to the back-phase image of the battlefield environment are obtained by the feature extractor output.

[0090] Step 802: Input the previous phase feature map and the subsequent phase feature map into the feature fuser of the encoder of the adaptive twin change detection subnetwork to obtain a fused feature map output by the feature fuser.

[0091] In this step, the feature fuser refers to a fuser that fuses the feature graphs of the previous phase and the subsequent phase. The feature fuser may be, for example, concat (splicing), which is not limited in this embodiment.

[0092] Specifically, the previous phase feature map and the subsequent phase feature map are input into the feature fusion device of the encoder of the adaptive twin change detection subnetwork, and the previous phase feature map and the subsequent phase feature map are concatenated to obtain the fused feature map output by the feature fusion device.

[0093] Step 803: Input the fused feature map into the decoder of the adaptive twin change detection subnetwork to obtain a rough multi-class change detection result output by the decoder.

[0094] In this step, the decoder of the adaptive twin change detection subnetwork refers to a tool that detects multiple types of change information on the fused feature map through the combined action of the self-attention mechanism and the cross-attention mechanism. The decoder can be, for example, a Deformable Transformer, which is not limited in this embodiment.

[0095] Specifically, the fused feature map is input into the decoder of the adaptive twin change detection subnetwork. The fused feature map is flattened and spliced ​​together through the joint action of the self-attention mechanism and the cross-attention mechanism in the decoder network, and then input into the fully connected layer for multi-classification processing to obtain the rough multi-class change detection result output by the decoder.

[0096] Step 804: Input the battlefield environment before-phase image and the battlefield environment after-phase image into the semantic segmentation subnetwork of the battlefield environment change detection model respectively, and obtain the battlefield environment before-phase element extraction result and the battlefield environment after-phase element extraction result output by the semantic segmentation subnetwork.

[0097] In this step, the semantic segmentation subnetwork refers to a network that extracts environmental elements from the pre-phase image and the post-phase image of the battlefield environment. The semantic segmentation subnetwork can be, for example, a symmetric dense hybrid full convolutional network, which is not limited in this embodiment.

[0098] Specifically, the pre-phase images and post-phase images of the battlefield environment are respectively input into the semantic segmentation subnetwork of the battlefield environment change detection model, and the semantic segmentation subnetwork automatically classifies the pre-phase images and post-phase images of the battlefield environment to obtain the pre-phase element extraction results of the battlefield environment corresponding to the pre-phase images of the battlefield environment and the post-phase element extraction results of the battlefield environment corresponding to the post-phase images of the battlefield environment output by the semantic segmentation subnetwork. The pre-phase element extraction results of the battlefield environment and the post-phase element extraction results of the battlefield environment include battlefield environment elements of six categories, namely buildings, roads, vegetation, water bodies, structures, and other elements.

[0099] Step 805: Input the rough multi-class change detection results, the battlefield environment pre-phase element extraction results and the battlefield environment post-phase element extraction results into the result fusion unit of the battlefield environment change detection model to obtain the battlefield environment change detection results output by the result fusion unit.

[0100] In this step, the result fusion device refers to a fusion device that fuses the rough multi-class change detection results, the battlefield environment pre-phase element extraction results, and the battlefield environment post-phase element extraction results.

[0101] Specifically, the rough multi-class change detection results, the battlefield environment pre-phase element extraction results, and the battlefield environment post-phase element extraction results are input into the result fusion device of the battlefield environment change detection model. The rough multi-class change detection results, the battlefield environment pre-phase element extraction results, and the battlefield environment post-phase element extraction results are fused by the result fusion device to obtain battlefield environment change detection results with precise contours and change attributes. The fusion process uses morphological operations such as dilation and erosion to make each obtained image patch an independent element as much as possible. The adaptive twin change detection subnetwork outputs the rough multi-class change detection results and the normalized multi-class probability Figure 6 Each pixel in the prediction results of typical battlefield environment elements corresponds to a multi-class change detection probability vector, which is 36-dimensional, representing the probabilities of 36 types of changes. For each spot in the prediction results of typical battlefield environment elements, the probability and of all pixels in each category in the spot are calculated, and the category with the largest probability and the largest probability are selected as the type and change type of the spot before and after the change, so as to determine the battlefield environment change detection results.

[0102] For example, Fig. 9 FIG. 1 is a schematic diagram of the processing process of the battlefield environment change detection model provided by the present invention. Fig. 9As shown in the figure, the structure of the adaptive twin change detection subnetwork is divided into two encoder branches and one decoder branch. The two encoder branches share training weights to extract multi-scale and multi-level features of the front and back phases respectively; the decoder branch uses these features to detect adaptive multi-class change information. The encoder stage uses the Swin Transformer network to divide the input image into fixed-size tiles, and then converts each patch into a high-dimensional vector through a linear embedding layer; the network contains multiple stages, each stage consists of multiple Swin Transformer blocks, and each block contains two attention mechanisms, Windows Multi-Head Self-Attention (W-MSA) and Shifted Windows Multi-Head Self-Attention (SW-MSA), to achieve the fusion of local and global features. At the end of each stage, downsampling is performed through the Patch Merging (pooling) layer to reduce the spatial resolution of the feature map and increase the feature dimension, thereby capturing features of different scales, and concat to obtain a fused feature map. The Deformable Transformer network is used in the decoder stage, and a deformable attention mechanism is introduced. Through adaptation, the attention weight can be focused on the key areas in the feature map. The location and number of sampling points are learnable and dynamically adjusted according to the weights, so as to more effectively extract battlefield environment change information. The Deformable Transformer network processes the multi-scale feature maps from the encoder, and enhances the model's ability to capture change information through cross-scale feature fusion; the self-attention mechanism and the cross-attention mechanism work together to fuse and generate feature maps of different levels, which are flattened and spliced ​​together; then they are input into the fully connected layer for multi-classification processing, and the change detection results are gradually refined through iterative optimization to generate rough multi-class change detection results with change attributes. The semantic segmentation subnetwork adopts a symmetric dense hybrid full convolutional network, which consists of a hybrid connection module, a maximum pooling layer based on spatial channel fusion squeeze activation structure, a shortcut connection across the encoder-decoder structure, an upsampling layer, and a softmax (normalized exponential function) layer.A hybrid connection module is used to improve the receptive field range and sparsity of the model; a weight recalibration structure based on spatial channel fusion squeeze activation is used to dynamically adjust the feature map in the model, enhance useful features and suppress useless features; a symmetrical encoder-decoder structure is used, the encoder part is used to extract image features, and the decoder part is used to restore the feature map to the size of the input image and perform pixel-level classification; in order to improve the segmentation ability of the network and retain the detail information in the image, a multi-scale unit mapping shortcut connection across the encoder and decoder is introduced on the basis of the symmetrical encoder-decoder structure. By using a symmetrical dense hybrid full convolutional network to automatically classify the battlefield environment pre-phase image and battlefield environment post-phase image, the battlefield environment pre-phase element extraction results and battlefield environment post-phase element extraction results containing six typical battlefield environment elements such as buildings, roads, vegetation, water bodies, structures, and others are generated. Finally, the rough multi-class change detection results, battlefield environment pre-phase element extraction results, battlefield environment post-phase element extraction results and battlefield rings are fused to obtain battlefield environment change detection results with precise contours and change attributes.

[0103] The present invention provides a battlefield environment change detection method, which obtains battlefield environment images; wherein the battlefield environment images include battlefield environment pre-phase images and battlefield environment post-phase images; the battlefield environment pre-phase images and battlefield environment post-phase images are input into a battlefield environment change detection model to obtain a battlefield environment change detection result output by the battlefield environment change detection model; wherein the battlefield environment change detection model is obtained by training based on a battlefield environment impact change sample data set, the battlefield environment change detection model is a model that performs battlefield environment element extraction and rough multi-class change detection on the battlefield environment pre-phase images and the battlefield environment post-phase images, and performs fusion processing on the battlefield environment element extraction and the rough multi-class change detection; the battlefield environment change detection result is an accurate multi-class change detection result obtained by the battlefield environment change detection model by fusion processing the battlefield environment element extraction and the rough multi-class change detection. The technical solution of the present invention is used to solve the defect in the prior art that it is difficult to effectively extract and accurately identify the change information of the battlefield environment, and to realize battlefield environment element extraction and rough multi-class change detection of the battlefield environment pre-phase images and battlefield environment post-phase images based on the battlefield environment change detection model, and to fuse the battlefield environment element extraction and rough multi-class change detection to obtain a battlefield environment change detection result of accurate multi-class change detection, thereby improving the accurate detection of battlefield environment changes.

[0104] The battlefield environment change detection device provided by the present invention is described below. The battlefield environment change detection device described below and the battlefield environment change detection method described above can be referenced to each other.

[0105] Fig.10 is a schematic diagram of the structure of the battlefield environment change detection device provided by the present invention, referring to Fig.10 As shown, the battlefield environment change detection device 1000 includes: an environment image acquisition module 1001 and a detection result determination module 1002.

[0106] The environmental image acquisition module 1001 is used to acquire battlefield environmental images; wherein the battlefield environmental images include battlefield environment pre-phase images and battlefield environment post-phase images.

[0107] The detection result determination module 1002 is used to input the battlefield environment pre-phase image and the battlefield environment post-phase image into the battlefield environment change detection model to obtain the battlefield environment change detection result output by the battlefield environment change detection model; wherein, the battlefield environment change detection model is trained based on a battlefield environment impact change sample data set, and the battlefield environment change detection model is a model that extracts battlefield environment elements and performs rough multi-class change detection on the battlefield environment pre-phase image and the battlefield environment post-phase image, and performs a fusion processing on the battlefield environment element extraction and the rough multi-class change detection; the battlefield environment change detection result is an accurate multi-class change detection result obtained by the battlefield environment change detection model through the fusion processing of the battlefield environment element extraction and the rough multi-class change detection.

[0108] In an exemplary embodiment, the detection result determination module 1002 is specifically used to: input the battlefield environment before-phase image and the battlefield environment after-phase image into the adaptive twin change detection subnetwork of the battlefield environment change detection model, and obtain the rough multi-class change detection result output by the adaptive twin change detection subnetwork; wherein the adaptive twin change detection subnetwork refers to a network that performs rough detection on the battlefield environment before-phase image and the battlefield environment after-phase image; respectively input the battlefield environment before-phase image and the battlefield environment after-phase image into the semantic segmentation subnetwork of the battlefield environment change detection model, and obtain the battlefield environment corresponding to the battlefield environment before-phase image output by the semantic segmentation subnetwork. The pre-phase element extraction results of the battlefield environment and the post-phase element extraction results of the battlefield environment corresponding to the post-phase image of the battlefield environment; wherein, the semantic segmentation sub-network refers to the network that extracts environmental elements from the pre-phase image of the battlefield environment and the post-phase image of the battlefield environment; the rough multi-class change detection results, the pre-phase element extraction results of the battlefield environment and the post-phase element extraction results of the battlefield environment are input into the result fusion device of the battlefield environment change detection model to obtain the battlefield environment change detection results output by the result fusion device; wherein, the result fusion device refers to a fusion device that fuses the rough multi-class change detection results, the pre-phase element extraction results of the battlefield environment and the post-phase element extraction results of the battlefield environment.

[0109] In an example embodiment, the detection result determination module 1002 inputs the pre-phase image and the post-phase image of the battlefield environment into the adaptive twin change detection subnetwork of the battlefield environment change detection model to obtain a rough multi-class change detection result output by the adaptive twin change detection subnetwork, and is specifically used for: inputting the pre-phase image and the post-phase image of the battlefield environment into the encoder of the adaptive twin change detection subnetwork to obtain a fused feature map output by the encoder; wherein the encoder refers to a tool for extracting multi-scale and multi-level feature maps of the pre-phase image and the post-phase image of the battlefield environment; inputting the fused feature map into the decoder of the adaptive twin change detection subnetwork to obtain a rough multi-class change detection result output by the decoder; wherein the decoder refers to a tool for performing multi-class change information detection on the fused feature map through the joint action of the self-attention mechanism and the cross-attention mechanism.

[0110] In an exemplary embodiment, the detection result determination module 1002 inputs the front-phase image and the back-phase image of the battlefield environment into the encoder of the adaptive twin change detection subnetwork to obtain a fused feature map output by the encoder, and is specifically used for: inputting the front-phase image and the back-phase image of the battlefield environment into the feature extractor of the encoder of the adaptive twin change detection subnetwork to obtain a front-phase feature map corresponding to the front-phase image of the battlefield environment and a back-phase feature map corresponding to the back-phase image of the battlefield environment output by the feature extractor; wherein, the feature extractor refers to an extractor that extracts feature maps from the front-phase image and the back-phase image of the battlefield environment; inputting the front-phase feature map and the back-phase feature map into the feature fusion device of the encoder of the adaptive twin change detection subnetwork to obtain a fused feature map output by the feature fusion device; wherein, the feature fusion device refers to a fusion device that fuses feature maps from the front-phase feature map and the back-phase feature map.

[0111] In an example embodiment, the device further includes: a model training module. The model training module is used to: obtain a battlefield environment impact change sample data set; wherein the battlefield environment impact change sample data set includes a battlefield environment change training sample set, a battlefield environment change verification sample set, and a battlefield environment change test sample set; iteratively train the adaptive twin network model according to the battlefield environment change training sample set to obtain an initial battlefield environment change detection model; wherein the adaptive twin network model is a pre-acquired basic model for battlefield environment detection; iteratively tune the initial battlefield environment change detection model according to the battlefield environment change verification sample set until the change detection loss of the initial battlefield environment change detection model is not reduced, and a candidate battlefield environment change detection model is obtained; test the candidate battlefield environment change detection model according to the battlefield environment change test sample set until the model accuracy of the candidate battlefield environment change detection model reaches a preset accuracy, and a battlefield environment change detection model is obtained.

[0112] In an example embodiment, a model training module obtains a sample data set of battlefield environment impact changes, and is specifically used to: obtain sample images of the battlefield environment before the phase, sample images of the battlefield environment after the phase, and typical battlefield environment elements; wherein the typical battlefield environment elements are predefined battlefield environment elements; construct a battlefield environment change matrix based on the typical battlefield environment elements; annotate change sample vector data based on the battlefield environment change matrix; and determine a sample data set of battlefield environment impact changes based on the sample images of the battlefield environment before the phase, sample images of the battlefield environment after the phase, and change sample vector data.

[0113] In an exemplary embodiment, a model training module determines a sample data set of battlefield environment impact changes based on a sample image of a battlefield environment before the phase, a sample image of a battlefield environment after the phase, and change sample vector data, and is specifically used to: when the change sample vector data quality check passes, perform vector rasterization processing on the change sample vector data to generate change sample raster label data; respectively crop the sample image of the battlefield environment before the phase, the sample image of the battlefield environment after the phase, and the change sample raster label data to generate a pre-phase image tile corresponding to the sample image of the battlefield environment before the phase, a post-phase image tile corresponding to the sample image of the battlefield environment after the phase, and a change sample label tile corresponding to the change sample raster label data; perform image enhancement processing on the pre-phase image tile and the post-phase image tile to generate a target image tile; perform pseudo-color processing on the change sample label tile to generate a color label tile; and determine the sample data set of battlefield environment impact changes based on the target image tile and the color label tile.

[0114] In an exemplary embodiment, a model training module determines a sample data set of battlefield environment impact changes based on target image tiles and color label tiles, and is specifically used to: perform validity checks on the target image tiles and color label tiles, respectively, to generate a valid image tile data set corresponding to the target image tiles and a sample label tile data set corresponding to the color label tiles; perform augmentation processing on the valid image tile data set and the sample label tile data set, respectively, to generate an augmented image tile data set corresponding to the valid image tile data set and an augmented sample label tile data set corresponding to the sample label tile data set; determine a sample data set of battlefield environment impact changes based on the augmented image tile data set and the augmented sample label tile data set; wherein the sample data set of battlefield environment impact changes is a data set that divides the augmented image tile data set and the augmented sample label tile data set into a battlefield environment change training sample set, a battlefield environment change verification sample set, and a battlefield environment change test sample set.

[0115] The device of this embodiment can be used to execute the method of any embodiment in the embodiment of the battlefield environment change detection method side. Its specific implementation process and technical effects are similar to those in the embodiment of the battlefield environment change detection method side. For details, please refer to the detailed description in the embodiment of the battlefield environment change detection method side, which will not be repeated here.

[0116] Fig.11 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Fig.11 As shown, the electronic device may include: a processor (processor) 1110 , a communication interface (Communications Interface) 1120 , a memory (memory) 1130 and a communication bus 1140 , wherein the processor 1110 , the communication interface 1120 , and the memory 1130 communicate with each other via the communication bus 1140 . Processor 1110 can call logic instructions in memory 1130 to execute a battlefield environment change detection method, which includes: obtaining battlefield environment images; wherein the battlefield environment images include battlefield environment pre-phase images and battlefield environment post-phase images; inputting battlefield environment pre-phase images and battlefield environment post-phase images into a battlefield environment change detection model to obtain a battlefield environment change detection result output by the battlefield environment change detection model; wherein the battlefield environment change detection model is trained based on a battlefield environment impact change sample data set, and the battlefield environment change detection model is a model that extracts battlefield environment elements and performs rough multi-class change detection on the battlefield environment pre-phase images and battlefield environment post-phase images, and performs fusion processing on the battlefield environment element extraction and the rough multi-class change detection; the battlefield environment change detection result is an accurate multi-class change detection result obtained by the battlefield environment change detection model through fusion processing of the battlefield environment element extraction and the rough multi-class change detection.

[0117] In addition, the logic instructions in the above-mentioned memory 1130 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0118] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the battlefield environment change detection method provided by the above-mentioned methods, which method includes: obtaining a battlefield environment image; wherein the battlefield environment image includes a pre-phase image of the battlefield environment and a post-phase image of the battlefield environment; inputting the pre-phase image of the battlefield environment and the post-phase image of the battlefield environment into a battlefield environment change detection model to obtain a battlefield environment change detection result output by the battlefield environment change detection model; wherein the battlefield environment change detection model is trained based on a battlefield environment impact change sample data set, and the battlefield environment change detection model is a model that extracts battlefield environment elements and performs rough multi-class change detection on the pre-phase image of the battlefield environment and the post-phase image of the battlefield environment, and performs fusion processing on the battlefield environment element extraction and the rough multi-class change detection; the battlefield environment change detection result is an accurate multi-class change detection result obtained by the battlefield environment change detection model by fusion processing the battlefield environment element extraction and the rough multi-class change detection.

[0119] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the battlefield environment change detection method provided by the above-mentioned methods, the method comprising: obtaining battlefield environment images; wherein the battlefield environment images include battlefield environment pre-phase images and battlefield environment post-phase images; inputting the battlefield environment pre-phase images and battlefield environment post-phase images into a battlefield environment change detection model to obtain a battlefield environment change detection result output by the battlefield environment change detection model; wherein the battlefield environment change detection model is trained based on a battlefield environment impact change sample data set, the battlefield environment change detection model is a model that performs battlefield environment element extraction and rough multi-class change detection on the battlefield environment pre-phase images and battlefield environment post-phase images, and performs fusion processing on the battlefield environment element extraction and the rough multi-class change detection; the battlefield environment change detection result is an accurate multi-class change detection result obtained by the battlefield environment change detection model by fusion processing the battlefield environment element extraction and the rough multi-class change detection.

[0120] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0121] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0122] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A battlefield environment change detection method, characterized in that: include: Acquire battlefield environment images; wherein the battlefield environment images include pre-battlefield environment images and post-battlefield environment images; The battlefield environment before-phase image and the battlefield environment after-phase image are input into the battlefield environment change detection model to obtain the battlefield environment change detection result output by the battlefield environment change detection model; wherein, the battlefield environment change detection model is obtained by training based on a sample data set of battlefield environment impact changes, and the battlefield environment change detection model is a model that performs battlefield environment element extraction and rough multi-class change detection on the battlefield environment before-phase image and the battlefield environment after-phase image, and fuses the battlefield environment element extraction and the rough multi-class change detection; the battlefield environment change detection result is an accurate multi-class change detection result obtained by fusing the battlefield environment element extraction and the rough multi-class change detection by the battlefield environment change detection model.

2. The battlefield environment change detection method according to claim 1, characterized in that: The step of inputting the battlefield environment pre-phase image and the battlefield environment post-phase image into a battlefield environment change detection model to obtain a battlefield environment change detection result output by the battlefield environment change detection model includes: Inputting the battlefield environment before-phase image and the battlefield environment after-phase image into the adaptive twin change detection subnetwork of the battlefield environment change detection model, and obtaining a rough multi-class change detection result output by the adaptive twin change detection subnetwork; wherein the adaptive twin change detection subnetwork refers to a network that performs rough detection on the battlefield environment before-phase image and the battlefield environment after-phase image; The battlefield environment pre-phase image and the battlefield environment post-phase image are respectively input into the semantic segmentation subnetwork of the battlefield environment change detection model, and the battlefield environment pre-phase element extraction result corresponding to the battlefield environment pre-phase image and the battlefield environment post-phase element extraction result corresponding to the battlefield environment post-phase image output by the semantic segmentation subnetwork are obtained; wherein the semantic segmentation subnetwork refers to a network that extracts environmental elements from the battlefield environment pre-phase image and the battlefield environment post-phase image; The rough multi-class change detection result, the battlefield environment pre-phase element extraction result and the battlefield environment post-phase element extraction result are input into the result fusion device of the battlefield environment change detection model to obtain the battlefield environment change detection result output by the result fusion device; wherein the result fusion device refers to a fusion device that fuses the rough multi-class change detection result, the battlefield environment pre-phase element extraction result and the battlefield environment post-phase element extraction result.

3. The battlefield environment change detection method according to claim 2, characterized in that: The step of inputting the battlefield environment pre-phase image and the battlefield environment post-phase image into the adaptive twin change detection subnetwork of the battlefield environment change detection model to obtain a rough multi-class change detection result output by the adaptive twin change detection subnetwork includes: Inputting the battlefield environment before-phase image and the battlefield environment after-phase image into the encoder of the adaptive twin change detection subnetwork to obtain a fusion feature map output by the encoder; wherein the encoder refers to a tool for extracting multi-scale and multi-level feature maps from the battlefield environment before-phase image and the battlefield environment after-phase image; The fused feature map is input into the decoder of the adaptive twin change detection subnetwork to obtain the rough multi-class change detection result output by the decoder; wherein the decoder refers to a tool that performs multi-class change information detection on the fused feature map through the joint action of the self-attention mechanism and the cross-attention mechanism.

4. The battlefield environment change detection method according to claim 3, characterized in that: The step of inputting the battlefield environment before-phase image and the battlefield environment after-phase image into the encoder of the adaptive twin change detection subnetwork to obtain a fusion feature map output by the encoder includes: Inputting the battlefield environment front-phase image and the battlefield environment back-phase image into the feature extractor of the encoder of the adaptive twin change detection subnetwork, and obtaining the front-phase feature map corresponding to the battlefield environment front-phase image and the back-phase feature map corresponding to the battlefield environment back-phase image output by the feature extractor; wherein the feature extractor refers to an extractor that extracts feature maps from the battlefield environment front-phase image and the battlefield environment back-phase image; The previous phase feature map and the subsequent phase feature map are input into the feature fusion device of the encoder of the adaptive twin change detection subnetwork to obtain a fused feature map output by the feature fusion device; wherein the feature fusion device refers to a fusion device that performs feature map fusion on the previous phase feature map and the subsequent phase feature map.

5. The battlefield environment change detection method according to claim 1, characterized in that: The training process of the battlefield environment change detection model includes the following: Acquire the battlefield environment impact change sample data set; wherein the battlefield environment impact change sample data set includes a battlefield environment change training sample set, a battlefield environment change verification sample set and a battlefield environment change test sample set; Iteratively training the adaptive twin network model according to the battlefield environment change training sample set to obtain an initial battlefield environment change detection model; wherein the adaptive twin network model is a pre-acquired basic model for battlefield environment detection; Iteratively tuning the initial battlefield environment change detection model according to the battlefield environment change verification sample set until a candidate battlefield environment change detection model is obtained without reducing the change detection loss of the initial battlefield environment change detection model; The candidate battlefield environment change detection model is tested according to the battlefield environment change test sample set until the model accuracy of the candidate battlefield environment change detection model reaches a preset accuracy, thereby obtaining the battlefield environment change detection model.

6. The battlefield environment change detection method according to claim 5, characterized in that: The obtaining of the battlefield environment impact change sample data set includes: Acquire a sample image of a battlefield environment before the time phase, a sample image of a battlefield environment after the time phase, and typical battlefield environment elements; wherein the typical battlefield environment elements are predefined battlefield environment elements; Constructing a battlefield environment change matrix according to the typical battlefield environment elements; Annotate the change sample vector data according to the battlefield environment change matrix; The battlefield environment impact change sample data set is determined according to the battlefield environment pre-phase sample image, the battlefield environment post-phase sample image and the change sample vector data.

7. The battlefield environment change detection method according to claim 6, characterized in that: The determining of the battlefield environment impact change sample data set according to the battlefield environment pre-phase sample image, the battlefield environment post-phase sample image and the change sample vector data includes: If the change sample vector data passes the quality check, performing vector rasterization processing on the change sample vector data to generate change sample raster label data; The battlefield environment front phase sample image, the battlefield environment back phase sample image and the change sample grid label data are respectively cropped to generate the front phase image tile corresponding to the battlefield environment front phase sample image, the back phase image tile corresponding to the battlefield environment back phase sample image and the change sample label tile corresponding to the change sample grid label data; Performing image enhancement processing on the previous phase image tiles and the next phase image tiles to generate target image tiles; Performing pseudo-color processing on the changed sample label tile to generate a color label tile; The battlefield environment impact change sample data set is determined according to the target image tile and the color label tile.

8. The battlefield environment change detection method according to claim 7, characterized in that: The determining of the battlefield environment impact change sample data set according to the target image tile and the color label tile includes: Performing validity checks on the target image tile and the color label tile respectively, and generating a valid image tile dataset corresponding to the target image tile and a sample label tile dataset corresponding to the color label tile; Performing augmentation processing on the effective image tile dataset and the sample label tile dataset respectively, to generate an augmented image tile dataset corresponding to the effective image tile dataset and an augmented sample label tile dataset corresponding to the sample label tile dataset; The battlefield environment impact change sample data set is determined according to the augmented image tile data set and the augmented sample label tile data set; wherein the battlefield environment impact change sample data set is a data set that divides the augmented image tile data set and the augmented sample label tile data set into the battlefield environment change training sample set, the battlefield environment change verification sample set and the battlefield environment change test sample set.

9. A battlefield environment change detection device, characterized in that: include: An environmental image acquisition module is used to acquire battlefield environmental images; wherein the battlefield environmental images include battlefield environmental pre-phase images and battlefield environmental post-phase images; A detection result determination module is used to input the battlefield environment before-phase image and the battlefield environment after-phase image into a battlefield environment change detection model to obtain a battlefield environment change detection result output by the battlefield environment change detection model; wherein the battlefield environment change detection model is trained based on a battlefield environment impact change sample data set, and the battlefield environment change detection model is a model that performs battlefield environment element extraction and rough multi-class change detection on the battlefield environment before-phase image and the battlefield environment after-phase image, and performs fusion processing on the battlefield environment element extraction and the rough multi-class change detection; the battlefield environment change detection result is an accurate multi-class change detection result obtained by the battlefield environment change detection model through fusion processing of the battlefield environment element extraction and the rough multi-class change detection.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the battlefield environment change detection method as described in any one of claims 1 to 8 is implemented.

Images