A data augmentation method, device and electronic device for remote sensing images

Abstract

The present invention discloses a data augmentation method for remote sensing images and a device-level electronic device, the method comprising: splicing any four sample images to obtain a first image; determining a fusion center in a pre-initialized grayscale image, taking the fusion center as the center of the first image, and fusing the first image with the grayscale image; determining a target fusion area within the overlapping area of ​​the grayscale image and the first image; determining a first fusion area containing a target object on the first image; the range of the first fusion area is less than or equal to the range of the target fusion area; fusing the image of the first fusion area with the image of the target area to obtain an augmented sample. Thus, by fusing the image containing the target object into the target fusion area, the obtained augmented sample contains the sample. Thus, the problem that the augmented sample contains a large number of empty samples in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a data augmentation method, device and electronic equipment for remote sensing images. Background Art

[0002] Object detection in remote sensing images has important applications in environmental monitoring, border inspection and other fields. Making full use of remote sensing image data can help improve the accuracy and stability of remote sensing image object detection networks.

[0003] Existing data augmentation methods include basic random cropping, mirroring and other methods. In order to make better use of training data, multi-image fusion sample augmentation methods have also been developed, such as cutmix, mixup, mosaic, etc. However, due to the sparse distribution of target objects in remote sensing images, the augmented samples may not contain the target objects. Such empty samples will cause the detection network training process to be unstable, thereby affecting the accuracy of target detection. Summary of the invention

[0004] In view of this, an embodiment of the present invention discloses a data augmentation method for remote sensing images, comprising:

[0005] Splicing any four sample images to obtain a first image;

[0006] Determine a fusion center in a pre-initialized grayscale image, use the fusion center as the center of the first image, and fuse the first image with the grayscale image;

[0007] Determine a target fusion area within the region where the grayscale image and the first image overlap;

[0008] Determine a first fusion region including a target object on the first image; the range of the first fusion region is less than or equal to the range of the target fusion region;

[0009] The image of the first fusion area is fused with the image of the target area to obtain an augmented sample.

[0010] Optionally, determining a target fusion area within an area where the grayscale image and the first image overlap includes:

[0011] Selecting a second fusion area including a fusion center within the region where the grayscale image and the first image overlap;

[0012] Dividing the second fusion area into four sub-areas based on the fusion center and the stitching line of the four sample images;

[0013] Any one or more of the four sub-regions represent a target fusion region.

[0014] Optionally, determining a first fusion area containing a target object on the first image includes:

[0015] Determine the center coordinates of the target object;

[0016] The first fusion region is determined by taking the central coordinates of the target object as the central coordinates of the first fusion region and taking the scale of the target fusion region as the scale of the first fusion region.

[0017] Optionally, also include:

[0018] Detecting whether the first fusion area exceeds the boundary of the first image;

[0019] If the first fusion region exceeds the boundary of the first image, the first fusion region is redefined based on the boundary of the first image and the boundary of the first fusion region, so that the first fusion region is within the boundary of the first image.

[0020] Optionally, also include:

[0021] An affine transformation is performed on the augmented samples.

[0022] Optionally, also include:

[0023] Get the detection accuracy of each target object;

[0024] Based on the detection accuracy of each target object, a first target object that does not meet the preset conditions is screened out;

[0025] A second image containing a first target object is extracted from an image of a sample in a first training sample set, and the second image is fused into a sample image in a second training sample set; the first training sample set is a sample set that has been used for training, and the second training sample set is a training sample set for subsequent training.

[0026] Optionally, the step of screening out the first target object that does not meet the preset condition based on the detection accuracy of each target object includes:

[0027] Based on the detection accuracy of each target object, determine a second target object with the highest accuracy and a third target object with the lowest accuracy;

[0028] If the difference between the detection accuracy of the second target object and the accuracy of the third target object is less than a preset threshold, it means that the third target object does not meet the preset condition.

[0029] The embodiment of the present invention discloses a data augmentation device for remote sensing images, comprising:

[0030] A stitching unit, used for stitching any four sample images to obtain a first image;

[0031] A first determining unit, configured to determine a fusion center in a pre-initialized grayscale image, use the fusion center as the center of the first image, and fuse the first image with the grayscale image;

[0032] A second determining unit, configured to determine a target fusion area within a region where the grayscale image and the first image overlap;

[0033] A third determining unit is used to determine a first fusion area containing a target object on the first image; the range of the first fusion area is smaller than or equal to the range of the target fusion area;

[0034] The first fusion unit is used to fuse the image of the first fusion area with the image of the target area to obtain an augmented sample.

[0035] Optionally, also include:

[0036] An acquisition unit, used for acquiring the detection accuracy of each target object;

[0037] A screening unit, configured to screen out first target objects that do not meet preset conditions based on the detection accuracy of each target object;

[0038] The second fusion unit is used to extract a second image containing the first target object from the image of the sample of the first training sample set, and fuse the second image into the sample image in the second training sample set; the first training sample set is a sample set that has been used for training, and the second training sample set is a training sample set for subsequent training.

[0039] An embodiment of the present invention discloses an electronic device, including:

[0040] Memory and processor;

[0041] The memory is used to store programs, and the processor is used to execute the above-mentioned data augmentation method for remote sensing images when executing the programs in the memory.

[0042] The embodiment of the present invention discloses a data augmentation method for remote sensing images and a device-level electronic device, the method comprising: splicing any four sample images to obtain a first image; determining a fusion center in a pre-initialized grayscale image, taking the fusion center as the center of the first image, and fusing the first image with the grayscale image; determining a target fusion area within the overlapping area of ​​the grayscale image and the first image; determining a first fusion area containing a target object on the first image; the range of the first fusion area is less than or equal to the range of the target fusion area; fusing the image of the first fusion area with the image of the target area to obtain an augmented sample. Thus, by fusing the image containing the target object into the target fusion area, the obtained augmented sample contains the sample. Thus, the problem that the augmented sample contains a large number of empty samples in the prior art is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0044] Figure 1 A schematic diagram of a process of a data augmentation method for remote sensing images provided by an embodiment of the present invention is shown;

[0045] Figure 2 A schematic diagram showing image stitching and image fusion in a data augmentation process provided by an embodiment of the present invention is shown;

[0046] Figure 3 A schematic diagram of selecting a fusion range provided by an embodiment of the present invention is shown;

[0047] Figure 4 A schematic diagram of a first fusion region provided by an embodiment of the present invention is shown;

[0048] Figure 5 Another schematic diagram of a process of a data augmentation method for remote sensing images disclosed in an embodiment of the present invention is shown;

[0049] Figure 6 A schematic structural diagram of a data augmentation device for remote sensing images provided by an embodiment of the present invention is shown;

[0050] Figure 7 A schematic structural diagram of an electronic device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] The data augmentation method in the prior art does not take into account the problem of sparse distribution of target objects in remote sensing images. Due to the sparse distribution of target objects, the augmented samples obtained by the data augmentation method in the prior art will have empty samples that do not contain target objects. Training the detection network with empty samples will make the detection network training process unstable, thereby affecting the accuracy of target detection.

[0053] In order to solve the above problems, an embodiment of the present invention discloses a data augmentation method for remote sensing images, which splices any four sample images to obtain a first image; determines a fusion center in a pre-initialized grayscale image, takes the fusion center as the center of the first image, and fuses the first image with the grayscale image; determines a target fusion area within the overlapping area of ​​the grayscale image and the first image; determines a first fusion area containing a target object on the first image; the range of the first fusion area is less than or equal to the range of the target fusion area; fuses the image of the first fusion area with the image of the target area to obtain an augmented sample. Thus, by fusing the image containing the target object into the target fusion area, the obtained augmented sample contains the sample. Thus, the problem that the augmented sample contains a large number of empty samples in the prior art is solved.

[0054] Example 1

[0055] refer to Figure 1 , which shows a schematic flow chart of a data augmentation method for remote sensing images provided by an embodiment of the present invention. In this embodiment, the method includes:

[0056] S101: splicing any four sample images to obtain a first image;

[0057] In this embodiment, four sample images can be randomly selected for stitching. Furthermore, in order to facilitate determination of the center position of the stitched first image, the resolutions of the four sample images can be the same, and the overlapping point of the four samples is the center point of the first image.

[0058] For example: Figure 2As shown, the four sample images A1, B1, C1, and D1 are spliced. The resolutions of the four sample images A1, B1, C1, and D1 are the same. Then the center of the first image after the four sample images A1, B1, C1, and D1 are spliced ​​is the coincidence point of the four sample images. The coincidence point is represented by (C x , C y ).

[0059] S102: determining a fusion center in a pre-initialized grayscale image, taking the fusion center as the center of the first image, and fusing the first image with the grayscale image;

[0060] In this embodiment, the fusion center may be a specified position in the grayscale image or an arbitrary position.

[0061] For example: Figure 2 As shown, the fusion center determined in the grayscale image is expressed as (C x , C y ), the fusion center can be any position in the grayscale image. The fusion center in the grayscale image is used as the center of the first image. The first image is fused with the grayscale image, so part of the image in the first image overlaps with the grayscale image.

[0062] S103: determining a target fusion area within a region where the grayscale image and the first image overlap;

[0063] In this embodiment, it can be seen from the above introduction that some areas in the grayscale image and the first image will overlap, and when the fusion center positions are different, the range of the area where the grayscale image and the first image overlap is also different.

[0064] The target fusion area may be one or more. In one implementation, the process of determining the target fusion area includes:

[0065] Selecting a second fusion area including a fusion center within the region where the grayscale image and the first image overlap;

[0066] Dividing the second fusion area into four sub-areas based on the fusion center and the stitching line of the four sample images;

[0067] Any one or more of the four sub-regions represent a target fusion region.

[0068] In this embodiment, the second fusion area is determined within the overlapping area of ​​the first image and the grayscale image. For example, it can be any area within the overlapping area of ​​the first image and the grayscale image. In one implementation, the two edges of the grayscale image can be used as the two edges of the second fusion range, and the second fusion range contains the fusion center. For example, the second fusion range can be one or more of the upper left corner, upper right corner, lower left corner and lower right corner of the grayscale image.

[0069] For example: Figure 2 As shown, the area range at the upper left corner of the grayscale image can be used as the first fusion area, and the fusion center is included in the area range.

[0070] In this embodiment, after every two images are stitched together, there will be a stitching line. The stitching line can be used as the edge line of the target fusion area, and the fusion center can be used as a corner of the target fusion area to determine four rectangular areas, wherein any one or more of the four rectangular areas can be the target fusion area.

[0071] For example: Figure 3 As shown in the figure, the fusion center is used as a corner of each area, and the stitching line is used as the edge to divide four areas, which are represented as A2, B2, C2, and D2. Therefore, any one or more area ranges of the four areas are any target fusion ranges.

[0072] For example: Figure 3 As shown, taking the second fusion range as the upper left portion of the grayscale image A2 as an example, for the region A2, xmax and ymax are the fusion centers (C x , C y ), when the fusion center is larger than the width and height of the image (w,h), xmin and ymin are C x -w and C y -h, otherwise 0, where w represents the width of each sample image, and h represents the height of each sample image. Thus, the area range of the target fusion area represented by the upper left image A2 can be expressed as the following formula 1):

[0073] 1)[Min xp1 ,Min yp1 ,Max xp1 ,Max yp1 ]=[max(C x -w,0),max(C y -h,0),C x ,C y ];

[0074] For the upper right image B2, xmin is C x, ymax is C y , when C y When it is greater than the image height h, ymin is C y -h, otherwise 0; when C x When +w is less than 2 times the image width w, xmax is C x +w, otherwise 2w, thus, the area range of the upper right corner B2 can be expressed as the following formula 2):

[0075] 2) [Min xp2 ,Min yp2 ,Max xp2 ,Max yp2 ]=[C x ,max(C y -h,0),max(C x +w,w×2),C y ];

[0076] For the lower left image C2, ymin is C y , xmax is C x , when C x When xmin is greater than the image width w, it is C x -w, otherwise 0; when C y When +h is less than 2 times the image height h, ymax is C y +h, otherwise 2h, then the area range of the lower left image can be expressed as the following formula 3):

[0077] 3) [Min xp3 ,Min yp3 ,Max xp3 ,Max yp3 ]=[max(C x -w,0),C y ,C x ,min(C y +h,h×2)];

[0078] For the lower right image D2, xmin and ymin are the fusion center (Cx, Cy). x When +w is less than 2 times the image width w, xmax is C x +w, otherwise 2w; when C y When +h is less than 2 times the image height h, ymax is C y +h, otherwise 2h, then the area range of the lower right image D2 is expressed as follows:

[0079] 4)[Min xp4 ,Min yp4 ,Max xp4 ,Max yp4]=[C x ,C y ,min(C x +w,w×2),min(C y +h,h×2)];

[0080] S104: determining a first fusion region including a target object on the first image; the range of the first fusion region is smaller than or equal to the range of the target fusion region;

[0081] In this embodiment, since the targets in the remote sensing image are relatively sparse, the determined target fusion area may not contain the target. In order to ensure that the obtained augmented sample contains the target object, in this embodiment, an image containing the target can be obtained from the first image, and the image containing the target object can be fused with the image in the target fusion range to obtain an augmented sample containing the target.

[0082] In this embodiment, there are multiple ways to determine the first fusion area containing the target object on the first image. For example, the first fusion area can be randomly determined in the first image, and the first fusion area contains the target object, and the scale of the first fusion area is the same as the scale of the target fusion area.

[0083] Alternatively, in one implementation, the process of determining the first fusion region includes:

[0084] Determine the center coordinates of the target object;

[0085] The first fusion region is determined by taking the central coordinates of the target object as the central coordinates of the first fusion region and taking the scale of the target fusion region as the scale of the first fusion region.

[0086] For example: Figure 4 As shown in FIG. B1, there is a target object (the dotted rectangular frame in FIG. B1). Taking this target object as an example, the coordinates of the center point of the target object are (C x_o ,C y_o ), the target fusion range is part B2, and the center point coordinates of part B2 are (C x_p ,C y_p ), the scale of the B2 area is used as the scale of the first fusion range, and the coordinates of the center point of the target object in B1 are used as the center of the first fusion range to determine the first fusion area.

[0087] Among them, Figure 4As shown, the area range of the first fusion area may exceed the boundary of the first image, and the part exceeding the boundary is invalid sample information. Therefore, in order to obtain valid sample information, the first fusion area fused into the target area only includes valid sample information. Then, when the range of the first fusion area exceeds the boundary of the first image, the first fusion area is re-determined based on the boundary of the first image. Specifically, it also includes: detecting whether the first fusion area exceeds the boundary of the first image;

[0088] If the first fusion region exceeds the boundary of the first image, the first fusion region is redefined based on the boundary of the first image and the boundary of the first fusion region, so that the first fusion region is within the boundary of the first image.

[0089] For example, the first fusion area may have an upper area, a lower area, a left area or a right area beyond the boundary of the first image. For the above situation, the following formula can be used to redefine the range of the first fusion area. For example, if Min xg , Min yg , Max xg , Max yg represents the range of the first fusion area. If the first fusion area exceeds the boundary of the first image, the area of ​​the first fusion area can be recalculated using the following formula 5):

[0090]

[0091] From the above introduction, it can be seen that if the first fusion area does not exceed the boundary of the first image, then the range of the first fusion area is the same as the range of the target fusion area; if the first fusion area exceeds the boundary of the first image, then the range of the first fusion area is smaller than the range of the target fusion area.

[0092] S105: Fusing the image of the first fusion area with the image of the target area to obtain an augmented sample.

[0093] In this embodiment, the target object exists in the image of the first fusion area obtained, and the image of the first fusion area is fused with the image of the target fusion area, or the image of the first fusion area is pasted onto the target fusion area, so as to obtain an augmented sample containing the target object.

[0094] In this embodiment, any four sample images are spliced ​​to obtain a first image; a fusion center is determined in a pre-initialized grayscale image, and the first image is fused with the grayscale image using the fusion center as the center of the first image; a target fusion area is determined within the overlapping region of the grayscale image and the first image; a first fusion area containing a target object is determined on the first image; the range of the first fusion area is less than or equal to the range of the target fusion area; the image of the first fusion area is fused with the image of the target area to obtain an augmented sample. Thus, by fusing the image containing the target object into the target fusion area, the obtained augmented sample contains the sample. Thus, the problem that the augmented sample contains a large number of empty samples in the prior art is solved.

[0095] Further, on the basis of the above-mentioned embodiment 1, in order to obtain more samples, an affine transformation may be performed on the obtained augmented samples containing the target object, specifically, further comprising:

[0096] An affine transformation is performed on the augmented samples.

[0097] In this embodiment, affine transformation is a linear transformation from two-dimensional coordinates to two-dimensional coordinates, which maintains the "straightness" (translator's note: straightness, that is, after the transformation, a straight line is still a straight line without bending, and an arc is still an arc) and "parallelism" (translator's note: parallelness, refers to the relative position relationship between two-dimensional graphics remains unchanged, parallel lines are still parallel lines, and the intersection angle of intersecting straight lines remains unchanged.) Then, performing affine transformation may include, for example, translating, scaling or reversing the augmented samples.

[0098] Example 2

[0099] In this embodiment, the remote sensing image includes a variety of different types of target objects, such as cars, airplanes, people, plants, etc. When the target detection network is trained through remote sensing images, the detection accuracy of the target detection network obtained for different types of target objects may be different. If the accuracy of some types of target objects is low, the detection accuracy of the target detection network will be affected. Therefore, in order to solve this problem, reference is made to Figure 5 , shows another flow chart of a data augmentation method for remote sensing images disclosed in an embodiment of the present invention. In this embodiment, the method includes:

[0100] S501: Obtaining the detection accuracy of each target object;

[0101] In this embodiment, after the target detection network is trained using training samples, the detection accuracy of the target detection network for each target object can be evaluated using a validation set.

[0102] S502: Screening out first target objects that do not meet preset conditions based on the detection accuracy of each target object;

[0103] In this embodiment, the first target object that does not meet the preset condition may be considered as a target object with low detection accuracy. In one implementation, S502 includes:

[0104] Based on the detection accuracy of each target object, determine a second target object with the highest accuracy and a third target object with the lowest accuracy;

[0105] If the difference between the detection accuracy of the second target object and the accuracy of the third target object is less than a preset threshold, it means that the third target object does not meet the preset condition.

[0106] In this embodiment, the preset threshold can be determined by the user based on the needs, or it can be obtained by calculation and simulation according to the needs. For example, the preset threshold can be 0.2, that is, if the difference between the lowest accuracy and the highest accuracy is less than 0.2, it means that the type of target object corresponding to the lowest accuracy does not meet the preset conditions.

[0107] The second target object is a type of target object with the highest accuracy, and the third target object is a type of target object with the lowest accuracy.

[0108] S503: extracting a first image containing a first target object from the images in the first training sample set, and fusing the second image into sample images in the second training sample set; the first training sample set is a sample set that has been used for training, and the second training sample set is a training sample set required for subsequent training.

[0109] In this embodiment, the size of the region of the first image containing the first target object may be random, or in order not to affect other samples in the training samples, the region of the first image may be as small as possible.

[0110] Specifically, the area range of the first image may be determined according to the size of the first target object.

[0111] When the target object is trained next time, the second training sample set may be used for training.

[0112] In this embodiment, based on the detection accuracy of each target object, the first target object that does not meet the preset conditions is screened out; the second image containing the first target object is extracted from the image of the sample in the first training sample set, and the second image is fused into the sample image in the second training sample set; the first training sample set is the sample set that has been used for training, and the second training sample set is the training sample set for subsequent training. In this way, the samples of target objects with weaker detection accuracy are increased, thereby improving the recognition accuracy of the target detection network.

[0113] refer to Figure 6 , shows a schematic diagram of the structure of a data augmentation device for remote sensing images provided by an embodiment of the present invention. In this embodiment, the device includes:

[0114] A stitching unit 601 is used to stitch any four sample images to obtain a first image;

[0115] A first determining unit 602 is configured to determine a fusion center in a pre-initialized grayscale image, use the fusion center as the center of the first image, and fuse the first image with the grayscale image;

[0116] A second determining unit 603 is used to determine a target fusion area within the region where the grayscale image and the first image overlap;

[0117] The third determining unit 604 is configured to determine a first fusion region including a target object on the first image; the range of the first fusion region is smaller than or equal to the range of the target fusion region;

[0118] The first fusion unit 605 is used to fuse the image of the first fusion area with the image of the target area to obtain an augmented sample.

[0119] Optionally, the second determining unit includes:

[0120] A selection subunit, configured to select a second fusion region including a fusion center within a region where the grayscale image and the first image overlap;

[0121] a dividing sub-unit, configured to divide the second fusion region into four sub-regions based on the fusion center and a stitching line of the four sample images;

[0122] Any one or more of the four sub-regions represent a target fusion region.

[0123] Optionally, the third determining unit includes:

[0124] A first determining subunit, used to determine the center coordinates of the target object;

[0125] The second determining subunit is configured to determine the first fusion region by taking the central coordinates of the target object as the central coordinates of the first fusion region and taking the scale of the target fusion region as the scale of the first fusion region.

[0126] Optionally, also include:

[0127] a detection subunit, configured to detect whether the first fusion area exceeds a boundary of the first image;

[0128] The third determining subunit is configured to, if the first fusion region exceeds a boundary of the first image, redefine the first fusion region based on a boundary of the first image and a boundary of the first fusion region, so that the first fusion region is within the boundary of the first image.

[0129] Optionally, also include:

[0130] An affine transformation unit is used to perform affine transformation on the augmented samples.

[0131] Optionally, also include:

[0132] An acquisition unit, used for acquiring the detection accuracy of each target object;

[0133] A screening unit, configured to screen out first target objects that do not meet preset conditions based on the detection accuracy of each target object;

[0134] The second fusion unit is used to extract a second image containing the first target object from the image of the sample of the first training sample set, and fuse the second image into the sample image in the second training sample set; the first training sample set is a sample set that has been used for training, and the second training sample set is a training sample set for subsequent training.

[0135] Through the device of this embodiment, based on the detection accuracy of each target object, the first target object that does not meet the preset conditions is screened out; the second image containing the first target object is extracted from the image of the sample in the first training sample set, and the second image is fused into the sample image in the second training sample set; the first training sample set is a sample set that has been used for training, and the second training sample set is a training sample set for subsequent training. In this way, the samples of target objects with weaker detection accuracy are increased, thereby improving the recognition accuracy of the target detection network.

[0136] refer to Figure 7 , shows a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. In this embodiment, the method includes:

[0137] Memory 701 and processor 702;

[0138] The memory 701 is used to store programs, and the processor 702 is used to execute the data augmentation method for remote sensing images described in the above-mentioned embodiment 1 and embodiment 2 when executing the programs in the memory.

[0139] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0140] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data augmentation method for remote sensing images, characterized in that: include: Splicing any four sample images to obtain a first image; Determine a fusion center in a pre-initialized grayscale image, use the fusion center as the center of the first image, and fuse the first image with the grayscale image; Determine a target fusion area within the region where the grayscale image and the first image overlap; Determine a first fusion region including a target object on the first image; the range of the first fusion region is less than or equal to the range of the target fusion region; Fusing the image of the first fusion area with the image of the target fusion area to obtain an augmented sample; Wherein, determining the target fusion area within the region where the grayscale image and the first image overlap includes: Selecting a second fusion area including a fusion center within the region where the grayscale image and the first image overlap; Dividing the second fusion area into four sub-areas based on the fusion center and the stitching line of the four sample images; Any one or more of the four sub-regions represent a target fusion region; Wherein, determining a first fusion area containing a target object on the first image includes: Determine the center coordinates of the target object; The first fusion region is determined by taking the central coordinates of the target object as the central coordinates of the first fusion region and taking the scale of the target fusion region as the scale of the first fusion region.

2. The method according to claim 1, characterized in that Also includes: Detecting whether the first fusion area exceeds the boundary of the first image; If the first fusion region exceeds the boundary of the first image, the first fusion region is redefined based on the boundary of the first image and the boundary of the first fusion region, so that the first fusion region is within the boundary of the first image.

3. The method according to claim 1, characterized in that Also includes: An affine transformation is performed on the augmented samples.

4. The method according to claim 1, characterized in that: Also includes: Get the detection accuracy of each target object; Based on the detection accuracy of each target object, a first target object that does not meet the preset conditions is screened out; Extracting a second image containing the first target object from an image of a sample in the first training sample set, and fusing the second image into a sample image in the second training sample set; The first training sample set is a sample set that has been used for training, and the second training sample set is a training sample set for subsequent training.

5. The method according to claim 4, characterized in that The step of screening out the first target object that does not meet the preset condition based on the detection accuracy of each target object includes: Based on the detection accuracy of each target object, determine a second target object with the highest accuracy and a third target object with the lowest accuracy; If the difference between the detection accuracy of the second target object and the accuracy of the third target object is less than a preset threshold, it means that the third target object does not meet the preset condition.

6. A data augmentation device for remote sensing images, characterized in that: The method for data augmentation of remote sensing images according to claim 1 is applied, wherein the data augmentation device comprises: A stitching unit, used for stitching any four sample images to obtain a first image; A first determining unit, configured to determine a fusion center in a pre-initialized grayscale image, use the fusion center as the center of the first image, and fuse the first image with the grayscale image; A second determining unit, configured to determine a target fusion area within a region where the grayscale image and the first image overlap; A third determining unit is used to determine a first fusion area containing a target object on the first image; the range of the first fusion area is smaller than or equal to the range of the target fusion area; The first fusion unit is used to fuse the image of the first fusion area with the image of the target fusion area to obtain an augmented sample.

7. The device according to claim 6, characterized in that Also includes: An acquisition unit, used for acquiring the detection accuracy of each target object; A screening unit, configured to screen out first target objects that do not meet preset conditions based on the detection accuracy of each target object; A second fusion unit, configured to extract a second image containing the first target object from the image of the sample in the first training sample set, and fuse the second image into the sample image in the second training sample set; The first training sample set is a sample set that has been used for training, and the second training sample set is a training sample set for subsequent training.

8. An electronic device, characterized in that: include: Memory and processor; The memory is used to store programs, and the processor is used to execute the data augmentation method for remote sensing images described in any one of claims 1 to 5 when executing the programs in the memory.

Images