Target identification method and device based on double-branch network

Abstract

The invention provides a target identification method and device based on a double-branch network. The method comprises the following steps: constructing a cross-view satellite data set; a controller is constructed, the controller is used for obtaining the gray value of each training sample in the training sample set, determining the similarity between the training samples and carrying out image enhancement on each training sample, and the similarity between the enhanced training samples is smaller than a preset threshold value; respectively inputting the training sample set after image enhancement into a BiFormer network model and a Resnet101 network model to respectively obtain a first feature vector set and a second feature vector set; splicing corresponding vectors in the first feature vector set and the second feature vector set to obtain a spliced vector set; constructing a loss function, inputting the splicing vector set into a network layer, and outputting a target recognition result of the cross-view satellite by the network layer; and training the BiFormer network model, the Resnet101 network model and the network layer on the basis of the loss function. The method is high in image target recognition accuracy.

Description

Technical Field

[0001] This invention relates to the field of image detection, and in particular to a target recognition method and apparatus based on a dual-branch network. Background Technology

[0002] The purpose of cross-view satellite recognition is to match images of different satellites from different perspectives, which has significant applications in the military field. However, there is currently no in-depth research on cross-view satellite recognition, and relevant datasets are difficult to find.

[0003] Currently, open-source spatial object recognition datasets have several drawbacks: 1) too many repeated viewpoints in a single sample; 2) simple backgrounds in sample images; 3) overly clean sample images (noise-free, blurry); 4) too few categories (only 20 categories). When using the above dataset for object re-identification, for example, using 14 categories as the training set and the remaining 6 categories as the validation set (test set), the query in the validation set (test set) includes some images with significantly different viewpoints from the gallery. Due to the large number of repeated viewpoints in the training set and the limited number of training data categories, object recognition performance is poor when performing challenging cross-viewpoint dataset partitioning. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention proposes a target recognition method and apparatus based on a dual-branch network. The method is used to solve the problems of insufficient cross-view satellite recognition datasets and low image target recognition accuracy.

[0005] According to a first aspect of the present invention, a target recognition method based on a dual-branch network is provided. The method of the present invention includes the following steps:

[0006] Step S1: Acquire satellite data, model and simulate the satellite, output simulated satellite images, construct a cross-view satellite dataset based on the simulated images, and determine the training sample set and test set from the cross-view satellite dataset;

[0007] Step S2: Construct a controller, which is used to obtain the gray values ​​of each training sample in the training sample set, determine the similarity between each training sample, perform image enhancement on each training sample, and the similarity between the enhanced training samples is less than a preset threshold.

[0008] Step S3: Input the image-enhanced training sample set into the BiFormer network model and the ResNet101 network model respectively to obtain the first feature vector set and the second feature vector set respectively; concatenate the corresponding vectors in the first feature vector set and the second feature vector set to obtain the concatenated vector set;

[0009] Step S4: Construct a loss function. The concatenated vector set is input into the network layer, and the network layer outputs the target recognition result of the satellite from a different perspective. The BiFormer network model, the ResNet101 network model, and the network layer are trained based on the loss function.

[0010] Preferably, step S1 involves: acquiring satellite data, modeling and simulating the satellite, outputting simulated satellite images, constructing a cross-view satellite dataset based on the simulated images, and determining a training sample set and a test set from the cross-view satellite dataset, including:

[0011] Step S11: Acquire several 3D satellite images. Blender simulation software performs modeling and simulation of the satellite based on the 3D satellite data. The modeling and simulation includes: Blender simulation software determining simulation parameters, including the number of viewing angles, camera distance, illumination parameters, and the size of the simulation image; for each 3D satellite image, Blender simulation software adjusts the viewing angle of the 3D satellite image based on the simulation parameters, generates a grayscale image of the 3D satellite image after adjusting the viewing angle, adjusts the size of the grayscale image, and uses the grayscale image after adjusting the size as the first simulation image of the satellite.

[0012] Step S12: Add Gaussian noise to the first simulated image, perform blurring processing with a convolution kernel of 1-4, and adjust the brightness and contrast to obtain the simulated image;

[0013] Step S13: All simulated images constitute a cross-view satellite dataset, and the training sample set and test set are determined from the cross-view satellite dataset.

[0014] Preferably, step S2: constructing a controller, the controller being used to acquire the grayscale values ​​of each training sample in the training sample set, determine the similarity between each training sample, perform image enhancement on each training sample, and the similarity between the enhanced training samples is less than a preset threshold, including:

[0015] The training sample set is divided into multiple batches, and each batch includes several training samples.

[0016] For each batch, the controller performs the following operations:

[0017] Step S21: The controller acquires the grayscale values ​​of each training sample in the batch; arranges each training sample according to its storage order in the batch, and stores the number of each training sample into the first array; sets the counter num = 2; sets the loop threshold to num1;

[0018] Step S22: If num is less than or equal to num1+1, proceed to step S23; otherwise, proceed to step S24.

[0019] Step S23: Randomly shuffle the storage order of each training sample in this batch, arrange each training sample in the new order, and store the number of each training sample into an array with the number num.

[0020] The two indices stored at the same array positions in the first array and the array numbered num are denoted as x and y, respectively;

[0021] Calculate the similarity of training sample pairs consisting of two training samples numbered x and y:

[0022]

[0023] Where, μ x Let μ be the average gray value of the training sample numbered x. y Let y be the average gray value of the training sample with ID y, where C1 and C2 are constants, and σ is the mean gray value of the training sample with ID y. xy Let σ be the covariance between the gray values ​​of training sample x and training sample y. x Let σ be the standard deviation of the gray level of the training sample numbered x. y y represents the standard deviation of the gray levels of the training sample with ID y;

[0024] For all training sample pairs between the first array and the array numbered num, calculate the average similarity and store the average in a similarity array, where the similarity data is an array storing the average.

[0025] The counter num is assigned the value num+1, and the process proceeds to step S22;

[0026] Step S24: Obtain the last stored similarity in the similarity array, remove the last stored similarity from the similarity array, and then calculate the average value avg of each element in the similarity array. If the last stored similarity is greater than avg, proceed to step S25; otherwise, the batch processing is complete.

[0027] Step S25: For each element in the first array corresponding to the training sample, select several regions from the training sample, and replace each region with the corresponding region of the replacement image; the replacement image is determined by the array numbered num, and is a training sample pair with the training sample; the batch processing is completed.

[0028] Preferably, step S4 involves: constructing a loss function, inputting the concatenated vector set into the network layer, and the network layer outputting the target recognition result from the cross-view satellite; training the BiFormer network model, the ResNet101 network model, and the network layer based on the loss function, wherein the loss function is:

[0029] Loss=0.3*CE+0.3*DF+0.4*Adasp

[0030] in:

[0031] DF = DF(p) t )=-(1+a)(1-p t ) γ+b log(p t )+β

[0032] DF is the calculation function, p t DF(p) is the probability of the predicted class of the training sample. t ) represents the DF loss function value, where a = weight1, b = weight1 * weight2, and β = (distance_small * α). 2 γ is an adjustable constant parameter, b is a modulation factor, distance_small is the reduced value of the distance from the training sample to its class center, weight1 is the standard deviation of the distance from the training sample to its class center, weight2 is the normalized value of distance_small; CE is the cross-entropy loss function, and Adasp is the loss caused by the concatenation vector compared with negative and positive samples.

[0033] Preferably, the method further includes step S5: acquiring the cross-view satellite image to be identified, inputting it into the trained BiFormer network model and ResNet101 network model respectively, and outputting the target recognition result of the cross-view satellite image to be identified by the trained network layers.

[0034] According to a second aspect of the present invention, a target recognition device based on a dual-branch network is provided, the device comprising:

[0035] Dataset generation module: configured to acquire satellite data, model and simulate satellites, output simulated satellite images, construct cross-view satellite dataset based on the simulated images, and determine training sample set and test set from the cross-view satellite dataset;

[0036] Image enhancement module: configured to build a controller, the controller is used to obtain the gray values ​​of each training sample in the training sample set, determine the similarity between each training sample, perform image enhancement on each training sample, and the similarity between the enhanced training samples is less than a preset threshold.

[0037] Feature extraction module: Configured to input the image-enhanced training sample set into the BiFormer network model and the ResNet101 network model respectively, to obtain the first feature vector set and the second feature vector set respectively; concatenate the corresponding vectors in the first feature vector set and the second feature vector set to obtain the concatenated vector set;

[0038] Training module: configured to construct a loss function, the concatenated vector set is input into the network layer, and the network layer outputs the target recognition results of cross-view satellites; the BiFormer network model, ResNet101 network model and the network layer are trained based on the loss function.

[0039] According to a third aspect of the present invention, an electronic device is provided, comprising:

[0040] A processor is used to execute multiple instructions;

[0041] Memory, used to store multiple instructions;

[0042] The plurality of instructions are to be stored in the memory and loaded and executed by the processor as described above.

[0043] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, wherein a plurality of instructions are stored therein; the plurality of instructions are configured to be loaded by a processor and executed as described above.

[0044] This invention utilizes software simulation of a satellite model to create a new dataset. Furthermore, it improves the Cutmix data augmentation method by altering the shape and number of replacement regions. A controller is also designed to manage the use of the improved data augmentation method, thereby enhancing its efficiency. To address the issue of varying degrees of imbalance between different categories, this invention proposes a novel loss function, DF loss, which independently rebalances the loss contributions of different categories based on their distance from the cluster centers.

[0045] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, the preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings. Attached Figure Description

[0046] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention, and the invention is illustrated by the following drawings. In the drawings:

[0047] Figure 1 This is a schematic flowchart of a target recognition method based on a dual-branch network according to one embodiment of the present invention.

[0048] Figure 2 This is a schematic diagram of the target recognition method based on a dual-branch network according to one embodiment of the present invention.

[0049] Figure 3 This is a schematic diagram of a dataset construction process according to one embodiment of the present invention;

[0050] Figure 4 This is a schematic diagram of spatial target data simulation according to one embodiment of the present invention;

[0051] Figure 5 This is a schematic diagram illustrating the effect of a target recognition method based on a dual-branch network according to one embodiment of the present invention.

[0052] Figure 6 This is a block diagram of a target recognition device based on a dual-branch network according to one embodiment of the present invention. Detailed Implementation

[0053] First, combine Figure 1 This invention describes a target recognition method based on a dual-branch network, according to one embodiment of the present invention. Figures 1-2 As shown, the method includes the following steps:

[0054] Step S1: Acquire satellite data, model and simulate the satellite, output simulated satellite images, construct a cross-view satellite dataset based on the simulated images, and determine the training sample set and test set from the cross-view satellite dataset;

[0055] Step S2: Construct a controller, which is used to obtain the gray values ​​of each training sample in the training sample set, determine the similarity between each training sample, perform image enhancement on each training sample, and the similarity between the enhanced training samples is less than a preset threshold.

[0056] Step S3: Input the image-enhanced training sample set into the BiFormer network model and the ResNet101 network model respectively to obtain the first feature vector set and the second feature vector set respectively; concatenate the corresponding vectors in the first feature vector set and the second feature vector set to obtain the concatenated vector set;

[0057] Step S4: Construct a loss function. The concatenated vector set is input into the network layer, and the network layer outputs the target recognition result of the satellite from a different perspective. The BiFormer network model, the ResNet101 network model, and the network layer are trained based on the loss function.

[0058] Furthermore, the method also includes step S5: acquiring the cross-view satellite image to be identified, inputting it into the trained BiFormer network model and Resnet101 network model respectively, and outputting the target recognition result of the cross-view satellite image to be identified by the trained network layers.

[0059] Step S1: Acquire satellite data, perform satellite modeling and simulation, output simulated satellite images, construct a cross-view satellite dataset based on the simulated images, and determine a training sample set and a test set from the cross-view satellite dataset, including:

[0060] Step S11: Acquire several 3D satellite images. Blender simulation software performs modeling and simulation of the satellite based on the 3D satellite data. The modeling and simulation includes: Blender simulation software determining simulation parameters, including the number of viewing angles, camera distance, illumination parameters, and the size of the simulation image; for each 3D satellite image, Blender simulation software adjusts the viewing angle of the 3D satellite image based on the simulation parameters, generates a grayscale image of the 3D satellite image after adjusting the viewing angle, adjusts the size of the grayscale image, and uses the grayscale image after adjusting the size as the first simulation image of the satellite.

[0061] Step S12: Add Gaussian noise to the first simulated image, perform blurring processing with a convolution kernel of 1-4, and adjust the brightness and contrast to obtain the simulated image;

[0062] Step S13: All simulated images constitute a cross-view satellite dataset, and the training sample set and test set are determined from the cross-view satellite dataset.

[0063] In this invention, to provide stronger data support for research on space target recognition algorithms, a space target simulation dataset that better reflects the actual requirements of space target recognition projects was created based on the relevant background requirements. The new space target simulation dataset is named "STSDs_1.0 (Space Target Simulation Datasets)," and its construction process is as follows: Figure 3 As shown.

[0064] 1. Collection of satellite 3D models

[0065] Currently, during the construction of the STSDs_1.0 dataset, 112 different satellite 3Dmax files were collected from the internet to support the simulation model. Before simulation, the collected 3Dmax files were registered, deduplicated, and quality-screened, resulting in 112 usable 3Dmax files. The dataset simulation was then conducted based on these 112 different satellite 3Dmax files.

[0066] 2. Blender simulation program design

[0067] The STSDs_1.0 dataset was created using Blender simulation software and Python as the programming language. Simulation details mainly include: viewpoint settings, camera distance settings, simulation image size settings, and lighting / darkness variation settings. During the actual simulation, the final data was generated based on empirical data used to achieve these settings. The simulation parameter settings are shown in Table 1.

[0068] Table 1 Blender Simulation Parameter Presets

[0069]

[0070] After the above steps, the resulting space target simulation dataset contains image sets of 112 different satellite targets. Each satellite target contains 25 images (each image corresponds to a viewpoint), and each image is 300×300 pixels in size. The dataset ultimately contains a total of 2800 images.

[0071] 3. Post-processing of simulation data

[0072] To make the simulated data more closely resemble reality, additional data processing operations are needed to increase the dataset's complexity. The data processing operations currently applied include: adding noise (Gaussian noise), blurring, and adjusting brightness and contrast. The preset parameters for these post-processing operations are shown in Table 2.

[0073] Table 2 Preset parameters for data post-processing

[0074]

[0075] The data post-processing process consists of the following steps:

[0076] (1) Read the file

[0077] (2) Add Gaussian noise with a standard deviation of 0.2 to each image.

[0078] (3) Blur the image using a convolution kernel of 1-4.

[0079] (4) Adjust the image brightness range to -20, 20, and the contrast to 0.8-1.2.

[0080] (5) Save the post-processed image.

[0081] 4. Cross-perspective dataset partitioning

[0082] To align with the practicalities of spatial target recognition projects, the dataset needs to be partitioned with a significant span of perspectives. The current partitioning results of the STSDs_1.0 dataset are shown in Table 3. First, the entire dataset is divided into a training set (train_data) and a test set (test_data) in an 8:2 ratio. The training set contains 90 classes of satellite target image data, and the test set contains 22 classes. Then, the test set is further divided into a query set and a gallery set in a 2:8 ratio. Each class in the query set contains 5 images, and each class in the gallery set contains 20 images.

[0083] Table 3 Statistical results of the STSDs_1.0 dataset

[0084]

[0085] In existing technologies, the Cutmix data augmentation method randomly selects a square area for replacement. However, this method has the drawback that since the background of satellite datasets is a black area and the target is in the center of the image, when the selected area is around the perimeter of the image and does not contain the target, the replaced image will not produce obvious differences. Therefore, this data augmentation method is not efficient.

[0086] Step S2: Constructing a controller, the controller is used to acquire the grayscale values ​​of each training sample in the training sample set, determine the similarity between each training sample, perform image enhancement on each training sample, and the similarity between the enhanced training samples is less than a preset threshold, including:

[0087] The training sample set is divided into multiple batches, and each batch includes several training samples.

[0088] For each batch, the controller performs the following operations:

[0089] Step S21: The controller acquires the grayscale values ​​of each training sample in the batch; arranges each training sample according to its storage order in the batch, and stores the number of each training sample into the first array; sets the counter num = 2; sets the loop threshold to num1;

[0090] Step S22: If num is less than or equal to num1+1, proceed to step S23; otherwise, proceed to step S24.

[0091] Step S23: Randomly shuffle the storage order of each training sample in this batch, arrange each training sample in the new order, and store the number of each training sample into an array with the number num.

[0092] The two indices stored at the same array positions in the first array and the array numbered num are denoted as x and y, respectively;

[0093] Calculate the similarity of training sample pairs consisting of two training samples numbered x and y:

[0094]

[0095] Where, μ x Let μ be the gray value of the training sample with ID x. y Let y be the grayscale value of the training sample with ID y, C1 be a constant used to maintain stability, C2 be a constant used to maintain stability, and σ be a constant. xy Let σ be the covariance between the gray levels of training sample x and training sample y. x Let σ be the standard deviation of the gray level of the training sample numbered x. y y represents the standard deviation of the gray levels of the training sample with ID y;

[0096] For all training sample pairs between the first array and the array numbered num, calculate the average similarity and store the average in a similarity array, where the similarity data is an array storing the average.

[0097] The counter num is assigned the value num+1, and the process proceeds to step S22;

[0098] Step S24: Obtain the last stored similarity in the similarity array, remove the last stored similarity from the similarity array, and then calculate the average value avg of each element in the similarity array. If the last stored similarity is greater than avg, proceed to step S25; otherwise, the batch processing is complete.

[0099] Step S25: For each element in the first array corresponding to the training sample, select several regions from the training sample, and replace each region with the corresponding region of the replacement image; the replacement image is determined by the array numbered num, and is a training sample pair with the training sample; the batch processing is completed.

[0100] Further, in step S21, before the controller acquires the grayscale values ​​of each training sample in the batch, the controller performs perspective transformation, color augmentation, random scaling, cutout, and mixup on each training sample in the batch. Afterwards, the controller acquires the grayscale values ​​of each processed training sample in the batch.

[0101] This invention also designs a novel data augmentation method, Annulus mix. In this method, the order of images in each batch is first shuffled. Then, the images in the original order are matched one-to-one with the shuffled images, and the similarity between the images is calculated using the SSIM method. The formula for the SSIM method is as follows:

[0102]

[0103] Where μ x and μ y σ is the average gray level of the image. x and σ y C represents the standard deviation of the image's gray levels, where C = (KL). 2 L is the number of gray levels in the image. For an 8-bit image, the value of L is 255, and K << 1.

[0104] The next step is to calculate the average similarity between the images in their original order and those after shuffling, and store this average similarity in the `similarity_sum` array. This average similarity will be stored in this array for each subsequent loop. Before operating on any images, a similarity check is performed. Specifically, the average similarity between the original and shuffled images in the current batch is compared to the average similarity in the `similarity_sum` array. If the average similarity of the current batch is higher, it means the images within that batch are relatively similar, and some operation is needed to reduce the similarity. Otherwise, no operation is performed on the images.

[0105] The image manipulation is roughly as follows. Assume the image to be modified is A, and another image, B, is also needed. Both images are the same size. On image B, using the center of image B as the origin, extract a ring-shaped portion of the image and overlay it onto the same position on image A. The number of rings is a random number between 1 and 3. The small and large radii of the rings are randomly selected (not exceeding the image size). If the number of overlays is greater than 1, efforts will be made to ensure that the covered areas do not overlap. For a batch of images, the modification correspondence is a one-to-one mapping between the images in the original order of the batch and the images after random shuffling.

[0106] In order to alleviate the problem of huge perspective differences between samples of the same type, this invention designs a feature extraction backbone network based on BiFormer and ResNet101. By feeding the image into BiFormer and ResNet101 respectively, global and local features are extracted. By combining global and local features, the semantic gap between different perspectives is eliminated, thereby achieving better results.

[0107] Step S4: Construct a loss function. The concatenated vector set is input into the network layer, and the network layer outputs the target recognition result from the cross-view satellite. The BiFormer network model, ResNet101 network model, and the network layer are trained based on the loss function, wherein the loss function is:

[0108] Loss=0.3*CE+0.3*DF+0.4*Adasp

[0109] in:

[0110] DF = DF(p) t )=-(1+a)(1-p t ) γ+b log(p t )+β

[0111] DF is the function for calculating DF loss, p t DF(p) is the probability of the predicted class of the training sample. t ) represents the DF loss function value, where a = weight1, b = weight1 * weight2, and β = (distance_small * α). 2 γ is an adjustable constant parameter, b is a modulation factor, distance_small is the reduced value of the distance from the training sample to its class center, weight1 is the standard deviation of the distance from the training sample to its class center, weight2 is the normalized value of distance_small; CE is the cross-entropy loss function, and Adasp is the loss caused by the concatenation vector compared with negative and positive samples.

[0112] The traditional Focal loss function has fixed adjustable parameters, which are determined before training. However, different classes in the dataset have varying degrees of imbalance, so the traditional Focal loss function cannot handle this problem.

[0113] This invention uses three loss functions: cross-entropy loss function, ASP loss function, and a self-designed loss function, DF loss function.

[0114] The Focal loss function is an improvement upon the DF loss function. The formula for the Focal loss function is as follows:

[0115] FL(p t )=-(1-p t ) γ log(p t )

[0116] The DF loss function dynamically improves upon the focal loss by using cluster centers to calculate dynamic parameters. First, a center needs to be initialized for all satellite categories. After each batch, the distance between the feature vectors extracted by the feature extraction network and the category center is calculated and denoted as distance. The specific steps are as follows:

[0117] Obtain the class center of each sample from the class center tensor.

[0118] Calculate the distance from each sample to its class center.

[0119] The distance is numerically reduced to obtain distance_small.

[0120] Calculate the weighted values ​​of distance, weight1 and weight2. Here, weight1 is the standard deviation of distance, and weight2 is the normalized distance_small value.

[0121] Calculate weighted loss

[0122] Update the category center based on the feature vector.

[0123] The improved form is roughly as follows:

[0124] DF(p t )=-(1+a)(1-p t ) γ+b log(p t )+β

[0125] Among them, a=weight1, b=weight1*weight2, β=(distance_small*α) 2

[0126] The overall loss function is

[0127] Loss=0.3*CE+0.3*DF+0.4*Adasp

[0128] In this invention, the data to be identified is compared with the target in the existing image library to measure feature similarity and obtain matching results. The basis for judging feature similarity is the size of the Euclidean distance. The smaller the distance, the higher the similarity and the earlier it appears in the matching results.

[0129] The technical effects of the present invention are shown in Table 4.

[0130] Table 4 Algorithm Performance Table

[0131]

[0132] As can be seen from the results above, this invention achieves excellent results. In terms of technical indicators, each indicator reached its highest value, with mAP reaching 79.0% and Rank-1 reaching 99.1%. This is mainly due to the fact that the information extracted by the ResNet101+BiFormer network includes both global and local features. Furthermore, data augmentation methods such as Annulusmix were used to increase the amount of data, and the Focal loss was improved to dynamically calculate the loss function for each category, thereby enabling the model to have a stronger ability to capture and recognize the features of spatial targets.

[0133] This invention provides a target recognition device based on a dual-branch network, the device comprising:

[0134] Dataset generation module: configured to acquire satellite data, model and simulate satellites, output simulated satellite images, construct cross-view satellite dataset based on the simulated images, and determine training sample set and test set from the cross-view satellite dataset;

[0135] Image enhancement module: configured to build a controller, the controller is used to obtain the gray values ​​of each training sample in the training sample set, determine the similarity between each training sample, perform image enhancement on each training sample, and the similarity between the enhanced training samples is less than a preset threshold.

[0136] Feature extraction module: Configured to input the image-enhanced training sample set into the BiFormer network model and the ResNet101 network model respectively, to obtain the first feature vector set and the second feature vector set respectively; concatenate the corresponding vectors in the first feature vector set and the second feature vector set to obtain the concatenated vector set;

[0137] Training module: configured to construct a loss function, the concatenated vector set is input into the network layer, and the network layer outputs the target recognition results of cross-view satellites; the BiFormer network model, ResNet101 network model and the network layer are trained based on the loss function.

[0138] The present invention further provides an electronic device, comprising:

[0139] A processor is used to execute multiple instructions;

[0140] Memory, used to store multiple instructions;

[0141] The plurality of instructions are to be stored in the memory and loaded and executed by the processor as described above.

[0142] The present invention further provides a computer-readable storage medium storing a plurality of instructions; the plurality of instructions are used by a processor to load and execute the method as described above.

[0143] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0144] In the embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or units through some interfaces, and may be electrical, mechanical, or other forms.

[0145] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0146] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0147] The integrated unit, implemented as a software functional unit, can be stored in a computer-readable storage medium. This software functional unit, stored in a storage medium, includes several instructions to cause a computer device (which may be a personal computer, a physical server, or a cloud server, etc., requiring the Ubuntu operating system to be installed) to execute some steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0148] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention shall still fall within the scope of the technical solution of the present invention.

Claims

1. A target recognition method based on a dual-branch network, characterized in that, The method includes the following steps: Step S1: Acquire satellite data, model and simulate the satellite, output simulated satellite images, construct a cross-view satellite dataset based on the simulated images, and determine the training sample set and test set from the cross-view satellite dataset; Step S2: Construct a controller, which is used to obtain the gray values ​​of each training sample in the training sample set, determine the similarity between each training sample, perform image enhancement on each training sample, and the similarity between the enhanced training samples is less than a preset threshold. Step S3: Input the image-enhanced training sample set into the BiFormer network model and the ResNet101 network model respectively to obtain the first feature vector set and the second feature vector set respectively; concatenate the corresponding vectors in the first feature vector set and the second feature vector set to obtain the concatenated vector set; Step S4: Construct a loss function. The concatenated vector set is input into the network layer, and the network layer outputs the target recognition result of the satellite from a different perspective. The BiFormer network model, the ResNet101 network model, and the network layer are trained based on the loss function.

2. The method as described in claim 1, characterized in that, Step S1: Acquire satellite data, perform satellite modeling and simulation, output simulated satellite images, construct a cross-view satellite dataset based on the simulated images, and determine a training sample set and a test set from the cross-view satellite dataset, including: Step S11: Acquire several 3D satellite images. Blender simulation software performs modeling and simulation of the satellite based on the 3D satellite data. The modeling and simulation includes: Blender simulation software determining simulation parameters, including the number of viewing angles, camera distance, illumination parameters, and the size of the simulation image; for each 3D satellite image, Blender simulation software adjusts the viewing angle of the 3D satellite image based on the simulation parameters, generates a grayscale image of the 3D satellite image after adjusting the viewing angle, adjusts the size of the grayscale image, and uses the grayscale image after adjusting the size as the first simulation image of the satellite. Step S12: Add Gaussian noise to the first simulated image, perform blurring, and adjust the brightness and contrast to obtain the simulated image; Step S13: All simulated images constitute a cross-view satellite dataset, and the training sample set and test set are determined from the cross-view satellite dataset.

3. The method as described in claim 2, characterized in that, Step S2: Constructing a controller, the controller is used to acquire the grayscale values ​​of each training sample in the training sample set, determine the similarity between each training sample, perform image enhancement on each training sample, and the similarity between the enhanced training samples is less than a preset threshold, including: The training sample set is divided into multiple batches, and each batch includes several training samples. For each batch, the controller performs the following operations: Step S21: The controller acquires the grayscale values ​​of each training sample in the batch; arranges each training sample according to its storage order in the batch, and stores the number of each training sample into the first array; sets the counter num = 2; sets the loop threshold to num1; Step S22: If num is less than or equal to num1+1, proceed to step S23; otherwise, proceed to step S24. Step S23: Randomly shuffle the storage order of each training sample in this batch, arrange each training sample in the new order, and store the number of each training sample into an array with the number num. The two indices stored at the same array positions in the first array and the array numbered num are denoted as x and y, respectively; Calculate the similarity of training sample pairs consisting of two training samples numbered x and y: Where, μ x Let μ be the average gray value of the training sample numbered x. y Let y be the average gray value of the training sample with ID y, where C1 and C2 are constants, and σ is the mean gray value of the training sample with ID y. xy Let σ be the covariance between the gray values ​​of training sample x and training sample y. x Let σ be the standard deviation of the gray level of the training sample numbered x. y y represents the standard deviation of the gray levels of the training sample with ID y; For all training sample pairs between the first array and the array numbered num, calculate the average similarity and store the average in a similarity array, where the similarity data is an array storing the average. The counter num is assigned the value num+1, and the process proceeds to step S22; Step S24: Obtain the last stored similarity in the similarity array, remove the last stored similarity from the similarity array, and then calculate the average value avg of each element in the similarity array. If the last stored similarity is greater than avg, proceed to step S25; otherwise, the batch processing is complete. Step S25: For each element in the first array corresponding to the training sample, select several regions from the training sample, and replace each region with the corresponding region of the replacement image; the replacement image is determined by the array numbered num, and is a training sample pair with the training sample; the batch processing is completed.

4. The method as described in claim 3, characterized in that, Step S4: Construct a loss function. The concatenated vector set is input into the network layer, and the network layer outputs the target recognition result from the cross-view satellite. The BiFormer network model, ResNet101 network model, and the network layer are trained based on the loss function, wherein the loss function is: Loss=0.3*CE+0.3*DF+0.4*Adasp in: DF=DF(p t )=-(1+a)(1-p t ) γ+b log(p t )+β DF is the calculation function, p t DF(p) is the probability of the predicted class of the training sample. t ) represents the DF loss function value, where a = weight1, b = weight1 * weight2, and β = (distance_small * α). 2 γ is an adjustable constant parameter, b is a modulation factor, distance_small is the reduced value of the distance from the training sample to its class center, weight1 is the standard deviation of the distance from the training sample to its class center, weight2 is the normalized value of distance_small; CE is the cross-entropy loss function, and Adasp is the loss caused by the concatenation vector compared with negative and positive samples.

5. The method according to any one of claims 1-4, characterized in that, The method further includes step S5: acquiring the cross-view satellite image to be identified, inputting it into the trained BiFormer network model and Resnet101 network model respectively, and outputting the target recognition result of the cross-view satellite image to be identified by the trained network layers.

6. A target recognition device based on a dual-branch network, characterized in that, The device includes: Dataset generation module: configured to acquire satellite data, model and simulate satellites, output simulated satellite images, construct cross-view satellite dataset based on the simulated images, and determine training sample set and test set from the cross-view satellite dataset; Image enhancement module: configured to build a controller, the controller is used to obtain the gray values ​​of each training sample in the training sample set, determine the similarity between each training sample, perform image enhancement on each training sample, and the similarity between the enhanced training samples is less than a preset threshold. Feature extraction module: Configured to input the image-enhanced training sample set into the BiFormer network model and the ResNet101 network model respectively, to obtain the first feature vector set and the second feature vector set respectively; concatenate the corresponding vectors in the first feature vector set and the second feature vector set to obtain the concatenated vector set; Training module: configured to construct a loss function, the concatenated vector set is input into the network layer, and the network layer outputs the target recognition results of cross-view satellites; the BiFormer network model, ResNet101 network model and the network layer are trained based on the loss function.

7. An electronic device, comprising: A processor is used to execute multiple instructions; Memory, used to store multiple instructions; The plurality of instructions are to be stored in the memory and loaded by the processor and executed as described in any one of claims 1-5.

8. A computer-readable storage medium storing a plurality of instructions; the plurality of instructions being loaded by a processor and executing the method as claimed in any one of claims 1-5.

Images