Multispectral image feature extraction method, device, electronic device and storage medium

Abstract

The present application provides a method, device, electronic device and computer-readable storage medium for extracting features from a multispectral image. The method comprises: obtaining a multispectral image to be processed, and synthesizing the multispectral image by bands to obtain at least two synthesized images, wherein the number of bands included in the synthesized image is less than the number of bands included in the multispectral image; extracting features from each synthesized image to obtain a first feature map corresponding to each synthesized image; recombining features in each first feature map to obtain a second feature map corresponding to each first feature map; and splicing each second feature map to obtain a spliced ​​feature map corresponding to the multispectral image. Through the present application, the feature extraction capability of multispectral images can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and relates to but is not limited to a method, device, electronic device, computer-readable storage medium, and computer program product for extracting features from a multispectral image. Background Art

[0002] With the rapid development of earth observation technologies such as remote sensing sensors, the data types of multispectral images are becoming more and more abundant, and the spatial resolution of images is also getting higher and higher. High-resolution images have rich and detailed information about ground objects, and the details of the ground objects are also clearer, which provides a good research basis for the extraction of ground object information in multispectral images. However, it also leads to the complex structure of the ground objects and the difficulty in processing interference information.

[0003] In the related technologies, for complex object recognition, due to the limitations of complex background, image quality, image resolution, seasonal and weather factors, object scale differences, sample imbalance, etc., when extracting features from multispectral images, the current main approach is to directly use multi-band multispectral images as the input of the model for feature extraction, or only use RGB (R stands for red, G stands for green, and B stands for blue) three-band true color images as the input of the model for feature extraction. This fails to fully utilize the information of different bands in the multispectral image, resulting in the extracted feature map either having too much redundant information or losing too many features, affecting the classification, segmentation, target recognition and other results of the multispectral image. Summary of the invention

[0004] The embodiments of the present application provide a method, device, electronic device, computer-readable storage medium and computer program product for extracting features of a multispectral image, which can extract data features in a multispectral image more comprehensively and accurately.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] The present invention provides a method for extracting features from a multispectral image, the method comprising:

[0007] Acquire a multispectral image to be processed, and perform band synthesis on the multispectral image to obtain at least two synthesized images, wherein the number of bands included in the synthesized images is less than the number of bands included in the multispectral image;

[0008] Performing feature extraction on each of the synthesized images respectively to obtain a first feature map corresponding to each of the synthesized images;

[0009] Recombining the features in each of the first feature maps respectively to obtain a second feature map corresponding to each of the first feature maps;

[0010] The second feature maps are spliced ​​to obtain a spliced ​​feature map corresponding to the multispectral image.

[0011] The present application provides a multi-spectral image feature extraction device, the device comprising:

[0012] An acquisition module, used for acquiring a multispectral image to be processed;

[0013] A band synthesis module, used for performing band synthesis on the multispectral image to obtain at least two synthesized images, wherein the number of bands included in the synthesized images is less than the number of bands included in the multispectral image;

[0014] A preliminary feature extraction module, used to extract features from each of the synthetic images to obtain a first feature map corresponding to each of the synthetic images;

[0015] A feature recombination module, used for respectively recombination the features in each of the first feature maps to obtain a second feature map corresponding to each of the first feature maps;

[0016] The stitching module is used to stitch the second feature maps to obtain a stitching feature map corresponding to the multispectral image.

[0017] In some embodiments, the band synthesis module is further used to:

[0018] Extracting the multiple bands included in the multispectral image to obtain single-band images corresponding to the bands;

[0019] Band synthesis is performed on the plurality of single-band images to obtain at least two synthesized images.

[0020] In some embodiments, the band synthesis module is further used to:

[0021] Obtaining the category to which the image content of the multispectral image belongs and the band parameters corresponding to each band;

[0022] Based on the category and the band parameter, selecting a single-band image from a plurality of the single-band images to form at least two single-band image groups, each of the single-band image groups including at least two single-band images;

[0023] Band synthesis is performed on each of the single-band image groups to obtain a corresponding synthesized image.

[0024] In some embodiments, the feature recombination module is further used to:

[0025] Perform band shuffling on the features in the at least two first feature maps obtained to obtain corresponding band shuffled feature maps;

[0026] The band shuffled feature maps are segmented to obtain second feature maps corresponding to each of the first feature maps.

[0027] In some embodiments, the feature recombination module is further used to:

[0028] Performing dimension expansion on each of the first feature maps respectively to obtain dimension-expanded first feature maps;

[0029] Perform feature superposition on the first feature map after expansion of each dimension on the expanded dimension to obtain a feature map after feature superposition;

[0030] The feature map after the feature superposition is reconstructed to obtain the band shuffled feature map.

[0031] In some embodiments, the multispectral image feature extraction device further includes:

[0032] A pooling module, used for performing pooling processing on each of the first feature maps before reorganizing the features in each of the first feature maps to obtain each pooled first feature map;

[0033] The feature recombination module is further used for:

[0034] The features in each of the pooled first feature maps are reorganized to obtain second feature maps corresponding to each of the first feature maps.

[0035] In some embodiments, the spliced ​​feature map is obtained by extracting features from the multispectral image through a feature extraction layer of an image classification model, the image classification model further includes a classification prediction layer, and the feature extraction device for the multispectral image further includes:

[0036] An input module, used for inputting the spliced ​​feature map into the classification prediction layer for image classification, to obtain an image classification result corresponding to the multispectral image;

[0037] The determination module is used to determine the image classification result as a target classification result when the image classification result meets a preset classification condition.

[0038] In some embodiments, the preliminary feature extraction module is further used to extract features from each second feature map respectively to obtain a third feature map corresponding to each second feature map when the image classification result does not meet the preset classification condition;

[0039] The feature recombination module is further used to respectively recombine the features in each of the third feature maps to obtain a fourth feature map corresponding to each of the third feature maps;

[0040] The splicing module is further used to splice the fourth feature maps to obtain an updated splicing feature map;

[0041] The input module is also used to input the updated splicing feature map into the classification prediction layer for image classification to obtain an image classification result corresponding to the multispectral image.

[0042] An embodiment of the present application provides an electronic device, including:

[0043] A memory for storing executable instructions;

[0044] The processor is used to implement the feature extraction method of the multispectral image provided in the embodiment of the present application when executing the executable instructions stored in the memory.

[0045] The embodiment of the present application provides a computer-readable storage medium storing executable instructions for causing a processor to execute and implement the feature extraction method of the multispectral image provided by the embodiment of the present application.

[0046] An embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the feature extraction method of the multispectral image provided by the embodiment of the present application.

[0047] The embodiments of the present application have the following beneficial effects:

[0048] Compared with the solution in the related art that only uses red, green and blue three-band true color images to extract multispectral image features, the embodiment of the present application obtains a multispectral image to be processed and performs band synthesis on the multispectral image to obtain at least two synthetic images; performs feature extraction on each synthetic image to obtain a first feature map corresponding to each synthetic image; then performs feature reorganization on the features in each first feature map to obtain a second feature map corresponding to each first feature map; and splices the second feature maps to obtain a spliced ​​feature map corresponding to the multispectral image, which can reduce the precision loss of feature extraction and make the features in the extracted feature maps more comprehensive and accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This is a schematic diagram of a network architecture of a multispectral image feature extraction system provided in an embodiment of the present application;

[0050] Figure 2 This is a schematic diagram of a composition structure of an electronic device provided in an embodiment of the present application;

[0051] Figure 3 This is a schematic diagram of an implementation flow of a feature extraction method for a multispectral image provided in an embodiment of the present application;

[0052] Figure 4This is a schematic diagram of an implementation flow of a band synthesis method for a multispectral image provided in an embodiment of the present application;

[0053] Figure 5 This is a schematic diagram of an implementation flow of a feature recombination method in a multispectral image provided in an embodiment of the present application;

[0054] Figure 6 This is another implementation flow diagram of the feature extraction method for multispectral images provided in an embodiment of the present application;

[0055] Figure 7 This is another implementation flow diagram of the multispectral image feature extraction method provided in the embodiment of the present application;

[0056] Figure 8 It is a schematic diagram of a composition structure of an image classification model where a feature extraction layer provided in an embodiment of the present application is located;

[0057] Fig. 9 It is another schematic diagram of the composition structure of the image classification model where the feature extraction layer provided in the embodiment of the present application is located;

[0058] Fig.10 This is another schematic diagram of the composition structure of the image classification model where the feature extraction layer provided in the embodiment of the present application is located;

[0059] Fig.11 This is another implementation flow diagram of the feature extraction method for multispectral images provided in the embodiment of the present application;

[0060] Fig.12 It is a schematic diagram of a feature extraction framework for a multispectral image provided in an embodiment of the present application;

[0061] Fig.13 It is a schematic diagram of an implementation flow of a channel shuffling method in a multispectral image provided in an embodiment of the present application. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0063] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0064] In the following description, the terms "first\second\third" involved are only used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0065] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0066] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0067] 1) Feature extraction refers to the method and process of using computers to extract characteristic information from images.

[0068] 2) Band, also known as spectral band or spectral band, is a continuous electromagnetic wave with a certain wavelength range. Bands are usually expressed as numerical values ​​of specific wavelength ranges, such as the fourth band of the Landsat multi-band scanner is 0.5 to 0.6 microns.

[0069] 3) Band shuffling, also known as channel shuffling or feature shuffling, rearranges the multiple channels of different bands in the multispectral image in a random order, disrupts the feature order of the original feature map, and fully integrates the channels.

[0070] Based on the above explanations of the nouns and terms involved in the embodiments of the present application, the feature extraction system for multispectral images provided in the embodiments of the present application is first described. Figure 1 , Figure 1 This is a network architecture diagram of a multispectral image feature extraction system provided in an embodiment of the present application. In the multispectral image feature system 100, a terminal 400 is connected to a server 200 via a network 300. The network 300 may be a wide area network or a local area network, or a combination of the two, and a wireless link is used to realize data transmission.

[0071] In some embodiments, the terminal 400 can be a laptop, a tablet computer, a desktop computer, a smart phone, a dedicated messaging device, a portable gaming device, a smart speaker, a smart watch, etc., but is not limited thereto. The server 200 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDN, Content Delivery Network), and big data and artificial intelligence platforms. The network 300 can be a wide area network or a local area network, or a combination of the two. The terminal 400 and the server 200 can be directly or indirectly connected by wired or wireless communication, which is not limited in the embodiments of the present application.

[0072] The terminal 400 is used to send a feature extraction request carrying a multispectral image to be processed to the server 200 to request the server 200 to perform feature extraction on the multispectral image.

[0073] The server 200 is used to parse the multispectral image to be processed from the feature extraction request, and perform band synthesis on the multispectral image to obtain at least two synthetic images, each of which includes less bands than the multispectral image; perform feature extraction on each synthetic image to obtain a first feature map corresponding to each synthetic image; perform feature recombination on the features in each first feature map to obtain a second feature map corresponding to each first feature map; perform splicing on each second feature map to obtain a spliced ​​feature map corresponding to the multispectral image, and return the spliced ​​feature map to the terminal 400.

[0074] The terminal 400 is also used to output a splicing feature map corresponding to the multispectral image to be processed.

[0075] In some embodiments, an image feature extraction client 410 is provided on the terminal 400. The user performs feature extraction on the multispectral image based on the image feature extraction client 410, and triggers a feature extraction instruction based on the selected multispectral image. The image feature extraction client 410 responds to the feature extraction instruction and sends a feature extraction request carrying the multispectral image to be processed to the server 200. After the server 200 parses the multispectral image to be processed from the feature extraction request, it performs band synthesis on the multispectral image to obtain at least two synthetic images, and performs feature extraction on each synthetic image to obtain a first feature map corresponding to each synthetic image, and then performs feature recombination on the features in each first feature map to obtain a second feature map corresponding to each first feature map, and then splices each second feature map to obtain a spliced ​​feature map corresponding to the multispectral image, and finally returns the spliced ​​feature map to the image feature extraction client 410, and the image feature extraction client 410 presents the spliced ​​feature map corresponding to the multispectral image to be processed.

[0076] See also Figure 2 , Figure 2 is a schematic diagram of a composition structure of an electronic device provided in an embodiment of the present application. In practical applications, the electronic device 500 can be implemented as Figure 1 The terminal 400 or the server 200 in the embodiment of the present invention is an electronic device. Figure 1 Taking the server 200 as an example, an electronic device for implementing the feature extraction method of multispectral images according to an embodiment of the present application is described. Figure 2 The electronic device 500 shown includes: at least one processor 510, a memory 550, at least one network interface 520 and a user interface 530. The various components in the electronic device 500 are coupled together via a bus system 540. It can be understood that the bus system 540 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 540 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 540 is not described in detail. Figure 2 Various buses are labeled as bus system 540 .

[0077] The processor 510 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0078] The user interface 530 includes one or more output devices 531 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 530 also includes one or more input devices 532, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0079] The memory 550 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 550 may optionally include one or more storage devices that are physically remote from the processor 510.

[0080] The memory 550 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 550 described in the embodiments of the present application is intended to include any suitable type of memory.

[0081] In some embodiments, the memory 550 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.

[0082] Operating system 551, including system programs for processing various basic system services and performing hardware-related tasks, such as framework layer, core library layer, driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0083] A network communication module 552, for reaching other computing devices via one or more (wired or wireless) network interfaces 520, exemplary network interfaces 520 include: Bluetooth, Wireless Compatibility Certification (WiFi), and Universal Serial Bus (USB);

[0084] a presentation module 553 for enabling presentation of information via one or more output devices 531 (e.g., display screen, speaker, etc.) associated with the user interface 530 (e.g., a user interface for operating peripherals and displaying content and information);

[0085] The input processing module 554 is used to detect one or more user inputs or interactions from one of the one or more input devices 532 and translate the detected inputs or interactions.

[0086] In some embodiments, the multispectral image feature extraction device provided in the embodiments of the present application can be implemented in a software manner. Figure 2 The feature extraction device 555 of the multispectral image stored in the memory 550 is shown, which can be software in the form of a program and a plug-in, etc., including the following software modules: an acquisition module 5551, a band synthesis module 5552, a preliminary feature extraction module 5553, a feature recombination module 5554 and a splicing module 5555. These modules are logical, so they can be arbitrarily combined or further split according to the functions implemented. The functions of each module will be explained below.

[0087] In other embodiments, the feature extraction device for multispectral images provided in the embodiments of the present application can be implemented in hardware. As an example, the feature extraction device for multispectral images provided in the embodiments of the present application can be a processor in the form of a hardware decoding processor, which is programmed to execute the feature extraction method for multispectral images provided in the embodiments of the present application. For example, the processor in the form of a hardware decoding processor can adopt one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs) or other electronic components.

[0088] The following describes the feature extraction method of the multispectral image provided by the embodiment of the present application. In some embodiments, the feature extraction method of the multispectral image provided by the embodiment of the present application can be implemented by the terminal or the server alone, or by the terminal and the server in collaboration. The following takes the server implementation as an example. Figure 3 , Figure 3 This is a schematic diagram of an implementation flow of the feature extraction method for multispectral images provided in the embodiment of the present application. Figure 3 The steps shown are explained.

[0089] Step S101, obtaining a multispectral image to be processed.

[0090] Here, the multispectral image to be processed contains at least three different bands. The band parameters contained in each band are different. Taking the multispectral image provided by the Landsat 8 satellite as an example, the bands included include the coastline band (Coastal, 1), blue band (Blue, 2), green band (Green, 3), red band (Red, 4), near infrared band (Nir, 5), short infrared band 1 (Swir1, 6), short infrared band 2 (Swir2, 7), panchromatic band (Pan, 8), cirrus band (Cirrus, 9), thermal infrared 1 (Tirs1, 10), thermal infrared 2 (Tirs2, 11).

[0091] In practical applications, if it is necessary to extract features from a multispectral image and obtain a feature map of the multispectral image, the multispectral image to be processed is sent to a server through a terminal.

[0092] Step S102: perform band synthesis on the multispectral image to obtain at least two synthesized images.

[0093] In practical applications, if you want to highlight the object category that the user is interested in in the multispectral image, you need to perform band synthesis on different bands to obtain different synthetic images, and the number of bands included in each synthetic image is less than the number of bands included in the multispectral image. Then, based on the synthetic image, the object category that the user is interested in is highlighted. Since the image processing system generally uses the three-color synthesis principle to form a color image, that is, 3 band images are placed on 3 channels, therefore, when performing band synthesis, 3 bands can be selected for synthesis to obtain a synthetic image.

[0094] For example, when the multispectral image is a multispectral remote sensing image provided by the Landsat 8 satellite, the object categories in the image include vegetation, waters, buildings, wetlands, grasslands, etc., and each single band in the image is extracted. After analyzing the amount of information contained in each single band and the correlation between bands, it is found that the near infrared-red-green (Nir-RB) band synthesis is the best band synthesis for vegetation, crops and wetland-related monitoring. Therefore, when it is necessary to highlight vegetation, the corresponding synthetic image can be composed of the three Nir-RB bands.

[0095] The method of band synthesis is described. In some embodiments, it is necessary to extract the information of each band of the multispectral image, obtain the corresponding multiple single-band images, and select three single-bands to synthesize to obtain a synthetic image according to the enhancement effect of band synthesis on different object categories. It can be implemented as follows: band extraction is performed on the multiple bands included in the multispectral image to obtain the single-band image corresponding to each band; band synthesis is performed on the multiple single-band images to obtain at least two synthetic images.

[0096] Exemplarily, when the multispectral image corresponds to a multispectral remote sensing image, taking Landsat 8 band synthesis as an example, the band synthesis method is selected according to the effect of enhancing different ground objects. Extract multiple bands such as red (R, No. 4), green (G, No. 3), blue (B, No. 2), near red (Nir, No. 5), short infrared (Swir1, No. 6) of the multispectral image. For example, 432 band synthesis can obtain a true color synthesized image; 543 band synthesis (standard false color synthesis) is used for vegetation, crops and wetland-related monitoring; 564 band effectively distinguishes between land and water. Since the synthesis of the above different bands highlights different features of the ground objects, different bands can be synthesized separately according to the actual situation, so that more favorable features in the multispectral image can be extracted. For example, when performing image classification, feature maps that are beneficial to image classification are extracted; when performing image segmentation, feature maps that are beneficial to image segmentation are extracted.

[0097] Step S103 , performing feature extraction on each synthetic image to obtain a first feature map corresponding to each synthetic image.

[0098] Here, the server can extract features from each composite image through a feature extraction algorithm to obtain a corresponding feature map. Here, the feature extraction performed on each composite image is referred to as preliminary feature extraction, and the feature map obtained by the preliminary feature extraction is referred to as a first feature map for distinction.

[0099] The server can obtain a trained preliminary feature extraction model in advance based on the sample image and feature extraction algorithm training, which can be implemented as follows: obtaining a sample image, inputting the sample image into the initialized preliminary feature extraction model for training, iteratively training until the convergence condition is met, and obtaining a trained preliminary feature extraction model. Here, the preliminary feature extraction model can be formed by neural network layers such as convolution layers, batch normalization layers, and activation layers, and each synthetic image is subjected to convolution, normalization, activation, and other processing to obtain a first feature map corresponding to each synthetic image.

[0100] Step S104, performing feature reorganization on the features in each first feature map to obtain a second feature map corresponding to each first feature map.

[0101] In the related art, after obtaining the first feature map, if the features extracted from the first feature map do not meet the image processing (such as classification) requirements, the above-mentioned preliminary feature extraction model continues to be used to extract features from the first feature map. Different channels are independent of each other. Such multiple convolutions will greatly reduce the accuracy of the extracted feature map, resulting in accuracy loss.

[0102] In order to solve the problem of precision loss, in an embodiment of the present application, the first feature map is processed before the first feature map is re-extracted. One implementation method for processing the first feature map can be: splicing each first feature map to connect the features between different feature maps, and performing group convolution on the spliced ​​first feature map to extract features, thereby reducing computational losses and improving precision. However, if multiple group convolutions are stacked together, between different first feature maps, except for edge features, most of the features are still convolved within each first feature map, which can only reduce part of the precision loss.

[0103] In the embodiment of the present application, another implementation method for processing the first feature map is given: the features of each first feature map are reorganized, that is, the features in each first feature map are randomized to obtain the second feature map of each first feature map object, and the features in each second feature map have a one-to-one correspondence with a feature in each first feature map, but the order and arrangement of the features in each second feature map are different from the order and arrangement of the features in the first feature map. In this way, when the feature extraction is performed later, the features of each convolution are random, which can greatly reduce the precision loss, making the features in the extracted feature map more comprehensive and accurate.

[0104] Step S105, stitching the second feature maps to obtain a stitched feature map corresponding to the multispectral image.

[0105] Here, the features of each second feature map are spliced, and the obtained spliced ​​feature map is the feature map obtained by extracting features from the multispectral image.

[0106] The feature extraction method of a multispectral image provided in an embodiment of the present application includes: obtaining a multispectral image to be processed, and performing band synthesis on the multispectral image to obtain at least two synthetic images, wherein the number of bands included in the synthetic image is less than the number of bands included in the multispectral image; performing feature extraction on each synthetic image to obtain a first feature map corresponding to each synthetic image; performing feature recombination on the features in each first feature map to obtain a second feature map corresponding to each first feature map; and splicing each second feature map to obtain a spliced ​​feature map corresponding to the multispectral image. By synthesizing a multispectral image into multiple synthetic images according to the bands, and performing feature recombination on the features in each first feature map obtained by feature extraction of multiple synthetic images, feature randomization is achieved. In this way, the precision loss during feature extraction can be greatly reduced, so that the features extracted in the spliced ​​feature map are more comprehensive and accurate.

[0107] In some embodiments, since the band parameters contained in each single band, such as the amount of band information, the band correlation coefficient, etc., are different, the bands can be synthesized by using the band parameters corresponding to each band and the category to which the image content of the multispectral image belongs to obtain a synthetic image.

[0108] See also Figure 4 , Figure 4 This is a schematic diagram of an implementation flow of the band synthesis method in the multispectral image provided by an embodiment of the present application. The band synthesis method for multiple single-band images can be implemented through steps S1021 to S1023, which will be described in combination with each step:

[0109] Step S1021, obtaining the category to which the image content of the multispectral image belongs and the band parameters corresponding to each band.

[0110] Here, the server obtains the image content contained in the multispectral image and determines the category of the image content. When the multispectral image is a multispectral image of the corresponding ground object, the category to which the image content belongs can be understood as different categories such as "arable land", "forest land", "grassland", "road", "building" and "water area".

[0111] Here, the band parameters include band information and band correlation coefficients. The band correlation coefficient is used to characterize the degree of overlap of band information between bands.

[0112] Step S1022 : selecting single-band images from a plurality of single-band images to form at least two single-band image groups based on the category and the band parameters.

[0113] Based on the band correlation coefficient, it can be determined which bands have low correlation, and the bands with low correlation are combined, which has obvious differences and is more conducive to highlighting the category information. According to the features to be extracted, multiple single-band images are selected to obtain at least two single-band image groups.

[0114] Step S1023 , performing band synthesis on each single-band image group to obtain a corresponding synthesized image.

[0115] Here, according to actual applications, when synthesizing different single-band images, in order to obtain a color picture, three single-band images are generally selected for synthesis to obtain a composite image.

[0116] Through steps S1021 to S1023, purposeful band synthesis is performed according to the band parameters of each band and the category (target category) corresponding to the image content to obtain multiple synthetic images, and then the display effects of different objects in the multispectral image are enhanced through different synthetic images to facilitate feature extraction.

[0117] Figure 5 This is a schematic diagram of an implementation flow of a feature recombination method in a multispectral image provided in an embodiment of the present application. Figure 3 In the example shown, step S104 of "respectively performing feature recombination on the features in each first feature map to obtain a second feature map corresponding to each first feature map" can be performed by Figure 5The implementation of step S1041 and step S1042 shown will be described in conjunction with each step.

[0118] Step S1041, performing band shuffling on the features in the obtained at least two first feature maps to obtain corresponding band shuffled feature maps.

[0119] Since the synthetic image is synthesized according to different bands, the obtained first feature maps are also different bands, and the features between different first feature maps are not communicated, which reduces the feature extraction capability. In an embodiment of the present application, the features in different first feature maps are reorganized by shuffling the bands. In one implementation method, a shuffling network can be used to shuffle the features of each first feature map using a shuffling module. In practice, it can be implemented as follows: dimensionally expand each first feature map to obtain a first feature map after dimension expansion; superimpose the features of each first feature map after dimension expansion on the expanded dimension to obtain a feature map after feature superposition; reconstruct the feature map after feature superposition to obtain a band shuffled feature map.

[0120] Assume that the features in the p first feature maps are shuffled by bands, and the sizes of the p first feature maps are all c*h*w, where c is the number of bands (i.e., the number of channels in the first feature map, which can also be called the number of features). The p first feature maps are output with expanded dimensions for dimensional expansion to obtain the first feature maps after dimensional expansion, and the size of each expanded first feature map becomes 1*c*h*w; the p first feature maps are superimposed on the first dimension to obtain a feature map after feature superposition, and the size of the feature map after feature superposition is p*c*h*w; then, the feature map after feature superposition is reconstructed. In the embodiment of the present application, feature reconstruction can be achieved by transposing the first dimension and the second dimension of the feature map after feature superposition, and the size of the band shuffled feature map obtained after reconstruction is c*p*h*w.

[0121] It should be noted that, in the method of the embodiment of the present application, the user can input only one multispectral image at a time, in which case the server only performs feature extraction on the multispectral image, and when performing band shuffling, band shuffling is performed on the features of the p first feature graphs of the multispectral image. In other embodiments, the user can input multiple multispectral images at a time, in which case the server can perform feature extraction on the multiple multispectral images in sequence, and when performing band shuffling, after the band shuffling of the features of the p first feature graphs of the previous multispectral image is completed, band shuffling of the features of the p first feature graphs of the next multispectral image is started; the server can also perform feature extraction on the multiple multispectral images at the same time to achieve batch band shuffling of multiple multispectral images.

[0122] Step S1042, performing feature map segmentation on the band shuffled feature map to obtain second feature maps corresponding to each first feature map.

[0123] When the band shuffle feature graph is segmented, the band shuffle feature graph can be segmented according to the number of features included in each first feature graph to obtain the second feature graph corresponding to each first feature graph. Exemplarily, the first feature graph includes 3 channels, and the band shuffle feature graph is segmented every 3 features.

[0124] Through step S1041 and step S1042, the features in each first feature map are shuffled by bands, so that different first feature maps can communicate with each other, thereby improving the ability of feature extraction. The band shuffled feature map obtained by band shuffling is subjected to feature map segmentation to obtain the second feature map corresponding to each first feature map, so that the number of feature maps and the number of features of each feature map remain unchanged, which is convenient for continuing feature extraction.

[0125] See also Figure 6 , Figure 6 is another implementation flow diagram of the feature extraction method for multispectral images provided in the embodiment of the present application, which is combined with Figure 6 The steps shown are explained.

[0126] Step S201, obtaining a multispectral image to be processed.

[0127] Steps S201 to S203 in the embodiment of the present application are respectively Figure 3 Steps S101 to S103 in the embodiment shown correspond to each other. The implementation methods and effects of steps S201 to S203 can be found in Figure 3 The corresponding description in the illustrated embodiment.

[0128] Step S202: perform band synthesis on the multispectral image to obtain at least two synthesized images.

[0129] Step S203: extract features from each composite image to obtain a first feature map corresponding to each composite image.

[0130] Step S204: perform pooling processing on each first feature map to obtain each pooled first feature map.

[0131] In the embodiment of the present application, before the features in each first feature map are reorganized, each first feature map can be pooled to obtain each pooled first feature map. For example, the first feature map is represented by c*h*w, where c is the number of synthesized bands. For example, 3 bands are selected for synthesis, and the number of bands of the first feature map obtained by feature extraction of the synthesized image remains unchanged, which is still 3. Continue to perform pooling on the first feature map, and the number of bands of the first feature map obtained after pooling remains unchanged, that is, pooling will not reduce the number of bands. The first feature map after pooling is represented as c*h1*w1. The pooling process belongs to downsampling processing, and the obtained h1<h, w1<w. By performing pooling on each first feature map, redundant data can be reduced, the amount of calculated data can be reduced, and the speed of feature extraction can be improved.

[0132] When the first feature map is pooled here, the pooling function used may be "maximum pooling", "average pooling", "L2-norm pooling", etc. Exemplarily, "maximum pooling" may be used, that is, according to a preset pooling template (such as 2*2), the first feature image is divided into several rectangular areas, and the maximum value is output for each rectangular area, so as to obtain the first feature map after pooling.

[0133] Step S205, reorganize the features in each pooled first feature map to obtain a second feature map corresponding to each first feature map.

[0134] Step S206, stitching the second feature maps to obtain a stitched feature map corresponding to the multispectral image.

[0135] The implementation methods of step S205 and step S206 are respectively Figure 3 In the illustrated embodiment, step S104 is the same as step S105, except that the images for feature reorganization are different, and step S205 and step S206 process the first feature map after pooling dimensionality reduction, so the dimension of the second feature map obtained in step S205 is smaller than the dimension of the second feature map obtained in step S104, and the dimension of the spliced ​​feature map obtained in step S206 is smaller than the dimension of the spliced ​​feature map obtained in step S105, thereby reducing the amount of data. The feature extraction method provided in the embodiment of the present application can reduce the redundant data of the spliced ​​feature map, reduce the amount of calculated data, and improve the efficiency of feature extraction.

[0136] In some embodiments, according to Figure 3 The embodiment shown or Figure 6After the illustrated embodiment obtains the spliced ​​feature map, the spliced ​​feature map can be further processed according to user needs. For example, when the user needs to classify the multispectral image, the spliced ​​feature map is input into the classification prediction layer for image classification to obtain a target classification result of the multispectral image; when the user needs to segment the multispectral image, the spliced ​​feature map is input into the segmentation prediction layer for image segmentation to obtain a target segmentation result of the multispectral image; when the user needs to identify a target in the multispectral image, the spliced ​​feature map is input into the target recognition layer for image recognition to obtain a target recognition result of the multispectral image.

[0137] In the embodiment of the present application, multispectral image classification is used as an example. Figure 7 , Figure 7 This is another implementation flow diagram of the multispectral image feature extraction method provided in the embodiment of the present application, which is combined with Figure 7 The steps shown are explained.

[0138] Step S301, obtaining a splicing feature map.

[0139] Here, the concatenated feature map can be based on Figure 3 or Figure 6 The steps of the illustrated embodiment are obtained.

[0140] Step S302: input the spliced ​​feature map into the classification prediction layer for image classification to obtain an image classification result corresponding to the multispectral image.

[0141] Here, the spliced ​​feature map can be obtained by extracting features from the multispectral image through the feature extraction layer of the image classification model. The processing of the feature extraction layer can be obtained by Figure 3 Steps S103 to S105 in the embodiment shown in the figure can also be implemented by Figure 6 In the illustrated embodiment, steps S203 to S206 are implemented.

[0142] Describes the structure of the image classification model where the feature extraction layer is located. Figure 8 , Figure 8 It is a schematic diagram of the composition structure of the image classification model where the feature extraction layer provided in the embodiment of the present application is located. In some embodiments, the image classification model includes a feature extraction layer (number 3) and a classification prediction layer (number 4). After the multispectral image (number 1) is subjected to band synthesis (number 2), multiple synthetic images are obtained, and the multiple synthetic images are input into the feature extraction layer (number 3) of the image classification model. Feature extraction is performed through multiple sub-layers in the feature extraction layer to obtain a spliced ​​feature map of the multispectral image, and the spliced ​​feature map is input into the classification prediction layer (number 4); the spliced ​​feature map is subjected to image classification through the classification prediction layer to obtain an image classification result.

[0143] Step S303, determining whether the image classification result meets a preset classification condition.

[0144] like Figure 8 As shown, the image classification model also includes a decision layer (code 5), and the image classification result output by the classification prediction layer (code 4) is input into the decision layer (code 5). The decision layer makes a decision on whether the image classification result meets the preset classification condition. When the image classification result meets the preset classification condition, it is determined that the classification is completed, and the process goes to step S304 to obtain the target classification result (code 6) corresponding to the multispectral image; when the image classification result does not meet the preset classification condition, it is determined that the classification is not completed, and the process goes to step S305 to continue to perform feature extraction based on the feature extraction layer.

[0145] The preset classification conditions here can be pre-defined by the user, such as dividing the multispectral image into three categories according to "house", "river" and "tree". The preset classification conditions can also be adaptively determined by the server. The server recognizes the features of four different categories: "house", "river", "tree" and "road", which can be divided into four categories.

[0146] Step S304: determine the image classification result as the target classification result.

[0147] The target classification result is obtained, the feature extraction is determined to be completed, and the entire image processing process is ended.

[0148] Step S305 , performing feature extraction on each second feature map to obtain a third feature map corresponding to each second feature map.

[0149] Here, the implementation method of feature extraction for each second feature map is similar to Figure 3 In the illustrated embodiment, the implementation method of feature extraction for each synthetic image in step S103 is the same, refer to the corresponding implementation description in step S103.

[0150] Step S306: Update the splicing feature map according to each third feature map to obtain an updated splicing feature map.

[0151] In one implementation, based on each third feature map, the spliced ​​feature map is updated to obtain an updated spliced ​​feature map. This can be achieved by: recombining the features in each third feature map to obtain the fourth feature maps corresponding to each third feature map, and splicing the fourth feature maps to obtain an updated spliced ​​feature map.

[0152] The structure of the feature extraction layer in this implementation is described. In some embodiments, the feature extraction layer may include an extraction sublayer, a feature recombination sublayer, and a splicing sublayer. Fig. 9 , Fig. 9It is another schematic diagram of the composition structure of the image classification model where the feature extraction layer provided in the embodiment of the present application is located. In some embodiments, the feature extraction layer (numbered 3) may include an extraction sublayer (numbered 31), a feature recombination sublayer (numbered 32) and a splicing sublayer (numbered 33), wherein the extraction sublayer performs feature extraction on each second feature map to obtain a third feature map corresponding to each second feature map; the feature recombination sublayer performs feature recombination on each third feature map to obtain a fourth feature map corresponding to each third feature map; the splicing sublayer splices each fourth feature map to obtain a spliced ​​feature map corresponding to the multispectral image. Continue to input the spliced ​​feature map into the classification prediction layer (No. 4) for image classification to obtain image classification results; input the image classification results output by the classification prediction layer into the decision layer (No. 5), and make a decision on whether the image classification results meet the preset classification conditions through the decision layer. When the image classification results meet the preset classification conditions, it is determined that the classification is completed, and the target classification results corresponding to the multispectral image are obtained (No. 6); when the image classification results do not meet the preset classification conditions, it is determined that the classification is not completed. At this time, the feature recombination sublayer (No. 32) inputs each fourth feature image into the extraction sublayer (No. 31) to continue feature extraction, and repeat the above process until the target classification results are obtained.

[0153] In another implementation, based on each third feature map, the spliced ​​feature map is updated to obtain an updated spliced ​​feature map, which can be implemented as follows: pooling the third feature maps to obtain each pooled third feature map; recombining the features in each pooled third feature map to obtain a fourth feature map corresponding to each third feature map, and splicing the fourth feature maps to obtain an updated spliced ​​feature map.

[0154] The structure of the feature extraction layer in this implementation is described. In some embodiments, the feature extraction layer may include a pooling sublayer, an extraction sublayer, a feature recombination sublayer, and a splicing sublayer. Fig.10 , Fig.10This is another composition structure diagram of the image classification model where the feature extraction layer provided in the embodiment of the present application is located. In some embodiments, the feature extraction layer (numbered 3) may include an extraction sublayer (numbered 31), a pooling sublayer (numbered 30), a feature recombination sublayer (numbered 32) and a splicing sublayer (numbered 33), wherein the extraction sublayer performs feature extraction on each second feature map to obtain a third feature map corresponding to each second feature map; the pooling sublayer performs pooling processing on each third feature map to obtain each pooled third feature map; the feature recombination sublayer performs feature recombination on each pooled third feature map to obtain a fourth feature map corresponding to each pooled third feature map; the splicing sublayer splices each fourth feature map to obtain a spliced ​​feature map corresponding to the multispectral image. Continue to input the spliced ​​feature map into the classification prediction layer (No. 4) for image classification to obtain image classification results; input the image classification results output by the classification prediction layer into the decision layer (No. 5), and make a decision on whether the image classification results meet the preset classification conditions through the decision layer. When the image classification results meet the preset classification conditions, it is determined that the classification is completed, and the target classification result (No. 6) corresponding to the multispectral image is obtained; when the image classification results do not meet the preset classification conditions, it is determined that the classification is not completed. At this time, the feature recombination sublayer (No. 32) inputs each fourth feature image into the pooling sublayer (No. 30) to continue feature extraction, and repeats the above process until the target classification result is obtained.

[0155] The implementation process of each step in the above two implementation methods refers to the corresponding steps in the above embodiments, which will not be described here. After obtaining the updated splicing feature map, the updated splicing feature map is used as a new splicing feature map, and the step S302 is returned to execute, that is, the updated splicing feature map is input into the classification prediction layer for image classification, and the image classification result corresponding to the multispectral image is obtained. This cycle is repeated until the target classification result that meets the classification conditions is obtained, and the classification of the multispectral image is realized.

[0156] The following is an introduction to the feature extraction method of multispectral images provided in the embodiment of the present application. Fig.11 This is another implementation flow diagram of the multispectral image feature extraction method provided in the embodiment of the present application, such as Fig.11 As shown, the feature extraction method of multispectral images provided in the embodiment of the present application is implemented collaboratively by the client and the server.

[0157] Step S401: In response to an upload operation on an original image sample, the client obtains an original multispectral image, and preprocesses the original multispectral image to obtain a multispectral image to be processed.

[0158] Here, the client receives the upload operation triggered by the user for the original multispectral image, obtains the local multispectral image uploaded by the user, and preprocesses the received original multispectral image, such as radiation correction, orthorectification, panchromatic sharpening, etc., and stretches the value corresponding to each pixel in the original multispectral image to the 0-255 natural scene image numerical space.

[0159] Step S402: the client sends the multispectral image to be processed to the server.

[0160] Step S403: the server obtains a synthetic image sample and a standard feature image corresponding to the synthetic image sample.

[0161] Here, after receiving the multispectral image to be processed, the server extracts the bands of the multispectral image to obtain the single-band information. According to actual needs, the band images corresponding to the single-band information are synthesized to obtain at least two synthesized images, each of which includes at least three band information, and the training samples are constructed based on the synthesized images.

[0162] And based on the category corresponding to the image content, the image content in each synthetic image is marked to obtain a standard feature image corresponding to the synthetic image.

[0163] Step S404: input the synthesized image sample into the feature extraction model, so that the feature extraction model extracts features from the synthesized image to obtain a corresponding splicing feature map.

[0164] Here, the synthetic image samples are input into the feature extraction model, and the feature extraction model is formally trained to obtain the predicted feature image.

[0165] Step S405, based on the difference between the predicted feature image and the standard feature image, update various parameters of the feature extraction model to obtain a trained feature extraction model.

[0166] Step S406, sending a prompt message indicating that the feature extraction model training is completed to the client.

[0167] Step S407 : the client sends a feature extraction request carrying the multispectral image to the server in response to the feature extraction instruction for the multispectral image to be processed.

[0168] In step S408, the server inputs the multispectral image to be processed into the trained feature extraction model for feature extraction to obtain a spliced ​​feature map.

[0169] Step S409: sending the spliced ​​feature map to the client.

[0170] Here, the client may present a stitching feature map corresponding to the input multispectral image to be processed in the human-computer interaction interface of the client.

[0171] The embodiment of the present application uses multi-channel shuffling to perform feature reorganization, so that the reorganized features are extracted during feature extraction. The features of each convolution are random, which can greatly reduce the precision loss, making the features in the extracted feature map more comprehensive and accurate. By means of band synthesis, different bands are synthesized separately and purposefully to extract more favorable category features, reducing the difficulty and feature loss of the model multi-band feature learning, and ultimately improving the accuracy of multispectral image processing.

[0172] The following is an explanation of an exemplary application of the embodiments of the present application in a practical application scenario.

[0173] Remote sensing satellite analysis is beneficial to land and resources surveys, economic analysis, and ecological environment monitoring. Compared with manual visual interpretation, automated remote sensing analysis technology can improve processing efficiency and cost. In recent years, the number of remote sensing satellites has gradually increased, and a large amount of high-resolution data has been acquired, enriching the detailed texture information. Remote sensing images are different from RGB natural images. They have many bands and a large value range, and they also have redundant information. When facing complex land object recognition, remote sensing recognition has a lower accuracy due to the complex background, image quality, image resolution, seasonal and weather factors, differences in land object scale, sample imbalance, etc.

[0174] Deep learning can automatically learn complex data features in a hierarchical manner. The current main approach is to directly use multi-band images as model input to train from scratch, or to only use RGB three-band true color images as model input. Different objects have different spectral reflectances at different wavelengths, and different band combinations have different responses to different objects.

[0175] Based on this, the embodiment of the present application provides a multi-spectral high-resolution remote sensing image feature extraction method based on deep learning, which performs feature extraction on multi-spectral high-resolution remote sensing images and constructs a feature extraction model for remote sensing feature extraction. Fig.12 , Fig.12 is a schematic diagram of a feature extraction framework for a multispectral image provided in an embodiment of the present application, combined with Fig.12 It can be seen that the multispectral image feature extraction model as a whole includes image preprocessing layer, band synthesis layer (Figure ① to Figure ②), preliminary feature extraction layer (Figure ② to Figure ③), pooling layer (Figure ③ to Figure ④), feature recombination layer (Figure ④ to Figure ⑤) and splicing layer (Figure ② to Figure ⑥, Figure ⑤ to Figure ⑥).

[0176] The image preprocessing layer is used to preprocess remote sensing images, including radiation correction, orthorectification, panchromatic sharpening, etc., and stretch the values ​​(such as percentage stretching, logarithmic stretching) to the 0-255 natural scene image numerical space.

[0177] Band synthesis layer, according to the characteristics of remote sensing images and target categories, the multispectral images of n bands of n*h*w are combined into different 3 bands, assuming that p synthetic images are formed, and the size of each synthetic image is expressed as 3*h*w. For example, GF2 images use RGB (321 bands) true color composition and Nir-RG (432 bands) false color composition, n = 4, p = 2, when manually interpreted (marked), true color is conducive to observing general objects, and false color is conducive to observing vegetation. When manually interpreted, humans will also actively combine bands to observe marks.

[0178] The initial feature extraction layer inputs the above p 3*h*w into a pre-trained preliminary feature extraction module for initial feature extraction. The initial feature extraction module can be trained by the sample image and the constructed convolutional feature extraction module, and the convolutional feature extraction module includes network layers such as convolutional layers, batch normalization layers, and activation layers.

[0179] The pooling layer performs a pooling operation on the results of the previous stage. The pooling layer is sandwiched between multiple preliminary feature extraction layers of multiple feature extractions, which can compress the input image and reduce overfitting.

[0180] Feature reorganization layer, using the shuffle block, see Fig.13 , Fig.13 This is a schematic diagram of an implementation flow of the channel shuffling method in a multispectral image provided by an embodiment of the present application. The above network combines data from different bands to extract features, and features of different groups are not communicated, which reduces the feature extraction ability of the network. Therefore, referring to the design of the shuffle network Shuffle Net, the shuffle module Shuffle Block is used to shuffle the features of different feature maps.

[0181] In specific operations, assume that the size of the p feature maps is B*c*h*w, where B is the model input batch size. Expand the dimensions of the above p feature map outputs to B*1*c*h*w, and superimpose them in the first dimension to obtain B*p*c*h*w. Transpose the first and second dimensions to obtain B*c*p*h*w. Reshape the result to B*(cxp)*h*w. Then redistribute the features, split the above results into p parts in order in the first dimension, and obtain the p feature maps corresponding to the shuffled features of the p feature maps.

[0182] The splicing layer splices the p feature maps obtained by the preliminary feature extraction layer to obtain a spliced ​​feature map (No. 1), and splices the p feature maps obtained by the feature recombination layer to obtain a spliced ​​feature map (No. 2). In the network of a specific task, the spliced ​​feature map is used as the input of the neural network, and is further input into the constructed network of the specific task to complete the specific task, such as a fully connected layer to construct a classification network, and an upsampling layer to construct a segmentation network. When the accuracy of a specific task does not meet the requirements, the p feature maps obtained by the feature recombination layer are used as input and re-input into the initial feature extraction layer, the pooling layer, and the feature recombination layer for updating to obtain new p feature maps, and then the new p feature maps are spliced ​​using the splicing layer to obtain a new spliced ​​feature map to continue to complete the specific task.

[0183] The embodiment of the present application proposes a multi-spectral high-resolution remote sensing image semantic segmentation method based on deep learning. By combining bands, different bands are purposefully combined to extract more favorable category features, so that the model can focus more on more useful bands and reduce the difficulty and feature loss of multi-band feature learning of the model. By using shuffling modules to connect, information can be exchanged, so that each branch of the model can obtain additional information, thereby improving the feature extraction capability. The feature extraction module is connected as input to different task modules, so that it can be used for remote sensing image tasks such as classification, segmentation, target detection, etc.

[0184] The following is a description of an exemplary structure of a multispectral image feature extraction device 555 provided in an embodiment of the present application implemented as a software module. In some embodiments, Figure 2 As shown, the software modules stored in the multispectral image feature extraction device 555 of the memory 540 may include:

[0185] An acquisition module 5551 is used to acquire a multispectral image to be processed;

[0186] A band synthesis module 5552 is used to perform band synthesis on the multispectral image to obtain at least two synthesized images, wherein the number of bands included in the synthesized images is less than the number of bands included in the multispectral image;

[0187] A preliminary feature extraction module 5553 is used to extract features from each of the synthetic images to obtain a first feature map corresponding to each of the synthetic images;

[0188] A feature recombination module 5554 is used to recombine the features in each of the first feature maps to obtain a second feature map corresponding to each of the first feature maps;

[0189] The stitching module 5555 is used to stitch the second feature maps to obtain a stitching feature map corresponding to the multispectral image.

[0190] In some embodiments, the band synthesis module 5552 is further used to:

[0191] Extracting the multiple bands included in the multispectral image to obtain single-band images corresponding to the bands;

[0192] Band synthesis is performed on the plurality of single-band images to obtain at least two synthesized images.

[0193] In some embodiments, the band synthesis module 5552 is further used to:

[0194] Obtaining the category to which the image content of the multispectral image belongs and the band parameters corresponding to each band;

[0195] Based on the category and the band parameter, selecting a single-band image from a plurality of the single-band images to form at least two single-band image groups, each of the single-band image groups including at least two single-band images;

[0196] Band synthesis is performed on each of the single-band image groups to obtain a corresponding synthesized image.

[0197] In some embodiments, the feature recombination module 5554 is further used to:

[0198] Perform band shuffling on the features in the at least two first feature maps obtained to obtain corresponding band shuffled feature maps;

[0199] The band shuffled feature maps are segmented to obtain second feature maps corresponding to each of the first feature maps.

[0200] In some embodiments, the feature recombination module 5554 is further configured to:

[0201] Performing dimension expansion on each of the first feature maps respectively to obtain dimension-expanded first feature maps;

[0202] Perform feature superposition on the first feature map after expansion of each dimension on the expanded dimension to obtain a feature map after feature superposition;

[0203] The feature map after the feature superposition is reconstructed to obtain the band shuffled feature map.

[0204] In some embodiments, the multispectral image feature extraction device 555 further includes:

[0205] A pooling module, used for performing pooling processing on each of the first feature maps before reorganizing the features in each of the first feature maps to obtain each pooled first feature map;

[0206] The feature recombination module 5554 is further used for:

[0207] The features in each of the pooled first feature maps are reorganized to obtain second feature maps corresponding to each of the first feature maps.

[0208] In some embodiments, the spliced ​​feature map is obtained by extracting features from the multispectral image through a feature extraction layer of an image classification model, the image classification model further includes a classification prediction layer, and the feature extraction device 555 of the multispectral image further includes:

[0209] An input module, used for inputting the spliced ​​feature map into the classification prediction layer for image classification, to obtain an image classification result corresponding to the multispectral image;

[0210] The determination module is used to determine the image classification result as a target classification result when the image classification result meets a preset classification condition.

[0211] In some embodiments, the preliminary feature extraction module 5553 is further used to extract features from each second feature map respectively to obtain a third feature map corresponding to each second feature map when the image classification result does not meet the preset classification condition;

[0212] The feature recombination module 5554 is further configured to recombine the features in each of the third feature maps to obtain a fourth feature map corresponding to each of the third feature maps;

[0213] The splicing module 5555 is further used to splice the fourth feature maps to obtain an updated spliced ​​feature map;

[0214] The input module is also used to input the updated splicing feature map into the classification prediction layer for image classification to obtain an image classification result corresponding to the multispectral image.

[0215] It should be noted that the description of the device of the embodiment of the present application is similar to the description of the above-mentioned method embodiment, and has similar beneficial effects as the method embodiment, so it will not be repeated.

[0216] An embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the feature extraction method of the multispectral image provided by the embodiment of the present application.

[0217] The present application embodiment provides a computer-readable storage medium storing executable instructions, wherein the executable instructions are stored. When the executable instructions are executed by a processor, the processor will execute the feature extraction method of the multispectral image provided by the present application embodiment, such as Figure 3 The feature extraction method of multispectral image is shown.

[0218] In some embodiments, the storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or various devices including one or any combination of the above memories.

[0219] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0220] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0221] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0222] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A feature extraction method for a multispectral image, characterized in that: The method comprises: Acquire a multispectral image to be processed, and perform band synthesis on the multispectral image to obtain at least two synthesized images, wherein the number of bands included in the synthesized images is less than the number of bands included in the multispectral image; Performing feature extraction on each of the synthesized images respectively to obtain a first feature map corresponding to each of the synthesized images; Recombining the features in each of the first feature maps respectively to obtain a second feature map corresponding to each of the first feature maps; Splicing the second feature maps to obtain a spliced ​​feature map corresponding to the multispectral image; The step of respectively reorganizing the features in each of the first feature maps to obtain a second feature map corresponding to each of the first feature maps includes: Performing dimension expansion on each of the first feature maps respectively to obtain dimension-expanded first feature maps; Perform feature superposition on the first feature map after expansion of each dimension on the expanded dimension to obtain a feature map after feature superposition; Performing feature reconstruction on the feature map after the feature superposition to obtain a band shuffled feature map; The band shuffled feature maps are segmented to obtain second feature maps corresponding to each of the first feature maps.

2. The method according to claim 1, characterized in that The step of performing band synthesis on the multispectral image to obtain at least two synthesized images includes: Extracting the multiple bands included in the multispectral image to obtain single-band images corresponding to the bands; Band synthesis is performed on the plurality of single-band images to obtain at least two synthesized images.

3. The method according to claim 2, characterized in that The step of performing band synthesis on the plurality of single-band images to obtain at least two synthesized images comprises: Obtaining the category to which the image content of the multispectral image belongs and the band parameters corresponding to each band; Based on the category and the band parameter, selecting a single-band image from a plurality of the single-band images to form at least two single-band image groups, each of the single-band image groups including at least two single-band images; Band synthesis is performed on each of the single-band image groups to obtain a corresponding synthesized image.

4. The method according to claim 1, characterized in that: Before respectively reorganizing the features in each of the first feature maps, the method further includes: Performing pooling processing on each of the first feature maps to obtain each pooled first feature map; The step of respectively reorganizing the features in the first feature maps to obtain second feature maps corresponding to the first feature maps comprises: The features in each of the pooled first feature maps are reorganized to obtain second feature maps corresponding to each of the first feature maps.

5. The method according to claim 1, characterized in that The splicing feature map is obtained by extracting features from the multispectral image through a feature extraction layer of an image classification model, the image classification model further includes a classification prediction layer, and the method further includes: Inputting the spliced ​​feature map into the classification prediction layer for image classification to obtain an image classification result corresponding to the multispectral image; When the image classification result meets the preset classification condition, the image classification result is determined as the target classification result.

6. The method according to claim 5, characterized in that The method comprises: When the image classification result does not meet the preset classification condition, performing feature extraction on each second feature map respectively to obtain a third feature map corresponding to each second feature map; According to each of the third feature maps, updating the splicing feature map to obtain an updated splicing feature map; The updated spliced ​​feature map is input into the classification prediction layer for image classification to obtain an image classification result corresponding to the multispectral image.

7. A feature extraction device for multispectral images, characterized in that: The device comprises: An acquisition module, used for acquiring a multispectral image to be processed; A band synthesis module, used for performing band synthesis on the multispectral image to obtain at least two synthesized images, wherein the number of bands included in the synthesized images is less than the number of bands included in the multispectral image; A preliminary feature extraction module, used to extract features from each of the synthetic images to obtain a first feature map corresponding to each of the synthetic images; A feature recombination module is used to perform feature recombination on the features in each of the first feature maps to obtain a second feature map corresponding to each of the first feature maps, wherein the feature recombination module is specifically used to perform dimension expansion on each of the first feature maps to obtain a first feature map after dimension expansion; perform feature superposition on the first feature map after dimension expansion on the expanded dimension to obtain a feature map after feature superposition; perform feature reconstruction on the feature map after feature superposition to obtain a band shuffled feature map; perform feature map segmentation on the band shuffled feature map to obtain a second feature map corresponding to each of the first feature maps; The stitching module is used to stitch the second feature maps to obtain a stitching feature map corresponding to the multispectral image.

8. An electronic device, characterized in that: include: A memory for storing executable instructions; The processor is used to implement the feature extraction method of the multispectral image according to any one of claims 1 to 6 when executing the executable instructions stored in the memory.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores executable instructions for causing a processor to execute instructions to implement the multispectral image feature extraction method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for extracting features from a multispectral image according to any one of claims 1 to 6 is implemented.

Images