Image feature extraction method and device, electronic device and storage medium

Abstract

The present invention discloses an image feature extraction method and device, electronic device, and storage medium thereof, and relates to financial technology. The method comprises: obtaining a data source image set of a remote sensing image, inputting multiple data source images into a pre-trained scene recognition model, wherein the scene recognition model extracts image features from the data source image, compares the image features with the object features of a preset target object, and analyzes whether the data source image is a target scene image associated with the preset target object. If the data source image is a target scene image, the target scene image is input into a pre-trained neural network model, and the image feature extraction results of the target scene image are spliced ​​with non-target scene images to obtain a feature extraction result graph of the remote sensing image. The present invention solves the technical problem in the related art of low target extraction accuracy and poor extraction efficiency when extracting small targets in large-scale remote sensing images.

Description

Technical Field

[0001] The present invention relates to the field of financial technology, and specifically to an image feature extraction method and device, electronic equipment, and storage medium. Background Art

[0002] In the credit information services provided by financial institutions (for example, pre-loan access, dynamic supervision, etc.), how to use remote sensing images to efficiently and accurately extract target objects has always been a research hotspot.

[0003] Existing methods for extracting targets from remote sensing images rely on basic information features of remote sensing images, such as single features like bands, textures, and gradients. These methods also require the use of manually collected training samples for target extraction and accuracy verification. Alternatively, deep learning-based methods for extracting targets from remote sensing images can be used to segment the images, train the model using large amounts of data, and automatically extract image features.

[0004] However, existing remote sensing image target extraction methods use low-level visual information as target extraction features. The extracted features are simple and require a large number of manual samples, resulting in semi-automated extraction and poor target extraction results. While deep learning-based remote sensing image target extraction methods can achieve automated target feature extraction and better target image extraction through network model training, for large-scale images with small extracted targets, it is difficult to distinguish between the target and background images. Direct feature extraction from segmented images results in poor model training results and low target extraction accuracy due to the uneven number of valid target and background images. Furthermore, a large amount of ineffective training leads to low efficiency.

[0005] To address the above-mentioned problems, no effective solutions have been proposed so far. Summary of the Invention

[0006] The embodiments of the present invention provide an image feature extraction method and device, an electronic device and a storage medium, so as to at least solve the technical problems in the related art of low target extraction accuracy and poor extraction efficiency when extracting small targets in large-scale remote sensing images.

[0007] According to one aspect of an embodiment of the present invention, there is provided an image feature extraction method, comprising: obtaining a data source image set of a remote sensing image, wherein the data source image set includes a plurality of data source images; inputting the plurality of data source images into a pre-trained scene recognition model, wherein the scene recognition model extracts image features from the data source image, compares the image features with object features of a preset target object, and analyzes whether the data source image is a target scene image associated with the preset target object based on the comparison result to obtain an analysis result; when the analysis result indicates that the data source image is a target scene image, inputting the target scene image into a pre-trained neural network model, wherein the neural network model extracts features from each of the target scene images to obtain an image feature extraction result; and splicing the image feature extraction result of the target scene image with a non-target scene image to obtain a feature extraction result map of the remote sensing image, wherein the pixel positions of the preset target object are identified in the feature extraction result map.

[0008] Optionally, the step of obtaining a data source image set of a remote sensing image includes: performing error correction on the remote sensing image, wherein the error correction method includes at least one of the following: radiation calibration, atmospheric correction, geometric correction and orthorectification; and performing non-overlapping area cropping on the remote sensing image after error correction to obtain the data source image set.

[0009] Optionally, the step of performing feature extraction on each of the target scene images to obtain an image feature extraction result includes: performing feature extraction on each of the input target scene images through the neural network model; identifying the object pixels associated with the preset target object in the target scene image as a first parameter, and identifying the object pixels associated with the non-preset target object in the target scene image as a second parameter to obtain the image feature extraction result.

[0010] Optionally, before obtaining the data source image set of remote sensing images, it also includes: taking a deep convolutional neural network as the basic network model structure; adjusting the output layer of the basic network model structure according to the image output category; using the target scene images and background images marked in the source data set to train the basic network model structure to obtain the scene recognition model.

[0011] Optionally, after adjusting the output layer of the basic network model structure according to the image output category, it also includes: initializing the basic network model structure through model parameters trained on a preset data set through transfer learning.

[0012] Optionally, after training the basic network model structure to obtain the scene recognition model, the method further includes: calculating an error through a loss function and adjusting a weight parameter of each model network layer in the scene recognition model.

[0013] Optionally, before obtaining the data source image set of remote sensing images, it also includes: classifying the pre-labeled remote sensing image set into a training set and a validation set, wherein the training set is used for model parameter training and tuning, and the validation set is used for testing the model extraction accuracy; constructing the model infrastructure of the neural network model and initializing the model parameters; using the training set to train the neural network model, and using the validation set to analyze the feature extraction accuracy of the neural network model; and confirming that the training of the neural network model is completed when the feature extraction accuracy is greater than a preset accuracy threshold.

[0014] According to another aspect of an embodiment of the present invention, an image feature extraction device is also provided, including: an acquisition unit, used to acquire a data source image set of a remote sensing image, wherein the data source image set includes multiple data source images; a first input unit, used to input the multiple data source images into a pre-trained scene recognition model, wherein the scene recognition model extracts image features from the data source image, compares the image features with object features of a preset target object, and analyzes whether the data source image is a target scene image associated with the preset target object based on the comparison result to obtain an analysis result; a second input unit, used to input the target scene image into a pre-trained neural network model when the analysis result indicates that the data source image is a target scene image, wherein the neural network model extracts features from each target scene image to obtain an image feature extraction result; a splicing unit, used to splice the image feature extraction results of the target scene image with non-target scene images to obtain a feature extraction result map of the remote sensing image, wherein the pixel positions of the preset target object are identified in the feature extraction result map.

[0015] Optionally, the acquisition unit includes: a first correction module, used to perform error correction on the remote sensing image, wherein the error correction method includes at least one of the following: radiation calibration, atmospheric correction, geometric correction and orthorectification; a first cropping module, used to crop the remote sensing image after error correction to obtain the data source image set.

[0016] Optionally, the second input unit includes: a first extraction module, used to extract features from each input target scene image through the neural network model; a first identification module, used to identify object pixels associated with the preset target object in the target scene image as a first parameter, and identify object pixels associated with non-preset target objects in the target scene image as a second parameter, to obtain the image feature extraction result.

[0017] Optionally, the extraction device also includes: a first output module, used to use a deep convolutional neural network as a basic network model structure before obtaining a data source image set of remote sensing images; a first adjustment module, used to adjust the output layer of the basic network model structure according to the image output category; a first training module, used to use the target scene images and background images marked in the source data set to train the basic network model structure to obtain the scene recognition model.

[0018] Optionally, the extraction device also includes: a first initialization module, which is used to initialize the basic network model structure through transfer learning and model parameters trained on a preset data set after adjusting the output layer of the basic network model structure according to the image output category.

[0019] Optionally, the extraction device also includes: a second adjustment module, which is used to adjust the weight parameters of each model network layer in the scene recognition model by calculating the error through a loss function after training the basic network model structure to obtain the scene recognition model.

[0020] Optionally, the extraction device also includes: a first classification module, used to classify the pre-labeled remote sensing image set into a training set and a verification set before obtaining the data source image set of the remote sensing image, wherein the training set is used for model parameter training and tuning, and the verification set is used for testing the model extraction accuracy; a first construction module, used to construct the model infrastructure of the neural network model and initialize the model parameters; a first analysis module, used to train the neural network model using the training set, and use the verification set to analyze the feature extraction accuracy of the neural network model; a first confirmation module, used to confirm that the training of the neural network model is completed when the feature extraction accuracy is greater than a preset accuracy threshold.

[0021] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is also provided, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the above-mentioned image feature extraction method.

[0022] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is also provided, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the above-mentioned image feature extraction method.

[0023] In the present disclosure, a data source image set of a remote sensing image is obtained, and multiple data source images are input into a pre-trained scene recognition model, wherein the scene recognition model extracts image features from the data source image, compares the image features with the object features of a preset target object, and analyzes whether the data source image is a target scene image associated with the preset target object based on the comparison result to obtain an analysis result. If the analysis result indicates that the data source image is a target scene image, the target scene image is input into a pre-trained neural network model, and the image feature extraction result of the target scene image is spliced ​​with the non-target scene image to obtain a feature extraction result map of the remote sensing image. In the present application, the obtained multiple data source images can be input into the scene recognition model for target scene image recognition, and the identified target scene image can be input into the neural network model for feature extraction, and then the image feature extraction result is spliced ​​with the non-target scene image, so as to obtain a feature extraction result map of the remote sensing image, which not only improves the target extraction efficiency, but also effectively avoids the misclassification of the background image, effectively improves the target extraction accuracy, and thus solves the technical problem of low target extraction accuracy and poor extraction efficiency in the related art when extracting small targets in large-scale remote sensing images. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0025] Figure 1 is a flow chart of an optional image feature extraction method according to an embodiment of the present invention;

[0026] Figure 2 is a flow chart of an optional method for automatically extracting targets from remote sensing images according to an embodiment of the present invention;

[0027] Figure 3 is a schematic diagram of an optional scene recognition model implementation according to an embodiment of the present invention;

[0028] Figure 4 is a schematic diagram of an optional neural network model implementation according to an embodiment of the present invention;

[0029] Figure 5is a schematic diagram of an optional image feature extraction device according to an embodiment of the present invention;

[0030] Figure 6 This is a hardware structure block diagram of an electronic device (or mobile device) for an image feature extraction method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] To facilitate those skilled in the art to understand the present invention, some of the terms or nouns involved in the embodiments of the present invention are explained below:

[0034] Remote sensing image scene recognition: remote sensing images of the same scene should have similar feature information, that is, the scene can be classified according to the feature information of the remote sensing image. Remote sensing image scenes can be divided into: forests, buildings, bridges and dams, reservoirs, cultivated land scenes, etc.

[0035] Background image: refers to the non-target scene remote sensing image based on scene classification after remote sensing image segmentation.

[0036] It should be noted that the image feature extraction method and device in the present disclosure can be used in the field of financial technology when performing image feature extraction, and can also be used in any field other than the field of financial technology when performing image feature extraction. The present disclosure does not limit the application field of the image feature extraction method and device.

[0037] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data for analysis, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties.

[0038] The following embodiments of the present invention can be applied to various systems / applications / devices for image feature extraction. This invention addresses the problem of automatically extracting small targets from large-scale remote sensing images and proposes a deep learning remote sensing image target automatic extraction method combined with scene classification. This method can effectively narrow the target image extraction range and only extract targets from the scene images of interest, thereby accelerating target extraction efficiency. Furthermore, it can solve the problem of misclassifying a large amount of background images due to the imbalance of positive and negative samples when directly extracting targets from large-scale remote sensing images, thereby improving target extraction accuracy.

[0039] The present invention will be described in detail below with reference to various embodiments.

[0040] Example 1

[0041] According to an embodiment of the present invention, an embodiment of an image feature extraction method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0042] Figure 1 is a flow chart of an optional image feature extraction method according to an embodiment of the present invention, such as Figure 1 As shown, the method includes the following steps:

[0043] Step S101: Acquire a data source image set of a remote sensing image, wherein the data source image set includes a plurality of data source images.

[0044] In step S102, a plurality of data source images are input into a pre-trained scene recognition model, wherein the scene recognition model extracts image features from the data source images, compares the image features with the object features of the preset target object, and analyzes whether the data source images are target scene images associated with the preset target object based on the comparison results to obtain analysis results.

[0045] Step S103: When the analysis result indicates that the data source image is a target scene image, the target scene image is input into a pre-trained neural network model, wherein the neural network model performs feature extraction on each target scene image to obtain an image feature extraction result.

[0046] Step S104 , splicing the image feature extraction result of the target scene image with the non-target scene image to obtain a feature extraction result map of the remote sensing image, wherein the pixel positions of the preset target object are marked in the feature extraction result map.

[0047] Through the above steps, a data source image set of a remote sensing image can be obtained, and multiple data source images are input into a pre-trained scene recognition model, wherein the scene recognition model extracts image features from the data source image, compares the image features with the object features of a preset target object, and analyzes whether the data source image is a target scene image associated with the preset target object based on the comparison result to obtain an analysis result. If the analysis result indicates that the data source image is a target scene image, the target scene image is input into a pre-trained neural network model, and the image feature extraction results of the target scene image are spliced ​​with the non-target scene image to obtain a feature extraction result map of the remote sensing image. In an embodiment of the present invention, the obtained multiple data source images can be input into the scene recognition model for target scene image recognition, and the identified target scene image is input into the neural network model for feature extraction, and then the image feature extraction results are spliced ​​with the non-target scene image, so as to obtain a feature extraction result map of the remote sensing image, which not only improves the target extraction efficiency, but also effectively avoids the misclassification of the background image, effectively improves the target extraction accuracy, and thus solves the technical problem of low target extraction accuracy and poor extraction efficiency in the related art when extracting small targets in large-scale remote sensing images.

[0048] The embodiment of the present invention is described in detail below in conjunction with the above steps.

[0049] In an embodiment of the present invention, an optional method, before obtaining a data source image set of remote sensing images, further includes: using a deep convolutional neural network as a basic network model structure; adjusting the output layer of the basic network model structure according to the image output category; using the target scene images and background images marked in the source data set to train the basic network model structure to obtain a scene recognition model.

[0050] In this embodiment, a deep convolutional neural network (for example, VGG16) can be used as the basic network model structure (the basic network model structure can include 13 convolutional layers, 3 fully connected layers, etc.), and the output layer of the basic network model structure can be adjusted according to the image output category (including: target scene image and background image), and the basic network model structure can be trained using the target scene images and background images marked in the source data set to obtain a scene recognition model.

[0051] Optionally, after adjusting the output layer of the basic network model structure according to the image output category, the method further includes: initializing the basic network model structure through model parameters trained on a preset data set through transfer learning.

[0052] In an embodiment of the present invention, after adjusting the output layer of the basic network model structure, the basic network model structure can be initialized through transfer learning using model parameters trained on a preset data set (for example, VGG16 model parameters pre-trained on the ImageNet data set).

[0053] Optionally, after training the basic network model structure to obtain the scene recognition model, the method further includes: calculating the error through a loss function and adjusting the weight parameters of each model network layer in the scene recognition model.

[0054] In an embodiment of the present invention, after obtaining the scene recognition model, the error can be calculated through the loss function to adjust and update the weights of the entire scene recognition network (i.e., adjust the weight parameters of each model network layer in the scene recognition model).

[0055] Optionally, before obtaining the data source image set of the remote sensing image, it also includes: classifying the pre-labeled remote sensing image set into a training set and a validation set, wherein the training set is used for model parameter training and tuning, and the validation set is used for testing the model extraction accuracy; building a model infrastructure of the neural network model and initializing the model parameters; using the training set to train the neural network model, and using the validation set to analyze the feature extraction accuracy of the neural network model; and confirming that the neural network model training is completed when the feature extraction accuracy is greater than a preset accuracy threshold.

[0056] In an embodiment of the present invention, the labeled remote sensing images can be divided into two categories according to a preset ratio (for example, 7:3): a training set and a validation set (i.e., the pre-labeled remote sensing image set is classified into a training set and a validation set). The training set is used for model parameter training and tuning, and the validation set is used to test the model extraction accuracy. In addition, if the remote sensing data set is too small, it can be scaled and rotated according to different ratios to expand the sample. Afterwards, a neural network model can be created and the model parameters can be initialized (i.e., the model infrastructure of the neural network model is constructed and the model parameters are initialized). The labeled remote sensing image training data set can be directly used for model training and the model parameters can be adjusted (i.e., the neural network model is trained using the training set). The validation set prepared in advance is used to verify the pixel-level target extraction accuracy of the network model (i.e., the feature extraction accuracy of the neural network model is analyzed using the validation set). If the accuracy verification passes, the trained model is saved (i.e., if the feature extraction accuracy is greater than a preset accuracy threshold (which can be set according to actual conditions), the neural network model training is confirmed to be complete). Otherwise, the network model parameters are adjusted and training continues until the feature extraction accuracy is greater than the preset accuracy threshold, thereby obtaining a pixel-level target extraction network model (i.e., a neural network model).

[0057] Step S101: Acquire a data source image set of a remote sensing image, wherein the data source image set includes a plurality of data source images.

[0058] Optionally, the step of obtaining a data source image set of the remote sensing image includes: performing error correction on the remote sensing image, wherein the error correction method includes at least one of the following: radiation calibration, atmospheric correction, geometric correction and orthorectification; and performing non-overlapping area cropping on the remote sensing image after the error correction to obtain the data source image set.

[0059] In an embodiment of the present invention, the data source image set of the remote sensing image contains multiple data source images. The remote sensing image can be error corrected first, including: radiation calibration, atmospheric correction, geometric correction and orthorectification, etc., and the remote sensing image after error correction is cropped with non-overlapping areas, so as to obtain the input data source (i.e., the data source image set).

[0060] In step S102, a plurality of data source images are input into a pre-trained scene recognition model, wherein the scene recognition model extracts image features from the data source images, compares the image features with the object features of the preset target object, and analyzes whether the data source images are target scene images associated with the preset target object based on the comparison results to obtain analysis results.

[0061] In an embodiment of the present invention, multiple data source images can be input into a pre-trained scene recognition model. The scene recognition model extracts image features from the data source image to determine whether the data source image is a target scene image (that is, the image features can be compared with the object features of a preset target object, and based on the comparison results, the data source image is analyzed to determine whether it is a target scene image associated with the preset target object to obtain an analysis result). For example, a building can be used as an extraction target, and multiple cropped image data source images can be input into a pre-trained VGG16 scene recognition model. The image features are extracted to determine whether it is a target scene image. The target scene image is marked as 1, and the non-target scene image is marked as 0.

[0062] Optionally, if it is a target scene image, the file indexes of all target scene images can be obtained to prepare for subsequent pixel-level file indexing. If it is a non-target scene image, the pixel value of the image can be changed to 0.

[0063] Step S103: When the analysis result indicates that the data source image is a target scene image, the target scene image is input into a pre-trained neural network model, wherein the neural network model performs feature extraction on each target scene image to obtain an image feature extraction result.

[0064] In an embodiment of the present invention, the file index of the target scene image can be obtained, and these target scene images can be input into a previously trained pixel-level deep convolutional neural network model to perform pixel-level target extraction (that is, when the analysis result indicates that the data source image is the target scene image, the target scene image is input into the pre-trained neural network model). In this embodiment, the neural network model can perform feature extraction on each target scene image to obtain image feature extraction results.

[0065] Optionally, the step of performing feature extraction on each target scene image to obtain an image feature extraction result includes: performing feature extraction on each input target scene image through a neural network model; identifying object pixels associated with a preset target object in the target scene image as a first parameter, and identifying object pixels associated with a non-preset target object in the target scene image as a second parameter to obtain an image feature extraction result.

[0066] In an embodiment of the present invention, a neural network model can be used to perform feature extraction on each input target scene image and perform pixel-level type prediction at the same time. The object pixels associated with the preset target object in the target scene image are identified as a first parameter (for example, 1), and the object pixels associated with the non-preset target object in the target scene image (i.e., background pixels) are identified as a second parameter (for example, 0) to obtain image feature extraction results. For example, if a building is used as the extraction target, the building pixel can be marked as 1, and other background pixels can be marked as 0, and the pixel-level target extraction result of the target scene image is output.

[0067] Step S104 , splicing the image feature extraction result of the target scene image with the non-target scene image to obtain a feature extraction result map of the remote sensing image, wherein the pixel positions of the preset target object are marked in the feature extraction result map.

[0068] In an embodiment of the present invention, the obtained pixel-level extraction results of the target scene image (i.e., the image feature extraction results of the target scene image) and the obtained non-target scene image can be sequentially spliced ​​through the file index of the scene image, so that a feature extraction result map of the entire remote sensing image can be obtained (the feature extraction result map identifies the pixel positions of the preset target object).

[0069] The following describes in detail another optional specific implementation.

[0070] Figure 2 FIG. 1 is a flow chart of an optional method for automatically extracting targets from remote sensing images according to an embodiment of the present invention. Figure 2 As shown, the process is as follows:

[0071] (1) Preprocessing of remote sensing image data sources: First, error correction is performed on the remote sensing image, including radiation calibration, atmospheric correction, geometric correction, and orthorectification, etc. The error-corrected remote sensing image is cropped without overlapping areas to obtain the input data source.

[0072] (2) Inputting the input data source into the scene recognition model: For example, taking a building as the extraction target, the cropped multiple image data source images are input into the pre-trained VGG16 scene recognition model, and the image features are extracted to determine whether it is a target scene image. The target scene image can be marked as 1, and the non-target scene image can be marked as 0.

[0073] (3) Target scene image judgment: If it is a target scene image, the file indexes of all target scene images are obtained to prepare for the subsequent pixel-level file indexing. If it is a non-target scene image, the pixel value of the non-target scene image is changed to 0.

[0074] (4) By obtaining the file index of the target scene image, these target scene images are input into the previously trained pixel-level deep convolutional neural network model to perform pixel-level target extraction.

[0075] (5) To extract building targets from images, a neural network model can be used to extract features from each input target scene image and perform pixel-level type prediction. If it is a building pixel, the pixel can be marked as 1, and if it is other background pixels, it can be marked as 0, and the pixel-level target extraction result of the target scene image can be output.

[0076] (6) Orderly image stitching: Through the file index of the scene image, the pixel-level target extraction results of the acquired target scene image can be orderly stitched with the acquired non-target scene image.

[0077] (7) Obtain the pixel-level extraction result map of the building target in the entire remote sensing image.

[0078] Figure 3 is a schematic diagram of an optional scene recognition model implementation according to an embodiment of the present invention, such as Figure 3 As shown, the process is as follows:

[0079] (1) Pre-trained model: The deep convolutional neural network VGG16 can be used as the basic network model structure (including: 13 convolutional layers, 3 fully connected layers, etc.). Since the output results are two categories (target image and background image), the final output layer can be changed to two categories. At the same time, through transfer learning, the VGG16 model parameters pre-trained on the ImageNet dataset can be used for initialization.

[0080] (2) Fine-tuning model parameters: Use the target scene graph and background image marked in the source data to train the scene recognition model, calculate the error through the loss function, and adjust and update the weights of the entire scene recognition network.

[0081] (3) Obtaining a scene recognition model: The scene recognition network model is trained using the non-overlapping cropped photo label data, that is, the target scene photo is ultimately labeled as 1 and the background photo is labeled as 0 through the scene model.

[0082] Figure 4 is a schematic diagram of an optional neural network model implementation according to an embodiment of the present invention, such as Figure 4 As shown, the process is as follows:

[0083] (1) Remote sensing dataset production: The labeled remote sensing images are classified into two categories according to a preset ratio (e.g., 7:3), namely, training set and validation set. The training set is used for model parameter training and tuning, and the validation set is used to test the model extraction accuracy. If the remote sensing dataset is too small, it can be scaled and rotated according to different ratios to expand the sample.

[0084] (2) Creating a neural network model: In this embodiment, VGG16 can be used to create a neural network model and initialize the model parameters.

[0085] (3) Model training: The labeled remote sensing image training dataset can be used directly to train the model and adjust the model parameters.

[0086] (4) Determine whether the extraction accuracy of the training model meets the requirements: Use the validation set prepared in advance to verify the pixel-level target extraction accuracy of the network model. If the extraction accuracy is greater than the preset threshold, the requirements are met, the accuracy verification is passed, and the training model is saved. Otherwise, adjust the network model parameters and continue training.

[0087] (5) Save the training model, that is, obtain the pixel-level target extraction neural network model.

[0088] In an embodiment of the present invention, in order to solve the problem of automatic extraction of small targets from large-scale remote sensing images in credit information services of financial institutions, a deep learning remote sensing image target automatic extraction method combined with scene classification is proposed. The method can effectively narrow the target image extraction range and only extract targets from scene images of interest, thereby accelerating the target extraction efficiency. At the same time, it solves the problem of misclassification of a large number of background images due to the imbalance of positive and negative samples when directly extracting targets from large-scale remote sensing images, thereby improving the target extraction accuracy.

[0089] Example 2

[0090] An image feature extraction device provided in this embodiment includes multiple implementation units, each implementation unit corresponding to each implementation step in the above-mentioned embodiment 1.

[0091] Figure 5 is a schematic diagram of an optional image feature extraction device according to an embodiment of the present invention, such as Figure 5 As shown, the extraction device may include: an acquisition unit 50, a first input unit 51, a second input unit 52, and a splicing unit 53, wherein:

[0092] An acquisition unit 50 is configured to acquire a data source image set of a remote sensing image, wherein the data source image set includes a plurality of data source images;

[0093] A first input unit 51 is configured to input a plurality of data source images into a pre-trained scene recognition model, wherein the scene recognition model extracts image features from the data source images, compares the image features with object features of a preset target object, and analyzes whether the data source images are target scene images associated with the preset target object based on the comparison results, thereby obtaining an analysis result;

[0094] A second input unit 52 is configured to input the target scene image into a pre-trained neural network model when the analysis result indicates that the data source image is a target scene image, wherein the neural network model performs feature extraction on each target scene image to obtain an image feature extraction result;

[0095] The stitching unit 53 is used to stitch the image feature extraction result of the target scene image with the non-target scene image to obtain a feature extraction result map of the remote sensing image, wherein the pixel positions of the preset target object are marked in the feature extraction result map.

[0096] The above-mentioned extraction device can obtain a data source image set of the remote sensing image through the acquisition unit 50, and input multiple data source images into a pre-trained scene recognition model through the first input unit 51, wherein the scene recognition model extracts image features from the data source image, compares the image features with the object features of the preset target object, and analyzes whether the data source image is a target scene image associated with the preset target object based on the comparison result to obtain an analysis result. When the analysis result indicates that the data source image is a target scene image, the target scene image is input into the pre-trained neural network model through the second input unit 52, and the image feature extraction result of the target scene image is spliced ​​with the non-target scene image through the splicing unit 53 to obtain a feature extraction result graph of the remote sensing image. In an embodiment of the present invention, the obtained multiple data source images can be input into a scene recognition model for target scene image recognition, and the recognized target scene image can be input into a neural network model for feature extraction. Thereafter, the image feature extraction result is spliced ​​with the non-target scene image, so as to obtain a feature extraction result map of the remote sensing image. This not only improves the target extraction efficiency, but also effectively avoids the misclassification of the background image, effectively improves the target extraction accuracy, and thus solves the technical problem in the related art of low target extraction accuracy and poor extraction efficiency when extracting small targets in large-scale remote sensing images.

[0097] Optionally, the acquisition unit includes: a first correction module, used to perform error correction on the remote sensing image, wherein the error correction method includes at least one of the following: radiation calibration, atmospheric correction, geometric correction and orthorectification; a first cropping module, used to crop the remote sensing image after error correction without overlapping areas to obtain a data source image set.

[0098] Optionally, the second input unit includes: a first extraction module, used to extract features from each input target scene image through a neural network model; a first identification module, used to identify object pixels associated with a preset target object in the target scene image as a first parameter, and to identify object pixels associated with a non-preset target object in the target scene image as a second parameter, to obtain image feature extraction results.

[0099] Optionally, the extraction device also includes: a first output module, used to use a deep convolutional neural network as the basic network model structure before obtaining the data source image set of the remote sensing image; a first adjustment module, used to adjust the output layer of the basic network model structure according to the image output category; a first training module, used to use the target scene images and background images marked in the source data set to train the basic network model structure to obtain a scene recognition model.

[0100] Optionally, the extraction device also includes: a first initialization module, which is used to initialize the basic network model structure through transfer learning and model parameters trained on a preset data set after adjusting the output layer of the basic network model structure according to the image output category.

[0101] Optionally, the extraction device also includes: a second adjustment module, which is used to adjust the weight parameters of each model network layer in the scene recognition model by calculating the error through a loss function after training the basic network model structure to obtain the scene recognition model.

[0102] Optionally, the extraction device also includes: a first classification module, used to classify the pre-labeled remote sensing image set into a training set and a verification set before obtaining the data source image set of the remote sensing image, wherein the training set is used for model parameter training and tuning, and the verification set is used for testing the model extraction accuracy; a first construction module, used to construct the model infrastructure of the neural network model and initialize the model parameters; a first analysis module, used to train the neural network model using the training set, and use the verification set to analyze the feature extraction accuracy of the neural network model; a first confirmation module, used to confirm that the neural network model training is completed when the feature extraction accuracy is greater than a preset accuracy threshold.

[0103] The above-mentioned extraction device can also include a processor and a memory. The above-mentioned acquisition unit 50, first input unit 51, second input unit 52, splicing unit 53, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize the corresponding functions.

[0104] The processor includes a kernel that retrieves the corresponding program unit from memory. One or more kernels can be configured, and kernel parameters are adjusted to combine the image feature extraction results of the target scene image with the non-target scene image to generate a feature extraction result map for the remote sensing image.

[0105] The above-mentioned memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0106] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program initialized with the following method steps: obtaining a set of data source images of a remote sensing image, inputting multiple data source images into a pre-trained scene recognition model, wherein the scene recognition model extracts image features from the data source image, compares the image features with object features of a preset target object, and analyzes whether the data source image is a target scene image associated with the preset target object based on the comparison result to obtain an analysis result. When the analysis result indicates that the data source image is a target scene image, the target scene image is input into a pre-trained neural network model, and the image feature extraction result of the target scene image is spliced ​​with the non-target scene image to obtain a feature extraction result graph of the remote sensing image.

[0107] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is also provided, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the above-mentioned image feature extraction method.

[0108] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is also provided, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the above-mentioned image feature extraction method.

[0109] Figure 6 FIG is a hardware structure block diagram of an electronic device (or mobile device) for an image feature extraction method according to an embodiment of the present invention. Figure 6As shown, the electronic device may include one or more (illustrated as 102a, 102b, ..., 102n in the figure) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a keyboard, a power supply and / or a camera. It will be understood by those skilled in the art that Figure 6 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 6 More or fewer components than shown, or with Figure 6 Different configurations shown.

[0110] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0111] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0112] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of the units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0113] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0114] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0115] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk, etc. Various media that can store program codes.

[0116] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A method for extracting image features, characterized in that: include: Acquire a data source image set of a remote sensing image, wherein the data source image set includes a plurality of data source images; Inputting the plurality of data source images into a pre-trained scene recognition model, wherein the scene recognition model extracts image features from the data source images, compares the image features with object features of a preset target object, and analyzes whether the data source images are target scene images associated with the preset target object based on the comparison results, thereby obtaining an analysis result; When the analysis result indicates that the data source image is a target scene image, obtaining a file index of the target scene image, and inputting the target scene image into a pre-trained neural network model, wherein the neural network model performs feature extraction on each target scene image to obtain an image feature extraction result; The step of extracting features from each target scene image comprises: extracting features from each input target scene image using the neural network model; identifying object pixels in the target scene image associated with the preset target object as a first parameter, and identifying object pixels in the target scene image associated with non-preset target objects as a second parameter, to obtain the image feature extraction result; The image feature extraction result of the target scene image is spliced ​​with the non-target scene image to obtain a feature extraction result map of the remote sensing image, wherein the pixel positions of the preset target object are identified in the feature extraction result map.

2. The method according to claim 1, characterized in that The steps for obtaining a data source image collection of remote sensing images include: Performing error correction on the remote sensing image, wherein the error correction comprises at least one of the following correction methods: radiometric calibration, atmospheric correction, geometric correction, and orthorectification; The error-corrected remote sensing images are cropped without overlapping areas to obtain the data source image set.

3. The method according to claim 1, characterized in that Before obtaining the data source image collection of remote sensing images, it also includes: The network model structure is based on deep convolutional neural network; Adjusting the output layer of the basic network model structure according to the image output category; The basic network model structure is trained using the target scene image and background image marked in the source data set to obtain the scene recognition model.

4. The method according to claim 3, characterized in that After adjusting the output layer of the basic network model structure according to the image output category, the method further includes: Through transfer learning, the basic network model structure is initialized using the model parameters trained on a preset data set.

5. The method according to claim 3, characterized in that After training the basic network model structure to obtain the scene recognition model, the method further includes: The error is calculated through the loss function, and the weight parameters of each model network layer in the scene recognition model are adjusted.

6. The method according to claim 1, characterized in that Before obtaining the data source image collection of remote sensing images, it also includes: Classify the pre-labeled remote sensing image set into a training set and a validation set, wherein the training set is used for model parameter training and tuning, and the validation set is used for testing the model extraction accuracy; Constructing a model infrastructure of the neural network model and initializing model parameters; Using the training set to train the neural network model, and using the validation set to analyze the feature extraction accuracy of the neural network model; When the feature extraction accuracy is greater than a preset accuracy threshold, it is confirmed that the neural network model training is completed.

7. An image feature extraction device, characterized in that: include: An acquisition unit, configured to acquire a data source image set of a remote sensing image, wherein the data source image set includes a plurality of data source images; a first input unit, configured to input the plurality of data source images into a pre-trained scene recognition model, wherein the scene recognition model extracts image features from the data source images, compares the image features with object features of a preset target object, and analyzes whether the data source images are target scene images associated with the preset target object based on the comparison results, thereby obtaining an analysis result; a second input unit, configured to, when the analysis result indicates that the data source image is a target scene image, obtain a file index of the target scene image, and input the target scene image into a pre-trained neural network model, wherein the neural network model performs feature extraction on each target scene image to obtain an image feature extraction result; The second input unit includes: a first extraction module, configured to perform feature extraction on each input target scene image through the neural network model; a first identification module, configured to identify object pixels associated with the preset target object in the target scene image as a first parameter, and identify object pixels associated with non-preset target objects in the target scene image as a second parameter, to obtain the image feature extraction result; The splicing unit is used to splice the image feature extraction result of the target scene image with the non-target scene image to obtain a feature extraction result map of the remote sensing image, wherein the pixel position of the preset target object is marked in the feature extraction result map.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the image feature extraction method according to any one of claims 1 to 6.

9. An electronic device, characterized in that: It includes one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the image feature extraction method described in any one of claims 1 to 6.

Images