Visual feature adaptation method based on flight scene category driving
By establishing a labeled scene classification dataset and evaluating feature extraction algorithms, the problem of poor feature adaptability under different flight scenarios was solved, a more comprehensive and objective evaluation of feature extraction algorithms was achieved, the matching degree between algorithms and scenarios was optimized, and the accuracy and reliability of aircraft vision tasks were improved.
Patent Information
- Application Number
- CN202511035411.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-25
- Publication Date
- 2025-11-18
AI Technical Summary
Existing technologies struggle to provide comprehensive and objective evaluations of feature extraction algorithms across different flight scenarios, resulting in poor feature adaptability and difficulty in meeting the demands of ever-changing flight environments.
A labeled scene classification dataset is established, divided into local blocks, and various image feature extraction algorithms are used for feature extraction and dimensionality reduction. The adaptability of the feature extraction algorithms in different scenes is evaluated through clustering and feature fit calculation.
It provides a more comprehensive and objective feature extraction algorithm evaluation system to help select the most suitable feature extraction algorithm, improve the accuracy and reliability of aircraft vision tasks, and enhance the adaptability and robustness of the algorithm.
Smart Images
Figure CN120976719A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of image processing, and in particular to a visual feature adaptation method based on flight scene category driving. BACKGROUND
[0002] In applications such as UAV navigation and remote sensing image analysis, feature extraction is a core step of environment perception, positioning and identification, and the performance of subsequent tasks is highly dependent on the results of feature extraction algorithms. In flight scenes, the limitation of computing resources often needs to be considered, and feature extraction compresses the dimension through key information such as edges and textures, which is beneficial to solving the "dimension curse" problem. However, remote sensing images of scenes such as mountains, forests and coasts have significant differences in characteristics, mainly in spectral features, texture structures, spatial scales and the like. Therefore, in the face of different scenes, the feature adaptation of various algorithms differs, and it is difficult to find a universal feature extraction algorithm to meet the needs of all scenes. For example, in the mountain scene, the complex terrain and variable vegetation types make feature extraction need to pay more attention to the contour of the terrain and the texture details of the vegetation; while in the coastal scene, the feature extraction at the junction of the sea and the land needs to consider the change of spectral reflectance and the shape feature of the coastline. This requires optimization and adjustment of algorithms for specific scenes to improve the accuracy and effectiveness of feature extraction.
[0003] Currently, the evaluation of the feature expression ability of algorithms often focuses on the improvement of the extracted feature parameters or the performance of specific tasks, such as the number of extracted features, recall rate, F1 score, etc. However, these evaluation indicators often only reflect some features of the feature extraction algorithm that are not closely related to feature adaptation, or the performance in specific situations, making it difficult to comprehensively measure the comprehensive feature expression ability of algorithms in different scenes and tasks. Therefore, a more comprehensive and objective evaluation system needs to be established to better guide the research and application of feature extraction algorithms. SUMMARY
[0004] In view of the above analysis, the present application aims to disclose a visual feature adaptation method based on flight scene category driving; to realize the accurate comparison of the feature adaptation degree of different feature extraction algorithms in various flight scene categories, and to provide a scientific basis for feature selection in aircraft vision tasks.
[0005] The visual feature adaptation method based on flight scene category driving disclosed by the present application comprises:
[0006] S1, a labeled scene classification data set is established, the same number of images are prepared for each scene category, and each image is divided into multiple local blocks;
[0007] S2, construct a set of image feature extraction algorithms, wherein each image feature extraction algorithm is used to extract local features from each local block in an image;
[0008] S3, use each algorithm in the set of algorithms to extract features from images of each type of flight scene in the scene classification data set, respectively, then perform feature merging and dimensionality reduction to obtain representative features corresponding to each type of scene image; then perform image clustering, and cluster the representative features of all images under each algorithm into clusters corresponding to the number of scene categories;
[0009] S4, according to the clustering results and the actual scene classification labels of the images, calculate the feature adaptation degree index, which is used to represent the adaptation degree of the image feature extraction algorithm to process a certain type of scene image.
[0010] Further, in S3, comprising:
[0011] S301, pre-process the images with scene labels in the scene classification data set, and divide each image into n local blocks;
[0012] S302, determine whether all feature extraction algorithms are completed; if yes, exit the process; otherwise, select an unused image feature extraction algorithm from the algorithm set, and perform feature extraction and feature dimensionality reduction on the input images to obtain representative features of the images;
[0013] S303, determine whether the feature extraction of all images is completed; if not, continue to perform feature extraction and feature dimensionality reduction of the images to obtain representative features of the images; if yes, exit the feature extraction process and proceed to the next step;
[0014] S304, cluster the obtained representative features of all images, and the number of clusters is the same as the number of scene categories, and the representative feature clusters corresponding to the scene categories are obtained after clustering, which are used for feature adaptation degree calculation in step S4;
[0015] Return to S302, and execute the process in a loop until the representative features of all images obtained by each image feature extraction algorithm in the algorithm set are clustered, and then exit the process.
[0016] Further, in S302, comprising:
[0017] 1) The image feature extraction algorithm extracts k-dimensional local features from each local block in the image and merges them into a preliminary feature vector;
[0018] The preliminary feature vector is obtained by directly merging the extracted n×k local feature vectors; let the feature vector set extracted from the bth image block by the image feature extraction algorithm be:
[0019]
[0020] The preliminary feature vector P of the image is:
[0021]
[0022] 2) Dimensionality reduction is performed on the preliminary feature vector by using principal component analysis to obtain representative features;
[0023] The principal component analysis is performed on each preliminary feature vector P, and the first r principal components whose cumulative variance contribution rate is not less than a set value are retained The representative features after dimensionality reduction are obtained
[0024] Wherein, u1, u2, …, u r ∈{1,2,…,n},v1,v2,…,v r ∈{1,2,…,k};i is the image index, and c is the scene category.
[0025] Further, in S304, clustering is performed based on the K-means++ clustering algorithm, and all representative features of the images under each algorithm are clustered into clusters corresponding to the number of scene categories;
[0026] In the clustering process of the representative features of all images , the number of clusters is consistent with the number of scene categories D, and the similarity between features is measured by cosine similarity; finally, D clusters are obtained.
[0027] Wherein, the c′th cluster F c′ is represented as: m′ is the number of samples in the cluster F c′ .
[0028] Further, in S4, the calculation process of the feature extraction algorithm and the feature adaptation degree between the scenes includes:
[0029] S401, obtaining any one cluster after clustering of the representative features by the feature extraction algorithm, counting the distribution of the real scene categories of the corresponding images, obtaining the scene category with the highest proportion in the cluster as the dominant scene category of the cluster;
[0030] S402, one-to-one correspondence of the representative features in the cluster to the original images, judging whether the scene category to which the original image belongs is consistent with the dominant scene category of the cluster; if consistent, the original image is classified as an “in-class” image; if not consistent, the original image is classified as an “out-of-class” image;
[0031] S403, according to the classification result, the representative features in the cluster are divided into the “in-class” image representative feature set and the “out-of-class” image representative feature set;
[0032] S404, calculating the average intra-class similarity of all intra-class samples in the "intra-class" image representative feature set, and the average inter-class similarity of the intra-class samples and the inter-class samples in the "intra-class" image representative feature set and the "inter-class" image representative feature set;
[0033] S405, determining the feature adaptation degree of the cluster according to the average intra-class similarity and the average inter-class similarity;
[0034] S406, obtaining the feature adaptation degree of the feature extraction algorithm and the dominant scene category according to the feature adaptation degrees of the multiple clusters with the common dominant scene category;
[0035] S407, checking whether the dominant scene category set of all the clusters covers all the scene categories, if yes, completing the calculation of the feature adaptation degree between the image feature extraction algorithm and the scene; if no, checking the number of the scene categories not covered, if there are multiple, repeating the clustering and the operations S401-S407 using only the representative features of the images belonging to the scene categories not covered to obtain the feature adaptation degree between the image feature extraction algorithm and the scene; if there is only one, entering the operation S408;
[0036] S408, for the scene category not dominant in any cluster, calculating the ratio of the average intra-class similarity of all the image representative features belonging to the scene category and the average inter-class similarity with other cluster centers as the feature adaptation degree between the image feature extraction algorithm and the scene category.
[0037] Further, in S401, the determination process of the dominant scene category includes:
[0038] 1) for any one cluster F c′ obtained in S304, counting the distribution of the real scene categories of the images corresponding to the cluster;
[0039] The cluster contains m' samples; through counting, the number of the samples belonging to the scene category i∈{1,2,…,D} is
[0040] 2) calculating the proportion of the scene category i∈{1,2,…,D} in the cluster;
[0041] The proportion of the scene category i is:
[0042]
[0043] 3) from the proportions of the various scene categories, finding out the category with the highest proportion and marking it as the dominant scene category d of the cluster.
[0044] Further, in S405, the feature fitness Score of the cluster is determined according to the average intra-class similarity and the average inter-class similarity d is:
[0045]
[0046] wherein, is the average intra-class similarity of all intra-class samples in the intra-class image representative feature set; is the average inter-class similarity of intra-class samples and extra-class samples in the intra-class image representative feature set and the extra-class image representative feature set.
[0047] Further, the average intra-class similarity of all intra-class samples in the intra-class image representative feature set is:
[0048]
[0049] the average inter-class similarity of intra-class samples and extra-class samples in the intra-class image representative feature set and the extra-class image representative feature set is:
[0050]
[0051] represents the i, j-th sample in the intra-class image representative feature set represents the k, l-th sample in the extra-class image representative feature set is the number of samples in the intra-class image representative feature set.
[0052] Further, in S406, the plurality of clusters c'1, c'2, …, c'N are determined according to the feature fitness Score of each cluster. e If the dominant scene category of the i-th cluster is the scene category d, then the feature fitness of the feature extraction algorithm under the scene d is:
[0053]
[0054] wherein, is the feature fitness of the i-th cluster when the dominant scene category of the i-th cluster is the scene category d.
[0055] Further, in S408, if the uncovered scene category is only one scene category d',
[0056] the cluster F of representative features of the scene category d' d′ is:
[0057] Then, the average intra-class similarity of all representative features belonging to the scene category d' is:
[0058]
[0059] wherein, is the s, t-th sample in the cluster F d′ .
[0060] The clustering centers corresponding to other scene categories except the scene category d' are respectively:
[0061] {f 1 ,f 2 ,…,f d′-1 ,f d′+1 ,…,f D};
[0062] Then, the average inter-class similarity of the cluster center corresponding to the scene category d' and other cluster centers is:
[0063]
[0064] wherein, f r is the r-th cluster center in {f 1 ,f 2 ,…,f d′-1 ,f d′+1 ,…,f D};
[0065] Then, the feature adaptation degree of the image feature extraction algorithm and the scene category d' is:
[0066]
[0067] The visual feature adaptation method based on flight scene category driving disclosed in the present application has the following beneficial effects:
[0068] 1. In view of the limitations of the existing feature extraction algorithm evaluation index, the method of the present application can comprehensively consider the performance of the feature extraction algorithm under different flight scenes, avoiding the evaluation one-sidedness caused by only focusing on the performance of a single feature or a specific task, and providing a more comprehensive and objective evaluation system for the research and application of feature extraction algorithms.
[0069] 2. By accurately comparing the feature adaptation degrees of different feature extraction algorithms with various flight scene categories, the method of the present application can provide a scientific basis for feature selection in actual tasks such as aircraft visual navigation, scene understanding or target recognition, helping researchers and engineers to select the most suitable feature extraction algorithm according to the specific flight scene requirements, thereby optimizing the matching degree of the algorithm and the application scene and improving the accuracy and reliability of the task.
[0070] 3. By comparing and analyzing the feature representation capabilities under different flight scenarios, we can gain a deeper understanding of how each feature extraction algorithm performs when faced with different environmental features (such as spectral features, texture structure, and spatial scale differences in scenes like mountains, forests, and coastlines). This provides direction for the improvement and optimization of the algorithms, enabling them to better adapt to various complex flight scenarios and enhance their adaptability and robustness. Attached Figure Description
[0071] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.
[0072] Figure 1 This is a flowchart of the visual feature adaptation method based on flight scene category in an embodiment of the present invention.
[0073] Figure 2 This is an example diagram illustrating the division of an image into n local blocks in an embodiment of the present invention. Detailed Implementation
[0074] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and, together with the embodiments of the present invention, serve to illustrate the principles of the present invention.
[0075] One embodiment of the present invention discloses a visual feature adaptation method driven by flight scene categories, such as... Figure 1 As shown, it includes:
[0076] S1. Establish a labeled scene classification dataset. Prepare the same number of images for each scene category and divide each image into multiple local blocks.
[0077] S2. Construct a set of image feature extraction algorithms, where each image feature extraction algorithm is used to extract local features from each local block in the image;
[0078] S3. Use the algorithms in the algorithm set to extract features from the images of various flight scenarios in the scene classification dataset, then merge and reduce the dimensions to obtain representative features corresponding to each scene image; then perform image clustering to cluster the representative features of all images under each algorithm into clusters with the same number of scene categories.
[0079] S4. Based on the clustering results and the actual scene classification labels of the images, calculate the feature fit index to represent the degree of fit of the image feature extraction algorithm in processing images of a certain scene.
[0080] Further, the feature extraction algorithm is concentrated on the feature adaptation degree of each scene category under each scene category, and the feature adaptation degree of each scene category under each scene category is listed in a table, so that the feature adaptation of the feature extraction algorithm and the scene category can be intuitively represented, so that the optimal feature extraction algorithm for the scene category image can be accurately and conveniently determined.
[0081] Specifically, in the labeled scene classification data set of S1, the same number m of images is prepared for each scene category (a total of D scene categories, c∈{1, 2, …, D}), and each image is divided into n local blocks, as shown in Figure 2 .
[0082] In the data set in this step, each scene category contains the same number and size of images to ensure the fairness and accuracy of the subsequent comparison. In addition, the long side and the wide side of the image are evenly divided to form a grid, so that the whole image is evenly divided into n local blocks of equal size, so that the subsequent image feature extraction can as evenly as possible obtain features from all parts of the image.
[0083] Specifically, in S2, the feature extraction algorithm includes but is not limited to: SIFT (Scale-invariant feature transform), SURF (Speeded Up Robust Features), FAST (Features from Accelerated Segment Test), SuperPoint, etc.
[0084] Specifically, in S3, it includes:
[0085] S301, the images with scene labels in the scene classification data set are preprocessed, and each image is divided into n local blocks;
[0086] S302, determine whether all feature extraction algorithms are completed; if yes, exit the process; if no, select an unused image feature extraction algorithm from the algorithm set, and perform feature extraction and feature dimension reduction on the input image one by one to obtain the representative features of the image;
[0087] S303, determine whether the feature extraction of all images is completed; if no, continue to perform feature extraction and feature dimension reduction on the image to obtain the representative features of the image; if yes, exit the feature extraction process and enter the next step;
[0088] S304, cluster all the representative features of the obtained images, and the number of clusters is the same as the number of scene categories. After clustering, the representative feature clusters corresponding to the scene categories are obtained, which are used for feature adaptation degree calculation in step S4;
[0089] Return to S302, and the process is executed in a loop until all representative features of images obtained by each image feature extraction algorithm in the algorithm set are clustered, and then the process is exited.
[0090] Specifically, in S302, the following steps are included:
[0091] 1) The image feature extraction algorithm extracts k-dimensional local features from each local block in the image and combines them into a preliminary feature vector;
[0092] The preliminary feature vector is obtained by directly combining the extracted n x k local feature vectors; let the feature vector set extracted by the image feature extraction algorithm from the bth image block be:
[0093]
[0094] The preliminary feature vector P of the image is:
[0095]
[0096] 2) Use principal component analysis to reduce the dimension of the preliminary feature vector and obtain representative features;
[0097] Perform principal component analysis on each preliminary feature vector P, and retain the first r principal components whose cumulative variance contribution rate is not less than a set value Obtain the reduced representative features
[0098] where u1, u2, …, u r ∈{1,2,…,n},v1,v2,…,v r ∈{1,2,…,k}; i is the image index, and c is the scene category; Preferably, the set value is 90%.
[0099] This step improves the robustness of the features by eliminating redundant information, while reducing the computational complexity of subsequent clustering.
[0100] Specifically, the representative features of all images obtained after S303 are
[0101] Specifically, in S304, the K-means++ clustering algorithm is used for clustering, and all representative features of images under each algorithm are clustered into clusters corresponding to the number of scene categories;
[0102] In the clustering process of all representative features of images , the number of clusters is consistent with the number of scene categories D, and the similarity between features is measured by cosine similarity; finally, D clusters are obtained.
[0103] where the c′th cluster Fc′ is represented as: m' is the number of samples in cluster F c′ .
[0104] In the specific clustering process, the upper limit of the number of iterations is set to 500 times, and the convergence threshold is set to 0.0001, to balance the calculation efficiency and clustering stability.
[0105] Specifically, in S4, the calculation process of the feature adaptation degree between the feature extraction algorithm and the scene includes:
[0106] S401, obtaining any one cluster after representative feature clustering by the feature extraction algorithm, counting the real scene category distribution of the corresponding image, obtaining the scene category with the highest proportion in the cluster as the dominant scene category of the cluster;
[0107] S402, one-to-one corresponding the representative features in the cluster to the original image, judging whether the scene category to which the original image belongs is consistent with the dominant scene category of the cluster; if consistent, the original image is classified as an "in-class" image; if not consistent, the original image is classified as an "out-of-class" image;
[0108] S403, according to the classification result, dividing the representative features in the cluster into an "in-class" image representative feature set and an "out-of-class" image representative feature set;
[0109] S404, calculating the average intra-class similarity of all intra-class samples in the "in-class" image representative feature set, and the average inter-class similarity between the intra-class samples and the out-of-class samples in the "in-class" image representative feature set and the "out-of-class" image representative feature set;
[0110] S405, determining the feature adaptation degree of the cluster according to the average intra-class similarity and the average inter-class similarity;
[0111] S406, obtaining the feature adaptation degree of the feature extraction algorithm and the dominant scene category according to the feature adaptation degrees of multiple clusters with the same dominant scene category;
[0112] S407, checking whether the dominant scene category set of all clusters covers all scene categories, yes, completing the calculation of the feature adaptation degree between the image feature extraction algorithm and the scene; no, then investigating the number of scene categories that are not covered, if there are multiple, only using the representative features of the images belonging to the un-covered scene categories to repeat the clustering and S401-S407 operations to obtain the feature adaptation degree between the image feature extraction algorithm and the scene; if there is only one, entering operation S408;
[0113] S408, for the scene category which is not dominant in any cluster, calculate the ratio of the average intra-class similarity of all image representative features belonging to the scene category and the average inter-class similarity with other cluster centers as the feature adaptation degree between the image feature extraction algorithm and the scene category.
[0114] Specifically, in S401, the determination process of the dominant scene category includes:
[0115] 1) For any one cluster F c′ obtained in S304, count the real scene category distribution of the corresponding image;
[0116] The cluster F contains m' samples; through statistics, the number of samples belonging to the scene category i∈{1,2,…,D} is
[0117] 2) Calculate the proportion of the scene category i∈{1,2,…,D} in the cluster;
[0118] The proportion of the scene category i is
[0119]
[0120] 3) From the proportion of each scene category, find the category with the highest proportion and mark it as the dominant scene category d of the cluster.
[0121] After S402, examine the scene category c c′ of the image feature corresponding to the original image in F c′ , the image conforming to the dominant scene category of the cluster and the image not conforming to the dominant scene category of the cluster are respectively regarded as "in-class" and "out-of-class" images;
[0122] To S403, according to the classification result, the representative features in the cluster are divided into "in-class" image representative feature set and "out-of-class" image representative feature set
[0123] In S404,
[0124] The average intra-class similarity of all intra-class samples in the "in-class" image representative feature set is:
[0125]
[0126] The average inter-class similarity of intra-class samples and out-of-class samples in the "in-class" image representative feature set and the "out-of-class" image representative feature set is:
[0127]
[0128] the representative feature set of the “in-class” images is the i, j-th sample in the representative feature set of the “in-class” images; the representative feature set of the “out-of-class” images is is the k, l-th sample in the representative feature set of the “out-of-class” images; is the number of samples in the representative feature set of the “in-class” images.
[0129] In S405, the feature fitness Score of the cluster is determined according to the average intra-class similarity and the average inter-class similarity d is:
[0130]
[0131] wherein, is the average intra-class similarity of all intra-class samples in the representative feature set of the “in-class” images; is the average inter-class similarity of the intra-class samples and the out-of-class samples in the representative feature set of the “in-class” images and the representative feature set of the “out-of-class” images.
[0132] The feature fitness Score d The smaller the feature fitness Score is, the more compact the intra-class samples are and the higher the inter-class discrimination is, and the better the feature expression ability is.
[0133] In S406, the plurality of clusters c'1, c'2, …, c'N are clustered according to the feature fitness Score e If the dominant scene category of the cluster is the scene category d, then the feature fitness of the feature extraction algorithm under the scene category d is:
[0134]
[0135] wherein, is the feature fitness of the i-th cluster when the dominant scene category of the cluster is the scene category d.
[0136] In S408, if the uncovered scene category is only one scene category d',
[0137] The cluster F of the representative features of the scene category d' d′ is:
[0138] Then, the average intra-class similarity of all the representative features belonging to the scene category d' is:
[0139]
[0140] wherein, is the cluster Fd′ The s-th and t-th samples in the dataset;
[0141] The cluster centers for the scene categories other than scene category d′ are as follows:
[0142] {f 1 ,f 2 ,…,f′ d-1 ,f′ d+1 ,…,f D}
[0143] Then, the average inter-class similarity between the cluster center corresponding to scene category d′ and other cluster centers is:
[0144]
[0145] Among them, f r For {f 1 ,f 2 ,…,f′ d-1 ,f′ d+1 ,…,f D The r-th cluster center in};
[0146] Therefore, the feature fit between the image feature extraction algorithm and scene category d′ is:
[0147]
[0148] Finally, the metrics of various feature extraction algorithms in different scenarios are listed in a table to compare feature adaptability.
[0149] Furthermore, when the cluster centers corresponding to other scene categories are {f 1 ,f 2 ,…,f d′-1 ,f d′+1 ,…,f D When a cluster is obtained from multiple dominant scene categories that share the same scene as a cluster, the cluster center is determined by the weighted sum of the cluster centers of the multiple clusters.
[0150] For example, {f 1 ,f 2 ,…,f d′-1 ,f d′+1 ,…,f D A center f in} d The f d Multiple clusters {c′1,…,c′} of the dominant scene category, each clustered around the same scene d. e}get;
[0151] but, in, a cluster center of the cluster {c′1, …, c′ e} of images, a number of "in-cluster" images of the cluster c′ i
[0152] In summary, the flight scene category-driven visual feature adaptation method disclosed in the embodiments of the present application considers the performance of feature extraction algorithms in different flight scenes, avoids the one-sidedness of evaluation caused by focusing only on single features or specific task performance, and provides a more comprehensive and objective evaluation system for the research and application of feature extraction algorithms.
[0153] By accurately comparing the feature adaptation degrees of different feature extraction algorithms with various flight scene categories, this method can provide a scientific basis for feature selection in actual tasks such as aircraft visual navigation, scene understanding, or target recognition, helping researchers and engineers to select the most suitable feature extraction algorithm according to specific flight scene requirements, thereby optimizing the matching degree of algorithms and application scenarios and improving the accuracy and reliability of tasks.
[0154] Furthermore, through comparative analysis of feature expression capabilities in different flight scenes, the performance of each feature extraction algorithm when facing different environmental features (such as spectral features, texture structures, and spatial scale differences of scenes such as mountains, forests, and coastlines) can be deeply understood, thereby providing a direction for algorithm improvement and optimization, enabling it to better adapt to various complex flight scenes and enhancing the adaptability and robustness of the algorithm.
[0155] The above description is only a preferred specific embodiment of the present application, but the protection scope of the present application is not limited thereto, and any changes or replacements within the technical scope disclosed by the present application can be easily thought of by those skilled in the art, which should be covered within the protection scope of the present application.
Claims
1. A method for driving visual feature adaptation based on flight scene category, characterized in that, The method comprises the following steps: S1, a labeled scene classification dataset is established, the same number of images are prepared for each scene category, and each image is divided into multiple local blocks; S2, an image feature extraction algorithm set is constructed, wherein each image feature extraction algorithm is used to extract local features from each local block in the image; S3, each algorithm in the algorithm set is used to extract features from the images of each flight scene category in the scene classification dataset one by one, and then the features are combined and reduced in dimension to obtain representative features corresponding to the images of each scene category; Then, image clustering is performed, and the representative features of all images under each algorithm are clustered into clusters with the same number of scene categories through clustering; S4, according to the clustering results and the actual scene classification labels of the images, an index of feature adaptation degree is calculated, which is used to represent the adaptation degree of the image feature extraction algorithm to process the scene image.
2. The method of claim 1, wherein the method is implemented by a device. In S3, the following steps are included: S301, the images with scene labels in the scene classification dataset are preprocessed, and each image is divided into n local blocks; S302, it is judged whether all feature extraction algorithms are completed; if yes, the process is exited; if not, an unused image feature extraction algorithm is selected from the algorithm set, and the feature extraction and feature reduction of the input image are performed to obtain the representative features of the image; S303, it is judged whether the feature extraction of all images is completed; if not, the feature extraction and feature reduction of the image are continued to obtain the representative features of the image; if yes, the feature extraction process is exited, and the next step is entered; S304, the representative features of all images obtained are clustered, the number of clusters is the same as the number of scene categories, and the representative feature clusters corresponding to the scene categories are obtained after clustering, which are used for feature adaptation degree calculation in step S4; Return to S302, and execute the process in a loop until the representative features of all images obtained by each image feature extraction algorithm in the algorithm set are clustered, and then the process is exited.
3. The method of claim 2, wherein the method further comprises: In S302, the following steps are included: 1) the image feature extraction algorithm extracts k-dimensional local features from each local block in the image and combines them into a preliminary feature vector; The preliminary feature vector is obtained by directly combining the extracted n×k local feature vectors; let the feature vector set extracted from the bth image block by the image feature extraction algorithm be: The preliminary feature vector P of the image is: 2) the principal component analysis method is used to reduce the dimension of the preliminary feature vector to obtain the representative feature; Carrying out principal component analysis on each preliminary feature vector P, and retaining the first r principal components whose cumulative variance contribution rate is not less than a set value Obtaining the representative features after dimension reduction where u1, u2,..., u r ∈ {1, 2,..., n}, v1, v2,..., v r ∈ {1, 2,..., k}; i is the image index and c is the scene category.
4. The method of claim 2, wherein the method further comprises: In S304, the K-means++ clustering algorithm is used for clustering, and the representative features of all images under each algorithm are clustered into clusters with the same number of scene categories; Representative features of all images In the clustering process, the number of clusters is consistent with the number of scene categories D, and the similarity between features is measured by cosine similarity; finally, D clusters are obtained. wherein the c'th cluster F c′ is represented as: m' is the number of samples in the cluster F c′ .
5. The method of claim 2, wherein the visual feature adaptation is driven based on a flight scene category. In S4, the calculation process of the feature adaptation degree between the feature extraction algorithm and the scene comprises the following steps: S401, any one cluster after the representative feature clustering by the feature extraction algorithm is obtained, the real scene category distribution of the corresponding image is counted, the scene category with the highest proportion in the cluster is obtained as the dominant scene category of the cluster; S402, one-to-one correspondence of representative features in the cluster to the original image, determine whether the original image belongs to the scene category consistent with the dominant scene category of the cluster; if consistent, the original image is classified as "in-class" image; if not consistent, the original image is classified as "out-of-class" image; S403, according to the classification result, the representative features in the cluster are divided into "in-class" image representative feature set and "out-of-class" image representative feature set; S404, calculate the average intra-class similarity of all intra-class samples in the "in-class" image representative feature set, and the average inter-class similarity of intra-class samples and out-of-class samples in the "in-class" image representative feature set and the "out-of-class" image representative feature set; S405, determine the feature adaptation degree of the cluster according to the average intra-class similarity and the average inter-class similarity; S406, obtain the feature adaptation degree of the feature extraction algorithm and the dominant scene category according to the feature adaptation degrees of multiple clusters with the same dominant scene category; S407, check whether the dominant scene category set of the current all clusters covers all scene categories, yes, complete the calculation of the feature adaptation degree between the image feature extraction algorithm and the scene; no, investigate the number of scene categories that are not covered, if there are multiple, only use the representative features of the images belonging to the un-covered scene categories to repeat the clustering and S401-S407 operations to obtain the feature adaptation degree between the image feature extraction algorithm and the scene; if there is only one, enter operation S408; S408, for the scene category which does not occupy the dominant in any cluster, calculate the ratio of the average intra-class similarity of all image representative features belonging to the scene category to the average inter-class similarity with other cluster centers as the feature adaptation degree between the image feature extraction algorithm and the scene category.
6. The method of claim 5, wherein the visual feature adaptation is driven based on a flight scene category. In S401, the determination process of the dominant scene category includes: 1) For any one cluster F obtained in S304 c′ , count the real scene category distribution of the corresponding image; clusters contains m' samples; by statistics, the number of samples belonging to the scene category i e {1,2,...,D} is 2) calculate the proportion of scene category i∈{1,2,…,D} in the cluster; Proportion of scene category i Is: 3) find the highest proportion from the proportion of each scene category, and mark it as the dominant scene category d of the cluster.
7. The method of claim 5, wherein the method further comprises: In S405, the feature fitness Score of the cluster is determined according to the average intra-class similarity and the average inter-class similarity d is: wherein, is the average intra-class similarity of all intra-class samples in the intra-class image representative feature set; is the average inter-class similarity of intra-class samples and extra-class samples in the intra-class image representative feature set and the extra-class image representative feature set.
8. The method of claim 7, wherein the method further comprises: "Within-class" image representative feature set is the average within-class similarity of all within-class samples is: Intra-class image representative feature set and out-of-class image representative feature set is: representative set of features of "in-class" images the i,jth sample in the set; representative set of features of "out-of-class" images is the k, lth sample in the set; is the number of samples in the representative set of features of "in-class" images.
9. The method of claim 8, wherein, In S406, multiple clusters c1′, c2′,…, c′ have the same scene d category as the dominant scene category of the cluster, then the feature adaptation degree of the corresponding feature extraction algorithm under the scene d is: wherein, is the feature fitness of the u-th cluster with the dominant scene category of the cluster being the scene d category.
10. The method of claim 5, wherein the method further comprises: In S408, if the un-covered scene category has only one scene category d′, a cluster F of representative features of the scene class d' d′ is: Then the average intra-class similarity of all representative features belonging to the scene category d′ is: wherein is the s, tth sample in cluster F d′ is the s, tth sample in cluster F The clustering centers corresponding to other scene categories excluding the scene category d′ are respectively: {f 1 ,f 2 ,…,f d′-1 ,f d′+1 ,…,f D} Then the average inter-class similarity between the cluster center corresponding to the scene category d′ and other cluster centers is: wherein f r is the rth cluster center in {f 1 ,f 2 ,…,f d′-1 ,f d′+1 ,…,f D} Then the feature adaptation degree of the image feature extraction algorithm and the scene category d′ is: