Image target recognition method and system based on feature recognition
By combining VAE, FCN and GAT hybrid neural network model, the problem of insufficient accuracy and robustness of traditional object detection methods in complex environments is solved, and more accurate and robust object detection is achieved, suitable for a variety of dynamics Complex environment.
Patent Information
- Application Number
- CN202410366821.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-28
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-03-28
Smart Images

Figure CN118196581B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of computer image recognition and processing, and in particular to an image target recognition method and system based on feature recognition. Background Art
[0002] In the field of modern image processing and computer vision, efficient and accurate target detection and recognition, especially in dynamic and complex environments, has always been a challenging technical problem. Although traditional target detection methods, such as feature engineering-based methods or early deep learning models, have achieved success to a certain extent, they are usually difficult to handle complex changes in images, such as illumination, occlusion, target scale changes, etc., which limits the accuracy and robustness of target detection. In addition, these methods often lack a deep understanding of the relationship between targets in the image and cannot make full use of the contextual information in the image to optimize the detection results. Therefore, how to effectively fuse and utilize image features to improve the performance of target detection has become a technical problem that needs to be solved urgently.
[0003] This solution aims to solve the above technical problems by introducing a hybrid neural network model that combines variational autoencoders (VAE), fully convolutional networks (FCN) and graph attention networks (GAT). The core of this solution is to use VAE to learn the deep potential representation of images, use FCN for accurate pixel-level target recognition, and dynamically fuse these features and optimize the weight distribution between features through GAT. This innovative feature fusion method can better capture the key information in the image and suppress irrelevant background, thereby achieving more accurate and robust target detection in complex environments. Summary of the invention
[0004] In view of the problems existing in the above-mentioned existing image target recognition methods based on feature recognition, the present invention is proposed.
[0005] Therefore, the problem to be solved by the present invention is that although traditional target detection methods, such as feature engineering-based methods or early deep learning models, have achieved success to a certain extent, they are usually difficult to handle complex changes in images, such as illumination, occlusion, target scale changes, etc., which limits the accuracy and robustness of target detection. In addition, these methods often lack a deep understanding of the relationship between targets in the image and cannot make full use of the contextual information in the image to optimize the detection results.
[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: an image target recognition method based on feature recognition, comprising:
[0007] Collect multi-source image data, perform preliminary image processing, and input them into the adaptive feature fusion model;
[0008] Using adaptive feature fusion model to enhance key features and suppress irrelevant information;
[0009] The enhanced key features are identified to obtain the image target.
[0010] As a preferred solution of the image target recognition method based on feature recognition described in the present invention, wherein: the collection of multi-source image data includes public data sets, lighting changes and target categories, and the preliminary processing includes data cleaning, data labeling, data enhancement and feature extraction.
[0011] As a preferred solution of the image target recognition method based on feature recognition described in the present invention, the data cleaning adopts an automated script to identify and delete duplicate, blurred or damaged image files, the data annotation adopts a semi-automated tool to assist in image annotation to annotate the category and position of the object, the data enhancement utilizes synthetic images generated by a generative adversarial network to enrich the data set, and the feature extraction selects a hybrid model based on a variational autoencoder and a convolutional neural network.
[0012] As a preferred solution of the image target recognition method based on feature recognition of the present invention, the adaptive feature fusion model combines VAE, FCN and GAT, and introduces an adaptive learning rate adjustment mechanism based on performance feedback, which is expressed as follows:
[0013]
[0014] Among them, λ i For each input x i The adaptive learning rate, γ is the parameter that controls the learning rate adjustment amplitude, x is the input image, D i is the derivative of the loss function for each sample i, GAT is the graph attention network, VAE is the variational autoencoder, and FCN is the fully convolutional network;
[0015] The formula of GAT is:
[0016] GAT(V,E)=∑ j∈N(i) α ij (W g v j )
[0017] Among them, V and E represent the nodes and edges in the graph respectively, N(i) represents the set of neighbor nodes of node i, and α ij is the attention coefficient, which indicates the importance of node i to its neighbor node j, W g is the learnable weight matrix, v j is the feature vector of node j;
[0018] The formula of VAE is expressed as:
[0019] VAE(x)=μ+σ⊙∈
[0020] Where μ and σ are the mean and standard deviation of the encoder output, ∈ is the noise sampled from a standard normal distribution, and ⊙ represents element-wise multiplication;
[0021] The formula of FCN is expressed as:
[0022] FCN(x)=ReLU(W f *x+b f )
[0023] Among them, W f is the convolution kernel weight, b f is the bias term, ReLU is the nonlinear activation function, and * represents the convolution operation.
[0024] As a preferred solution of the image target recognition method based on feature recognition described in the present invention, the enhanced key features are: using VAE to extract feature representation from the output image, and then using FCN to perform pixel-level target recognition and semantic segmentation on the features, and using GAT to dynamically fuse the features extracted by VAE and FCN.
[0025] As a preferred solution of the image target recognition method based on feature recognition described in the present invention, the suppression of irrelevant information is to apply NMS to further clean and refine the feature fusion results, remove overlapping detection frames, retain the detection frames representing the real targets, sort the results after NMS processing according to confidence, and remove the detection results with low confidence again, and use additional classifiers or rules for the remaining detection results, and perform final verification and adjustment based on the global information of the image.
[0026] As a preferred solution of the image target recognition method based on feature recognition described in the present invention, wherein: the identification of enhanced key features includes inputting the enhanced key feature vector into the ResNet network, identifying and classifying the feature vector, and finally outputting the category of the target and its corresponding confidence.
[0027] Another object of the present invention is to provide an image target recognition system based on feature recognition, comprising a feature extraction module, an image processing module and a target recognition module;
[0028] The feature extraction module is used to extract image characteristics from the input image;
[0029] The image processing module is used to integrate visual features, semantic features and image complexity to generate fusion features suitable for target recognition;
[0030] The object recognition module is used to recognize and classify objects in images.
[0031] A computer device comprises: a memory and a processor; the memory stores a computer program, and the processor implements the steps of an image target recognition method based on feature recognition when executing the computer program.
[0032] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of an image target recognition method based on feature recognition.
[0033] The beneficial effects of the present invention are as follows: the present invention significantly improves the performance of target detection in dynamic and complex scenes. First, through the dynamic feature fusion mechanism, the model can adaptively adjust the feature fusion strategy, effectively deal with problems such as occlusion and illumination changes in the image, and improve the accuracy of detection. Secondly, combined with the graph attention mechanism of GAT, the relationship between features is optimized, so that the model can better utilize the contextual information in the image and enhance the understanding of the interaction between targets, further improving the accuracy and robustness of target detection. Finally, the scheme has high versatility and flexibility, and is suitable for target detection tasks in a variety of dynamic and complex environments, providing new research directions and application possibilities for the future development of image processing technology. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.
[0035] Figure 1 Schematic diagram of the flow of the image target recognition method based on feature recognition.
[0036] Figure 2 Schematic diagram of the structure of the image target recognition method based on feature recognition. DETAILED DESCRIPTION
[0037] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings.
[0038] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.
[0039] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor is it a separate or selective embodiment that is mutually exclusive with other embodiments.
[0040] Example 1
[0041] Reference Figure 1 , which is the first embodiment of the present invention, and which provides an image target recognition method based on feature recognition. The image target recognition method based on feature recognition includes:
[0042] S1: Collect multi-source image data, perform preliminary image processing, and input them into the adaptive feature fusion model;
[0043] Specifically, multi-source image data including public data sets, illumination changes and target categories are collected, and the preliminary processing includes data cleaning, data annotation, data enhancement and feature extraction to ensure data diversity, covering different environmental conditions, illumination changes and target categories;
[0044] It should be noted that data cleaning uses automated scripts to identify and delete duplicate, blurred or damaged image files. The data annotation uses semi-automated tools to assist in image annotation, annotating the category and location of the object. The data enhancement uses synthetic images generated by generative adversarial networks to enrich the data set. The feature extraction selects a hybrid model based on variational autoencoders and convolutional neural networks to achieve complex scenes. A professional annotation team is introduced to ensure the accuracy and consistency of annotations. Data enhancement uses a variety of data enhancement techniques (such as rotation, scaling, color transformation) to increase the generalization ability of the model.
[0045] It should also be noted that the image data source is traffic flow images collected by the urban traffic monitoring system in multiple time periods and weather conditions. These images contain different types of vehicles, pedestrians, traffic signs, etc., captured under different lighting and weather conditions, providing rich scene changes and target diversity. Timestamps and environmental condition labels such as "day / night" and "sunny / rainy" are attached to each image. These labels provide additional contextual information, which helps the model learn feature extraction and fusion strategies under different environmental conditions. Professional image annotation tools are used to accurately annotate bounding boxes and classify targets such as vehicles, pedestrians, and traffic signs in the image.
[0046] S2: Use adaptive feature fusion model to enhance key features and suppress irrelevant information;
[0047] Specifically, the adaptive feature fusion model combines VAE, FCN and GAT, and introduces an adaptive learning rate adjustment mechanism based on performance feedback. The formula is expressed as follows:
[0048]
[0049] Among them, λ i For each input x i The adaptive learning rate, γ is the parameter that controls the learning rate adjustment amplitude, x is the input image, D i is the derivative of the loss function for each sample i, GAT is the graph attention network, VAE is the variational autoencoder, and FCN is the fully convolutional network;
[0050] The formula of GAT is:
[0051] GAT(V,E)=∑ j∈N(i) α ij (W g v j )
[0052] Among them, V and E represent the nodes and edges in the graph respectively, N(i) represents the set of neighbor nodes of node i, and α ij is the attention coefficient, which indicates the importance of node i to its neighbor node j, W g is the learnable weight matrix, v j is the feature vector of node j;
[0053] The formula of VAE is expressed as:
[0054] VAE(x)=μ+σ⊙∈
[0055] Where μ and σ are the mean and standard deviation of the encoder output, ∈ is the noise sampled from a standard normal distribution, and ⊙ represents element-wise multiplication;
[0056] The formula of FCN is expressed as:
[0057] FCN(x)=ReLU(W f *x+b f )
[0058] Among them, W f is the convolution kernel weight, b f is the bias term, ReLU is the nonlinear activation function, and * represents the convolution operation;
[0059] It should be noted that the VAE part is used to learn and encode the deep potential representation of image data. It compresses the input image into a latent space vector through the encoder network, and then tries to reconstruct the input image through the decoder network. During this period, the model is able to perform gradient descent training through the re-parameterization technique. This process helps the model capture the key features in the image and learn the probability distribution of image data. The FCN part focuses on pixel-level image understanding and is mainly used for semantic segmentation tasks. It uses a series of convolutional layers and upsampling layers to determine the category to which each pixel belongs, which is very useful for understanding the shape and position of each target in the image. The GAT part uses the attention mechanism to dynamically weight features to highlight important features and suppress irrelevant information. This process takes into account the relationship between different areas in the image. , by learning the "attention" of each area (node) to other areas, the weight of feature fusion is dynamically adjusted. The variational autoencoder is based on the existing traffic sign recognition dataset and trains an FCN model to perform pixel-level traffic sign recognition and semantic segmentation tasks, which enables the model to understand the key traffic instructions in the image. The graph attention network uses GAT to process features from VAE and FCN, and introduces the spatial relationship of objects in the image and environmental condition labels as additional inputs. In this way, GAT can dynamically adjust the weight of feature fusion and optimize feature representation based on the interaction between objects and contextual information. The above formula combines the features extracted by VAE and FCN, and dynamically adjusts the fusion weights of these features through GAT, while optimizing the model training process using an adaptive learning rate mechanism.
[0060] Furthermore, the key feature of the enhancement is to use VAE to extract feature representation from the output image, then use FCN to perform pixel-level target recognition and semantic segmentation on the features, and use GAT to dynamically fuse the features extracted by VAE and FCN.
[0061] By using VAE to extract feature representations from the output image and then applying FCN to perform pixel-level target recognition and semantic segmentation, we can achieve in-depth mining and refined understanding of the potential information in the image. The application of VAE enables the model to capture the deep features and global structure of the image, while the pixel-level recognition and segmentation of FCN further accurately identify the specific targets in the image, enhancing the ability to recognize the target, so that subsequent processing steps can be performed on the basis of more accurate features, improving the accuracy of target recognition and the ability to capture details, and providing a solid foundation for accurately identifying and understanding image content.
[0062] Furthermore, irrelevant information is suppressed by applying NMS to further clean and refine the feature fusion results, remove overlapping detection boxes, retain the detection boxes representing the real targets, sort the results after NMS processing according to confidence, and remove the detection results with low confidence again. For the remaining detection results, additional classifiers or rules are used to perform final verification and adjustment based on the global information of the image.
[0063] By applying NMS to further clean and refine the feature fusion results, the overlapping detection frames are effectively removed, and the detection frames representing the real targets are retained. The results after NMS processing are sorted by confidence, and the detection results with low confidence are further removed, which suppresses irrelevant information in the image, reduces the occurrence of false positives, improves the accuracy of detection, ensures the high credibility of the recognition results, and reduces the burden of subsequent processing.
[0064] S3, identify the enhanced key features and obtain the image target;
[0065] Specifically, identifying the enhanced key features includes inputting the enhanced key feature vector into the ResNet network, identifying and classifying the feature vector, and finally outputting the target category and its corresponding confidence.
[0066] The enhanced key feature vector is input into the ResNet network for identification and classification, and the final output includes the category of the target and its corresponding confidence. The deep network structure and residual learning mechanism of ResNet are utilized to effectively identify and classify the targets in the image, and the powerful ability of the deep learning model is used for accurate classification. At the same time, the residual connection helps to avoid the gradient vanishing problem in deep network training, improves the efficiency of model training and the accuracy of classification. It should also be noted that at the data level, the real-time images of the traffic monitoring system and the precise environment and time labels are combined to provide rich contextual information and target diversity. At the model level, by combining VAE, FCN and GAT, and introducing an adaptive learning rate adjustment mechanism based on performance feedback, the model can better understand and utilize the complex features and contextual information in the image.
[0067] Example 2
[0068] Reference Figure 2 , which is the second embodiment of the present invention, and which is different from the previous embodiment, provides an image target recognition system based on feature recognition, including a feature extraction module, an image processing module and a target recognition module;
[0069] The feature extraction module is used to extract image characteristics from the input image;
[0070] The image processing module is used to integrate visual features, semantic features and image complexity to generate fusion features suitable for target recognition;
[0071] The object recognition module is used to recognize and classify objects in images.
[0072] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program codes.
[0073] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.
[0074] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.
[0075] It should be understood that the various parts of the present invention can be implemented with hardware, software, firmware or a combination thereof. In the above embodiments, a plurality of steps or methods can be implemented with software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented with hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, an application specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
[0076] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.
Claims
1. A method for image target recognition based on feature recognition, characterized in that: include, Collect multi-source image data, perform preliminary image processing, and input them into the adaptive feature fusion model; Using adaptive feature fusion model to enhance key features and suppress irrelevant information; Identify the enhanced key features and obtain the image target; The collecting of multi-source image data includes public data sets, illumination changes and target categories, and the preliminary processing includes data cleaning, data annotation, data enhancement and feature extraction; The data cleaning refers to the use of automated scripts to identify and delete duplicate, blurred or damaged image files; the data annotation refers to the use of semi-automated tools to assist in image annotation, annotating the category and location of objects; the data enhancement refers to enriching the data set with synthetic images generated by generative adversarial networks; the feature extraction refers to the selection of a hybrid model based on variational autoencoders and convolutional neural networks; The adaptive feature fusion model combines VAE, FCN and GAT, and introduces an adaptive learning rate adjustment mechanism based on performance feedback, which is expressed as follows: Among them, λ i For each input x i The adaptive learning rate, γ is the parameter that controls the learning rate adjustment amplitude, x is the input image, D i is the derivative of the loss function for each sample i, GAT is the graph attention network, VAE is the variational autoencoder, and FCN is the fully convolutional network; The formula of GAT is: GAT(V,E)(∑). j∈N(i) α ij (W g v j ) Among them, V and E represent the nodes and edges in the graph respectively, N(i) represents the set of neighbor nodes of node i, and α ij is the attention coefficient, which indicates the importance of node i to its neighbor node j, W g is the learnable weight matrix, v j is the feature vector of node j; The formula of VAE is expressed as: VAE(x)=μ+σ⊙∈ Where μ and σ are the mean and standard deviation of the encoder output, ∈ is the noise sampled from a standard normal distribution, and ⊙ represents element-wise multiplication; The formula of FCN is expressed as: FCN(x)=ReLU(W f *x+b f ) Among them, W f is the convolution kernel weight, b f is the bias term, ReLU is the nonlinear activation function, and * represents the convolution operation.
2. The image target recognition method based on feature recognition as claimed in claim 1, characterized in that: The enhanced key feature is to use VAE to extract feature representation from the output image, then use FCN to perform pixel-level target recognition and semantic segmentation on the features, and use GAT to dynamically fuse the features extracted by VAE and FCN.
3. The image target recognition method based on feature recognition as claimed in claim 2, characterized in that: The suppression of irrelevant information is to apply NMS to further clean and refine the feature fusion results, remove overlapping detection frames, retain the detection frames representing the real targets, sort the results after NMS processing according to confidence, and remove the detection results with low confidence again. For the remaining detection results, additional classifiers or rules are used to perform final verification and adjustment based on the global information of the image.
4. The image target recognition method based on feature recognition as claimed in claim 3, characterized in that: The identifying of the enhanced key features includes inputting the enhanced key feature vector into the ResNet network, identifying and classifying the feature vector, and finally outputting the category of the target and its corresponding confidence.
5. A feature-based image target recognition system based on the feature-based image target recognition method according to any one of claims 1 to 4, characterized in that: Including, feature extraction module, image processing module and target recognition module; The feature extraction module is used to extract image characteristics from the input image; The image processing module is used to integrate visual features, semantic features and image complexity to generate fusion features suitable for target recognition; The object recognition module is used to recognize and classify objects in images.
6. A computer device comprising: A memory and a processor; the memory stores a computer program, wherein the processor implements the steps of the image target recognition method based on feature recognition as described in any one of claims 1 to 4 when executing the computer program.
7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the image object recognition method based on feature recognition described in any one of claims 1 to 4 are implemented.
Citation Information
Patent Citations
Roof type identification method and device based on multi-source geographic information and remote sensing image
CN117746247A
