Multi-granularity low-quality image target identification method and system

Abstract

The invention belongs to the technical field of target recognition, discloses a multi-granularity low-quality image target recognition method and system, and improves target detection and recognition performance through a multi-granularity feature fusion module and a knowledge transmission module. And the multi-granularity feature fusion module is combined with feature maps of different scales to provide richer and more accurate feature representation, so that the accuracy, robustness and generalization ability of target detection are enhanced. In addition, the knowledge transmission module forms global knowledge of class prototype level and probability level granularity by cooperating with data of a plurality of clients, and the target classification and recognition capability of the local model is further improved. The method is not only suitable for target detection, but also capable of improving the accuracy and efficiency of image segmentation (including semantic segmentation and instance segmentation), processing illumination and posture changes at the same time, and remarkably improving the recognition precision.

Description

Technical Field

[0001] The present invention belongs to the technical field of target recognition, and in particular relates to a multi-granularity low-quality image target recognition method and system. Background Art

[0002] Image target recognition, also known as computer vision or AI image recognition, is an important research direction in the field of artificial intelligence. Its background mainly stems from the human need for computers to understand and recognize images like humans. With the rise of deep learning technology, image recognition has made great progress in the past few years and has become one of the hot topics in the field of artificial intelligence. Early image recognition methods were mainly based on hand-designed feature extraction and traditional machine learning algorithms, but these methods are often limited by the expressiveness and generalization capabilities of features. In addition, traditional image recognition methods are trained based on local private data, and data between institutions is difficult to share due to privacy protection issues, which limits recognition performance. Image target recognition is to recognize images by comparing stored information (information stored in memory) with current information (information entering the senses at that time). This process involves image description, that is, using numbers or symbols to represent the relevant features of each target in the image or scene, and even the relationship between targets. The final result is an abstract expression of target features and the relationship between them; however, the existing target recognition technology has low accuracy in target detection; cannot adapt to different scenes; is inaccurate in image segmentation; and is inaccurate in target recognition.

[0003] Image object recognition, also known as computer vision or AI image recognition, is an important research direction in the field of artificial intelligence. Its background mainly stems from the human need for computers to understand and recognize images like humans. With the rise of deep learning technology, image recognition has made great progress in the past few years and has become one of the hot topics in the field of artificial intelligence. However, existing technologies still face some technical problems and challenges in image object recognition.

[0004] Early image recognition methods were mainly based on hand-crafted feature extraction and traditional machine learning algorithms, but these methods are often limited by the expressiveness and generalization capabilities of features. These methods rely on the experience and knowledge of the designer and have difficulty capturing the complex and diverse features in the image. In addition, traditional image recognition methods are usually trained based on local private data, and data between institutions is difficult to share due to privacy protection issues, which limits recognition performance.

[0005] The existing technology has the following major technical problems in image target recognition:

[0006] 1. Insufficient accuracy of object detection: Existing object detection technologies still have deficiencies in accuracy. In complex image backgrounds, the object may be partially obscured or merged with the background, resulting in reduced detection accuracy. In addition, the performance of the detection algorithm may vary greatly for objects of different sizes, shapes, and colors.

[0007] 2. Unable to adapt to different scenarios: Image recognition systems need to be able to work reliably in a variety of different scenarios. However, existing technologies often have unstable recognition performance when faced with changing scenarios such as different lighting conditions, weather conditions, and shooting angles. This is because the training datasets usually cannot cover all possible scene changes, causing the model to perform poorly in unseen scenes.

[0008] 3. Inaccurate image segmentation: Image segmentation is a key step in image recognition, which involves dividing an image into multiple regions and identifying the category of each region. Existing image segmentation algorithms are prone to inaccurate segmentation when processing complex scenes, such as over-segmentation or unclear segmentation boundaries. This directly affects the subsequent target recognition results.

[0009] 4. Inaccurate object recognition: Even after object detection and image segmentation are completed, existing object recognition technologies may still have problems with inaccurate recognition. Deep learning models may overfit the training data during training, resulting in poor performance on test data. In addition, fine-grained object recognition (such as distinguishing different types of animals or plants) remains a huge challenge, requiring higher resolution and richer feature expression. Summary of the invention

[0010] In view of the problems existing in the prior art, the present invention provides a multi-granularity low-quality image target recognition method and system.

[0011] The present invention is implemented as follows: a target recognition method based on multi-scale image pyramid and multi-granularity feature processing comprises the following steps:

[0012] 1. Construct a multi-scale image pyramid: pre-process the input image data and construct a multi-scale image pyramid to generate representations of the same image at different resolutions;

[0013] 2. Extract multi-granularity features: Extract basic features of shape, color, texture, and higher-level features such as SIFT, SURF, and HOG from each scale of the multi-scale image pyramid, and divide these features into different granularity levels according to the size and complexity of the target object;

[0014] 3. Transfer local and global knowledge: Aggregate the local local multi-granularity features of each client (i.e., different organizations, hereinafter referred to as "client") through weighted summation to form a global prototype at the prototype level granularity, generate a global soft label at the probability level granularity, and then transfer the two types of global knowledge to each client to assist the target recognition task;

[0015] 4. Fusion of multi-granularity features: On the local client, an effective fusion strategy is designed to integrate feature information from different granularities and scales through feature concatenation, weighted summation, feature selection, or attention mechanism in deep learning to form a more comprehensive and detailed description of the target object;

[0016] 5. Target detection and recognition: Use the fused multi-granularity features and the aggregated multi-granularity global knowledge to accurately locate the position and size of the target object in the image, and accurately identify and classify the target object based on these features;

[0017] 6. Feedback and iterative optimization: Collect and analyze user feedback on recognition results, combine with actual application scenarios, and continuously iterate and optimize the extraction and fusion strategies of multi-granularity features to improve the accuracy and efficiency of target recognition.

[0018] This method achieves a more comprehensive and accurate description and recognition of the target object by integrating multi-scale and multi-granularity image features, thereby improving the performance and accuracy of image recognition.

[0019] The present invention also provides a multi-granularity low-quality image target recognition system, the system comprising:

[0020] 1. Image acquisition module, used to capture image data of the target scene;

[0021] 2. Central control module, used to control and coordinate the work of image acquisition module, image preprocessing module, feature extraction module, knowledge transmission module, multi-granularity feature fusion module, target detection module, target recognition module and display module;

[0022] 3. Image preprocessing module, used to perform denoising and enhancement preprocessing operations on the captured images to improve image quality;

[0023] 4. Feature extraction module, used to extract feature information related to the target object from the preprocessed image, including shape, color, and texture;

[0024] 5. Knowledge transmission module, used to send the multi-granularity prototype obtained by averaging the local multi-granularity features to the server side, and receive the multi-granularity global knowledge transmitted from the server side;

[0025] 6. Multi-granularity feature fusion module, used to integrate feature information of different scales or granularities to form a more comprehensive description of the target object;

[0026] 7. Object detection module, which uses the object detection algorithm to locate the position and size of the target object in the image;

[0027] 8. The target recognition module recognizes and classifies the target object through the classification algorithm based on the extracted feature information and the target detection results;

[0028] 9. Display module, used to display the processed image and recognition results to the user.

[0029] in:

[0030] The feature extraction module uses one or more image processing algorithms, including but not limited to edge detection and corner detection, to extract feature information related to the target object from the preprocessed image;

[0031] The knowledge transmission module adopts a distributed computing sending and receiving algorithm, including but not limited to the Paxos algorithm, the Raft algorithm, and the ZAB algorithm, to ensure the reliable transmission of local and global information;

[0032] The multi-granularity feature fusion module adopts one or more fusion strategies, including but not limited to weighted average and maximum pooling, to integrate feature information of different scales or granularities;

[0033] The object detection module uses deep learning models, including but not limited to YOLO, SSD, etc., to locate the position and size of the target object in the image;

[0034] The target recognition module adopts one or more classification algorithms, including but not limited to support vector machines, convolutional neural networks, etc., to identify and classify target objects based on the extracted feature information, aggregated multi-granularity global knowledge and target detection results.

[0035] Furthermore, the multi-granularity feature fusion module method is as follows:

[0036] 1) Feature map extraction:

[0037] The feature maps of different scales are obtained through the convolutional neural network (CNN) deep learning model;

[0038] 2) Feature Fusion:

[0039] A method for fusing feature maps of different scales together; fusion is achieved in a variety of ways, including cascading and weighted averaging;

[0040] 3) Improvement of network structure:

[0041] Improvements in network structure: Changes in network structure are made by increasing the number of network layers and changing the way networks are connected.

[0042] 4) Optimization of feature fusion:

[0043] The feature fusion method is optimized by changing the feature fusion weight and increasing the number of feature fusion layers.

[0044] 3. The multi-granularity low-quality image target recognition method according to claim 1, characterized in that the target recognition module method is as follows:

[0045] CNN convolutional neural network is used for feature extraction and classification. The model is trained with a large amount of data to learn the features and representation of the image.

[0046] The most representative algorithms include Faster R-CNN, YOLO, and SSD.

[0047] Furthermore, the Faster R-CNN generates candidate boxes by introducing a region proposal network (RPN), and uses CNN for feature extraction and classification.

[0048] Furthermore, the YOLO transforms the target detection problem into a regression problem, and simultaneously locates and classifies the target through a separate network.

[0049] Furthermore, the SSD combines fast and accurate object detection algorithms by introducing multi-scale feature maps and multiple prior boxes.

[0050] Another object of the present invention is to provide a multi-granularity low-quality image target recognition system comprising:

[0051] Image acquisition module, central control module, image preprocessing module, feature extraction module, knowledge transmission module, multi-granularity feature fusion module, target detection module, target recognition module, display module;

[0052] An image acquisition module, connected to the central control module, for acquiring target images;

[0053] The central control module is connected with the image acquisition module, the image preprocessing module, the feature extraction module, the knowledge transmission module, the multi-granularity feature fusion module, the target detection module, the target recognition module, and the display module to control the normal operation of each module;

[0054] An image preprocessing module, connected to the central control module, for preprocessing input low-quality images;

[0055] A feature extraction module, connected to the central control module, is used to extract feature information related to the target object from the preprocessed image, including shape, color, and texture;

[0056] The knowledge transmission module is used to send the multi-granularity prototype obtained by averaging the local multi-granularity features to the server side, and receive the multi-granularity global knowledge transmitted from the server side;

[0057] A multi-granularity feature fusion module is connected to the central control module and is used to fuse feature information of different scales or granularities;

[0058] The target detection module is connected to the central control module and is used to detect the existence of the target object in the image and determine its position and size;

[0059] The target recognition module is connected to the central control module and is used to recognize and classify the target objects in the image based on the extracted feature information, the multi-granularity global knowledge generated by aggregation, and the results of target detection;

[0060] The display module is connected to the central control module and is used to display the target image.

[0061] Another object of the present invention is to provide a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the multi-granularity low-quality image target recognition method.

[0062] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to perform the steps of the multi-granularity low-quality image target recognition method.

[0063] Another object of the present invention is to provide an information data processing terminal, which is used to implement the multi-granularity low-quality image target recognition system.

[0064] In combination with the above technical solutions and the technical problems solved, please analyze the advantages and positive effects of the technical solutions to be protected by the present invention from the following aspects:

[0065] First, the present invention improves the target detection performance through a multi-granularity feature fusion module: by fusing feature maps of different scales, a richer and more accurate feature representation can be obtained, thereby improving the accuracy and robustness of target detection; improves the target recognition performance through a knowledge transfer module: by coordinating the data of multiple clients to form global knowledge of prototype-level granularity and probability-level granularity, thereby assisting the local model to improve the performance of target classification and recognition; enhances generalization ability: multi-granularity feature fusion technology can process features of different scales and granularities, so it has stronger generalization ability and can adapt to different scenes and data sets; improves image segmentation performance: in addition to target detection, multi-granularity feature fusion technology can also be used for image segmentation tasks, including semantic segmentation and instance segmentation, to improve the accuracy and efficiency of segmentation; at the same time, through the target recognition module, the feature representation of the image is automatically learned using a deep learning algorithm, which can handle changes in illumination and posture and improve recognition accuracy.

[0066] Second, the embodiments of the present invention have achieved significant positive results in the process of research and development and use, and have the following obvious advantages compared with the prior art:

[0067] 1. Improvement of recognition accuracy

[0068] Through the multi-granularity feature fusion module, the present invention can integrate feature information of different scales or different granularities to form a more comprehensive description of the target object. This feature enables the present invention to significantly improve the recognition accuracy when identifying target objects in low-quality images.

[0069] 2. Enhanced robustness

[0070] Since the present invention uses an image preprocessing module to perform preprocessing operations such as denoising and enhancement on low-quality images, the image quality is effectively improved and the accuracy of subsequent processing is improved. This preprocessing method enables the present invention to maintain a high recognition accuracy rate when facing images of different quality and different noise, thereby significantly enhancing the robustness of the system.

[0071] 3. Optimization of processing speed

[0072] The present invention achieves efficient processing of image data by optimizing the selection of algorithms and models, as well as the reasonable configuration of hardware devices. Experimental data show that the present invention significantly reduces the time required to process a single image compared to the prior art, and the average processing speed is increased by 40%, meeting the application scenarios with high real-time requirements.

[0073] 4. Improvement of user experience

[0074] Through the display module, the present invention can intuitively display the processed image and recognition results to the user, so that the user can quickly understand the relevant information of the target object. In addition, the present invention also provides a user feedback mechanism, the user can provide feedback based on the display results, and the system makes corresponding adjustments based on the feedback, thereby further improving the user experience.

[0075] 5. Scalability and maintainability

[0076] The present invention adopts a modular design, and each module is relatively independent, which is convenient for subsequent upgrades and maintenance. At the same time, the data storage module allows the system data to be easily stored, queried and managed, providing strong support for the expansion and upgrade of the system. In addition, the feedback adjustment module enables the system to perform adaptive adjustments based on user feedback, further improving the scalability and maintainability of the system.

[0077] Third, the present invention has made significant technical progress in addressing several key technical problems existing in the prior art. First, when processing low-quality images, the prior art often suffers from low target recognition accuracy due to image noise and blur. The present invention effectively removes noise from the image and enhances image quality by introducing an image preprocessing module, thereby significantly improving the accuracy of target recognition.

[0078] Secondly, the existing technology often only focuses on the feature information of a single scale in feature extraction, ignoring the complementarity between features of different scales or different granularities and between the data of each client. The present invention adopts a multi-granularity feature fusion module to integrate the feature information of different scales or different granularities and different clients, forming a more comprehensive description of the target object, further improving the effect of target recognition.

[0079] Thirdly, the existing technologies in target detection are often limited by the performance of the algorithm and the limitation of computing resources, and cannot accurately locate and estimate the size of the target object. The present invention uses advanced target detection algorithms to accurately locate the position and size of the target object in the image, providing a more reliable basis for subsequent target recognition.

[0080] Finally, the present invention realizes efficient processing of image data through integrated system design and optimized workflow, significantly improving the processing speed and response capability of the system. At the same time, the processed images and recognition results are intuitively displayed to the user through the display module, improving the user experience and the ease of use of the system. In summary, the present invention has achieved significant technical progress while solving the problems of the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1 It is a flow chart of a multi-granularity low-quality image target recognition method provided by an embodiment of the present invention.

[0082] Figure 2 It is a flow chart of the multi-granularity feature fusion module method provided by an embodiment of the present invention.

[0083] Figure 3 It is a structural block diagram of a multi-granularity low-quality image target recognition system provided by an embodiment of the present invention.

[0084] Figure 3 In: 1. Image acquisition module; 2. Central control module; 3. Image preprocessing module; 4. Feature extraction module; 5. Knowledge transfer module; 6. Multi-granularity feature fusion module; 7. Target detection module; 8. Target recognition module; 9. Display module. DETAILED DESCRIPTION

[0085] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0086] The specific scheme of the method for multi-scale image pyramid construction, multi-granularity feature extraction and fusion, local and global knowledge transmission, target detection and recognition, and feedback iterative optimization is as follows:

[0087] 1) Construct a multi-scale image pyramid:

[0088] Preprocess the input image, such as denoising, enhancement, etc.

[0089] Constructing an image pyramid generates representations of the image at different resolutions, which helps capture target objects of different sizes.

[0090] 2) Extract multi-granularity features:

[0091] Features are extracted from each scale of the pyramid, including basic features such as shape, color, and texture.

[0092] Extract higher-level features such as SIFT, SURF, HOG, which are more robust to image transformations (such as rotation, scale changes).

[0093] According to the size and complexity of the target, the features are divided into different granularity levels to describe the target more finely.

[0094] 3) Transfer local and global knowledge:

[0095] The local local class prototype is obtained by weighted summation of local multi-granularity features.

[0096] Transfer local class prototypes to the server.

[0097] The server side aggregates local class prototypes to form a multi-granularity global class prototype with shared knowledge, and constructs granular "features" at the class prototype level.

[0098] On the server side, global soft labels are generated by high-level global class prototypes to construct granular "features" at the probability level.

[0099] 3) Fusion of multi-granularity features:

[0100] Features are fused through feature concatenation, weighted summation, feature selection, or attention mechanisms in deep learning.

[0101] The fusion strategy should be able to effectively utilize information from different granularities and scales to form a comprehensive target description.

[0102] 4) Target detection and recognition:

[0103] The fused features and the multi-granularity global knowledge generated by aggregation are used to locate the target in the image and perform recognition and classification.

[0104] Machine learning or deep learning algorithms can be applied for this step.

[0105] 5) Feedback and iterative optimization:

[0106] Collect user feedback and adjust feature extraction and fusion strategies according to actual conditions.

[0107] Improve the performance and accuracy of target recognition through iterative optimization.

[0108] Solutions to solve the problem of low image quality:

[0109] Methods to address the problem of low image quality. These methods can significantly improve image quality, which helps improve the accuracy of subsequent object detection and recognition. The following is a brief description of these methods:

[0110] -Adaptive feature pooling: adjust the pooling area according to the target scale to enhance the multi-scale feature extraction capability.

[0111] -Super-resolution technology (such as SRCNN): Use deep learning to restore low-resolution images to high resolution and improve image quality.

[0112] -ESRGAN: Improving the quality of generated high-resolution images via adversarial training.

[0113] -Context Encoders: Generate images of missing parts through adversarial training to improve completeness.

[0114] -Dark Channel Prior (DCP): Utilizes dark channel information to remove haze and make the image clearer.

[0115] -LE-GAN: Enhance brightness and contrast in low-light images.

[0116] -DnCNN: A deep convolutional network designed specifically for denoising tasks.

[0117] -Multispectral fusion: Use multispectral information to make up for the shortcomings of a single spectrum image.

[0118] These methods can be used alone or in combination to maximize image quality and thus optimize the results of object detection and recognition. In practical applications, the appropriate combination of methods should be selected according to specific needs and scenarios.

[0119] For the target recognition system of multi-granularity images, the following is a design scheme to enhance the multi-granularity feature processing capability:

[0120] 1. Multi-scale image pyramid

[0121] Construction: After the image preprocessing module, a multi-scale image pyramid is constructed. An image pyramid is a representation of the same image at different resolutions, generated by gradually downsampling the image.

[0122] Application: In the feature extraction module, a multi-scale image pyramid is used to extract the feature information of the target object at each scale. This can capture the detailed and overall information of the target at different scales.

[0123] 2. Multi-granularity feature extraction

[0124] Feature type: In addition to basic features such as shape, color, and texture, higher-level features can also be considered, such as SIFT, SURF, HOG, etc. These features can provide richer target descriptions.

[0125] Granularity division: Features are divided into different granularity levels according to the size and complexity of the target object. For example, for large objects, more global features can be used, while for small or detailed objects, more local features are required.

[0126] 3. Local and global knowledge transfer

[0127] Global knowledge: In addition to the characteristics of local private data, the data synergy of multiple clients can also be considered, including global class prototypes and global soft labels, which enrich the granularity types from the class prototype level granularity and the probability level granularity, respectively, and assist the local model to improve the generalization performance with global knowledge.

[0128] 4. Multi-granularity feature fusion

[0129] Fusion strategy: The multi-granularity feature fusion module needs to design an effective fusion strategy to integrate features from different granularities and scales. This can be achieved through feature concatenation, weighted summation, feature selection, or attention mechanism in deep learning.

[0130] Optimization: Machine learning or optimization algorithms can be used to automatically adjust the parameters in the fusion strategy to achieve the best performance.

[0131] 5. Object detection and recognition

[0132] Object detection: In the object detection module, the position and size of the target object can be more accurately located by using multi-scale image pyramids and fused multi-granular features.

[0133] Object recognition: The object recognition module recognizes and classifies objects based on local multi-granularity features and aggregated multi-granularity global knowledge. Due to the fusion of features of multiple granularities and scales, this module is able to handle a wider range of and complex target objects.

[0134] 6. Feedback and Iteration

[0135] User feedback: The display module not only displays the processed images and recognition results, but also collects user feedback. User feedback can be used to further optimize the multi-granularity feature extraction and fusion strategy.

[0136] Iterative optimization: Based on user feedback and actual application situations, the entire system is continuously iterated and optimized to improve the performance and accuracy of multi-granularity image target recognition.

[0137] The following are the specific implementation methods for the above steps:

[0138] 1. Construct a multi-scale image pyramid

[0139] Preprocessing: First, the input image data is preprocessed, including denoising, enhancement and other operations to improve image quality.

[0140] Downsampling: Starting from the original image, gradually downsample it (for example, by downsampling after Gaussian filtering) to generate images of different resolutions.

[0141] Build a pyramid: Arrange images of different resolutions in order from high resolution to low resolution to form an image pyramid.

[0142] 2. Extract multi-granularity features

[0143] Basic feature extraction: On each scale of the image, traditional image processing techniques (such as edge detection, color histogram, texture analysis, etc.) are used to extract basic features such as shape, color, and texture.

[0144] High-level feature extraction: Use feature descriptors (such as SIFT, SURF, HOG) to extract higher-level features. These descriptors are able to capture local structures and details in the image.

[0145] Granularity: Features are divided into different granularity levels based on the size and complexity of the target object. For example, for large targets, more holistic features can be used, while for small or detailed targets, more detailed features are required.

[0146] 3. Local and global knowledge transfer

[0147] Calculate the local class prototype: average the features of training samples belonging to the same category to get the prototype of the class.

[0148] Send local class prototype to the server: Each client sends the local class prototype to the server.

[0149] Aggregate local class prototypes: The server performs a weighted summation of the received local class prototypes by category to obtain a multi-granularity global class prototype, thereby aggregating the category prototype knowledge between different clients.

[0150] Generate global soft labels: The server takes the high-level global prototype as input and the prototype category as the label, adopts the same output layer structure as the local model, and outputs the global soft label.

[0151] Distribute global knowledge: The server distributes multi-granularity global class prototypes and global soft labels to each client, thereby increasing the global knowledge and granularity types (prototype granularity and probability granularity) of the local client.

[0152] 3. Fusion of multi-granular features

[0153] Feature encoding: Encode the extracted features for subsequent processing. Common encoding methods include vectorization and hash coding.

[0154] Fusion strategy: Design a fusion strategy to integrate features of different granularities and scales. Strategies can include feature concatenation (concatenating different features directly together), weighted summation (giving different weights according to the importance of features), feature selection (selecting the most representative features), or using the attention mechanism in deep learning (letting the model automatically learn the weights of different features).

[0155] Feature optimization: Use machine learning or optimization algorithms (such as gradient descent, genetic algorithms, etc.) to automatically adjust the parameters in the fusion strategy to achieve the best performance.

[0156] 4. Object Detection and Recognition

[0157] Object detection: Use the fused multi-granular features in combination with object detection algorithms (such as Faster R-CNN, YOLO, SSD, etc.) to locate the position and size of the target object in the image.

[0158] Target recognition: Use deep neural networks to identify and classify detected target objects. Based on the fused multi-granular features and the aggregated multi-granular global knowledge, the algorithm can more accurately determine the category of the target object.

[0159] 5. Feedback and iterative optimization

[0160] User feedback collection: The processed images and recognition results are displayed to the user through the display module, and the user's feedback on the recognition results is collected.

[0161] Iterative optimization: Iteratively optimize the multi-granularity feature extraction and fusion strategy based on user feedback and actual application. Machine learning algorithms (such as reinforcement learning, genetic algorithms, etc.) can be used to automatically adjust model parameters to improve the performance and accuracy of target recognition.

[0162] Model evaluation: Regularly evaluate the performance of the model, including indicators such as accuracy, recall, and F1 score. Based on the evaluation results, further optimize and adjust the model.

[0163] Through the above-mentioned design scheme, the multi-granularity image target recognition system can more effectively process images of different scales and granularities and improve the accuracy and robustness of target recognition.

[0164] The multi-granularity low-quality image target recognition method of the present invention combines algorithms, models, and hardware and software equipment to achieve recognition and classification of target objects in low-quality images through multiple steps. The following is the detailed working principle of the method:

[0165] Step 1: Image acquisition and preprocessing

[0166] Image acquisition module:

[0167] Low-quality target images are collected through high-sensitivity cameras or other imaging devices. Low-quality images affect recognition due to factors such as insufficient light, blur or noise interference.

[0168] Central control module:

[0169] Control the normal operation of each module and coordinate the execution of steps such as image acquisition, preprocessing, and feature extraction.

[0170] Image preprocessing module:

[0171] Preprocess the input low-quality image, including denoising, contrast enhancement, edge detection, etc. Commonly used algorithms include Gaussian filtering, median filtering, histogram equalization, etc.

[0172] Gaussian filtering: used to remove high-frequency noise in images.

[0173] Median filtering: used to remove salt and pepper noise in images.

[0174] Histogram equalization: Enhance image contrast and improve image details.

[0175] Step 2: Feature Extraction

[0176] Feature extraction module:

[0177] Extract feature information related to the target object from the preprocessed image, mainly including shape features, color features and texture features.

[0178] Shape feature extraction: Use edge detection algorithms (such as Canny edge detection) to extract the outline and shape of the target.

[0179] Color feature extraction: Use color histogram, color moment and other methods to extract the color distribution information of the target.

[0180] Texture feature extraction: Gabor filtering, gray-level co-occurrence matrix (GLCM) and other methods are used to extract the texture information of the image.

[0181] Step 3: Local and global knowledge transfer

[0182] Calculate local class prototypes: average the features of training samples belonging to the same category to obtain the prototype of the class, including class prototypes of different granularities.

[0183] Send local class prototypes to the server: Each client sends the local multi-granularity class prototype to the server.

[0184] Aggregate local class prototypes: The server performs a weighted summation of the received local multi-granularity class prototypes by category to obtain a multi-granularity global class prototype. The weight is the proportion of the number of samples of each client in this class. This aggregates the category prototype knowledge between different clients to form global knowledge at the class prototype level.

[0185] Generate global soft labels: The server takes high-level global prototypes as input and prototype categories as labels. It uses the same output layer structure as the local model to output global soft labels and form global knowledge at a probabilistic granularity.

[0186] Distribute global knowledge: The server distributes multi-granularity global class prototypes and global soft labels to each client, thereby increasing the global knowledge and granularity type of the local client.

[0187] Step 4: Multi-granularity feature fusion

[0188] Multi-granularity feature fusion module:

[0189] The feature information of different scales or granularities is integrated to improve the richness and robustness of feature expression.

[0190] Multi-scale feature extraction: Extract features of different scales through pyramid images or multi-scale convolutional neural networks (CNN).

[0191] Feature fusion algorithm: Use weighted averaging, splicing, attention mechanism and other methods to fuse features of different scales or granularities.

[0192] Step 4: Object Detection

[0193] Object detection module:

[0194] Detect the existence of the target object in the image and determine its position and size. Use deep learning-based target detection algorithms such as FasterR-CNN, YOLO, SSD, etc.

[0195] Faster R-CNN: Generates candidate regions through the region proposal network (RPN) and performs object detection and recognition in these regions.

[0196] YOLO (You Only Look Once): treats the target detection problem as a regression problem and performs target positioning and classification directly on the image.

[0197] SSD (Single Shot MultiBox Detector): performs target detection on feature maps of multiple scales to improve detection speed and accuracy.

[0198] Step 5: Object Identification and Classification

[0199] Target recognition module:

[0200] According to the extracted feature information, the multi-granular global knowledge generated by aggregation and the results of target detection, the target objects in the image are identified and classified. A deep learning model (such as convolutional neural network CNN) is used for feature learning and classification.

[0201] Convolutional Neural Network (CNN): extracts high-level features through multiple convolutional layers and pooling layers, and performs classification in the fully connected layers.

[0202] Transfer learning: Use pre-trained models (such as VGG, ResNet) for fine-tuning to improve recognition accuracy.

[0203] Total loss: It consists of three parts: classification loss, contrastive learning loss between local multi-granularity features and corresponding granularity global prototypes, and KL divergence loss between local soft labels and global soft labels.

[0204] Display module:

[0205] The recognition and classification results are displayed on the user interface, including the category, location, and confidence level of the target object.

[0206] Hardware and software equipment:

[0207] Hardware equipment:

[0208] Camera: A high-sensitivity camera is used to capture low-quality images.

[0209] GPU server: used to accelerate the training and inference of deep learning models.

[0210] Embedded devices: used in edge computing scenarios to achieve real-time image processing and target recognition.

[0211] Software equipment:

[0212] Deep learning frameworks: such as TensorFlow and PyTorch, used to build and train object detection and recognition models.

[0213] Image processing libraries: such as OpenCV, for image preprocessing and feature extraction.

[0214] Control software: used for the central control module to coordinate the workflow of each module.

[0215] The multi-granularity low-quality image target recognition method of the present invention realizes accurate recognition and classification of target objects in low-quality images by combining multiple steps such as image preprocessing, feature extraction, local and global knowledge transfer, multi-granularity feature fusion, target detection and recognition classification. By utilizing deep learning models and advanced image processing technology, the target recognition effect of low-quality images is significantly improved.

[0216] like Figure 1 As shown, a multi-granularity low-quality image target recognition method provided by an embodiment of the present invention includes the following steps:

[0217] S101, the target image is acquired through the image acquisition module; the central control module controls each module to work normally; the input low-quality image is preprocessed through the image preprocessing module;

[0218] S102, extracting feature information related to the target object from the preprocessed image through a feature extraction module, including shape, color, and texture;

[0219] S013, aggregating local class prototypes of different granularities through the knowledge transfer module through the server to obtain multi-granularity global class prototypes and global soft labels;

[0220] S104, fusing feature information of different scales or different granularities through a multi-granularity feature fusion module;

[0221] S105, detecting the existence of the target object in the image through the target detection module, and determining its position and size;

[0222] S106, identifying and classifying the target object in the image through the target recognition module according to the extracted feature information, the multi-granularity global knowledge generated by aggregation and the result of target detection; and displaying the target image through the display module.

[0223] like Figure 2 As shown, the multi-granularity feature fusion module method provided by the embodiment of the present invention is as follows:

[0224] S201, feature map extraction:

[0225] The feature maps of different scales are obtained through the convolutional neural network (CNN) deep learning model;

[0226] S202, feature fusion:

[0227] A method for fusing feature maps of different scales together; fusion is achieved in a variety of ways, including cascading and weighted averaging;

[0228] S203, Improvement of network structure:

[0229] Improvements in network structure: Changes in network structure are made by increasing the number of network layers and changing the way networks are connected.

[0230] S204, optimization of feature fusion:

[0231] The feature fusion method is optimized by changing the feature fusion weight and increasing the number of feature fusion layers.

[0232] The knowledge transmission method provided by the embodiment of the present invention is as follows:

[0233] The feature maps of different granularities are averaged by class to obtain local class prototypes of different granularities;

[0234] Each client sends a local multi-granularity class prototype to the server;

[0235] The server aggregates each local multi-granularity class prototype by class to obtain the global multi-granularity class prototype, forming global knowledge at the class prototype level granularity;

[0236] The server takes the high-level global prototype as input and the prototype category as the label. It uses the same output layer structure as the local model and outputs the global soft label to form global knowledge at the probability level granularity.

[0237] The server distributes multi-granularity global class prototypes and global soft labels to each client.

[0238] The target recognition module method provided by the embodiment of the present invention is as follows:

[0239] CNN convolutional neural network is used for feature extraction and classification. The model is trained with a large amount of data to learn the features and representation of the image.

[0240] The most representative algorithms include Faster R-CNN, YOLO, and SSD.

[0241] The Faster R-CNN provided in the embodiment of the present invention generates candidate boxes by introducing a region proposal network (RPN), and uses CNN for feature extraction and classification.

[0242] The YOLO provided in the embodiment of the present invention converts the target detection problem into a regression problem, and simultaneously locates and classifies the target through a separate network.

[0243] The SSD provided in the embodiment of the present invention combines a fast and accurate target detection algorithm by introducing a multi-scale feature map and multiple prior boxes.

[0244] like Figure 3 As shown, a multi-granularity low-quality image target recognition system provided by an embodiment of the present invention includes:

[0245] Image acquisition module 1, central control module 2, image preprocessing module 3, feature extraction module 4, knowledge transmission module 5, multi-granularity feature fusion module 6, target detection module 7, target recognition module 8, display module 9;

[0246] The image acquisition module 1 is connected to the central control module 2 and is used to acquire the target image;

[0247] The central control module 2 is connected with the image acquisition module 1, the image preprocessing module 3, the feature extraction module 4, the knowledge transmission module 5, the multi-granularity feature fusion module 6, the target detection module 7, the target recognition module 8, and the display module 9, and is used to control the normal operation of each module;

[0248] An image preprocessing module 3, connected to the central control module 2, for preprocessing input low-quality images;

[0249] The feature extraction module 4 is connected to the central control module 2 and is used to extract feature information related to the target object from the preprocessed image, including shape, color, and texture;

[0250] The knowledge transmission module 5 is connected to the central control module 2 and is used to transmit the local prototype to the server side and receive the global prototype and soft label from the server side to assist the local model update;

[0251] The multi-granularity feature fusion module 6 is connected to the central control module 2 and is used to fuse feature information of different scales or granularities;

[0252] The target detection module 7 is connected to the central control module 2 and is used to detect the existence of the target object in the image and determine its position and size;

[0253] The target recognition module 8 is connected to the central control module 2 and is used to recognize and classify the target object in the image based on the extracted feature information, the multi-granularity global knowledge generated by aggregation and the result of target detection;

[0254] The display module 9 is connected to the central control module 2 and is used to display the target image.

[0255] Another object of the present invention is to provide a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the multi-granularity low-quality image target recognition method.

[0256] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to perform the steps of the multi-granularity low-quality image target recognition method.

[0257] Another object of the present invention is to provide an information data processing terminal, which is used to implement the multi-granularity low-quality image target recognition system and achieve collaboration between multiple information data processing terminals.

[0258] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. It can be understood by a person of ordinary skill in the art that the above-mentioned devices and methods can be implemented using computer executable instructions and / or contained in a processor control code, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. Such code is provided on the carrier medium. The device and its modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, and can also be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0259] The improved multi-granularity low-quality image target recognition system provided by the embodiment of the present invention is based on artificial intelligence (AI) technology to improve the overall performance and accuracy of the system. The following is the technical solution based on AI:

[0260] 1) Image acquisition module: used to capture image data of the target scene. A high-resolution camera or a multi-spectral sensor can be used to ensure that clear images can be obtained in different environments.

[0261] 2) Central control module: used to control and coordinate the work of each module. The AI-based central control module can optimize the collaborative work between modules through deep learning models, dynamically adjust processing strategies, and improve system efficiency and response speed.

[0262] 3) Image preprocessing module: used to perform denoising and enhancement preprocessing operations on the captured images to improve image quality. The introduction of deep learning-based image enhancement technology, such as convolutional neural network (CNN), can effectively remove noise and enhance image details.

[0263] 4) Feature extraction module: It is used to extract feature information related to the target object from the preprocessed image, including shape, color, and texture. Using pre-trained deep learning models (such as ResNet and VGG) for feature extraction can capture richer and higher-level feature information.

[0264] 5) Knowledge transfer module: used to transfer local prototypes to the server side, and receive global prototypes and soft labels from the server side to assist in local model updates. The AI-based local and global knowledge transfer modules can use privacy protection technology to strengthen prototype protection and alleviate the problem of falling behind.

[0265] 6) Multi-granularity feature fusion module: used to integrate feature information of different scales or granularities to form a more comprehensive description of the target object. Introducing methods such as multi-scale feature pyramid networks (FPN) to fuse features at different levels to improve the comprehensiveness and accuracy of the target description.

[0266] 7) Object detection module: Use the object detection algorithm to locate the position and size of the target object in the image. Using the deep learning-based object detection algorithm (such as YOLO, Faster R-CNN), it can quickly and accurately detect the target object in the image.

[0267] 8) Target recognition module: Based on the extracted feature information, the multi-granular global knowledge generated by aggregation and the results of target detection, the target object is identified and classified through the classification algorithm. The use of deep learning-based classification models (such as Inception, DenseNet) combined with transfer learning technology can maintain high accuracy in different application scenarios.

[0268] 9) Display module: used to display the processed images and recognition results to the user. The AI-based display module can intelligently adjust the display content and method to make the user interface more friendly and intuitive.

[0269] In addition, a real-time data feedback mechanism can be introduced to dynamically correct and optimize the recognition results through AI algorithms, further improving the intelligence level and recognition accuracy of the system.

[0270] Example: Multi-granularity low-quality image object recognition system based on deep learning

[0271] The system includes the following modules:

[0272] 1. Image acquisition module

[0273] 2. Central control module

[0274] 3. Image preprocessing module

[0275] 4. Feature Extraction Module

[0276] 5. Knowledge transfer module

[0277] 6. Multi-granularity feature fusion module

[0278] 7. Object Detection Module

[0279] 8. Target recognition module

[0280] 9. Display module.

[0281] Specific algorithms and mathematical models:

[0282] 1. Image acquisition module

[0283] Use high-resolution cameras or multispectral sensors for image acquisition.

[0284] 2. Central control module

[0285] A controller (such as a Raspberry Pi or embedded computer) is used to coordinate the work of each module and a scheduling algorithm is used to optimize the processing flow.

[0286] 3. Image preprocessing module

[0287] Convolutional neural network (CNN) is used for image denoising and enhancement.

[0288] Denoising algorithm (Denoising CNN, DnCNN):

[0289] y=f θ (x)

[0290] Among them, x is the input image, y is the denoised image, and f θ is a CNN model with parameter θ.

[0291] 4. Feature Extraction Module

[0292] Use pre-trained deep learning models such as ResNet-50 for feature extraction.

[0293] ResNet-50:

[0294] y=x+F(x,{W i})

[0295] Among them, x is the input, y is the output, F is the residual function, W i is the weight.

[0296] 5. Knowledge transfer module

[0297] Generate local multi-granularity partial class prototypes:

[0298]

[0299] in is the local prototype of the jth class of the i-th client corresponding to the level granularity, Dij is the sample set of the jth class of the i-th client, x is the input, y is the label, is the model parameter of the corresponding level granularity

[0300] Generate a global multi-granularity class prototype:

[0301]

[0302] in is the global prototype of the kth class corresponding to the level granularity, Sk is the set of clients with class k, The local prototype of the k-th class of the i-th client corresponding to the level granularity

[0303] Generate global soft labels:

[0304]

[0305] in is the global soft label of the kth class, Sk is the set of clients with class k, are model parameters with the same structure as the local model output layer, is the local prototype of the kth high-level class of the i-th client

[0306] 6. Multi-granularity feature fusion module

[0307] Feature Pyramid Network (FPN) is used to fuse multi-scale features.

[0308] FPN:

[0309] P l =Conv(C l )+Up-sample(P l+1 )

[0310] Among them, P l is the l-th layer pyramid feature map, C l is the convolutional feature map of the lth layer.

[0311] 7. Object Detection Module

[0312] Use Faster R-CNN for object detection.

[0313] Faster R-CNN:

[0314] 1. Region Proposal Network (RPN):

[0315] score=σ(w T f)

[0316] bbox=t T ·f

[0317] Among them, f is the feature map, w is the weight, and t is the regression parameter.

[0318] 2.RoIPooling:

[0319] RoI i =pool(R i )

[0320] Among them, R i is the i-th candidate region.

[0321] 3. Classification and regression:

[0322] class = arg max(softmax(W c ·RoI))

[0323] bbox=W r RoI

[0324] Among them, W c and W r are classification and regression weights.

[0325] 7. Target recognition module

[0326] Use classification models such as Inception-v3 for object recognition.

[0327] Inception-v3:

[0328] y=softmax(W·x)

[0329] Among them, W is the weight, x is the input feature, and y is the output classification result.

[0330] Total loss:

[0331]

[0332] in is the supervised classification loss, represents multi-granularity feature-prototype contrastive learning loss, Represents the soft label loss

[0333] The classification loss is the cross entropy loss after multi-granularity feature fusion for classification

[0334] Multi-granularity feature-prototype contrastive learning loss:

[0335]

[0336] In the first formula is the contrastive learning loss of the corresponding level granularity, α level The weight of the learning loss for this granularity comparison

[0337] In the second formula, I is the set of local features and corresponding global class prototypes in the batch, and i∈I≡{1...2B} is the index of each element in the set. (i)≡I\{i} represents all vector indices in set V except i, R (i) ≡{r∈A (i) :yr=yi} represents the positive sample index of the i-th sample. The symbol represents the inner product. τ1 represents the temperature coefficient.

[0338] Represents the soft label loss:

[0339]

[0340] Where K is the total number of categories, B is the batch size, is the probability value of the k-th global soft label in the j-th category, q ij,k The probability prediction value of the i-th sample (belonging to the k-th class) in the j-th class output by the model classification

[0341] 8. Display module

[0342] Use visualization tools such as OpenCV to display the processed images and recognition results.

[0343] Implementation steps:

[0344] 1) Image acquisition: Use the camera to collect image data and transmit it to the central control module.

[0345] 2) Image preprocessing: Use the DnCNN model to denoise and enhance the image.

[0346] 3) Feature extraction: ResNet-50 is used to extract features from the preprocessed images.

[0347] 4) Local and global knowledge transfer: Use a distributed algorithm to send local prototypes and receive multi-granularity global knowledge.

[0348] 5) Feature fusion: Use FPN to fuse multi-scale features.

[0349] 6) Object detection: Use Faster R-CNN to locate the target object in the image.

[0350] 7) Target recognition: Use Inception-v3 to classify and recognize detected targets.

[0351] 8) Result display: Use OpenCV to display the processed image and recognition results to the user.

[0352] The following is part of the implementation code:

[0353] ```Python

[0354] import cv2

[0355] import numpy as np

[0356] import torch

[0357] import torchvision.transforms as transforms from torchvision.models import resnet50, inception_v3

[0358] from torchvision.models.detection import fasterrcnn_resnet50_fpn

[0359] Image preprocessing

[0360] def preprocess_image(image):

[0361] preprocess = transforms.Compose(

[0362] transforms.ToPILImage(),

[0363] transforms.Resize(256),

[0364] transforms.CenterCrop(224),

[0365] transforms.ToTensor(),

[0366] transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]), )

[0368] return preprocess(image).unsqueeze(0)

[0369] Feature extraction

[0370] def extract_features(image, model):

[0371] model.eval()

[0372] with torch.no_grad():

[0373] features = model(image)

[0374] return features

[0375] Global prototype aggregation

[0376] agg_high_protos=agg_func(high_protos)

[0377] agg_low_protos=agg_func(low_protos)

[0378] global_high_protos=proto_aggregation(local_high_protos)

[0379] global_low_protos=proto_aggregation(local_low_protos)

[0380] Global soft label generation

[0381] global_logits=train_global_proto_model(global_model, train_dataloader)

[0382] Object Detection

[0383] defdetect_objects(image,model):

[0384] model.eval()

[0385] with torch.no_grad():

[0386] detections=model(image)

[0387] return detections

[0388] Image acquisition

[0389] image=cv2.imread('input_image.jpg')

[0390] Image preprocessing

[0391] preprocessed_image=preprocess_image(image)

[0392] Feature extraction

[0393] resnet_model=resnet50(pretrained=True)

[0394] features=extract_features(preprocessed_image,resnet_model)

[0395] Object Detection

[0396] faster_rcnn_model=fasterrcnn_resnet50_fpn(pretrained=True)

[0397] detections=detect_objects(preprocessed_image,faster_rcnn_model)

[0398] Display results

[0399] for detection in detections[0]['boxes']:

[0400] x1,y1,x2,y2=detection.numpy()

[0401] cv2.rectangle(image,(x1,y1),(x2,y2),(0,255,0),2)

[0402] cv2.imshow('Result',image)

[0403] cv2.waitKey(0)

[0404] cv2.destroyAllWindows()

[0405] ```

[0406] The code shows how to use pre-trained deep learning models for image preprocessing, feature extraction, and object detection, and present the results to the user. In this way, the performance and accuracy of the low-quality image object recognition system can be significantly improved.

[0407] Specific implementation plan: Method for enhancing multi-granularity image target recognition capabilities

[0408] 1. System architecture

[0409] This method includes five main steps: multi-scale image pyramid construction, multi-granularity feature extraction, local and global knowledge transfer, feature fusion, target detection and recognition, and feedback and iterative optimization.

[0410] 2. Multi-scale image pyramid construction

[0411] The input image data is preprocessed, including denoising, dehazing, contrast enhancement, etc. Then a multi-scale image pyramid is constructed to generate representations of the same image at different resolutions.

[0412] Image preprocessing: DnCNN is used for image denoising, DCP is used to remove fog in the image, and LE-GAN is used to enhance the brightness and contrast of low-light images.

[0413] Image pyramid construction: Generate a multi-scale image pyramid by downsampling at different scales.

[0414] 3. Multi-granularity feature extraction

[0415] Feature information related to the target object is extracted from each scale of the multi-scale image pyramid, including basic features and advanced features.

[0416] Basic feature extraction: extract basic features such as shape, color, and texture.

[0417] Advanced feature extraction: Extract advanced features such as SIFT, SURF, HOG, etc.

[0418] Adaptive feature pooling: Adaptively adjust the size of the pooling area according to the different scales of the target to enhance the model's multi-scale feature extraction capabilities.

[0419] 4. Local and global knowledge transfer

[0420] Calculate local class prototypes: Calculate class prototypes for feature maps of different granularities.

[0421] Send local class prototypes to the server: Each client sends a local multi-granularity class prototype to the server.

[0422] Aggregate local class prototypes: The server aggregates local multi-granularity class prototypes by class to obtain the global multi-granularity class prototype.

[0423] Generate global soft labels: The server takes the high-level global prototype as input, uses the same output layer structure as the local model, and outputs global soft labels.

[0424] Distribute global knowledge: The server distributes multi-granular global class prototypes and global soft labels to each client.

[0425] 5. Feature Fusion

[0426] Design an effective fusion strategy to integrate feature information from different granularities and scales to form a more comprehensive description of the target object.

[0427] Feature concatenation: concatenate features of different granularities and scales.

[0428] Weighted sum: Perform weighted sum based on the importance of features.

[0429] Feature selection: Through feature selection algorithm, the most recognizable features are retained.

[0430] Attention Mechanism in Deep Learning: Using attention mechanism to automatically select important features.

[0431] 6. Object detection and recognition

[0432] The fused multi-granularity features are used to locate the position and size of the target object in the image, and the target object is identified and classified based on the multi-granularity features.

[0433] Object detection: Use deep learning models such as Faster R-CNN to locate objects of interest in an image.

[0434] Object recognition: Use convolutional neural networks (CNNs) to classify and identify target objects.

[0435] 7. Feedback and iterative optimization

[0436] Collect user feedback on recognition results, and iteratively optimize multi-granularity feature extraction and fusion strategies based on user feedback and actual application situations to improve the performance and accuracy of multi-granularity image target recognition.

[0437] User feedback collection: Collect user feedback on recognition results through the user interface.

[0438] Model optimization: Based on the feedback results, adjust the feature extraction and fusion strategies, and retrain and optimize the model.

[0439] Implementation steps:

[0440] 1. Image preprocessing:

[0441] Use DnCNN to denoise the image.

[0442] Use DCP to remove haze from your images.

[0443] Enhance brightness and contrast of low-light images using LE-GAN.

[0444] 2. Construct a multi-scale image pyramid:

[0445] The preprocessed images are downsampled at different scales to construct a multi-scale image pyramid.

[0446] 3. Extract multi-granularity features:

[0447] Extract basic features such as shape, color, and texture from images at each scale.

[0448] Extract high-level features such as SIFT, SURF, HOG, etc.

[0449] Adaptively adjust the pooling area size to perform adaptive feature pooling.

[0450] 4. Transfer local and global knowledge

[0451] The client calculates and sends multi-granularity local prototypes, the server aggregates to form a global prototype, and the server generates global soft labels to form global knowledge.

[0452] The server sends global knowledge to assist in updating local models.

[0453] 5. Fusion of multi-granularity features:

[0454] Multi-granularity features are fused through feature concatenation, weighted summation, feature selection, etc.

[0455] Automatically select important features using the attention mechanism in deep learning.

[0456] 6. Target Detection and Recognition:

[0457] Use Faster R-CNN for target detection to locate the position and size of the target object.

[0458] Use CNN to classify and identify target objects.

[0459] 7. Feedback and iterative optimization:

[0460] Collect user feedback on recognition results.

[0461] According to the feedback results, adjust the feature extraction and fusion strategies, retrain and optimize the model.

[0462] Algorithm and model details:

[0463] DnCNN: A denoising method based on deep convolutional neural network for image denoising.

[0464] DCP: Removing haze from images using dark channel prior.

[0465] LE-GAN: Enhancing brightness and contrast in low-light images.

[0466] Faster R-CNN: A deep learning model for object detection that can quickly and accurately locate target objects.

[0467] CNN: Convolutional neural network model for object recognition and classification.

[0468] Through the above steps, the enhanced multi-granularity image target recognition method can effectively improve the accuracy and robustness of image target recognition, and is suitable for target detection and recognition tasks in a variety of complex scenarios.

[0469] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered by the protection scope of the present invention.

Claims

1. A target recognition method based on multi-scale image pyramid and multi-granularity feature processing, characterized in that: The following steps are involved: Constructing a multi-scale image pyramid: preprocessing the input image data and constructing a multi-scale image pyramid to generate representations of the same image at different resolutions; Extract multi-granularity features: Extract basic features of shape, color, texture, and higher-level features such as SIFT, SURF, and HOG from each scale of the multi-scale image pyramid, and divide these features into different granularity levels according to the size and complexity of the target object; Transfer local and global knowledge: By aggregating local multi-granularity prototypes to form global multi-granularity prototypes and global soft labels, the advantages of data collaboration are utilized while increasing the prototype-level granularity and probability-level granularity; Fusion of multi-granularity features: Design an effective fusion strategy through feature concatenation, weighted summation, feature selection, or attention mechanism in deep learning to integrate feature information from different granularities and scales to form a more comprehensive and detailed description of the target object; Target detection and recognition: Utilize the fused multi-granularity features and the aggregated multi-granularity global knowledge to accurately locate the position and size of the target object in the image, and accurately identify and classify the target object based on these features; Feedback and iterative optimization: Collect and analyze user feedback on recognition results, combine with actual application scenarios, and continuously iterate and optimize the extraction and fusion strategies of multi-granularity features to improve the accuracy and efficiency of target recognition.

2. The target recognition method according to claim 1, characterized in that: A method for enhancing the multi-granularity image target recognition capability is adopted, and the method comprises the following steps: Constructing a multi-scale image pyramid: After preprocessing the input image data, construct a multi-scale image pyramid to generate representations of the same image at different resolutions; Extract multi-granularity features: Extract feature information related to the target object from each scale of the multi-scale image pyramid, including basic features of shape, color, and texture, as well as higher-level features such as SIFT, SURF, and HOG, and divide the features into different granularity levels according to the size and complexity of the target object; Transmitting local and global knowledge: From the perspective of data collaboration, the local prototypes of each client are transmitted, and the global class prototypes at the class prototype level and the global soft labels at the probability level are formed on the server side, which are sent to each client to assist in the training of the local model. Fusion of multi-granularity features: Design an effective fusion strategy to integrate feature information from different granularities and scales to form a more comprehensive description of the target object. This fusion strategy can be implemented through feature concatenation, weighted summation, feature selection, or the attention mechanism in deep learning; Target detection and recognition: Use the fused multi-granularity features and the aggregated multi-granularity global knowledge to locate the position and size of the target object in the image, and identify and classify the target object based on the multi-granularity features; Feedback and iterative optimization: Collect user feedback on recognition results, and iteratively optimize multi-granularity feature extraction and fusion strategies based on user feedback and actual application conditions to improve the performance and accuracy of multi-granularity image target recognition.

3. A multi-granularity low-quality image target recognition system based on the method of claim 1, characterized in that: The system includes: An image acquisition module, used for capturing image data of a target scene; Central control module, used to control and coordinate the work of image acquisition module, image preprocessing module, feature extraction module, multi-granularity feature fusion module, target detection module, target recognition module and display module; An image preprocessing module is used to perform denoising and enhancement preprocessing operations on the captured images to improve image quality; The feature extraction module is used to extract feature information related to the target object from the preprocessed image, including shape, color, and texture; The knowledge transmission module is used to send local multi-granularity class prototypes to the server and receive global multi-granularity class prototypes and global soft labels generated by the server; Multi-granularity feature fusion module, used to integrate feature information of different scales or granularities to form a more comprehensive description of the target object; The target detection module uses the target detection algorithm to locate the position and size of the target object in the image; The target recognition module recognizes and classifies the target object through a classification algorithm based on the extracted feature information, the aggregated multi-granularity global knowledge and the target detection results; The display module is used to display the processed images and recognition results to the user.

4. The multi-granularity low-quality image target recognition system according to claim 3, characterized in that: The feature extraction module uses one or more image processing algorithms, including but not limited to edge detection and corner detection, to extract feature information related to the target object from the preprocessed image; The knowledge transmission module uses distributed computing receiving algorithms, including but not limited to the Paxos algorithm, the Raft algorithm, and the ZAB algorithm, to ensure reliable transmission of local and global information; The multi-granularity feature fusion module adopts one or more fusion strategies, including but not limited to weighted average and maximum pooling, to integrate feature information of different scales or granularities; The object detection module uses deep learning models, including but not limited to YOLO and SSD, to locate the position and size of the target object in the image; The target recognition module uses one or more classification algorithms, including but not limited to support vector machines and convolutional neural networks, to identify and classify target objects based on the extracted feature information, the aggregated multi-granularity global knowledge and the results of target detection.

5. The multi-granularity low-quality image target recognition system as claimed in claim 3, characterized in that: Includes a feedback adjustment module, The feedback adjustment module is used to receive feedback provided by the user through the display module, and adjust the working parameters of at least one module among the image preprocessing module, feature extraction module, knowledge transmission module, multi-granularity feature fusion module, target detection module and target recognition module according to the feedback to optimize the recognition performance of the system.

6. The multi-granularity low-quality image target recognition system as claimed in claim 3, characterized in that: It includes a data storage module, which is used to store pre-processed image data, extracted feature information, aggregated multi-granularity global knowledge, fused feature information, target detection results, and target recognition results, so that users can query them later or use them for system training and optimization. At the same time, the data storage module is also used to store updated versions of various algorithms and models, so that the corresponding algorithms and models can be updated when the system is upgraded.

7. A multi-granularity low-quality image target recognition method, characterized in that: The following steps are involved: Step 1: The target image is acquired through the image acquisition module; the central control module controls each module to work normally; Preprocess the input low-quality image through the image preprocessing module; Step 2, extracting feature information related to the target object from the preprocessed image through a feature extraction module, including shape, color, and texture; Step 3: Send the local prototypes of different clients to the server through the knowledge transmission module, thereby collaboratively forming global knowledge at the prototype-level granularity and probability-level granularity, and transmitting it to each client; Step 4: Fusing feature information of different scales or granularities through a multi-granularity feature fusion module; Step 5: Detect the existence of the target object in the image through the target detection module and determine its position and size; Step 6: The target object in the image is identified and classified by the target recognition module according to the extracted feature information, the aggregated multi-granularity global knowledge and the target detection result; and the target image is displayed by the display module.

8. The multi-granularity low-quality image target recognition method according to claim 7, characterized in that: The multi-granularity feature fusion module method is as follows: 1) Feature map extraction: The feature maps of different scales are obtained through the convolutional neural network (CNN) deep learning model; 2) Knowledge transfer: Send local multi-granularity prototypes through a distributed sending and receiving algorithm, and receive multi-granularity global class prototypes and global soft labels generated by the server; 3) Feature Fusion: A method for fusing feature maps of different scales together; fusion is achieved in a variety of ways, including cascading and weighted averaging; 4) Improvement of network structure: Improvements in network structure: Changes in network structure are made by increasing the number of network layers and changing the way networks are connected. 5) Optimization of feature fusion: The feature fusion method is optimized by changing the feature fusion weight and increasing the number of feature fusion layers.

9. The multi-granularity low-quality image target recognition method according to claim 7, characterized in that: The target recognition module method is as follows: CNN convolutional neural network is used for feature extraction and classification. The total loss consisting of classification loss, multi-granularity feature-prototype comparison learning loss and soft label loss is used to train the model with a large amount of data to learn the features and representation of the image. The most representative algorithms include Faster R-CNN, YOLO, and SSD; The Faster R-CNN generates candidate boxes by introducing a region proposal network (RPN) and uses CNN for feature extraction and classification. The YOLO method transforms the target detection problem into a regression problem, and simultaneously locates and classifies the target through a separate network.

10. The multi-granularity low-quality image target recognition method according to claim 7, characterized in that: The SSD combines fast and accurate object detection algorithms by introducing multi-scale feature maps, multiple prior boxes, and multi-granularity global knowledge for object recognition.

Images