Intelligent image recognition method and system based on big data

Abstract

The invention discloses an intelligent image recognition method and an intelligent image recognition system based on big data. The method comprises the following steps: preprocessing an image, performing deep feature extraction, constructing progressive retrieval through a hierarchical filtering mechanism, obtaining a plurality of Top-K images through multi-feature joint hash generation of an LSH layer and graph structure similarity calculation of a GSN layer, outputting the Top-K images, and automatically updating training data and model parameters according to user confirmation information. According to the method, depth feature extraction, big data indexing and a similarity judgment mechanism are combined, through data-driven model training and feature optimization, image objects similar to the target image in visual semantics can be quickly recognized, and the recognition efficiency and accuracy are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing and big data technology, and particularly relates to a technical method and system for identifying whether a target picture is similar to a picture in big data, which is suitable for application scenarios such as image search and visual retrieval. BACKGROUND

[0002] With the development of image recognition and computer vision, image retrieval has become one of the core research fields. In real-world applications, users often need to determine whether a certain target picture has appeared in a historical database or whether there is a highly similar image. However, existing technologies mainly rely on manual feature extraction or inefficient full-image traversal comparison, and when faced with large-scale image databases, there are problems such as slow recognition, low accuracy, and poor scalability. Therefore, there is an urgent need for an image similarity identification system that combines deep learning feature extraction and big data vector comparison technology to efficiently determine whether a target picture is similar to an image in a database. SUMMARY

[0003] The present application provides an intelligent image recognition method and system based on big data, which combines deep feature extraction, big data indexing, and similarity judgment mechanisms. Through data-driven model training and feature optimization, it can quickly identify image objects that are similar to the target image in visual semantics, improving recognition efficiency and accuracy.

[0004] An intelligent image recognition method based on big data includes the following steps:

[0005] S1 image input and acquisition: obtaining image data to be identified;

[0006] S2 image preprocessing: normalizing the size, enhancing the color, and segmenting the image to be identified into background and target regions;

[0007] S3 deep feature extraction: extracting multi-layer feature quantities with semantic discriminability of the background and target through a pre-trained or self-trained deep neural network model, i.e., extracting shallow features and deep features of the background region and the target region, respectively. The shallow features contain edge texture information, and the deep features contain object and scene semantic information. Cross-modal feature fusion is achieved through a cross-attention mechanism, and the edge texture and semantic information features are balanced through learnable weight coefficients.

[0008] The feature vector is defined as:

[0009] F = a · f s + β · f d

[0010] where f s represents shallow features, and f dDeep features are represented, and a and b are learnable weight coefficients.

[0011] S4 Similarity calculation and comparison: Through the hierarchical filtering mechanism, a progressive retrieval is constructed. Through the multi-feature joint hashing generation of the LSH layer and the graph structure similarity calculation of the GSN layer, the first stage performs similarity comparison between the feature vectors of the background region in the image and the feature vectors of the images in the database. The cosine similarity is used for scoring, and the Top-N similarity of several images is used as the comparison image set. In the second stage, the feature vector of the target region is compared with the feature vector of the comparison image set composed of the Top-N similarity of several images in the background region. The cosine similarity is used for scoring to obtain the accurate similarity score result. Then the similarity score results of the first and second stages are weighted and fused, and the final ranking of the candidate images is performed based on the fusion score. Top-K images are obtained based on the scoring results.

[0012] S5 Similar image output: Output the Top-K similarity of several images and their metadata.

[0013] S6 Model optimization feedback: Automatically update the training data and model parameters according to the user confirmation information.

[0014] S1 Image input and collection: User upload (JPEG / PNG), API interface call (HTTP / REST), etc.

[0015] S4 Similarity calculation and comparison: The scoring strategy is that the LSH layer accounts for 40%, and the GSN layer accounts for 60%.

[0016] An intelligent image recognition system for implementing the above method, comprising: an image collection module, an image preprocessing module, a feature extraction module, an image feature database, a similarity matching module, an identification result output module, and a model optimization module.

[0017] The image collection module is used to receive and upload target images, supports multiple upload methods, and can be a file upload interface (front-end upload JPG / PNG), an API interface call (HTTP / REST), etc.

[0018] The image preprocessing module is used to complete format conversion and image enhancement, and uniformly converts the format of the input image to a standard format, including image size normalization, color channel correction, noise filtering, image enhancement (brightness contrast, sharpening), and using Mask R-CNN to segment the image into background region and target region.

[0019] The feature extraction module is used to extract the shallow and deep features of the background region and the target region and fuse the feature vectors; the feature extraction module comprises a feature extraction network submodule, a feature partition module, a cross-attention fusion module, and a feature weighting combination module. The feature partition module is used to divide the feature space of an input image into regions, divide the image into two subdomains of a background region and a target region, so that the feature extraction network submodule extracts and processes the features of each region respectively; the feature extraction network submodule extracts the features of the background region and the target region based on a pre-trained or self-trained deep neural network model, respectively, and obtains shallow features and deep features, the shallow features contain low-level visual information such as edges and textures, and are suitable for local detail recognition; the deep features contain high-level semantic information such as objects and scenes, and are suitable for semantic recognition and classification, and the feature extraction network submodule can be implemented by using a convolutional neural network (CNN), a visual Transformer (ViT), a Swin Transformer, etc. The cross-attention fusion module is used to realize the cross-semantic level association between the shallow features and the deep features, introduce a cross-attention mechanism to calculate the attention weights between different levels of features, and guide the information transmission between the features according to the attention weights, so as to realize the dynamic fusion of the shallow details and the deep semantics. The feature weighting combination module is used to weight and fuse the shallow features and the deep features according to learnable weight coefficients, generate a unified high-dimensional feature vector, and is defined as follows:

[0020] F=α·f s +β·f d

[0021] Wherein, f s represents the shallow features, f d represents the deep features, and α and β are learnable weight coefficients.

[0022] The image feature database module is used to store the vector information and metadata of a large number of images.

[0023] The similarity matching module is used to calculate the similarity score between the to-be-identified image and the images in the database, and to filter out the most similar images. The similarity matching module includes a feature input module, a multi-feature hash index module and a first-stage similarity calculation module, a graph structure similarity enhancement module and a second-stage similarity calculation module, a similarity fusion module, the feature input module is used to receive and input the background region feature vector and the target region feature vector of the to-be-identified image and the target region feature vector of the database image, and the corresponding feature vector set of the image in the database; the feature hash index module and the first-stage similarity calculation module are used to perform multi-modal joint hash coding on the image feature vector, construct an efficient feature index structure through a locality sensitive hash (LSH) mechanism, and compare the background region feature vector of the to-be-identified image with the background region feature vector of the image in the database by cosine similarity, to obtain the first-stage similarity score result, and select the Top-N similarity ranked images according to the score result to form a comparison image set; the graph structure similarity enhancement module and the second-stage similarity calculation module use a graph neural network to model the target region of the to-be-identified image and the images in the comparison image set, compare the target region feature vector of the to-be-identified image with the image feature vector in the Top-N comparison image set obtained by the first-stage screening by cosine similarity, to obtain the second-stage accurate similarity score result, and update the similarity ranking of the final matching image based on the result; the similarity fusion module is used to weight and fuse the similarity score results of the first stage and the second stage, and based on the fusion score, the candidate images are finally ranked, and a number of images ranked Top-K in similarity are selected according to the score result.

[0024] The identification result output module is used to display similar images and information; the identification result output module includes a Top-K image screening unit, a metadata extraction and association unit, and a result output interface; the Top-K image screening unit outputs the final similarity score result, and extracts a number of images ranked Top-K in similarity as the final retrieval result; the metadata extraction and association unit extracts and outputs the corresponding metadata information of the selected Top-K images, including but not limited to image number, source path, label information, timestamp, location parameter, etc.; the result output interface provides a unified data output interface, supports outputting similar images and their metadata to an upper module, a terminal device or a user interface in a visual form or a structured data form; this module also includes an interface adaptation mechanism, supports standard data protocol docking with a database, a front-end display system and an API service system, and ensures efficient transmission and compatible display of similar image retrieval results.

[0025] The model optimization module is used for dynamically adjusting system recognition performance according to user interaction feedback, realizing continuous learning and model adaptive updating. The model optimization module comprises an incremental training data generation unit and a model update execution unit; the incremental training data generation unit generates positive / negative sample pairs according to user feedback for subsequent fine-tuning; the model update execution unit updates neural network model parameters based on new samples through transfer learning or fine-tuning; the module supports continuous learning, meta-learning (Meta-Learning) or light fine-tuning strategies, realizing long-term performance maintenance of the model under low resource conditions. BRIEF DESCRIPTION OF DRAWINGS

[0026] Figure 1 A flowchart of the intelligent image recognition method based on big data of the application; DETAILED DESCRIPTION

[0027] Figure 1 A flowchart of the intelligent image recognition method based on big data of the application, the intelligent image recognition method based on big data of the application comprises the following steps:

[0028] S1 image input and acquisition: acquiring image data to be recognized;

[0029] Receiving image data input by a user, supporting formats such as JPEG and PNG. Image input modes support various interfaces, including a file upload interface and API interface (such as HTTP / REST) calling, etc.

[0030] S2 image preprocessing: performing size normalization, color enhancement and segmentation on the image, and dividing the image into a background and a target;

[0031] Performing preprocessing operations on the image, including size normalization, color channel enhancement, brightness contrast improvement, sharpening processing and noise filtering. The image is segmented by using Mask R-CNN to obtain images of the background region and the target region.

[0032] S3 deep feature extraction: extracting multi-layer feature quantities with semantic distinguishability of the background and the target through a pre-trained or self-trained deep neural network model, wherein shallow layer features contain edge texture information, and deep layer features contain object and scene semantic information; cross-modal feature fusion is realized through a cross-attention mechanism, and edge texture and semantic information features are balanced through a learnable weight coefficient.

[0033] The feature vector is defined as:

[0034] F=α·f s +β·f d

[0035] Wherein, f s represents shallow layer features, and f dDeep features are represented, and a and β are learnable weight coefficients.

[0036] S4 Similarity calculation and comparison: Progressive image retrieval is constructed through a hierarchical filtering mechanism. Through multi-feature joint hashing of the LSH layer and graph structure similarity calculation of the GSN layer, the similarity of the image feature vectors in the database of the background region features of the image is compared first. Cosine similarity is used for scoring, and the Top-N similar images are used as the comparison image set. Then, the feature vector of the target region is compared with the feature vector of the comparison image set composed of the Top-K similar images, and cosine similarity is used for scoring.

[0037] Preferably, in the present application, the similarity calculation module is based on a hierarchical filtering mechanism to construct a progressive image retrieval process. First, through the local sensitive hashing (LSH) mechanism, the extracted multi-image feature vectors are jointly hashed to form a hash index table for fast rough screening, so as to realize the preliminary compression and approximate matching of the feature space. Then, it enters the graph structure similarity network (GSN) layer, and the graph modeling method is used to build the graph of multi-dimensional semantic relationship, and high-precision similarity calculation is performed in the structure. In the first stage, the background region feature vector of the image is compared with the background feature vector in the pre-constructed image feature database. The comparison process preferably uses cosine similarity as the measurement standard, and all candidate images are scored and sorted according to the calculation results, and the top N image samples are selected to form a preliminary comparison image set. In the second stage, the feature vector of the target region in the image to be identified is compared with the preliminary comparison image set again, and cosine similarity is used as the similarity score index, so as to realize further screening and sorting of the candidate images. This staged comparison strategy can effectively improve the accuracy and calculation efficiency of the similar image retrieval, and reduce the false matching rate. Then, the similarity score results of the first and second stages are weighted and fused, and the candidate images are finally sorted based on the fusion score, and the Top-K images are obtained according to the score results.

[0038] S5 Similar image retrieval and output: Output the Top-K similar images and their metadata;

[0039] The final screened Top-K similar images and their corresponding metadata information (such as image ID, label, source path, timestamp, etc.) are provided to the upper layer system or user front-end interface through a unified data output interface, supporting structured data or image display forms.

[0040] S6 Model optimization feedback: automatically update the training data and model parameters according to the user confirmation information.

[0041] Incremental training samples are dynamically generated according to the interactive feedback (such as confirmation / denial, scoring, etc.) of the user to the retrieval results, and are used for fine-tuning training of the deep model. This module supports strategies such as continuous learning, transfer learning and meta-learning, and maintains the long-term recognition performance of the system.

[0042] The S1 image input and collection supports image input and collection, user upload (JPEG / PNG), API interface call (HTTP / REST), etc.

[0043] The S4 similarity calculation and comparison has a scoring strategy of LSH layer accounting for 40% and GSN layer accounting for 60%.

[0044] An intelligent image recognition system implementing the above method, comprising: an image collection module, an image preprocessing module, a feature extraction module, an image feature database, a similarity matching module, an identification result output module, and a model optimization module.

[0045] The image collection module is used to receive and upload target images, supports multiple upload methods, and can be a file upload interface (front-end upload JPG / PNG), an API interface call (HTTP / REST), etc.

[0046] The image preprocessing module is used to complete format conversion and image enhancement, uniformly converts the format of the input image to a standard format, including image size normalization, color channel correction, noise filtering, image enhancement (brightness contrast, sharpening), and uses Mask R-CNN to segment the image into background and target regions.

[0047] The feature extraction module comprises a feature extraction network submodule, a feature partition module, a cross-attention fusion module, and a feature weighted combination module. The feature extraction network submodule extracts hierarchical features of background regions and target regions from an input image based on a pre-trained or self-trained deep neural network model, and obtains shallow features and deep features, respectively. The shallow features contain low-level visual information such as edges and textures, and are suitable for local detail recognition. The deep features contain high-level semantic information such as objects and scenes, and are suitable for semantic recognition and classification. The feature extraction network submodule can be implemented by using a convolutional neural network (CNN), a visual Transformer (ViT), a Swin Transformer, or the like. The cross-attention fusion module is used to realize cross-semantic level association between the shallow features and the deep features. By introducing a cross-attention mechanism, the attention weights between different levels of features are calculated, and the information transmission between the features is guided accordingly, so as to realize dynamic fusion of shallow details and deep semantics. The feature weighted combination module is used to weight and combine the shallow features and the deep features according to learnable weight coefficients, and generate a unified high-dimensional feature vector, which is defined as follows:

[0048] F=α·f s +β·f d

[0049] wherein f s represents the shallow features, f d represents the deep features, and α and β are learnable weight coefficients.

[0050] The image feature database module is used to store vector information and metadata of a large number of images.

[0051] The similarity matching module is used to calculate the similarity score between the target image and the images in the feature database, and to filter out the most similar images. The similarity matching module includes a feature input module, a multi-feature hash index module and a first stage similarity calculation module, a graph structure similarity enhancement module and a second stage similarity calculation module, a similarity fusion module, the feature input module is used to receive and input the background area feature vector and the target area feature vector of the image to be identified and the corresponding feature vector set of the image in the database; the feature hash index module and the first stage similarity calculation module are used to perform multi-modal joint hash coding on the image feature vector, construct an efficient feature index structure through the locality sensitive hash (Locality Sensitive Hashing, LSH) mechanism, and compare the background area feature vectors of the image to be identified and the background area feature vectors of the images in the database by cosine similarity, to obtain the first stage similarity score result, and select the Top-N similarity ranked images according to the score result to form a comparison image set; the graph structure similarity enhancement module and the second stage similarity calculation module use a graph neural network to model the target area of the image to be identified and the images in the comparison image set, compare the target area feature vector of the image to be identified and the image feature vector in the Top-N comparison image set obtained by the first stage screening by cosine similarity, to obtain the second stage accurate similarity score result, and update the similarity ranking of the final matching image accordingly; the similarity fusion and output module is used to weight and fuse the similarity score results of the first stage and the second stage, and based on the fusion score, the candidate images are finally sorted, and according to the score result, a number of images ranked Top-K in similarity are selected.

[0052] The identification result output module is used to display similar images and information; the identification result output module includes a Top-K image screening unit, a metadata extraction and association unit, a result output interface, and an interface adaptation mechanism: the Top-K image screening unit outputs the final similarity score result, and extracts a number of images ranked Top-K in similarity as the final retrieval result; the metadata extraction and association unit extracts and outputs the corresponding metadata information of the selected Top-K images, including but not limited to image number, source path, label information, timestamp, location parameter, etc.; the result output interface provides a unified data output interface, supports outputting similar images and their metadata to upper layer modules, terminal devices or user interfaces in a visual form or a structured data form; the interface adaptation mechanism supports standard data protocol docking with databases, front-end display systems and API service systems, to ensure efficient transmission and compatible display of similar image retrieval results.

[0053] The model optimization module is used to realize adaptive training and continuous learning.

[0054] The model optimization module is used to dynamically adjust the system recognition performance according to user interaction feedback, realize continuous learning and model adaptive update. The module includes an incremental training data generation unit and a model update execution unit; the incremental training data generation unit generates positive / negative sample pairs according to user feedback for subsequent fine-tuning; the model update execution unit updates the neural network model parameters based on new samples through transfer learning or fine-tuning; the module supports continuous learning, meta-learning (Meta-Learning) or light fine-tuning strategy, and realizes long-term performance maintenance of the model under low resource condition.

[0055] The following takes the image of a horse running on the grass as an example to illustrate the method of the application:

[0056] (1) the user uploads an image,

[0057] (2) the system unifies the size to 224x224 and enhances the contrast of the image, and automatically segments the image into "background area (grass)" and "target area (horse)" by Mask R-CNN;

[0058] (3) shallow features fs (such as grass texture and horse edge) and deep features (such as semantic classification as "animal" and "grass") are extracted for the two areas respectively;

[0059] The final feature vector is generated after feature fusion:

[0060] F=0.4*f s +0.6*f d

[0061] (4) the system performs fast hash screening through the LSH layer, and then performs fine structure matching through the GSN layer, and sequentially executes background area and target area comparison, after the first stage comparison of the background area, the Top-N candidate image set is obtained, the target area is compared again in the second stage, then the similarity score results of the first stage and the second stage are weighted and fused, and the candidate images are finally sorted based on the fusion score, and the Top-K image set is obtained based on the score result.

[0062] (5) output the Top-K image set and its metadata

[0063] (6) the user selects the most matched one from the results, and the system records the feedback for model continuous optimization training.

Claims

1. An intelligent image recognition method based on big data, characterized in that: The following steps are involved: S1 Image input and acquisition: obtain image data to be identified; S2 Image preprocessing: normalize the size, enhance the color, and segment the image to be identified to obtain the background area and the target area; S3 Deep Feature Extraction: Use a pre-trained or self-trained deep neural network model to extract shallow features and deep features from the background area and target area respectively, where the shallow features are edge texture information and the deep features are object and scene semantic information; the shallow features and deep features are fused through a cross-attention mechanism to obtain a fused feature vector F, and the shallow features and deep features are balanced through learnable weight coefficients. The feature vector is defined as: F=α·f s +β·f d Among them, f s represents shallow features, f d represents deep features, and α and β are learnable weight coefficients. S4 Similarity calculation and comparison: Through the multi-feature joint hash generation of the LSH layer and the graph structure similarity calculation of the GSN layer, the first stage compares the background area feature quantity in the image to be identified with the image feature vector in the database, uses cosine similarity to score, and screens out several images with the top-N similarity as the comparison image set; the second stage compares the feature vector of the target area with the feature vector of the comparison image set consisting of several images with the top-N similarity in the background area, uses cosine similarity to score, obtains an accurate similarity score result, performs a weighted fusion on the similarity score results of the first and second stages, and finally sorts the candidate images based on the fusion score, and obtains several top-K images based on the score result; S5 similar image output: outputs several images with top-K similarity and their metadata; S6 model optimization feedback: Automatically update training data and model parameters based on user confirmation information.

2. The method according to claim 1, characterized in that S4 similarity calculation and comparison, α is the LSH layer accounting for 40%, and β is the GSN layer accounting for 60%.

3. The method according to claim 1, characterized in that The S2 image preprocessing uses Mask R-CNN to segment the image to be identified and distinguish between the background area and the target area.

4. The method according to claim 1, wherein The deep neural network model in the S3 deep feature extraction step is a convolutional neural network (CNN), a visual transformer (ViT) or a Swin transformer structure.

5. An intelligent image recognition system based on big data, characterized in that: include: Image acquisition module, used to collect images to be identified, which is a file upload interface (front-end upload JPG / PNG) or API interface call (HTTP / REST); Image preprocessing module, which performs image size normalization, color enhancement, and uses Mask R-CNN to segment the image into background and target areas; Feature extraction module, used to extract shallow features and deep features of background area and target area and fuse feature vectors; Image feature database module, used to store vector information and metadata of massive images; The similarity matching module is used to calculate the similarity score between the image to be identified and the images in the database, and to select the most similar images; Recognition result output module, used to display similar images and information; The model optimization module is used to dynamically adjust the system recognition performance based on user interaction feedback.

6. The intelligent image recognition system based on big data according to claim 5, characterized in that: The feature extraction module includes a feature extraction network submodule, a feature partitioning module, a cross attention fusion module, and a feature weighted combination module; The feature extraction network submodule performs hierarchical feature extraction on the background area and target area of ​​the input image based on a pre-trained or self-trained deep neural network model; the cross-attention fusion module is used to fuse shallow features with deep features; the feature weighted combination module is used to perform weighted fusion of shallow features with deep features based on learnable weight coefficients, and the fusion feature vector F is defined as follows: F=α·f s +β·f d Among them, f s represents shallow features, f d represents deep features, and α and β are learnable weight coefficients.

7. The intelligent image recognition system based on big data according to claim 5, characterized in that: The similarity matching module includes a feature input module, a multi-feature hash index module and a first-stage similarity calculation module, a graph structure similarity enhancement module and a second-stage similarity calculation module, and a similarity fusion module. The feature input module is used to receive and input the feature vector of the image to be identified and the background area feature vector of the database image and the target area feature vector target, as well as the corresponding feature vector set of the image in the database; the feature hash index module and the first-stage similarity calculation module are used to perform multimodal joint hash coding on the image feature vector, and use locality sensitive hashing (LSH) to obtain the image feature vector. The LSH (Sensitive Hashing) mechanism constructs an efficient feature index structure and performs a cosine similarity comparison between the background area feature vector of the image to be identified and the background area feature vector of the image in the database to obtain a similarity scoring result of the first stage, and selects several images ranked in the top-N similarity based on the scoring result to form a comparison image set; the graph structure similarity enhancement module and the second stage similarity calculation module use a graph neural network to model the target area of ​​the image to be identified and the images in the comparison image set, and performs a cosine similarity comparison between the target area feature vector of the image to be identified and the image feature vectors in the top-N comparison image set obtained by screening in the first stage to obtain an accurate similarity scoring result of the second stage, and updates the similarity ranking of the matching images accordingly; the similarity fusion module is used to perform a weighted fusion on the similarity scoring results of the first stage and the second stage, and finally sorts the candidate images based on the fusion score, and selects several images ranked in the top-K similarity based on the scoring result.

Images