Cultural relic classification and identification system and method based on image recognition and natural language processing
Through image recognition and natural language processing technology, combined with pre-trained models and database matching algorithms, efficient and accurate classification and identification of ancient cultural relics are achieved, and the problems of low efficiency and low accuracy in the existing technology are solved, and systematic cultural relics analysis and report generation.
Patent Information
- Application Number
- CN202411554560.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-04
- Publication Date
- 2024-11-29
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing technology cannot efficiently and accurately carry out automatic classification and identification of ancient cultural relics, and is inefficient and has low accuracy.
The cultural relics classification and identification system based on image recognition and natural language processing is adopted, including image acquisition, image processing, feature extraction and feature matching modules. Through pre-training models and database matching algorithms, the visual features and text features of cultural relics are merged and matched. Generate classification identification results.
It improves the accuracy and efficiency of cultural relics identification, provides systematic cultural relics image acquisition and analysis reports, enhances user interaction experience, and continuously optimizes system performance through feedback learning.
Smart Images

Figure CN119048854A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing, and in particular to a system and method for classifying and identifying cultural relics based on image recognition and natural language processing. Background Art
[0002] In the research and management of cultural relics, the classification and identification of cultural relics is a basic and important task. Traditional classification and identification work mostly relies on the experience and knowledge of experts, which is not only inefficient, but also easily affected by subjective factors, and the accuracy and objectivity cannot be fully guaranteed. With the advancement of science and technology, especially the rapid development of computer vision and artificial intelligence technology, automatic image recognition technology has been widely used in many fields, such as medical image analysis, biometric authentication and surveillance video analysis, which provides new technical means for the automatic classification and identification of cultural relics. However, ancient cultural relics are of various types and shapes, and their images are highly complex. At present, there is still a lack of a system that can automatically classify and identify ancient cultural relic images efficiently and accurately; at the same time, most of the existing related technologies are still in the theoretical research stage or the preliminary application stage, which is difficult to meet the needs of practical work.
[0003] The current classification and identification of cultural relics is inefficient and inaccurate, which is a technical problem that needs to be solved. Summary of the invention
[0004] The technical task of the present invention is to address the above shortcomings and provide a cultural relics classification and identification system and method based on image recognition and natural language processing to solve the technical problems of low efficiency and low accuracy in the previous cultural relics classification and identification.
[0005] In a first aspect, the present invention provides a cultural relics classification and identification system based on image recognition and natural language processing, comprising an image acquisition module, an image processing module, a feature extraction module, a feature matching module and a result output module, wherein the result output module provides a visual interface; The image acquisition module is used to obtain the cultural relic images collected under the shooting rules, and perform image preprocessing on the collected cultural relic images to obtain the final cultural relic images, and send the final cultural relic images to the image processing module; The image processing module is used to pre-process the cultural relic images, extract and classify the features of the cultural relics through the pre-trained cultural relic pre-classification model, fuse the features of related types to obtain feature vectors, and perform dimensionality reduction optimization on the feature vectors to obtain feature subsets for cultural relic classification and identification; The feature extraction module is used to take the feature subset as input, analyze the visual features of the cultural relics through the pre-trained cultural relics visual feature extraction model, obtain the visual features of the cultural relics, and take the feature subset and the cultural relics visual features as input, analyze the historical background and cultural significance of the cultural relics through the pre-trained natural language model, obtain the text features of the cultural relics, and merge the visual features and text features into a feature vector of fixed length as the cultural relics feature vector output; The feature matching module is configured with a cultural relic database, which records cultural relic images, basic cultural relic information, cultural relic descriptions, cultural relic feature vectors, cultural relic categories and cultural relic age information, and is used to perform feature matching based on the cultural relic feature vector of the cultural relic to be tested and the cultural relic feature vector in the cultural relic database, and output the classification and identification results of the cultural relic to be tested, and the classification and identification results include the age and category of the cultural relic; The result output module is used to display the cultural relic images of the cultural relic to be tested and the classification and identification results of the cultural relic to be tested, generate a cultural relic report based on the cultural relic images of the cultural relic to be tested and the classification and identification results, and display the cultural relic report.
[0006] Preferably, the image acquisition module is used to perform the following steps to obtain the cultural relic image acquired under the shooting rule: Taking the image of cultural relics reflecting the outline and status of cultural relics as the input image, the category and shooting of the current cultural relics are analyzed by the morphological classification algorithm to obtain the morphological classification results including the category of cultural relics, image RGB and image shooting parameters; Based on the morphological classification results, the shooting of cultural relics is analyzed and adjusted through the pre-built shooting rule database, and shooting rules and light and shadow adjustment suggestions are output. The shooting rule database records the shooting rules and light and shadow requirements of each type of cultural relic in various shooting environments; Get images of cultural relics collected under shooting rules and light and shadow adjustment suggestions.
[0007] Preferably, for the image acquisition module, the image preprocessing of the cultural relic image includes grayscale correction, color balance and image denoising of the cultural relic image.
[0008] Preferably, the image acquisition module supports the user to configure the shooting scheme of the cultural relic image through the user interface, and supports the user to adjust and optimize the collected cultural relic image in real time, and adjusts the clarity, light intensity and shooting angle of the cultural relic image through adjustment and optimization; Among them, the shooting plan is used to limit the shooting equipment and shooting content of the cultural relics image. The shooting equipment includes digital cameras with high dynamic range and high-resolution capture capabilities, scanners, infrared cameras, and LED lighting systems with adjustable color temperature and brightness. The shooting content is used to limit the shooting angle of the cultural relics to obtain the three-dimensional structure and style details of the cultural relics.
[0009] Preferably, the image processing module is used to perform the following preprocessing on the cultural relic image: The image is subjected to denoising operation and contrast and brightness adjustment through image preprocessing, wherein the denoising operation removes random noise in the cultural relic image based on a convolutional neural network, and the contrast and brightness of the cultural relic image are adjusted based on a histogram equalization technique; The cultural relic image after image preprocessing is subjected to image enhancement, including edge enhancement and texture enhancement. The edge enhancement of the cultural relic image is performed based on an edge detection algorithm, and the texture enhancement of the cultural relic image is performed through a high-pass filter or a Gabor filter.
[0010] Preferably, the cultural relic pre-classification model is a network model constructed based on a CNN algorithm, which is used to extract deep features of the input cultural relic image to obtain features reflecting the shape, texture and color of the cultural relic, and classify the features to obtain feature categories; The image processing module is used to reduce the dimension of the feature vector by the principal component analysis method or the t-SNE method to remove redundant information, and to perform feature evaluation and screening on the feature vector after dimension reduction based on a machine learning algorithm to obtain a feature subset for classification and identification of cultural relics, wherein the machine learning algorithm includes a support vector machine and a random forest.
[0011] Preferably, the cultural relic visual feature extraction model is a network model constructed by introducing a CNN algorithm with an attention mechanism, and is used to extract features of the cultural relics based on an input feature subset, and obtain visual features that can reflect the edges, colors, textures, specific patterns, inscriptions, and shapes of the cultural relics; The natural language model is a language model built based on the BERT model or GPT, which is used to analyze the attribute features of the cultural relics based on the input feature subset and visual features, and output text features that reflect the historical background and cultural significance of the cultural relics; The feature extraction module is used to combine visual features and text features into a feature vector of fixed length as text feature output through feature vectorization technology.
[0012] As a preferred embodiment, the cultural relics database is configured with a cultural relics basic information table, a cultural relics feature vector table, a cultural relics category table and a cultural relics chronology table, and the cultural relics integrated information table is used to record the cultural relics basic information, the cultural relics basic information includes the cultural relics number, name, category, age and cultural relics image path, the cultural relics feature vector table is used to record the cultural relics characteristics, the cultural relics category table is used to record the category information to which the cultural relics belong, and the cultural relics chronology table is used to record the historical age information to which the cultural relics belong; The feature matching module is configured with feature matching algorithms, including cosine similarity, Euclidean distance and Manhattan distance. The feature matching module is used to support the selection of a matching algorithm to calculate the vector similarity between the cultural relic feature vector of the cultural relic to be tested and the cultural relic feature vector in the cultural relic database, compare the calculated vector similarity with a preset threshold, and select a matching cultural relic feature vector from the cultural relic database; and is used to support the calculation of the vector similarity between the cultural relic feature vector of the cultural relic to be tested and the cultural relic feature vector in the cultural relic database by each matching algorithm, and sum the similarities corresponding to various matching algorithms as the final generated vector similarity, compare the final generated vector similarity with a preset threshold, and select a matching cultural relic feature vector from the cultural relic database; correspondingly, the feature matching module is used to determine the category information and age information of the cultural relic based on the matched feature vector.
[0013] When calculating the vector similarity between the cultural relic feature vector of the cultural relic to be tested and the cultural relic feature vector in the cultural relic database, a weight is set for each element in the cultural relic feature vector, and the vector similarity between the cultural relic feature vector of the cultural relic to be tested and the cultural relic feature vector in the cultural relic database is calculated by weighting the elements of the feature vector.
[0014] Preferably, the result output module includes a result visualization submodule, a report generation submodule, a user interaction submodule and a feedback learning submodule; The result visualization submodule realizes dynamic data visualization based on SVG or Canvas technology and JavaScript programming, and is used to display the classification and identification results of cultural relics in the form of charts and images. The report generation submodule is used to generate cultural relic reports in various formats based on preconfigured analysis report templates. The contents of the cultural relic reports include cultural relic images, classification and identification results of cultural relics, and text descriptions generated based on natural language technology. The contents of the text descriptions include relevant data analysis of cultural relics, similar cultural relic information, and historical background interpretation; The user interaction submodule provides a visual interface, and provides query and classification identification of cultural relic images through the cooperation of the AI chat robot, the image processing module, the feature extraction module, and the feature matching module, and receives user feedback; The feedback learning submodule is used to adjust the models and algorithms configured in the image acquisition module, the image processing module, the feature extraction module and the feature matching module based on user feedback.
[0015] In a second aspect, the present invention provides a method for classifying and identifying cultural relics based on image recognition and natural language processing, which is used to provide classification and identification of ancient cultural relics through a cultural relics classification and identification system based on image recognition and natural language processing as described in any one of the first aspects.
[0016] The cultural relics classification and identification system and method based on image recognition and natural language processing of the present invention have the following advantages: 1. Collect cultural relic images under shooting rules and pre-process the cultural relic images so that the collected cultural relic images meet the requirements for cultural relic identification, which helps to improve the accuracy of identification; 2. The image acquisition module is equipped with a pre-built shooting rule database. For the cultural relics to be tested, an image showing the outline and state of the cultural relics is taken as the input image. The category and shooting of the current cultural relics are analyzed through the morphological classification algorithm. The analysis results and the shooting rule database are decomposed to analyze and adjust the shooting of the cultural relics. The shooting rules and light and shadow adjustment suggestions are output, so that users can shoot the cultural relics according to the shooting rules and light and shadow adjustment suggestions, so that users can quickly, accurately and systematically realize the cultural relic image acquisition of different cultural relics; 3. The image processing module can provide four operations for cultural relic images: image preprocessing, image enhancement, feature extraction and feature optimization. It uses the most advanced natural language processing and image recognition technology to process and analyze images of ancient cultural relics, thus achieving rapid and accurate identification of ancient cultural relics. 4. In the image enhancement stage, edge enhancement and texture enhancement techniques are used to emphasize the key features of the cultural relic images. Feature optimization uses dimensionality reduction technology and feature selection algorithms to optimize feature vectors and improve the accuracy of identification and classification. 5. In the feature extraction module, the visual features of cultural relics are analyzed through the pre-trained cultural relic visual feature extraction model to obtain the visual features of the cultural relics. The historical background and cultural significance of the cultural relics are analyzed through the pre-trained natural language model to obtain the text features of the cultural relics, thus achieving a comprehensive analysis of the cultural relics and improving the accuracy of cultural relic identification and classification; 6. The feature matching module is equipped with a cultural relic database and a variety of matching algorithms. The matching algorithm can be selected as needed to match the cultural relic feature vector of the cultural relic to be tested with the cultural relic feature vector in the cultural relic database, query similar cultural relics to obtain the classification and identification results of the cultural relics, and realize efficient and accurate matching of the cultural relic features in the database with the cultural relic features to be identified; 7. When matching features in the feature matching module, it supports feature weighted calculation, multi-matching algorithm fusion and threshold adjustment, which improves the accuracy and efficiency of matching and ensures the reliability of cultural relics identification; 8. The result output module includes the result visualization submodule, report generation submodule, user interaction submodule and feedback and learning submodule. It can display the results of cultural relic identification and classification in the form of intuitive charts and image comparisons, generate detailed analysis reports, and continuously improve system performance through user feedback, further improving the accuracy of cultural relic identification; 9. The result output module provides an AI chatbot, which improves the user interaction experience and facilitates users to query and understand cultural relics. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0018] The present invention is further described below in conjunction with the accompanying drawings.
[0019] Figure 1 This is a structural diagram of a cultural relics classification and identification system based on image recognition and natural language processing in Example 1. DETAILED DESCRIPTION
[0020] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it. However, the embodiments are not intended to limit the present invention. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments may be combined with each other.
[0021] The embodiments of the present invention provide a system and method for classifying and identifying cultural relics based on image recognition and natural language processing, which are used to solve the technical problems of low efficiency and low accuracy in current classification and identification of cultural relics.
[0022] Embodiment 1: The present invention provides a cultural relics classification and identification system based on image recognition and natural language processing, including an image acquisition module, an image processing module, a feature extraction module, a feature matching module and a result output module, wherein the result output module provides a visual interface.
[0023] The image acquisition module is used to obtain the cultural relic images collected under the shooting rules, and perform image preprocessing on the collected cultural relic images to obtain the final cultural relic images, and send the final cultural relic images to the image processing module.
[0024] As a specific implementation of the image acquisition module, the module is used to perform the following steps to obtain the cultural relic image acquired under the shooting rules: (1) Using a cultural relic image that reflects the outline and status of the cultural relic as the input image, the morphological classification algorithm is used to analyze the category and shooting of the current cultural relic, and the morphological classification results including the cultural relic category, image RGB, and image shooting parameters are obtained; (2) Based on the morphological classification results, the shooting of cultural relics is analyzed and adjusted through the pre-built shooting rule database, and shooting rules and light and shadow adjustment suggestions are output. The shooting rule database records the shooting rules and light and shadow requirements for each type of cultural relic in various shooting environments; (3) Obtain images of cultural relics collected according to shooting rules and light and shadow adjustment suggestions.
[0025] As a specific implementation of the image acquisition module, the module preprocesses the image of the cultural relic, including grayscale correction, color balance and image denoising.
[0026] As a specific implementation of the image acquisition module, the module supports users to configure the shooting plan of cultural relic images through the user interface, and supports users to adjust and optimize the collected cultural relic images in real time, and adjust the clarity, light intensity and shooting angle of the cultural relic images through adjustment and optimization. Among them, the shooting plan is used to limit the shooting equipment and shooting content of the cultural relic images. The shooting equipment includes a digital camera with high dynamic range and high resolution capture capability, a scanner, an infrared camera, and an LED lighting system with adjustable color temperature and brightness. The shooting content is used to limit the shooting angle of the cultural relic to obtain the three-dimensional structure and style details of the cultural relic.
[0027] In the classification and identification of ancient cultural relics, the image acquisition module plays a vital role. It is responsible for collecting high-quality image data from ancient cultural relics for subsequent processing and analysis. The image acquisition module of this embodiment meets the following requirements: High-resolution image acquisition: High-performance digital cameras and scanners are used to obtain high-resolution images of cultural relics to ensure that the images contain sufficient details to support accurate feature extraction. These devices can work under different lighting conditions to ensure that the image quality is not affected by the external environment.
[0028] Multi-angle shooting: Based on the different forms of different cultural relics, machine learning methods are used to determine the multi-angle shooting plan using the classification network, and the images of cultural relics are captured from different perspectives to obtain more comprehensive cultural relic information. These perspective data are particularly important for understanding the three-dimensional structure and style details of cultural relics.
[0029] Non-contact acquisition technology: Non-contact image acquisition technologies such as laser scanning and infrared imaging are used to obtain surface texture and internal structure information of cultural relics. These technologies are essential for protecting fragile and precious cultural relics, while providing data types that cannot be obtained by traditional photography methods.
[0030] Image preprocessing: Basic preprocessing operations are introduced during the image acquisition phase, including but not limited to grayscale correction, color balance, and image denoising, to improve the image recognition rate and the efficiency of subsequent processing. This step ensures the quality of the collected image data and lays a solid foundation for feature extraction and database matching.
[0031] Real-time feedback mechanism: Design a real-time feedback mechanism that allows operators to instantly evaluate image quality during acquisition and make necessary adjustments. This includes image clarity, lighting intensity, and optimization of shooting angles.
[0032] The technical implementation of the image acquisition module involves carefully designed hardware equipment, including but not limited to a customized lighting system, a multi-directional adjustable camera stand, and high-performance image processing software. All equipment and software are selected and configured in accordance with strict standards for cultural relics protection to ensure that the highest possible quality image data is obtained without causing any damage to the cultural relics. The equipment selection and configuration are designed as follows: (1) High-performance digital camera: Choose a digital camera with high dynamic range (HDR) and high-resolution capture capabilities, as well as a lens compatible with macro photography to capture the details of the cultural relics; (2) Customized lighting system: Design an LED lighting system with adjustable color temperature and brightness, and use image processing technology to dynamically adjust the light source to provide balanced and soft light, reduce image damage and reflections, and improve image quality; (3) Software development and configuration: Develop dedicated image acquisition software and integrate it into the image acquisition module to achieve the following functions: (3-1) Automatic control: Automatically adjust camera parameters, including exposure time, ISO sensitivity, focus mode, etc., to adapt to different shooting conditions; (3-2) Image preprocessing function: Integrate image preprocessing algorithms to automatically correct and optimize the collected images, thereby improving the efficiency and accuracy of subsequent processing.
[0033] The image processing module is used to preprocess the cultural relic images, extract and classify the features of the cultural relics through the pre-trained cultural relic pre-classification model, fuse the features of related types to obtain the feature vector, and perform dimensionality reduction optimization on the feature vector to obtain the feature subset used for cultural relic classification and identification.
[0034] As a specific implementation of the image processing module, the module is used to perform the following preprocessing of the cultural relic image: (1) Performing denoising operations and adjusting contrast and brightness of cultural relic images through image preprocessing. In particular, during the denoising operation, random noise in the cultural relic image is removed based on a convolutional neural network, and the contrast and brightness of the cultural relic image are adjusted based on a histogram equalization technique; (2) Performing image enhancement on the cultural relic image after image preprocessing, including edge enhancement and texture enhancement. The edge enhancement of the cultural relic image is performed based on an edge detection algorithm, and the texture enhancement of the cultural relic image is performed by high-pass filtering or Gabor filtering.
[0035] The cultural relic pre-classification model in this embodiment is a network model constructed based on the CNN algorithm, which is used to extract deep features of the input cultural relic image, obtain features reflecting the shape, texture and color of the cultural relic, and classify the features to obtain feature categories. The image processing module is used to reduce the dimension of the feature vector by the principal component analysis method or the t-SNE method to remove redundant information, and to perform feature evaluation and screening on the feature vector after dimension reduction based on the machine learning algorithm to obtain a feature subset for cultural relic classification and identification, wherein the machine learning algorithm includes a support vector machine and a random forest.
[0036] The image processing module is the core component of the automatic classification and identification system of ancient cultural relics, and is responsible for realizing the key transformation from image acquisition to feature extraction. The image acquisition module of this embodiment processes and analyzes the images of ancient cultural relics based on the most advanced natural language processing and image recognition technology, thereby realizing the rapid and accurate identification of the age, source and category of ancient cultural relics. The image processing module provides four service links: image preprocessing, image enhancement, feature extraction and feature optimization. Each link adopts the most advanced algorithms and technologies to ensure the accuracy and efficiency of image processing.
[0037] Image preprocessing is the first step in the image processing process, which aims to improve image quality by removing noise, adjusting contrast and brightness, and preparing high-quality input for subsequent processing steps. The specific techniques are as follows: (1) Denoising: Using deep learning-based denoising algorithms, such as convolutional neural network (CNN) denoising technology, to effectively remove random noise in the image and retain the detailed features of the cultural relic image; (2) Adjust contrast and brightness: Use histogram equalization technology to automatically adjust the contrast and brightness of the image, improve the visual effect of the image, and make the details of the cultural relics clearer and more visible.
[0038] The image enhancement stage aims to further enhance the characteristics of cultural relics in the image through technical means to facilitate feature extraction. The technical steps include: (1) Edge enhancement: Sobel, Canny and other edge detection algorithms are used to emphasize the edge information of the cultural relic image, laying the foundation for subsequent feature extraction; (2) Texture enhancement: Through high-pass filtering or Gabor filter and other technologies, the texture features of cultural relic images are enhanced to highlight their unique features.
[0039] Feature extraction is a key part of the image processing module, and its purpose is to extract effective feature information that can be used for classification and identification from the enhanced image. The technical details are as follows: (1) Feature extraction based on deep learning: Using deep learning models such as convolutional neural networks (CNNs) to automatically learn and extract deep features of cultural relic images. These deep features can fully reflect information such as the shape, texture, and color of cultural relics, providing strong support for accurate classification and identification; (2) Feature fusion: In order to more comprehensively describe the characteristics of cultural relics, feature fusion technology is used to merge the features extracted from different levels to form a final, high-dimensional feature vector.
[0040] The feature optimization part is to improve the accuracy of identification and classification. The feature optimization step optimizes the feature representation through dimensionality reduction technology and feature selection algorithm. It specifically includes the following two operations: (1) Dimensionality reduction: Use principal component analysis (PCA), t-SNE and other techniques to reduce the dimension of feature vectors, remove redundant information, and retain the most critical discriminant information; (2) Feature selection: Based on machine learning algorithms, such as support vector machines (SVM) or random forests, evaluate and select the subset of features that are most helpful for the classification and identification of cultural relics.
[0041] The feature extraction module is used to take the feature subset as input, perform visual feature analysis on the cultural relics through a pre-trained cultural relic visual feature extraction model to obtain the visual features of the cultural relics, and take the feature subset and the cultural relic visual features as input, analyze the historical background and cultural significance of the cultural relics through a pre-trained natural language model to obtain the text features of the cultural relics, and merge the visual features and text features into a feature vector of fixed length as the output of the cultural relic feature vector.
[0042] As a specific implementation, the cultural relic visual feature extraction model is a network model constructed by introducing the CNN algorithm with an attention mechanism, which is used to extract features of cultural relics based on the input feature subset, and obtain visual features that can reflect the edges, colors, textures, specific patterns, inscriptions, and shapes of cultural relics. The natural language model is a language model built based on the BERT model or GPT, which is used to analyze the attribute characteristics of cultural relics based on the input feature subset and visual features, and output text features that reflect the historical background and cultural significance of cultural relics. The feature extraction module is used to merge visual features and text features into a fixed-length feature vector as a text feature output through feature vectorization technology.
[0043] The feature extraction module of this embodiment mainly extracts key information from the image of ancient cultural relics through deep learning algorithm and natural language processing technology. The module first receives the pre-processed cultural relic image transmitted from the image processing module, and then extracts the deep features of the image through a series of deep neural network models, especially convolutional neural network (CNN) and attention mechanism. Next, the historical background, cultural significance and other descriptive information of the cultural relics are analyzed in combination with natural language processing technology, and finally a comprehensive feature vector is formed for further analysis and comparison by the database matching module. The technology provided by this module includes the following: (1) Deep Convolutional Neural Network (CNN): CNN is the cornerstone technology for image recognition and classification. This embodiment uses a modified CNN architecture to extract visual features of cultural relic images. By superimposing multiple convolutional layers and pooling layers, the model can capture features at different levels from local to global. Each convolutional layer can extract specific features of the image, such as edges, colors, textures, etc. As the network level deepens, the extracted features also change from simple to complex, gradually forming a high-level feature representation representing the cultural relic itself. (2) Attention mechanism: In order to further improve the accuracy and efficiency of feature extraction, the attention mechanism is introduced on the basis of CNN. The attention mechanism allows the model to focus on the most informative areas in the image, such as specific patterns, inscriptions or shapes of cultural relics. In this way, the model can not only reduce the interference of irrelevant information such as background, but also more accurately capture key features, providing more accurate data support for subsequent classification and identification work; (3) Natural language processing technology In addition to image features, the value and classification of ancient cultural relics are also closely related to their historical background, cultural significance and other non-visual information. Therefore, in the feature extraction module, natural language processing technology, such as pre-trained language models (such as BERT, GPT) and text classification algorithms, are used to analyze and extract key information in the description of cultural relics. Through in-depth understanding of these text data, the attributes and characteristics of cultural relics can be more comprehensively grasped, and then combined with visual features to form a more comprehensive representation of cultural relic features; (4) Feature fusion and vectorization: After extracting the visual features and text features of the cultural relics, the next step is to effectively fuse these two types of features to form a unified feature representation vector. This process is achieved through feature vectorization technology, that is, the extracted visual features and text features are merged through a certain algorithm and converted into a feature vector of fixed length. This feature vector contains both the visual information of the image and the cultural and historical information of the cultural relics, providing a rich and accurate data foundation for the database matching module.
[0044] The feature matching module is configured with a cultural relic database, which records cultural relic images, basic cultural relic information, cultural relic descriptions, cultural relic feature vectors, cultural relic categories and cultural relic age information. It is used to perform feature matching based on the cultural relic feature vector of the cultural relic to be tested and the cultural relic feature vector in the cultural relic database, and output the classification and identification results of the cultural relic to be tested, which include the age and category of the cultural relic.
[0045] As a specific implementation of the cultural relics database, it is configured with a cultural relics basic information table, a cultural relics feature vector table, a cultural relics category table and a cultural relics chronology table. The cultural relics integrated information table records the basic information of cultural relics. The basic information of cultural relics includes the cultural relics number, name, category, age and cultural relics image path. The cultural relics feature vector table is used to record the characteristics of cultural relics, the cultural relics category table is used to record the category information of the cultural relics, and the cultural relics chronology is used to record the historical age information of the cultural relics.
[0046] As a specific implementation of the feature matching module, the module is configured with feature matching algorithms, including cosine similarity, Euclidean distance and Manhattan distance. The feature matching module is used to support the selection of a matching algorithm to calculate the vector similarity between the cultural relic feature vector of the cultural relic to be tested and the cultural relic feature vector in the cultural relic database, compare the calculated vector similarity with a preset threshold, and select a matching cultural relic feature vector from the cultural relic database; and is used to support the calculation of the vector similarity between the cultural relic feature vector of the cultural relic to be tested and the cultural relic feature vector in the cultural relic database through each matching algorithm, and sum up the similarities corresponding to various matching algorithms as the final generated vector similarity, compare the final generated vector similarity with a preset threshold, and select a matching cultural relic feature vector from the cultural relic database; correspondingly, the feature matching module is used to determine the category information and age information of the cultural relic based on the matched feature vector.
[0047] When calculating the vector similarity between the cultural relic feature vector of the cultural relic to be tested and the cultural relic feature vector in the cultural relic database, a weight is set for each element in the cultural relic feature vector, and the vector similarity between the cultural relic feature vector of the cultural relic to be tested and the cultural relic feature vector in the cultural relic database is calculated by weighting the elements of the feature vector.
[0048] The feature matching module is a key link in achieving the accuracy of cultural relics identification. The design and implementation of this module aims to efficiently and accurately match the cultural relics features output by the feature extraction module with the cultural relics features in the database through advanced natural language processing and image recognition technology, so as to achieve accurate identification of information such as the age, source, and category of the cultural relics. In this embodiment, the feature matching module provides processes such as the selection and optimization of matching algorithms, database design, and information processing processes.
[0049] Database design is the prerequisite for achieving accurate matching. To achieve this goal, this embodiment constructs a multi-dimensional cultural relic feature database structure, which can store both basic cultural relic image data and multi-dimensional feature data (such as shape, pattern, material, text description, etc.) extracted by image recognition and natural language processing technology. The cultural relic database design structure is as follows: The cultural relic feature database will use a relational database management system (RDBMS) to take advantage of its efficient query capabilities and mature technical support. The database will contain the following core tables: (1) Cultural Relics Basic Information Table: stores the basic information of cultural relics, such as the cultural relic number, name, category, age, and cultural relic image path; (2) Cultural Relics Feature Vector Table: stores feature vectors of cultural relics, which are extracted from cultural relic images and descriptions using image processing and natural language processing techniques; (3) Cultural Relics Category Table: stores information about the categories to which cultural relics may belong, to facilitate classification and retrieval; (4) Cultural Relics Chronology: records the historical period to which the cultural relics belong.
[0050] The above-mentioned cultural relic database can effectively support the subsequent matching process and realize fast and efficient data retrieval. In order to improve the efficiency of database matching, it is necessary to index the key fields in the database. In particular, the cultural relic feature vector table can greatly improve the speed of similarity search by establishing a reasonable indexing strategy.
[0051] The core of the database matching module is to realize an efficient and accurate matching algorithm. This embodiment is based on a similarity calculation method based on deep learning, which can accurately measure the similarity between different cultural relics features. Specific algorithm selection includes but is not limited to: (1) Cosine similarity: The cosine value between the feature vectors of two artifacts is calculated to evaluate their similarity. (2) Euclidean distance: Calculate the Euclidean distance between two feature vectors in multidimensional space. The smaller the distance, the higher the similarity. (3) Manhattan distance: Calculates the sum of the absolute values of the differences between vectors in each dimension. It is applicable to feature vectors with the same importance in each dimension.
[0052] In view of the diversity and complexity of cultural relics features, the selected algorithms are optimized to improve the accuracy and efficiency of matching. The optimization measures include: (1) Feature weighting: weight the elements in the feature vector according to the discriminative power of different features to enhance the system's sensitivity to important features; (2) Multi-algorithm fusion: Combining multiple similarity calculation methods to increase the reliability of identification through algorithm fusion; (3) Dynamic threshold adjustment: Based on the statistical data of the matching results, the similarity threshold is dynamically adjusted to achieve better matching results.
[0053] In summary of the information processing process, the information processing flow of the feature matching module is as follows: (1) Feature extraction: Obtain the cultural relic feature vector from the image processing module.
[0054] (2) Feature conversion and standardization: The extracted feature vectors are converted and standardized to meet the requirements of database matching; (3) Feature matching: Match the standardized feature vector with the feature vector of the cultural relics stored in the database, and calculate the similarity using the selected matching algorithm; (4) Result analysis: Based on the similarity results returned by the matching algorithm, analyze and determine the most likely category, age, and other information of the cultural relics; (5) Result output: The results of the matching analysis are output to the result output module for user reference or further academic research.
[0055] The result output module is used to display the cultural relic images of the cultural relic to be tested and the classification and identification results of the cultural relic to be tested, generate a cultural relic report based on the cultural relic images of the cultural relic to be tested and the classification and identification results, and display the cultural relic report.
[0056] As a specific implementation of the result output module, the module includes a result visualization submodule, a report generation submodule, a user interaction submodule and a feedback learning submodule.
[0057] The result visualization submodule is based on SVG or Canvas technology and combines JavaScript programming to achieve dynamic data visualization display, which is used to display the classification and identification results of cultural relics in the form of charts and image comparisons.
[0058] In this embodiment, the result visualization submodule uses SVG or Canvas technology, combined with JavaScript programming to achieve dynamic data visualization, ensuring a good interactive experience and high-definition visual effects. The results of cultural relics identification and classification are intuitively displayed in the form of charts, image comparisons, etc., including but not limited to chronological flow charts, classification radar charts, etc. In addition, in order to facilitate in-depth analysis by professionals, high-definition cultural relics images and feature highlighting functions will also be provided to make the details and features of the cultural relics more prominent.
[0059] The report generation submodule is used to generate cultural relics reports in various formats based on preconfigured analysis report templates. The contents of the cultural relics reports include cultural relic images, classification and identification results of cultural relics, and text descriptions generated based on natural language technology. The contents of the text descriptions include relevant data analysis of cultural relics, similar cultural relics information, and historical background interpretation.
[0060] In the present embodiment, the report generation submodule can automatically generate a detailed analysis report according to user needs. Utilize template engine technology to support the generation of reports in various formats such as PDF and HTML. Through natural language generation (NLG) technology, the text description in the report will be closer to human language and easy to understand. These reports not only contain the basic information and classification and identification results of cultural relics, but also provide the relevant data analysis, similar cultural relics information comparison and possible historical background interpretation of this result, which can be used for cultural relics protection, research and teaching.
[0061] The user interaction submodule provides a visual interface, and provides an AI chat robot. Through the cooperation of the AI chat robot and the image processing module, feature extraction module, and feature matching module, it provides query and classification identification of cultural relics images and receives user feedback.
[0062] In this embodiment, the user interaction submodule is designed with a friendly user interface, and a responsive web page is built using the latest Web front-end framework Vue.js to ensure that the user interface is beautiful and easy to use. At the same time, the latest NLP technology is integrated to optimize the interactive capabilities of the AI chatbot. Users can easily query results, submit new cultural relic images for analysis, and customize the content and format of the analysis report as needed. At the same time, by integrating the AI chatbot, users can interact with the system, such as asking about the detailed identification process of cultural relics, providing feedback, etc.
[0063] The feedback learning submodule is used to adjust the models and algorithms configured in the image acquisition module, image processing module, feature extraction module and feature matching module based on user feedback.
[0064] The feedback and learning submodule in this embodiment is the cornerstone of the continuous optimization of the result output module. This module not only collects user feedback information to adjust and improve the user experience, but also analyzes user usage habits and feedback content through machine learning technology, continuously improves the accuracy of cultural relics data analysis and the interaction quality of the system, and combines deep learning technology to continuously train the model through user usage and feedback data, automatically optimize data matching and analysis algorithms, so as to improve the overall performance of the system and user satisfaction.
[0065] Traditional cultural relics classification and identification work mainly relies on manual experience, which is not only time-consuming, but also difficult to ensure accuracy and efficiency. The system of this embodiment can perform in-depth analysis of cultural relic images, extract key features, match them with cultural relic information in the database, and realize automatic classification and identification through advanced image recognition technology. The classification and identification of cultural relics through this system can greatly improve the speed and accuracy of cultural relic identification, and provide strong technical support for the research, protection and management of ancient cultural relics.
[0066] In the system of this embodiment, a database containing a large amount of cultural relic data is constructed. The database includes not only images of cultural relics, but also detailed description information corresponding to them, such as age, source, category, etc. The establishment of this database provides a rich and reliable reference for the automatic classification and identification of cultural relics. The database will be continuously updated to include more cultural relic information and improve the identification scope and accuracy of the system.
[0067] In addition to image recognition technology, the system of this embodiment also introduces natural language processing technology, which can process and understand the semantic content in the description information of cultural relics, such as the historical background and artistic style of the cultural relics. By combining the analysis results of image recognition and natural language processing technology, the system can comprehensively evaluate and identify cultural relics from multiple dimensions, greatly improving the comprehensiveness and depth of identification.
[0068] The system of this embodiment provides a user-friendly interactive interface. Users only need to upload images of cultural relics to quickly obtain classification and identification results of cultural relics, including the age, source, category and related historical document information of the cultural relics. This interactive method greatly reduces the user threshold, allowing non-professionals to easily use this system to identify cultural relics.
[0069] Through rapid and accurate classification and identification of cultural relics, the system of this embodiment not only provides technical support for cultural relic protection, but also provides a solid foundation for further research on cultural relics. The cultural relic information identified by the system can provide important clues for researchers and help to deeply explore the historical and cultural value behind the cultural relics. At the same time, the application of the system can also improve the efficiency of cultural relic management and optimize resource allocation.
[0070] The system of this embodiment combines natural language processing with image recognition technology, which not only improves the efficiency and accuracy of cultural relic identification, but also opens up new directions for subsequent technical research and application. This will promote the continuous innovation and development of cultural relic identification technology and bring more possibilities for cultural relic protection and research.
[0071] By comprehensively applying the most advanced natural language processing and image recognition technologies, a new and efficient technical means is provided for the automatic classification and identification of cultural relics. The system of this embodiment can not only improve the efficiency and accuracy of cultural relic identification, promote the in-depth protection and research of cultural relics, but also contribute to the innovation and development of the field of cultural relic scientific and technological protection.
[0072] Embodiment 2: The present invention provides a method for classifying and identifying cultural relics based on image recognition and natural language processing, which provides classification and identification of ancient cultural relics through the system disclosed in Embodiment 1.
[0073] The present invention has been shown and described in detail above through the accompanying drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Based on the above multiple embodiments, those skilled in the art can know that the means in the above different embodiments can be combined to obtain more embodiments of the present invention, and these embodiments are also within the protection scope of the present invention.
Claims
1. A cultural relics classification and identification system based on image recognition and natural language processing, characterized in that: It includes an image acquisition module, an image processing module, a feature extraction module, a feature matching module and a result output module, and the result output module provides a visual interface; The image acquisition module is used to obtain the cultural relic images collected under the shooting rules, and perform image preprocessing on the collected cultural relic images to obtain the final cultural relic images, and send the final cultural relic images to the image processing module; The image processing module is used to pre-process the cultural relic images, extract and classify the features of the cultural relics through the pre-trained cultural relic pre-classification model, fuse the features of related types to obtain feature vectors, and perform dimensionality reduction optimization on the feature vectors to obtain feature subsets for cultural relic classification and identification; The feature extraction module is used to take the feature subset as input, analyze the visual features of the cultural relics through the pre-trained cultural relics visual feature extraction model, obtain the visual features of the cultural relics, and take the feature subset and the cultural relics visual features as input, analyze the historical background and cultural significance of the cultural relics through the pre-trained natural language model, obtain the text features of the cultural relics, and merge the visual features and text features into a feature vector of fixed length as the cultural relics feature vector output; The feature matching module is configured with a cultural relic database, which records cultural relic images, basic cultural relic information, cultural relic descriptions, cultural relic feature vectors, cultural relic categories and cultural relic age information, and is used to perform feature matching based on the cultural relic feature vector of the cultural relic to be tested and the cultural relic feature vector in the cultural relic database, and output the classification and identification results of the cultural relic to be tested, and the classification and identification results include the age and category of the cultural relic; The result output module is used to display the cultural relic images of the cultural relic to be tested and the classification and identification results of the cultural relic to be tested, generate a cultural relic report based on the cultural relic images of the cultural relic to be tested and the classification and identification results, and display the cultural relic report.
2. The cultural relics classification and identification system based on image recognition and natural language processing according to claim 1 is characterized in that: The image acquisition module is used to perform the following steps to obtain the cultural relic image acquired under the shooting rules: Taking the image of cultural relics reflecting the outline and status of cultural relics as the input image, the category and shooting of the current cultural relics are analyzed by the morphological classification algorithm to obtain the morphological classification results including the category of cultural relics, image RGB and image shooting parameters; Based on the morphological classification results, the shooting of cultural relics is analyzed and adjusted through the pre-built shooting rule database, and shooting rules and light and shadow adjustment suggestions are output. The shooting rule database records the shooting rules and light and shadow requirements of each type of cultural relic in various shooting environments; Get images of cultural relics collected under shooting rules and light and shadow adjustment suggestions.
3. The cultural relics classification and identification system based on image recognition and natural language processing according to claim 1 is characterized in that: For the image acquisition module, the image preprocessing of the cultural relic image includes grayscale correction, color balancing and image denoising.
4. The cultural relics classification and identification system based on image recognition and natural language processing according to claim 1 is characterized in that: The image acquisition module supports the user to configure the shooting plan of the cultural relic image through the user interface, and supports the user to adjust and optimize the collected cultural relic image in real time, and adjusts the clarity, light intensity and shooting angle of the cultural relic image through adjustment and optimization; Among them, the shooting plan is used to limit the shooting equipment and shooting content of the cultural relics image. The shooting equipment includes digital cameras with high dynamic range and high-resolution capture capabilities, scanners, infrared cameras, and LED lighting systems with adjustable color temperature and brightness. The shooting content is used to limit the shooting angle of the cultural relics to obtain the three-dimensional structure and style details of the cultural relics.
5. The cultural relics classification and identification system based on image recognition and natural language processing according to claim 1 is characterized in that: The image processing module is used to perform the following preprocessing on the cultural relic image: The image is subjected to denoising operation and contrast and brightness adjustment through image preprocessing, wherein the denoising operation removes random noise in the cultural relic image based on a convolutional neural network, and the contrast and brightness of the cultural relic image are adjusted based on a histogram equalization technique; The cultural relic image after image preprocessing is subjected to image enhancement, including edge enhancement and texture enhancement. The edge enhancement of the cultural relic image is performed based on an edge detection algorithm, and the texture enhancement of the cultural relic image is performed through a high-pass filter or a Gabor filter.
6. The cultural relics classification and identification system based on image recognition and natural language processing according to claim 1 is characterized in that: The cultural relic pre-classification model is a network model built based on the CNN algorithm, which is used to extract deep features of the input cultural relic image, obtain features reflecting the shape, texture and color of the cultural relic, and classify the features to obtain feature categories; The image processing module is used to reduce the dimension of the feature vector by the principal component analysis method or the t-SNE method to remove redundant information, and to perform feature evaluation and screening on the feature vector after dimension reduction based on a machine learning algorithm to obtain a feature subset for classification and identification of cultural relics, wherein the machine learning algorithm includes a support vector machine and a random forest.
7. The cultural relics classification and identification system based on image recognition and natural language processing according to claim 1 is characterized in that: The cultural relic visual feature extraction model is a network model constructed by introducing a CNN algorithm with an attention mechanism, which is used to extract features of cultural relics based on an input feature subset, and obtain visual features that can reflect the edges, colors, textures, specific patterns, inscriptions, and shapes of cultural relics; The natural language model is a language model built based on the BERT model or GPT, which is used to analyze the attribute features of the cultural relics based on the input feature subset and visual features, and output text features that reflect the historical background and cultural significance of the cultural relics; The feature extraction module is used to combine visual features and text features into a feature vector of fixed length as text feature output through feature vectorization technology.
8. The cultural relics classification and identification system based on image recognition and natural language processing according to claim 1 is characterized in that: The cultural relics database is configured with a cultural relics basic information table, a cultural relics feature vector table, a cultural relics category table and a cultural relics chronology table. The cultural relics integrated information table records the basic information of cultural relics. The basic information of cultural relics includes the cultural relics number, name, category, age and cultural relics image path. The cultural relics feature vector table is used to record the characteristics of cultural relics. The cultural relics category table is used to record the category information of the cultural relics. The cultural relics chronology table is used to record the historical age information of the cultural relics. The feature matching module is configured with feature matching algorithms, including cosine similarity, Euclidean distance and Manhattan distance. The feature matching module is used to support the selection of a matching algorithm to calculate the vector similarity between the feature vector of the cultural relic to be tested and the feature vector of the cultural relic in the cultural relic database, compare the calculated vector similarity with a preset threshold, and select a matching cultural relic feature vector from the cultural relic database; and to support the calculation of the vector similarity between the feature vector of the cultural relic to be tested and the feature vector of the cultural relic in the cultural relic database by each matching algorithm, and sum the similarities corresponding to various matching algorithms as the final generated vector similarity, compare the final generated vector similarity with a preset threshold, and select a matching cultural relic feature vector from the cultural relic database; correspondingly, the feature matching module is used to determine the category information and age information of the cultural relic based on the matched feature vector; When calculating the vector similarity between the cultural relic feature vector of the cultural relic to be tested and the cultural relic feature vector in the cultural relic database, a weight is set for each element in the cultural relic feature vector, and the vector similarity between the cultural relic feature vector of the cultural relic to be tested and the cultural relic feature vector in the cultural relic database is calculated by weighting the elements of the feature vector.
9. The cultural relics classification and identification system based on image recognition and natural language processing according to claim 1 is characterized in that: The result output module includes a result visualization submodule, a report generation submodule, a user interaction submodule and a feedback learning submodule; The result visualization submodule realizes dynamic data visualization based on SVG or Canvas technology and JavaScript programming, and is used to display the classification and identification results of cultural relics in the form of charts and images. The report generation submodule is used to generate cultural relic reports in various formats based on preconfigured analysis report templates. The contents of the cultural relic reports include cultural relic images, classification and identification results of cultural relics, and text descriptions generated based on natural language technology. The contents of the text descriptions include relevant data analysis of cultural relics, similar cultural relic information, and historical background interpretation; The user interaction submodule provides a visual interface, and provides query and classification identification of cultural relic images through the cooperation of the AI chat robot, the image processing module, the feature extraction module, and the feature matching module, and receives user feedback; The feedback learning submodule is used to adjust the models and algorithms configured in the image acquisition module, the image processing module, the feature extraction module and the feature matching module based on user feedback.
10. A method for classifying and identifying cultural relics based on image recognition and natural language processing, characterized in that: Used to provide classification and identification of ancient cultural relics through a cultural relic classification and identification system based on image recognition and natural language processing as described in any one of claims 1-9.
Citation Information
Patent Citations
Cultural relic image recognition system and method based on composite features
CN115661824A
Generative large model-based natural language interactive security video retrieval system and device
CN118551077A
Intelligent identification system for 3D objects
CN118644707A
Method for batch identification and automatic classification and archiving of picture content
CN118799619A
Cited By
AI-driven ancient coin identification method and system
CN120510620A
Yuan blue-and-white decoration gene pool construction and AI derivative design system and method
CN120599405A
Jade ware micromark detection method and device based on multi-modal data fusion
CN120685645A
A social perception-based cultural relic resource colony identification method
CN122780782A
