Intelligent recognition method of paper documents based on full perception

Abstract

The present invention discloses a method for intelligent recognition of paper documents based on full perception, including using optical character recognition technology to convert text in an image into an editable format. Using pressure sensors and surface scanning technology to measure the thickness and surface texture of paper. Using spectral analysis technology to detect the chemical composition of paper and ink. Through the acoustic sensor module, the sound waves generated during writing are captured to analyze the paper texture and writing pressure. The data obtained by each sensor are integrated to construct a multimodal data set. The data processing and deep learning module is input for preprocessing, feature extraction and pattern recognition to generate preliminary document feature data. The semantic understanding and document analysis module is input to parse the document content and context. The document protection and management suggestion module is input to evaluate the preservation status and potential risks, and generate protection and preservation suggestions. The present invention ensures the long-term preservation and value maintenance of documents, and greatly improves the efficiency and accuracy of document management.

Description

Technical Field

[0001] The present invention relates to the technical field of document intelligent recognition, and in particular to a paper document intelligent recognition method based on full perception. Background Art

[0002] With the rapid development of information technology, the demand for digitalization and intelligent processing of paper documents is increasing. Among many technologies, optical character recognition (OCR) technology has always been an important means of document digitization. Traditional OCR systems usually capture digital images of documents through image acquisition modules, improve image quality through preprocessing steps, and then use pattern recognition algorithms to convert text in the image into editable text. However, existing OCR technology still has certain limitations when processing complex document structures and diverse content.

[0003] First, existing OCR technology mainly focuses on text recognition and lacks the ability to fully perceive and analyze the physical and chemical properties of documents. These properties include paper material, printing technology, ink characteristics, etc., which are important for the preservation, authentication and historical value assessment of documents. However, existing technologies do not pay enough attention to this, resulting in the inability to provide comprehensive and accurate analysis results when processing historical documents and other important documents.

[0004] Secondly, for documents with complex formats, existing OCR and intelligent document processing (IDP) technologies still have much room for improvement in recognition rate and accuracy. Documents with complex formats may contain multiple fonts, images, tables, and handwritten annotations, which places higher demands on existing technologies. When processing these complex documents, existing technologies often find it difficult to maintain high-precision recognition, resulting in information loss or misrecognition.

[0005] In addition, although existing IDP systems attempt to parse the semantic content of documents, they still have significant deficiencies in understanding complex contexts, metaphors, or the deep meaning of documents. When dealing with texts involving complex contexts or metaphors, these systems often have difficulty accurately understanding and interpreting the true meaning of documents, limiting their application in advanced information retrieval and content analysis.

[0006] In addition, the existing technology also has obvious deficiencies in the identification of the physical and chemical properties of documents. The physical and chemical properties of paper are crucial to the preservation and identification of documents, such as the thickness, texture, chemical composition of paper and the chemical substances of ink. However, the existing OCR and IDP technologies are difficult to obtain and analyze these key physical and chemical information, resulting in limited applications in document preservation and protection. Therefore, how to provide a paper document intelligent recognition method based on full perception is a problem that technicians in this field urgently need to solve. Summary of the invention

[0007] One purpose of the present invention is to propose a method for intelligent recognition of paper documents based on full perception, which ensures the long-term preservation and value maintenance of documents and greatly improves the efficiency and accuracy of document management.

[0008] A paper document intelligent recognition method based on full perception according to an embodiment of the present invention includes the following steps:

[0009] S1, an initialization step of the multimodal sensor module, using the optical sensor submodule, the tactile sensor submodule, the chemical sensor submodule and the acoustic sensor submodule to synchronously scan the paper document to obtain the visual image, physical perception data, chemical information and sound data of the paper document;

[0010] S2, using an optical sensor module, using a camera and a scanning device, to convert text in a digital image into an editable format through optical character recognition technology;

[0011] S3, combined with the tactile sensor module, uses pressure sensors and surface scanning technology to measure the thickness and surface texture of paper;

[0012] S4, using the chemical sensor module, detecting the chemical substances of paper and ink through spectral analysis technology and chemical sensors to generate a chemical property profile;

[0013] S5. Use the acoustic sensor module to capture the sound waves generated when writing, analyze the paper texture and writing pressure, and use the audio analysis function to analyze the vibration caused by document writing or interaction;

[0014] S6, integrating the data acquired from the optical sensor module, the tactile sensor module, the chemical sensor module, and the acoustic sensor module to construct a multimodal data set;

[0015] S7, inputting the acquired multimodal data set into the data processing and deep learning module for preprocessing, and standardizing and cleaning the multimodal data set through the data preprocessing submodule, removing noise, adjusting brightness, and performing normalization processing;

[0016] S8, extract visual features from visual images in the multimodal dataset using the feature extraction submodule, and extract relevant features from acoustic and chemical data using the sound and chemical analysis tools;

[0017] S9, classify and identify the extracted multimodal data set through the pattern recognition submodule combined with the machine learning model to generate preliminary document feature data;

[0018] S10, inputting the preliminary document feature data into the semantic understanding and document analysis module, parsing the document content through the text parsing submodule, and extracting the structure and semantic information;

[0019] S11. Use the context understanding submodule to analyze the deep meaning and context of the document based on the document content and external knowledge base, and combine historical data and cultural background knowledge to perform semantic analysis and understanding;

[0020] S12, inputting the parsed semantic information and document feature data into the document protection and management suggestion module, and evaluating the preservation status and potential risks of the document through the condition evaluation submodule;

[0021] S13. Based on the evaluation results, the protection suggestion submodule generates preservation and restoration suggestions for the document type, and provides appropriate temperature, humidity conditions and processing methods;

[0022] S14. Output a complete document identification and protection recommendation report to support subsequent document management and maintenance work.

[0023] Optionally, S2 includes the following steps:

[0024] S21, using the high-definition camera and scanning device in the optical sensor module to capture high-resolution images of paper documents and generate digital images of the documents ;

[0025] S22. Digital images Perform image preprocessing:

[0026] Apply the adaptive denoising algorithm to remove the noise in the image and obtain the denoised image :

[0027] ;

[0028] in, is the denoising coefficient, Indicates that the image is The second derivative in the direction, is the direction weight, n is the image dimension;

[0029] Use a multi-scale sharpening filter to enhance the edge details of the image and generate a sharpened image :

[0030] ;

[0031] in, is the sharpening factor, is the multi-scale Gaussian weight, For is the multi-scale Gaussian filtering result of the standard deviation, M is the number of scales;

[0032] Execute the adaptive contrast adjustment function to optimize the contrast of the image and obtain the preprocessed image :

[0033] ;

[0034] in, and Represent the maximum and minimum pixel values ​​of the image, respectively. and are the maximum and minimum values ​​of the target contrast range, is the contrast adjustment factor, is the standard deviation of the image;

[0035] S23, the preprocessed image Input into the optical character recognition software for character recognition processing, and analyze the pre-processed image through the optical character recognition software engine , identify text regions in images , the text area Convert the characters in to machine-readable text , generate preliminary editable text ;

[0036] S24. Use natural language processing technology to process preliminary editable text Perform spelling and grammar correction and generate corrected text :

[0037] ;

[0038] in, Indicates that it is based on preliminary text The probability of is the i-th word in the text, and N is the total number of words in the text;

[0039] Apply formatting algorithms to the corrected text Perform typesetting, adjust paragraphs, fonts and spacing, and generate final editable text :

[0040] ;

[0041] in, It represents the typesetting weight of the text paragraph, and M is the total number of text paragraphs.

[0042] Optionally, S3 includes the following steps:

[0043] S31, using the pressure sensor in the tactile sensor module to measure the pressure of the paper document to generate paper thickness data ;

[0044] ;

[0045] Where A is the area of ​​the paper, is the pressure value of the i-th measuring point, is the displacement of the i-th measuring point, and N is the total number of measuring points;

[0046] S32. Scan the surface of the paper document using surface scanning technology to generate surface texture data ;

[0047] ;

[0048] in, is the three-dimensional coordinate of the jth scanning point, and M is the total number of scanning points;

[0049] S33, the paper thickness data of the pressure sensor and surface texture data generated by surface scanning technology Fusion to build a physical property dataset of paper :

[0050] ;

[0051] in, , , is the fusion coefficient, represents the outer product of tensors;

[0052] S34. Physical property data set Perform feature extraction to generate paper thickness features and surface texture features :

[0053] ;

[0054] ;

[0055] in, and is the feature extraction weight, and They are and The mean of and They are and The standard deviation of is the surface angle;

[0056] S35. Extract the thickness feature and surface texture features Integrate to generate comprehensive feature data :

[0057] ;

[0058] in, , , is the integration coefficient, Represents a convolution operation.

[0059] Optionally, S4 includes the following steps:

[0060] S41. Use the spectral analysis device in the chemical sensor module to scan the paper document and obtain the spectral data of the paper and ink. :

[0061] ;

[0062] in, is the i-th spectral wavelength, is the reflectivity at this wavelength, and L is the total number of measured wavelengths;

[0063] S42. Spectral data Preprocessing is performed to improve the signal-to-noise ratio of the spectral data through baseline correction and denoising, and the preprocessed spectral data is obtained. :

[0064] ;

[0065] in, represents the minimum value in the spectral data, is the denoising coefficient, is the standard deviation of the spectral data;

[0066] S43. Use chemical analysis tools to analyze the preprocessed spectral data Perform component analysis to identify chemical substances in paper and ink and generate chemical composition data :

[0067] ;

[0068] in, and are the mean and standard deviation of the spectral data reflectance, and are the mean and standard deviation of wavelength, is the modulation coefficient, is the maximum measurement wavelength;

[0069] S44, chemical composition data Identify and classify chemical species in paper and ink by comparing with known chemical databases to generate chemical property profiles :

[0070] ;

[0071] in, represents the jth chemical substance identified, K is the total number of chemical substances identified, is the feature vector of the chemical substances in the database.

[0072] Optionally, S5 includes the following steps:

[0073] S51. Use the micro-microphone in the acoustic sensor module to capture the sound waves generated during writing and obtain the acoustic data generated during the writing process. :

[0074] ;

[0075] in, is the kth time point, is the amplitude of the sound wave captured at the kth time point, and T is the total number of time points;

[0076] ;

[0077] ;

[0078] Where A(f) represents the amplitude at frequency f, is the phase angle, is the noise term, is the amplitude of the mth noise source, is the time center of the noise source, is the standard deviation of the noise;

[0079] S52. Acoustic Data Perform preprocessing to improve the signal-to-noise ratio of acoustic data through denoising and normalization, and obtain preprocessed acoustic data ;

[0080] S53. Using audio analysis functions After preprocessing, the acoustic data Perform spectrum analysis to extract features related to paper texture and writing pressure and generate acoustic feature data :

[0081] ;

[0082] in, and are the mean and standard deviation of the time points, respectively;

[0083] ;

[0084] ;

[0085] S54, the data obtained from the optical sensor module, the tactile sensor module, the chemical sensor module and the acoustic sensor module Fusion to build a multimodal dataset :

[0086] ;

[0087] in, Represents a union operation on datasets.

[0088] Optionally, the S8 includes the following steps:

[0089] S81, using feature extraction submodule from multimodal dataset The preprocessed image in Extract visual features and generate visual feature data :

[0090] ;

[0091] in, Represents the grayscale value of pixel (i, j), H and W are the height and width of the image respectively, and are the horizontal and vertical coordinates of the image center, and are the standard deviations of the horizontal and vertical coordinates of the image, respectively;

[0092] S82, use chemical analysis tools to analyze the pre-processed chemical composition data Perform feature extraction to generate chemical feature data :

[0093] ;

[0094] in, is the reflectivity, is the wavelength, and are the mean and standard deviation of wavelength, is the modulation coefficient, is the maximum measurement wavelength;

[0095] S83, extracting visual feature data , acoustic feature data and chemical characterization data Integrate to generate comprehensive feature data :

[0096] ;

[0097] in, , and is the integration weight coefficient.

[0098] Optionally, the S9 includes the following steps:

[0099] S91, the comprehensive feature data Input pattern recognition submodule, select the most important features through adaptive weighted feature selection algorithm, and generate optimized feature set :

[0100] ;

[0101] in, Features With the target variable The correlation coefficient between Features The standard deviation of is the weight, is the importance threshold, Features With the target variable The weighting parameter between The target variable The mean of , K is the total number of target variables;

[0102] S92. Use the deep neural network model combined with the improved self-attention mechanism to optimize the feature set Perform classification and generate preliminary document feature data :

[0103] ;

[0104] in, and is the weight matrix, and is the bias term, is the activation function, is the adjustment factor, is the residual function, is the improved self-attention mechanism function, are query, key, and value matrices respectively, is the self-attention weight coefficient, is context feature data;

[0105] The improved self-attention mechanism function is calculated as follows:

[0106] ;

[0107] in, represents the query matrix, represents the key matrix, represents the value matrix, are the weight matrices for query, key, and value, respectively. is the dimension of the key, To adjust the parameters, It is a hybrid context vector that combines context information, temporal features, and spatial features:

[0108] ;

[0109] in, represents the norm of a vector, is an indicator function, which takes the value 1 when the condition is met, otherwise it takes the value 0. , and are the adjustment parameters of context, time and space features respectively, is the time characteristic data, is the spatial feature data, is the threshold of the context vector;

[0110] S93. Use self-supervised learning methods to pre-train the model through the generated pseudo labels and optimize the model parameters :

[0111] ;

[0112] Among them, the loss function combines MSE loss, regularization term and KL divergence. is the regularization parameter, is the model weight matrix, is the variance function, is the KL divergence weight, is the KL divergence function.

[0113] S94. Perform multiple cross-validations on the trained deep neural network model combined with the self-attention mechanism to evaluate the performance of the model:

[0114] ;

[0115] in, is the weight adjustment coefficient, The loss function parameter The gradient of is the predicted label of the mth sample in the kth fold, For parameters The mean of

[0116] S95. Apply the model evaluated by cross-validation to the multimodal dataset to be processed to generate the final preliminary document feature data :

[0117] ;

[0118] in, For optimized parameters The weights and biases of is the improved self-attention mechanism function, are the optimized query, key and value matrices respectively, They are the optimized contextual feature, temporal feature and spatial feature data respectively.

[0119] Optionally, the S10 includes the following steps:

[0120] S101, preliminary document feature data Input semantic understanding and document analysis module, preprocess the document content through text parsing submodule, including word segmentation, stop word removal and word form restoration, and generate preprocessed text data :

[0121] ;

[0122] Among them, the word segmentation function , stop word removal function and the lemmatization function This is achieved by:

[0123] ;

[0124] ;

[0125] ;

[0126] S102, using syntactic analysis algorithm to pre-process text data Perform syntactic structure analysis and generate a syntactic structure tree :

[0127] ;

[0128] Among them, the syntax structure tree generation function This is achieved by:

[0129] ;

[0130] in, is the normalization factor, are model parameters, is the characteristic function;

[0131] S103, using named entity recognition technology to pre-process text data Extract entity information from , generate a list of entities ;

[0132] S104. Preprocess text data using semantic role labeling algorithm Perform semantic role labeling and extract semantic role information :

[0133] ;

[0134] Among them, the semantic role labeling function This is achieved by:

[0135] ;

[0136] in, is the weight matrix, is the hidden layer state, is the bias term;

[0137] S105, the syntax structure tree , Entity List and semantic role information Integrate and generate the structure and semantic information of the document :

[0138] ;

[0139] in, Represents joint operations of information;

[0140] S106. Use context analysis algorithms to analyze the structure and semantic information of documents Perform contextual correlation analysis to generate contextual feature data ;

[0141] S107: context feature data Preprocessing text data Integrate to generate the final structure and semantic information :

[0142] .

[0143] Optionally, the S11 includes the following steps:

[0144] S111, structural and semantic information Input context understanding submodule, use context analysis algorithm to analyze document content and extract context related features ;

[0145] S112, context-related features Associate with knowledge in external knowledge base to generate knowledge-enhanced context features ;

[0146] S113. Contextual features for knowledge enhancement using historical data and cultural background knowledge Conduct further analysis to generate historically and culturally relevant contextual features ;

[0147] S114. Incorporate historical and cultural contextual features With structural and semantic information Integrate to generate final semantic information and document feature data .

[0148] Optionally, the S12 includes the following steps:

[0149] S121, the parsed semantic information and document feature data Input the document protection and management suggestion module, and conduct a preliminary assessment of the document's preservation status and potential risks through the condition assessment submodule to generate preliminary assessment data :

[0150] ;

[0151] in, and are model parameters, and is the characteristic function;

[0152] S122. Based on the preliminary assessment data , using physical and chemical property information to assess the vulnerability and degradation risk of documents and generate vulnerability assessment data :

[0153] ;

[0154] in, and is the normalization constant, is the kth measured value of the ith physical property, is its displacement, is the lth measured value of the ith chemical property, For its weight, and is the new evaluation parameter;

[0155] S123. Combine the historical data and usage frequency of the document to evaluate the preservation priority and repair requirements of the document and generate preservation priority data :

[0156] ;

[0157] in, is the mth historical data point, For its timestamp, and are the mean and standard deviation of historical data, is the nth frequency data point, is its frequency value, and are the mean and standard deviation of the usage frequency, and is the new evaluation parameter;

[0158] S124. Vulnerability assessment data and save priority data Integration to generate comprehensive conservation status assessment data for documents :

[0159] ;

[0160] in, is the fusion parameter, is the fusion mean, and is the new evaluation parameter;

[0161] S125. Based on the comprehensive preservation status assessment data Assess the potential risks of documents and generate risk assessment reports :

[0162] ;

[0163] in, is the risk weight, and are the mean and standard deviation of the comprehensive evaluation data, and is the new evaluation parameter;

[0164] S126, automatically generating protection and preservation suggestions based on the analysis results, including the following steps:

[0165] S1261, evaluating the preservation status and potential risk of the document through the condition evaluation submodule, and evaluating the vulnerability and degradation risk of the document based on the physical and chemical information collected from the sensor and analysis modules;

[0166] S1262, the condition evaluation submodule uses the chemical properties and tactile measurement results to evaluate the physical and chemical conditions of the document through functions, and calculates the minimum values ​​of the chemical and tactile data to evaluate the overall condition of the document:

[0167] ;

[0168] in, is the jth measured value in the chemical property, For its wavelength, is the kth measurement value in the tactile characteristic, For its depth, and are the mean and standard deviation of wavelength, and are the mean and standard deviation of the depth, is the new evaluation parameter;

[0169] S1263, generating preservation and restoration suggestions for each document type based on the evaluation results through the protection suggestion submodule, such as suitable temperature, humidity conditions and processing methods;

[0170] S1264, protection suggestion module evaluates according to conditions through function Provide protection advice , based on the condition evaluation results, if it is below a certain threshold, protective measures are recommended; if it is stable, no additional measures are required:

[0171] ;

[0172] in, The threshold for protection measures.

[0173] The beneficial effects of the present invention are:

[0174] The present invention achieves a full range of technical effects from perception to analysis to protection by integrating a multimodal sensor module, a data processing and deep learning module, a semantic understanding and document analysis module, and a document protection and management suggestion module. It not only solves the problems of text recognition limitations, insufficient complex format processing capabilities, and insufficient contextual understanding in the prior art, but also significantly improves the ability to recognize the physical and chemical properties of documents. Through the comprehensive perception of the multimodal sensor module, the system can capture the visual, tactile, chemical, and acoustic information of the document, providing rich raw data for subsequent data processing and analysis. The multimodal data is efficiently processed and feature extracted by the deep learning module to ensure data standardization and optimization, so that the system can accurately identify and classify key information in the document. In the semantic understanding and document analysis module, the system combines external knowledge bases, historical data, and cultural background knowledge to conduct in-depth semantic parsing and contextual analysis of document content, accurately understand the complex context and deep meaning of documents, support advanced information retrieval and content analysis, and automatically generate document preservation and repair suggestions based on multimodal data and semantic analysis results through the document protection and management suggestion module, evaluate the preservation status and potential risks of documents, and provide personalized protection measures. This comprehensive evaluation and automated protection solution ensures the long-term preservation and value maintenance of documents, greatly improving the efficiency and accuracy of document management. BRIEF DESCRIPTION OF THE DRAWINGS

[0175] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0176] Figure 1 This is a flow chart of a method for intelligent recognition of paper documents based on full perception proposed by the present invention. DETAILED DESCRIPTION

[0177] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0178] refer to Figure 1 , a paper document intelligent recognition method based on full perception, comprising the following steps:

[0179] S1, an initialization step of the multimodal sensor module, using the optical sensor submodule, the tactile sensor submodule, the chemical sensor submodule and the acoustic sensor submodule to synchronously scan the paper document to obtain the visual image, physical perception data, chemical information and sound data of the paper document;

[0180] S2, using an optical sensor module, using a camera and a scanning device, to convert text in a digital image into an editable format through optical character recognition technology;

[0181] In this implementation, S2 includes the following steps:

[0182] S21, using the high-definition camera and scanning device in the optical sensor module to capture high-resolution images of paper documents and generate digital images of the documents ;

[0183] S22. Digital images Perform image preprocessing:

[0184] Apply the adaptive denoising algorithm to remove the noise in the image and obtain the denoised image :

[0185] ;

[0186] in, is the denoising coefficient, Indicates that the image is The second derivative in the direction, is the direction weight, n is the image dimension;

[0187] Use a multi-scale sharpening filter to enhance the edge details of the image and generate a sharpened image :

[0188] ;

[0189] in, is the sharpening factor, is the multi-scale Gaussian weight, For is the multi-scale Gaussian filtering result of the standard deviation, M is the number of scales;

[0190] Execute the adaptive contrast adjustment function to optimize the contrast of the image and obtain the preprocessed image :

[0191] ;

[0192] in, and Represent the maximum and minimum pixel values ​​of the image, respectively. and are the maximum and minimum values ​​of the target contrast range, is the contrast adjustment factor, is the standard deviation of the image;

[0193] S23, the preprocessed image Input into the optical character recognition software for character recognition processing, and analyze the pre-processed image through the optical character recognition software engine , identify text regions in images , the text area Convert the characters in to machine-readable text , generate preliminary editable text ;

[0194] S24. Use natural language processing technology to process preliminary editable text Perform spelling and grammar correction and generate corrected text :

[0195] ;

[0196] in, Indicates that it is based on preliminary text The probability of is the i-th word in the text, and N is the total number of words in the text;

[0197] Apply formatting algorithms to the corrected text Perform typesetting, adjust paragraphs, fonts and spacing, and generate final editable text :

[0198] ;

[0199] in, It represents the typesetting weight of the text paragraph, and M is the total number of text paragraphs.

[0200] S3, combined with the tactile sensor module, uses pressure sensors and surface scanning technology to measure the thickness and surface texture of paper;

[0201] In this implementation, S3 includes the following steps:

[0202] S31, using the pressure sensor in the tactile sensor module to measure the pressure of the paper document to generate paper thickness data ;

[0203] ;

[0204] Where A is the area of ​​the paper, is the pressure value of the i-th measuring point, is the displacement of the i-th measuring point, and N is the total number of measuring points;

[0205] S32. Scan the surface of the paper document using surface scanning technology to generate surface texture data ;

[0206] ;

[0207] in, is the three-dimensional coordinate of the jth scanning point, and M is the total number of scanning points;

[0208] S33, the paper thickness data of the pressure sensor and surface texture data generated by surface scanning technology Fusion to build a physical property dataset of paper :

[0209] ;

[0210] in, , , is the fusion coefficient, represents the outer product of tensors;

[0211] S34. Physical property data set Perform feature extraction to generate paper thickness features and surface texture features :

[0212] ;

[0213] ;

[0214] in, and is the feature extraction weight, and They are and The mean of and They are and The standard deviation of is the surface angle;

[0215] S35. Extract the thickness feature and surface texture features Integrate to generate comprehensive feature data :

[0216] ;

[0217] in, , , is the integration coefficient, Represents a convolution operation.

[0218] S4, using the chemical sensor module, detecting the chemical substances of paper and ink through spectral analysis technology and chemical sensors to generate a chemical property profile;

[0219] In this implementation, S4 includes the following steps:

[0220] S41. Use the spectral analysis device in the chemical sensor module to scan the paper document and obtain the spectral data of the paper and ink. :

[0221] ;

[0222] in, is the i-th spectral wavelength, is the reflectivity at this wavelength, and L is the total number of measured wavelengths;

[0223] S42. Spectral data Preprocessing is performed to improve the signal-to-noise ratio of the spectral data through baseline correction and denoising, and the preprocessed spectral data is obtained. :

[0224] ;

[0225] in, represents the minimum value in the spectral data, is the denoising coefficient, is the standard deviation of the spectral data;

[0226] S43. Use chemical analysis tools to analyze the preprocessed spectral data Perform component analysis to identify chemical substances in paper and ink and generate chemical composition data :

[0227] ;

[0228] in, and are the mean and standard deviation of the spectral data reflectance, and are the mean and standard deviation of wavelength, is the modulation coefficient, is the maximum measurement wavelength;

[0229] S44, chemical composition data Identify and classify chemical species in paper and ink by comparing with known chemical databases to generate chemical property profiles :

[0230] ;

[0231] in, represents the jth chemical substance identified, K is the total number of chemical substances identified, is the feature vector of the chemical substances in the database.

[0232] S5. Use the acoustic sensor module to capture the sound waves generated when writing, analyze the paper texture and writing pressure, and use the audio analysis function to analyze the vibration caused by document writing or interaction;

[0233] In this implementation, S5 includes the following steps:

[0234] S51. Use the micro-microphone in the acoustic sensor module to capture the sound waves generated during writing and obtain the acoustic data generated during the writing process. :

[0235] ;

[0236] in, is the kth time point, is the amplitude of the sound wave captured at the kth time point, and T is the total number of time points;

[0237] ;

[0238] ;

[0239] Where A(f) represents the amplitude at frequency f, is the phase angle, is the noise term, is the amplitude of the mth noise source, is the time center of the noise source, is the standard deviation of the noise;

[0240] S52. Acoustic Data Perform preprocessing to improve the signal-to-noise ratio of acoustic data through denoising and normalization, and obtain preprocessed acoustic data ;

[0241] S53. Using audio analysis functions After preprocessing, the acoustic data Perform spectrum analysis to extract features related to paper texture and writing pressure and generate acoustic feature data :

[0242] ;

[0243] in, and are the mean and standard deviation of the time points, respectively;

[0244] ;

[0245] ;

[0246] S54, the data obtained from the optical sensor module, the tactile sensor module, the chemical sensor module and the acoustic sensor module Fusion to build a multimodal dataset :

[0247] ;

[0248] in, Represents a union operation on datasets.

[0249] In traditional document recognition methods, usually relying on a single optical character recognition technology, only the text in the document can be recognized, and there is a lack of effective recognition means for the physical and chemical properties of paper. However, this embodiment combines optical, tactile, chemical and acoustic sensors to comprehensively collect and analyze multimodal data of documents, which can achieve more accurate and comprehensive document recognition and analysis.

[0250] S6, integrating the data acquired from the optical sensor module, the tactile sensor module, the chemical sensor module, and the acoustic sensor module to construct a multimodal data set;

[0251] S7, inputting the acquired multimodal data set into the data processing and deep learning module for preprocessing, and standardizing and cleaning the multimodal data set through the data preprocessing submodule, removing noise, adjusting brightness, and performing normalization processing;

[0252] S8, extract visual features from visual images in the multimodal dataset using the feature extraction submodule, and extract relevant features from acoustic and chemical data using the sound and chemical analysis tools;

[0253] In this implementation, S8 includes the following steps:

[0254] S81, using feature extraction submodule from multimodal dataset The preprocessed image in Extract visual features and generate visual feature data :

[0255] ;

[0256] in, Represents the grayscale value of pixel (i, j), H and W are the height and width of the image respectively, and are the horizontal and vertical coordinates of the image center, and are the standard deviations of the horizontal and vertical coordinates of the image, respectively;

[0257] S82, use chemical analysis tools to analyze the pre-processed chemical composition data Perform feature extraction to generate chemical feature data :

[0258] ;

[0259] in, is the reflectivity, is the wavelength, and are the mean and standard deviation of wavelength, is the modulation coefficient, is the maximum measurement wavelength;

[0260] S83, extracting visual feature data , acoustic feature data and chemical characterization data Integrate to generate comprehensive feature data :

[0261] ;

[0262] in, , and is the integration weight coefficient.

[0263] S9, classify and identify the extracted multimodal data set through the pattern recognition submodule combined with the machine learning model to generate preliminary document feature data;

[0264] In this implementation, S9 includes the following steps:

[0265] S91, the comprehensive feature data Input pattern recognition submodule, select the most important features through adaptive weighted feature selection algorithm, and generate optimized feature set :

[0266] ;

[0267] in, Features With the target variable The correlation coefficient between Features The standard deviation of is the weight, is the importance threshold, Features With the target variable The weighting parameter between The target variable The mean of , K is the total number of target variables;

[0268] S92. Use the deep neural network model combined with the improved self-attention mechanism to optimize the feature set Perform classification and generate preliminary document feature data :

[0269] ;

[0270] in, and is the weight matrix, and is the bias term, is the activation function, is the adjustment factor, is the residual function, is the improved self-attention mechanism function, are query, key, and value matrices respectively, is the self-attention weight coefficient, is context feature data;

[0271] The improved self-attention mechanism function is calculated as follows:

[0272] ;

[0273] in, represents the query matrix, represents the key matrix, represents the value matrix, are the weight matrices for query, key, and value, respectively. is the dimension of the key, To adjust the parameters, It is a hybrid context vector that combines context information, temporal features, and spatial features:

[0274] ;

[0275] in, represents the norm of a vector, is an indicator function, which takes the value 1 when the condition is met, otherwise it takes the value 0. , and are the adjustment parameters of context, time and space features respectively, is the time characteristic data, is the spatial feature data, is the threshold of the context vector;

[0276] S93. Use self-supervised learning methods to pre-train the model through the generated pseudo labels and optimize the model parameters :

[0277] ;

[0278] Among them, the loss function combines MSE loss, regularization term and KL divergence. is the regularization parameter, is the model weight matrix, is the variance function, is the KL divergence weight, is the KL divergence function.

[0279] S94. Perform multiple cross-validations on the trained deep neural network model combined with the self-attention mechanism to evaluate the performance of the model:

[0280] ;

[0281] in, is the weight adjustment coefficient, The loss function parameter The gradient of is the predicted label of the mth sample in the kth fold, For parameters The mean of

[0282] S95. Apply the model evaluated by cross-validation to the multimodal dataset to be processed to generate the final preliminary document feature data :

[0283] ;

[0284] in, For optimized parameters The weights and biases of is the improved self-attention mechanism function, are the optimized query, key and value matrices respectively, They are the optimized contextual features, temporal features and spatial feature data respectively.

[0285] The improved self-attention mechanism in this embodiment can capture the multimodal feature information of the document more comprehensively by combining context, time and space features, and show higher accuracy and robustness when processing complex document layout and content, especially in the intelligent recognition task of paper documents. It enables the model to not only recognize the text content, but also perceive the physical and chemical properties of the document, as well as dynamically changing environmental information, thereby greatly improving the overall performance and application value of document recognition.

[0286] At the same time, when processing complex format documents containing multiple fonts, images, tables and handwritten annotations, this method significantly improves the recognition rate and accuracy through the combination of multimodal sensor modules and data processing with deep learning modules.

[0287] S10, inputting the preliminary document feature data into the semantic understanding and document analysis module, parsing the document content through the text parsing submodule, and extracting the structure and semantic information;

[0288] In this implementation, S10 includes the following steps:

[0289] S101, preliminary document feature data Input semantic understanding and document analysis module, preprocess the document content through text parsing submodule, including word segmentation, stop word removal and word form restoration, and generate preprocessed text data :

[0290] ;

[0291] Among them, the word segmentation function , stop word removal function and the lemmatization function This is achieved by:

[0292] ;

[0293] ;

[0294] ;

[0295] S102, using syntactic analysis algorithm to pre-process text data Perform syntactic structure analysis and generate a syntactic structure tree :

[0296] ;

[0297] Among them, the syntax structure tree generation function This is achieved by:

[0298] ;

[0299] in, is the normalization factor, are model parameters, is the characteristic function;

[0300] S103, using named entity recognition technology to pre-process text data Extract entity information from , generate a list of entities ;

[0301] S104. Preprocess text data using semantic role labeling algorithm Perform semantic role labeling and extract semantic role information :

[0302] ;

[0303] Among them, the semantic role labeling function This is achieved by:

[0304] ;

[0305] in, is the weight matrix, is the hidden layer state, is the bias term;

[0306] S105, the syntax structure tree , Entity List and semantic role information Integrate and generate the structure and semantic information of the document :

[0307] ;

[0308] in, Represents joint operations of information;

[0309] S106. Use context analysis algorithms to analyze the structure and semantic information of documents Perform contextual correlation analysis to generate contextual feature data ;

[0310] S107: context feature data Preprocessing text data Integrate to generate the final structure and semantic information :

[0311] .

[0312] S11. Use the context understanding submodule to analyze the deep meaning and context of the document based on the document content and external knowledge base, and combine historical data and cultural background knowledge to perform semantic analysis and understanding;

[0313] In this implementation, S11 includes the following steps:

[0314] S111, structural and semantic information Input context understanding submodule, use context analysis algorithm to analyze document content and extract context related features ;

[0315] S112, context-related features Associate with knowledge in external knowledge base to generate knowledge-enhanced context features ;

[0316] S113. Contextual features for knowledge enhancement using historical data and cultural background knowledge Conduct further analysis to generate historically and culturally relevant contextual features ;

[0317] S114. Incorporate historical and cultural contextual features With structural and semantic information Integrate to generate final semantic information and document feature data .

[0318] S12, inputting the parsed semantic information and document feature data into the document protection and management suggestion module, and evaluating the preservation status and potential risks of the document through the condition evaluation submodule;

[0319] In this implementation, S12 includes the following steps:

[0320] S121, the parsed semantic information and document feature data Input the document protection and management suggestion module, and conduct a preliminary assessment of the document's preservation status and potential risks through the condition assessment submodule to generate preliminary assessment data :

[0321] ;

[0322] in, and are model parameters, and is the characteristic function;

[0323] S122. Based on the preliminary assessment data , using physical and chemical property information to assess the vulnerability and degradation risk of documents and generate vulnerability assessment data :

[0324] ;

[0325] in, and is the normalization constant, is the kth measured value of the ith physical property, is its displacement, is the lth measured value of the ith chemical property, For its weight, and is the new evaluation parameter;

[0326] S123. Combine the historical data and usage frequency of the document to evaluate the preservation priority and repair requirements of the document and generate preservation priority data :

[0327] ;

[0328] in, is the mth historical data point, For its timestamp, and are the mean and standard deviation of historical data, is the nth frequency data point, is its frequency value, and are the mean and standard deviation of the usage frequency, and is the new evaluation parameter;

[0329] S124. Vulnerability assessment data and save priority data Integration to generate comprehensive conservation status assessment data for documents :

[0330] ;

[0331] in, is the fusion parameter, is the fusion mean, and is the new evaluation parameter;

[0332] S125. Based on the comprehensive preservation status assessment data Assess the potential risks of documents and generate risk assessment reports :

[0333] ;

[0334] in, is the risk weight, and are the mean and standard deviation of the comprehensive evaluation data, and is the new evaluation parameter;

[0335] S126, automatically generating protection and preservation suggestions based on the analysis results, including the following steps:

[0336] S1261, evaluating the preservation status and potential risk of the document through the condition evaluation submodule, and evaluating the vulnerability and degradation risk of the document based on the physical and chemical information collected from the sensor and analysis modules;

[0337] S1262, the condition evaluation submodule uses the chemical properties and tactile measurement results to evaluate the physical and chemical conditions of the document through functions, and calculates the minimum values ​​of the chemical and tactile data to evaluate the overall condition of the document:

[0338] ;

[0339] in, is the jth measured value in the chemical property, For its wavelength, is the kth measurement value in the tactile characteristic, For its depth, and are the mean and standard deviation of wavelength, and are the mean and standard deviation of the depth, is the new evaluation parameter;

[0340] S1263, generating preservation and restoration suggestions for each document type based on the evaluation results through the protection suggestion submodule, such as suitable temperature, humidity conditions and processing methods;

[0341] S1264, protection suggestion module evaluates according to conditions through function Provide protection advice , based on the condition evaluation results, if it is below a certain threshold, protective measures are recommended; if it is stable, no additional measures are required:

[0342] ;

[0343] in, The threshold for protection measures.

[0344] S13. Based on the evaluation results, the protection suggestion submodule generates preservation and restoration suggestions for the document type, and provides appropriate temperature, humidity conditions and processing methods;

[0345] S14. Output a complete document identification and protection recommendation report to support subsequent document management and maintenance work.

[0346] The paper document intelligent recognition method based on full perception in this embodiment solves many problems in the prior art by integrating a multimodal sensor module, a data processing and deep learning module, a semantic understanding and document analysis module, and a document protection and management suggestion module. Specifically, this method can not only recognize the text in the document, but also perceive and analyze the physical and chemical properties of the document, such as paper type, ink composition, and printing technology, to provide a more comprehensive document analysis capability. In addition, when processing documents in complex formats, this method significantly improves the recognition rate and accuracy by combining tactile, chemical, and acoustic sensors. Advanced semantic and contextual understanding technology can accurately parse the deep meaning and complex context in the document, thereby supporting more advanced information retrieval and content analysis. Ultimately, by automatically generating protection and preservation recommendations, it can help institutions and individuals manage and maintain their document assets more effectively and ensure the long-term preservation and security of documents.

[0347] Embodiment 1:

[0348] The paper of some manuscripts has begun to show obvious signs of brittleness and degradation. In addition, the content of these manuscripts is complex, containing multiple fonts, images, tables and handwritten annotations. Traditional OCR technology faces problems of low recognition rate and high error rate when processing these documents. In order to better protect and manage these ancient manuscripts, the museum decided to introduce a paper document intelligent recognition system (ADIS) based on full perception for comprehensive analysis and processing.

[0349] Museum staff placed these ancient manuscripts one by one on the scanning platform of the ADIS system. The system's multimodal sensor module was activated, using a high-definition camera to capture high-resolution images of the manuscripts, while using a pressure sensor to measure the thickness and surface texture of the paper. Chemical sensors detect the chemical composition of paper and ink through spectral analysis technology, while acoustic sensors capture sound waves generated during writing to analyze paper texture and writing pressure. Through these steps, the system collected multimodal data of the manuscripts, including visual images, physical perception data, chemical information, and sound data.

[0350] These data are fed into the Data Processing and Deep Learning module for analysis. First, the Data Preprocessing submodule performs denoising, sharpening, and contrast adjustment on the images, while standardizing and cleaning the physical and chemical data. The Feature Extraction submodule extracts visual features from the images and relevant features from the acoustic and chemical data. The Pattern Recognition submodule combines machine learning models to classify and identify the extracted multimodal data and generate preliminary document feature data.

[0351] The preliminary document feature data is input into the semantic understanding and document analysis module for further analysis. The text parsing submodule parses the document content and extracts structural and semantic information. The context understanding submodule combines historical data and cultural background knowledge to analyze the deep meaning and context of the document and generate detailed semantic information.

[0352] Finally, the parsed semantic information and document feature data are input into the document protection and management recommendation module. The condition assessment submodule evaluates the preservation status and potential risks of the manuscripts, and calculates the overall condition of the document using chemical properties and tactile measurements. Based on the evaluation results, the protection recommendation submodule generates preservation and restoration recommendations for each manuscript, such as appropriate temperature, humidity conditions and treatment methods. If the preservation status of some manuscripts is below a certain threshold, the system will recommend appropriate protection measures.

[0353] The following is a detailed process description and related data:

[0354] 1. Document identification and analysis:

[0355] When recognizing the text content of these 50 manuscripts, the ADIS system achieved an accuracy rate of 98.5%, significantly higher than the 85.3% of the traditional OCR system. The specific data are shown in Table 1:

[0356] Table 1 Comparison of document recognition accuracy

[0357] System Type Total number of documents Recognition accuracy ADIS system 50 98.5% Traditional OCR system 50 85.3%

[0358] As can be seen from Table 1 above, the ADIS system can not only accurately recognize text content through multimodal sensing technology, but also process complex document formats, including multiple fonts, images, tables and handwritten annotations. When processing documents in these complex formats, traditional OCR systems have a high error rate and cannot guarantee high-precision recognition.

[0359] 2. Physical and chemical property analysis:

[0360] When evaluating the thickness and surface texture of the paper, the pressure sensor measurements of the ADIS system showed that 23.1% of the manuscripts had a paper thickness of less than 0.1 mm, indicating that the paper of these manuscripts was already very fragile. The chemical analysis results showed that 34.4% of the manuscripts contained high levels of acidic substances, increasing the risk of paper degradation. The specific data are shown in Table 2:

[0361] Table 2 Physical and chemical properties analysis results

[0362] Manuscript number Thickness (mm) Surface texture Acid content Status Assessment Save suggestion 001 0.08 normal high Below threshold Temperature 18°C, humidity 45% 002 0.12 Rough Low Above threshold No additional measures required 003 0.09 normal middle Below threshold Temperature 18°C, humidity 45% ... ... ... ... ... ... 050 0.11 smooth high Below threshold Temperature 18°C, humidity 45%

[0363] 3. Preservation status and risk assessment:

[0364] Through the condition assessment submodule, the ADIS system generated a detailed preservation status assessment report. The assessment results showed that the preservation status of 15.4% of the manuscripts was below the threshold and immediate protection measures were required. The protection suggestion submodule recommended that these manuscripts be stored in an environment with a temperature of 18°C ​​and a humidity of 45% to delay the degradation of the paper. The specific data are shown in Table 3:

[0365] Table 3 Conservation status and risk assessment

[0366] Manuscript number Save Status Rating risk assessment Risk Level Suggested Actions 001 45 high High risk Immediate protective measures, temperature 18°C, humidity 45% 002 75 Low No risk No additional measures required 003 48 middle Medium risk Check regularly, temperature 18°C, humidity 45% ... ... ... ... ... 050 42 high High risk Immediate protective measures, temperature 18°C, humidity 45%

[0367] 4. Historical and cultural background analysis:

[0368] Through the semantic understanding module, the ADIS system extracts the historical context and cultural significance of the manuscript. The system identifies important historical events contained in a manuscript, which are missed in the analysis of traditional OCR systems. In addition, the ADIS system also combines external knowledge bases to provide detailed background information about these events, helping museums better understand and display the value of manuscripts. The specific data is shown in Table 4:

[0369] Table 4 Historical and cultural background information extraction

[0370] Manuscript number Identify important historical events Background information provided Remark 001 yes yes Provide detailed background information 002 no no No relevant information 003 yes yes Provide detailed background information ... ... ... ... 050 yes yes Provide detailed background information

[0371] 5. Time efficiency:

[0372] In the process of processing these 50 manuscripts, the ADIS system took a total of about 30 hours, with an average of 36 minutes per manuscript. The traditional manual analysis and OCR system took about 100 hours to process this batch of manuscripts, with an average of 2 hours per manuscript. The ADIS system significantly improved the efficiency of document processing. The specific data is shown in Table 5:

[0373] Table 5 Processing time comparison

[0374] System Type Total processing time (hours) Average time per manuscript (minutes) ADIS system 30 36 Traditional OCR system 100 120

[0375] From the data in Table 5, we can see that the ADIS system is superior to the traditional OCR system in terms of recognition accuracy, physical and chemical property analysis, preservation status assessment and risk prediction. In addition, the ADIS system can also provide detailed historical and cultural background analysis, which greatly improves the efficiency and quality of document processing.

[0376] In this example, the ADIS system demonstrates its great potential in the protection and management of ancient manuscripts. Through multimodal sensing technology, deep learning algorithms, and advanced semantic understanding capabilities, the ADIS system can comprehensively and accurately identify and analyze paper documents and provide personalized protection and preservation recommendations. This new document processing method not only significantly improves the accuracy and processing efficiency of document recognition, but also provides strong support for the long-term preservation and value mining of documents.

[0377] The present invention achieves a full range of technical effects from perception to analysis to protection by integrating a multimodal sensor module, a data processing and deep learning module, a semantic understanding and document analysis module, and a document protection and management suggestion module. It not only solves the problems of text recognition limitations, insufficient complex format processing capabilities, and insufficient contextual understanding in the prior art, but also significantly improves the ability to recognize the physical and chemical properties of documents. Through the comprehensive perception of the multimodal sensor module, the system can capture the visual, tactile, chemical, and acoustic information of the document, providing rich raw data for subsequent data processing and analysis. The multimodal data is efficiently processed and feature extracted by the deep learning module to ensure data standardization and optimization, so that the system can accurately identify and classify key information in the document. In the semantic understanding and document analysis module, the system combines external knowledge bases, historical data, and cultural background knowledge to conduct in-depth semantic parsing and contextual analysis of document content, accurately understand the complex context and deep meaning of documents, support advanced information retrieval and content analysis, and automatically generate document preservation and repair suggestions based on multimodal data and semantic analysis results through the document protection and management suggestion module, evaluate the preservation status and potential risks of documents, and provide personalized protection measures. This comprehensive evaluation and automated protection solution ensures the long-term preservation and value maintenance of documents, greatly improving the efficiency and accuracy of document management.

[0378] In summary, the paper document intelligent recognition method based on full perception realizes the comprehensive, accurate and efficient recognition and management of paper documents by integrating multimodal perception, deep learning, semantic understanding and automated protection suggestion technology, providing a new solution for document research, management and protection.

[0379] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A paper document intelligent recognition method based on full perception, characterized in that: The following steps are involved: S1, an initialization step of the multimodal sensor module, using the optical sensor module, the tactile sensor module, the chemical sensor module and the acoustic sensor module to synchronously scan the paper document to obtain the visual image, physical perception data, chemical information and sound data of the paper document; S2, using an optical sensor module, using a camera and a scanning device, to convert text in a digital image into an editable format through optical character recognition technology; S3, combined with the tactile sensor module, uses pressure sensors and surface scanning technology to measure the thickness and surface texture of paper; S4, using the chemical sensor module, detecting the chemical substances of paper and ink through spectral analysis technology and chemical sensors to generate a chemical property profile; S5. Use the acoustic sensor module to capture the sound waves generated when writing, analyze the paper texture and writing pressure, and use the audio analysis function to analyze the vibration caused by document writing or interaction; S6, integrating the data acquired from the optical sensor module, the tactile sensor module, the chemical sensor module, and the acoustic sensor module to construct a multimodal data set; S7, inputting the acquired multimodal data set into the data processing and deep learning module for preprocessing, and standardizing and cleaning the multimodal data set through the data preprocessing submodule, removing noise, adjusting brightness, and performing normalization processing; S8, extract visual features from digital images in the multimodal dataset using the feature extraction submodule, and extract relevant features from acoustic and chemical data using the sound and chemical analysis tools; S9, classify and identify the extracted multimodal data set through the pattern recognition submodule combined with the machine learning model to generate preliminary document feature data; S10, inputting the preliminary document feature data into the semantic understanding and document analysis module, parsing the document content through the text parsing submodule, and extracting the structure and semantic information; S11. Use the context understanding submodule to analyze the deep meaning and context of the document based on the document content and external knowledge base, and combine historical data and cultural background knowledge to perform semantic analysis and understanding; S12, inputting the parsed semantic information and document feature data into the document protection and management suggestion module, and evaluating the preservation status and potential risks of the document through the condition evaluation submodule; S13. Based on the evaluation results, the protection suggestion submodule generates preservation and restoration suggestions for the document type, and provides appropriate temperature, humidity conditions and processing methods; S14. Output a complete document identification and protection recommendation report to support subsequent document management and maintenance work; The S2 comprises the following steps: S21, using the high-definition camera and scanning device in the optical sensor module to capture high-resolution images of paper documents and generate digital images of the documents ; S22. Digital images Perform image preprocessing: Apply the adaptive denoising algorithm to remove the noise in the image and obtain the denoised image : ； in, is the denoising coefficient, Indicates that the image is The second derivative in the direction, is the direction weight, n is the image dimension; Use a multi-scale sharpening filter to enhance the edge details of the image and generate a sharpened image : ； in, is the sharpening factor, is the multi-scale Gaussian weight, For is the multi-scale Gaussian filtering result of the standard deviation, M is the number of scales; Execute the adaptive contrast adjustment function to optimize the contrast of the image and obtain the preprocessed image : ； in, and Represent the maximum and minimum pixel values ​​of the image, respectively. and are the maximum and minimum values ​​of the target contrast range, is the contrast adjustment factor, is the standard deviation of the image; S23, the preprocessed image Input into the optical character recognition software for character recognition processing, and analyze the pre-processed image through the optical character recognition software engine , identify text regions in images , the text area Convert the characters in to machine-readable text , generate preliminary editable text ; S24. Use natural language processing technology to process preliminary editable text Perform spelling and grammar correction and generate corrected text : ； in, Indicates that the text is editable based on the The probability of is the i-th word in the text, and N is the total number of words in the text; Apply formatting algorithms to the corrected text Perform typesetting, adjust paragraphs, fonts and spacing, and generate final editable text : ； in, Indicates the typesetting weight of a text paragraph. is the total number of text paragraphs; The S3 comprises the following steps: S31, using the pressure sensor in the tactile sensor module to measure the pressure of the paper document to generate paper thickness data ; ； Where A is the area of ​​the paper, is the pressure value of the i-th measuring point, is the displacement of the ith measurement point, is the total number of measurement points; S32. Scan the surface of the paper document using surface scanning technology to generate surface texture data ; ； in, is the three-dimensional coordinate of the j-th scanning point, is the total number of scan points; S33, the paper thickness data of the pressure sensor and surface texture data generated by surface scanning technology Fusion to build a physical property dataset of paper : ； in, , , is the fusion coefficient, represents the outer product of tensors; S34. Physical property data set Perform feature extraction to generate paper thickness features and surface texture features : ； ； in, and is the feature extraction weight, and They are and The mean of and They are and The standard deviation of is the surface angle; S35. Extract the thickness feature and surface texture features Integrate to generate comprehensive feature data : ； in, , , is the integration coefficient, Represents the convolution operation; The S4 comprises the following steps: S41. Use the spectral analysis device in the chemical sensor module to scan the paper document and obtain the spectral data of the paper and ink. : ； in, is the i-th spectral wavelength, is the reflectivity at this wavelength, and L is the total number of measured wavelengths; S42. Spectral data Preprocessing is performed to improve the signal-to-noise ratio of the spectral data through baseline correction and denoising, and the preprocessed spectral data is obtained. : ； in, represents the minimum value in the spectral data, is the denoising coefficient, is the standard deviation of the spectral data; S43. Use chemical analysis tools to analyze the preprocessed spectral data Perform component analysis to identify chemical substances in paper and ink and generate chemical composition data : ； in, and are the mean and standard deviation of the spectral data reflectance, and are the mean and standard deviation of wavelength, is the modulation coefficient, is the maximum measurement wavelength; S44, chemical composition data Identify and classify chemical species in paper and ink by comparing with known chemical databases to generate chemical property profiles : ； in, represents the g-th chemical substance identified, K is the total number of chemical substances identified, is the feature vector of the chemical substances in the database.

Images