Natural language processing platform and processing method based on deep learning

Abstract

The invention belongs to the technical field of natural language processing, and discloses a natural language processing platform and method based on deep learning, and the platform comprises a data collection module which is used for collecting font data, text data, image data and environment data; the data processing module is used for preprocessing the collected font data, text data, image data and environment data and obtaining a comprehensive feature data set; the character recognition module is used for predicting character recognition results through a character recognition model and judging whether the character recognition results are consistent or not; the semantic recognition module is used for generating corresponding character paraphrases through a semantic recognition model; collecting character recognition results and corresponding character paraphrases through the software terminal; the voice output module is used for converting the character recognition result and the paraphrases of the corresponding characters into natural voices and outputting the natural voices by adopting a TTS technology; through the implementation of the technology, students can be helped to better learn and understand.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing technology, and more specifically, to a natural language processing platform and processing method based on deep learning. Background Art

[0002] The patent application publication number CN108427670A discloses a sentiment analysis method based on contextual word vectors and deep learning. The method discloses a sentiment analysis method in the field of natural language processing, involving the use of contextual word vectors and deep learning to analyze the sentiment tendency of user comments on e-commerce platforms. The method trains contextual word vectors and combines deep learning to solve the current problems of sentiment analysis, such as difficulty in processing high-dimensional text data, inability to extract sentiment features, and low accuracy. The present invention uses contextual word vectors to obtain the semantic information and sentiment information of words in the context. Compared with the existing technology, the method can extract more effective features for user comments on e-commerce platforms, and has a higher accuracy and recall rate for sentiment tendency analysis.

[0003] Traditional natural language processing methods have the following main problems:

[0004] It lacks the ability to accurately identify fonts, has a low accuracy rate for font recognition, cannot achieve higher-precision font recognition and matching, and has poor results in identifying subtle differences between similar fonts; fonts are not effectively classified and organized; fonts between different software and platforms are not compatible, and visual problems due to font rendering differences frequently occur; the algorithm has low flexibility and cannot adapt to different font styles and outline features; it cannot reduce computational complexity while ensuring outline accuracy;

[0005] The feature dimension is high and the computational efficiency is low. It cannot reflect the importance of words in the text and has a weak ability to distinguish different text contents. It cannot accurately analyze the sentiment of the text. It does not have a voice output function, so users need to spend time and energy to read a large amount of text and cannot obtain information more intuitively.

[0006] In view of this, the present invention proposes a natural language processing platform and processing method based on deep learning to solve the above problems. Summary of the invention

[0007] In order to overcome the above-mentioned defects of the prior art and to achieve the above-mentioned purpose, the present invention provides the following technical solution: a natural language processing platform based on deep learning, comprising:

[0008] A data collection module, used for collecting font data, text data, image data and environmental data;

[0009] The data processing module is used to pre-process the collected font data, text data, image data and environmental data to obtain a font feature data set, a text feature data set, an image feature data set and an environmental feature data set; and weightedly fuse the font feature data set, the image feature data set and the environmental feature data set to obtain a comprehensive feature data set;

[0010] The text recognition module is used to train and obtain a text recognition model based on the comprehensive feature data set, and predict the text recognition result through the text recognition model; compare the predicted text recognition result with the preset text recognition result to determine whether the text recognition results are consistent;

[0011] Semantic recognition module: if the text recognition results are inconsistent, the recognition is performed again; if the text recognition results are consistent, the semantic recognition model is obtained through text feature data set training, and the corresponding text interpretation is generated through the semantic recognition model; the text recognition results and the corresponding text interpretation are collected through the software terminal;

[0012] The speech output module uses TTS technology to convert the text recognition results and the corresponding text interpretation into natural speech output; each module is connected by wired and / or wireless means.

[0013] Furthermore, the font data includes font style data, font size data, font thickness data and spacing data; the text data includes grammar data, character data, text content data and text language data; the image data includes text image data; and the environmental data includes light intensity data and noise level data of the collection environment.

[0014] Furthermore, the method of preprocessing the collected font data to obtain a font feature data set includes:

[0015] The collected font data is respectively extracted with contour feature data, geometric parameter feature data, structural feature data and attribute coding data. The specific method is as follows:

[0016] S31, using Canny edge detection to identify the outline of the font style data; tracking the outline along the detected edge, and recording the coordinate points of the outline to form an outline path; measuring the average width of the outline path, thereby obtaining line width features; analyzing and quantifying the shape of the outline corners, thereby obtaining corner shape features; collecting line width features and corner shape features to obtain outline feature data;

[0017] S32, geometrically parameterizing the font data; simplifying the outline of the font style data using the Douglas-Peucker algorithm; selecting the starting point and the ending point on the outline path, and setting a threshold α;

[0018] The value of the threshold α is adjusted by a threshold adjustment formula; the threshold adjustment formula is: Among them, α′ is the adjusted threshold; n is the total number of coordinate points on the contour path; j is the index of the coordinate point; θ j is the normal direction of the jth coordinate point; θ j+1 is the normal direction of the j+1th coordinate point; is the curvature change rate of the contour path; γ1 is the weight coefficient of the curvature change rate of the contour path; x b and b are the horizontal and vertical coordinates of the end point of the contour path; x1 and y1 are the horizontal and vertical coordinates of the starting point of the contour path; b is the index of the end point of the contour path; is the closure degree of the contour path; dk ​​is the total length of the contour path; γ2 is the weight coefficient of the closure degree of the contour path;

[0019] For each coordinate point on the contour path, calculate the vertical distance between the point and the starting point and the end point; for any coordinate point P i , if d(P i )<α, then the coordinate point is discarded; otherwise, if d(P i )>α, then keep the coordinate point; if d(P i )=α, when the coordinate point P i If it is a bending point or turning point on the contour path, the coordinate point is retained, otherwise it is discarded; where d(P i ) is the coordinate point P i The vertical distance between the starting point and the end point; P i is the i-th coordinate point; i is the index of the coordinate point; repeat the above process until all coordinate points are processed; collect all the retained coordinate points to obtain the contour point set; select β control points based on the contour point set;

[0020] The number β of the control points is determined by a number adjustment formula; the number adjustment formula is: Among them, β′ is the number of control points after adjustment; n is the total number of coordinate points on the contour path; o is the order of the Bezier curve; dk is the total length of the contour path; is the square root of contour complexity; w is the width of the contour path display area; h is the height of the contour path display area; μ1 and μ2 are weight coefficients; is the operator for rounding up;

[0021] The control points are fitted into Bezier curves using the least square method; the contour point set and the Bezier curve are collected to obtain geometric parameter feature data;

[0022] S33, performing a ratio analysis based on the font size data and the spacing data in the font data, calculating the height and width ratios of different fonts, and the ratio relationship between different fonts, to obtain structural feature data;

[0023] S34, attribute mapping is performed on the font thickness data, the font thickness is mapped to a numerical range, and the font thickness is represented by a numerical attribute code; the attribute codes corresponding to all the font thickness data are collected to obtain attribute code data;

[0024] The acquired contour feature data, geometric parameter feature data, structural feature data and attribute coding data are integrated to obtain a font feature data set.

[0025] Furthermore, the method of preprocessing the collected text data, image data and environmental data to obtain a text feature data set, an image feature data set and an environmental feature data set includes:

[0026] S41. Preprocessing of text data: Clean the text data to remove text noise, and use the jieba tool to segment the text data; calculate the frequency of each word, count the number of texts containing each word, and calculate the TF-IDF value of each word; compose the TF-IDF value of each word into a feature vector to obtain TF-IDF feature data;

[0027] Generate N-gram sequences based on the text data after word segmentation; for each N-gram sequence, calculate its frequency of occurrence in the text data, and calculate the conditional probability of the N-gram sequence based on the frequency of occurrence; collect the frequency of occurrence and conditional probability corresponding to each N-gram sequence to obtain N-gram feature data;

[0028] Integrate the acquired TF-IDF feature data and N-gram feature data to obtain a text feature dataset;

[0029] S42, preprocessing the image data: converting the image data into a grayscale image and performing binarization processing; ensuring that the text area in the image data remains horizontal by rotating and cropping; determining the position and range of the text by detecting the text area in the image data; performing character segmentation in the text area, and segmenting the detected text area into individual text blocks; collecting all the individual text blocks to obtain an image feature data set;

[0030] S43. Preprocessing of environmental data: Identify and eliminate outliers in environmental data, and use interpolation to fill in missing environmental data; standardize and normalize the filled environmental data and convert it into a standard normal distribution with a mean of 0 and a standard deviation of 1; use principal component analysis to extract key environmental features, and then obtain an environmental feature data set.

[0031] Furthermore, the method of weightedly fusing the font feature dataset, the image feature dataset and the environment feature dataset to obtain a comprehensive feature dataset includes:

[0032] The font feature dataset, the image feature dataset and the environment feature dataset are fused by a weighted formula to obtain a comprehensive feature dataset, where the font feature dataset is denoted as E1, the image feature dataset is denoted as E2, and the environment feature dataset is denoted as E3;

[0033] The weighted formula is: GQ=E1·σ1+E2·σ2+E3·σ3; wherein σ1 is the weight coefficient of the font feature data set; σ2 is the weight coefficient of the image feature data set; and σ3 is the weight coefficient of the environment feature data set.

[0034] Furthermore, the method for training and acquiring a text recognition model based on a comprehensive feature data set includes:

[0035] Constructing a text recognition model, which includes an input layer, a convolution layer, a pooling layer, a fully connected layer and an output layer; the input layer is used to input a historical comprehensive feature data set, and the output layer is used to output a text recognition result; the text recognition model is a convolutional neural network model;

[0036] Use the cross entropy loss function as the loss function to measure the error between the model's predicted value and the actual value; use the training set data to train the model and minimize the loss function through the Adam optimizer; use the validation set to evaluate the performance of the model and tune the model's hyperparameters until the model performance reaches the preset stopping condition; use the test set to evaluate the model's performance in the prediction task and input the current comprehensive feature data set into the trained text recognition model to obtain the text recognition result.

[0037] Furthermore, the method of comparing the predicted text recognition result with the preset text recognition result to determine whether the text recognition results are consistent includes:

[0038] If the predicted text recognition result matches the preset text recognition result, the text recognition is determined to be correct;

[0039] If the predicted text recognition result does not match the preset text recognition result, the text recognition is judged to be wrong;

[0040] Furthermore, if the text recognition results are consistent, a semantic recognition model is obtained by training a text feature data set, and a method for generating corresponding text interpretations by the semantic recognition model includes:

[0041] The data set is divided into a training set, a validation set and a test set; the sample set is a subset of the data set, and each sample set includes a historical text feature data set and a corresponding semantic label; a semantic recognition model is constructed, including an input layer, a hidden layer and an output layer; the input layer is used to input the historical text feature data set, and the output layer is used to generate the corresponding semantic label; the semantic recognition model is a recurrent neural network model;

[0042] Initialize the weight matrix and bias term, and use the training set to train the model; select the cross entropy loss function to measure the difference between the model prediction result and the true label; select the Adam optimizer to update the model parameters to minimize the loss function; use the validation set to evaluate the performance of the model, and measure the performance of the model by calculating the accuracy index; adjust the structure and parameters of the model and retrain according to the performance feedback of the validation set accuracy to obtain a trained semantic recognition model;

[0043] The predicted text recognition results that are consistent with the preset text recognition results are collected, segmented and encoded, converted into input sequences of the same length, and input into the trained semantic recognition model to obtain the corresponding text interpretation.

[0044] Furthermore, the method of converting the text recognition result and the corresponding text interpretation into natural speech output using TTS technology includes:

[0045] The text recognition results and the corresponding text interpretations are collected through the software terminal, input into the selected TTS engine, and the speech speed, pitch, volume and timbre parameters are set; the text recognition results and the corresponding text interpretations are converted into speech waveforms through the TTS engine to generate audio files; the audio files are listened to and the parameters are adjusted to optimize the output effect; the optimized audio files are embedded into the application to obtain natural speech output.

[0046] Natural language processing methods based on deep learning, including:

[0047] S1, collect font data, text data, image data and environment data;

[0048] S2, preprocessing the collected font data, text data, image data and environment data to obtain a font feature data set, a text feature data set, an image feature data set and an environment feature data set; weighted fusion of the font feature data set, the image feature data set and the environment feature data set to obtain a comprehensive feature data set;

[0049] S3, training and acquiring a text recognition model according to the comprehensive feature data set, and predicting a text recognition result through the text recognition model; comparing the predicted text recognition result with the preset text recognition result to determine whether the text recognition results are consistent;

[0050] S4. If the text recognition results are inconsistent, the recognition is performed again; if the text recognition results are consistent, a semantic recognition model is obtained through text feature data training, and the corresponding text interpretation is generated through the semantic recognition model; the text recognition results and the corresponding text interpretation are collected through the software terminal;

[0051] S5. Use TTS technology to convert text recognition results and corresponding text interpretations into natural speech output.

[0052] The technical effects and advantages of the natural language processing platform based on deep learning of the present invention are as follows:

[0053] Through precise contour feature extraction and geometric parameterization, the accuracy of font recognition can be significantly improved and the misrecognition rate can be reduced, achieving higher-precision font recognition and matching, especially in the recognition of subtle differences between similar fonts. Structural feature data and attribute coding data can help to effectively classify and organize fonts for easy retrieval and use. Standardized font feature data sets can help improve the compatibility and consistency of fonts between different software and platforms, and reduce visual problems caused by font rendering differences.

[0054] The threshold adjustment formula allows the threshold to be dynamically adjusted according to factors such as the complexity of the actual contour path and the curvature change rate, which improves the processing flexibility and enables the algorithm to adapt to different font styles and contour features. The quantity adjustment formula can accurately calculate the number of control points based on factors such as the complexity of the contour path, the total length, and the size of the display area, which helps to reduce the number of unnecessary control points and reduce the calculation complexity while ensuring the contour accuracy.

[0055] By calculating TF-IDF feature data, the feature dimension is reduced to improve the calculation efficiency. At the same time, it can reflect the importance of words in the text and help distinguish different text contents. Calculating N-gram feature data can capture local sequence patterns in the text, which helps to identify key patterns in tasks such as text classification and sentiment analysis.

[0056] Through voice output, users can obtain information more intuitively without spending time and energy reading large amounts of text, which greatly improves the user experience; TTS technology can be widely used in multiple fields to provide users with more intelligent and convenient services. In the field of education, TTS technology can convert text content such as textbooks and articles into voice to help students learn and understand better. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 This is a schematic diagram of the structure of a natural language processing platform based on deep learning of the present invention;

[0058] Figure 2This is a flow chart of the natural language processing method based on deep learning of the present invention. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] Example 1

[0061] See also Figure 1 As shown, the natural language processing platform based on deep learning in this embodiment includes:

[0062] A data collection module, used for collecting font data, text data, image data and environmental data;

[0063] The data processing module is used to pre-process the collected font data, text data, image data and environmental data to obtain a font feature data set, a text feature data set, an image feature data set and an environmental feature data set; and weightedly fuse the font feature data set, the image feature data set and the environmental feature data set to obtain a comprehensive feature data set;

[0064] The text recognition module is used to train and obtain a text recognition model based on the comprehensive feature data set, and predict the text recognition result through the text recognition model; compare the predicted text recognition result with the preset text recognition result to determine whether the text recognition results are consistent;

[0065] Semantic recognition module: if the text recognition results are inconsistent, the recognition is performed again; if the text recognition results are consistent, the semantic recognition model is obtained through text feature data set training, and the corresponding text interpretation is generated through the semantic recognition model; the text recognition results and the corresponding text interpretation are collected through the software terminal;

[0066] The speech output module uses TTS technology to convert the text recognition results and the corresponding text interpretation into natural speech output; each module is connected by wired and / or wireless means.

[0067] Font data includes font style data, font size data, font thickness data and spacing data; text data includes grammar data, character data, text content data and text language data; image data includes text image data; environmental data includes light intensity data and noise level data of the collection environment.

[0068] The method of preprocessing the collected font data to obtain a font feature data set includes:

[0069] The collected font data is respectively extracted with contour feature data, geometric parameter feature data, structural feature data and attribute coding data. The specific method is as follows:

[0070] S31, using Canny edge detection to identify the outline of the font style data; tracking the outline along the detected edge, and recording the coordinate points of the outline to form an outline path; measuring the average width of the outline path, thereby obtaining line width features; analyzing and quantifying the shape of the outline corners, thereby obtaining corner shape features; collecting line width features and corner shape features to obtain outline feature data;

[0071] S32, geometrically parameterizing the font data; simplifying the outline of the font style data using the Douglas-Peucker algorithm; selecting the starting point and the ending point on the outline path, and setting a threshold α;

[0072] The value of the threshold α is adjusted by the threshold adjustment formula; the threshold adjustment formula is: Among them, α′ is the adjusted threshold; n is the total number of coordinate points on the contour path; j is the index of the coordinate point; θ j is the normal direction of the jth coordinate point; θ j+1 is the normal direction of the j+1th coordinate point; is the curvature change rate of the contour path; γ1 is the weight coefficient of the curvature change rate of the contour path; x b and b are the horizontal and vertical coordinates of the end point of the contour path; x1 and y1 are the horizontal and vertical coordinates of the starting point of the contour path; b is the index of the end point of the contour path; is the closure degree of the contour path; dk ​​is the total length of the contour path; γ2 is the weight coefficient of the closure degree of the contour path;

[0073] For example: Assume that the total length of a font's outline path is 12, and perform geometric parameterization on the coordinate points of the font's outline path; the font's outline path coordinate points are: [(0,0), (1,2), (2,3), (3,5), (5,6), (6,7), (8,6), (10,5)]; the starting point of this group of font outline path coordinate points is (0,0), and the ending point is (10,5); the normal direction on the font's outline path gradually increases with the point, θ = 10°, 15°, ..., 40°;

[0074] Assume that the closure degree of the contour path is 0.9, the curvature change rate is 0.15, and the weight coefficients γ1 and γ2 are 0.3 and 0.2 respectively; then the threshold adjustment formula can be used to obtain α′=0.15×0.3+0.9×0.2=0.225, that is, the value of the threshold α is 0.225;

[0075] For each coordinate point on the contour path, calculate the vertical distance between the point and the starting point and the end point; for any coordinate point P i , if d(P i )<α, then the coordinate point is discarded; otherwise, if d(P i )>α, then keep the coordinate point; if d(P i )=α, when the coordinate point P i If it is a bending point or turning point on the contour path, the coordinate point is retained, otherwise it is discarded; where d(P i ) is the coordinate point P i The vertical distance between the starting point and the end point; P i is the i-th coordinate point; i is the index of the coordinate point; repeat the above process until all coordinate points are processed; collect all the retained coordinate points to obtain the contour point set; select β control points based on the contour point set, usually the number of control points is less than the number of original contour points;

[0076] The number of control points β is determined by the quantity adjustment formula; the quantity adjustment formula is: Among them, β′ is the number of control points after adjustment; n is the total number of coordinate points on the contour path; o is the order of the Bezier curve; dk is the total length of the contour path; is the square root of contour complexity; w is the width of the contour path display area; h is the height of the contour path display area; μ1 and μ2 are weight coefficients; is the operator for rounding up;

[0077] For example: suppose we have a contour path with a total of 100 coordinate points, the order of the Bezier curve is 3, the total length of the contour path is 10, the square root of the contour complexity is 2, the width of the contour path display area is 5, the height is 4, and the weight coefficients μ1 and μ2 are set to 0.5 and 0.7 respectively;

[0078] According to the quantity adjustment formula, we can get: That is, the number of control points is 19;

[0079] Use the least squares method to fit the control points into a Bezier curve. The specific mathematical formula is: B(t) = (1-t) 3 D0+3(1-t) 2 tD1+3(1-t)t 2 D2+t3 D3, t∈[0,1]; where B(t) is the result of the Bezier curve equation; t is a parameter variable used to define the position of any point on the curve; D0, D1, D2 and D3 are control points; collect the contour point set and the Bezier curve to obtain the geometric parameter feature data;

[0080] S33, performing a ratio analysis based on the font size data and the spacing data in the font data, calculating the height and width ratios of different fonts, and the ratio relationship between different fonts, to obtain structural feature data;

[0081] S34, attribute mapping is performed on the font thickness data, the font thickness is mapped to a numerical range, and the font thickness is represented by a numerical attribute code; for example, the mapped numerical range is an integer of 0-100, where 0 represents the thinnest and 100 represents the thickest; the attribute codes corresponding to all the font thickness data are collected to obtain attribute code data;

[0082] The acquired contour feature data, geometric parameter feature data, structural feature data and attribute coding data are integrated to obtain a font feature data set.

[0083] The method of preprocessing the collected text data, image data and environmental data to obtain a text feature data set, an image feature data set and an environmental feature data set includes:

[0084] S41. Preprocessing of text data: Clean the text data to remove text noise, and use the jieba tool to segment the text data; calculate the frequency of each word, count the number of texts containing each word, and calculate the TF-IDF value of each word; compose the TF-IDF value of each word into a feature vector to obtain TF-IDF feature data;

[0085] Generate N-gram sequences based on the text data after word segmentation; for each N-gram sequence, calculate its frequency of occurrence in the text data, and calculate the conditional probability of the N-gram sequence based on the frequency of occurrence; collect the frequency of occurrence and conditional probability corresponding to each N-gram sequence to obtain N-gram feature data;

[0086] Integrate the acquired TF-IDF feature data and N-gram feature data to obtain a text feature dataset;

[0087] For example, suppose there is a text: "Today's weather is good, and today's mood is also good"; this text is segmented, and the segmentation result is ["today", "weather", "good", "today", "mood", "also", "very good"]; generate a 2-gram sequence to get ["today's weather", "good weather", "today's mood", "mood also", "also very good"]; count the number of occurrences of each 2-gram and calculate its frequency. In this text:

[0088] 2-gram Occurrence Frequency "Today's weather" 1 0.2 "Nice weather" 1 0.2 "Today's mood" 1 0.2 "My mood" 1 0.2 "Very good" 1 0.2

[0089] Calculate the conditional probability of each 2-gram, that is, the probability of generating the second word given the first word; for example, the conditional probability of "today's weather" = 0.5; the conditional probability of "nice weather" = 1.0; the conditional probability of "today's mood" = 0.5; the conditional probability of "mood also" = 1.0; the conditional probability of "also very good" = 1.0; collect the occurrence frequency and conditional probability corresponding to each N-gram sequence to obtain N-gram feature data;

[0090] S42, preprocessing the image data: converting the image data into a grayscale image and performing binarization processing; ensuring that the text area in the image data remains horizontal by rotating and cropping; determining the position and range of the text by detecting the text area in the image data; performing character segmentation in the text area, and segmenting the detected text area into individual text blocks; collecting all the individual text blocks to obtain an image feature data set;

[0091] S43. Preprocessing of environmental data: Identify and eliminate outliers in environmental data, and use interpolation to fill in missing environmental data; standardize and normalize the filled environmental data and convert it into a standard normal distribution with a mean of 0 and a standard deviation of 1; use principal component analysis to extract key environmental features, and then obtain an environmental feature data set.

[0092] The method of weightedly fusing the font feature dataset, the image feature dataset and the environment feature dataset to obtain a comprehensive feature dataset includes:

[0093] The font feature dataset, the image feature dataset and the environment feature dataset are fused by a weighted formula to obtain a comprehensive feature dataset, where the font feature dataset is denoted as E1, the image feature dataset is denoted as E2, and the environment feature dataset is denoted as E3;

[0094] The weighted formula is: GQ=E1·σ1+E2·σ2+E3·σ3; where σ1 is the weight coefficient of the font feature data set; σ2 is the weight coefficient of the image feature data set; and σ3 is the weight coefficient of the environmental feature data set.

[0095] Methods for training and obtaining a text recognition model based on a comprehensive feature dataset include:

[0096] Construct a text recognition model, which includes an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer. The input layer is used to input a historical comprehensive feature data set, and the output layer is used to output text recognition results. The text recognition model is a convolutional neural network model.

[0097] Use the cross entropy loss function as the loss function to measure the error between the model's predicted value and the actual value; use the training set data to train the model and minimize the loss function through the Adam optimizer; use the validation set to evaluate the performance of the model and tune the model's hyperparameters until the model performance reaches the preset stopping condition; use the test set to evaluate the model's performance in the prediction task and input the current comprehensive feature data set into the trained text recognition model to obtain the text recognition result.

[0098] The method of comparing the predicted text recognition result with the preset text recognition result to determine whether the text recognition results are consistent includes:

[0099] If the predicted text recognition result matches the preset text recognition result, the text recognition is determined to be correct;

[0100] If the predicted text recognition result does not match the preset text recognition result, the text recognition is judged to be wrong;

[0101] If the text recognition results are consistent, a semantic recognition model is obtained by training the text feature data set. The method of generating the corresponding text interpretation through the semantic recognition model includes:

[0102] The dataset is divided into a training set, a validation set, and a test set; the sample set is a subset of the dataset, and each sample set includes a historical text feature dataset and corresponding semantic labels; a semantic recognition model is constructed, including an input layer, a hidden layer, and an output layer; the input layer is used to input the historical text feature dataset, and the output layer is used to generate corresponding semantic labels; the semantic recognition model is a recurrent neural network model;

[0103] Initialize the weight matrix and bias term, and use the training set to train the model; select the cross entropy loss function to measure the difference between the model prediction result and the true label; select the Adam optimizer to update the model parameters to minimize the loss function; use the validation set to evaluate the performance of the model, and measure the performance of the model by calculating the accuracy index; adjust the structure and parameters of the model and retrain according to the performance feedback of the validation set accuracy to obtain a trained semantic recognition model;

[0104] The predicted text recognition results that are consistent with the preset text recognition results are collected, segmented and encoded, converted into input sequences of the same length, and input into the trained semantic recognition model to obtain the corresponding text interpretation.

[0105] Methods for converting text recognition results and corresponding text interpretations into natural speech output using TTS technology include:

[0106] The text recognition results and the corresponding text interpretations are collected through the software terminal, input into the selected TTS engine, and the speech speed, pitch, volume and timbre parameters are set; the text recognition results and the corresponding text interpretations are converted into speech waveforms through the TTS engine to generate audio files; the audio files are listened to and the parameters are adjusted to optimize the output effect; the optimized audio files are embedded into the application to obtain natural speech output.

[0107] This embodiment can significantly improve the accuracy of font recognition and reduce the misrecognition rate through precise contour feature extraction and geometric parameterization, and achieve higher-precision font recognition and matching, especially in the recognition of subtle differences between similar fonts. The structural feature data and attribute coding data are helpful for effective classification and organization of fonts, which is convenient for retrieval and use. The standardized font feature data set helps to improve the compatibility and consistency of fonts between different software and platforms, and reduce visual problems caused by font rendering differences.

[0108] The threshold adjustment formula allows the threshold to be dynamically adjusted according to factors such as the complexity of the actual contour path and the curvature change rate, which improves the processing flexibility and enables the algorithm to adapt to different font styles and contour features. The quantity adjustment formula can accurately calculate the number of control points based on factors such as the complexity of the contour path, the total length, and the size of the display area, which helps to reduce the number of unnecessary control points and reduce the calculation complexity while ensuring the contour accuracy.

[0109] By calculating TF-IDF feature data, the feature dimension is reduced to improve the calculation efficiency. At the same time, it can reflect the importance of words in the text and help distinguish different text contents. Calculating N-gram feature data can capture local sequence patterns in the text, which helps to identify key patterns in tasks such as text classification and sentiment analysis.

[0110] Through voice output, users can obtain information more intuitively without spending time and energy reading large amounts of text, which greatly improves the user experience; TTS technology can be widely used in multiple fields to provide users with more intelligent and convenient services. In the field of education, TTS technology can convert text content such as textbooks and articles into voice to help students learn and understand better.

[0111] Example 2

[0112] See also Figure 2 As shown, the part not described in detail in this embodiment is described in Example 1, and a natural language processing method based on deep learning is provided, including:

[0113] S1, collect font data, text data, image data and environment data;

[0114] S2, preprocessing the collected font data, text data, image data and environment data to obtain a font feature data set, a text feature data set, an image feature data set and an environment feature data set; weighted fusion of the font feature data set, the image feature data set and the environment feature data set to obtain a comprehensive feature data set;

[0115] S3, training and acquiring a text recognition model according to the comprehensive feature data set, and predicting a text recognition result through the text recognition model; comparing the predicted text recognition result with the preset text recognition result to determine whether the text recognition results are consistent;

[0116] S4. If the text recognition results are inconsistent, the recognition is performed again; if the text recognition results are consistent, a semantic recognition model is obtained through text feature data training, and the corresponding text interpretation is generated through the semantic recognition model; the text recognition results and the corresponding text interpretation are collected through the software terminal;

[0117] S5. Use TTS technology to convert text recognition results and corresponding text interpretations into natural speech output.

[0118] Since the electronic device introduced in this embodiment is an electronic device used to implement the natural language processing platform and processing method based on deep learning in the embodiment of this application, based on the natural language processing platform and processing method based on deep learning introduced in the embodiment of this application, the technical personnel of this field can understand the specific implementation of the electronic device of this embodiment and its various variations, so how the electronic device implements the method in the embodiment of this application is not described in detail here. As long as the technical personnel of this field implement the electronic device used by the natural language processing platform and processing method based on deep learning in the embodiment of this application, they all belong to the scope of protection of this application.

[0119] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters and thresholds in the formula are set by technicians in this field according to actual conditions.

[0120] The above are only preferred embodiments of the present invention. The protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technical users in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A natural language processing platform based on deep learning, characterized by: include: A data collection module, used for collecting font data, text data, image data and environmental data; The data processing module is used to pre-process the collected font data, text data, image data and environmental data to obtain a font feature data set, a text feature data set, an image feature data set and an environmental feature data set; The font feature dataset, the image feature dataset and the environment feature dataset are weightedly fused to obtain a comprehensive feature dataset; The text recognition module is used to train and obtain a text recognition model based on the comprehensive feature data set, and predict the text recognition result through the text recognition model; compare the predicted text recognition result with the preset text recognition result to determine whether the text recognition results are consistent; Semantic recognition module, if the text recognition results are inconsistent, re-recognition; If the text recognition results are consistent, a semantic recognition model is obtained through training of the text feature data set, and the corresponding text interpretation is generated through the semantic recognition model; the text recognition results and the corresponding text interpretation are collected through the software terminal; The speech output module uses TTS technology to convert the text recognition results and the corresponding text interpretation into natural speech output; each module is connected by wired and / or wireless means.

2. The natural language processing platform based on deep learning according to claim 1, characterized in that: The font data includes font style data, font size data, font thickness data and spacing data; the text data includes grammar data, character data, text content data and text language data; the image data includes text image data; and the environmental data includes light intensity data and noise level data of the collection environment.

3. The natural language processing platform based on deep learning according to claim 2, characterized in that: The method of preprocessing the collected font data to obtain a font feature data set includes: The collected font data is respectively extracted with contour feature data, geometric parameter feature data, structural feature data and attribute coding data. The specific method is as follows: S31, using Canny edge detection to identify the outline of the font style data; tracking the outline along the detected edge, and recording the coordinate points of the outline to form an outline path; measuring the average width of the outline path, thereby obtaining line width features; analyzing and quantifying the shape of the outline corners, thereby obtaining corner shape features; collecting line width features and corner shape features to obtain outline feature data; S32, geometrically parameterizing the font data; simplifying the outline of the font style data using the Douglas-Peucker algorithm; selecting the starting point and the ending point on the outline path, and setting a threshold α; The value of the threshold α is adjusted by a threshold adjustment formula; the threshold adjustment formula is: Among them, α′ is the adjusted threshold; n is the total number of coordinate points on the contour path; j is the index of the coordinate point; θ j is the normal direction of the jth coordinate point; θ j+1 is the normal direction of the j+1th coordinate point; is the curvature change rate of the contour path; γ1 is the weight coefficient of the curvature change rate of the contour path; x b and b are the horizontal and vertical coordinates of the end point of the contour path; x1 and y1 are the horizontal and vertical coordinates of the starting point of the contour path; b is the index of the end point of the contour path; is the closure degree of the contour path; dk ​​is the total length of the contour path; γ2 is the weight coefficient of the closure degree of the contour path; For each coordinate point on the contour path, calculate the vertical distance between the point and the starting point and the end point; for any coordinate point P i , if d(P i )<α, then the coordinate point is discarded; otherwise, if d(P i )>α, then keep the coordinate point; if d(P i )=α, when the coordinate point P i If it is a bending point or turning point on the contour path, the coordinate point is retained, otherwise it is discarded; where d(P i ) is the coordinate point P i The vertical distance between the starting point and the end point; P i is the i-th coordinate point; i is the index of the coordinate point; repeat the above process until all coordinate points are processed; collect all the retained coordinate points to obtain the contour point set; select β control points based on the contour point set; The number β of the control points is determined by a number adjustment formula; the number adjustment formula is: Among them, β′ is the number of control points after adjustment; n is the total number of coordinate points on the contour path; o is the order of the Bezier curve; dk is the total length of the contour path; is the square root of contour complexity; w is the width of the contour path display area; h is the height of the contour path display area; μ1 and μ2 are weight coefficients; is the operator for rounding up; The control points are fitted into Bezier curves using the least square method; the contour point set and the Bezier curve are collected to obtain geometric parameter feature data; S33, performing a ratio analysis based on the font size data and the spacing data in the font data, calculating the height and width ratios of different fonts, and the ratio relationship between different fonts, to obtain structural feature data; S34, attribute mapping is performed on the font thickness data, the font thickness is mapped to a numerical range, and the font thickness is represented by a numerical attribute code; the attribute codes corresponding to all the font thickness data are collected to obtain attribute code data; The acquired contour feature data, geometric parameter feature data, structural feature data and attribute coding data are integrated to obtain a font feature data set.

4. The natural language processing platform based on deep learning according to claim 3, characterized in that: The method of preprocessing the collected text data, image data and environmental data to obtain a text feature data set, an image feature data set and an environmental feature data set includes: S41. Preprocessing of text data: Clean the text data to remove text noise, and use the jieba tool to segment the text data; calculate the frequency of each word, count the number of texts containing each word, and calculate the TF-IDF value of each word; compose the TF-IDF value of each word into a feature vector to obtain TF-IDF feature data; Generate N-gram sequences based on the text data after word segmentation; for each N-gram sequence, calculate its frequency of occurrence in the text data, and calculate the conditional probability of the N-gram sequence based on the frequency of occurrence; collect the frequency of occurrence and conditional probability corresponding to each N-gram sequence to obtain N-gram feature data; Integrate the acquired TF-IDF feature data and N-gram feature data to obtain a text feature dataset; S42, preprocessing the image data: converting the image data into a grayscale image and performing binarization processing; ensuring that the text area in the image data remains horizontal by rotating and cropping; determining the position and range of the text by detecting the text area in the image data; performing character segmentation in the text area, and segmenting the detected text area into individual text blocks; collecting all the individual text blocks to obtain an image feature data set; S43. Preprocessing of environmental data: Identify and eliminate outliers in environmental data, and use interpolation to fill in missing environmental data; standardize and normalize the filled environmental data and convert it into a standard normal distribution with a mean of 0 and a standard deviation of 1; use principal component analysis to extract key environmental features, and then obtain an environmental feature data set.

5. The natural language processing platform based on deep learning according to claim 4, characterized in that: The method of weightedly fusing the font feature dataset, the image feature dataset and the environment feature dataset to obtain a comprehensive feature dataset comprises: The font feature dataset, the image feature dataset and the environment feature dataset are fused by a weighted formula to obtain a comprehensive feature dataset, where the font feature dataset is denoted as E1, the image feature dataset is denoted as E2, and the environment feature dataset is denoted as E3; The weighted formula is: GQ=E1·σ1+E2·σ2+E3·σ3; wherein σ1 is the weight coefficient of the font feature data set; σ2 is the weight coefficient of the image feature data set; and σ3 is the weight coefficient of the environment feature data set.

6. The natural language processing platform based on deep learning according to claim 5, characterized in that: The method for training and acquiring a text recognition model based on a comprehensive feature data set includes: Constructing a text recognition model, which includes an input layer, a convolution layer, a pooling layer, a fully connected layer and an output layer; the input layer is used to input a historical comprehensive feature data set, and the output layer is used to output a text recognition result; the text recognition model is a convolutional neural network model; Use the cross entropy loss function as the loss function to measure the error between the model's predicted value and the actual value; use the training set data to train the model and minimize the loss function through the Adam optimizer; use the validation set to evaluate the performance of the model and tune the model's hyperparameters until the model performance reaches the preset stopping condition; use the test set to evaluate the model's performance in the prediction task and input the current comprehensive feature data set into the trained text recognition model to obtain the text recognition result.

7. The natural language processing platform based on deep learning according to claim 6, characterized in that: The method of comparing the predicted text recognition result with the preset text recognition result to determine whether the text recognition results are consistent includes: If the predicted text recognition result matches the preset text recognition result, the text recognition is determined to be correct; If the predicted text recognition result does not match the preset text recognition result, the text recognition is determined to be wrong.

8. The natural language processing platform based on deep learning according to claim 7, characterized in that: If the text recognition results are consistent, a semantic recognition model is obtained by training the text feature data set, and a method for generating corresponding text interpretations by the semantic recognition model includes: The data set is divided into a training set, a validation set and a test set; the sample set is a subset of the data set, and each sample set includes a historical text feature data set and a corresponding semantic label; a semantic recognition model is constructed, including an input layer, a hidden layer and an output layer; the input layer is used to input the historical text feature data set, and the output layer is used to generate the corresponding semantic label; the semantic recognition model is a recurrent neural network model; Initialize the weight matrix and bias term, and use the training set to train the model; select the cross entropy loss function to measure the difference between the model prediction result and the true label; select the Adam optimizer to update the model parameters to minimize the loss function; use the validation set to evaluate the performance of the model, and measure the performance of the model by calculating the accuracy index; adjust the structure and parameters of the model and retrain according to the performance feedback of the validation set accuracy to obtain a trained semantic recognition model; The predicted text recognition results that are consistent with the preset text recognition results are collected, segmented and encoded, converted into input sequences of the same length, and input into the trained semantic recognition model to obtain the corresponding text interpretation.

9. The natural language processing platform based on deep learning according to claim 8, characterized in that: The method of converting the text recognition result and the interpretation of the corresponding text into natural speech output by using TTS technology includes: The text recognition results and the corresponding text interpretations are collected through the software terminal, input into the selected TTS engine, and the speech speed, pitch, volume and timbre parameters are set; the text recognition results and the corresponding text interpretations are converted into speech waveforms through the TTS engine to generate audio files; the audio files are listened to and the parameters are adjusted to optimize the output effect; the optimized audio files are embedded into the application to obtain natural speech output.

10. A natural language processing method based on deep learning, applied to a natural language processing platform based on deep learning as claimed in any one of claims 1 to 9, characterized in that: include: S1, collect font data, text data, image data and environment data; S2, preprocessing the collected font data, text data, image data and environment data to obtain a font feature data set, a text feature data set, an image feature data set and an environment feature data set; weighted fusion of the font feature data set, the image feature data set and the environment feature data set to obtain a comprehensive feature data set; S3, training and acquiring a text recognition model according to the comprehensive feature data set, and predicting a text recognition result through the text recognition model; comparing the predicted text recognition result with the preset text recognition result to determine whether the text recognition results are consistent; S4. If the text recognition results are inconsistent, re-recognition is performed; If the text recognition results are consistent, a semantic recognition model is obtained through text feature data training, and the corresponding text interpretation is generated through the semantic recognition model; the text recognition results and the corresponding text interpretation are collected through the software terminal; S5. Use TTS technology to convert text recognition results and corresponding text interpretations into natural speech output.

Images