Text information recognition method, device and storage medium

Abstract

The present application provides a text information recognition method, device and storage medium, wherein the method comprises: obtaining content to be recognized including text information, determining a first recognition result for the text information in the content to be recognized based on a pinyin sequence and a recognition model corresponding to the content to be recognized, and determining a second recognition result for the text information in the content to be recognized based on a Chinese character sequence corresponding to the content to be recognized and the recognition model, and finally obtaining a text information recognition result of the content to be recognized based on a first vector corresponding to the first recognition result and a second vector corresponding to the second recognition result. The technical solution can improve the recognition effect of the text information in the content to be recognized, improve the recognition accuracy of the text information, and enhance the user experience by processing the pinyin sequence and the Chinese character sequence.

Description

Technical Field

[0001] The present application relates to the field of information processing technology, and in particular to a text information recognition method, device and storage medium. Background Art

[0002] With the continuous development of science and technology, users can purchase the goods they need on e-commerce websites without leaving their homes. After receiving the purchased goods, users usually comment on the quality of the purchased goods and the seller's service quality, thereby providing a reference for other buyers.

[0003] In the prior art, user comments may contain advertisements, uncivilized language, and other types of text information. Usually, when a new user comment is added to an e-commerce website, it can be determined whether the user comment contains the above types of text information. If so, such text information is processed to ensure the user experience of other users.

[0004] However, existing text information recognition methods can only recognize part of the target text information in the content to be recognized (for example, user reviews), and have the problems of low recognition accuracy and poor user experience. Summary of the invention

[0005] The present application provides a text information recognition method, device and storage medium to solve the problems of low text information recognition accuracy and poor user experience in existing text information recognition methods.

[0006] The first aspect of the present application provides a text information recognition method, comprising:

[0007] Acquire content to be identified, wherein the content to be identified includes: text information;

[0008] Determine a first recognition result for text information in the content to be recognized based on the pinyin sequence corresponding to the content to be recognized and the recognition model;

[0009] Determining a second recognition result for text information in the content to be recognized based on the Chinese character sequence corresponding to the content to be recognized and the recognition model;

[0010] Based on the first vector corresponding to the first recognition result and the second vector corresponding to the second recognition result, a text information recognition result of the content to be recognized is obtained.

[0011] In this embodiment, the recognition model is used to process the pinyin sequence and the Chinese character sequence of the content to be recognized, which can improve the recognition effect of the text information in the content to be recognized, improve the recognition accuracy of the text information, and enhance the user experience.

[0012] In a possible design of the first aspect, obtaining the text information recognition result of the to-be-recognized content based on the first vector corresponding to the first recognition result and the second vector corresponding to the second recognition result includes:

[0013] Based on the principle of superposition of vectors at the same position, a first vector corresponding to the first recognition result and a second vector corresponding to the second recognition result are combined to obtain a composite vector;

[0014] The synthesized vector is processed through a fully connected network and then input into a discriminant model to obtain a text information recognition result of the content to be recognized.

[0015] In another possible design of the first aspect, before determining a first recognition result for text information in the content to be recognized based on the pinyin sequence corresponding to the content to be recognized and the recognition model, the method further includes:

[0016] Preprocessing the content to be recognized to obtain a Chinese character sequence corresponding to the content to be recognized;

[0017] The recognition model is used to convert the Chinese character sequence into the pinyin sequence.

[0018] In another possible design of the first aspect, before determining a first recognition result for text information in the content to be recognized based on the pinyin sequence corresponding to the content to be recognized and the recognition model, the method further includes:

[0019] Acquire a historical recognition sample set, wherein the historical recognition sample set includes: a plurality of historical recognition contents carrying text information;

[0020] Based on the pinyin sequence corresponding to each historical recognition content in the historical recognition sample set, the first sub-network in the preset network is trained to obtain the first recognition network included in the recognition model, wherein the first recognition network has the function of recognizing text information in the pinyin sequence;

[0021] The second sub-network in the preset network is trained based on the Chinese character sequence corresponding to each historical recognition content in the historical recognition sample set to obtain the second recognition network included in the recognition model, and the second recognition network has the function of recognizing text information in the Chinese character sequence.

[0022] In the above possible design of the first aspect, before training the first subnetwork in the preset network based on the pinyin sequence corresponding to each historical recognition content in the historical recognition sample set to obtain the first recognition network included in the recognition model, the method further includes:

[0023] Preprocessing each piece of historical recognition content in the historical recognition sample set to obtain a Chinese character sequence corresponding to each piece of historical recognition content;

[0024] Obtaining a pinyin annotation result of a Chinese character sequence corresponding to each piece of historical recognition content, wherein the pinyin annotation result is manually determined based on context information of each piece of historical recognition content;

[0025] Based on the pinyin annotation result, converting the Chinese character sequence corresponding to each historical recognition content in the historical recognition sample set into a pinyin sequence;

[0026] The third sub-network in the preset network is trained based on the Chinese character sequence and the pinyin sequence corresponding to each historical recognition content in the historical recognition sample set to obtain the conversion network included in the recognition model, and the conversion network has the function of converting the Chinese character sequence into the pinyin sequence.

[0027] In another possible design of the first aspect, the method further includes:

[0028] Detecting whether there is text information in a text word library in the Chinese character sequence and / or the pinyin sequence corresponding to the content to be recognized, and obtaining a third recognition result, wherein the text word library includes the Chinese character sequence and the pinyin sequence corresponding to the text information determined in the historical recognition process;

[0029] Based on the third recognition result, the text information recognition result is updated, and the updated text information recognition result is used as the text information recognition result of the content to be recognized.

[0030] In any of the above possible designs of the first aspect, the content to be identified includes: user comment content on the e-commerce website, and the text information in the content to be identified includes: keywords in the user comment content.

[0031] In any of the above possible designs of the first aspect, the content to be identified includes: the content to be desensitized in the cloud storage system, and the text information in the content to be identified includes: the sensitive information in the content to be desensitized.

[0032] A second aspect of the present application provides a text information recognition device, including: an acquisition module, a recognition module and a determination module;

[0033] The acquisition module is used to acquire the content to be identified, and the content to be identified includes: text information;

[0034] The recognition module is used to determine a first recognition result for the text information in the content to be recognized based on the pinyin sequence corresponding to the content to be recognized and the recognition model, and to determine a second recognition result for the text information in the content to be recognized based on the Chinese character sequence corresponding to the content to be recognized and the recognition model;

[0035] The determination module is used to obtain a text information recognition result of the content to be recognized based on a first vector corresponding to the first recognition result and a second vector corresponding to the second recognition result.

[0036] In a possible design of the second aspect, the determination module is specifically used to combine the first vector corresponding to the first recognition result and the second vector corresponding to the second recognition result based on the principle of superposition of vectors at the same position to obtain a synthetic vector, and input the synthetic vector into the discriminant model after processing it through a fully connected network to obtain a text information recognition result of the content to be recognized.

[0037] In another possible design of the second aspect, the device further includes: a processing module;

[0038] The processing module is used to preprocess the content to be recognized before the recognition module determines the first recognition result for the text information in the content to be recognized based on the pinyin sequence corresponding to the content to be recognized and the recognition model, so as to obtain the Chinese character sequence corresponding to the content to be recognized, and use the recognition model to convert the Chinese character sequence into the pinyin sequence.

[0039] In another possible design of the second aspect, the acquisition module is further used to acquire a historical recognition sample set, wherein the historical recognition sample set includes: a plurality of historical recognition contents carrying text information;

[0040] The device also includes: a training module;

[0041] The training module is used to train the first sub-network in the preset network based on the pinyin sequence corresponding to each historical recognition content in the historical recognition sample set, so as to obtain the first recognition network included in the recognition model, and the first recognition network has the function of recognizing text information in the pinyin sequence; and to train the second sub-network in the preset network based on the Chinese character sequence corresponding to each historical recognition content in the historical recognition sample set, so as to obtain the second recognition network included in the recognition model, and the second recognition network has the function of recognizing text information in the Chinese character sequence.

[0042] In the above possible design of the second aspect, the processing module is further used to pre-process each historical recognition content in the historical recognition sample set to obtain a Chinese character sequence corresponding to each historical recognition content;

[0043] The acquisition module is further used to obtain the pinyin annotation result of the Chinese character sequence corresponding to each historical recognition content, the pinyin annotation result is manually determined based on the context information of each historical recognition content, based on the pinyin annotation result;

[0044] The processing module is further used to convert the Chinese character sequence corresponding to each historical recognition content in the historical recognition sample set into a pinyin sequence;

[0045] The training module is also used to train the third sub-network in the preset network based on the Chinese character sequence and pinyin sequence corresponding to each historical recognition content in the historical recognition sample set, so as to obtain the conversion network included in the recognition model, and the conversion network has the function of converting the Chinese character sequence into the pinyin sequence.

[0046] In yet another possible design of the second aspect, the apparatus further includes: a detection module and an update module;

[0047] The detection module is used to detect whether there is text information in the text word library in the Chinese character sequence and / or the pinyin sequence corresponding to the content to be recognized, and obtain a third recognition result, wherein the text word library includes the Chinese character sequence and the pinyin sequence corresponding to the text information determined in the historical recognition process;

[0048] The updating module is used to update the text information recognition result based on the third recognition result, and use the updated text information recognition result as the text information recognition result of the content to be recognized.

[0049] In any of the above possible designs of the second aspect, the content to be identified includes: user comment content on the e-commerce website, and the text information in the content to be identified includes: keywords in the user comment content.

[0050] In any of the above possible designs of the second aspect, the content to be identified includes: the content to be desensitized in the cloud storage system, and the text information in the content to be identified includes: the sensitive information in the content to be desensitized.

[0051] The third aspect of the present application provides a text information recognition device, including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in the first aspect and each possible design of the first aspect is implemented.

[0052] A fourth aspect of the present application provides a storage medium, wherein instructions are stored in the storage medium. When the storage medium is run on a computer, the computer executes the method described in the first aspect and each possible design of the first aspect.

[0053] The text information recognition method, device and storage medium provided in the embodiments of the present application obtain the content to be recognized including text information, determine the first recognition result for the text information in the content to be recognized based on the pinyin sequence corresponding to the content to be recognized and the recognition model, determine the second recognition result for the text information in the content to be recognized based on the Chinese character sequence corresponding to the content to be recognized and the recognition model, and obtain the text information recognition result of the content to be recognized based on the first vector corresponding to the first recognition result and the second vector corresponding to the second recognition result. The method can improve the recognition effect of the text information in the content to be recognized, improve the recognition accuracy of the text information, and enhance the user experience by processing the pinyin sequence and the Chinese character sequence. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 A flowchart of a first embodiment of a text information recognition method provided in an embodiment of the present application;

[0055] Figure 2 A flowchart of a second embodiment of a text information recognition method provided in an embodiment of the present application;

[0056] Figure 3 A flowchart of a third embodiment of a text information recognition method provided in an embodiment of the present application;

[0057] Figure 4 A flowchart of a fourth embodiment of a text information recognition method provided in an embodiment of the present application;

[0058] Figure 5 A flowchart of a fifth embodiment of a text information recognition method provided in an embodiment of the present application;

[0059] Figure 6 A schematic diagram of the structure of the preset network corresponding to the recognition model;

[0060] Figure 7 A flowchart of a sixth embodiment of a text information recognition method provided in an embodiment of the present application;

[0061] Figure 8 A schematic diagram of the structure of a first embodiment of a text information recognition device provided in an embodiment of the present application;

[0062] Fig. 9 A schematic diagram of the structure of a second embodiment of a text information recognition device provided in an embodiment of the present application;

[0063] Fig.10 A schematic diagram of the structure of a third embodiment of a text information recognition device provided in an embodiment of the present application;

[0064] Fig.11 This is a structural diagram of Embodiment 4 of the text information recognition device provided in the embodiments of the present application. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0066] Below, some terms in the embodiments of the present application are explained to facilitate understanding by those skilled in the art:

[0067] Machine learning: Machine learning refers to the process of using certain algorithms to guide computers to use known data to derive appropriate models and use this model to make judgments on new situations.

[0068] Content to be identified: Content to be identified may refer to a certain piece of content obtained from a variety of platforms such as big data platforms, e-commerce platforms, and business platforms. The content may include: text information, symbol information, graphic information, expression information, and other content in various forms. Exemplarily, the text information in the content to be identified may be, for example, content to be desensitized obtained from a big data platform, user comment content obtained from an e-commerce platform, business information obtained from a business platform, etc. The embodiments of this application do not limit the specific form of expression of the content to be identified, which can be determined according to actual conditions.

[0069] Correspondingly, the text information in the content to be identified may refer to some target text information, for example, sensitive information such as user identity information and age information contained in the content to be desensitized, keywords with emotional connotations such as insults and praise contained in user comments, or keywords with advertising nature such as platform names, etc. Among them, keywords such as insults, pornography, and advertising contained in user comments may also be called banned words. This embodiment does not limit the text information in the content to be identified.

[0070] LSTM: Long short-term memory (LSTM) network is a time recurrent neural network. LSTM is suitable for processing and predicting important events with very long intervals and delays in time series.

[0071] Bi-LSTM: Bidirectional long short-term memory network is composed of forward LSTM and backward LSTM. It is composed of two LSTMs. The forward LSTM obtains sequence information from the forward direction, and the backward LSTM obtains sequence information from the backward direction. In this way, the complete information of the entire sequence and the order of each position in the sequence can be integrated.

[0072] CRF: Conditional random field (CRF) is a discriminant model and a type of random field, commonly used to label or analyze sequence data, such as natural language text or biological sequences.

[0073] The text information identification method provided in the embodiment of the present application can be applied to a server, which can be used to identify text information in a platform to be processed. Exemplarily, the platform to be processed can be an e-commerce platform, a big data platform, or a business platform, etc. Therefore, the text information identification method can be used to identify keywords in user comments on the e-commerce platform, identify sensitive information in the content to be desensitized on the big data platform, and identify target business information on the business platform.

[0074] It is worth noting that, in actual applications, the text information recognition method can also be applied to text information recognition in other scenarios. This embodiment does not limit the specific application scenario technology, which can be determined according to actual conditions.

[0075] Exemplarily, the technical solution of the present application can be explained below by taking the text information recognition method of the present application as applicable to keyword recognition in user comment content on an e-commerce platform.

[0076] In the field of e-commerce technology, with the rapid development of network technology, it is becoming more and more common for users to purchase the items they need on e-commerce websites. When shopping online, users can only understand the basic information of the product based on the content recorded in the product details and cannot see the real product. Therefore, the quality evaluation of the product and the service quality evaluation of the seller after the relevant user purchases the product is very important. It can not only reflect the seller's reputation, but also may affect the consumer's order judgment. Moreover, it is also the most direct way to obtain consumer feedback and interaction. Therefore, user evaluation has a very high guiding value and can provide reference for other buyers to a large extent. Therefore, user review content is very important data in the field of e-commerce.

[0077] In real life, some buyers, based on interests or emotional factors, may include keywords with advertising nature or uncivilized language with extreme emotions in their comments when commenting on the quality of goods or sellers' services, thus affecting the shopping experience of other users. Therefore, how to identify keywords in user comments is an important part of the e-commerce field and a prerequisite for ensuring the quality of user comments.

[0078] Similarly, with the continuous maturity and development of big data technology and the expansion of its application scope, there are more and more development projects involving big data technology. The demand for desensitizing large-scale data and using the desensitized data to build a test environment for a big data project is becoming increasingly urgent. Therefore, how to identify sensitive information in the content to be desensitized on the big data platform or the target business information on the business platform is the key to achieving desensitization of sensitive information in the data to be desensitized and protecting the privacy of sensitive information.

[0079] The text information recognition method in the prior art can only recognize part of the target text information in the content to be recognized, and there are problems of low recognition accuracy and poor user experience. To solve this problem, the embodiment of the present application provides a text information recognition method, device and storage medium, which obtains the content to be recognized including text information, determines a first recognition result for the text information in the content to be recognized based on the pinyin sequence and recognition model corresponding to the content to be recognized, determines a second recognition result for the text information in the content to be recognized based on the Chinese character sequence corresponding to the content to be recognized and the recognition model, and obtains the text information recognition result of the content to be recognized based on the first vector corresponding to the first recognition result and the second vector corresponding to the second recognition result. The method can improve the recognition effect of the text information in the content to be recognized, improve the recognition accuracy of the text information, and enhance the user experience by processing the pinyin sequence and the Chinese character sequence.

[0080] The technical solution of the present application is described in detail below through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0081] Figure 1 The flowchart of the first embodiment of the text information recognition method provided in the embodiment of the present application is shown in FIG. The method can be applied to a server, which is used to process transactions on a processing platform. For example, the server can be used to process transactions such as product information maintenance, order acquisition, order processing, and product evaluation management on an e-commerce website. For example, in this embodiment, Figure 1 As shown, the text information recognition method may include the following steps:

[0082] Step 11: Obtain the content to be identified, which includes: text information.

[0083] Optionally, in this embodiment, when there is content to be identified on the platform to be processed, the server can obtain the content to be identified on the platform to be processed. It is worth noting that in order to reflect the high accuracy of the text information identification method of this embodiment, this embodiment is described as the content to be identified including text information.

[0084] For example, in the e-commerce field, the server can detect whether there are transactions for evaluating products on the e-commerce website. If so, it can detect whether there are new user comments. When a new user comment is detected on the e-commerce website, the server can obtain the user comment, laying the foundation for subsequent detection and identification of whether there are keywords in the user comment content.

[0085] Step 12: Based on the pinyin sequence corresponding to the content to be recognized and the recognition model, determine a first recognition result for the text information in the content to be recognized.

[0086] In this embodiment, in order to avoid the problem of inaccurate text information recognition caused by users converting the text information in the content to be recognized, the recognition model in this embodiment can be obtained by training the pinyin sequence corresponding to the text information included in each recognized content in the historical recognition content set during the training process. Therefore, the recognition model has the function of recognizing text information in the pinyin sequence. Therefore, the server can use the recognition model to recognize the text information of the pinyin sequence of the content to be recognized, and then obtain the first recognition result for the text information in the content to be recognized.

[0087] Exemplarily, a trained recognition model is running on the server, and the recognition model has at least the function of recognizing text information in a pinyin sequence. Therefore, when the content to be recognized includes text information, the recognition model can recognize the text information in the content to be recognized.

[0088] In actual applications, when the content to be recognized is obtained, a first recognition result can be obtained by inputting the pinyin sequence corresponding to the content to be recognized into the recognition model. The first recognition result is a recognition result for the text information in the pinyin sequence. That is, when the server obtains the pinyin sequence corresponding to the content to be recognized, the recognition model running on the server can be used to recognize the text information in the content to be recognized, thereby obtaining the first recognition result.

[0089] For example, the following example uses the content to be identified as user comments on an e-commerce website, and the text information as keywords in the user comments.

[0090] For example, suppose a user comment on an e-commerce network wants to express the meaning of "this scarf is the same style as Xiaohong", and "Xiaohong" is a keyword. In order to avoid the keyword "Xiaohong" being detected, the commenter may publish the user comment as "this scarf is the same style as Xiaohong". Since the recognition model in the embodiment of the present application is obtained based on the pinyin sequence "zhe ge wei jin shi xiao hong de tong kuan" corresponding to the user comment content, the recognition model can recognize "xiao hong", thereby improving the accuracy of keyword recognition.

[0091] Step 13: Based on the Chinese character sequence corresponding to the content to be recognized and the recognition model, determine a second recognition result for the text information in the content to be recognized.

[0092] Optionally, in this embodiment, in order to further improve the accuracy of text information recognition, the recognition model running on the server also has the function of recognizing text information in a Chinese character sequence. Therefore, the server can use the recognition model to recognize the text information of the Chinese character sequence of the content to be recognized, and then obtain a second recognition result for the text information in the content to be recognized.

[0093] In this embodiment, during the process of training the above recognition model, the server may also perform training based on the Chinese character sequences corresponding to the historical recognition contents in the historical recognition sample set, so that the recognition model also has the function of recognizing text information in the Chinese character sequence.

[0094] In this embodiment, by mapping the Chinese character sequence to pinyin and training the model based on the Chinese character sequence and the pinyin sequence, the training of the recognition model can be made more sufficient, and the generalization ability of the recognition model is greatly improved. It can not only recognize the text information expressed in Chinese characters in the content to be recognized, but also recognize the text information expressed in pinyin, which improves the interpretability and credibility of the recognition model, and ultimately can accurately locate the text information in the content to be recognized.

[0095] Step 14: Based on the first vector corresponding to the first recognition result and the second vector corresponding to the second recognition result, obtain the text information recognition result of the content to be recognized.

[0096] In this embodiment, in order to further improve the accuracy of text information recognition and prevent the inaccuracy problems existing in the recognition of a single pinyin sequence or a single Chinese character sequence, this embodiment can combine the above-mentioned first recognition result and the second recognition result, and use the processing result based on the first vector corresponding to the first recognition result and the second vector corresponding to the second recognition result as the recognition result of the text information, which realizes the complementarity of text information and improves the recognition accuracy of text information in the content to be recognized.

[0097] For the specific implementation of this step, please refer to the following Figure 3 The description in the illustrated embodiment will not be repeated here.

[0098] Exemplarily, in this embodiment, the recognition model may include two symmetrical sub-networks, for example, Bi-LSTM networks with the same structure. One Bi-LSTM network is used to recognize text information in the pinyin sequence corresponding to the content to be recognized, and the other Bi-LSTM network is used to recognize text information in the Chinese character sequence corresponding to the content to be recognized.

[0099] Specifically, the content to be recognized is input into the recognition model, and the recognition model can use a Bi-LSTM network to recognize the text information in the pinyin sequence corresponding to the content to be recognized, and use another Bi-LSTM network to recognize the text information in the Chinese character sequence corresponding to the content to be recognized.

[0100] Exemplarily, in this embodiment, the Bi-LSTM network used to identify text information in the pinyin sequence corresponding to the content to be identified is called the first recognition network, and the Bi-LSTM network used to identify text information in the Chinese character sequence corresponding to the content to be identified is called the second recognition network.

[0101] Since the Bi-LSTM network contains forward and backward LSTMs, it can obtain the contextual information of the complete sequence in both forward and reverse order. These two symmetrical Bi-LSTM networks with consistent network structures can effectively learn the complete contextual information of Chinese character sequences and pinyin sequences through the network, laying the foundation for the subsequent determination of the position and information of text information in the space vector.

[0102] In this embodiment, the server uses a recognition model that can recognize text information in pinyin sequences and Chinese character sequences. This can effectively solve the problem of inaccurate recognition caused by converting Chinese characters into pinyin in the text information in the content to be recognized, making the text information recognition results more interpretable and credible, thereby improving the recognition accuracy of the text information in the content to be recognized.

[0103] The text information recognition method provided in the embodiment of the present application obtains the content to be recognized including text information, determines the first recognition result for the text information in the content to be recognized based on the pinyin sequence and the recognition model corresponding to the content to be recognized, and determines the second recognition result for the text information in the content to be recognized based on the Chinese character sequence corresponding to the content to be recognized and the recognition model, and finally obtains the text information recognition result of the content to be recognized based on the first vector corresponding to the first recognition result and the second vector corresponding to the second recognition result. That is, the technical solution can improve the recognition effect of the text information in the content to be recognized, improve the recognition accuracy of the text information, and enhance the user experience through the processing of the pinyin sequence and the Chinese character sequence.

[0104] Exemplarily, based on the above embodiments, Figure 2 The flowchart of the second embodiment of the text information recognition method provided in the embodiment of the present application is as follows. Figure 2 As shown, in this embodiment, before the above step 12, the text information recognition method may further include the following steps:

[0105] Step 21: Preprocess the content to be recognized to obtain a Chinese character sequence corresponding to the content to be recognized.

[0106] Exemplarily, after the server obtains the content to be recognized, it may first pre-process the content to be recognized to obtain a Chinese character sequence that meets the recognition standard.

[0107] Exemplarily, preprocessing of the content to be identified may include the following: filtering out empty segments or content of text information by parsing the content to be identified, splitting each segment of text information into short texts with a length less than a preset length, and the like.

[0108] Step 22: Use the above recognition model to convert the Chinese character sequence into a pinyin sequence.

[0109] In this embodiment, the above-mentioned recognition model may also include a conversion network having the function of converting a Chinese character sequence into a pinyin sequence. During the training process of the recognition model, the conversion network is trained to have the ability to analyze the context information of the acquired content to be recognized and the ability to convert the Chinese character sequence into a pinyin sequence. Therefore, the server can use the recognition model to perform pinyin annotation on the obtained Chinese character sequence based on the context information of the content to be recognized, and convert it into a pinyin sequence.

[0110] Exemplarily, the server can translate the Chinese character sequence into a pinyin sequence through the conversion network in the recognition model to obtain an accurate and complete pinyin sequence. Exemplarily, the conversion network can be implemented by an LSTM network.

[0111] Optionally, in actual applications, after the server obtains the content to be recognized and preprocesses the content to be recognized, the preprocessed Chinese character sequence can be converted into a corresponding pinyin sequence through a recognition model, thereby improving the accuracy of the conversion.

[0112] The text information recognition method provided in the embodiment of the present application obtains the Chinese character sequence corresponding to the content to be recognized by preprocessing the content to be recognized, and converts the Chinese character sequence into a pinyin sequence using the above recognition model. The technical solution first preprocesses the content to be recognized and converts the Chinese character sequence into a pinyin sequence, which lays a foundation for the subsequent recognition of text information and indirectly improves the recognition accuracy of text information.

[0113] Optionally, based on the above embodiment, Figure 3 The flowchart of the third embodiment of the text information recognition method provided in the embodiment of the present application is as follows. Figure 3 As shown, in this embodiment, the above step 14 can be implemented by the following steps:

[0114] Step 31: Based on the principle of superimposing vectors at the same position, the first vector corresponding to the first recognition result and the second vector corresponding to the second recognition result are combined to obtain a composite vector.

[0115] Optionally, in this embodiment, it can be seen from the above analysis that by inputting the above content to be recognized into the above recognition model, a first recognition result corresponding to the pinyin sequence and a second recognition result corresponding to the Chinese character sequence will be obtained. In practical applications, the first recognition network of the recognition model outputs the above first recognition result in the form of a vector, and the second recognition network of the recognition model outputs the above second recognition result in the form of a vector.

[0116] Therefore, for the combination of the first recognition result and the second recognition result, based on the principle of superposition of vectors at the same position, the first vector corresponding to the first recognition result and the second vector corresponding to the second recognition result can be combined to obtain a synthetic vector, so as to ensure that the recognition model can retain the information of the Chinese character sequence and the pinyin sequence at the same position as much as possible, so as to enhance the probability of text information recognition.

[0117] Step 32: The synthesized vector is processed through a fully connected network and then input into a discriminant model to obtain a text information recognition result of the content to be recognized.

[0118] In practical applications, the above-obtained synthetic vector needs to be mapped into a fixed vector in order to obtain the text information recognition result of the content to be recognized. Optionally, the recognition model can input the above-obtained synthetic vector into a discriminant model, and after the labeling processing of the discriminant model, the position and amount of the text information in the content to be recognized can be located.

[0119] In this embodiment, the discriminant model can be a CRF model, that is, the CRF model is used to realize the final recognition of text information. This is because the two symmetrical sub-networks (Bi-LSTM network) can map the Chinese character sequence and the pinyin sequence in the content to be recognized to the specified sample tag space to obtain a synthetic vector, which contains the context information of the complete sequence, and the synthetic vector is input into the CRF to identify the position of the text information in the content to be recognized.

[0120] The text information recognition method provided in the embodiment of the present application is based on the principle of superimposing vectors at the same position, combining the first vector corresponding to the first recognition result and the second vector corresponding to the second recognition result to obtain a composite vector, and inputting the composite vector into the discriminant model after processing through a fully connected network to obtain the text information recognition result of the content to be recognized. In this technical solution, the composite vectors corresponding to the first recognition result and the second recognition result are respectively passed through the fully connected network and the discriminant model, so as to determine the position of the text information, and provide a basis for the subsequent processing of the text information.

[0121] Exemplarily, based on any of the above embodiments, Figure 4 This is a flow chart of the fourth embodiment of the text information recognition method provided in the present application. Figure 4 As shown, in this embodiment, the text information recognition method may further include the following steps:

[0122] Step 41: Obtain a historical recognition sample set, where the historical recognition sample set includes: multiple historical recognition contents carrying text information.

[0123] Optionally, if the server wants to train a recognition model with high recognition accuracy, it can obtain a historical recognition sample set online, and the number of historical recognition contents carrying text information in the historical recognition sample set should be sufficient, and the text information can cover multiple categories based on the actual distribution of the historical recognition content.

[0124] Step 42: Based on the pinyin sequence corresponding to each historical recognition content in the historical recognition sample set, the first sub-network in the preset network is trained to obtain the first recognition network included in the recognition model, and the first recognition network has the function of recognizing text information in the pinyin sequence.

[0125] Optionally, in this embodiment, it is assumed that the recognition model to be trained corresponds to the above-mentioned preset network, and the above-mentioned preset network includes at least three sub-networks. In practical applications, it is assumed that the server uses the first sub-network included in the preset network to train the first recognition network with the function of recognizing text information in the pinyin sequence, and uses the second sub-network included in the preset network to train the second recognition network with the function of recognizing text information in the Chinese character sequence. Optionally, the first sub-network and the second sub-network can be implemented by a symmetrically distributed Bi-LSTM network.

[0126] Specifically, in this embodiment, the pinyin sequence corresponding to each historical recognition content in the historical recognition sample set and the first information recognition result corresponding to each historical recognition content are respectively used as inputs of the first sub-network to train the first sub-network, thereby obtaining the first recognition network included in the recognition model, and the first recognition network has the function of recognizing text information in the pinyin sequence. Among them, the first information recognition result corresponding to each historical recognition content is a known text information recognition result for the pinyin sequence.

[0127] Step 43: Based on the Chinese character sequence corresponding to each historical recognition content in the historical recognition sample set, the second sub-network in the preset network is trained to obtain a second recognition network included in the recognition model, and the second recognition network has the function of recognizing text information in the Chinese character sequence.

[0128] Specifically, in this embodiment, the Chinese character sequence corresponding to each historical recognition content in the historical recognition sample set and the second information recognition result corresponding to each historical recognition content are respectively used as inputs of the second sub-network to train the second sub-network, thereby obtaining the second recognition network included in the recognition model, and the second recognition network has the function of recognizing text information in the Chinese character sequence. The second information recognition result corresponding to each historical recognition content is a known text information recognition result for the Chinese character sequence.

[0129] The text information recognition method according to the embodiment of the present application obtains a historical recognition sample set, which includes: multiple pieces of historical recognition content carrying text information. Based on the pinyin sequence corresponding to each piece of historical recognition content in the historical recognition sample set, the first sub-network in the preset network is trained to obtain the first recognition network included in the recognition model. The first recognition network has the function of recognizing text information in the pinyin sequence. Based on the Chinese character sequence corresponding to each piece of historical recognition content in the historical recognition sample set, the second sub-network in the preset network is trained to obtain the second recognition network included in the recognition model. The second recognition network has the function of recognizing text information in the Chinese character sequence. In this technical solution, the trained recognition model includes a first recognition network with the function of recognizing text information in the pinyin sequence and a second recognition network with the function of recognizing text information in the Chinese character sequence, which provides the implementation conditions for the accurate recognition of the content to be recognized.

[0130] Exemplarily, on the basis of the above embodiment, Figure 5 It is a schematic flowchart of the fifth embodiment of the text information recognition method provided by the embodiment of the present application. As Figure 5 shown, in this embodiment, before the above step 42, the text information recognition method may further include the following steps:

[0131] Step 51: Preprocess each piece of historical recognition content in the historical recognition sample set to obtain the Chinese character sequence corresponding to each piece of historical recognition content.

[0132] Optionally, in order to improve the training efficiency and training accuracy of the recognition model, the server may first perform a series of preprocessing on the obtained historical recognition sample set to obtain a relatively clean historical recognition sample set with rich sample categories and the Chinese character sequence corresponding to each piece of historical recognition content.

[0133] Step 52: Obtain the pinyin annotation result of the Chinese character sequence corresponding to each piece of historical recognition content, and the pinyin annotation result is determined manually based on the context information of each piece of historical recognition content.

[0134] Optionally, in the prior art, the mapping from Chinese characters to pinyin can be realized through some toolkits, but this method cannot solve the problem of the same Chinese character having different pinyins. For example: "growing up" and "length", the pinyin corresponding to "长" is inconsistent in different scenarios. For example, based on the context information, the pinyin sequence of "growing up" should be "zhang da", and the pinyin sequence of "length" should be "chang duan".

[0135] Therefore, in this embodiment, considering the context information of the Chinese character sequence corresponding to the historical recognition content, the pinyin annotation result of the Chinese character sequence corresponding to each piece of historical recognition content can be determined manually based on the context information of each piece of historical recognition content.

[0136] Step 53: Based on the pinyin annotation result, convert the Chinese character sequence corresponding to each historical recognition content in the historical recognition sample set into a pinyin sequence.

[0137] Optionally, in this embodiment, the server can directly convert the Chinese character sequence corresponding to each historical recognition content into a pinyin sequence based on the obtained pinyin annotation results, thereby obtaining the Chinese character sequence and pinyin sequence corresponding to each historical recognition content in the historical recognition sample set.

[0138] Step 54: Based on the Chinese character sequence and pinyin sequence corresponding to each historical recognition content in the historical recognition sample set, the third sub-network in the preset network is trained to obtain the conversion network included in the above recognition model, which has the function of converting the Chinese character sequence into the pinyin sequence.

[0139] In this embodiment, the server can use the third sub-network included in the preset network to train a conversion network having the function of converting a Chinese character sequence into a pinyin sequence. Optionally, the third sub-network can be implemented by an LSTM network.

[0140] Exemplarily, in this embodiment, the third sub-network can be trained so that the third sub-network can accurately convert the Chinese character sequence into a pinyin sequence according to the context information of the content to be recognized.

[0141] Specifically, in this embodiment, the Chinese character sequence and pinyin sequence corresponding to each historical recognition content in the historical recognition sample set are used as the input and output of the third sub-network respectively to train the third sub-network, and the conversion network included in the recognition model is obtained. The conversion network has the function of converting the Chinese character sequence into a pinyin sequence.

[0142] The text information recognition method provided by the embodiment of the present application pre-processes each historical recognition content in the historical recognition sample set to obtain the Chinese character sequence corresponding to each historical recognition content, obtains the pinyin annotation result of the Chinese character sequence corresponding to each historical recognition content, and the pinyin annotation result is manually determined based on the context information of each historical recognition content. Based on the pinyin annotation result, the Chinese character sequence corresponding to each historical recognition content in the historical recognition sample set is converted into a pinyin sequence, and finally, based on the Chinese character sequence and pinyin sequence corresponding to each historical recognition content in the historical recognition sample set, the third sub-network in the preset network is trained to obtain the conversion network included in the recognition model, and the conversion network has the function of converting the Chinese character sequence into the pinyin sequence. In this technical solution, the conversion network with the function of converting the Chinese character sequence into the pinyin sequence is obtained by manual annotation, which lays the foundation for the accurate recognition of subsequent text information.

[0143] For example, Figure 6This is a schematic diagram of the structure of the preset network corresponding to the recognition model. Figure 6 As shown, the recognition model may include: a first recognition network and a second recognition network with the same structure and symmetrical distribution, a conversion network, a connection network and a discriminant model. Optionally, the first recognition network and the second recognition network are implemented by a Bi-LSTM network, the conversion network can be implemented by an LSTM network, the connection network may include: a connection vector and a fully connected layer, and the discriminant model may be a CRF model.

[0144] For example, refer to Figure 6 As shown, in this embodiment, the content to be identified is the user comment content, and the text information to be identified is the keywords in the user comment content. The specific identification scheme is as follows: the user comment content is processed to obtain a Chinese character sequence. On the one hand, the Chinese character sequence is input into a second recognition network with a function of identifying text information corresponding to the Chinese character sequence to obtain a second vector corresponding to the second recognition result. On the other hand, the Chinese character sequence is input into a conversion network to obtain a pinyin sequence. The pinyin sequence is input into a first recognition network with a function of identifying text information corresponding to the pinyin sequence to obtain a first vector corresponding to the first recognition result. The first vector and the second vector are processed by the connection network and the discriminant model to obtain a keyword recognition result.

[0145] The following description is made by taking the example that the historical recognition sample set is the historical user comment content on the e-commerce website, and the text information to be recognized is the keywords in the historical user comment content.

[0146] Specifically, based on Figure 6 The training process of the preset network for the recognition model is summarized as follows:

[0147] (1) Obtain a set of historical user review samples online and perform a series of preprocessing on them to obtain a relatively clean user review sequence dataset.

[0148] (2) Obtain the Chinese character sequence and the pinyin sequence corresponding to each historical comment sample in the above historical user comment sample set, wherein the pinyin sequence is obtained by converting the Chinese character sequence based on the manually annotated pinyin annotation result.

[0149] (3) Using the Chinese character sequence and pinyin sequence corresponding to each historical comment sample in the above historical user comment sample set as a training data set to train the conversion network, the first recognition network and the second recognition network in the recognition model, the first recognition network and the second recognition network are networks with the same structure and symmetrical distribution.

[0150] Specifically, in this embodiment, the conversion network can be an LSTM network. The conversion network can be trained by taking the Chinese character sequence corresponding to the historical comment samples as the input of the conversion network and the pinyin sequence corresponding to the historical comment samples as the output of the conversion network.

[0151] The first recognition network and the second recognition network are Bi-LSTM networks with the same structure. Bi-LSTM contains forward and backward LSTM, and can obtain the contextual information of the positive and reverse order of the complete sequence based on the historical comment samples. The scheme uses the pinyin sequence corresponding to the historical comment content and the first text recognition result corresponding to the historical comment content to train the first recognition network, and uses the Chinese character sequence corresponding to the historical comment content and the second text recognition result corresponding to the historical comment content to train the second recognition network, that is, the Chinese character sequence and the pinyin sequence express different information in the entire text in the vector space. By training two symmetrical first recognition networks and second recognition networks, it is necessary to mine the information of the text sequence from two angles, so as to more accurately determine the position of keywords in historical user comments. Therefore, in this embodiment, the first recognition network, the second recognition network and the conversion network are obtained through network learning training, which can provide sufficient information and knowledge for the discriminant model to accurately identify the keywords in the user comment content.

[0152] (4) The first text vector corresponding to the first text recognition result and the second text vector corresponding to the second text recognition result are output to the connection network for combination, and the resultant vector corresponding to the combination result is mapped to a fixed vector.

[0153] Optionally, the concatenation vector (Concat vector) can receive two Bi-LSTM networks to obtain two output vectors, and connect them at the same position of the vectors to form a new vector, namely a synthetic vector, and input the synthetic vector into the fully connected layer to obtain a fixed vector, which ensures that the network retains the information of the Chinese sequence and the pinyin sequence at the same position as much as possible.

[0154] (5) Using the discriminant model to identify the fixed vector to obtain the final keyword recognition result. In this embodiment, the CRF model can realize the recognition of keywords, that is, recognize the position of the keywords in the text corresponding to the user comment content.

[0155] Based on the above embodiments, the text information recognition method realizes the recognition of keywords in user comment content. Exemplarily, the keywords may include: advertising, fraud, insults and other banned words. Optionally, in practical applications, the identifiers of these banned words in user comment samples may be B-adv, I-adv, B-lie, I-lie, B-abuse, I-abuse, where B represents the starting byte and I represents the middle byte, that is, the bytes other than the starting byte.

[0156] The following example uses keywords in user comments as banned words to illustrate the rationality of this solution:

[0157] Assume that a new user comment on an e-commerce website contains banned words for advertising, and the advertisement is to promote the Wangsan platform, that is, the banned word is "Wangsan". For example, the user comment is: "Welfare is here, Wangsan low price free order", from which it can be seen that the user comment contains banned advertising, and the banned words are "Wangsan" and (equivalent to Wangsan with the same pronunciation but different characters).

[0158] In practical applications, after obtaining the Chinese character sequence corresponding to the user's comment content, you can use Figure 6 The conversion network (i.e., LSTM network) shown in the figure obtains the pinyin sequence corresponding to the Chinese character sequence: "fu li lai la wang san di jiamian dan". In this embodiment, the Chinese character sequence and the pinyin sequence are respectively input into the symmetrical first recognition network and the second recognition network (Bi-LSTM network) to obtain two output vectors, namely, output vector A and output vector B. The output vector A and the output vector B are connected to obtain a composite vector C, and finally the composite vector C is input into the CRF to realize the prohibited identification of network three.

[0159] Exemplarily, based on any of the above embodiments, Figure 7 This is a flow chart of the sixth embodiment of the text information recognition method provided in the present application. Figure 7 As shown, the text information recognition method may also include the following steps:

[0160] Step 71: Detect whether there is text information in the text word library in the Chinese character sequence and / or pinyin sequence corresponding to the content to be recognized, and obtain a third recognition result.

[0161] The text word library includes Chinese character sequences and pinyin sequences corresponding to the text information determined in the historical recognition process.

[0162] Optionally, for a specific application field, you can collect text information determined in the historical recognition process, and use the Chinese character sequence and pinyin sequence corresponding to the text information to establish a text vocabulary. If new text information of the target type appears during subsequent use, you can also add the pinyin sequence and Chinese character sequence corresponding to the new text information to the text vocabulary to achieve an update of the text vocabulary.

[0163] For example, in the e-commerce field, by collecting multiple banned words identified in the historical banned word identification process, a banned word library can be established using the Chinese character sequences and pinyin sequences corresponding to all the banned words. If new banned words appear during subsequent use, the new banned words can be added to the banned word library to achieve updates to the banned word library.

[0164] Therefore, in this embodiment, in addition to identifying the text information in the content to be identified based on the recognition model, the server can also detect whether there is text information existing in the text vocabulary in the Chinese character sequence and pinyin sequence corresponding to the content to be identified, and obtain a third recognition result based on the detection result.

[0165] Step 72: Based on the third recognition result, the above text information recognition result is updated, and the updated text information recognition result is used as the text information recognition result of the content to be recognized.

[0166] In this embodiment, the server can determine whether the third recognition result is consistent with the text information recognition result obtained through the recognition model. If inconsistent, the third recognition result is merged into the recognition result of the above text information, thereby updating the text information recognition result, and using the updated text information recognition result as the text information recognition result corresponding to the content to be recognized.

[0167] The text information recognition result provided in the embodiment of the present application identifies whether there is text information in the content to be recognized by matching the content to be recognized with the text information in the text vocabulary. This can make up for the prediction result of the recognition model to a certain extent. The method is simple and easy to implement, which further improves the accuracy of text information recognition.

[0168] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0169] Figure 8 This is a schematic diagram of the structure of the first embodiment of the text information recognition device provided in the embodiment of the present application. The device can be integrated in the server or implemented by the server. Figure 8 As shown, the device may include: an acquisition module 81, an identification module 82 and a determination module 83.

[0170] The acquisition module 81 is used to acquire the content to be identified, and the content to be identified includes: text information;

[0171] The recognition module 82 is used to determine a first recognition result for the text information in the content to be recognized based on the pinyin sequence corresponding to the content to be recognized and the recognition model, and to determine a second recognition result for the text information in the content to be recognized based on the Chinese character sequence corresponding to the content to be recognized and the recognition model;

[0172] The determination module 83 is used to obtain a text information recognition result of the content to be recognized based on the first vector corresponding to the first recognition result and the second vector corresponding to the second recognition result.

[0173] Exemplarily, in a possible design of the present embodiment, the determination module 83 is specifically used to combine the first vector corresponding to the first recognition result and the second vector corresponding to the second recognition result based on the principle of superposition of vectors at the same position to obtain a composite vector, and input the composite vector into the discriminant model after processing through a fully connected network to obtain the text information recognition result of the content to be recognized.

[0174] Exemplarily, based on the above embodiments, Fig. 9 This is a schematic diagram of the structure of the second embodiment of the text information recognition device provided in the embodiment of the present application. Fig. 9 As shown, the device also includes: a processing module 91.

[0175] The processing module 91 is used to preprocess the content to be recognized before the recognition module 82 determines the first recognition result for the text information in the content to be recognized based on the pinyin sequence corresponding to the content to be recognized and the recognition model, so as to obtain the Chinese character sequence corresponding to the content to be recognized, and use the recognition model to convert the Chinese character sequence into the pinyin sequence.

[0176] Exemplarily, in this embodiment, the acquisition module 81 is further used to acquire a historical recognition sample set, wherein the historical recognition sample set includes: a plurality of historical recognition contents carrying text information;

[0177] In this embodiment, the device further includes: a training module 92 .

[0178] The training module 92 is used to train the first sub-network in the preset network based on the pinyin sequence corresponding to each historical recognition content in the historical recognition sample set to obtain the first recognition network included in the recognition model, and the first recognition network has the function of recognizing text information in the pinyin sequence; and to train the second sub-network in the preset network based on the Chinese character sequence corresponding to each historical recognition content in the historical recognition sample set to obtain the second recognition network included in the recognition model, and the second recognition network has the function of recognizing text information in the Chinese character sequence.

[0179] Optionally, in this embodiment, the processing module 91 is further used to pre-process each piece of historical recognition content in the historical recognition sample set to obtain a Chinese character sequence corresponding to each piece of historical recognition content;

[0180] The acquisition module 81 is also used to obtain the pinyin annotation result of the Chinese character sequence corresponding to each historical recognition content, and the pinyin annotation result is manually determined based on the context information of each historical recognition content, based on the pinyin annotation result;

[0181] The processing module 91 is also used to convert the Chinese character sequence corresponding to each historical recognition content in the historical recognition sample set into a pinyin sequence;

[0182] The training module 92 is also used to train the third sub-network in the preset network based on the Chinese character sequence and pinyin sequence corresponding to each historical recognition content in the historical recognition sample set, so as to obtain the conversion network included in the recognition model, and the conversion network has the function of converting the Chinese character sequence into the pinyin sequence.

[0183] Exemplarily, based on any of the above embodiments, Fig.10 This is a schematic diagram of the structure of the third embodiment of the text information recognition device provided in the embodiment of the present application. Fig.10 As shown, the device may further include: a detection module 101 and an update module 102.

[0184] The detection module 101 is used to detect whether there is text information in the text word library in the Chinese character sequence and / or the pinyin sequence corresponding to the content to be recognized, and obtain a third recognition result, wherein the text word library includes the Chinese character sequence and the pinyin sequence corresponding to the text information determined in the historical recognition process;

[0185] The updating module 102 is used to update the text information recognition result based on the third recognition result, and use the updated text information recognition result as the text information recognition result of the content to be recognized.

[0186] In a possible design of this embodiment, the above-mentioned content to be identified includes: user comment content on the e-commerce website, and the text information in the content to be identified includes: keywords in the user comment content.

[0187] In a possible design of this embodiment, the above-mentioned content to be identified includes: content to be desensitized in the cloud storage system, and the text information in the content to be identified includes: sensitive information in the content to be desensitized.

[0188] The device provided in the embodiment of the present application can be used to perform Figures 1 to 7 The implementation principle and technical effect of the method in the illustrated embodiment are similar and will not be described in detail here.

[0189] It should be noted that it should be understood that the division of the various modules of the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software called by processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. For example, the determination module can be a separately established processing element, or it can be integrated in a chip of the above device. In addition, it can also be stored in the memory of the above device in the form of program code, and called and executed by a processing element of the above device. The implementation of other modules is similar. In addition, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each module above can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.

[0190] For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASIC), or one or more microprocessors (digital signal processors, DSP), or one or more field programmable gate arrays (FPGA), etc. For another example, when a module above is implemented in the form of a processing element scheduling program code, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0191] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0192] Fig.11 This is a schematic diagram of the structure of the fourth embodiment of the text information recognition device provided in the embodiment of the present application. Fig.11 As shown, the device may include: a processor 111, a memory 112, a communication interface 113 and a system bus 114, wherein the memory 112 and the communication interface 113 are connected to the processor 111 through the system bus 114 and communicate with each other, the memory 112 is used to store computer-executable instructions, the communication interface 113 is used to communicate with other devices, and the processor 111 implements the above-mentioned Figures 1 to 7 A scheme of the illustrated embodiment.

[0193] Should Fig.11The system bus mentioned in the specification may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The communication interface is used to realize the communication between the database access device and other devices (such as a client, a read-write library, and a read-only library). The memory may include a random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage.

[0194] The above-mentioned processor can be a general-purpose processor, including a central processing unit CPU, a network processor (NP), etc.; it can also be a digital signal processor DSP, an application-specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0195] Optionally, the present application also provides a storage medium, wherein the storage medium stores instructions, which, when executed on a computer, cause the computer to execute the above-mentioned Figures 1 to 7 The method of the illustrated embodiment.

[0196] Optionally, the present application embodiment further provides a chip for executing instructions, the chip being used to execute the above Figures 1 to 7 The method of the illustrated embodiment.

[0197] In the present application, "at least one" means one or more, and "plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship; in the formula, the character " / " indicates that the previous and next associated objects are in a "division" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0198] It should be understood that the various numerical numbers involved in the embodiments of the present application are only used for the convenience of description and are not used to limit the scope of the embodiments of the present application.

[0199] It can be understood that in the embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0200] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A text information recognition method, characterized in that: include: Acquire content to be identified, wherein the content to be identified includes: text information; Determine a first recognition result for text information in the content to be recognized based on the pinyin sequence corresponding to the content to be recognized and the recognition model; Determining a second recognition result for text information in the content to be recognized based on the Chinese character sequence corresponding to the content to be recognized and the recognition model; Based on the principle of superposition of vectors at the same position, a first vector corresponding to the first recognition result and a second vector corresponding to the second recognition result are combined to obtain a composite vector; The synthesized vector is processed through a fully connected network and then input into a discriminant model to obtain a text information recognition result of the content to be recognized.

2. The method according to claim 1, characterized in that Before determining a first recognition result for text information in the content to be recognized based on the pinyin sequence corresponding to the content to be recognized and the recognition model, the method further includes: Preprocessing the content to be recognized to obtain a Chinese character sequence corresponding to the content to be recognized; The recognition model is used to convert the Chinese character sequence into the pinyin sequence.

3. The method according to claim 1 or 2, characterized in that: Before determining a first recognition result for text information in the content to be recognized based on the pinyin sequence corresponding to the content to be recognized and the recognition model, the method further includes: Acquire a historical recognition sample set, wherein the historical recognition sample set includes: a plurality of historical recognition contents carrying text information; Based on the pinyin sequence corresponding to each historical recognition content in the historical recognition sample set, the first sub-network in the preset network is trained to obtain the first recognition network included in the recognition model, wherein the first recognition network has the function of recognizing text information in the pinyin sequence; The second sub-network in the preset network is trained based on the Chinese character sequence corresponding to each historical recognition content in the historical recognition sample set to obtain the second recognition network included in the recognition model, and the second recognition network has the function of recognizing text information in the Chinese character sequence.

4. The method according to claim 3, characterized in that Before training the first sub-network in the preset network based on the pinyin sequence corresponding to each historical recognition content in the historical recognition sample set to obtain the first recognition network included in the recognition model, the method further includes: Preprocessing each piece of historical recognition content in the historical recognition sample set to obtain a Chinese character sequence corresponding to each piece of historical recognition content; Obtaining a pinyin annotation result of a Chinese character sequence corresponding to each piece of historical recognition content, wherein the pinyin annotation result is manually determined based on context information of each piece of historical recognition content; Based on the pinyin annotation result, converting the Chinese character sequence corresponding to each historical recognition content in the historical recognition sample set into a pinyin sequence; The third sub-network in the preset network is trained based on the Chinese character sequence and the pinyin sequence corresponding to each historical recognition content in the historical recognition sample set to obtain the conversion network included in the recognition model, and the conversion network has the function of converting the Chinese character sequence into the pinyin sequence.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: Detecting whether there is text information in a text word library in the Chinese character sequence and / or the pinyin sequence corresponding to the content to be recognized, and obtaining a third recognition result, wherein the text word library includes the Chinese character sequence and the pinyin sequence corresponding to the text information determined in the historical recognition process; Based on the third recognition result, the text information recognition result is updated, and the updated text information recognition result is used as the text information recognition result of the content to be recognized.

6. The method according to any one of claims 1 to 5, characterized in that: The content to be identified includes: user comment content on the e-commerce website, and the text information in the content to be identified includes: keywords in the user comment content.

7. The method according to any one of claims 1 to 5, characterized in that: The content to be identified includes: content to be desensitized in the cloud storage system, and the text information in the content to be identified includes: sensitive information in the content to be desensitized.

8. A text information recognition device, characterized in that: include: Acquisition module, identification module and determination module; The acquisition module is used to acquire the content to be identified, and the content to be identified includes: text information; The recognition module is used to determine a first recognition result for the text information in the content to be recognized based on the pinyin sequence corresponding to the content to be recognized and the recognition model, and to determine a second recognition result for the text information in the content to be recognized based on the Chinese character sequence corresponding to the content to be recognized and the recognition model; The determination module is used to obtain a text information recognition result of the content to be recognized based on a first vector corresponding to the first recognition result and a second vector corresponding to the second recognition result; The determination module is specifically used to combine the first vector corresponding to the first recognition result and the second vector corresponding to the second recognition result based on the principle of superposition of vectors at the same position to obtain a synthetic vector, and input the synthetic vector into the discriminant model after processing it through a fully connected network to obtain the text information recognition result of the content to be recognized.

9. A text information recognition device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

10. A storage medium, characterized in that: The storage medium stores instructions, which, when executed on a computer, enable the computer to execute the method according to any one of claims 1 to 7.

Images