Method, device, equipment and readable storage medium for identifying wrong characters

Abstract

The present application provides a method, device, equipment and readable storage medium for identifying typos, which relate to the field of artificial intelligence technology, in order to improve the accuracy of identifying typos in media content. The method includes: obtaining target comment data for published media content, extracting target text features corresponding to the target comment data based on the context information of each word contained in the target comment data, determining the target comment content for typos contained in the target comment data based on the target text features, and determining the typo information in the media content based on the target comment content. The method can identify typos or special words that have not appeared in historical media content, thereby improving the accuracy of identifying typos in media content, and in the process of identifying typos, it is not necessary to detect the entire media content, thereby improving the efficiency of identifying typos in media content.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device, equipment and readable storage medium for identifying misspelled characters. Background Art

[0002] In the related art, a typo candidate set is generally created in advance, and by replacing the candidate words that appear in the typo candidate set in the media content, it is determined whether the candidate words are typos based on the degree of influence of the candidate words on the text fluency of the media content. However, this method can only detect typos that already exist in the typo candidate set, and cannot discover typos or special words that have not appeared in the typo candidate set. Therefore, the scope of typo recognition is limited. How to reduce the limitations of typo recognition to improve the accuracy of typo recognition is an issue that needs to be considered. Summary of the invention

[0003] Embodiments of the present application provide a method, apparatus, device and readable storage medium for identifying wrong characters, which are used to improve the accuracy of identifying wrong characters in media content.

[0004] In a first aspect of the present application, a method for identifying typos is provided, comprising:

[0005] Obtaining target comment data for published media content;

[0006] Extracting target text features corresponding to the target comment data according to context information of each word contained in the target comment data;

[0007] Based on the target text feature, determining that the target comment data includes target comment content for typos;

[0008] Based on the target comment content, typo information in the media content is determined.

[0009] In a second aspect of the present application, a device for identifying wrong characters is provided, comprising:

[0010] A data acquisition unit, used to acquire target comment data for published media content;

[0011] A feature extraction unit, configured to extract target text features corresponding to the target comment data according to context information of each word contained in the target comment data;

[0012] A first recognition unit is used to determine, based on the target text feature, that the target comment data contains target comment content for typos;

[0013] The second recognition unit is used to determine the typo information in the media content based on the target comment content.

[0014] In a possible implementation manner, the feature extraction unit is specifically used to:

[0015] Inputting the target review data into a trained review data classification model;

[0016] Based on the language learning sub-model in the comment data classification model, feature extraction is performed on the context information of each word contained in the target comment data to obtain the target text feature corresponding to the target comment data;

[0017] The language learning sub-model is obtained by using historical comment data as training samples and performing feature learning training on the language learning sub-model based on context information of each word contained in the training samples.

[0018] In a possible implementation, the comment data classification model further includes a prediction sub-model, and the first recognition unit is specifically used to:

[0019] Inputting the target text features into the prediction sub-model;

[0020] Based on the first degree of association learned by the prediction sub-model, predict a second degree of association between the target text feature and the target data recognition result, wherein the first degree of association is determined based on the degree of association between the historical text feature corresponding to the historical comment data and the target data recognition result, and the target data recognition result is used to characterize the comment content containing typos in the text data;

[0021] If the second relevance is greater than the relevance threshold, it is determined that the target comment data contains the target comment content.

[0022] In a possible implementation, the feature extraction unit is further used to train the language learning model in the following manner:

[0023] Based on the historical comment data set, a training operation is performed on the language learning sub-model, wherein one training operation comprises: performing a text prediction operation on each historical comment data obtained from the historical comment data set, and determining a prediction deviation corresponding to each historical comment data; and adjusting parameters of the language learning sub-model based on the prediction deviation corresponding to each historical comment data;

[0024] Wherein, the text prediction operation includes:

[0025] According to the word segmentation rule associated with the language form of one historical comment data among the various historical comment data, the one historical comment data is subjected to word segmentation processing to obtain at least one word included in the one historical comment data;

[0026] Based on a preset word mask, masking some words in the at least one word; and

[0027] Determining context information of the partial words in the one historical review data, and selecting candidate words whose matching degree with the determined context information satisfies a matching degree condition from a pre-configured candidate word library, wherein the candidate word library is determined based on the historical review data set;

[0028] The deviation information between the partial words and the selected candidate words is determined as the prediction deviation corresponding to the one historical comment data.

[0029] In a possible implementation manner, the second identification unit is specifically configured to:

[0030] Parsing the target comment content based on a preconfigured regular expression for identifying the typo information to obtain a corresponding parsing result;

[0031] Based on the parsing result, determining at least one wrong character associated with the target comment content, and text position information of the at least one wrong character in the media content;

[0032] The at least one incorrect word and the text position information are determined as incorrect word information in the media content.

[0033] In a possible implementation manner, the second identification unit is further configured to:

[0034] If the typo information is not obtained based on the parsing result, performing typo detection on the media content based on a preconfigured typo detection rule to obtain a detection result;

[0035] According to the detection result, it is determined whether the media content contains corresponding wrong character information.

[0036] In a possible implementation manner, the second identification unit is further configured to:

[0037] Obtaining account information of a target account that published the target comment data;

[0038] Determining the confidence level of the target review data based on the account information;

[0039] When it is determined that the confidence reaches a confidence threshold, typo information in the media content is determined based on the target comment content.

[0040] In a third aspect of the present application, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the program.

[0041] In a fourth aspect of the present application, a computer program product is provided, the computer program product comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided in the first aspect above.

[0042] In a fifth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions. When the computer instructions are executed on a computer, the computer executes the method described in the first aspect.

[0043] Since the embodiment of the present application adopts the above technical solution, it has at least the following technical effects:

[0044] On the one hand, in the embodiments of the present application, typos in media content are identified directly based on information in comment data of the media content. In this process, there is no need to collect typos in advance and create a candidate set of typos. Typos or special words that have not appeared in historical media content can be identified, thereby improving the accuracy of identifying typos in media content. On the other hand, in the process of identifying typos in the embodiments of the present application, only the comment data needs to be identified, and there is no need to detect the entire media content, which significantly reduces the detection range of typos identification and improves the efficiency of identifying typos in media content. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A schematic diagram of an application scenario of typo recognition provided in an embodiment of the present application;

[0046] Figure 2 A schematic diagram of a process of a method for identifying typos provided in an embodiment of the present application;

[0047] Figure 3 A schematic diagram of a process for obtaining comment data for media content provided in an embodiment of the present application;

[0048] Figure 4 A schematic diagram of the structure of a review data classification model provided in an embodiment of the present application;

[0049] Figure 5 An example diagram of the structure of a language learning sub-model provided in an embodiment of the present application;

[0050] Figure 6A schematic diagram of a process of a training operation for a language learning sub-model provided in an embodiment of the present application;

[0051] Figure 7 A complete flow chart of a method for identifying typos provided in an embodiment of the present application;

[0052] Figure 8 A complete flowchart of a method for identifying typos provided in an embodiment of the present application;

[0053] Fig. 9 A schematic diagram of the structure of a device for identifying wrong characters provided in an embodiment of the present application;

[0054] Fig.10 A structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0055] In order to better understand the technical solution provided by the embodiments of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0056] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0057] In order to help those skilled in the art better understand the technical solution of the present application, some concepts involved in the present application are explained below.

[0058] 1) Media Content

[0059] In the era of self-media, content can generally refer to audio, video, graphics, and text, etc.; in the embodiments of the present application, media content can be but is not limited to content pushed to an account by a content platform (such as but is not limited to a content sharing system or a content recommendation system, etc.), and media content can be but is not limited to multimedia resources including at least one information or any combination of text, audio, video, articles, graphics, pictures, etc.; among them, in the embodiments of the present application, typos can be identified for text information in the media content. For example, when the media content is an article or graphics, the above-mentioned text information can be the text in the article or graphics; when the media content is a picture, the above-mentioned text information can be the description text in the picture; when the media content is a video, the above-mentioned text information can be but is not limited to the description text in the video (such as the narration information or character introduction information of a TV series, etc.) or subtitles in the video, etc.

[0060] 2) Review data and target review data

[0061] The comment data in the embodiments of the present application may include, but is not limited to, an account that receives media content and comment information on the media content; the target comment data is comment data on the media content that requires typos recognition, and can also be understood as currently processed comment data, etc.

[0062] 3)Bert (Bidirectional Encoder Representations from Transformer) model

[0063] The Bert model is an encoder network (Encoder) of a bidirectional Transformer. The goal of the Bert model is to use large-scale unlabeled corpus training to obtain a semantic representation (Representation) of the text that contains rich semantic information, and then fine-tune the semantic representation of the text in a specific natural language processing (NLP) task, and finally apply it to the specific NLP task.

[0064] 4) Natural Language Processing

[0065] Natural language processing is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that can achieve effective communication between people and computers using natural language. Natural language processing is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field will involve natural language, that is, the language people use in daily life, so it is closely related to the study of linguistics. Natural language processing technology usually includes text processing, semantic understanding, machine translation, robot question answering, knowledge graph and other technologies.

[0066] The embodiments of the present application relate to artificial intelligence (AI) and machine learning technology, and are designed based on computer vision technology and machine learning (ML) in artificial intelligence.

[0067] Artificial intelligence is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence.

[0068] Artificial intelligence is the study of the design principles and implementation methods of various intelligent machines, so that the machines have the functions of perception, reasoning and decision-making. Artificial intelligence technology mainly includes computer vision technology, natural language processing technology, and machine learning or deep learning. With the research and progress of artificial intelligence technology, artificial intelligence has been studied and applied in many fields, such as common smart homes, smart customer service, virtual assistants, smart speakers, smart marketing, unmanned driving, automatic driving, robots, smart medical care, etc. It is believed that with the development of technology, artificial intelligence will be applied in more fields and play an increasingly important role.

[0069] Machine learning is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning and other technologies.

[0070] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0071] The design concept of this application is described below.

[0072] In the related art, in the process of identifying typos in media content, known typos samples are generally collected to establish a candidate set containing typos with a high frequency of occurrence, and candidate words appearing in the candidate set in the media content are replaced in some way. Then, based on the degree of influence of the replaced candidate words on the text fluency of the media content, it is determined whether the replaced candidate words are typos. This method can only detect typos already in the typos candidate set, and cannot find typos that have not appeared in the typos candidate set, that is, it cannot find typos that have not appeared in the sample, and it cannot find special new words (such as but not limited to names, new Internet terms, etc.) and other special words with strong prior knowledge, and it cannot find whether such special words are wrong. For example, in "A1's wife is A2", A2 is a normal name, but A2 is not A1's wife. In the current typos recognition method, A2 can only be identified as a correct name, but cannot be identified whether A2 is A1's wife.

[0073] In view of this, the inventor has designed a method, apparatus, device and readable storage medium for identifying typos in media content, which are used to improve the efficiency of identifying typos in media content; the method takes into account that comment data on media content may contain information related to typos, and therefore the implementation of the present application is designed to identify whether the media content contains typos through information in comment data on media content; specifically, target comment data on published media content is obtained, and target text features corresponding to the target comment data are extracted based on context information of each word contained in the target comment data, and then based on the target text features, it is determined whether the target comment data contains target comment content for typos, and when it is determined that the target comment data contains target comment content for typos, the typo information in the media content can be determined based on the above target comment content.

[0074] As an embodiment, the language form of the text information in the media content is not limited in the embodiment of the present application. Technical personnel in the field can set it according to actual needs. The language form of the above text information can be but not limited to at least one language including Chinese, English, Korean, Japanese, Italian, Hindi, etc.; the language form of the comment data on the media content is not excessively limited. Technical personnel in the field can set it according to actual needs. The language form of the above comment data can be but not limited to at least one language including Chinese, English, Korean, Japanese, Italian, Hindi, etc.; the following content takes Chinese as an example to exemplify the typo recognition method provided in the embodiment of the present application.

[0075] Furthermore, in order to further improve the accuracy of identifying typos in media content, the language format of the comment data on the media content in the embodiment of the present application can be consistent with the language of the text information in the media content.

[0076] It should be noted that the specific implementation methods of this application involve operations related to object data such as obtaining demographic attribute information, social attribute information, etc. of an account. Therefore, when the above embodiments of this application are applied to specific products or technologies, it is necessary to obtain the object's permission or consent, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0077] In order to more clearly understand the design ideas of the present application, the application scenarios in the embodiments of the present application are introduced as follows.

[0078] Please refer to Figure 1 , represents an application scenario of wrong character recognition, which may include a terminal device 110, a content server 120 and a wrong character recognition server 130; the terminal device 110, the content server 120 and the wrong character recognition server 130 may communicate with each other through a network, wherein:

[0079] The terminal device 110 is used to receive media content and send the media content to the content server 120; the terminal device 110 can also receive media content distributed by the above-mentioned content server 120, and respond to the comment operation of the account receiving the media content on the published media content, obtain comment data on the media content, and send the comment data to the typo recognition server 130.

[0080] As an embodiment, a client of the content platform may be installed on the terminal device 110 , and the terminal device 110 may publish media content to the content server 120 through the client, and receive media content distributed by the content server 120 through the client.

[0081] The content server 120 is used to receive media content uploaded by the terminal device 110 and distribute the media content to one or more terminal devices 110 .

[0082] The typo recognition server 130 is used to obtain target comment data for published media content, extract target text features corresponding to the target comment data based on the context information of each word contained in the target comment data, and when it is determined that the target comment data contains target comment content for typos based on the target text features, the typo information in the media content is determined based on the target comment content.

[0083] The terminal device 110 in the embodiment of the present application can be a mobile terminal, a fixed terminal or a portable terminal, such as a mobile phone, a station, a unit, a device, a multimedia computer, a multimedia tablet, an Internet node, a communicator, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a personal communication system (PCS) device, a personal navigation device, a personal digital assistant (PDA), an audio / video player, a digital camera or a camcorder, a positioning device, a television receiver, a radio broadcast receiver, an e-book device, a gaming device or any combination thereof, including accessories and peripherals of these devices or any combination thereof.

[0084] In the embodiment of the present application, the content server 120 and the typo recognition server 130 can be the same server or different servers; and the content server 120 and the typo recognition server 130 can be independent physical servers, or a server cluster or distributed system composed of multiple physical servers, or multiple cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms in cloud service technology (such as the content server 120 can be but not limited to including the server 120-1, server 120-2 or server 120-3 shown in the figure; such as the typo recognition server 130 can be but not limited to including the server 130-1, server 130-2 or server 130-3 shown in the figure); the functions of the above-mentioned content server 120 can be implemented by one or more cloud servers, and can also be implemented by one or more cloud server clusters, etc.; the functions of the above-mentioned typo recognition server 130 can be implemented by one or more cloud servers, and can also be implemented by one or more cloud server clusters, etc.

[0085] Among them, cloud service technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model application, which can form a resource pool, which can be used on demand and is flexible and convenient. Cloud service technology is an important support; the background service of the technical network system requires a large amount of computing and storage resources, such as video websites, picture websites and more portal websites. With the rapid development and application of the Internet industry, in the future, each item may have its own identification mark, which needs to be transmitted to the background system for logical processing. Data of different levels will be processed separately. All kinds of industry data need strong system backing support, which can only be achieved through cloud service technology.

[0086] based on Figure 1 The following is an example of a method for identifying typos in an embodiment of the present application; please refer to Figure 2 , which is a schematic diagram of a method for identifying typos designed in an embodiment of the present application, specifically comprises the following steps:

[0087] Step S201: Acquire target comment data for published media content.

[0088] As an embodiment, the above-mentioned target comment data can be but is not limited to being selected from a comment data set associated with the above-mentioned media content, and the above-mentioned comment data set includes comment data for the above-mentioned media content; the above-mentioned comment data may include comment content for the above-mentioned media content, and the above-mentioned comment data may also include information such as content identification of the above-mentioned media content, account information of the account that published the above-mentioned comment data, time information of publishing the above-mentioned comment data and at least one information.

[0089] As an embodiment, the comment data may be collected by the terminal device 110. For details, see Figure 3 In (a), the account receiving the media content can trigger a comment operation on the media content, and the terminal device 110 can respond to the comment operation, obtain the comment content indicated by the comment operation as comment data, and send the comment data to the typo recognition server 130; then the typo server 130 can receive the comment data sent by each terminal device, record the comment data for the same media content into a comment data set associated with the media content, and then select comment data from the comment data set as target comment data.

[0090] As an embodiment, the above comment data may also be directly collected by the above wrong character recognition server 130; specifically, see Figure 3 In (b), the account receiving the media content can trigger a comment operation on the media content, and the terminal device 110 can respond to the above comment operation and send a comment instruction to the typo recognition server 130; then the typo server 130 can respond to the above comment instruction, obtain the comment content indicated by the above comment operation as comment data, and record the comment data for the same media content into the comment data set associated with the media content, and then select the comment data from the above comment data set as the target comment data.

[0091] Step S202: extracting target text features corresponding to the target comment data according to the context information of each word contained in the target comment data.

[0092] As an embodiment, in order to improve the accuracy of extracting the above-mentioned target text features, the embodiment of the present application can also use a model to extract the above-mentioned target text features. For example, in the embodiment of the present application, it is possible but not limited to performing feature extraction on the target comment data through at least one of a vector space model (Vector Space Model, VSM), a probability statistics model, or a trained neural network model, etc., to obtain the target text features; wherein, a more detailed method for extracting target text features will be further explained below.

[0093] In the embodiments of the present application, the above-mentioned neural network model is not limited. Technical personnel in this field can set it according to actual needs. The above-mentioned neural network model can include but is not limited to convolutional neural network (CNN), recurrent neural network (RNN), language learning model (Bert model), Ernie model, Albert model, etc.

[0094] Step S203: Based on the target text features, determine whether the target comment data contains target comment content for typos.

[0095] For ease of description, in the following contents of the embodiments of the present application, the target comment data containing target comment content targeting typos will be referred to as typo comment data, and the target comment data not containing target comment content targeting typos will be referred to as non-typo comment data; in order to improve the efficiency and accuracy of identifying typo comments in the target comment data, the embodiments of the present application can, but are not limited to, identify typo comments and non-typo comments in the target comment data through a trained comment data recognition model.

[0096] Step S204: determining the typo information in the media content based on the target comment content.

[0097] As an embodiment, in order to improve the efficiency and accuracy of determining typo information in media content based on target comment content, in an embodiment of the present application, the target comment content can be parsed based on a preconfigured regular expression for identifying the above-mentioned typo information, and the typo information in the media content can be obtained based on the parsing result; among which, the specific method of determining typo information in media content based on regular expressions will be explained below.

[0098] The following content of the embodiment of the present application further explains the above-mentioned comment data classification model involved in step S203.

[0099] As an example, the present application example provides an example of a review data classification model, see Figure 4 , a structural diagram of a review data classification model is provided, the review data classification model may include but is not limited to an input layer, a language learning sub-model, a prediction sub-model and an output layer, wherein:

[0100] The input layer is used to receive the target comment data and pass it to the language learning sub-model; the output layer is used to output the prediction results of the prediction sub-model.

[0101] The above-mentioned language learning sub-model can perform feature extraction on the context information of each word contained in the target comment data to obtain the target text features corresponding to the target comment data; in order to further improve the accuracy of the obtained target text features, the language learning sub-model can perform feature extraction on the context information of each of the above-mentioned words respectively to obtain the word feature vectors mapped by each word, and then obtain the target text features based on the word feature vectors mapped by each word, wherein the average value of the word feature vectors mapped by each of the above-mentioned words can be but is not limited to being determined as the above-mentioned target text features; wherein one context information can be but is limited to including one of the information such as the word vector after encoding (Embedding) the word, the position information of the word in the target comment data, and the data identifier of the target comment data where the word is located.

[0102] As an embodiment, in order to improve the accuracy of the language learning sub-model in extracting target text features, the embodiments of the present application can, but are not limited to, use historical comment data as training samples, and perform feature learning training on the above-mentioned language learning sub-model based on the context information of each word contained in the above-mentioned training samples; wherein, during the above-mentioned training process, the language learning sub-model can learn the text features of words by predicting the words based on the context information of the words in the historical comment data, and then the training process can improve the accuracy of the language learning sub-model in extracting text features, so that the trained language learning sub-model improves the accuracy of extracting target text features.

[0103] The above-mentioned prediction sub-model can identify whether the target comment data is typo comment data or non-typo comment data based on the target text features. Specifically, in the above-mentioned step S203, the target text features extracted by the above-mentioned language learning sub-model can be but not limited to being input into the above-mentioned prediction sub-model; based on the first correlation degree learned by the above-mentioned prediction sub-model, the second correlation degree between the above-mentioned target text features and the target data recognition results is predicted, and if the above-mentioned second correlation degree is greater than the correlation degree threshold, it is determined that the above-mentioned target comment data contains the above-mentioned target comment content; wherein the above-mentioned first correlation degree is determined based on the degree of correlation between the historical text features corresponding to the historical comment data and the above-mentioned target data recognition results, and the above-mentioned target data recognition results are used to characterize that the text data contains comment content for typos, that is, the above-mentioned target data recognition results characterize that the currently processed comment data is typo comment data; wherein the above-mentioned prediction sub-model can be but not limited to being a binary classification model or other classification models.

[0104] As an embodiment, in the embodiment of the present application, the language learning sub-model can be first trained to obtain a trained language learning sub-model, and then after the trained language learning sub-model, a prediction sub-model for identifying typo comment data and non-typo comment data is created to obtain an initial comment data classification model, and the initial comment data classification model is trained to obtain a trained comment data classification model.

[0105] The following content further explains the training process of the above language learning sub-model and comment data classification model.

[0106] First, the structure of the language learning sub-model involved in the present application is exemplified. The language learning sub-model involved in the embodiments of the present application may be, but is not limited to, a Bert model, a Fast-Bert model, a Tiny-Bert model, etc. For ease of understanding, a structural example diagram of a language learning sub-model is provided here, see Figure 5 , the language learning sub-model may include a word representation layer 501, a feature extraction layer 502 and a feature output layer 503; wherein:

[0107] The above-mentioned word representation layer 501 can be used, but is not limited to, to perform word segmentation processing on the above-mentioned comment data (such as the above-mentioned target comment data or historical comment data) according to the word segmentation rules associated with the language form of the comment data, obtain at least one word contained in the comment data, and then process the context information of each word in the at least one word through processing methods such as encoding (embedding) to obtain word representations of each word, such as but not limited to including the word representation E1, word representation E2 and word representation E shown in the figure. N(N is a positive integer), etc., wherein the description of the context information can be found in the above content and will not be repeated here.

[0108] The feature extraction layer 502 may be used, but is not limited to, to extract a word feature for each word representation after processing by the multi-layer unit Trm. As shown in the figure, the word representations E1 to E N , respectively extract word features T1 to T N .

[0109] The feature output layer 503 may, but is not limited to, output the word features extracted by the feature extraction layer 502 .

[0110] Secondly, the training process of the above language learning sub-model is explained in detail:

[0111] In an embodiment of the present application, historical comment data can be used as training samples, and feature extraction training can be performed on the language learning sub-model based on the context information of each word contained in the training samples; specifically, the language learning sub-model can be trained at least once based on a historical comment data set including multiple historical comment data to obtain a trained language learning sub-model.

[0112] As an embodiment, in order to improve the accuracy of text feature extraction by the language learning model, in the embodiment of the present application, a first training end condition can be set for the training process of the language learning sub-model, and then in the process of training the language learning sub-model, when it is determined that the first training end condition is met, the language learning sub-model being trained is output;

[0113] In the embodiment of the present application, the first training end condition is not excessively limited, and those skilled in the art may set it according to actual needs. For example, the first training end condition may be set to one or any combination of the following training end conditions A1 to A3, but is not limited to: training end condition A1) the number of training operations for the above language learning sub-model reaches a first number threshold; training end condition A2) the duration of the training operation for the above language learning sub-model reaches a first duration threshold; training end condition A3) the model prediction error of the language learning sub-model currently being trained is less than a first prediction error threshold, wherein the above model prediction error will be further described below.

[0114] As an embodiment, in a training operation, the language learning sub-model can be used to perform text prediction operations on each historical comment data obtained from the above historical comment data set, determine the model prediction error of the language learning sub-model, and adjust the model parameters of the above language learning sub-model based on the model prediction error. Specifically, see Figure 6, a schematic diagram of a training operation process is provided, and a training operation may include but is not limited to the following steps S601 and S602:

[0115] Step S601 : performing text prediction operations on each piece of historical comment data obtained from the above historical comment data set, and determining the prediction deviation corresponding to each piece of the above historical comment data.

[0116] As an embodiment, the prediction deviation corresponding to a historical comment data can represent the error information of text prediction of some words in the above-mentioned historical comment data through a language learning sub-model; in the text prediction operation for a historical comment data, the above-mentioned part of the words can be predicted based on but not limited to the context information of the above-mentioned part of the words in the above-mentioned historical comment data. Specifically, the above-mentioned text prediction operation can be but not limited to including the following steps S6011 to S6014.

[0117] Step S6011, performing word segmentation processing on the above-mentioned historical comment data according to the word segmentation rule associated with the language form of the above-mentioned historical comment data, to obtain at least one word contained in the above-mentioned historical comment data.

[0118] In this step, the historical comment data is segmented based on the word segmentation rules associated with the language form of the historical comment data, which can improve the accuracy of the obtained words and further improve the accuracy of subsequent word predictions.

[0119] As an embodiment, technicians in this field can determine the word segmentation rules associated with each language form according to actual business needs and language characteristics of the language form. In the embodiments of the present application, examples of word segmentation rules associated with several language forms are given below.

[0120] Step S6012: masking some words in the at least one word based on a preset word mask.

[0121] As an embodiment, in this step, a preset word mask Mask may be used but is not limited to randomly masking one or more words in the above-mentioned missing word. The specific form of the above-mentioned word mask Mask is not limited, and technicians in this field can set it according to actual needs.

[0122] Step S6013, determining the context information of the above-mentioned partial words in the above-mentioned historical comment data, and selecting candidate words whose matching degree with the determined context information meets the matching condition from the pre-configured candidate word library, and the above-mentioned candidate word library is determined based on the above-mentioned historical comment data set.

[0123] As an embodiment, before training the language learning sub-model, it is possible but not limited to performing word segmentation processing on each historical comment data in the historical comment data set, determining each word obtained by the word segmentation processing as a candidate word, and determining the obtained candidate word set as the above-mentioned candidate word library, wherein the specific method of performing word segmentation processing on each historical comment data can be referred to step S6011, which will not be repeated here.

[0124] As an embodiment, there is no limitation on the situation where the above-mentioned matching degree satisfies the matching degree condition. Technical personnel in this field can set it according to actual needs. For example, the matching degree with the largest value among the matching degrees between the candidate words and the determined context information can be determined as the matching degree that satisfies the matching degree condition, or the matching degree closest to the matching degree threshold among the matching degrees between the candidate words and the determined context information can be determined as the matching degree that satisfies the matching degree condition.

[0125] As an embodiment, the above step S6013 can, but is not limited to, perform the following text prediction operations on each word in the partial words, respectively select candidate words that match each word in the partial words, and determine them as the corresponding predicted words in the partial words: for a word in the partial words, determine the context information of the above word in the above historical comment data as the target context information, and then determine the matching degree of each candidate word in the candidate vocabulary and the target context information, select candidate words whose matching degree reaches the matching degree threshold, and determine the selected candidate words as the predicted words matching the above word; wherein the above text prediction operation method is only an exemplary description, and technicians in this field can flexibly adopt other methods to predict words.

[0126] Step S6014: Determine the deviation information between the above-mentioned partial words and the selected candidate words as the prediction deviation corresponding to the above-mentioned one historical comment data.

[0127] As an embodiment, the above prediction deviation may be, but is not limited to, representing the degree of deviation between the above partial words and the selected candidate words, and the degree of deviation may be negatively correlated with the degree of matching between the partial words and the selected candidate words; in the embodiment of the present application, a specific method for determining the above prediction deviation may be set according to actual needs, and several examples of determining the prediction deviation are given below:

[0128] In the embodiment of the present application, the prediction deviation corresponding to the above-mentioned one historical comment data can be determined based on the string matching degree or semantic matching degree between the partial words and the selected candidate words. For example, when the partial words include multiple words, the semantic matching degree between each word and the corresponding predicted word (that is, the candidate words selected for each word) can be determined as the error information corresponding to each word, and then the average of the error information corresponding to each word in the above-mentioned multiple words is determined as the prediction deviation corresponding to the above-mentioned one historical comment data;

[0129] In the embodiment of the present application, when some words include multiple words, the prediction correct probability for the multiple words can also be determined as the prediction error corresponding to the above-mentioned one historical comment data. The above-mentioned prediction probability can be determined by, but is not limited to, formula 1:

[0130]

[0131] In formula (1), K2 is the total number of words that are blocked in the above historical comment data, K1 is the number of candidate words selected as blocked words, and P1 is the prediction deviation corresponding to a historical comment data; for example, if the blocked words in a historical comment data are word 1, word 2, and word 3, the candidate word selected for word 1 is word 1, the candidate word selected for word 2 is word 5, and the candidate word selected for word 3 is word 4, then the above K2 is 3, K1 is 1, and the prediction deviation corresponding to the historical comment data is 1 / 3.

[0132] Step S602: adjusting the parameters of the language learning sub-model based on the prediction deviations corresponding to the various historical comment data.

[0133] As an embodiment, the model prediction error of the language learning sub-model can be determined based on the prediction deviations corresponding to each historical comment data, and the parameters of the language learning sub-model can be adjusted based on the model prediction error, such as adjusting the model parameters of the language learning sub-model in the direction of reducing the model prediction error, etc., but not limited to.

[0134] In order to improve the flexibility of the implementation of the solution, the specific method of determining the above-mentioned model prediction error is not too limited in the embodiment of the present application. In the embodiment of the present application, it can be flexibly set according to actual business needs, such as but not limited to determining the mean of the prediction deviations corresponding to each historical comment data as the above-mentioned model prediction error; the model prediction error of the language learning sub-model can also be determined based on the principle of the following formula 2:

[0135]

[0136] In formula (2), K4 is the total number of historical review data in the historical review data set, K3 is the number of historical review data with correct text prediction, and P2 is the prediction error of the above model; among them, the historical review data with correct text prediction can be historical review data with a prediction deviation greater than the prediction deviation threshold, and the historical review data with correct text prediction can also be historical review data where the selected candidate words are the blocked partial words themselves.

[0137] As an embodiment, to facilitate further understanding of the solution of the embodiments of the present application, the following gives examples of word segmentation rules associated with several language forms:

[0138] The word segmentation rules associated with the language forms involved in the embodiments of the present application can, but are not limited to, dividing the consecutive characters that form the reference words in the reference word segmentation set associated with the language form in the historical review data; among them, the above reference words can, but are not limited to, be the first character group composed of at least one character with a combined usage frequency higher than the first frequency threshold in the above language form, where the arrangement positions of at least some characters in the above first character group can have a sequence, and the combined usage frequency can be determined based on the statistics of the historical usage of each character in the language form, etc.; here, Chinese is used as an example of the language form of a historical review data for illustration, a Chinese character can be a character. Assuming that the above first frequency threshold is 0.65, the combined usage frequency of the character '长' and the character '江' to form '长江' is 0.68, the combined usage frequency of the character '陈' and the character '列' to form '列陈' is 0.25, the combined usage frequency of the character '黄' and the character '河' to form '黄河' is 0.75, and the combined usage frequency of the character '黄' and the character '河' to form '黄河' and then form '河黄' is 0.20, then '长江' and '黄河' can be used as the reference words in the reference word segmentation set associated with Chinese respectively, and '列陈' and '河黄' are not used as reference words. Furthermore, if the target review data contains the character '长' and the character '江' is in the consecutive position after '长', then '长' and '江' in the target review data can be segmented into one word.

[0139] In the embodiments of the present application, the word segmentation rule associated with the language form may also be to divide consecutive characters with a combined usage frequency greater than a second frequency threshold in the language form of the above-mentioned one historical comment data in the target comment data into one word; where the above-mentioned consecutive characters are denoted as a second character group. Here, taking Chinese as the language form of the above-mentioned one historical comment data as an example, the determination of the above-mentioned combined usage frequency and the obtained second character group will be exemplarily described; if a second character group includes consecutive characters B1 and B2 in position, then the probability of character B2 appearing after character B1 in Chinese can be determined as the combined usage frequency of this second character group. For example, the probability of the character "江" appearing after the character "长" in Chinese is 0.8, and the combined usage frequency of "长江" is determined to be 0.8; if a second character group includes consecutive characters B3, B4, and B5 in position, then based on the first probability of character B4 appearing after character B3 in Chinese and the second probability of character B5 appearing after character B4 in Chinese, the combined usage frequency of this second character group can be determined. For example, the product of the first probability and the second probability can be directly determined as the combined usage frequency of this second character group. Here, a specific example is given. If there are consecutive "黄", "浦", and "江" in the target comment data, and the probability of the character "浦" appearing after the character "黄" in Chinese is 0.9, and the probability of the character "江" appearing after the character "浦" is 0.7, the combined usage frequency of the consecutive appearance of "黄浦江" is "0.9×0.8 = 0.72". If the second frequency threshold is 0.6, then the consecutive "黄", "浦", and "江" in the target comment data can be divided into one word "黄浦江"; when the number of characters in the second character combination exceeds 3, the combined frequency of each second character combination can be determined based on the above method, and no more details will be described here;

[0140] It should be noted that the above-mentioned word segmentation rule associated with the language form is only an exemplary introduction. Those skilled in the art can determine the word segmentation rule associated with each language form according to the actual business requirements and the language characteristics of the language form. For different language forms, the above-mentioned characters can be set according to the actual situation. For example, when the language form is Chinese, one Chinese character can be one character, and when the language form is English, one English word can be one character.

[0141] The following content of the embodiments of the present application further describes the training process of the above-mentioned comment data classification model.

[0142] As an embodiment, in the embodiment of the present application, during the process of training the comment data classification model, the historical comment data can be annotated with data types and used as training samples, and the comment data classification model can be trained based on a training sample set consisting of multiple training samples. During the training process, the data type annotated for each historical comment data can be used as the annotated data type, and the historical text features of each historical comment data can be extracted through the trained language learning sub-model, and the data type of each historical comment data can be estimated based on the above historical text features through the prediction sub-model, and then the estimated data type can be used as the predicted data type, and the prediction error of the comment data classification model is determined based on the deviation information between the annotated data type and the predicted data type corresponding to each historical comment data, and then the parameters of the above prediction sub-model are adjusted in the direction of reducing the prediction error of the comment data classification model, and when the second training end condition is met, the above trained language learning sub-model and the current prediction sub-model are output as the trained comment data classification model;

[0143] Among them, the above-mentioned marked data type can be the above-mentioned typo comment data or non-typo comment data. If a historical comment data contains target comment content for typos, then the historical comment data can be marked as typo comment data; if a historical comment data does not contain target comment content for typos, then the historical comment data can be marked as non-typo comment data.

[0144] As an embodiment, the second training end condition mentioned above is not excessively limited in the embodiments of the present application. Technical personnel in this field can set it according to actual needs, such as but not limited to setting the second training end condition to one or any combination of the following training end conditions C1 to C3: training end condition C1) the number of training operations for the above comment data classification model reaches the second number threshold; training end condition C2) the duration of the training operation on the above comment data classification model reaches the second duration threshold; training end condition C3) the prediction error of the comment data classification model currently being trained is less than the second prediction error threshold, etc.

[0145] As an embodiment, in order to improve the efficiency and flexibility of identifying typos in media content based on target comment data for the media content, in an embodiment of the present application, the target comment data in the target comment data set for the media content can be input into the above-mentioned comment data classification model in batches, or the target comment data in the target comment data can be input into the comment data classification model one by one for identification.

[0146] As an embodiment, in order to further improve the accuracy of identifying typos in media content in different language forms, historical comment data sets in different language forms can be used to train corresponding language learning sub-models and comment data classification models to obtain language learning sub-models and comment data classification sub-models associated with different language forms; when identifying typos in media content, the language form of the above media content can be determined as the target language form, and comment data in the target language form can be selected from the comment data set for the media content to be processed as target comment data, and then the target comment data can be processed using the comment data classification model associated with the above target language form to determine whether the target comment data is typo comment data. The specific processing method can be found in the above content and will not be repeated here; wherein, the language learning sub-model in the comment data classification model associated with the above target language form is a trained language learning sub-model associated with the above target language form.

[0147] The following content of the embodiment of the present application further illustrates the process of parsing the target comment content based on the regular expression in the above step S204 to determine the typos in the media content:

[0148] First, the regular expression involved in the embodiment of the present application is further explained; the above regular expression may include but is not limited to at least one reference text for indicating a typo, and at least one of the information such as a typo placeholder and a language symbol indicating a typo, wherein the above typo placeholder may be in one of the reference texts or between two reference texts. Those skilled in the art may set the reference text and typo placeholder in the above regular expression according to actual needs, such as the regular expression may be but is not limited to the following regular expressions 1 to 5:

[0149] Regular expression 1: "S1";

[0150] Regular expression 2: "S1-P1, S2";

[0151] Regular expression 3: "P1-S1";

[0152] Regular expression 4: "D1-P1-D2";

[0153] Regular expression 5: "D1-P1, E1";

[0154] Among them, in the above regular expressions 1 to 5, S1 and S2 are different reference texts, P1 is a wrong character placeholder, D1 and D2 are different character segments in the same reference text, and E1 is a target word placeholder, which can be but is not limited to being a placeholder for the correct word corresponding to the wrong character.

[0155] As an embodiment, in order to improve the accuracy of parsing the target comment content, in the embodiments of the present application, different sets of regular expressions can be pre-configured for comment data in different language forms, and then, based on each regular expression in the set of regular expressions associated with the language form of the target comment data, the target comment content is parsed; for the convenience of understanding, the following takes Chinese as an example to give the set of regular expressions associated with Chinese, and this set of regular expressions may but is not limited to include the following Examples 1 to 5:

[0156] Example 1: "There is a character written wrongly."

[0157] Example 2: "The 'which' in the text seems to be written wrongly."

[0158] Example 3: "The character 'Wen' is written wrongly."

[0159] Example 4: "The 'Wen' in 'Li Wen' is written wrongly."

[0160] Example 5: "It's not 'Li Wen', but should be 'Li Wen'."

[0161] Among them, the above Example 1 is an example of Regular Expression 1, and the "There is a character written wrongly." in it is S1 in Regular Expression 1; the above Example 2 is an example of Regular Expression 2, where the "in the text" and "seems to be written wrongly" are S1 and S2 in Regular Expression 2 respectively, and the "which" is P1 in Regular Expression 2; the above Example 3 is an example of Regular Expression 3, where the "Wen" is P1 in Regular Expression 2, and the "character is written wrongly" is S1 in Regular Expression 3; the above Example 4 is an example of Regular Expression 4, where the "in 'Li Wen'" and "is written wrongly" are D1 and D2 in Regular Expression 4 respectively, and the "Wen" is P1 in Regular Expression 4; the above Example 5 is an example of Regular Expression 5, where the "not" and "but should be" are D1 and D2 in Regular Expression 5 respectively, the "Li Wen" is P1 in Regular Expression 5, and the "Li Wen" is E1 in Regular Expression 5.

[0162] As an embodiment, there is no excessive limitation on the number of regular expressions in the set of regular expressions associated with different language forms respectively. Those skilled in the art can set according to actual needs. In the above step S204, the regular expressions can be sequentially selected from the set of regular expressions associated with the language form of the target comment data to parse the target comment content in the target comment data.

[0163] Specifically, in step S204, the target comment content contained in the target comment data can be parsed based on the selected regular expression to obtain a corresponding parsing result; based on the above parsing result, at least one typo associated with the above target comment content and the text position information of the above at least one typo in the above media content are determined; the above at least one typo and the above text position information are determined as typo information in the above media content.

[0164] As an embodiment, in order to improve the accuracy of the obtained parsing result, in the embodiment of the present application, the character matching degree between the target comment data and the reference text in the regular expression can be determined as the above parsing result, but is not limited to; specifically, in step S204, for each selected regular expression, the following operations can be performed to determine the typo information in the above media content:

[0165] Determine the character matching degree between the target comment data and a regular expression; if the above character matching degree is greater than the matching degree threshold, then based on the text position information of the typo placeholder in the above regular expression, determine the character at the text position of the typo placeholder in the above target comment data as the typo, and determine the text position of the typo placeholder as the text position information of the typo, etc.

[0166] As an embodiment, in order to improve the accuracy of typo detection, in an embodiment of the present application, if the typo information is not obtained based on the analysis result in step S204, typo detection can be performed on the media content based on but not limited to pre-configured typo detection rules to obtain detection results, and based on the detection results, it is determined whether the media content contains corresponding typo information.

[0167] As an embodiment, the failure to obtain the above-mentioned typo information based on the above-mentioned parsing results may include but is not limited to the following situations 1) and 2): Situation 1) In the set of regular expressions associated with the language form of the target comment data, there is no regular expression whose character matching degree with the target comment data is greater than the matching degree threshold; Situation 2) In the set of regular expressions associated with the language form of the target comment data, among the regular expressions whose character matching degree with the target comment data is greater than the matching degree threshold, no specific typo information is indicated, such as may be but is not limited to the situation where the regular expression whose character matching degree is greater than the matching degree threshold is the above-mentioned regular expression 1.

[0168] As an embodiment, the preconfigured typo detection rule may be, but is not limited to, based on a trained typo detection model to directly detect typos in the media content, or the typo detection may be performed by a reviewer on the media content.

[0169] As an embodiment, the typo information indicated in the target comment data is not necessarily reliable. Therefore, in order to improve the accuracy of identifying typos, in the embodiment of the present application, after determining that the target comment data contains target comment content for typos based on the target text features, before determining the typo information in the media content based on the target comment content, the confidence of the target comment data can be determined based on the account information of the target account that published the target comment data. If it is determined that the confidence of the target comment data reaches the confidence threshold, the typo information in the media content can be determined based on the target comment content. ; In this way, target comment data with unreliable information can be filtered out, thereby improving the accuracy of identifying typos; the above-mentioned account information may be but is not limited to the account portrait data of the target account, which is also called account portrait or account portrait (UserProfile), which refers to labeling the information of accounts associated with the account; the account portrait data may be but is not limited to including at least one of the following information: demographic attribute information and social attributes such as the account's gender, age, region, height, education, marital status, industry, etc., account information such as the account's account level and account reputation, and information mined from the account's historical behavior data.

[0170] As an embodiment, since the target comment data is not necessarily accurate, the typo information obtained in the above step S204 may be wrong. Therefore, in the embodiment of the present application, in step S204, after obtaining the typo information, the typo information can also be fed back to the content reviewer, and the content reviewer can determine whether the above typo information is correct. If it is determined that the typo information is correct, the content reviewer can correct the typo indicated by the above typo information in the media content.

[0171] As an embodiment, the following content provides an example of a complete process of a method for identifying a wrong character, see Figure 7 , specifically including the following steps:

[0172] Step S701 : selecting currently unprocessed target comment data from a set of comment data managed for published media content.

[0173] Step S702: input the selected target comment data into the trained comment data classification model, and extract the target text features corresponding to the target comment data through the language learning sub-model in the comment data classification model.

[0174] Step S703: Identify the selected target comment data through the prediction sub-model in the comment data classification model.

[0175] Step S704, determining whether the target comment data is a comment data with a typo, if so, proceeding to step S705, otherwise proceeding to step S709.

[0176] Step S705: parsing the target comment data based on a regular expression associated with the language form of the media content to obtain a parsing result.

[0177] Step S706, determining whether the parsing result contains wrong character indication information, if so, proceeding to step S707, otherwise proceeding to step S708;

[0178] The typo indication information may include but is not limited to at least one typo associated with the target comment content in the target comment data, and text position information of the at least one typo in the media content.

[0179] Step S707, based on the parsing result, determine the typo information in the above media content, and proceed to step S709.

[0180] Step S708, based on the pre-configured typo detection rules, perform typo detection on the above media content to obtain a detection result, and determine whether the above media content contains corresponding typo information based on the detection result, and then enter step S709.

[0181] Step S709, determining whether there is unprocessed target comment data in the comment data set associated with the above media content, if yes, proceed to step S701, otherwise, end the processing.

[0182] The specific contents of the above steps S701 to S709 can be found in the above description and will not be repeated here.

[0183] See also Figure 8, a specific example of a method for identifying typos is provided, in which an article is taken as an example of media content. After an account on a content platform creates an article, the article can be published on the content platform through a content production end; and then, after the article is published through a content production end, the content platform can distribute the article to a content consumption end through a content distribution outlet; as the article is continuously exposed and recommended on the information flow, the account on the content platform can click and read the article, and can trigger a comment operation on the article, and then the above-mentioned typos identification server 130 can use the comment data obtained based on the above-mentioned comment operation as the target comment data, identify the target comment data based on the comment data classification model, determine whether the above-mentioned target comment data contains the target comment content for the typos, and when it is determined that the target comment data contains the target comment content, determine the typos information in the above-mentioned article based on the target comment content; further, the above-mentioned target comment data and the above-mentioned typos information can also be fed back to the content reviewer of the manual review system, and the content reviewer determines whether the above-mentioned typos information is correct. When it is determined that the typos information is correct, the content reviewer can modify the typos indicated by the above-mentioned typos information, etc.

[0184] In the embodiments of the present application, on the one hand, typos in media content can be identified based on target comment data for the media content, thereby improving the accuracy of identifying typos in media content; experiments have shown that using the typo identification method provided in the embodiments of the present application, more than 90% of the typo information in typo comment data can be identified, and the comment data classification model can achieve an accuracy of more than 95% for identifying typo comment data; on the other hand, if content reviewers conduct typos in media content based on the identified typo information.

[0185] Please refer to Fig. 9 Based on the same inventive concept, the embodiment of the present application provides a wrong character recognition device 900, comprising:

[0186] The data acquisition unit 901 is used to acquire target comment data for the published media content;

[0187] A feature extraction unit 902 is used to extract target text features corresponding to the target comment data according to context information of each word contained in the target comment data;

[0188] The first recognition unit 903 is used to determine, based on the target text features, whether the target comment data contains target comment content for typos;

[0189] The second recognition unit 904 determines the typo information in the media content based on the target comment content.

[0190] As an embodiment, the feature extraction unit 902 is specifically used for:

[0191] Input the above target review data into the trained review data classification model;

[0192] Based on the language learning sub-model in the above comment data classification model, feature extraction is performed on the context information of each word contained in the above target comment data to obtain the target text features corresponding to the above target comment data;

[0193] The above-mentioned language learning sub-model is obtained by using the historical comment data as training samples and performing feature learning training on the above-mentioned language learning sub-model based on the context information of each word contained in the above-mentioned training samples.

[0194] As an embodiment, the comment data classification model further includes a prediction sub-model, and the first recognition unit 903 is specifically used for:

[0195] Input the above target text features into the above prediction sub-model;

[0196] Based on the first correlation degree learned by the prediction sub-model, predict a second correlation degree between the target text feature and the target data recognition result, wherein the first correlation degree is determined based on the correlation degree between the historical text feature corresponding to the historical comment data and the target data recognition result, and the target data recognition result is used to characterize the comment content containing typos in the text data;

[0197] If the second correlation degree is greater than the correlation degree threshold, it is determined that the target comment data contains the target comment content.

[0198] As an embodiment, the feature extraction unit 902 is further configured to train the above-mentioned language learning model in the following manner:

[0199] Based on the historical comment data set, a training operation is performed on the language learning sub-model, and one training operation includes: performing a text prediction operation on each historical comment data obtained from the historical comment data set to determine the prediction deviation corresponding to each of the historical comment data; and adjusting the parameters of the language learning sub-model based on the prediction deviation corresponding to each of the historical comment data;

[0200] The above text prediction operation includes:

[0201] According to the word segmentation rule associated with the language form of one of the historical comment data, the one historical comment data is subjected to word segmentation processing to obtain at least one word contained in the one historical comment data;

[0202] Based on a preset word mask, masking some words in the at least one word; and

[0203] Determine context information of the above-mentioned part of words in the above-mentioned one historical comment data, and select candidate words whose matching degree with the determined context information satisfies the matching degree condition from a pre-configured candidate word library, wherein the above-mentioned candidate word library is determined based on the above-mentioned historical comment data set;

[0204] The deviation information between the above-mentioned partial words and the selected candidate words is determined as the prediction deviation corresponding to the above-mentioned one historical comment data.

[0205] As an embodiment, the second identification unit 904 is specifically configured to:

[0206] Based on a pre-configured regular expression for identifying the typos, the target comment content is parsed to obtain a corresponding parsing result;

[0207] Based on the analysis result, determining at least one wrong character associated with the target comment content and text position information of the at least one wrong character in the media content;

[0208] The at least one wrong character and the text position information are determined as wrong character information in the media content.

[0209] As an embodiment, the second identification unit 904 is further configured to:

[0210] If the typo information is not obtained based on the analysis result, then based on the pre-configured typo detection rule, typo detection is performed on the media content to obtain a detection result;

[0211] According to the detection result, it is determined whether the media content contains corresponding wrong character information.

[0212] As an embodiment, the second identification unit 904 is used for:

[0213] Obtain the account information of the target account that published the target comment data;

[0214] Based on the above account information, determine the confidence level of the above target review data;

[0215] When it is determined that the confidence reaches a confidence threshold, the typo information in the media content is determined based on the target comment content.

[0216] As an example, Fig. 9 The device in can be used to implement any of the methods for identifying wrong characters discussed above.

[0217] Based on the same inventive concept as the above method embodiment, a computer device is also provided in the embodiment of the present application. The computer device can be used for data processing based on push content. In one embodiment, the computer device can be a server, such as Figure 1 The typo recognition server 130 shown in FIG. 1 is shown in FIG. 1 . In this embodiment, the structure of the computer device can be as follows: Fig.10 As shown, it includes a memory 1001 , a communication module 1003 and one or more processors 1002 .

[0218] The memory 1001 is used to store computer programs executed by the processor 1002. The memory 1001 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and programs required for running the instant messaging function, etc.; the data storage area may store various instant messaging information and operation instruction sets, etc.

[0219] The memory 1001 may be a volatile memory, such as a random-access memory (RAM); the memory 1001 may also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); or the memory 1001 may be any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 1001 may be a combination of the above memories.

[0220] The processor 1002 may include one or more central processing units (CPU) or a digital processing unit, etc. The processor 1002 is configured to implement the above-mentioned data processing method based on content push when calling the computer program stored in the memory 1001 .

[0221] The communication module 1003 is used to communicate with terminal devices and other servers.

[0222] The specific connection medium between the memory 1001, the communication module 1003 and the processor 1002 is not limited in the embodiments of the present application. Fig.10 In the embodiment, the memory 1001 and the processor 1002 are connected via a bus 1004. The bus 1004 is connected to the processor 1002 via a bus 1004. Fig.10The connections between other components are shown in bold lines, which are only for illustration and are not intended to be limiting. Bus 1004 can be divided into address bus, data bus, control bus, etc. For ease of illustration, Fig.10 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0223] The memory 1001 stores a computer storage medium, and the computer storage medium stores computer executable instructions, and the computer executable instructions are used to implement the content recommendation method of the embodiment of the present application. The processor 1002 is used to execute the above-mentioned wrong character recognition method.

[0224] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks or optical disks, etc., various media that can store program codes.

[0225] Alternatively, if the above-mentioned integrated unit of the invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the above-mentioned typo recognition method of each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0226] Based on the same technical concept, the embodiment of the present application also provides a computer-readable storage medium, which stores computer instructions. When the above computer instructions are executed on a computer, the computer executes the method for identifying typos as discussed above.

[0227] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0228] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for identifying typos, characterized in that: include: Obtaining target comment data for published media content; Based on the language learning sub-model in the comment data classification model, feature extraction is performed on the context information of each word contained in the target comment data to obtain the target text feature corresponding to the target comment data; wherein the language learning sub-model is obtained by training the historical comment data through word segmentation processing based on the word segmentation rule associated with the language form, the word segmentation rule associated with the language form is to divide the continuous characters of the reference words in the reference word segmentation set associated with the language form in the historical comment data together, the reference words are a first character group composed of at least one character in the above-mentioned language form whose combined use frequency is higher than a first frequency threshold, the arrangement positions of at least one character in the first character group exist in a chronological order, and the combined use frequency is determined based on the statistics of the historical use of each character in the language form; Based on the prediction sub-model of the comment data classification model and the target text features, classify and identify whether the target comment data contains target comment content for typos, and obtain the target comment content for typos in the target comment data; Based on a preconfigured regular expression for identifying typos, the target comment content is parsed to obtain a corresponding parsing result; wherein the regular expression includes: at least one reference text for indicating typos, and at least one of a typos placeholder and a language symbol indicating typos, wherein the typos placeholder is in one of the reference texts or between two reference texts; Based on the parsing result, at least one typo associated with the target comment content and text position information of the at least one typo in the media content are determined, and the at least one typo and the text position information are determined as typo information in the media content.

2. The method according to claim 1, characterized in that The prediction sub-model based on the comment data classification model and the target text features classifies and identifies whether the target comment data contains target comment content for typos, and obtains the target comment content for typos in the target comment data, including: Inputting the target text features into the prediction sub-model; Predicting a second degree of association between the target text feature and the target data recognition result based on the first degree of association learned by the prediction sub-model, wherein the first degree of association is determined based on the degree of association between the historical text feature corresponding to the historical comment data and the target data recognition result; If the second relevance is greater than the relevance threshold, it is determined that the target comment data contains the target comment content.

3. The method according to claim 1, characterized in that The language learning sub-model is trained in the following way: Based on the historical comment data set, a training operation is performed on the language learning sub-model, wherein one training operation comprises: performing a text prediction operation on each historical comment data obtained from the historical comment data set, and determining a prediction deviation corresponding to each historical comment data; and adjusting parameters of the language learning sub-model based on the prediction deviation corresponding to each historical comment data; Wherein, the text prediction operation includes: According to the word segmentation rule associated with the language form of one historical comment data among the various historical comment data, the one historical comment data is subjected to word segmentation processing to obtain at least one word included in the one historical comment data; Based on a preset word mask, masking some words in the at least one word; and Determining context information of the partial words in the one historical review data, and selecting candidate words whose matching degree with the determined context information satisfies a matching degree condition from a pre-configured candidate word library, wherein the candidate word library is determined based on the historical review data set; The deviation information between the partial words and the selected candidate words is determined as the prediction deviation corresponding to the one historical comment data.

4. The method according to claim 1, characterized in that The method further comprises: If the typo information is not obtained based on the parsing result, performing typo detection on the media content based on a preconfigured typo detection rule to obtain a detection result; According to the detection result, it is determined whether the media content contains corresponding wrong character information.

5. The method according to any one of claims 1 to 3, characterized in that: Determining the typo information in the media content based on the target comment content includes: Obtaining account information of a target account that published the target comment data; Determining the confidence level of the target review data based on the account information; When it is determined that the confidence reaches a confidence threshold, typo information in the media content is determined based on the target comment content.

6. A device for identifying wrong characters, characterized in that: include: A data acquisition unit, used to acquire target comment data for published media content; A feature extraction unit is used to extract features from context information of each word contained in the target comment data based on a language learning sub-model in a comment data classification model, so as to obtain target text features corresponding to the target comment data; wherein the language learning sub-model is obtained by training historical comment data that performs word segmentation processing based on a word segmentation rule associated with a language form, wherein the word segmentation rule associated with a language form is to divide together consecutive characters of reference words in a reference word segmentation set associated with a language form in the historical comment data, wherein the reference words are a first character group composed of at least one character in the above-mentioned language form whose combined use frequency is higher than a first frequency threshold, wherein there is a sequence of arrangement positions between at least one character in the first character group, and the combined use frequency is determined based on statistics of historical usage of each character in the language form; A first recognition unit is used to classify and recognize whether the target review data contains target review content for typos, and obtain the target review content for typos contained in the target review data; A second recognition unit is used to parse the target comment content based on a preconfigured regular expression for identifying wrong characters to obtain a corresponding parsing result; wherein the regular expression includes: at least one reference text for indicating wrong characters, and at least one of a wrong character placeholder and a language symbol indicating the wrong characters, wherein the wrong character placeholder is in one of the reference texts or between two reference texts; Based on the parsing result, determining at least one wrong character associated with the target comment content, and text position information of the at least one wrong character in the media content; The at least one wrong character and the text position information are determined as wrong character information in the media content.

7. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 5.

Images