Data processing method and device based on combined model, equipment and storage medium

Abstract

The embodiment of the invention discloses a data processing method and device based on a combined model, equipment and a storage medium, and the method comprises the steps: obtaining text information, carrying out the sensitive information detection of the text information through a first language model, and obtaining a plurality of sensitive words; determining a text position of each sensitive word in the text information, and determining a context analysis position and text content corresponding to the context analysis position according to the text position; determining target sensitive words in the sensitive words by performing context analysis on the text content, and filtering the target sensitive words in the text information to obtain security data; and reporting the security data to a server for the server to perform word segmentation processing on the security data based on a second language model to obtain a plurality of keywords, and generating response information corresponding to the text information according to the plurality of keywords and the part-of-speech of each keyword. The security of the data and the deep understanding capability of the data are improved, the situations of false report and missing report are reduced, and the satisfaction degree of the user is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular to a data processing method, apparatus, device and storage medium based on a combination model. Background Art

[0002] The application of commercial large models mainly stems from the rapid development of deep learning technology, especially the rise of large-scale pre-trained language models (PLMs). In recent years, with the rapid advancement of deep learning technology, large-scale pre-trained language models have become an important paradigm in the field of natural language processing (NLP). These models can learn rich language knowledge and general representations through self-supervised pre-training on a large amount of unlabeled text data. Through fine-tuning and other methods, these models can be applied to tasks such as question and answer, dialogue, summarization, and translation, and have achieved significant performance improvements. In particular, general large models such as BERT, GPT, T5, etc. have demonstrated powerful language understanding and generation capabilities. The release of ChatGPT has aroused widespread attention in the industry to commercial large model technology.

[0003] However, the method of processing data through commercial big models has data security issues, which may lead to data leakage and abuse. In order to solve the data security risks existing in the use of commercial big models, in the related technology, data is filtered by text matching to avoid data insecurity. However, this text matching method is prone to false positives and false negatives, lacks a deep understanding of context and semantics, and affects user satisfaction. Summary of the invention

[0004] The embodiments of the present application provide a data processing method, apparatus, device and storage medium based on a combined model, which solves the problem of security risks in the data processing process of large commercial models and the false positives and false negatives caused by the lack of deep understanding of context and semantics in the text matching process, improves the security of data and the ability to deeply understand data, reduces false positives and false negatives, and improves user satisfaction.

[0005] In a first aspect, an embodiment of the present application provides a data processing method based on a combination model, comprising:

[0006] Acquire text information, and perform sensitive information detection on the text information through a first language model to obtain a plurality of sensitive words, where the first language model is a small local model;

[0007] Determine the text position of each sensitive word in the text information, and determine the context analysis position and the text content corresponding to the context analysis position according to the text position;

[0008] Determine a target sensitive word among the sensitive words by performing context analysis on the text content, and filter the target sensitive words in the text information to obtain security data;

[0009] The security data is reported to a server, and the server performs word segmentation processing on the security data based on a second language model to obtain a plurality of keywords, and generates response information corresponding to the text information according to the plurality of keywords and the part of speech of each keyword.

[0010] Optionally, determining the text position of each sensitive word in the text information, and determining the context analysis position and the text content corresponding to the context analysis position according to the text position includes:

[0011] Determine the text paragraph and text line number to which each sensitive word belongs, determine the target text line number for context analysis based on the text paragraph and the text line number, and determine the text content corresponding to the target text line number.

[0012] Optionally, determining the target text line number for context analysis according to the text paragraph and the text line number includes:

[0013] Perform word count on the text paragraph, and determine whether the target text paragraph for context analysis is the same as the text paragraph based on the statistical result;

[0014] In the case where the target text paragraph is the same as the text paragraph, determining the text line number and the text line number adjacent to the text line number as the target text line number;

[0015] In the case that the target text paragraph is different from the text paragraph, all text line numbers contained in the target text paragraph are determined as target text line numbers.

[0016] Optionally, determining a target sensitive word among the sensitive words by performing context analysis on the text content includes:

[0017] Determine the category of each sensitive word and the sensitivity level of the text content by performing contextual analysis on the text content, and make sensitivity judgments on each category and the sensitivity level;

[0018] When the sensitivity level is greater than the preset sensitivity level and there is a sensitive category in each category, the sensitive word corresponding to the sensitive category is determined as the target sensitive word.

[0019] Optionally, generating response information corresponding to the text information according to the multiple keywords and the part of speech of each keyword includes:

[0020] The response type of the text information and a response database corresponding to the response type are determined according to the multiple keywords and the part of speech of each keyword, and the response information corresponding to the text information is generated according to the response database.

[0021] Optionally, determining the response type of the text information and the response database corresponding to the response type according to the multiple keywords and the part of speech of each keyword includes:

[0022] Determining a sentence type of the text information according to the multiple keywords and the part of speech of each of the keywords, and when the sentence type is a question type, determining a response type corresponding to the text information as a reply type, and determining a response database corresponding to the reply type as a reply database;

[0023] In the case where the sentence type is a declarative sentence type, the response type corresponding to the text information is determined to be a translation type, and the response database corresponding to the translation type is determined to be a translation database.

[0024] Optionally, determining the sentence type of the text information according to the multiple keywords and the part of speech of each keyword includes:

[0025] Identify whether there are question words or language identifiers among the multiple keywords. If there are question words or language identifiers among the multiple keywords, determine the text meaning of the text information based on the multiple keywords and the part of speech of each of the keywords, and determine the sentence type of the text information based on the text meaning.

[0026] In a second aspect, an embodiment of the present application provides a data processing device based on a combination model, comprising:

[0027] A sensitive information detection module, used to obtain text information, perform sensitive information detection on the text information through a first language model, and obtain a plurality of sensitive words, wherein the first language model is a small local model;

[0028] A text position determination module, used to determine the text position of each sensitive word in the text information;

[0029] A text content determination module, used to determine a context analysis position and text content corresponding to the context analysis position according to the text position;

[0030] A target sensitive word determination module, configured to determine a target sensitive word among the sensitive words by performing context analysis on the text content;

[0031] A sensitive word filtering module, used to filter each target sensitive word in the text information to obtain safe data;

[0032] A data reporting module is used to report the security data to a server, and the server is used to perform word segmentation processing on the security data based on a second language model to obtain multiple keywords, and generate response information corresponding to the text information according to the multiple keywords and the part of speech of each keyword.

[0033] In a third aspect, an embodiment of the present application provides an electronic device, comprising: one or more processors; a storage device configured to store one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the data processing method based on the combination model described in the first aspect.

[0034] In a fourth aspect, an embodiment of the present application provides a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to execute the data processing method based on the combination model as described in the first aspect.

[0035] After acquiring text information, the embodiment of the present application performs sensitive information detection on the text information through the first language model to obtain multiple sensitive words; determine the text position of each sensitive word in the text information, determine the context analysis position and the text content corresponding to the context analysis position according to the text position; determine the target sensitive word in each sensitive word by performing context analysis on the text content, filter each target sensitive word in the text information, and obtain security data; report the security data to the server, which is used for the server to perform word segmentation processing on the security data based on the second language model to obtain multiple keywords, and generate response information corresponding to the text information according to multiple keywords and the part of speech of each keyword. The information security of user information can be ensured by performing sensitive information detection on the text through the first language model, and the target sensitive word in multiple sensitive words is determined by combining the context analysis results, which improves the accuracy of identifying sensitive information in complex environments and avoids false positives and false negatives. By processing the security data through the second language model, the accuracy of the response information can be improved, thereby improving the service quality of the smart terminal and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is a flow chart of a data processing method based on a combination model provided in an embodiment of the present application;

[0037] Figure 2 is a flow chart of a method for determining a target text line number provided in an embodiment of the present application;

[0038] Figure 3 It is a schematic diagram for displaying text information provided by an embodiment of the present application;

[0039] Figure 4is another schematic diagram for displaying text information provided in an embodiment of the present application;

[0040] Figure 5 is a flow chart of a method for determining a target sensitive word provided by an embodiment of the present application;

[0041] Figure 6 is a flow chart of a method for generating response information provided by an embodiment of the present application;

[0042] Figure 7 is a structural schematic diagram of a data processing device based on a combination model provided in an embodiment of the present application;

[0043] Figure 8 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical scheme and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for the convenience of description, only the part related to the present application but not all the contents are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow chart describes each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of each operation can be rearranged. The process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The process can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc.

[0045] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0046] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0047] In the following, in conjunction with the accompanying drawings, the data processing method, device, equipment and medium based on the combined model provided in the embodiments of the present application are described in detail through specific embodiments and their application scenarios.

[0048] The data processing method based on the combined model provided in the embodiment of the present application is applied to scenarios where voice or text input by users is intelligently processed, such as intelligent customer service systems and navigation robots that intelligently process information input by users. Based on the above application scenarios, it can be understood that the executor of this solution can be an electronic device with intelligent data processing functions, such as a mobile phone, a laptop computer, a tablet computer, etc.

[0049] Figure 1 is a flow chart of a data processing method based on a combination model provided in an embodiment of the present application, such as Figure 1 As shown, including:

[0050] Step S101: obtain text information, perform sensitive information detection on the text information through a first language model, and obtain multiple sensitive words, where the first language model is a small local model.

[0051] Among them, the text information is the text content used to express the user's needs. For example, if the user wants to know the route to the library, the text information that can be input in the intelligent terminal can be "check the route to the library". The first language model can be a local small language model. The small language model is relative to the large language model, which refers to a model with a smaller scale and a smaller number of parameters. These models can usually achieve relatively good performance at a lower computing cost when processing specific tasks or fields. The design of the small language model is to be more compact and efficient to address the need for feasible AI solutions in resource-constrained environments. Sensitive information detection refers to matching each keyword in the text information with a sensitive word and determining the matched keyword as a sensitive word. Among them, sensitive information refers to personal information that may cause the user's personal dignity to be violated or the personal and property safety to be endangered once it is leaked or illegally used, such as biometric information, religious belief information, specific identity information, medical health information, financial account information, and whereabouts information. In one embodiment, if the information input by the user is text information, the acquired text information is directly subjected to sensitive information detection. If the information input by the user is voice information, the voice information is first converted into text information, and then the text information is subjected to sensitive information detection. The text information is input into the first language model to identify whether there are sensitive words in the text information that match the sensitive information in the first language model, and the sensitive words in the identification result are screened out to obtain a plurality of sensitive words.

[0052] Step S102: determining the text position of each sensitive word in the text information, and determining a context analysis position and text content corresponding to the context analysis position according to the text position.

[0053] The text position is used to indicate the location of the sensitive word in the text information, such as the position of the 4th to 7th characters in the 3rd line. The context analysis position is used to indicate the location of the text content for context analysis. In one embodiment, after determining the text position of each sensitive word in the text information, the behavioral context analysis position of the text position is determined, and the text content of the line is determined to be the text content for context analysis. Exemplarily, if the text position of the sensitive word in the text information is the position of the 4th to 7th characters in the 3rd line, the context analysis position is determined to be the 3rd line, and the content in the 3rd line is determined to be the text content for context analysis.

[0054] Step S103: determining target sensitive words among the sensitive words by performing context analysis on the text content, filtering the target sensitive words in the text information, and obtaining security data.

[0055] Context analysis is a method of understanding and explaining language phenomena by analyzing the environment in which the language is used. It involves analyzing the language and culture produced by the original language under the influence of external factors, aiming to correctly understand the background of the corpus. The target sensitive words are sensitive words that threaten the information security of user information after analyzing the sensitive words. Since the same words may have different meanings in different contexts, the detected sensitive words will not threaten the information security of users in the appropriate context. Therefore, the target sensitive words that only threaten the information security of users are determined according to the results of the context analysis, and the target sensitive words in the text information are filtered to obtain security data.

[0056] Step S104: reporting the security data to a server, for the server to perform word segmentation processing on the security data based on a second language model to obtain a plurality of keywords, and generating response information corresponding to the text information according to the plurality of keywords and the part of speech of each keyword.

[0057] The second language model can be a large language model in the server. Large language models (LLMs) are language models built by deep neural networks with tens of billions of parameters. This model is trained using a large amount of unlabeled text through self-supervised learning methods to solve general language processing problems such as text classification, question answering, document summarization, and text generation. Word segmentation is a process of dividing text into a sequence of strings, and its basic unit is called token or word. This processing has different applications for texts in different languages, especially for languages ​​without explicit marks between words, such as Chinese, Japanese, Vietnamese, etc. For Chinese, word segmentation refers to dividing a sequence of Chinese characters into individual words. Response information is a text message reply information. For example, if the text message is "check the route to the library", the corresponding response information is the specific route from the current location of the smart terminal to the library.

[0058] In one embodiment, the obtained security data is reported to the server, so that the server performs word segmentation processing on the security data based on the second language model to obtain multiple keywords, and performs part-of-speech tagging on each keyword, and determines the information associated with the keyword of the corresponding part-of-speech according to the multiple keywords and the part-of-speech of each keyword, and organizes the information to obtain the response information corresponding to the text information. Among them, the information associated with each keyword is first determined, and it is determined whether the various information obtained are consistent. If they are consistent, the information is determined as the response information corresponding to the text information. If they are inconsistent, the various information are combined to generate the response information corresponding to the text information.

[0059] After obtaining the text information, the text information is detected for sensitive information through the first language model to obtain multiple sensitive words, and the first language model is a local small model; the text position of each sensitive word in the text information is determined, and the context analysis position and the text content corresponding to the context analysis position are determined according to the text position; the target sensitive word in each sensitive word is determined by context analysis of the text content, and each target sensitive word in the text information is filtered to obtain security data; the security data is reported to the server, and the server performs word segmentation processing on the security data based on the second language model to obtain multiple keywords, and generates response information corresponding to the text information according to the multiple keywords and the part of speech of each keyword. The sensitive information detection of the text can be performed by the first language model to ensure the information security of the user information, and the target sensitive word in multiple sensitive words is determined by combining the context analysis results, which improves the accuracy of identifying sensitive information in complex environments and avoids false positives and false negatives. By processing the security data with the second language model, the accuracy of the response information can be improved, thereby improving the service quality of the smart terminal and the user experience.

[0060] In one embodiment, determining the text position of each sensitive word in the text information, and determining the context analysis position and the text content corresponding to the context analysis position according to the text position includes:

[0061] Determine the text paragraph and text line number to which each sensitive word belongs, determine the target text line number for context analysis based on the text paragraph and the text line number, and determine the text content corresponding to the target text line number.

[0062] The target text line number refers to the line number where the text content to be analyzed is located. The text paragraph and text line number to which each sensitive word belongs are determined, and the text line number in the text paragraph is used as the target text line number, and the text content corresponding to the text line number is determined. In another embodiment, after determining the text paragraph and text line number to which each sensitive word belongs, the text line number adjacent to the text line number in the text paragraph is determined as the target text line number, and the text content corresponding to the target text line number is determined.

[0063] In the above, the text paragraph and text line number to which each sensitive word belongs are determined, the target text line number of the context analysis is determined according to the text paragraph and text line number, and the text content corresponding to the target text line number is determined. The specific text content of the context analysis can be determined quickly and accurately, avoiding the influence of the efficiency of the context analysis due to the excessive amount of text information, and ensuring the accuracy of the context analysis.

[0064] Figure 2 is a flow chart of a method for determining a target text line number provided in an embodiment of the present application, such as Figure 2 As shown, including:

[0065] Step S201: Count the number of words in the text paragraph, and determine whether the target text paragraph for context analysis is the same as the text paragraph based on the statistical result.

[0066] Among them, the target text paragraph is a text paragraph that can ensure that there are enough words for context analysis to ensure the accuracy of context analysis. In one embodiment, after determining the text paragraph, the number of words in the text paragraph is counted. If the counted number of words is greater than the preset number of words, the text paragraph is determined to be the target text paragraph, that is, the text paragraph is the same as the target text paragraph. If the counted number of words is less than or equal to the preset number of words, the text paragraph adjacent to the text paragraph is used as the target text paragraph, that is, the text paragraph is different from the target text paragraph.

[0067] Step S202: When the target text paragraph is the same as the text paragraph, determine the text line number and the text line number adjacent to the text line number as the target text line number.

[0068] Figure 3 is a schematic diagram showing a text information provided by an embodiment of the present application, such as Figure 3 As shown, the text paragraph to which the sensitive word belongs is the 3rd paragraph, and the text line number is the 2nd line, that is, the text position to which the sensitive word belongs is the 2nd line of the 3rd paragraph. When the target text paragraph is the same as the text paragraph to which the sensitive word belongs, the text line number is the 2nd line, and the text line numbers adjacent to the text line number are the 1st line and the 3rd line, respectively. In this case, the 1st line, the 2nd line, and the 3rd line are determined as the target text line numbers.

[0069] Step S203: When the target text paragraph is different from the text paragraph, determine all text line numbers contained in the target text paragraph as target text line numbers.

[0070] Figure 4 is another display diagram of text information provided by an embodiment of the present application. The text paragraph to which the sensitive word belongs is the third paragraph, and the text line number to which it belongs is the second line, that is, the text position to which the sensitive word belongs is the second line of the third paragraph. In the case where the target text paragraph is different from the text paragraph to which the sensitive word belongs, the text paragraph adjacent to the text paragraph is used as the target text paragraph, that is, the target text paragraph is the second paragraph or the fourth paragraph, such as Figure 4 As shown, there are 5 lines of text in the second paragraph, so the 1st to 5th lines in the second paragraph are determined as the target text line numbers, and there are 3 lines of text in the fourth paragraph, so the 1st to 3rd lines in the fourth paragraph are determined as the target text line numbers.

[0071] In the above, by counting the number of words in the text paragraph, it is determined whether the target text paragraph of the context analysis is the same as the text paragraph according to the statistical result; when the target text paragraph is the same as the text paragraph, the text line number and the text line number adjacent to the text line number are determined as the target text line number; when the target text paragraph is different from the text paragraph, all the text line numbers contained in the target text paragraph are determined as the target text line number. The accuracy of the context analysis can be improved by ensuring the data volume of the text content, and the appropriate amount of text content can be obtained by accurately determining the location of the text content, thereby avoiding the excessive amount of data for the context analysis, which affects the efficiency of the context analysis.

[0072] Figure 5 is a flow chart of a method for determining a target sensitive word provided by an embodiment of the present application. Figure 5 As shown, including:

[0073] Step S501: Determine the category of each sensitive word and the sensitivity level of the text content by performing context analysis on the text content, and perform sensitivity judgment on each category and the sensitivity level.

[0074] In one embodiment, different words have different meanings in different contexts, and therefore belong to different categories. For example, if one of the sensitive words is "apple", since the sensitive word can represent a kind of fruit, the category of the sensitive word is the fruit category, and can also represent a mobile phone brand, the category of the sensitive word is the mobile phone brand category, and can also represent a company brand, the category of the sensitive word is the company / institution name category. The specific meaning of the sensitive word in the text information and the category to which it belongs can be determined through context analysis, and the sensitivity level of the text content is determined according to the number of sensitive words in the text content. For example, when the number of sensitive words in the text content is greater than or equal to 5, the sensitivity level is determined to be level three, when the number of sensitive words in the text content is greater than or equal to 3 and less than 5, the sensitivity level is determined to be level two, and when the number of sensitive words in the text content is less than 3, the sensitivity level is determined to be level one. The sensitivity of the text content is determined according to the category to which each sensitive word belongs and the sensitivity level of the text content.

[0075] Step S502: When the sensitivity level is greater than a preset sensitivity level and there is a sensitive category in each category, determine the sensitive word corresponding to the sensitive category as a target sensitive word.

[0076] Among them, the sensitive category is used to indicate that a category of keywords are all categories that pose a threat to the user's information security, such as passwords, organization / company names, etc. For example, if the sensitive word "apple" is a fruit name category, it does not belong to the sensitive category. If the sensitive word "apple" is an organization / company name category, it belongs to the sensitive category. In one embodiment, when the sensitivity level is greater than level 2 and there is a sensitive category in the sensitive category to which each sensitive word belongs, the sensitive word corresponding to the sensitive category is determined as the target sensitive word.

[0077] In a possible embodiment, when the sensitivity level is lower than the preset sensitivity level and there are sensitive categories in each category, it is analyzed whether the sensitive words in the text content can pose a threat to the information security of the text information. If it can affect the security of the text information, the sensitive words corresponding to the sensitive category are determined as target sensitive words.

[0078] In the above, the category of each sensitive word and the sensitivity level of the text content are determined by contextual analysis of the text content, and sensitivity judgment is made on each category and sensitivity level; when the sensitivity level is greater than the preset sensitivity level and there is a sensitive category in each category, the sensitive word corresponding to the sensitive category is determined as the target sensitive word. The accuracy of determining the target sensitive word can be improved by judging the overall sensitivity of the content and the sensitive category.

[0079] Figure 6 is a flow chart of a method for generating response information provided by an embodiment of the present application, such as Figure 6 As shown, including:

[0080] Step S601: obtain text information, perform sensitive information detection on the text information through a first language model, and obtain multiple sensitive words, where the first language model is a small local model.

[0081] Step S602: Determine the text position of each sensitive word in the text information, and determine the context analysis position and the text content corresponding to the context analysis position according to the text position.

[0082] Step S603: determining target sensitive words among the sensitive words by performing context analysis on the text content, filtering the target sensitive words in the text information, and obtaining security data.

[0083] Step S604: reporting the security data to a server, for the server to perform word segmentation processing on the security data based on a second language model to obtain a plurality of keywords, determining a response type of the text information and a response database corresponding to the response type according to the plurality of keywords and the part of speech of each of the keywords, and generating response information corresponding to the text information according to the response database.

[0084] In one embodiment, after the security data is segmented to obtain multiple keywords, the keywords are tagged with parts of speech. This tagging process not only helps to analyze the grammatical structure of the text, but also reveals how words interact with each other and the structure of sentences, thereby deepening the understanding of the semantic meaning of the text and accurately determining the response type based on the understanding of the semantic meaning of the text. Among them, the response type can include the answer type, navigation type, etc. Each response type corresponds to a database, such as the answer type corresponds to the answer question database, and the navigation type corresponds to the navigation database. The response information corresponding to each keyword is searched in the database corresponding to each keyword.

[0085] In the above, based on multiple keywords and each of the keywords, the response type of the text information and the response database corresponding to the response type are determined, and the response information corresponding to the text information is generated according to the response database. By tagging the keywords with parts of speech, the understanding of the text information can be deepened, thereby improving the accuracy and efficiency of generating the response information.

[0086] In one embodiment, determining the response type of the text information and the response database corresponding to the response type according to the multiple keywords and the part of speech of each keyword includes:

[0087] Determining a sentence type of the text information according to the multiple keywords and the part of speech of each of the keywords, and when the sentence type is a question type, determining a response type corresponding to the text information as a reply type, and determining a response database corresponding to the reply type as a reply database;

[0088] In the case where the sentence type is a declarative sentence type, the response type corresponding to the text information is determined to be a translation type, and the response database corresponding to the translation type is determined to be a translation database.

[0089] Among them, the text information is determined to be a sentence type based on multiple keywords and the part of speech of each keyword, and the sentence type may include a rhetorical question type, an exclamatory sentence type, etc. The response type is used to indicate the feedback type for user needs, and the response type may be a navigation type, etc. Optionally, the sentence type is a question type and a declarative sentence type, and the response type is a reply type and a translation type. In one embodiment, the sentence type of the text information is determined based on multiple keywords and the part of speech of each keyword. When the sentence type is a question type, the response type corresponding to the text information is determined to be a reply type, and the response database corresponding to the reply type is determined to be a reply database, so as to search for the response information corresponding to each keyword in the reply database. When the sentence type is a declarative sentence type, the response type corresponding to the text information is determined to be a translation type, and the response database corresponding to the translation type is determined to be a translation database, so as to search for the translation content corresponding to each keyword in the translation database.

[0090] In the above, the sentence type of the text information is determined based on multiple keywords and the part of speech of each keyword. When the sentence type is a question sentence type, the response type corresponding to the text information is determined to be a reply type, and the response database corresponding to the reply type is determined to be a reply database; when the sentence type is a declarative sentence type, the response type corresponding to the text information is determined to be a translation type, and the response database corresponding to the translation type is determined to be a translation database. The ability to set up corresponding types of databases for different sentence types improves data query efficiency, and correspondingly improves the efficiency of generating response information.

[0091] In one embodiment, determining the sentence type of the text information according to the multiple keywords and the part of speech of each keyword includes:

[0092] Identify whether there are question words or language identifiers among the multiple keywords. If there are question words or language identifiers among the multiple keywords, determine the text meaning of the text information based on the multiple keywords and the part of speech of each of the keywords, and determine the sentence type of the text information based on the text meaning.

[0093] Among them, interrogative words are words used to construct interrogative sentences, mainly including interrogative pronouns, interrogative particles and interrogative adverbs. These words are used to ask questions in language to guide listeners or readers to answer. Interrogative words can be "what", "where", "how" and "how". Language identifiers are words used to indicate language types, such as Chinese, English, French and Spanish. In one embodiment, it is identified whether there are interrogative words or language identifiers in multiple keywords. When interrogative words or language identifiers exist, the text meaning of the text information is determined according to the multiple keywords and the part of speech of each keyword, and the sentence type of the text information is determined according to the text meaning.

[0094] In the above, it is identified whether there are interrogative words or language identifiers in multiple keywords. When there are interrogative words or language identifiers in multiple keywords, the text meaning of the text information is determined according to the multiple keywords and the part of speech of each keyword, and the sentence type of the text information is determined according to the text meaning. It is possible to first determine whether there are interrogative words or language identifiers in multiple keywords to make a preliminary judgment on the text information input by the user, determine whether it meets the response conditions of the intelligent terminal device, and search for the response information corresponding to each keyword in the corresponding database when the response conditions are met, so as to avoid excessive load, thereby affecting the query efficiency of the response information.

[0095] Figure 7 is a structural diagram of a data processing device based on a combination model provided in an embodiment of the present application, such as Figure 7 As shown, including:

[0096] A sensitive information detection module 71 is used to obtain text information, perform sensitive information detection on the text information through a first language model, and obtain a plurality of sensitive words, where the first language model is a small local model;

[0097] A text position determination module 72, used to determine the text position of each sensitive word in the text information;

[0098] A text content determination module 73, used to determine a context analysis position and text content corresponding to the context analysis position according to the text position;

[0099] A target sensitive word determination module 74 is used to determine a target sensitive word among the sensitive words by performing context analysis on the text content;

[0100] A sensitive word filtering module 75 is used to filter each target sensitive word in the text information to obtain safe data;

[0101] The data reporting module 76 is used to report the security data to the server, and the server performs word segmentation processing on the security data based on the second language model to obtain multiple keywords, and generates response information corresponding to the text information according to the multiple keywords and the part of speech of each keyword.

[0102] After acquiring text information, the embodiment of the present application performs sensitive information detection on the text information through the first language model to obtain multiple sensitive words; determine the text position of each sensitive word in the text information, determine the context analysis position and the text content corresponding to the context analysis position according to the text position; determine the target sensitive word in each sensitive word by performing context analysis on the text content, filter each target sensitive word in the text information, and obtain security data; report the security data to the server, which is used for the server to perform word segmentation processing on the security data based on the second language model to obtain multiple keywords, and generate response information corresponding to the text information according to multiple keywords and the part of speech of each keyword. The information security of user information can be ensured by performing sensitive information detection on the text through the first language model, and the target sensitive word in multiple sensitive words is determined by combining the context analysis results, which improves the accuracy of identifying sensitive information in complex environments and avoids false positives and false negatives. By processing the security data through the second language model, the accuracy of the response information can be improved, thereby improving the service quality of the smart terminal and improving the user experience.

[0103] In a possible embodiment, the text content determination module 73 is used to:

[0104] Determine the text paragraph and text line number to which each sensitive word belongs, determine the target text line number for context analysis based on the text paragraph and the text line number, and determine the text content corresponding to the target text line number

[0105] In a possible embodiment, the text content determination module 73 is used to:

[0106] Perform word count on the text paragraph, and determine whether the target text paragraph for context analysis is the same as the text paragraph based on the statistical result;

[0107] In the case where the target text paragraph is the same as the text paragraph, determining the text line number and the text line number adjacent to the text line number as the target text line number;

[0108] In the case that the target text paragraph is different from the text paragraph, all text line numbers contained in the target text paragraph are determined as target text line numbers.

[0109] In a possible embodiment, the target sensitive word determination module 74 is used to:

[0110] Determine the category of each sensitive word and the sensitivity level of the text content by performing contextual analysis on the text content, and make sensitivity judgments on each category and the sensitivity level;

[0111] When the sensitivity level is greater than the preset sensitivity level and there is a sensitive category in each category, the sensitive word corresponding to the sensitive category is determined as the target sensitive word.

[0112] In a possible embodiment, the response type of the text information and a response database corresponding to the response type are determined according to the multiple keywords and the part of speech of each keyword, and the response information corresponding to the text information is generated according to the response database.

[0113] In a possible embodiment, the sentence type of the text information is determined according to the multiple keywords and the part of speech of each of the keywords, and when the sentence type is a question type, the response type corresponding to the text information is determined to be a reply type, and the response database corresponding to the reply type is determined to be a reply database;

[0114] In the case where the sentence type is a declarative sentence type, the response type corresponding to the text information is determined to be a translation type, and the response database corresponding to the translation type is determined to be a translation database.

[0115] In a possible embodiment, it is identified whether there are question words or language identifiers among the multiple keywords. If question words or language identifiers are present among the multiple keywords, the text meaning of the text information is determined based on the multiple keywords and the part of speech of each of the keywords, and the sentence type of the text information is determined based on the text meaning.

[0116] An embodiment of the present application also provides an electronic device, and the data processing device based on the combination model can integrate a data processing device based on the combination model provided in an embodiment of the present application. Figure 8 Schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Figure 8 The data processing device based on the combined model includes: an input device 83, an output device 84, a memory 82 and one or more processors 81; the memory 82 is used to store one or more programs; when one or more programs are executed by one or more processors 81, the one or more processors 81 implement the data processing method based on the combined model provided in the above embodiment. The input device 83, the output device 84, the memory 82 and the processor 81 can be connected by a bus or other means. Figure 8 The example of connecting through bus is taken in the following.

[0117] The memory 82, as a computing device readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the data processing method based on the combined model provided in any embodiment of the present application. The memory 82 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the device, etc. In addition, the memory 82 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 82 may further include a memory remotely arranged relative to the processor 81, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0118] The input device 83 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the device. The output device 84 may include a display device such as a display screen.

[0119] The processor 81 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 82, that is, implements the above-mentioned data processing method based on the combination model.

[0120] The above-mentioned data processing apparatus, device and computer based on the combination model can be used to execute the data processing method based on the combination model provided in any of the above-mentioned embodiments, and have corresponding functions and beneficial effects.

[0121] The present application also provides a storage medium storing computer executable instructions. When the computer executable instructions are executed by a computer processor, they are used to execute the data processing method based on the combination model provided in the above embodiment. The data processing method based on the combination model includes:

[0122] Acquire text information, and perform sensitive information detection on the text information through a first language model to obtain a plurality of sensitive words, where the first language model is a small local model;

[0123] Determine the text position of each sensitive word in the text information, and determine the context analysis position and the text content corresponding to the context analysis position according to the text position;

[0124] Determine a target sensitive word among the sensitive words by performing context analysis on the text content, and filter the target sensitive words in the text information to obtain security data;

[0125] The security data is reported to a server, and the server performs word segmentation processing on the security data based on a second language model to obtain a plurality of keywords, and generates response information corresponding to the text information according to the plurality of keywords and the part of speech of each keyword.

[0126] Storage medium - any of various types of memory devices or storage devices. The term "storage medium" is intended to include: installation media, such as CD-ROM, floppy disk or tape device; computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory, such as flash memory, magnetic media (such as hard disk or optical storage); registers or other similar types of memory elements, etc. Storage media may also include other types of memory or combinations thereof. In addition, the storage medium may be located in the first computer system in which the program is executed, or may be located in a different second computer system, which is connected to the first computer system via a network (such as the Internet). The second computer system can provide program instructions to the first computer for execution. The term "storage medium" may include two or more storage media that may reside in different locations (for example, in different computer systems connected by a network). The storage medium may store program instructions (for example, embodied as a computer program) that can be executed by one or more processors.

[0127] Of course, the storage medium containing computer executable instructions provided in an embodiment of the present application, whose computer executable instructions are not limited to the data processing method based on the combination model as described above, can also execute related operations in the data processing method based on the combination model provided in any embodiment of the present application.

[0128] The data processing apparatus, device and storage medium based on the combination model provided in the above embodiments can execute the data processing method based on the combination model provided in any embodiment of the present application. For technical details not described in detail in the above embodiments, please refer to the data processing method based on the combination model provided in any embodiment of the present application.

[0129] The above are only preferred embodiments of the present application and the technical principles used. The present application is not limited to the specific embodiments described herein, and various obvious changes, readjustments and substitutions that can be made by those skilled in the art will not deviate from the scope of protection of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the claims.

Claims

1. A data processing method based on a combination model, characterized in that: include: Acquire text information, and perform sensitive information detection on the text information through a first language model to obtain a plurality of sensitive words, where the first language model is a small local model; Determine the text position of each sensitive word in the text information, and determine the context analysis position and the text content corresponding to the context analysis position according to the text position; Determine a target sensitive word among the sensitive words by performing context analysis on the text content, and filter the target sensitive words in the text information to obtain security data; The security data is reported to a server, and the server performs word segmentation processing on the security data based on a second language model to obtain a plurality of keywords, and generates response information corresponding to the text information according to the plurality of keywords and the part of speech of each keyword.

2. The data processing method based on the combined model according to claim 1, characterized in that: The step of determining the text position of each sensitive word in the text information, and determining the context analysis position and the text content corresponding to the context analysis position according to the text position includes: Determine the text paragraph and text line number to which each sensitive word belongs, determine the target text line number for context analysis based on the text paragraph and the text line number, and determine the text content corresponding to the target text line number.

3. The data processing method based on the combined model according to claim 2, characterized in that: The step of determining the target text line number for context analysis according to the text paragraph and the text line number includes: Perform word count on the text paragraph, and determine whether the target text paragraph for context analysis is the same as the text paragraph based on the statistical result; In the case where the target text paragraph is the same as the text paragraph, determining the text line number and the text line number adjacent to the text line number as the target text line number; In the case that the target text paragraph is different from the text paragraph, all text line numbers contained in the target text paragraph are determined as target text line numbers.

4. The data processing method based on the combined model according to claim 1, characterized in that: The determining a target sensitive word among the sensitive words by performing context analysis on the text content includes: Determine the category of each sensitive word and the sensitivity level of the text content by performing contextual analysis on the text content, and make sensitivity judgments on each category and the sensitivity level; When the sensitivity level is greater than the preset sensitivity level and there is a sensitive category in each category, the sensitive word corresponding to the sensitive category is determined as the target sensitive word.

5. The data processing method based on the combined model according to claim 1, characterized in that: The step of generating response information corresponding to the text information according to the multiple keywords and the part of speech of each keyword includes: The response type of the text information and a response database corresponding to the response type are determined according to the multiple keywords and the part of speech of each keyword, and the response information corresponding to the text information is generated according to the response database.

6. The data processing method based on the combined model according to claim 5 is characterized in that: The step of determining the response type of the text information and the response database corresponding to the response type according to the multiple keywords and the part of speech of each keyword includes: Determining a sentence type of the text information according to the multiple keywords and the part of speech of each of the keywords, and when the sentence type is a question type, determining a response type corresponding to the text information as a reply type, and determining a response database corresponding to the reply type as a reply database; In the case where the sentence type is a declarative sentence type, the response type corresponding to the text information is determined to be a translation type, and the response database corresponding to the translation type is determined to be a translation database.

7. The data processing method based on the combined model according to claim 6 is characterized in that: The determining the sentence type of the text information according to the multiple keywords and the part of speech of each keyword includes: Identify whether there are question words or language identifiers among the multiple keywords. If there are question words or language identifiers among the multiple keywords, determine the text meaning of the text information based on the multiple keywords and the part of speech of each of the keywords, and determine the sentence type of the text information based on the text meaning.

8. A data processing device based on a combination model, characterized in that: include: A sensitive information detection module, used to obtain text information, perform sensitive information detection on the text information through a first language model, and obtain a plurality of sensitive words, wherein the first language model is a small local model; A text position determination module, used to determine the text position of each sensitive word in the text information; A text content determination module, used to determine a context analysis position and text content corresponding to the context analysis position according to the text position; A target sensitive word determination module, configured to determine a target sensitive word among the sensitive words by performing context analysis on the text content; A sensitive word filtering module, used to filter each target sensitive word in the text information to obtain safe data; A data reporting module is used to report the security data to a server, and the server is used to perform word segmentation processing on the security data based on a second language model to obtain multiple keywords, and generate response information corresponding to the text information according to the multiple keywords and the part of speech of each keyword.

9. An electronic device, comprising: one or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, enables the one or more processors to implement the data processing method based on the combination model as described in any one of claims 1-7.

10. A storage medium storing computer executable instructions, wherein the computer executable instructions are used to execute the data processing method based on the combination model as described in any one of claims 1 to 7 when executed by a computer processor.

Images