Intelligent device and effective semantic word extraction method

Abstract

The present application provides an intelligent device and a method for extracting effective semantic words, the intelligent device includes a storage module and a processing module, the processing module is configured to obtain a text to be extracted; segment the text to be extracted to obtain a word set, the word set includes multiple semantic words; replace the semantic words in the word set with a universal tag to generate an annotated text set; input the text to be extracted and the annotated text set into a semantic extraction model; obtain the semantic similarity output by the semantic extraction model, and filter the semantic words in the word set according to a similarity threshold to obtain effective semantic words. The present application can extract key semantic words that affect semantic understanding from the user's query text, help the search engine better understand the user's intention, and thus help the intelligent device give accurate search results and improve the user experience.

Description

Technical Field

[0001] The present application relates to the technical field of natural language processing, and in particular to an intelligent device and an effective semantic word extraction method. Background Art

[0002] Natural language processing is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that can achieve effective communication between people and computers using natural language. Natural language processing is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field will involve natural language, that is, the language used by people in daily life, so it is closely related to the study of linguistics, but there are important differences. Natural language processing is not a general study of natural language, but the development of computer systems that can effectively realize natural language communication, especially the software systems therein. Therefore, it is a part of computer science. Natural language processing is mainly used in machine translation, public opinion monitoring, automatic summarization, opinion extraction, text classification, question answering, text semantic comparison, speech recognition, Chinese OCR, etc.

[0003] In the field of smart device consumption, consumers' query statements tend to be colloquial, and the query statements contain more personal habits and regional habits. For example, Shandong people like to use inverted sentences to query. The current understanding of consumer intentions is generally directly analyzed based on the user's query statements. Some manufacturers also use classification algorithms that integrate business parts of speech, but when there are many meaningless words or polysemous words in the user's query statements, this classification method often misclassifies. In the field of smart device search, due to the diversification and networking of each search target title, the title content is full of messy self-constructed words, exaggerated words and other meaningless words, which seriously affect the accuracy and efficiency of the search. The more popular method currently used in the field of smart device search is to directly match the user query and the search target title, and then give some of the highest-scoring content. The main reason is that on the one hand, it is impossible to effectively analyze the meaningful words in the user query or title, and on the other hand, it is difficult to collect training data with a wide coverage. Summary of the invention

[0004] The present application provides a smart device and an effective semantic word extraction method to solve the problem that existing smart device search methods cannot effectively analyze meaningful words in user queries or titles.

[0005] In one aspect, the present application provides a smart device, comprising:

[0006] A storage module, configured to store the semantic extraction model;

[0007] The processing module is configured as follows:

[0008] Get the text to be extracted;

[0009] Segmenting the text to be extracted to obtain a word set, wherein the word set includes a plurality of semantic words;

[0010] Replacing semantic words in the word set with universal tags to generate an annotated text set;

[0011] The text to be extracted and the annotated text set are input into a semantic extraction model, wherein the semantic extraction model is generated by training a training sample set and an annotated sample set, wherein the training sample set includes training sentences with semantic labels; the annotated sample set includes annotated sample sentences with annotated probabilities, and the annotated sample sentences are sentences formed by replacing keywords in the training sentences with general tags;

[0012] The semantic similarity output by the semantic extraction model is obtained, and the semantic words in the word set are filtered according to a similarity threshold to obtain valid semantic words.

[0013] On the other hand, the present application also provides an effective semantic word extraction method, comprising the following steps:

[0014] Get the text to be extracted;

[0015] Segmenting the text to be extracted to obtain a word set, wherein the word set includes a plurality of semantic words;

[0016] Replacing semantic words in the word set with universal tags to generate an annotated text set;

[0017] The text to be extracted and the annotated text set are input into a semantic extraction model, wherein the semantic extraction model is generated by training a training sample set and an annotated sample set, wherein the training sample set includes training sentences with semantic labels; the annotated sample set includes annotated sample sentences with annotated probabilities, and the annotated sample sentences are sentences formed by replacing keywords in the training sentences with general tags;

[0018] The semantic similarity output by the semantic extraction model is obtained, and the semantic words in the word set are filtered according to a similarity threshold to obtain valid semantic words.

[0019] It can be seen from the above technical scheme that the present application provides an intelligent device and an effective semantic word extraction method, the intelligent device includes a storage module and a processing module, the processing module is configured to obtain a text to be extracted; segment the text to be extracted to obtain a word set, the word set includes multiple semantic words; replace the semantic words in the word set with a universal tag to generate an annotated text set; input the text to be extracted and the annotated text set into a semantic extraction model, wherein the semantic extraction model is generated by training a training sample set and an annotated sample set, the training sample set includes training sentences with semantic labels; the annotated sample set includes annotated sample sentences with annotated probabilities, the annotated sample sentences are sentences formed by replacing the keywords in the training sentences with universal tags; obtain the semantic similarity output by the semantic extraction model, and filter the semantic words in the word set according to the similarity threshold to obtain effective semantic words. The present application can extract key semantic words that affect semantic understanding from the user's query text, help search engines better understand user intent, and thus help intelligent devices give accurate search results and improve user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0021] Figure 1 This is a schematic diagram of the processing flow of the smart device in the embodiment of this application;

[0022] Figure 2 This is a schematic diagram of the model training process in the embodiment of this application;

[0023] Figure 3 This is a schematic diagram of a semantic extraction model in an embodiment of the present application;

[0024] Figure 4 A schematic diagram of the process of obtaining the text to be extracted in the embodiment of the present application;

[0025] Figure 5 This is a schematic diagram of the word segmentation process in the embodiment of this application;

[0026] Figure 6 A schematic diagram of the process of generating annotated text sets in an embodiment of the present application;

[0027] Figure 7 A schematic diagram of the process of obtaining label classification results in an embodiment of the present application;

[0028] Figure 8 This is a schematic diagram of the threshold filtering process in an embodiment of the present application. DETAILED DESCRIPTION

[0029] The following embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following embodiments do not represent all implementations consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application as detailed in the claims.

[0030] In the embodiment of the present application, the semantic extraction method can be applied to smart devices with data processing functions and semantic extraction requirements. The smart devices include but are not limited to: computers, smart terminals, smart TVs, smart wearable devices, smart display devices, servers, etc. The smart devices can have a built-in or external storage module and provide a processing module to form a text classification system capable of executing the text classification method.

[0031] In the embodiment of the present application, the smart device may be a smart TV device, which has a built-in memory and a controller, wherein the memory may be used to store data such as text, natural language processing models, and control programs. The controller may call data from the memory and process the called data by running the control program.

[0032] The application scenario of this application is the field of smart device search. It can extract key semantic words that affect semantic understanding from the user's query text, help the search engine better understand the user's intention, and thus help the smart device give accurate answers and improve the user experience. On the other hand, since the titles of the current search target data are various, the extraction of effective semantic words can more easily help the retrieval system retrieve the content that the user is interested in.

[0033] like Figure 1 The figure shows a schematic diagram of the processing flow of the intelligent device in the embodiment of the present application. In the embodiment of the present application, natural language processing may include two stages, namely, the model training stage and the semantic extraction stage. In the model training stage, the processing module can obtain training sample data from the network or other channels, and then input the training sample data into the initial semantic model for model training. The training model can output the classification probability according to the input sample data. Then, the loss value is calculated based on the classification probability and the labeling probability, and the model parameters of the initial semantic model are adjusted according to the loss value to obtain the semantic extraction model. In this way, the parameters of the training model are repeatedly tuned after a certain volume of training sample data, so as to obtain a semantic extraction model with better training effect. After completing the model training process, the processing module stores the semantic extraction model obtained through training in the storage module for subsequent applications to call.

[0034] In the embodiment of the present application, the semantic extraction phase processing module can call the semantic extraction model obtained through training in the storage module, and input the text data that needs to be semantically extracted into the called semantic extraction model. Then, through the internal operation of the semantic extraction model, each semantic word extracted from the current text data can be obtained, so as to perform related queries.

[0035] In an embodiment of the present application, the training model or semantic extraction model described in the above embodiment can be a model based on natural language processing (Natural Language Processing, NLP). For example, BERT model and other NLP models obtained by optimizing or modifying based on BERT model. It should be noted that the model described in the above embodiment can be called a training model in the model training stage, and the model in the semantic extraction stage is called a semantic extraction model. Since the training model and the semantic extraction model are only different stages of a model, in the semantic extraction process, the training model and the semantic extraction model are both input with text data, so the text data that can be input into the training model can also be input into the semantic extraction model, that is, in any of the embodiments described below, unless otherwise specified, the training model and the semantic extraction model are no longer distinguished, and the semantic extraction method can be applied to the processing of text data in the model training stage, and can also be applied to the processing of text data in the semantic extraction stage.

[0036] In the embodiment of the present application, the above-mentioned semantic extraction process can set different annotation probabilities so that the intelligent device can determine the semantic similarity of semantic words in text data according to the annotation probability, thereby determining the semantic words in the text data. That is, the semantic extraction process can determine the machine language from the natural language text data and realize machine learning. It can be seen that the semantic extraction process can be applied to fields related to natural language processing, such as intelligent voice control, intelligent question and answer, image recognition processing, business statistical analysis, etc.

[0037] In an embodiment of the present application, in order to realize the above-mentioned semantic extraction process, the smart device can perform model training and semantic extraction by building an artificial intelligence (AI) algorithm in the operating system. Obviously, for smart devices with different purposes, the artificial intelligence algorithms built into their operating systems are different due to the different functions they implement, but they are essentially all for realizing the semantic extraction process. For example, the AI ​​algorithm built into the intelligent question-and-answer robot performs semantic extraction on the text input by the user's voice to provide relevant answers. In the intelligent search system built into the smart TV, semantic extraction is performed on the text input by the user's voice or typing to search for related programs.

[0038] In an embodiment of the present application, in addition to building the AI ​​algorithm into the operating system, the AI ​​algorithm corresponding to the semantic extraction function can also be built into the application. That is, the smart device can also implement the semantic extraction function by installing the application. The application that can implement the semantic extraction function can be a system application or a third-party application. For example, in order to implement the intelligent question and answer function, the computer can download and install the "intelligent question and answer robot" application, and call the semantic extraction model by running the application, and then obtain the text data input by the user in real time, and input the text data into the semantic extraction model to implement the semantic extraction function of the text data.

[0039] In the embodiments of the present application, the semantic extraction function is not limited to the operation of one smart device, but can also be achieved through the collaborative operation of multiple devices, that is, the smart device can establish a communication connection with the server, and in actual applications, the smart device can obtain the user's input text in real time, the server executes model training, semantic extraction process and query process, and then the smart device displays the query results.

[0040] In an embodiment of the present application, the smart device can obtain text data input by the user in real time during operation and send the text data to the server. The server has a built-in AI algorithm and a semantic extraction model that implements the semantic extraction function. After receiving the text data sent by the smart device, the server can input the text data into the semantic extraction model to obtain the semantic extraction result output by the model. The server then feeds back the semantic extraction result to the smart device so that the semantic extraction result and related query information can be fed back to the user through the smart device.

[0041] In an embodiment of the present application, in order to realize more business needs and reduce the amount of data processing, in actual applications, the specific device data for realizing the semantic extraction function through the collaborative operation of multiple devices can be flexibly set according to the requirements of the functions to be realized. And the specific semantic extraction process can be flexibly set according to the device hardware configuration and data volume requirements to reduce repeated data processing processes and save the computing power of the device. For example, multiple smart devices can establish a communication connection with the server at the same time. Among them, the server is used to uniformly provide a semantic extraction model for multiple smart devices. After obtaining the semantic extraction model, different smart devices can perform data input, model calculation, result output and other procedures by themselves to realize the semantic extraction function. At the same time, the smart device can also report its processed text data to the server to further perform model training in the server and continuously improve the semantic extraction model. Therefore, the server can push the semantic extraction model to the smart device at a predetermined time, and update the semantic extraction model in each smart device to maintain its timeliness.

[0042] In an embodiment of the present application, when the semantic extraction function is realized by the collaborative operation of multiple devices, the computing load status of each device can also be monitored in real time, and the actual execution subject of the model training phase and / or the semantic extraction phase can be dynamically adjusted according to the real-time load status. The application used to realize semantic extraction can be a network application. Then, the smart devices and servers connected to the same network can realize the semantic extraction function by running the network application after installing the network application. In the process of running the network application, the network application can monitor the computing load of each smart device and server in real time, including CPU usage, memory usage, network delay, etc. When the data corresponding to any computing load is abnormal, the AI ​​algorithm execution subject of the corresponding device can be adjusted in real time to enable the semantic extraction process to run smoothly.

[0043] In an embodiment of the present application, under normal conditions, the model calculation in the semantic extraction process can be performed by the smart device. When it is monitored that the memory usage of the smart device exceeds a threshold, the process of the smart device executing the model calculation can be paused, and the smart device can be automatically controlled to send the acquired text data to the server, so that the model calculation is executed through the server, and the semantic extraction results are fed back to reduce the processing load of the smart device and improve the timeliness of the semantic extraction.

[0044] It can be seen from the above implementation that in the process of applying semantic extraction, the smart device or server needs to input text data into the training model (or semantic extraction model) in the model training stage and the semantic extraction stage. Since the text data to be extracted is natural language text data, it has different text forms depending on the source of the text data. For example, for the text data generated by the voice information input by the user, its content tends to be colloquial according to the user's voice input process, and the actual text length is relatively low, generally only the length of a sentence. For the input text, the text length is generally higher.

[0045] like Figure 2 The figure shows a schematic diagram of the model training process in an embodiment of the present application. In the embodiment of the present application, the processing module is configured to obtain a training sample set, wherein the training sample set is constructed by a query text and a media asset title text input by a user; extract training sentences from the training sample set; use a universal tag to replace keywords in the training sentences to generate the annotated sample sentences; set an annotation probability for the annotated sample sentences; and use the training sentences and the annotated sample sentences to train the semantic extraction model.

[0046] In an embodiment of the present application, the processing module can collect training data from the text input by the user and the title text in the media resource library in the server. There is no restriction on the way the user inputs the text. The user can input the text to be searched by voice or by typing. The media resource title text in the server is the title text of the user's final search target. Combining the two as training data can improve the comprehensiveness of model training. The processing module can extract sample sentences for training from the training sample set. The keywords in the training sentence are replaced with a universal tag to generate the annotated sample sentence. There is no restriction on the style of the universal tag, which can be a single character or a character group, as long as it can replace the semantic word. A labeling probability is set for the annotated sample sentence, and the labeling probability is obtained by labeling in the form of a soft label. The semantic extraction model is trained using the training sentence and the annotated sample sentence.

[0047] In an embodiment of the present application, the universal tag replaces a keyword in the training sentence, and the other contents in the training sentence remain unchanged. A labeled sample sentence is generated, and a labeling probability is set for each labeled sample sentence. The set of the training sentence, the labeled sample sentence and the labeling probability is the training sample of the semantic extraction model.

[0048] In the embodiment of the present application, the annotation form of the soft label can very accurately express the similarity between the original text and the replaced annotated text. The closer the annotation probability is to 1, the more similar the original text is to the replaced annotated text. The closer the annotation probability is to 0, the less similar the original text is to the replaced annotated text. The semantic extraction model annotates the annotation probability in the form of soft labels with a certain gradient, that is, the importance of different words is distinguished by using standard labels with a certain gradient.

[0049] In an embodiment of the present application, the processing module is configured to call the initial semantic model in the step of training the semantic extraction model using the training sentences and the annotated sample sentences; input the training sentences and the annotated sample sentences into the initial semantic model; obtain the classification probability output by the initial semantic model; calculate the loss value based on the classification probability and the annotated probability; and adjust the model parameters of the initial semantic model based on the loss value to obtain the semantic extraction model.

[0050] like Figure 3The figure shows a schematic diagram of a semantic extraction model in an embodiment of the present application. In the embodiment of the present application, the initial semantic model is a Query2Query interactive matching model, Query1 is the training sentence, Query2 is the labeled sample sentence, sim_prob_new is the labeling probability, [prob0, prob1] is the classification probability, [labels_0, labels_1] is the label classification result of the training sentence and the labeled sample sentence, Loss is the loss value, and the training sentence and the labeled sample sentence are input into the initial semantic model to calculate the classification probability [prob0, prob1], where:

[0051] The calculation method of the loss value Loss is:

[0052] Loss=labels_g-[prob0,prob1];

[0053] Among them, the calculation method of labels_g is:

[0054] labels_g=[labels_0,labels_1]*[1-sim_prob_new,sim_prob_new];

[0055] The calculation method of the label classification result [labels_0, labels_1] is:

[0056]

[0057] The model parameters of the initial semantic model are adjusted according to the calculated loss value to obtain the semantic extraction model, and the original semantic word extraction task is converted into a task of calculating semantic similarity using the semantic similarity calculation method, thereby effectively solving the problem that the same word has different importance in different texts.

[0058] In an embodiment of the present application, the semantic extraction model effectively incorporates label information annotated with a certain gradient when calculating the loss value loss, that is, the dynamic perception of text similarity is effectively incorporated into the entire training process.

[0059] like Figure 4 The figure shows a schematic diagram of the process of obtaining the text to be extracted in an embodiment of the present application. In the embodiment of the present application, the processing module is configured to receive a query instruction input by a user in the step of obtaining the text to be extracted, wherein the query instruction includes a query text; parse the query text from the query instruction; and use a preset vocabulary to delete meaningless words in the query text to obtain the text to be extracted.

[0060] In an embodiment of the present application, when a user uses a smart device to search, he or she directly enters a voice query instruction through the voice input function of the smart device. The processing module of the smart device parses the query text from the voice query instruction. The query text may include many meaningless modal particles, such as: "Then check actor A's movie 1" and "actor B's movie 2". The processing module uses a preset vocabulary to delete the meaningless words "then" and "this" in the query text to obtain the text to be extracted, namely, "check actor A's movie 1" and "actor B's movie 2".

[0061] In an embodiment of the present application, when a user uses a smart device to search, he or she inputs a text query instruction through the text input function of the smart device. The processing module of the smart device parses the query text from the text query instruction. The query text may include many meaningless modal particles and punctuation marks, such as: "Check actor A's movie 1, ha" and "actor B's movie 2, ah!". The processing module uses a preset vocabulary to delete the meaningless words "ha", "ah" and "!" in the query text to obtain the text to be extracted, namely "Check actor A's movie 1, "actor B's movie 2".

[0062] In an embodiment of the present application, the preset vocabulary is a self-built library. There may be other meaningless modal particles, punctuation marks and special symbols in the query instructions entered by the user when searching using a smart device. All meaningless modal particles, punctuation marks and special symbols can be deleted through the preset vocabulary; the preset vocabulary can be added, modified and deleted at any time, so that the preset vocabulary can better handle meaningless words in the query text.

[0063] In an embodiment of the present application, some repeated word segments with extremely low semantic similarity rankings and extremely low semantic similarity scores may be obtained during the model training process. These repeated meaningless word segments can be added to the preset vocabulary to further improve the preset vocabulary.

[0064] like Figure 5 The figure shows a schematic diagram of the word segmentation process in an embodiment of the present application. In the embodiment of the present application, the processing module is configured to call a word segmentation tool in the step of segmenting the text to be extracted; the text to be extracted is input into the word segmentation tool to divide the text to be extracted into multiple semantic words to form the word set.

[0065] In the embodiment of the present application, there is no restriction on the word segmentation tool, and the text to be extracted, such as "Here is the crossroads of Europe! Explore the old town of Tallinn, Estonia and appreciate the intact medieval buildings" is input into the word segmentation tool for word segmentation, and the word segmentation results obtained are "here", "Europe", "crossroads", "explore", "Estonia", "Tallinn", "old town", "appreciate", "medieval", "complete", "architecture", and input "actor A's movie 1 is played" to obtain "actor A", "of", "movie 1", "played".

[0066] In the embodiment of the present application, Chinese word segmentation refers to dividing a sequence of Chinese characters into separate words. Word segmentation is the process of recombining a continuous sequence of characters into a word sequence according to certain specifications. The word segmentation result conforms to the objective cognition of the phrase. The word segmentation tool can completely and thoroughly divide the text to be extracted into independent words regardless of the length of the text to be extracted.

[0067] For example, Jieba word segmentation can be selected as the word segmentation tool, which is a very popular tool in the field of NLP (natural language processing). The main function of Jieba is to do Chinese word segmentation, which can perform simple word segmentation, parallel word segmentation, and command line word segmentation. Its functions currently also support keyword extraction, part-of-speech tagging, word position query, etc. Although Jieba is based on Python, it also supports other languages ​​and platforms, such as C++, Go, R, Rust, Node.js, PHP, iOS, Android, etc., so Jieba can meet the needs of various developers. The number of github stars of the Jieba project has reached 24k; because Jieba word segmentation has powerful versatility and rich database, it is selected as a word segmentation tool; the Jieba word segmentation has three modes: the first is the full mode: all words that can be formed into words in the sentence are scanned out; the second is the precise mode: trying to cut the sentence most accurately (Jieba default mode); the third is the search mode: on the basis of the precise mode, long words are segmented again to improve the recall rate, which is suitable for search engine word segmentation. In the embodiment of the present application, any of the above three modes can be used to perform word segmentation operations.

[0068] like Figure 6The figure shows a schematic diagram of the process of generating an annotated text set in an embodiment of the present application. In the embodiment of the present application, the processing module is configured to replace the semantic words in the word set with a universal tag to traverse the semantic words in the word set in the step of generating the annotated text set; replace one semantic word in the word set with a universal tag in turn to obtain an annotated text sentence in each process of replacing a semantic word, wherein the annotated text sentence includes a universal tag and the semantic words in the word set that are not replaced by the universal tag, and a labeling probability is set for the annotated text sentence; and the annotated text sentences generated in each process of replacing a semantic word are combined to form the annotated text set.

[0069] In an embodiment of the present application, the processing module combines the word segmentation results, replaces each semantic word with the general marker "N", and keeps other semantic words unchanged to obtain annotated text sentences. For example, from the text to be extracted "actor A's movie 1 played", "actor AN movie 1 played", "actor A's N played", and "actor A's 1N movies" can be obtained. The annotation probability is set for each annotated text sentence, such as "N's movie 1 played, 0.9", "actor AN movie 1 played, 1", "actor A's N played, 0.7", and "actor A's 1N movies, 0.8"; all annotated text sentences are combined to constitute the annotated text set.

[0070] In the embodiment of the present application, the use of the universal tag to replace the semantic word plays the role of marking the semantic word. The model training step and the semantic extraction step are all involved in the annotated text sentences obtained by replacing the semantic word with the universal tag and keeping other semantic words unchanged, and the result output by the semantic extraction step is the semantic similarity of the replaced semantic word, that is, the semantic similarity of the marked semantic word is obtained.

[0071] In an embodiment of the present application, a labeling probability is set for each labeled text sentence in the form of a soft label. The soft label refers to a numerical value between 0 and 1 that represents the degree of similarity between the text to be extracted and the labeled text sentence. The labeling format of the soft label is based on a teacher-student design.

[0072] In the embodiment of the present application, a labeling probability is set for the labeling text generated by replacing a semantic word with each general mark in the form of a soft label, that is, the original text is compared with the replaced labeling text, and the labeling probability indicates the similarity between the original text and the replaced labeling text. For example, "N's movie 1 is played, 0.9" means that the similarity between "N's movie 1 is played" and "Actor A's movie 1 is played" is 0.9, which is very similar; "Actor AN movie 1 is played, 1" means that the similarity between "Actor AN movie 1 is played" and "Actor A's movie 1 is played" is 1, which is basically completely similar.

[0073] In the embodiment of the present application, the annotation form of the soft label can very accurately express the similarity between the original text and the replaced annotated text. The closer the annotation probability is to 1, the more similar the original text is to the replaced annotated text. The closer the annotation probability is to 0, the less similar the original text is to the replaced annotated text. The semantic extraction model annotates the annotation probability in the form of soft labels with a certain gradient, that is, the importance of different words is distinguished by using standard labels with a certain gradient.

[0074] In an embodiment of the present application, the processing module is configured to input the text to be extracted and the annotated text set into a semantic extraction model, wherein the semantic extraction model is generated by training a training sample set and an annotated sample set, the training sample set includes training sentences with semantic labels; the annotated sample set includes annotated sample sentences with annotation probabilities, and the annotated sample sentences are sentences formed by replacing keywords in the training sentences with general tags.

[0075] In an embodiment of the present application, the semantic extraction model is an improvement on the text interaction matching model. The semantic extraction model can convert the input text into a vector, and the output value is also a vector. The present application selects a value in the label classification result vector output by the model to obtain the semantic similarity.

[0076] In an embodiment of the present application, the text to be extracted and the annotated text set are input into the trained semantic extraction model to obtain the semantic similarity of the semantic words output by replacing the common tags, wherein the annotated text set and the annotated sample set have the same structure and both contain several annotated text sentences with annotation probabilities.

[0077] like Figure 7 The figure shows a schematic diagram of the process of obtaining the label classification result in the embodiment of the present application. In the embodiment of the present application, the text to be extracted Query1 and the annotated text sentence Query2 are input into the semantic extraction model to obtain the label classification result [labels_0, labels_1], wherein the label classification result [labels_0, labels_1] is calculated as follows:

[0078]

[0079] In an embodiment of the present application, the model parameters of the initial semantic model are adjusted according to the calculated loss value to obtain the semantic extraction model, and the original semantic word extraction task is converted into a task of calculating semantic similarity using a semantic similarity calculation method, thereby effectively solving the problem that the same word has different importance in different texts.

[0080] In an embodiment of the present application, the processing module is configured to obtain the semantic similarity output by the semantic extraction model in the step of obtaining the semantic similarity output by the semantic extraction model for the semantic words replaced by universal tags in the text to be extracted and each of the annotated text sentences; sort the semantic words replaced by universal tags in each of the annotated text sentences in order of the semantic similarity from large to small to obtain a semantic sorting result; and filter valid semantic words in the word set according to the semantic sorting result.

[0081] like Figure 8 The figure shows a schematic diagram of the threshold filtering process in an embodiment of the present application. In the embodiment of the present application, prob is the semantic similarity, and the value of the semantic similarity prob is equal to the value of labels_1 in the label classification result [labels_0, labels_1]. The semantic similarity output by the semantic extraction model for the text to be extracted and the annotated text sentence is obtained according to the label classification result [labels_0, labels_1]. For example, for the text to be extracted "actor A's movie 1 plays", "actor A, prob = 0.99924904", "of, prob = 0.3582316", "movie 1, prob = 0.99599165", and "play, prob = 0.9925249" can be obtained; after sorting according to the semantic similarity prob score, the semantic sorting result can be obtained: "actor A", "movie 1", "play", "of".

[0082] In an embodiment of the present application, the importance of semantic words in the text to be extracted can be determined based on the semantic sorting result. For example, in the text to be extracted "actor A's movie 1 is played", the most important semantic word is "actor A", the second most important semantic word is "movie 1", the third most important semantic word is "played", and the least important semantic word is "of"; the semantic sorting result can be used to identify the most important semantic words in the text to be extracted, and big data statistics of user preferences and habits can be performed based on this, and search associations and personalized recommendations can be performed for users based on the most important semantic words in the query texts frequently entered by users.

[0083] In an embodiment of the present application, the processing module is configured to set a filtering threshold in the step of screening valid semantic words in the word set according to the semantic ranking result; compare the semantic similarity of the semantic words replaced by the general marker in each of the annotated text sentences with the filtering threshold; and extract valid semantic words, wherein the valid semantic words are semantic words replaced by the general marker in the annotated text sentences whose semantic similarity is greater than or equal to the filtering threshold.

[0084] In the embodiment of the present application, the filtering threshold is set as needed. For example, if the filtering threshold is set to 0.5, the semantic words in the text to be extracted "actor A's movie 1 is played" can be obtained as "actor A", "movie 1", and "played".

[0085] In an embodiment of the present application, the filtering threshold is set according to the requirements of the query result range. If the filtering threshold is set to be larger, the fewer valid semantic words in the text to be extracted will be, and the range of the query result will be smaller, and the query result will be more accurate. Conversely, if the filtering threshold is set to be smaller, the more semantic words in the text to be extracted will be, and the range of the query result will be larger, and the query result will be broader, that is, the filtering threshold is negatively correlated with the query result range.

[0086] In an embodiment of the present application, if the filtering threshold is set to 0.1 during the query process, a lot of content with low relevance will appear in the search results; if the filtering threshold is set to 0.9 during the query process, only content with extremely high relevance will appear in the search results; if the query text is "Actor A's Movie 1 Play", the filtering threshold is 0.1, then the search results will include all content in the title that includes any of the semantic words "actor A", "of", "movie 1", "play", and contain a large amount of search content; if the query text is "Actor A's Movie 1 Play", the filtering threshold is 0.995, then the search results will only include all content in the title that includes any of the semantic words "actor A" and "movie 1", and the search results are very accurate.

[0087] In an embodiment of the present application, a filtering threshold can be set according to a specific application scenario. For example, in the application context of news search, the filtering threshold can be set to a higher value to ensure the accuracy of the searched news and meet the needs of users; in the application context of short video search, the filtering threshold can be set to a moderate value to ensure that the searched short video content is sufficient to meet the needs of users; in the application context of entertainment message search, the filtering threshold can be set to a lower value to ensure that the searched entertainment messages have a wide coverage and meet the needs of users.

[0088] In an embodiment of the present application, the processing module is configured to use the valid semantic words to query related media asset items in the storage module after the step of obtaining the valid semantic words; or, use the valid semantic words to send a query instruction for querying related media asset items to the server.

[0089] In the embodiment of the present application, after obtaining the valid semantic words in the word set, the processing module uses the valid semantic words to query the related media asset items in the storage module, that is, to search the related content in the training data through the valid semantic words, or uses the valid semantic words to send a query instruction for querying the related media asset items to the server, that is, to search the related content in the server data through the valid semantic words. For example, if the valid semantic words are "actor A", "movie 1", and "play", the related content of the three words "actor A", "movie 1", and "play" is queried in the storage module or server.

[0090] In an embodiment of the present application, after obtaining valid semantic words through the semantic extraction model, the smart device can search in its own database according to the relevant valid semantic words to obtain related content; the relevant valid semantic words can also be sent to other servers for search to meet the user's query needs. For example, the user can search for information about actor B stored on the user and on the Internet through the smart phone he carries with him; the user can search for videos related to food documentaries in the Internet server media library through the smart TV at home; the user can search for the location of a pizza shop in the mall in its own information database through the smart question-and-answer robot commonly found in shopping malls.

[0091] Based on the above-mentioned smart device, some embodiments of the present application also provide an effective semantic word extraction method, including:

[0092] Get the text to be extracted;

[0093] Segmenting the text to be extracted to obtain a word set, wherein the word set includes a plurality of semantic words;

[0094] Replacing semantic words in the word set with universal tags to generate an annotated text set;

[0095] The text to be extracted and the annotated text set are input into a semantic extraction model, wherein the semantic extraction model is generated by training a training sample set and an annotated sample set, wherein the training sample set includes training sentences with semantic labels; the annotated sample set includes annotated sample sentences with annotated probabilities, and the annotated sample sentences are sentences formed by replacing keywords in the training sentences with general tags;

[0096] The semantic similarity output by the semantic extraction model is obtained, and the semantic words in the word set are filtered according to a similarity threshold to obtain valid semantic words.

[0097] In an embodiment of the present application, the effective semantic word extraction method obtains a set of multiple semantic words by segmenting the text to be extracted; uses universal tags to replace the semantic words in the semantic word set to generate an annotated text set; inputs the text to be extracted and the annotated text set into a semantic extraction model, wherein the semantic extraction model is generated by training a training sample set and an annotated sample set, and the training sample set includes training sentences with semantic labels; the annotated sample set includes annotated sample sentences with annotation probabilities, and the annotated sample sentences are sentences formed by replacing keywords in the training sentences with universal tags; obtains the semantic similarity output by the semantic extraction model, and filters the semantic words in the word set according to a similarity threshold to obtain effective semantic words.

[0098] In an embodiment of the present application, the semantic extraction model in the effective semantic word extraction method is annotated with a certain gradient of soft label form annotation probability, that is, the importance of different words is distinguished by using standard labels with a certain gradient; the model parameters of the initial semantic model are adjusted according to the calculated loss value to obtain the semantic extraction model, and the original semantic word extraction task is converted into a task of calculating semantic similarity using a semantic similarity calculation method, thereby effectively solving the problem that the same word has different importance in different texts; the semantic extraction model effectively incorporates the label information annotated with a certain gradient when calculating the loss value, that is, the dynamic perception of text similarity is effectively integrated into the entire training process.

[0099] It can be seen from the above technical scheme that the present application provides an intelligent device and an effective semantic word extraction method, the intelligent device includes a storage module and a processing module, the processing module is configured to obtain a text to be extracted; segment the text to be extracted to obtain a word set, the word set includes multiple semantic words; replace the semantic words in the word set with a universal tag to generate an annotated text set; input the text to be extracted and the annotated text set into a semantic extraction model, wherein the semantic extraction model is generated by training a training sample set and an annotated sample set, the training sample set includes training sentences with semantic labels; the annotated sample set includes annotated sample sentences with annotated probabilities, the annotated sample sentences are sentences formed by replacing the keywords in the training sentences with universal tags; obtain the semantic similarity output by the semantic extraction model, and filter the semantic words in the word set according to the similarity threshold to obtain effective semantic words. The present application can extract key semantic words that affect semantic understanding from the user's query text, help search engines better understand user intent, and thus help intelligent devices give accurate search results and improve user experience.

[0100] Similar parts between the embodiments provided in this application can be referenced to each other. The specific implementation methods provided above are only a few examples under the general concept of this application and do not constitute a limitation on the protection scope of this application. For those skilled in the art, any other implementation methods expanded based on the scheme of this application without creative work belong to the protection scope of this application.

Claims

1. A smart device, characterized in that: include: A storage module, configured to store the semantic extraction model; The processing module is configured as follows: Get the text to be extracted; Segmenting the text to be extracted to obtain a word set, wherein the word set includes a plurality of semantic words; Traversing the semantic words in the word set; sequentially replacing a semantic word in the word set with a universal tag, so as to obtain a marked text sentence in each process of replacing a semantic word, wherein the marked text sentence includes the universal tag and the semantic word in the word set that is not replaced by the universal tag, and setting a marking probability for the marked text sentence; Combining the annotated text sentences generated in each semantic word replacement process to form an annotated text set; The text to be extracted and the annotated text set are input into a semantic extraction model, wherein the semantic extraction model is generated by training a training sample set and an annotated sample set, wherein the training sample set includes training sentences with semantic labels; the annotated sample set includes annotated sample sentences with annotated probabilities, wherein the annotated sample sentences are sentences formed by replacing keywords in the training sentences with general tags; the annotated probabilities are obtained by annotating in the form of soft labels, wherein the annotated form of the soft labels is used to express the degree of similarity between the original text and the replaced annotated text; the annotated probabilities are used to calculate the label classification results; and the label classification results are used to calculate the value of semantic similarity; Obtaining the semantic similarity output by the semantic extraction model for the text to be extracted and the semantic words replaced by the universal marker in each of the annotated text sentences; Sort the semantic words replaced by the universal markers in each annotated text sentence in descending order of semantic similarity to obtain a semantic sorting result; the semantic sorting result is used to screen valid semantic words; Setting a filtering threshold, comparing the semantic similarity of the semantic words replaced by the universal marker in each of the annotated text sentences with the filtering threshold; Valid semantic words are extracted, where the valid semantic words are semantic words replaced by universal tags in the annotated text sentences whose semantic similarity is greater than or equal to the filtering threshold.

2. The smart device according to claim 1, characterized in that: The processing module is configured to: In the step of segmenting the text to be extracted, calling a segmentation tool; The text to be extracted is input into the word segmentation tool to divide the text to be extracted into a plurality of semantic words to form the word set.

3. The smart device according to claim 1, characterized in that: The processing module is configured to: Acquire a training sample set, where the training sample set is constructed by a query text and a media asset title text input by a user; Extracting training sentences from the training sample set; Replacing the keywords in the training sentences with universal tags to generate the labeled sample sentences; Setting a labeling probability for the labeling sample sentence; The semantic extraction model is trained using the training sentences and the annotated sample sentences.

4. The smart device according to claim 1, characterized in that: The processing module is configured to: In the step of training the semantic extraction model using the training sentences and the annotated sample sentences, calling an initial semantic model; Inputting the training sentences and the labeled sample sentences into the initial semantic model; Obtaining the classification probability output by the initial semantic model; Calculate the loss value according to the classification probability and the labeling probability; The model parameters of the initial semantic model are adjusted according to the loss value to obtain the semantic extraction model.

5. The smart device according to claim 1, characterized in that: The processing module is configured to: In the step of obtaining the text to be extracted, a query instruction input by a user is received, wherein the query instruction includes the query text; Parsing a query text from the query instruction; The meaningless words in the query text are deleted using a preset word library to obtain the text to be extracted.

6. The smart device according to claim 1, characterized in that: The processing module is configured to: After the step of obtaining the valid semantic word, the valid semantic word is used to search for the associated media asset items in the storage module; or, The valid semantic words are used to send a query instruction for querying related media asset items to the server.

7. A method for extracting effective semantic words, characterized in that: include: Get the text to be extracted; Segmenting the text to be extracted to obtain a word set, wherein the word set includes a plurality of semantic words; Traversing the semantic words in the word set; sequentially replacing a semantic word in the word set with a universal tag, so as to obtain a marked text sentence in each process of replacing a semantic word, wherein the marked text sentence includes the universal tag and the semantic word in the word set that is not replaced by the universal tag, and setting a marking probability for the marked text sentence; Combining the annotated text sentences generated in each semantic word replacement process to form an annotated text set; The text to be extracted and the annotated text set are input into a semantic extraction model, wherein the semantic extraction model is generated by training a training sample set and an annotated sample set, wherein the training sample set includes training sentences with semantic labels; the annotated sample set includes annotated sample sentences with annotated probabilities, wherein the annotated sample sentences are sentences formed by replacing keywords in the training sentences with general tags; the annotated probabilities are obtained by annotating in the form of soft labels, and the annotated form of the soft labels is used to express the degree of similarity between the original text and the replaced annotated text; Obtaining the semantic similarity output by the semantic extraction model for the text to be extracted and the semantic words replaced by the universal marker in each of the annotated text sentences; Sort the semantic words replaced by the universal markers in each annotated text sentence in descending order of semantic similarity to obtain a semantic sorting result; the semantic sorting result is used to screen valid semantic words; Setting a filtering threshold, comparing the semantic similarity of the semantic words replaced by the universal marker in each of the annotated text sentences with the filtering threshold; Valid semantic words are extracted, where the valid semantic words are semantic words replaced by universal tags in the annotated text sentences whose semantic similarity is greater than or equal to the filtering threshold.

Images