Semantic parsing method and device

Abstract

The present application provides a semantic parsing method and apparatus. The method includes: obtaining an original text, performing semantic parsing on the original text through a multi-level word segmentation dictionary to obtain a semantic parsing result, where the multi-level word segmentation dictionary includes a first word segmentation dictionary, a second word segmentation dictionary, and a third word segmentation dictionary, and outputting the semantic parsing result. The technical solution provided by the present application effectively screens and streamlines domain entity words in a general dictionary through an entity word classification model, and at the same time classifies domain entity words through domain conflict judgment to obtain a multi-level word segmentation dictionary, effectively reducing the data volume of the word segmentation dictionary, reducing the processing time in the semantic parsing process, and improving the processing efficiency. At the same time, the accuracy of business positioning is improved. In summary, the technical solution provided by the embodiments of the present application solves the problems existing in the prior art, such as the excessive data volume of the word segmentation dictionary, resulting in excessive storage resource occupation, too long semantic parsing processing time, and poor accuracy of business positioning.

Description

Technical Field

[0001] The present application relates to the technical field of display devices, and in particular, to a semantic parsing method and apparatus. Background Art

[0002] A display device, such as a smart TV, can provide a user with playback screens such as audio, video, pictures, etc. Compared with a traditional TV that provides live TV programs for users through data broadcasting, a smart TV generally focuses on online interactive media, Internet TV, and on-demand streaming media, and can provide users with richer content and services, such as various application and service contents such as online videos and online games.

[0003] During the use of a display device, a user needs to send an instruction to the display device to achieve remote control of the display device.Currently, with the popularization of smart display devices and the continuous development of the artificial intelligence field, a user can achieve real-time control of the display device by inputting voice, and the display device performs semantic parsing on the voice instruction through a word segmentation dictionary to achieve voice control of the display device by the user.

[0004] Due to the continuous development of fields such as film and television, music, news, and games, a large number of new words are generated, such as movie names, song names, Internet words, etc. The new words make the word segmentation dictionary have a huge amount of data. Therefore, when performing semantic parsing, it is necessary to call a huge word segmentation dictionary for semantic parsing, resulting in too long semantic parsing processing time, seriously affecting the user experience. At the same time, due to the huge amount of data in the word segmentation dictionary, the storage resources occupied by the word segmentation dictionary are too large, and there is a risk of memory overflow during cache refreshing. Summary of the Invention

[0005] The present application provides a semantic parsing method and apparatus, which effectively screen and streamline an existing semantic word segmentation dictionary, reduce the amount of data in the semantic word segmentation dictionary, and solve the problem that the processing time of existing semantic parsing is too long and affects the user experience.

[0006] In a first aspect, the present application provides a semantic parsing method, including:

[0007] Obtaining an original text;

[0008] Performing semantic parsing on the original text through a multi-level word segmentation dictionary to obtain a semantic parsing result, where the multi-level word segmentation dictionary includes a first word segmentation dictionary, a second word segmentation dictionary, and a third word segmentation dictionary. The first word segmentation dictionary includes basic grammar words, domain core words, and enhanced domain entity words. The second word segmentation dictionary includes basic grammar words, domain core words, and ordinary domain entity words. The third word segmentation dictionary includes basic grammar words, domain core words, and weakened domain entity words;

[0009] Outputting the semantic parsing result.

[0010] Performing semantic parsing on the original text through a multi-level word segmentation dictionary to obtain a semantic parsing result. The multi-level word segmentation dictionary includes a first word segmentation dictionary, a second word segmentation dictionary, and a third word segmentation dictionary, and specifically includes the following steps:

[0011] Performing semantic parsing on the original text through the first word segmentation dictionary to obtain a first semantic calculation score and a first semantic parsing result;

[0012] Judging whether the first semantic calculation score is greater than a preset threshold;

[0013] If the first semantic calculation score is greater than the preset threshold, outputting the first semantic parsing result as the semantic parsing result;

[0014] If the first semantic calculation score is not greater than the preset threshold;

[0015] Performing semantic parsing on the original text through the second word segmentation dictionary to obtain a second semantic calculation score and a second semantic parsing result;

[0016] Judging whether the second semantic calculation score is greater than the preset threshold;

[0017] If the second semantic calculation score is greater than the preset threshold, outputting the second semantic parsing result as the semantic parsing result;

[0018] If the second semantic calculation score is not greater than the preset threshold;

[0019] Performing semantic parsing on the original text through the third word segmentation dictionary to obtain a third semantic calculation score and a third semantic parsing result;

[0020] Judging whether the third semantic calculation score is greater than the preset threshold;

[0021] If the third semantic calculation score is greater than the preset threshold, outputting the third semantic parsing result as the semantic parsing result.

[0022] Before obtaining the original text, it further includes:

[0023] Obtaining a general word segmentation dictionary;

[0024] Constructing a multi-level word segmentation dictionary according to the general word segmentation dictionary.

[0025] The constructing of the multi-level word segmentation dictionary model according to the general dictionary includes the following steps:

[0026] Constructing an entity word classification model;

[0027] Obtain the set of domain entity words from the general dictionary;

[0028] Input the set of domain entity words into the entity word classification model for classification to obtain an enhanced set of domain entity words and a non-enhanced set of domain entity words;

[0029] Split the non-enhanced set of domain entity words according to domain conflict judgment to obtain a set of ordinary domain entity words and a set of weakened domain entity words;

[0030] Obtain the set of basic grammar words and the set of domain core words from the general dictionary;

[0031] Generate the first word segmentation dictionary according to the set of basic grammar words, the set of domain core words, and the enhanced set of domain entity words, generate the second word segmentation dictionary according to the set of basic grammar words, the set of domain core words, and the set of ordinary domain entity words, and generate the third word segmentation dictionary according to the set of basic grammar words, the set of domain core words, and the set of weakened domain entity words;

[0032] Generate the multi-level word segmentation dictionary according to the first word segmentation dictionary, the second word segmentation dictionary, and the third word segmentation dictionary.

[0033] The construction of the entity word classification model includes the following steps:

[0034] Obtain network entity word popularity data and the request text input by the user. The network entity word popularity data includes Internet entity data, rankings, and click-through rates. The request text includes domain entity words, access rankings, and access times;

[0035] Calculate the influence factors of various entity words in turn according to the network entity word popularity data and the request text input by the user;

[0036] Perform K_means clustering training on the influence factors to generate an entity word classification model.

[0037] Input the set of domain entity words into the entity word classifier for classification to obtain an enhanced set of domain entity words and a non-enhanced set of domain entity words, which specifically includes the following steps:

[0038] Input the domain entity words in the set of domain entity words into the entity word classifier in turn;

[0039] Calculate the influence factor corresponding to the domain entity word;

[0040] If the influence factor of the current entity word is greater than the preset threshold, it is an enhanced domain entity word;

[0041] If the influence factor of the current entity word is not greater than the preset threshold, it is a non-enhanced domain entity word;

[0042] Generate an enhanced domain entity word set based on the enhanced domain entity words, and generate a non-enhanced domain entity word set based on the non-enhanced domain entity words.

[0043] Split the non-enhanced domain entity word set according to domain conflict judgment to obtain a general domain entity word set and a weakened domain entity word set, which specifically includes the following steps:

[0044] Obtain Internet popular text corpus data, perform short sentence splitting, count the sentence frequencies of each short sentence, and obtain domain short sentences;

[0045] Perform word segmentation and semantic parsing on the domain short sentences to establish a domain conflict syntactic set;

[0046] Input the entity words in the non-enhanced domain entity word set into the domain conflict syntactic set in sequence to perform domain conflict judgment detection;

[0047] If there is a conflict, the current entity word is a weakened domain entity word;

[0048] If there is no conflict, the current entity word is a general domain entity word;

[0049] Generate a weakened domain entity word set according to the weakened domain entity words, and generate a general domain entity word set according to the general domain entity words.

[0050] Before outputting the semantic parsing result, the following steps are further included:

[0051] Obtain a domain entity word set and a domain core word set from a general dictionary;

[0052] Establish a domain entity word classification table according to the domain entity word set, and establish a domain core word mapping table according to the domain core word set;

[0053] Divide the domain core word set and the domain entity word set into several domain dictionaries and corresponding domain labels according to the above domain entity word classification table and the above domain core word mapping table.

[0054] The following steps are further included:

[0055] Call the domain dictionary with the corresponding domain label for semantic parsing according to the semantic parsing result;

[0056] If the semantic parsing result is different from the domain label, filter the semantic parsing result.

[0057] If the semantic parsing result is the same as the domain label, retain the semantic parsing result.

[0058] Second aspect, the present application provides a semantic parsing device, including:

[0059] A text acquisition unit: to acquire the original text;

[0060] A semantic parsing unit: to perform semantic parsing on the original text through a multi-level word segmentation dictionary to obtain a semantic parsing result, where the multi-level word segmentation dictionary includes a first word segmentation dictionary, a second word segmentation dictionary, and a third word segmentation dictionary. The first word segmentation dictionary includes basic grammar words, domain core words, and enhanced domain entity words. The second word segmentation dictionary includes basic grammar words, domain core words, and ordinary domain entity words. The third word segmentation dictionary includes basic grammar words, domain core words, and weakened domain entity words;

[0061] A result output unit: to output the semantic parsing result.

[0062] As can be seen from the above technical solutions, the embodiments of the present application provide a semantic parsing method and device, including: acquiring the original text, performing semantic parsing on the original text through a multi-level word segmentation dictionary to obtain a semantic parsing result, where the multi-level word segmentation dictionary includes a first word segmentation dictionary, a second word segmentation dictionary, and a third word segmentation dictionary, and outputting the semantic parsing result. The technical solution provided by the present application effectively screens and streamlines the domain entity words in the general dictionary through an entity word classification model, and at the same time classifies the domain entity words through domain conflict judgment to obtain a multi-level word segmentation dictionary, effectively reducing the data volume of the word segmentation dictionary, reducing the processing time in the semantic parsing process, and improving the processing efficiency. At the same time, the accuracy of business positioning is improved. In summary, the technical solution provided by the embodiments of the present application solves the problems in the prior art that the data volume of the word segmentation dictionary is too large, resulting in excessive occupation of storage resources, too long semantic parsing processing time, and poor accuracy of business positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the embodiments of the present application or the implementation manners in related technologies, the following will briefly introduce the drawings required to be used in the description of the embodiments or related technologies. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained according to these drawings.

[0064] Figure 1 It is a schematic diagram of a semantic parsing network architecture provided by an embodiment of the present application;

[0065] Figure 2 It is a schematic diagram of an application scenario of the semantic parsing method in some embodiments;

[0066] Figure 3 It is a flowchart of a semantic parsing method shown in an exemplary embodiment of the present application;

[0067] Figure 4 A schematic flowchart of semantic parsing shown in an exemplary embodiment of the present application;

[0068] Figure 5 A flowchart of the method in the first embodiment provided by the present application;

[0069] Figure 6 A flowchart of the method in the second embodiment provided by the present application;

[0070] Figure 7 A flowchart of the method in the third embodiment provided by the present application;

[0071] Figure 8 A flowchart of the method in the fourth embodiment provided by the present application;

[0072] Figure 9 A flowchart of the method in the fifth embodiment provided by the present application;

[0073] Figure 10 A flowchart of the method in the sixth embodiment provided by the present application;

[0074] Figure 11 A flowchart of the method in the seventh embodiment provided by the present application;

[0075] Figure 12 A schematic connection diagram of a semantic parsing device shown in an exemplary embodiment of the present application. Detailed implementation manners

[0076] To make the purpose and implementation manners of the present application clearer, the following will clearly and completely describe the exemplary implementation manners of the present application with reference to the accompanying drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only a part of the embodiments of the present application, rather than all of the embodiments.

[0077] It should be noted that the brief description of the terms in the present application is only for the convenience of understanding the subsequent described implementation manners, rather than intending to limit the implementation manners of the present application. Unless otherwise specified, these terms should be understood according to their ordinary and common meanings.

[0078] The terms "first", "second", "third", etc. in the specification, claims and the above-mentioned drawings of the present application are used to distinguish similar or homogeneous objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that such terms can be interchanged under appropriate circumstances.

[0079] The terms "comprising" and "having" and any variations thereof are intended to cover inclusion without exclusion. For example, a product or device comprising a series of components need not be limited to all the components clearly listed, but may include other components not clearly listed or inherent to such products or devices.

[0080] The term "module" refers to any known or later-developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that can perform functions related to that element.

[0081] To make the objectives and implementation manners of this application clearer, the following will clearly and completely describe the exemplary implementation manners of this application in conjunction with the accompanying drawings in the exemplary embodiments of this application. Obviously, the described exemplary embodiments are only a part of the embodiments of this application, rather than all of the embodiments.

[0082] It should be noted that the brief description of the terms in this application is only for the convenience of understanding the subsequent described implementation manners, rather than intending to limit the implementation manners of this application. Unless otherwise specified, these terms should be understood in their ordinary and common meanings.

[0083] The terms "first", "to be supplemented", "third", etc. in the description, claims and the above-mentioned drawings of this application are used to distinguish similar or like objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that such terms can be interchanged under appropriate circumstances.

[0084] The terms "comprising" and "having" and any variations thereof are intended to cover inclusion without exclusion. For example, a product or device comprising a series of components need not be limited to all the components clearly listed, but may include other components not clearly listed or inherent to such products or devices.

[0085] The term "module" refers to any known or later-developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that can perform functions related to that element.

[0086] To clearly illustrate the embodiments of this application, the following will be combined with Figure 1 Describe a speech recognition network architecture provided by an embodiment of this application.

[0087] See Figure 1 , Figure 1 It is a schematic diagram of a semantic parsing network architecture provided by an embodiment of this application. Figure 1Among them, the intelligent device is used to receive the input information and output the processing result of the information. The speech recognition service device is an electronic device deployed with a speech recognition service, the semantic service device is an electronic device deployed with a semantic service, and the business service device is an electronic device deployed with a business service. The electronic devices here may include servers, computers, etc. The speech recognition service, semantic service (also referred to as a semantic engine), and business service here are web services that can be deployed on electronic devices. Among them, the speech recognition service is used to recognize audio as text, the semantic service is used to perform semantic parsing on the text, and the business service is used to provide specific services such as weather query services of Moji Weather, music query services of QQ Music, etc. In one embodiment, Figure 1 In the architecture shown, there may be multiple entity service devices deployed with different business services, or one or more entity service devices may integrate one or more functional services.

[0088] In some embodiments, the following is an example description of the process of processing the information input to the intelligent device based on Figure 1 the architecture shown. Taking the information input to the intelligent device as a query statement input by voice as an example, the above process may include the following three processes:

[0089] [Speech Recognition]

[0090] After receiving the query statement input by voice, the intelligent device can upload the audio of the query statement to the speech recognition service device, so that the speech recognition service device can recognize the audio as text through the speech recognition service and return it to the intelligent device. In one embodiment, before uploading the audio of the query statement to the speech recognition service device, the intelligent device can perform noise reduction processing on the audio of the query statement. The noise reduction processing here may include steps such as removing echo and ambient noise.

[0091] [Semantic Parsing]

[0092] The intelligent device uploads the text of the query statement recognized by the speech recognition service to the semantic service device, so that the semantic service device can perform semantic parsing on the text through the semantic service to obtain the business field, intention, etc. of the text.

[0093] [Semantic Response]

[0094] The semantic service device issues a query instruction to the corresponding business service device according to the semantic parsing result of the text of the query statement to obtain the query result given by the business service. The intelligent device can obtain the query result from the semantic service device and output it. As an embodiment, the semantic service device can also send the semantic parsing result of the query statement to the intelligent device, so that the intelligent device outputs the feedback statement in the semantic parsing result.

[0095] In some embodiments, Figure 1The intelligent device shown may be a display device, such as a smart TV. The functions of the speech recognition service device can be realized by the cooperation of the sound collector and the controller set on the display device, and the functions of the semantic service device and the service service device can be realized by the controller of the display device or by the server of the display device.

[0096] During the use of the intelligent device, the user needs to send instructions to the intelligent device to realize remote control of the intelligent device. At present, with the popularization of intelligent devices and the continuous development of the artificial intelligence field, the user can realize real-time control of the intelligent device by inputting speech. The intelligent device performs semantic parsing on the speech instruction through a word segmentation dictionary to realize the speech control of the display device by the user.

[0097] Due to the continuous development of fields such as film and television, music, news, and games, a large number of new words are generated, such as movie names, song names, and network words. The new words make the word segmentation dictionary have a huge amount of data. Therefore, when performing semantic parsing, it is necessary to call a huge word segmentation dictionary for semantic parsing, resulting in too long semantic parsing processing time, seriously affecting the user experience. At the same time, due to the huge amount of data in the word segmentation dictionary, the storage resources occupied by the word segmentation dictionary are too large, and there is a risk of memory overflow when the cache is refreshed.

[0098] To solve the above problems, the embodiments of the present application provide a semantic parsing method and device. The method performs semantic parsing on the original text input by the user through a multi-level word segmentation dictionary, reduces the processing time, improves the processing efficiency, and thus better realizes the operations required by the user and improves the user experience.

[0099] Figure 2 The figure shows a schematic diagram of the application scenario of the semantic parsing method in some embodiments. The method can be applied to the scenario where the user uses speech to control an intelligent electronic device and is executed by the intelligent electronic device or by a device dedicated to voice interaction set in the intelligent electronic device. Hereinafter, taking the execution subject as the intelligent electronic device as an exemplary description, rather than limiting it. The intelligent electronic device can be a display device, such as a smart TV, or can also be a mobile phone, a computer, a television, a washing machine, an air conditioner, a speaker, etc.

[0100] Figure 3 It is a flowchart of a semantic parsing method shown in an exemplary embodiment of the present application. The method can be applied to various devices such as a display device, a server, a PC (personal computer), a tablet computer, a mobile phone, a virtual reality device, and a smart wearable device.

[0101] See Figure 3 As shown, the method may include the following steps:

[0102] S110: Obtain the original text;

[0103] S120: performing semantic analysis on the original text using a multi-level word segmentation dictionary to obtain a semantic analysis result;

[0104] The multi-level word segmentation dictionary includes a first word segmentation dictionary, a second word segmentation dictionary and a third word segmentation dictionary, wherein the first word segmentation dictionary includes basic grammatical words, domain core words and enhanced domain entity words, the second word segmentation dictionary includes basic grammatical words, domain core words and general domain entity words, and the third word segmentation dictionary includes basic grammatical words, domain core words and weakened domain entity words;

[0105] It should be noted that from the perspective of vertical domain analysis, vocabulary can be divided into basic grammatical words, domain core words, and domain entity words. Basic grammatical words are the smallest units that make up a sentence, such as verbs, adjectives, auxiliary words, pronouns, directional words, etc., and are the basis for conventional word segmentation. Domain core words are key feature identifiers of domain intent. For example, "movie" is a core word in the film and television field, "weather" is a core word in the weather field, and "takeout" is a keyword in the takeout field. These words can clearly identify the intent of the field. Domain entity words are words that identify entity names within the field. These words have very strong domain attributes and may not represent any meaningful entities outside the field, such as movie names, song names, game names, scenic spot names, etc. These words account for the highest proportion in terms of data volume and are also an important cause of word segmentation ambiguity.

[0106] Among them, domain entity words are also divided into enhanced domain entity words, ordinary domain entity words and weakened domain entity words. For example, domain entity words such as the movie "Wolf Warrior" and the song "Dimple" are very popular and appear frequently on the Internet, and are enhanced domain entity words. However, the movie "Fireman's Ball" (movie, 1967) and "Ride a Bicycle to the Beach to Find You" (song, 2014) only have conventional domain attributes and are ordinary domain entity words. However, "It's Raining in Beijing" (song, 2013) and "Make a Phone Call" (song, 2000) have serious attribute conflicts with the weather domain and the video call domain, and are weakened domain entity words. In one embodiment of the present application, the first word segmentation dictionary includes basic grammatical words, domain core words and enhanced domain entity words, the second word segmentation dictionary includes basic grammatical words, domain core words and ordinary domain entity words, and the third word segmentation dictionary includes basic grammatical words, domain core words and weakened domain entity words. By effectively dividing domain entity words and reducing the amount of data in the word segmentation dictionary, and storing basic grammatical words and domain core words in a multi-level word segmentation dictionary, the accuracy and completeness of the semantic analysis results can be guaranteed.

[0107] S130: Output semantic parsing results.

[0108] Figure 4The flowchart of step S120 of a semantic parsing method shown in an exemplary embodiment of this application specifically includes the following steps:

[0109] S121: Perform semantic parsing on the original text through the first word segmentation dictionary to obtain the first semantic calculation score and the first semantic parsing result;

[0110] S122: Determine whether the first semantic calculation score is greater than the preset threshold. If the first semantic calculation score is greater than the preset threshold, output the semantic parsing result;

[0111] S123: If the first semantic calculation score is not greater than the preset threshold, perform semantic parsing on the original text through the second word segmentation dictionary to obtain the second semantic calculation score and the second semantic parsing result;

[0112] S124: Determine whether the second semantic calculation score is greater than the preset threshold. If the second semantic calculation score is greater than the preset threshold, output the semantic parsing result;

[0113] S125: If the second semantic calculation score is not greater than the preset threshold, perform semantic parsing on the original text through the third word segmentation dictionary to obtain the third semantic calculation score and the third semantic parsing result;

[0114] S126: Determine whether the third semantic calculation score is greater than the preset threshold,

[0115] S127: If the third semantic calculation score is greater than the preset threshold, output the semantic parsing result.

[0116] S128: When the third semantic calculation score is not greater than the preset threshold, terminate the program.

[0117] After the program terminates, cancel the output of the semantic parsing result and process the text input by the user this time as a chat service.

[0118]

[0119] Table 1

[0120] Table 1 shows the semantic parsing result of an embodiment provided by this application. The original text input by the user is "I want to make a call". Perform semantic parsing on the original text through the first word segmentation dictionary to obtain the first semantic calculation score score = 1.0 and the first semantic parsing result domain = video_call. Since the first semantic calculation score is greater than the preset threshold at this time, the first semantic parsing result is output. There is no need to perform semantic parsing on the original text through the second and third word segmentation dictionaries again, effectively saving processing time and improving the semantic parsing efficiency.

[0121] See Figure 5, in an embodiment provided by the present application, before obtaining the original text, it further includes:

[0122] S210: Obtain a general word segmentation dictionary;

[0123] S220: Construct a multi-level word segmentation dictionary according to the general word segmentation dictionary.

[0124] In an embodiment provided by the present application, referring to Figure 6 , constructing a multi-level word segmentation dictionary model according to the general dictionary includes the following steps:

[0125] S2201: Construct an entity word classification model;

[0126] S2202: Obtain a domain entity word set from the general dictionary;

[0127] S2203: Input the domain entity word set into the entity word classification model for classification to obtain an enhanced domain entity word set and a non-enhanced domain entity word set;

[0128] S2204: Split the non-enhanced domain entity word set according to the domain conflict judgment to obtain a common domain entity word set and a weakened domain entity word set;

[0129] S2205: Obtain a basic grammar word set and a domain core word set from the general dictionary;

[0130] S2206: Generate a first word segmentation dictionary according to the basic grammar word set, the domain core word set and the enhanced domain entity word set, generate a second word segmentation dictionary according to the basic grammar word set, the domain core word set and the common domain entity word set, and generate a third word segmentation dictionary according to the basic grammar word set, the domain core word set and the weakened domain entity word set;

[0131] S2207: Generate a multi-level word segmentation dictionary according to the first word segmentation dictionary, the second word segmentation dictionary and the third word segmentation dictionary.

[0132] In an embodiment provided by the present application, referring to Figure 7 , constructing an entity word classification model includes the following steps:

[0133] S310: Obtain network entity word heat data and the request text input by the user. The network entity word heat data includes Internet entity data, rankings and click-through rates, and the request text includes domain entity words, access rankings and access times;

[0134] S320: Calculate the influence factors of various entity words in turn according to the network entity word heat data and the request text input by the user;

[0135] More specifically, first, by obtaining the popularity data of network entity words, converting the popularity data of network entity words into text vectors, where the popularity data of network entity words includes Internet entity data, rankings, and click-through rates, the normalized network influence factor α is calculated based on the text vectors. w And the network influence factor α of entity word i wi , and the calculation formula is:

[0136]

[0137] ]>Among them, rank wi is the network ranking of entity word i, rank wmax is the maximum network ranking, rank wmin is the minimum network ranking, count wi is the click-through rate of entity word i, count wmax为 is the maximum click-through rate, count wmin is the minimum click-through rate.

[0138] In an embodiment of the present application, the minimum click-through rate count wmin = 0, and the minimum network ranking rank wmin = 1

[0139] Secondly, obtain the request text input by the user. The request text includes domain entity words, access rankings, and access times, and calculate the normalized user influence factor α based on the request text. u And the user influence factor α of entity word i ui , and the calculation formula is:

[0140]

[0141] Among them, rank ui is the access ranking of entity word i, rank umax is the maximum access ranking, rank umin is the minimum access ranking, count i is the access times of entity word i, count umax is the maximum access times, count umin is the minimum access times;

[0142] In an embodiment of the present application, the minimum access times count umin = 0, and the minimum access ranking rank umin = 1.

[0143] According to the normalized network influence factor α w , the network influence factor α of entity word i wi , the normalized user influence factor α u and the user influence factor α of entity word iui Calculate the entity influence factor α, and the calculation formula is:

[0144] α = α wi × w1 + α ui × w2, where α wi is the network influence factor of entity word i, w1 is the network weight, and α ui is the user influence factor of entity word i

[0145] and w2 is the user weight; w1 + w2 = 1

[0146] S330: Perform K_means clustering training on the influence factors to generate an entity word classification model.

[0147] Specifically, calculate the influence factors of each entity word in the existing domain entity words in the general word segmentation dictionary according to the above formula, and perform K_means clustering training on the influence factors of the entity words to obtain the boundary value Tedge of the two classifications of enhanced domain entity words and general domain entity words. The boundary value Tedge is a preset threshold, and an entity word classification model is generated according to the preset threshold.

[0148] Furthermore, when classifying newly generated entity words, input the entity words into the entity word classification model, that is, the entity words are divided into enhanced domain entity words or non-enhanced domain entity words.

[0149] In an embodiment provided by the present application, refer to Figure .8 , input the domain entity word set into the entity word classifier for classification to obtain an enhanced domain entity word set and a non-enhanced domain entity word set, which specifically includes the following steps:

[0150] S410: Input the domain entity words in the domain entity word set into the entity word classifier in sequence;

[0151] S420: Calculate the influence factors corresponding to the domain entity words;

[0152] S430: Determine whether the influence factor corresponding to the domain entity word is greater than the preset threshold;

[0153] S440: If the influence factor of the current entity word is greater than the preset threshold, the current domain entity word is an enhanced domain entity word.

[0154] S450: If the influence factor of the current entity word is not greater than the preset threshold, the current domain entity word is a non-enhanced domain entity word.

[0155] S460: Generate an enhanced domain entity word set according to the enhanced domain entity words, and generate a non-enhanced domain entity word set according to the non-enhanced domain entity words.

[0156] In an embodiment provided by the present application, refer toFigure 9 , split the non-enhanced domain entity word set according to domain conflict judgment to obtain a general domain entity word set and a weakened domain entity word set, which specifically includes the following steps:

[0157] S510: Obtain Internet popular text corpus data, perform short sentence splitting, count the frequency of each short sentence, and obtain domain short sentences;

[0158] S520: Perform word segmentation and semantic parsing on the domain short sentences to establish a set of domain conflict syntax sets;

[0159] S530: Input the entity words in the non-enhanced domain entity word set into the set of domain conflict syntax sets in sequence for domain conflict judgment detection;

[0160] S540: Determine whether there is a conflict;

[0161] S550: If there is a conflict, the current domain entity word is a weakened domain entity word.

[0162] S560: If there is no conflict, the current domain entity word is a general domain entity word.

[0163] S570: Generate a weakened domain entity word set according to the weakened domain entity words, and generate a general domain entity word set according to the general domain entity words.

[0164] In an embodiment of the present application, refer to Figure 10 , and further includes the following steps:

[0165] S610: Obtain a domain entity word set and a domain core word set from a general dictionary;

[0166] S620: Establish a domain entity word classification table according to the domain entity word set, and establish a domain core word mapping table according to the domain core word set;

[0167] S630: Divide the domain core word set and the domain entity word set into several domain dictionaries and corresponding domain labels according to the above domain entity word classification table and the above domain core word mapping table.

[0168] Specifically, create a domain entity word classification table and a domain core word mapping table based on the domain entity word set and the domain core word set obtained from the general dictionary to form a corresponding classification system. Each domain expert classifies the general dictionary according to the domain, and establishes a unified attribute mapping relationship for each domain in the dictionary, and each attribute corresponds to a domain.

[0169] For the domain core words in the domain core word set in the general dictionary in sequence, they are split and classified according to the domain core word mapping table, their corresponding domains are obtained, and stored in the corresponding domain dictionaries. For example, the domain core word "TV drama" belongs to the film and television domain. According to the mapping table, it is queried as the film and television domain and should be assigned to the film and television dictionary. At the same time, for the domain entity words in the domain entity word set in the general dictionary in sequence, they are classified, their corresponding domains are obtained according to the domain entity word classification table, and stored in the domain dictionaries of the corresponding domains. It should be noted that when a domain entity word belongs to multiple domains, it is stored separately in multiple corresponding domain dictionaries. For example, the domain entity word "Journey to the West" has multiple attributes such as song name, TV drama name, game name, and novel name, and multiple attributes belong to different domains respectively. Therefore, it is assigned to the corresponding domain dictionaries of film and television, music, game, and novel respectively.

[0170] In an embodiment of the present application, referring to Figure 11 , before outputting the semantic parsing result, the following steps are further included:

[0171] S710: Invoke the domain dictionary with the corresponding domain label according to the semantic parsing result for semantic parsing;

[0172] S720: Determine whether the semantic parsing result is the same as the domain label;

[0173] S730: If the semantic parsing result is different from the domain label, filter the semantic parsing result;

[0174] S740: If the semantic parsing result is the same as the domain label, retain the semantic parsing result.

[0175] As can be seen from the above technical solutions, the embodiments of the present application provide a semantic parsing method and device, including: obtaining the original text, performing semantic parsing on the original text through a multi-level segmentation dictionary to obtain a semantic parsing result. The multi-level segmentation dictionary includes a first segmentation dictionary, a second segmentation dictionary, and a third segmentation dictionary, and outputting the semantic parsing result. The technical solution provided by the present application effectively screens and streamlines the domain entity words in the general dictionary through the entity word classification model, and at the same time classifies the domain entity words through the domain conflict judgment to obtain a multi-level segmentation dictionary, effectively reducing the data volume of the segmentation dictionary, reducing the processing time in the semantic parsing process, and improving the processing efficiency. At the same time, the accuracy of business positioning is improved. In summary, the technical solution provided by the embodiments of the present application solves the problems in the prior art that the data volume of the segmentation dictionary is too large, resulting in excessive occupation of storage resources, too long semantic parsing processing time, and poor accuracy of business positioning.

[0176] The embodiments of the present application provide a semantic parsing device, Figure 12 which is a structural block diagram of a semantic parsing device provided by the embodiments of the present application, asFigure 12 As shown in Figure 12 , the device includes:

[0177] A text acquisition unit 801: to acquire the original text.

[0178] A semantic parsing unit 802: to perform semantic parsing on the original text through a multi-level word segmentation dictionary to obtain a semantic parsing result. The multi-level word segmentation dictionary includes a first word segmentation dictionary, a second word segmentation dictionary, and a third word segmentation dictionary. The first word segmentation dictionary includes basic grammar words, domain core words, and enhanced domain entity words. The second word segmentation dictionary includes basic grammar words, domain core words, and common domain entity words. The third word segmentation dictionary includes basic grammar words, domain core words, and weakened domain entity words.

[0179] A result output unit 803: to output the semantic parsing result.

[0180] As can be seen from the above technical solutions, the embodiments of the present application provide a semantic parsing method and device, including: acquiring the original text, performing semantic parsing on the original text through a multi-level word segmentation dictionary to obtain a semantic parsing result. The multi-level word segmentation dictionary includes a first word segmentation dictionary, a second word segmentation dictionary, and a third word segmentation dictionary, and outputting the semantic parsing result. The technical solution provided by the present application effectively screens and streamlines the domain entity words in the general dictionary through an entity word classification model, and at the same time classifies the domain entity words through domain conflict judgment to obtain a multi-level word segmentation dictionary, effectively reducing the data volume of the word segmentation dictionary, reducing the processing time in the semantic parsing process, and improving the processing efficiency. At the same time, the accuracy of business positioning is improved. In summary, the technical solution provided by the embodiments of the present application solves the problems in the prior art that the data volume of the word segmentation dictionary is too large, resulting in excessive storage resource occupation, too long semantic parsing processing time, and poor accuracy of business positioning.

[0181] The present application can be used in many general-purpose or special-purpose computing system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and so on.

[0182] The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in a distributed computing environment, where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0183] It should be noted that, in this document, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or apparatus that comprises a series of elements includes not only those elements but also other elements not expressly listed, or elements that are inherent to such process, method, article or apparatus.

[0184] Those skilled in the art will readily conceive of other embodiments of the present application after considering the specification and practicing the application disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include known common knowledge or conventional technical means in the technical field not disclosed in the present application. The specification and examples are only regarded as exemplary, and the true scope and spirit of the present application are pointed out by the following claims.

[0185] It should be understood that the present application is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present application is only limited by the appended claims.

Claims

1. A semantic parsing method, characterized in that, Including: Obtain the original text; Obtain a multi-level word segmentation dictionary, where the multi-level word segmentation dictionary includes a first word segmentation dictionary, a second word segmentation dictionary, and a third word segmentation dictionary. The first word segmentation dictionary includes basic grammar words, domain core words, and enhanced domain entity words. The second word segmentation dictionary includes basic grammar words, domain core words, and common domain entity words. The third word segmentation dictionary includes basic grammar words, domain core words, and weakened domain entity words. Among them, the basic grammar words are the smallest units that make up a sentence, the domain core words are the characteristic identifiers of domain intentions, and the enhanced domain entity words, the common domain entity words, and the weakened domain entity words are all used to identify entity names within the domain. The conflict degree between the enhanced domain entity words, the common domain entity words, and the weakened domain entity words and the attributes of the domain increases in sequence; Perform semantic parsing on the original text through the first word segmentation dictionary to obtain a first semantic calculation score and a first semantic parsing result; Determine whether the first semantic calculation score is greater than a preset threshold; If the first semantic calculation score is greater than the preset threshold, output the first semantic parsing result as the semantic parsing result; If the first semantic calculation score is not greater than the preset threshold; Perform semantic parsing on the original text through the second word segmentation dictionary to obtain a second semantic calculation score and a second semantic parsing result; Determine whether the second semantic calculation score is greater than the preset threshold; If the second semantic calculation score is greater than the preset threshold, output the second semantic parsing result as the semantic parsing result; If the second semantic calculation score is not greater than the preset threshold; Perform semantic parsing on the original text through the third word segmentation dictionary to obtain a third semantic calculation score and a third semantic parsing result; Determine whether the third semantic calculation score is greater than the preset threshold; If the third semantic calculation score is greater than the preset threshold, output the third semantic parsing result as the semantic parsing result; Output the semantic parsing result.

2. The semantic parsing method according to claim 1, wherein Before obtaining the original text, it further includes: Obtain a general word segmentation dictionary; Construct the multi-level word segmentation dictionary according to the general word segmentation dictionary.

3. The semantic parsing method according to claim 2, wherein The constructing the multi-level word segmentation dictionary according to the general word segmentation dictionary includes the following steps: Construct an entity word classification model; Obtain a set of domain entity words from the general word segmentation dictionary; Input the set of domain entity words into the entity word classification model for classification to obtain a set of enhanced domain entity words and a set of non-enhanced domain entity words; Split the set of non-enhanced domain entity words according to domain conflict judgment to obtain a set of common domain entity words and a set of weakened domain entity words; Obtain a set of basic grammar words and a set of domain core words from the general word segmentation dictionary; Generate the first word segmentation dictionary according to the basic grammar word set, the domain core word set, and the enhanced domain entity word set, generate the second word segmentation dictionary according to the basic grammar word set, the domain core word set, and the general domain entity word set, and generate the third word segmentation dictionary according to the basic grammar word set, the domain core word set, and the weakened domain entity word set; Generate the multi-level word segmentation dictionary according to the first word segmentation dictionary, the second word segmentation dictionary, and the third word segmentation dictionary.

4. The semantic parsing method according to claim 3, wherein The constructing of the entity word classification model includes the following steps: Obtain network entity word popularity data and the request text input by the user. The network entity word popularity data includes Internet entity data, rankings, and click-through rates. The request text includes domain entity words, access rankings, and access times; Calculate the influence factors of various entity words in turn according to the network entity word popularity data and the request text input by the user; Perform K_means clustering training on the influence factors to generate an entity word classification model.

5. The semantic parsing method according to claim 4, wherein Input the domain entity word set into the entity word classification model for classification to obtain an enhanced domain entity word set and a non-enhanced domain entity word set, which specifically includes the following steps: Input the domain entity words in the domain entity word set into the entity word classification model in turn; Calculate the influence factor corresponding to the domain entity word; If the influence factor of the current entity word is greater than the preset threshold, the current domain entity word is an enhanced domain entity word; If the influence factor of the current entity word is not greater than the preset threshold, the current domain entity word is a non-enhanced domain entity word; Generate an enhanced domain entity word set according to the enhanced domain entity words, and generate a non-enhanced domain entity word set according to the non-enhanced domain entity words.

6. The semantic parsing method according to claim 5, characterized in that Split the non-enhanced domain entity word set according to domain conflict judgment to obtain a general domain entity word set and a weakened domain entity word set, which specifically includes the following steps: Obtain Internet popular text corpus data, perform short sentence splitting, count the frequency of each short sentence, and obtain domain short sentences; Perform word segmentation and semantic parsing on the domain short sentences to establish a domain conflict syntax set; Input the entity words in the non-enhanced domain entity word set into the domain conflict syntax set in turn to perform domain conflict judgment detection; If there is a conflict, the current domain entity word is a weakened domain entity word; If there is no conflict, the current domain entity word is a general domain entity word; Generate a weakened domain entity word set according to the weakened domain entity words, and generate a general domain entity word set according to the general domain entity words.

7. The semantic parsing method according to claim 1, wherein Before outputting the semantic parsing result, it also includes the following steps: Obtain the domain entity word set and the domain core word set from the general word segmentation dictionary; Establish a domain entity word classification table according to the domain entity word set, and establish a domain core word mapping table according to the domain core word set; Divide the domain core word set and the domain entity word set into several domain dictionaries and corresponding domain labels according to the above domain entity word classification table and the above domain core word mapping table.

8. The semantic parsing method according to claim 7, wherein It also includes the following steps: Call the domain dictionary with the corresponding domain label according to the semantic parsing result for semantic parsing; If the semantic parsing result is different from the domain label, filter the semantic parsing result; If the semantic parsing result is the same as the domain label, retain the semantic parsing result.

9. A semantic parsing device, characterized in that, It includes: Text acquisition unit: Acquire the original text; Semantic parsing unit: Acquire a multi-level word segmentation dictionary, where the multi-level word segmentation dictionary includes a first word segmentation dictionary, a second word segmentation dictionary, and a third word segmentation dictionary. The first word segmentation dictionary includes basic grammar words, domain core words, and enhanced domain entity words. The second word segmentation dictionary includes basic grammar words, domain core words, and ordinary domain entity words. The third word segmentation dictionary includes basic grammar words, domain core words, and weakened domain entity words; perform semantic parsing on the original text through the first word segmentation dictionary to obtain a first semantic calculation score and a first semantic parsing result; determine whether the first semantic calculation score is greater than a preset threshold; If the first semantic calculation score is greater than the preset threshold, output the first semantic parsing result as the semantic parsing result; if the first semantic calculation score is not greater than the preset threshold; Perform semantic parsing on the original text through the second word segmentation dictionary to obtain a second semantic calculation score and a second semantic parsing result; determine whether the second semantic calculation score is greater than the preset threshold; If the second semantic calculation score is greater than the preset threshold, output the second semantic parsing result as the semantic parsing result; if the second semantic calculation score is not greater than the preset threshold; Perform semantic parsing on the original text through the third word segmentation dictionary to obtain a third semantic calculation score and a third semantic parsing result; determine whether the third semantic calculation score is greater than the preset threshold; if the third semantic calculation score is greater than the preset threshold, output the third semantic parsing result as the semantic parsing result; where the basic grammar word is the smallest unit that constitutes a sentence, the domain core word is a characteristic identifier of the domain intention, and the enhanced domain entity word, the ordinary domain entity word, and the weakened domain entity word are all used to identify the entity names within the domain, and the conflict degree of the enhanced domain entity word, the ordinary domain entity word, and the weakened domain entity word with the domain attribute increases in turn; Result output unit: Output the semantic parsing result.

Images