A data processing method, device, equipment and storage medium

Abstract

The present invention discloses a data processing method, device, equipment and storage medium, wherein the method comprises: obtaining input text, matching the input text with a target dictionary tree; the target dictionary tree is constructed based on the operation information of the operation object; the operation information comprises the name of the operation object, the object generalization text and the operation purpose; according to the matching result, the input text is annotated to obtain private domain corpus. The labor cost of annotating the private domain corpus can be reduced, and the efficiency of annotating the private domain corpus can be improved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the computer field, and in particular to a data processing method, apparatus, device and storage medium. Background Art

[0002] With the development of artificial intelligence, intelligent voice products have provided convenience for users, thus effectively improving work efficiency. However, in intelligent voice systems, manual annotation methods are often used to annotate private domain corpora, resulting in low annotation efficiency and high labor costs. Therefore, how to reduce the labor cost of annotating private domain corpora and improve the annotation efficiency of private domain corpora is a problem that needs to be solved. Summary of the invention

[0003] The present invention provides a data processing method, apparatus, device and storage medium, which can reduce the labor cost of annotating private domain corpus and improve the efficiency of annotating private domain corpus.

[0004] According to one aspect of the present invention, there is provided a data processing method, comprising:

[0005] Acquire input text, and match the input text with a target dictionary tree; the target dictionary tree is constructed based on operation information of the operation object; the operation information includes the name of the operation object, the object generalization text and the operation purpose;

[0006] According to the matching result, the input text is annotated to obtain private domain corpus.

[0007] According to another aspect of the present invention, there is provided a data processing device, the device comprising:

[0008] An input text matching module is used to obtain input text and match the input text with a target dictionary tree; the target dictionary tree is constructed based on the operation information of the operation object; the operation information includes the name of the operation object, the object generalization text and the operation purpose;

[0009] The corpus annotation module is used to annotate the input text according to the matching result to obtain private domain corpus.

[0010] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0011] at least one processor; and

[0012] a memory communicatively connected to the at least one processor; wherein,

[0013] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the data processing method described in any embodiment of the present invention.

[0014] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the data processing method described in any embodiment of the present invention when executed.

[0015] The technical solution of the embodiment of the present invention first obtains the input text, matches the input text with the target dictionary tree, and then annotates the input text according to the matching result to obtain the private domain corpus. The above solution annotates the input text with the target dictionary tree, which solves the problem of low annotation efficiency when manually annotating the input text. It realizes the effect of automatic annotation of private domain corpus and saves labor costs; in addition, the above solution improves the efficiency of corpus annotation while ensuring the accuracy of corpus annotation.

[0016] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 A flow chart of a data processing method provided in Embodiment 1 of the present invention;

[0019] Figure 2 A flow chart of a data processing method provided in Embodiment 2 of the present invention;

[0020] Figure 3 A flow chart of a data processing method provided in Embodiment 3 of the present invention;

[0021] Figure 4 A schematic diagram of the structure of a data processing device provided in Embodiment 4 of the present invention;

[0022] Figure 5 This is a schematic diagram of the structure of an electronic device provided in Embodiment 5 of the present invention. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0024] It should be noted that the terms "first", "second", "target", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "etc." and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0025] Embodiment 1

[0026] Figure 1 A flowchart of a data processing method is provided for the first embodiment of the present invention. This embodiment is applicable to the case of processing data, and is particularly applicable to the case of annotating input text to obtain private domain text. The method can be executed by a data processing device, which can be implemented in the form of hardware and / or software, and can be configured in an electronic device, such as an intelligent voice system in an electronic device. Figure 1 As shown, the method includes:

[0027] S110, obtaining input text, and matching the input text with the target dictionary tree.

[0028] The target dictionary tree is constructed based on the operation information of the operation object; the operation information includes the name of the operation object, the object generalization text and the purpose of the operation.

[0029] Among them, the input text is the corpus information that needs to be marked. For example, in the in-vehicle intelligent voice system, it is necessary to store a large amount of corpus information in the field of automobile control that has been marked, so that when the user makes a voice containing corpus information, the vehicle can be controlled based on the voice uttered by the user. Therefore, in the actual use of the in-vehicle intelligent voice system, it is necessary to add new corpus information in the field of automobile control according to the actual situation. The method of adding new corpus information in the field of automobile control can be to mark the corpus information in the field of automobile control in the input text, and store the marked input text in the in-vehicle intelligent voice system.

[0030] The operation object refers to the object that can be controlled by the intelligent voice system. In a vehicle, the operation object can be a vehicle structure such as windows, doors, air conditioners and lights. Object generalization text refers to the generalized vocabulary of the operation object name. For example, when the name of the operation object is door, the object generalization text may include: front door, car door and back door, etc. The operation purpose is the purpose to be achieved by the intelligent voice system controlling the operation object. For example, if the corpus information stored in the intelligent voice system is to open the car window, the name of the operation object in the corpus information is car window, and the operation purpose is to open the car window.

[0031] A dictionary tree, also known as a word search tree or a Trie tree, is a tree structure, such as a variant of a hash tree. A dictionary tree can be used to count, sort, and save a large number of strings. A dictionary tree can use the common prefix of strings to reduce query time, minimize unnecessary string comparisons, and have high query efficiency.

[0032] Optionally, a target dictionary tree is pre-built, and the target dictionary tree stores the operation object name identifier, the object generalized text identifier, and the operation purpose identifier. The target dictionary tree is stored in the intelligent voice system.

[0033] When the input text needs to be marked, the intelligent voice system obtains the input text and extracts the text operation object information from the input text. Text operation object information refers to the information of the operation object included in the input text. The text operation object information includes the text object name and the text operation purpose. The intelligent voice system performs prefix matching on the operation object information in the input text through the dictionary tree to obtain the matching result. When performing prefix matching, you can first perform prefix matching on the text object name in the input text through the dictionary tree to obtain the prefix matching result of the operation object name; then perform prefix matching on the text operation purpose in the input text through the target dictionary tree to obtain the prefix matching result of the operation purpose.

[0034] Exemplarily, the target dictionary tree can be constructed according to the name of the operation object, the object generalization text and the operation purpose. Specifically, it can be implemented through the following sub-steps:

[0035] S1101. Construct a first dictionary tree in a target dictionary tree according to an operation object, an object generalization text, and a first attribute value of the operation object.

[0036] The first attribute value of the operation object refers to a character string that can be used as the name identifier of the operation object, and the first attribute value of the operation object can be represented by value0. The first dictionary tree is one of the components of the target dictionary tree, and the first dictionary tree can be named handler1_trie.

[0037] Specifically, taking the intelligent voice system in the vehicle field as an example, the intelligent voice system can pre-acquire the operation object information in the vehicle field. The operation object information acquired by the intelligent voice system may be historical operation object information. After the intelligent voice system acquires the operation object information, it assigns an object key value to the operation object name, and at the same time assigns a first text key value to the object generalization text. Among them, the object key value can be used as an identifier of the operation object name, and the object key value can be a number, a character, or a keyword of the operation object name; the first text key value can be used as an identifier of the object generalization text, and the first text key value can be a number, a character, or a keyword of the object generalization text. The intelligent voice system acquires a unique identification string of the operation object name based on the operation object information, and uses the unique identification string of the operation object name as the first attribute value. Among them, the unique identification string of the operation object name can be generated based on the namespace of the operation object name and the random number in the operation object name.

[0038] Furthermore, the object key value and value0 are stored correspondingly in the nodes of handler1_trie, and the first text key value and value0 are stored correspondingly in the nodes of handler1_trie, thereby obtaining handler1_trie that stores the correspondence between the object key value and value0, and the correspondence between the first text key value and value0.

[0039] S1102: Construct a second dictionary tree in the target dictionary tree according to the operation purpose, the second attribute value of the operation object, and the purpose generalization text of the operation purpose.

[0040] The second trie can be named handler2_trie.

[0041] The second attribute value refers to a character string that can be used as an operation purpose identifier. The second attribute value can be represented by value1. The purpose generalization text refers to the generalized vocabulary of the operation purpose. For example, if the operation purpose is to open the door, the purpose generalization text may include open the door, please open the door, and please open the door.

[0042] Specifically, after the intelligent voice system obtains the operation object information, it assigns a purpose key value to the operation purpose, and assigns a second text key value to the purpose generalized text. The purpose key value can be used as an identifier of the operation purpose, and the purpose key value can be a number, a character, or a keyword of the operation purpose; the second text key value can be used as an identifier of the purpose generalized text, and the second text key value can be a number, a character, or a keyword of the purpose generalized text.

[0043] The purpose key value is concatenated with the unique identification string of the operation object name to obtain the purpose concatenation attribute value; at the same time, the second text key value is concatenated with the unique identification string of the operation object name to obtain the text concatenation attribute value. The intelligent voice system obtains the unique identification string of the operation purpose according to the operation object information, and uses the unique identification string of the operation purpose as the second attribute value. Among them, the unique identification string of the operation purpose can be generated according to the namespace of the operation purpose and the random number in the operation purpose.

[0044] Furthermore, the destination splicing attribute value and value1 are stored in correspondence in the nodes of handler2_trie, and the text splicing attribute value and value1 are stored in correspondence in the nodes of handler2_trie, thereby obtaining a handler2_trie storing the correspondence between the destination splicing attribute value and value1, and the correspondence between the text splicing attribute value and value1.

[0045] S1103: Generate a composite key value according to the operation object name and the operation purpose.

[0046] Specifically, the intelligent voice system concatenates the unique identification string of the operation object name and the unique identification string of the operation purpose, and uses the concatenation result as the composite key value.

[0047] S1104. Concatenate the combined key value and the keyword character to obtain a concatenated character, and construct a third dictionary tree in the target dictionary tree based on the concatenated character string.

[0048] Among them, the third dictionary tree can be named handler3_trie.

[0049] The keyword is a keyword of the operation purpose, and the keyword character refers to a unique identification string of the keyword.

[0050] Specifically, keywords are extracted from the operation purpose, and the keyword extraction criteria can be preset according to actual needs. According to the keyword of the operation purpose, a unique identification string of the keyword is obtained, and the unique identification string of the keyword is used as the keyword character. The unique identification string of the keyword can be generated according to the namespace of the keyword and the random number in the keyword. The combined key value and the keyword character are spliced ​​to obtain the spliced ​​character. The spliced ​​character is stored in the node of handler3_trie, and the handler3_trie storing the spliced ​​character can be obtained.

[0051] S1105. Construct a fourth dictionary tree in the target dictionary tree according to the correspondence between the keyword key value and the keyword character.

[0052] Among them, the fourth dictionary tree can be named handler4_trie.

[0053] Specifically, a keyword key value is assigned to the keyword, and the keyword key value can be a number or a character. The keyword key value and the keyword character are stored in the node of handler4_trie in correspondence, and the handler4_trie storing the correspondence between the keyword key value and the keyword character can be obtained.

[0054] It should be noted that this embodiment constructs a target dictionary tree according to the operation object name, object generalization text and operation purpose, and can construct a target dictionary tree that stores the operation object information, which is convenient for subsequent corpus annotation of the input text through the target dictionary tree.

[0055] S120: Based on the matching results, the input text is annotated to obtain private domain corpus.

[0056] Among them, private domain refers to a carrier of private data on the Internet. Private domain corpus refers to corpus data in a specific field.

[0057] Specifically, after the intelligent voice system obtains the operation object name prefix matching result and the operation purpose prefix matching result from the target dictionary tree, it performs corpus annotation on the text object name in the input text and the text purpose in the input text respectively. The method for performing corpus annotation on the text object name in the input text may be to replace the text object name prefix in the input text with the operation object name prefix matching result that matches the input text in the target dictionary tree; the method for performing corpus annotation on the operation purpose in the input text may be to replace the text purpose prefix in the input text with the operation purpose prefix matching result that matches the input text in the target dictionary tree. The input text after corpus annotation is stored in the target dictionary tree as private domain corpus.

[0058] The technical solution provided in this embodiment first obtains the input text, matches the input text with the target dictionary tree, and then annotates the input text according to the matching result to obtain the private domain corpus. The above solution annotates the input text with the target dictionary tree, which solves the problem of low annotation efficiency when manually annotating the input text. It realizes the effect of automatic annotation of private domain corpus and saves labor costs; in addition, the above solution improves the efficiency of corpus annotation while ensuring the accuracy of corpus annotation.

[0059] Exemplarily, based on the above embodiment, while annotating the input text to obtain private domain corpus, voice data can also be obtained, and the voice data can be compared with the private domain corpus for consistency. According to the consistency comparison result, the operation information of the specified object in the voice data can be determined.

[0060] The voice data is the instruction information sent by the user to the operation object.

[0061] Specifically, after receiving the voice data sent by the user, the intelligent voice system performs voice data analysis on the voice data, determines the operation object information contained in the voice data according to the analysis result, and uses the operation object information contained in the voice data as the voice object information. The voice object information is compared with the operation object information contained in the private domain corpus stored in the target dictionary tree for consistency. If the operation object information is consistent with the voice object information, the operation object name and operation purpose in the operation object information consistent with the voice object information are determined, and the operation object is controlled based on the voice data.

[0062] It should be noted that by obtaining voice data, comparing the voice data with private domain corpus for consistency, and determining the operation information of the specified object in the voice data based on the consistency comparison result, the user's operation needs can be met based on the user's voice data, saving the user's time and providing the user with a more flexible control method.

[0063] Embodiment 2

[0064] Figure 2 This is a flow chart of a data processing method provided in the second embodiment of the present invention. This embodiment is optimized on the basis of the above embodiment, and provides a preferred embodiment of using the first dictionary tree, the second dictionary tree, the third dictionary tree and the fourth dictionary tree to match the input text. Specifically, Figure 2 As shown, the method includes:

[0065] S210: Obtain input text, use the first dictionary tree to match the input text, and obtain a matching object.

[0066] Specifically, after receiving the input text, the intelligent voice system extracts text operation information from the input text, and performs prefix matching on the text object name in the text operation information and the operation object information stored in handler1_trie. If the text object name is consistent with the operation object name corresponding to value0 stored in handler1_trie, the operation object corresponding to value0 is used as the matching object, and the matching object is stored in the storage space of the intelligent voice system.

[0067] S220: If the matching object is unique, the second dictionary tree is used to match the input text to obtain the matching object.

[0068] Specifically, if the intelligent voice system determines that the input text contains only one matching object based on the matching result of handler1_trie and the input text, the text purpose in the text operation information is prefix matched with the operation object information stored in handler2_trie, where the text purpose is the operation purpose contained in the input text. If the text purpose is consistent with the operation purpose corresponding to value1 stored in handler2_trie, the operation purpose corresponding to value1 is used as the matching purpose, and the matching purpose is stored in the storage space of the intelligent voice system.

[0069] S230: If the matching purpose is unique, use the fourth dictionary tree to match the input text to obtain matching keywords.

[0070] Specifically, if the intelligent voice system determines that the input text contains only one matching purpose based on the matching result of handler2_trie and the input text, the operation object information in handler4_trie is prefix matched with the text keyword in the text operation information, and the text keyword is the keyword extracted from the text purpose. If the text keyword is consistent with the keyword corresponding to the keyword character stored in handler4_trie, the keyword corresponding to the keyword character is used as the matching keyword, and the matching keyword is stored in the storage space of the intelligent voice system.

[0071] S240: Acquire matching characters from the third dictionary tree according to the matching object name, matching purpose and matching keyword.

[0072] Specifically, after the intelligent voice system obtains the matching object name, matching purpose and matching keyword, it concatenates the matching object unique identification string, the matching purpose unique identification string and the matching keyword unique identification string, and uses the concatenation result as the matching integration character. Then, the matching integration character is matched with the concatenation character in handler3_trie. If the matching integration character is consistent with the concatenation character in handler3_trie, the concatenation character consistent with the matching integration character is used as the matching character.

[0073] S250: Annotate the input text according to the matching characters to obtain private domain corpus.

[0074] Specifically, the matching characters are obtained, and the combined key values ​​and keyword characters contained in the matching characters can be determined according to the matching characters. The input text is annotated according to the matching characters, and the annotated input text is used as the private domain corpus.

[0075] Exemplarily, the method of annotating the input text according to the matching characters may be to replace the matching object name and the matching purpose with the combined key value contained in the matching characters, and to replace the matching keyword with the keyword character contained in the matching characters.

[0076] Preferably, when the number of matching objects is at least two, a target object needs to be selected; and the input text is annotated according to the target object. Specifically, this can be achieved through the following sub-steps:

[0077] S2501: When the number of matching objects in the matching result is at least two, determine whether there is a profile relationship between all matching objects.

[0078] Specifically, if the intelligent voice system obtains more than one matching object, it is necessary to further determine whether there is a profile relationship between the obtained matching objects. For example, if the intelligent voice system obtains two matching objects, namely, the door and the second row of doors, there is a profile relationship between the matching objects; if the intelligent voice system obtains two matching objects, namely, the door and the window, there is no profile relationship between the matching objects.

[0079] S2502: If a profile relationship exists, the matching object with the longest character among all matching objects is taken as the target object.

[0080] Specifically, if the number of matching objects is at least two, and there is a profile relationship between all matching objects, the matching object with the longest characters among all matching objects is selected, and the matching object with the longest characters is used as the target matching object. For example, if the matching objects are: window, rear window, and right rear window, the matching object with the longest characters is determined to be the right rear window, and the right rear window can be used as the target object.

[0081] S2503. According to the target object, the input text is annotated to obtain private domain corpus.

[0082] Specifically, after determining the target object, the other matching objects except the target object in the matching objects included in the input text are deleted to obtain the target input text containing only one matching object. The target input text is annotated to obtain the private domain corpus.

[0083] Optionally, if the number of matching objects in the matching result is at least two, and there are matching objects with or without a matching relationship, the input text is forwarded to a manual annotation platform for corpus annotation.

[0084] It should be noted that after obtaining the matching results, the number of matching objects in the matching results is determined, and when the number of matching objects is at least two, the target object is screened out from the matching objects based on the profile relationship between the matching objects, and the input text is annotated according to the target object. This can ensure that when the input text contains multiple matching objects, the input text can be accurately annotated.

[0085] The technical solution of this embodiment is to obtain the input text, use the first dictionary tree to match the input text, and obtain the matching object; if the matching object is unique, use the second dictionary tree to match the input text to obtain the matching purpose; if the matching purpose is unique, use the fourth dictionary tree to match the input text to obtain the matching keywords; according to the matching object, matching purpose and matching keywords, obtain the matching characters from the third dictionary tree; according to the matching results, perform corpus annotation on the input text to obtain the private domain corpus. The above scheme matches the input text according to the first dictionary tree, the second dictionary tree, the third dictionary tree and the fourth dictionary tree respectively, and performs corpus annotation on the input text according to the matching results, which improves the accuracy of corpus annotation while ensuring the efficiency of corpus annotation.

[0086] Exemplarily, based on the matching results, the input text is annotated with corpus, and the private domain corpus can be obtained by, when the number of matching purposes in the matching results is at least two, determining whether there is a profile relationship between all the matching purposes; if there is a profile relationship, taking the matching purpose with the longest characters among all the matching purposes as the target purpose; based on the target purpose, the input text is annotated with corpus to obtain the private domain corpus.

[0087] Specifically, if the intelligent voice system obtains more than one matching purpose, it is necessary to further determine whether there is a profile relationship between the obtained matching purposes. For example, if the intelligent voice system obtains two matching purposes, namely, opening the window and opening the second row of windows, there is a profile relationship between the matching purposes; if the intelligent voice system obtains two matching purposes, namely, opening the door and opening the second row of windows, there is no profile relationship between the matching purposes.

[0088] If the number of matching purposes is at least two, and there is a profile relationship between all matching purposes, the matching purpose with the longest characters among all matching purposes is selected, and the matching purpose with the longest characters is used as the target matching purpose. For example, if the matching purposes are: open window, open rear window, and open right rear window, the matching object with the longest characters is determined to be open right rear window, and in this case, open right rear window can be used as the target purpose.

[0089] After determining the target purpose, the matching purposes other than the target purpose are deleted from the matching purposes contained in the input text, and the target input text containing only one matching purpose is obtained. The target input text is annotated with corpus to obtain the private domain corpus. The method for annotating the target input text with corpus is consistent with the method for annotating the input text with corpus in the above embodiment, and will not be repeated here.

[0090] Optionally, if the number of matching purposes in the matching result is at least two, and there are matching purposes for which there is no matching relationship, the input text is forwarded to a manual annotation platform for corpus annotation.

[0091] It should be noted that after obtaining the matching results, the number of matching purposes in the matching results is determined, and when the number of matching purposes is at least two, the target purposes are screened out from the matching purposes based on the profile relationship between the matching purposes, and the input text is annotated according to the target purposes. This can ensure that when the input text contains multiple matching purposes, the input text can be accurately annotated.

[0092] Optionally, in this embodiment, the method for matching the input text with the target dictionary tree may also be: extracting text object information from the input text based on a regular expression; and matching the extracted text object information with the target dictionary tree.

[0093] Among them, regular expressions are commonly used speech templates. The intelligent voice system can construct regular expressions based on the existing operation object information. The regular expression can be one or more according to the actual situation.

[0094] Specifically, after the intelligent voice system obtains the input text, it compares the input text with a pre-built regular expression. If the input text has a matching relationship with the regular expression, the text object name that matches the regular expression in the text object information is extracted as the regular matching name, the text purpose that matches the regular expression is extracted as the regular matching purpose, and the text keyword that matches the regular expression is extracted as the regular matching keyword.

[0095] The regular matching name is compared with the matching object, the regular matching purpose is compared with the matching purpose, and the regular matching keyword is compared with the matching keyword. If the regular matching name is consistent with the matching object, the regular matching purpose is consistent with the matching purpose, and the regular matching keyword is consistent with the matching keyword, the input text is annotated to obtain the private domain corpus.

[0096] Embodiment 3

[0097] Figure 3 This is a flowchart of a data processing method provided in the third embodiment of the present invention. This embodiment is optimized on the basis of the above embodiment, and provides a preferred embodiment of determining the relevance between the input text and the noise word library and the enhanced word library according to the matching results, obtaining the enhanced text according to the related noise words and enhanced words, and then annotating the enhanced text. Specifically, Figure 3 As shown, the method includes:

[0098] S310, obtaining input text, and matching the input text with the target dictionary tree.

[0099] The target dictionary tree is constructed based on the operation information of the operation object; the operation information includes the name of the operation object, the object generalization text and the purpose of the operation.

[0100] S320: Determine, according to the matching result, an enhancement word in the enhancement word library that is associated with the input text.

[0101] The enhanced word library refers to a database used to store the correspondence between the denoised text data and enhanced words. Enhanced words are emphatic words associated with the text data. For example, enhanced words for car windows may include: back row, right, left, and front row.

[0102] Specifically, the intelligent speech system matches the input text with the target dictionary tree to obtain the matching object, matching purpose and matching keyword in the input text. Further, the matching object, matching purpose and matching keyword in the input text are compared with the text data in the enhanced database to determine whether there is text data in the enhanced database that is consistent with the text object information in the input text.

[0103] S330: Add the enhancement word to the input text to obtain the enhanced text.

[0104] Specifically, the matching object is matched with the text data stored in the enhanced word library. If the matching object is consistent with the text data, the enhanced word corresponding to the text data consistent with the matching object is added to the matching object to obtain enhanced object data; the matching purpose is matched with the text data stored in the enhanced word library. If the matching purpose is consistent with the text data, the enhanced word corresponding to the text data consistent with the matching purpose is added to the matching purpose to obtain enhanced purpose data; the matching keyword is matched with the text data stored in the enhanced word library. If the matching keyword is consistent with the text data, the enhanced word corresponding to the text data consistent with the matching keyword is added to the matching keyword to obtain enhanced keyword data. The enhanced text is obtained according to the enhanced object data, the enhanced purpose data and the enhanced keyword data.

[0105] S340: Annotate the enhanced text to obtain private domain corpus.

[0106] Specifically, the intelligent voice system obtains the operation object name prefix matching result and the operation purpose prefix matching result from the target dictionary tree, and then respectively performs corpus annotation on the enhanced object data and enhanced purpose data in the enhanced text. The method for performing corpus annotation on the enhanced object data in the enhanced text may be to replace the enhanced object data prefix in the enhanced text with the operation object name prefix matching result that matches the enhanced text in the target dictionary tree; the method for performing corpus annotation on the enhanced purpose data in the enhanced text may be to replace the enhanced purpose data prefix in the enhanced text with the operation purpose prefix matching result that matches the enhanced text in the target dictionary tree. The enhanced text after corpus annotation is stored in the target dictionary tree as private domain corpus.

[0107] Optionally, a noise word library containing common noise data may be pre-built, and the common noise data may include common request words, modal particles, and numbers. For example, if the input text is "Please help me open the door", the noise data in the input text is: Please help me. After obtaining the enhanced text, the noise words in the noise word library may be randomly added to the head of the input text and / or the tail of the input text to form a new private domain corpus.

[0108] The technical solution of this embodiment determines the enhancement words in the enhancement word library that are related to the input text according to the matching results; adds the enhancement words related to the input text to the input text to obtain the enhanced text; and annotates the enhanced text to obtain the private domain corpus. The above solution can expand the input text by adding enhancement words to the input text to form an enhanced text, and annotate the enhanced text in advance, thereby increasing the integrity and diversity of the private domain corpus.

[0109] Embodiment 4

[0110] Figure 4 This is a schematic diagram of the structure of a data processing device provided in Embodiment 4 of the present invention. This embodiment is applicable to the case of processing data. Figure 4 As shown, the data processing device includes: an input text matching module 410 and a corpus annotation module 420.

[0111] The input text matching module 410 is used to obtain the input text and match the input text with the target dictionary tree; the target dictionary tree is constructed based on the operation information of the operation object; the operation information includes the name of the operation object, the object generalization text and the operation purpose;

[0112] The corpus annotation module is used to annotate the input text according to the matching results to obtain private domain corpus.

[0113] The technical solution provided in this embodiment first obtains the input text, matches the input text with the target dictionary tree, and then annotates the input text according to the matching result to obtain the private domain corpus. The above solution annotates the input text with the target dictionary tree, which solves the problem of low annotation efficiency when manually annotating the input text. It realizes the effect of automatic annotation of private domain corpus and saves labor costs; in addition, the above solution improves the efficiency of corpus annotation while ensuring the accuracy of corpus annotation.

[0114] Furthermore, the above data processing device also includes:

[0115] A first dictionary tree construction module, used to construct a first dictionary tree in the target dictionary tree according to the operation object, the object generalization text and the first attribute value of the operation object;

[0116] The second dictionary tree construction module is used to construct the second dictionary tree in the target dictionary tree according to the operation purpose, the second attribute value of the operation object and the purpose generalization text of the operation purpose

[0117] A composite key value generation module is used to generate a composite key value according to the operation object name and the operation purpose;

[0118] A third dictionary tree construction module is used to concatenate the combined key value with the keyword character to obtain a concatenated character, and to construct a third dictionary tree in the target dictionary tree based on the concatenated character string;

[0119] The fourth dictionary tree construction module is used to construct a fourth dictionary tree in the target dictionary tree according to the corresponding relationship between the keyword key value and the keyword character.

[0120] Exemplarily, the input text matching module 410 is specifically used for:

[0121] The first dictionary tree is used to match the input text to obtain a matching object;

[0122] If the matching object is unique, the second dictionary tree is used to match the input text to obtain the matching object;

[0123] If the matching purpose is unique, the fourth dictionary tree is used to match the input text to obtain matching keywords;

[0124] According to the matching object, matching purpose and matching keyword, matching characters are obtained from the third dictionary tree.

[0125] Exemplarily, the corpus annotation module 420 is specifically used for:

[0126] When the number of matching objects in the matching result is at least two, determining whether there is a profile relationship between all matching objects;

[0127] If there is a profile relationship, the matching object with the longest character among all matching objects is taken as the target object;

[0128] According to the target object, the input text is annotated to obtain private domain corpus.

[0129] Furthermore, the corpus annotation module 420 is specifically used for:

[0130] When the number of matching purposes in the matching result is at least two, determining whether there is a profile relationship between all matching purposes;

[0131] If there is a general relationship, the matching target with the longest character among all matching targets is taken as the target target;

[0132] According to the target purpose, the input text is annotated to obtain private domain corpus.

[0133] Exemplarily, the corpus annotation module 420 is further used for:

[0134] According to the matching results, determining the enhancement words in the enhancement word library that are associated with the input text;

[0135] Adding enhancement words to the input text to obtain enhanced text;

[0136] Annotate the enhanced text to obtain private domain corpus.

[0137] Exemplarily, the above data processing device further includes:

[0138] A voice data acquisition module, used for acquiring voice data;

[0139] The voice data comparison module is used to compare the voice data with the private domain corpus for consistency;

[0140] The operation information determination module is used to determine the operation information of the specified object in the voice data according to the consistency comparison result.

[0141] The data processing device provided in this embodiment can be applied to the data processing method provided in any of the above embodiments, and has corresponding functions and beneficial effects.

[0142] Embodiment 5

[0143] Figure 5 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0144] like Figure 5 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0145] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0146] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as a data processing method.

[0147] In some embodiments, the data processing method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the data processing method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the data processing method in any other appropriate manner (e.g., by means of firmware).

[0148] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0149] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0150] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0151] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0152] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0153] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0154] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0155] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A data processing method, characterized in that: include: Acquire input text, and match the input text with a target dictionary tree; the target dictionary tree is constructed based on operation information of the operation object; The operation information includes the name of the operation object, the object generalization text and the operation purpose; According to the matching results, the input text is annotated to obtain private domain corpus; The method further comprises: Constructing a first dictionary tree in the target dictionary tree according to the operation object, the object generalization text and the first attribute value of the operation object; Constructing a second dictionary tree in the target dictionary tree according to the operation purpose, the second attribute value of the operation object and the purpose generalization text of the operation purpose; Generate a composite key value according to the operation object name and the operation purpose; Concatenate the combined key value and the keyword character to obtain a concatenated character, and construct a third dictionary tree in the target dictionary tree based on the concatenated character string; Constructing a fourth dictionary tree in the target dictionary tree according to the correspondence between the keyword key value and the keyword character; The step of matching the input text with the target dictionary tree includes: Using the first dictionary tree to match the input text to obtain a matching object; If the matching object is unique, the second dictionary tree is used to match the input text to obtain the matching object; If the matching purpose is unique, the fourth dictionary tree is used to match the input text to obtain matching keywords; According to the matching object, the matching purpose and the matching keyword, matching characters are acquired from the third dictionary tree.

2. The method according to claim 1, characterized in that The step of annotating the input text according to the matching result to obtain private domain corpus includes: When the number of matching objects in the matching result is at least two, determining whether there is a profile relationship between all matching objects; If there is a profile relationship, the matching object with the longest character among all matching objects is taken as the target object; According to the target object, the input text is annotated to obtain private domain corpus.

3. The method according to claim 1, characterized in that The step of annotating the input text according to the matching result to obtain private domain corpus includes: When the number of matching purposes in the matching result is at least two, determining whether there is a profile relationship between all matching purposes; If there is a general relationship, the matching target with the longest character among all matching targets is taken as the target target; According to the target purpose, the input text is annotated to obtain private domain corpus.

4. The method according to claim 1, characterized in that: The step of annotating the input text according to the matching result to obtain private domain corpus includes: According to the matching results, determining the enhancement words in the enhancement word library that are associated with the input text; Adding the enhancement word to the input text to obtain an enhanced text; The enhanced text is annotated to obtain private domain corpus.

5. The method according to claim 1, characterized in that Also includes: Get voice data; Performing consistency comparison between the voice data and the private domain corpus; According to the consistency comparison result, the operation information of the specified object in the voice data is determined.

6. A data processing device, characterized in that: include: An input text matching module is used to obtain input text and match the input text with a target dictionary tree; the target dictionary tree is constructed based on the operation information of the operation object; the operation information includes the name of the operation object, the object generalization text and the operation purpose; A corpus annotation module, used to annotate the input text according to the matching result to obtain private domain corpus; The device also includes: A first dictionary tree construction module, used to construct a first dictionary tree in the target dictionary tree according to the operation object, the object generalization text and the first attribute value of the operation object; A second dictionary tree construction module is used to construct a second dictionary tree in the target dictionary tree according to the operation purpose, the second attribute value of the operation object and the purpose generalization text of the operation purpose; A combination key value generating module, used for generating a combination key value according to the operation object name and the operation purpose; A third dictionary tree construction module is used to concatenate the combined key value with the keyword character to obtain a concatenated character, and to construct a third dictionary tree in the target dictionary tree based on the concatenated character string; A fourth dictionary tree construction module, used to construct a fourth dictionary tree in the target dictionary tree according to the correspondence between the keyword key value and the keyword character; Wherein, the input text matching module is specifically used for: Using the first dictionary tree to match the input text to obtain a matching object; If the matching object is unique, the second dictionary tree is used to match the input text to obtain the matching object; If the matching purpose is unique, the fourth dictionary tree is used to match the input text to obtain matching keywords; According to the matching object, the matching purpose and the matching keyword, matching characters are acquired from the third dictionary tree.

7. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the data processing method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the data processing method according to any one of claims 1 to 5 when executed.

Images