Method and device for detecting keyword error extraction in imprecise text

Abstract

The present application discloses a method and apparatus for detecting incorrect keyword extraction from imprecise text. The method comprises: obtaining a target text to be detected and a target keyword corresponding to the target text, wherein the target text is text data with noise, and the target keyword is a keyword obtained by extracting keywords from the target text using keyword extraction technology; determining keyword information and context information of the target keyword, wherein the keyword information includes each keyword of the target keyword, keyword position information, and keyword segmentation information, and the context information includes the current sentence, the previous sentence, and the next sentence of the current sentence in which the target keyword is located; determining a keyword feature vector of the keyword information and a context feature vector of the context information; and inputting the keyword feature vector and the context feature vector into a preset detection model, thereby determining whether the target keyword is incorrectly extracted based on the output result of the detection model.

Description

Technical Field

[0001] The present application relates to the technical field of text detection, and in particular to a method and apparatus for detecting incorrect keyword extraction in imprecise text. Background Art

[0002] Keyword extraction involves extracting the most interesting elements from natural language text, enabling rapid identification of information topics. It is a crucial foundational technology in the field of natural language processing (NLP). In addition to directly outputting keywords to end users, it also provides valuable information for a wide range of downstream tasks, such as information retrieval, sentiment analysis, knowledge graphs, public opinion monitoring, complaint analysis, document summarization, and voice quality inspection. Years of development in various fields have resulted in the accumulation of diverse and rich keyword libraries and corresponding systems. For example, knowledge graphs may prioritize keywords in the form of named entities (personal names, organization names, and place names), while document summarization tends to favor technical terms. Sentiment / public opinion / quality inspection focuses on profanity, insults, and sensitive terms (e.g., "regulation," "315," and "contact family"). Leveraging this domain knowledge, whether through machine learning or direct matching, can more or less meet business needs.

[0003] However, with the increasing number of text sources, such as ASR-translated text and user input errors, real-world text inevitably introduces noise, such as homophones and similar characters. This poses a challenge to keyword extraction. Current solutions include: 1. Correcting errors before extracting keywords; 2. Making do with errors and tagging noisy text with keywords, aiming to first identify noisy keywords and then match them with normal keywords; 3. End-to-end direct recognition. For example, in voice scenarios, this approach abandons the model of recognizing text first and then identifying keywords, and instead outputs keywords directly from the voice. Each of these approaches has different drawbacks: 1. They rely on error correction, which typically targets the entire text, while our goal is simply to extract keywords, resulting in a waste of effort; 2. They are time-consuming and labor-intensive, not leveraging existing keyword libraries, and different ASR engines may introduce different noise errors, making annotations difficult to generalize; 3. The technology is immature, involving modeling and parameter tuning, requiring a long cycle and limited generalizability.

[0004] With respect to the technical problem in the prior art mentioned above that there is a lack of a method that can quickly, accurately and specifically detect incorrect keyword extraction from imprecise texts, no effective solution has been proposed yet. Summary of the Invention

[0005] The embodiments of the present disclosure provide a method and apparatus for detecting incorrect keyword extraction from imprecise text, so as to at least solve the technical problem in the prior art of lacking a method that can quickly, accurately and specifically detect incorrect keyword extraction from imprecise text.

[0006] According to one aspect of an embodiment of the present disclosure, a method for detecting incorrect keyword extraction of imprecise text is provided, comprising: obtaining a target text to be detected and a target keyword corresponding to the target text, wherein the target text is text data with noise, and the target keyword is a keyword obtained by extracting keywords from the target text using keyword extraction technology; determining keyword information of the target keyword and context information of the target keyword, wherein the keyword information includes each keyword of the target keyword, location information of the keyword, and segmentation information of the keyword, and the context information includes the current sentence, the previous sentence, and the next sentence of the current sentence in which the target keyword is located; determining a keyword feature vector of the keyword information and a context feature vector of the context information; and inputting the keyword feature vector and the context feature vector into a preset detection model, thereby determining whether the target keyword is extracted incorrectly based on the output result of the detection model.

[0007] According to another aspect of an embodiment of the present disclosure, a storage medium is further provided, the storage medium including a stored program, wherein when the program is run, a processor executes any one of the above methods.

[0008] According to another aspect of an embodiment of the present disclosure, a device for detecting incorrect keyword extraction of imprecise text is also provided, including: an acquisition module, used to acquire a target text to be detected and a target keyword corresponding to the target text, wherein the target text is text data with noise, and the target keyword is a keyword obtained by extracting the target text by using keyword extraction technology; a first determination module, used to determine the keyword information of the target keyword and the context information of the target keyword, wherein the keyword information includes each keyword of the target keyword, the position information of the keyword and the segmentation information of the keyword, and the context information includes the current sentence, the previous sentence and the next sentence of the current sentence where the target keyword is located; a second determination module, used to determine the keyword feature vector of the keyword information and the context feature vector of the context information; and a judgment module, used to input the keyword feature vector and the context feature vector into a preset detection model, so as to judge whether the target keyword is extracted incorrectly according to the output result of the detection model.

[0009] According to another aspect of an embodiment of the present disclosure, a device for detecting incorrect keyword extraction of imprecise text is also provided, comprising: a processor; and a memory connected to the processor, for providing the processor with instructions for processing the following processing steps: obtaining a target text to be detected and a target keyword corresponding to the target text, wherein the target text is text data with noise, and the target keyword is a keyword obtained by extracting keywords from the target text using keyword extraction technology; determining keyword information of the target keyword and context information of the target keyword, wherein the keyword information includes each keyword of the target keyword, keyword position information, and keyword segmentation information, and the context information includes the current sentence, the previous sentence, and the next sentence of the current sentence in which the target keyword is located; determining a keyword feature vector of the keyword information and a context feature vector of the context information; and inputting the keyword feature vector and the context feature vector into a preset detection model, thereby determining whether the target keyword is extracted incorrectly based on the output result of the detection model.

[0010] In the disclosed embodiment, a method for detecting erroneous extraction is proposed to optimize the keyword extraction in imprecise text scenarios (such as but not limited to text noise caused by ASR translation). The method does not invade upstream and downstream modules and can reuse existing mature technologies (such as ASR, keyword extraction) and accumulated keyword libraries to the greatest extent. Because the keyword extraction results are directly detected, no modification is required upstream; and the problem of erroneous extraction is solved in a focused manner, with strong pertinence and guaranteed effect. From the above content, it can be seen that by introducing contextual information and specific keyword text and position information, the detection model can find that the names of ordinary people are easily misrecognized by ASR through learning some examples, so erroneous keywords are likely to appear during verification; in addition, keywords generally appear in similar semantic segments in the context. If they are isolated semantic blocks, they may be misrecognized. Therefore, contextual information needs to be introduced to assist in determining whether the keyword is extracted incorrectly. Therefore, the detection method proposed in this embodiment does not invade the upstream and downstream modules, can reuse existing mature technologies (such as ASR, keyword extraction, etc.) and accumulated keyword libraries to the greatest extent, and focus on solving the problem of erroneous extraction. It is highly targeted and has a high detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of this application. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation of the present disclosure. In the drawings:

[0012] Figure 1 is a hardware structure block diagram of a computing device for implementing the method according to embodiment 1 of the present disclosure;

[0013] Figure 2is a flowchart of a method for detecting incorrect keyword extraction from imprecise text according to the first aspect of embodiment 1 of the present disclosure;

[0014] Figure 3 is a schematic diagram of the structural framework of the detection model according to Example 1 of the present disclosure;

[0015] Figure 4 This is a schematic diagram of vectorized representation of each word, position information of each word, and segmentation information according to Embodiment 1 of the present disclosure;

[0016] Figure 5 is a schematic diagram of an apparatus for detecting incorrect keyword extraction from imprecise text according to embodiment 2 of the present disclosure; and

[0017] Figure 6 3 is a schematic diagram of an apparatus for detecting incorrect keyword extraction from imprecise text according to embodiment 3 of the present disclosure. DETAILED DESCRIPTION

[0018] In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present disclosure.

[0019] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0020] First, some nouns or terms that appear in the description of the embodiments of the present disclosure are subject to the following explanations:

[0021] Term 1: ASR (Automatic Speech Recognition) is a technology that converts human speech into text.

[0022] Term 2: Embedding refers to converting a word or a character (token) into a vector representation.

[0023] Term 3: BERT (Bidirectional Encoder Representations from Transformers) is a pre-trained model for natural language processing.

[0024] Example 1

[0025] According to this embodiment, an embodiment of a method for detecting incorrect keyword extraction from imprecise text is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0026] The method embodiment provided in this embodiment can be executed in a server or similar computing device. Figure 1 FIG1 shows a hardware structure block diagram of a computing device for implementing a method for detecting incorrect keyword extraction from imprecise text. Figure 1 As shown, the computing device may include one or more processors (the processor may include but is not limited to a microprocessor MCU or a programmable logic device FPGA, etc.), a memory for storing data, and a transmission device for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply and / or a camera. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0027] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuitry." The data processing circuitry may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuitry may be a single, independent processing module, or may be incorporated in whole or in part into any of the other components of the computing device. As described in the embodiments of the present disclosure, the data processing circuitry serves as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0028] The memory can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the method for detecting incorrect keyword extraction of imprecise text in the embodiment of the present disclosure. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, implementing the method for detecting incorrect keyword extraction of imprecise text in the above-mentioned application. The memory may include a high-speed random access memory and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely located relative to the processor, and these remote memories may be connected to the computing device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0029] The transmission device is used to receive or send data via a network. Specific examples of the aforementioned network may include a wireless network provided by a communications provider of the computing device. In one embodiment, the transmission device includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0030] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computing device.

[0031] It should be noted that, in some optional embodiments, the above Figure 1 The computing device shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware and software elements. Figure 1 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the computing device described above.

[0032] In the above operating environment, according to the first aspect of this embodiment, a method for detecting incorrect keyword extraction from imprecise text is provided. Figure 2 A schematic diagram of the process is shown in FIG. Figure 2 As shown, the method includes:

[0033] S201: Obtain a target text to be detected and a target keyword corresponding to the target text, wherein the target text is text data with noise, and the target keyword is a keyword extracted from the target text using a keyword extraction technology;

[0034] S202: Determine the keyword information and context information of the target keyword. The keyword information includes each keyword of the target keyword, the position information of the keyword, and the segmentation information of the keyword. The context information includes the current sentence where the target keyword is located, the previous sentence and the next sentence of the current sentence.

[0035] S203: Determine the keyword information and context information of the target keyword. The keyword information includes each keyword of the target keyword, the position information of the keyword, and the segmentation information of the keyword. The context information includes the current sentence where the target keyword is located, the previous sentence and the next sentence of the current sentence; and

[0036] S204: Input the keyword feature vector and context feature vector into a preset detection model, and then determine whether the extraction of the target keyword is incorrect according to the output result of the detection model.

[0037] Specifically, in the process of detecting incorrect extraction in this embodiment, first determine the target text to be detected and the target keyword corresponding to the target text. Among them, the target text is text data with noise. For example, the noise in the target text can come from text noise caused by ASR technology translation, or from text noise caused by OCR technology recognition, or can also be from manual input errors, etc. There is no specific limitation here. And, the target keyword is a keyword obtained by using keyword extraction technology to extract keywords from the target text.

[0038] The following uses a specific example to illustrate how this embodiment detects incorrect keyword extraction:

[0039] Among them, the target text to be detected is as follows:

[0040] Agent: Seat: Ah, for the consumption company, may I ask if you are Ms. Weihaimei?

[0041] Customer: Huaihai has no culture and is not here.

[0042] Agent: Hello

[0043] The keyword obtained by using keyword extraction technology to extract keywords from the above target text is "has no culture" in the second sentence.

[0044] Therefore, after obtaining the target text and target keywords, this embodiment needs to determine the keyword information of the target keyword and the context information of the target keyword. Among them, the keyword information includes each keyword of the target keyword, the position information of the keyword, and the segmentation information of the keyword. Therefore, according to the above example, it can be determined that the keywords included in the keyword information of the target keyword "no culture" are the three keywords "no", "culture", and "culture". The position information of these three keywords are respectively the 26th, 27th, and 28th words in the target text, and the segmentation information of these three keywords is in the keyword part. In addition, the context information of the target keyword includes the current sentence where the target keyword is located, the previous sentence and the next sentence of the current sentence. Therefore, according to the above example, it can be determined that the current sentence where the target keyword is located is "Customer: Huaihai is not cultured and is not here", the previous sentence of the target keyword is "Agent: Ah, I am from the consumer management company. May I ask if you are Ms. Weihaimei?", and the next sentence of the target keyword is "Agent: Hello".

[0045] Furthermore, this embodiment also needs to determine the keyword feature vector of the keyword information and the context feature vector of the context information, and then input the determined keyword feature vector and context feature vector into a preset detection model, so as to determine whether the target keyword is extracted incorrectly based on the output result of the detection model. The detection model is a model trained using sample noise text with information on whether the keyword extraction is incorrect or not. Therefore, referring to Figure 3 As shown in FIG, after the keyword feature vector and the context feature vector are input into the preset detection model, the detection model can output a classification label. A classification label of 1 output by the detection model indicates that the target keyword is incorrectly extracted, and a classification label of 0 output indicates that the target keyword is correctly extracted.

[0046] Therefore, the detection method proposed in this embodiment directly detects the keyword extraction results and does not require any upstream modifications. Therefore, this method does not invade the upstream and downstream modules, and focuses on solving the problem of incorrect keyword extraction from imprecise texts. It is highly targeted and can reuse existing mature technologies (such as ASR technology, keyword extraction technology) and accumulated keyword libraries to the greatest extent. In addition, by introducing contextual information and specific keyword text and location information, the detection model can find that the names of ordinary people are easily misrecognized by ASR through learning from some samples, so that it can quickly and accurately detect incorrectly extracted keywords. This solves the technical problem in the prior art of lacking a method that can quickly, accurately and specifically detect incorrect keyword extraction from imprecise texts.

[0047] Optionally, the operation of determining the keyword feature vector of keyword information includes: converting each keyword of the target keyword into a corresponding keyword vector; converting the position information of the keyword into a corresponding position vector; converting the segmentation information of the keyword into a corresponding segmentation vector; and determining the keyword feature vector of the keyword information according to the keyword vector, the position vector, and the segmentation vector.

[0048] Specifically, each keyword of the target keyword is "mei", "wen", "hua", the position information of each keyword is the 26th, 27th, and 28th characters in the target text, and the segmentation information of each keyword belongs to the keyword part. Thus, in the process of determining the keyword feature vector of the keyword information, referring to Figure 4 as shown, it is necessary to convert the three keywords "mei", "wen", "hua" into corresponding keyword vectors, which are E 没 , E 文 , E 化 , it is necessary to convert the position information of these three keywords into corresponding position vectors, which are E 26 , E 27 , E 28 , and it is also necessary to convert the segmentation information of these three keywords into corresponding segmentation vectors, which are E A , E A , E A . Among them, E represents embedding, and E A represents the keyword part. Finally, according to the converted keyword vector, position vector, and segmentation vector, determine the keyword feature vector of the keyword information. According to the above content, the determined keyword feature vector is, for example: E [CLS] E0E A E 没 E 26 E A E 文 E 27 E A E 化 E 28 E A . Among them, [CLS] represents the classification label character. Thus, the obtained keyword feature vector is the vector sum of the character vector of each keyword + the position vector of each keyword + the segmentation vector of each keyword.

[0049] Optionally, the operation of determining the context feature vector of the context information includes: determining the current sentence feature vector of the current sentence where the target keyword is located in the context information; determining the previous sentence feature vector of the previous sentence of the current sentence in the context information; determining the next sentence feature vector of the next sentence of the current sentence in the context information; and determining the context feature vector of the context information based on the current sentence feature vector, the previous sentence feature vector and the next sentence feature vector.

[0050] Specifically, in the process of determining the context feature vector of the context information, the current sentence feature vector of the current sentence where the target keyword in the context information is located is first determined. The current sentence feature vector is the vector sum of the word vector of each word in the current sentence, including the speaking role, the position vector of each word, and the segment vector of each word. Figure 4 As shown, the current sentence feature vector can be E1E 22 E C E ： E 23 E C E 淮 E 24 E C E 海 E 25 E C E 没 E 26 E C E 文 E 27 E C E 化 E 28 E C E 没 E 29 E C E 在 E 30 E C E.E 24 E C , where the speaking role is represented by the number "1" when it is a customer, and by the number "2" when it is an agent. Then, it is necessary to determine the previous sentence feature vector of the previous sentence of the current sentence in the context information. The previous sentence feature vector is the vector sum of the word vector of each word in the previous sentence, including the speaking role, the position vector of each word, and the segment vector of each word. Figure 4 As shown, the feature vector of the previous sentence can be E1E0E B E:E2E B E 啊 E3E B E 上 E4E B ......E 吗 E 20 E BE ？ E 21 E B Similarly, it is also necessary to determine the next sentence feature vector of the next sentence of the current sentence in the context information. Figure 4 As shown, the current sentence feature vector can be E2E 32 E D E:E 33 E D E 喂 E 34 E D .

[0051] Finally, the context feature vector of the context information is determined based on the current sentence feature vector, the previous sentence feature vector, and the next sentence feature vector, for example, E1E0E B E:E2E B E 啊 E3E B E 上 E4E B ......E 吗 E 20 E B E ？ E 21 E B E [SEP] E1E 22 E C E:E 23 E C E 淮 E 24 E C ......E 在 E 30 E C E.E 24 E C E [SEP] E2E 32 E D E:E 33 E D E 喂 E 34 E D Among them, [SEP] represents the separator character, E B Indicates the previous sentence where the keyword is located, E C Indicates the current sentence where the keyword is located, E D Indicates the next sentence where the keyword appears. By combining the speaker's role, each word's word vector, each word's position vector, and each word's segment vector, a comprehensive contextual feature vector can be generated. Furthermore, if the target keyword has no previous or next sentence, the feature vector for that part can be left blank.

[0052] Optionally, refer to Figure 4 As shown, the operation of determining the current sentence feature vector of the current sentence where the target keyword in the context information is located includes: converting each word in the current sentence, including the speaking role, into a corresponding first word vector, where different speaking roles are referred to by different numbers; converting the position information of each word in the current sentence, including the speaking role, into a corresponding first position vector; converting the segmentation information of each word in the current sentence, including the speaking role, into a corresponding first segmentation vector; and determining the current sentence feature vector of the current sentence based on the first word vector, the first position vector and the first segmentation vector.

[0053] Optionally, refer to Figure 4 As shown, the operation of determining the previous sentence feature vector of the previous sentence of the current sentence in the context information includes: converting each word including the speaking role in the previous sentence into a corresponding second word vector, where different speaking roles are referred to by different numbers; converting the position information of each word including the speaking role in the previous sentence into a corresponding second position vector; converting the segmentation information of each word including the speaking role in the previous sentence into a corresponding second segmentation vector; and determining the previous sentence feature vector of the previous sentence based on the second word vector, the second position vector and the second segmentation vector.

[0054] Optionally, refer to Figure 4 As shown, the operation of determining the next sentence feature vector of the next sentence of the current sentence in the context information includes: converting each word including the speaking role in the next sentence into a corresponding third word vector, where different speaking roles are referred to by different numbers; converting the position information of each word including the speaking role in the next sentence into a corresponding third position vector; converting the segmentation information of each word including the speaking role in the next sentence into a corresponding third segmentation vector; and determining the next sentence feature vector of the next sentence based on the third word vector, the third position vector and the third segmentation vector.

[0055] Optionally, refer to Figure 3As shown, the detection model is a model trained using sample text labeled with information on whether keyword extraction is incorrect or not, wherein the sample text is text data with noise, and the keyword feature vector and the context feature vector are input into the preset detection model, thereby determining whether the target keyword is extracted incorrectly based on the output result of the detection model, including: inputting the keyword feature vector and the context feature vector into the detection model, outputting a classification label corresponding to the target keyword, wherein the classification label includes an incorrect extraction label and a correct extraction label; when the classification label output by the detection model is an incorrect extraction label, determining that the target keyword is extracted incorrectly; and when the classification label output by the detection model is a correct extraction label, determining that the target keyword is extracted correctly. The detection model can be a BERT model, or other deep neural network models can be selected, such as a GPT model and an LSTM model.

[0056] Therefore, this embodiment aims at the keyword extraction optimization problem in the scenario of inaccurate text (such as but not limited to text noise caused by ASR translation), and proposes a method for detecting erroneous extraction. This method does not invade the upstream and downstream modules, and can reuse existing mature technologies (such as ASR, keyword extraction) and accumulated keyword libraries to the greatest extent. Because the keyword extraction results are directly detected, no modification is required upstream; and the problem of erroneous extraction is solved in a focused manner, with strong pertinence and guaranteed effect. From the above content, it can be seen that by introducing contextual information and specific keyword text and position information, the detection model can find that the names of ordinary people are easily misrecognized by ASR through learning some samples, so erroneous keywords are likely to appear during verification; in addition, keywords generally appear in similar semantic segments in the context. If they are isolated semantic blocks, they may be misrecognized, so it is necessary to introduce contextual information to assist in determining whether the keyword is extracted incorrectly. Therefore, the detection method proposed in this embodiment does not invade the upstream and downstream modules, can reuse existing mature technologies (such as ASR, keyword extraction, etc.) and accumulated keyword libraries to the greatest extent, and focus on solving the problem of erroneous extraction. It is highly targeted and has a high detection accuracy.

[0057] In addition, reference Figure 1 As shown, according to the second aspect of this embodiment, a storage medium is provided, which includes a stored program, wherein when the program is run, a processor executes any one of the above methods.

[0058] It should be noted that for the aforementioned method embodiments, for simplicity of description, they are all expressed as a series of action combinations. However, those skilled in the art should be aware that the present invention is not limited by the order of the actions described, because according to the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0059] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0060] Example 2

[0061] Figure 5 FIG2 shows an apparatus 500 for detecting incorrect keyword extraction from imprecise text according to this embodiment, which corresponds to the method according to the first aspect of embodiment 1. Figure 5 As shown, the device 500 includes: an acquisition module 510, which is used to acquire a target text to be detected and a target keyword corresponding to the target text, wherein the target text is text data with noise, and the target keyword is a keyword obtained by extracting the target text by using keyword extraction technology; a first determination module 520, which is used to determine the keyword information of the target keyword and the context information of the target keyword, wherein the keyword information includes each keyword of the target keyword, the position information of the keyword and the segmentation information of the keyword, and the context information includes the current sentence, the previous sentence and the next sentence of the current sentence where the target keyword is located; a second determination module 530, which is used to determine the keyword feature vector of the keyword information and the context feature vector of the context information; and a judgment module 540, which is used to input the keyword feature vector and the context feature vector into a preset detection model, so as to determine whether the target keyword is extracted incorrectly according to the output result of the detection model.

[0062] Optionally, the second determination module 530 includes: a first conversion submodule, used to convert each keyword of the target keyword into a corresponding keyword vector; a second conversion submodule, used to convert the position information of the keyword into a corresponding position vector; a third conversion submodule, used to convert the segmentation information of the keyword into a corresponding segmentation vector; and a first determination submodule, used to determine the keyword feature vector of the keyword information based on the keyword vector, position vector and segmentation vector.

[0063] Optionally, the second determination module 530 includes: a second determination submodule, used to determine the current sentence feature vector of the current sentence where the target keyword is located in the context information; a third determination submodule, used to determine the previous sentence feature vector of the previous sentence of the current sentence in the context information; a fourth determination submodule, used to determine the next sentence feature vector of the next sentence of the current sentence in the context information; and a fifth determination submodule, used to determine the context feature vector of the context information based on the current sentence feature vector, the previous sentence feature vector and the next sentence feature vector.

[0064] Optionally, the second determination submodule includes: a first conversion unit, used to convert each word in the current sentence, including the speaking role, into a corresponding first word vector, where different speaking roles are referred to by different numbers; a second conversion unit, used to convert the position information of each word in the current sentence, including the speaking role, into a corresponding first position vector; a third conversion unit, used to convert the segmentation information of each word in the current sentence, including the speaking role, into a corresponding first segmentation vector; and a first determination unit, used to determine the current sentence feature vector of the current sentence based on the first word vector, the first position vector and the first segmentation vector.

[0065] Optionally, the third determination submodule includes: a fourth conversion unit, used to convert each word in the previous sentence, including the speaking role, into a corresponding second word vector, where different speaking roles are referred to by different numbers; a fifth conversion unit, used to convert the position information of each word in the previous sentence, including the speaking role, into a corresponding second position vector; a sixth conversion unit, used to convert the segmentation information of each word in the previous sentence, including the speaking role, into a corresponding second segmentation vector; and a second determination unit, used to determine the previous sentence feature vector of the previous sentence based on the second word vector, the second position vector and the second segmentation vector.

[0066] Optionally, the fourth determination submodule includes: a seventh conversion unit, used to convert each word in the next sentence, including the speaking role, into a corresponding third word vector, where different speaking roles are referred to by different numbers; an eighth conversion unit, used to convert the position information of each word in the next sentence, including the speaking role, into a corresponding third position vector; a ninth conversion unit, used to convert the segmentation information of each word in the next sentence, including the speaking role, into a corresponding third segmentation vector; and a third determination unit, used to determine the next sentence feature vector of the next sentence based on the third word vector, the third position vector and the third segmentation vector.

[0067] Optionally, the detection model is a model trained using sample text marked with information on whether keyword extraction is incorrect or not, wherein the sample text is text data with noise, and the judgment module 540 includes: a detection submodule, which is used to input the keyword feature vector and the context feature vector into the detection model, and output a classification label corresponding to the target keyword, wherein the classification label includes an incorrect extraction label and a correct extraction label; a first judgment submodule, which is used to determine that the target keyword extraction is incorrect when the classification label output by the detection model is an incorrect extraction label; and a second judgment submodule, which is used to determine that the target keyword extraction is correct when the classification label output by the detection model is a correct extraction label.

[0068] Therefore, according to this embodiment, a method for detecting erroneous extraction is proposed for the optimization problem of keyword extraction in imprecise text scenarios (such as but not limited to text noise caused by ASR translation). The method does not invade the upstream and downstream modules, and can reuse existing mature technologies (such as ASR, keyword extraction) and accumulated keyword libraries to the greatest extent. Because the keyword extraction results are directly detected, no modification is required upstream; and the problem of erroneous extraction is solved in a focused manner, with strong pertinence and guaranteed effect. From the above content, it can be seen that by introducing contextual information and specific keyword text and position information, the detection model can find that the names of ordinary people are easily misrecognized by ASR through learning some samples, so erroneous keywords are likely to appear during verification; in addition, keywords generally appear in similar semantic segments in the context. If they are isolated semantic blocks, they may be misrecognized. Therefore, contextual information needs to be introduced to assist in determining whether the keyword is extracted incorrectly. Therefore, the detection method proposed in this embodiment does not invade the upstream and downstream modules, can reuse existing mature technologies (such as ASR, keyword extraction, etc.) and accumulated keyword libraries to the greatest extent, and focus on solving the problem of erroneous extraction. It is highly targeted and has a high detection accuracy.

[0069] Example 3

[0070] Figure 6FIG. 6 shows an apparatus 600 for detecting incorrect keyword extraction from imprecise text according to this embodiment, which corresponds to the method according to the first aspect of embodiment 1. Figure 6 As shown, the device 600 includes: a processor 610; and a memory 620, which is connected to the processor 610 and is used to provide the processor 610 with instructions for processing the following processing steps: obtaining a target text to be detected and a target keyword corresponding to the target text, wherein the target text is text data with noise, and the target keyword is a keyword obtained by extracting the target text using keyword extraction technology; determining keyword information of the target keyword and context information of the target keyword, wherein the keyword information includes each keyword of the target keyword, the position information of the keyword, and the segmentation information of the keyword, and the context information includes the current sentence, the previous sentence, and the next sentence of the current sentence in which the target keyword is located; determining a keyword feature vector of the keyword information and a context feature vector of the context information; and inputting the keyword feature vector and the context feature vector into a preset detection model, thereby determining whether the target keyword is extracted incorrectly based on the output result of the detection model.

[0071] Optionally, the operation of determining the keyword feature vector of the keyword information includes: converting each keyword of the target keyword into a corresponding keyword vector; converting the position information of the keyword into a corresponding position vector; converting the segmentation information of the keyword into a corresponding segmentation vector; and determining the keyword feature vector of the keyword information based on the keyword vector, position vector and segmentation vector.

[0072] Optionally, the operation of determining the context feature vector of the context information includes: determining the current sentence feature vector of the current sentence where the target keyword is located in the context information; determining the previous sentence feature vector of the previous sentence of the current sentence in the context information; determining the next sentence feature vector of the next sentence of the current sentence in the context information; and determining the context feature vector of the context information based on the current sentence feature vector, the previous sentence feature vector and the next sentence feature vector.

[0073] Optionally, the operation of determining the current sentence feature vector of the current sentence where the target keyword in the context information is located includes: converting each word in the current sentence, including the speaking role, into a corresponding first word vector, where different speaking roles are referred to by different numbers; converting the position information of each word in the current sentence, including the speaking role, into a corresponding first position vector; converting the segmentation information of each word in the current sentence, including the speaking role, into a corresponding first segmentation vector; and determining the current sentence feature vector of the current sentence based on the first word vector, the first position vector and the first segmentation vector.

[0074] Optionally, the operation of determining the previous sentence feature vector of the previous sentence of the current sentence in the context information includes: converting each word in the previous sentence, including the speaking role, into a corresponding second word vector, where different speaking roles are referred to by different numbers; converting the position information of each word in the previous sentence, including the speaking role, into a corresponding second position vector; converting the segmentation information of each word in the previous sentence, including the speaking role, into a corresponding second segmentation vector; and determining the previous sentence feature vector of the previous sentence based on the second word vector, the second position vector and the second segmentation vector.

[0075] Optionally, the operation of determining the next sentence feature vector of the next sentence of the current sentence in the context information includes: converting each word in the next sentence, including the speaking role, into a corresponding third word vector, where different speaking roles are referred to by different numbers; converting the position information of each word in the next sentence, including the speaking role, into a corresponding third position vector; converting the segmentation information of each word in the next sentence, including the speaking role, into a corresponding third segmentation vector; and determining the next sentence feature vector of the next sentence based on the third word vector, the third position vector and the third segmentation vector.

[0076] Optionally, the detection model is a model trained using sample text marked with information on whether keyword extraction is incorrect or not, wherein the sample text is text data with noise, and the keyword feature vector and the context feature vector are input into a preset detection model, thereby determining whether the target keyword is extracted incorrectly based on the output result of the detection model, including: inputting the keyword feature vector and the context feature vector into the detection model, outputting a classification label corresponding to the target keyword, wherein the classification label includes an incorrect extraction label and a correct extraction label; when the classification label output by the detection model is an incorrect extraction label, determining that the target keyword is extracted incorrectly; and when the classification label output by the detection model is a correct extraction label, determining that the target keyword is extracted correctly.

[0077] Therefore, according to this embodiment, a method for detecting erroneous extraction is proposed for the optimization problem of keyword extraction in imprecise text scenarios (such as but not limited to text noise caused by ASR translation). The method does not invade the upstream and downstream modules, and can reuse existing mature technologies (such as ASR, keyword extraction) and accumulated keyword libraries to the greatest extent. Because the keyword extraction results are directly detected, no modification is required upstream; and the problem of erroneous extraction is solved in a focused manner, with strong pertinence and guaranteed effect. From the above content, it can be seen that by introducing contextual information and specific keyword text and position information, the detection model can find that the names of ordinary people are easily misrecognized by ASR through learning some samples, so erroneous keywords are likely to appear during verification; in addition, keywords generally appear in similar semantic segments in the context. If they are isolated semantic blocks, they may be misrecognized. Therefore, contextual information needs to be introduced to assist in determining whether the keyword is extracted incorrectly. Therefore, the detection method proposed in this embodiment does not invade the upstream and downstream modules, can reuse existing mature technologies (such as ASR, keyword extraction, etc.) and accumulated keyword libraries to the greatest extent, and focus on solving the problem of erroneous extraction. It is highly targeted and has a high detection accuracy.

[0078] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0079] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0080] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0081] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0082] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0083] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk, etc. Various media that can store program codes.

[0084] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A method for detecting incorrect keyword extraction from imprecise text, characterized in that: include: Obtaining a target text to be detected and a target keyword corresponding to the target text, wherein the target text is text data with noise, and the target keyword is a keyword obtained by extracting keywords from the target text using a keyword extraction technology; Determining keyword information of the target keyword and context information of the target keyword, wherein the keyword information includes each keyword of the target keyword, position information of the keyword, and segment information of the keyword, and the context information includes a current sentence in which the target keyword is located, a previous sentence, and a next sentence of the current sentence; Determining a keyword feature vector of the keyword information and a context feature vector of the context information; as well as The keyword feature vector and the context feature vector are input into a preset detection model, so as to determine whether the target keyword is extracted incorrectly according to the output result of the detection model.

2. The method according to claim 1, characterized in that The operation of determining the keyword feature vector of the keyword information includes: Convert each keyword of the target keyword into a corresponding keyword vector; Converting the position information of the keyword into a corresponding position vector; Converting the segmentation information of the keyword into a corresponding segmentation vector; and A keyword feature vector of the keyword information is determined according to the keyword vector, the position vector, and the segment vector.

3. The method according to claim 1, characterized in that The operation of determining the context feature vector of the context information includes: Determining a current sentence feature vector of a current sentence in which the target keyword is located in the context information; Determine a previous sentence feature vector of a previous sentence of the current sentence in the context information; Determining a next sentence feature vector of the next sentence of the current sentence in the context information; and A context feature vector of the context information is determined according to the current sentence feature vector, the previous sentence feature vector, and the next sentence feature vector.

4. The method according to claim 3, characterized in that The operation of determining a current sentence feature vector of a current sentence in which the target keyword in the context information is located includes: Convert each word in the current sentence, including the speaking role, into a corresponding first word vector, where different speaking roles are represented by different numbers; Converting position information of each word in the current sentence, including the speaking character, into a corresponding first position vector; Converting the segmentation information of each word in the current sentence, including the speaking role, into a corresponding first segmentation vector; and A current sentence feature vector of the current sentence is determined according to the first word vector, the first position vector, and the first segment vector.

5. The method according to claim 3, characterized in that The operation of determining a previous sentence feature vector of a previous sentence of the current sentence in the context information includes: Convert each word in the previous sentence, including the speaking role, into a corresponding second word vector, where different speaking roles are represented by different numbers; Converting the position information of each word in the previous sentence, including the speaking character, into a corresponding second position vector; Converting the segmentation information of each word in the previous sentence, including the speaking role, into a corresponding second segmentation vector; and A previous sentence feature vector of the previous sentence is determined according to the second word vector, the second position vector, and the second segment vector.

6. The method according to claim 3, characterized in that The operation of determining a next sentence feature vector of the next sentence of the current sentence in the context information includes: Convert each word in the next sentence, including the speaking role, into a corresponding third word vector, where different speaking roles are represented by different numbers; Converting the position information of each word in the next sentence, including the speaking character, into a corresponding third position vector; Converting the segmentation information of each word in the next sentence, including the speaking role, into a corresponding third segmentation vector; and A next sentence feature vector of the next sentence is determined according to the third word vector, the third position vector and the third segment vector.

7. The method according to claim 1, characterized in that The detection model is a model trained using sample texts annotated with information on whether keyword extraction is wrong, wherein the sample texts are text data with noise, and Inputting the keyword feature vector and the context feature vector into a preset detection model, thereby determining whether the target keyword is extracted incorrectly based on an output result of the detection model, includes: Inputting the keyword feature vector and the context feature vector into the detection model, and outputting a classification label corresponding to the target keyword, wherein the classification label includes an incorrectly extracted label and a correctly extracted label; When the classification label output by the detection model is an erroneous extraction label, determining that the target keyword is extracted incorrectly; and When the classification label output by the detection model is a correct extraction label, it is determined that the target keyword is correctly extracted.

8. A storage medium, characterized in that: The storage medium includes a stored program, wherein when the program is executed, the processor executes the method according to any one of claims 1 to 7.

9. A device for detecting incorrect keyword extraction from imprecise text, characterized in that: include: An acquisition module is used to acquire a target text to be detected and a target keyword corresponding to the target text, wherein the target text is text data with noise, and the target keyword is a keyword obtained by extracting the target text using a keyword extraction technology; A first determination module is configured to determine keyword information of the target keyword and context information of the target keyword, wherein the keyword information includes each keyword of the target keyword, position information of the keyword, and segmentation information of the keyword, and the context information includes a current sentence in which the target keyword is located, a previous sentence, and a next sentence of the current sentence; a second determining module, configured to determine a keyword feature vector of the keyword information and a context feature vector of the context information; as well as The determination module is used to input the keyword feature vector and the context feature vector into a preset detection model, thereby determining whether the target keyword is extracted incorrectly based on the output result of the detection model.

10. A device for detecting incorrect keyword extraction from imprecise text, characterized in that: include: processor; as well as A memory, connected to the processor, configured to provide the processor with instructions for processing the following steps: Obtaining a target text to be detected and a target keyword corresponding to the target text, wherein the target text is text data with noise, and the target keyword is a keyword obtained by extracting keywords from the target text using a keyword extraction technology; Determining keyword information of the target keyword and context information of the target keyword, wherein the keyword information includes each keyword of the target keyword, position information of the keyword, and segment information of the keyword, and the context information includes a current sentence in which the target keyword is located, a previous sentence, and a next sentence of the current sentence; Determining a keyword feature vector of the keyword information and a context feature vector of the context information; as well as The keyword feature vector and the context feature vector are input into a preset detection model, so as to determine whether the target keyword is extracted incorrectly according to the output result of the detection model.

Images