Method and system for processing local information context-related word components

Abstract

The present invention provides a method and system for processing local information context-related word components. The method inputs a sentence into a syntactic model, sets extraction windows of different widths for sentence segmentation, obtains a first word component, determines the length of the context content according to the extraction window, performs association calculations with the previous and next sentences respectively, determines the context content, performs semantic analysis on the first word component, and uses its context content as a reference input for semantic analysis to derive the word meaning corresponding to the first word component. By predicting the word meaning of the context content, the word meaning corresponding to the first word component can be corrected.

Description

Technical Field

[0001] The present application relates to the field of network multimedia, and in particular to a method and system for processing local information context-related word components. Background Art

[0002] Word embedding is a static assumption about a word. To address the grammatical differences between words in different contexts, existing technologies use context-dependent word components. However, this also introduces the technical issue of long processing time.

[0003] Therefore, there is an urgent need for a targeted method and system for processing local information context-related word components. Summary of the Invention

[0004] The purpose of the present invention is to provide a method and system for processing local information context-related word components, by inputting a sentence into a syntactic model, setting extraction windows of different widths for sentence segmentation, obtaining a first word component, and determining the length of the context content based on the extraction window, performing association calculations with the previous and next sentences respectively, determining the context content, performing semantic analysis on the first word component, and using its context content as a reference input for semantic analysis to derive the word meaning corresponding to the first word component. By predicting the word meaning of the context content, the word meaning corresponding to the first word component can be corrected.

[0005] In a first aspect, the present application provides a method for processing local information context-related word components, the method comprising:

[0006] Acquire a network data stream, extract sentences from it, input the sentences into a syntactic model, and perform sentence segmentation, wherein the syntactic model sets extraction windows of different widths according to each word type, uses the extraction windows as a basis for sentence segmentation, and the words within the window width constitute the first word component, and outputs and caches the first word component;

[0007] Querying the length of the corresponding context content according to the width of the extraction window;

[0008] Performing an association calculation between the first word component and the previous sentence, and determining the previous content of the first word component based on the degree of association;

[0009] During the next sentence segmentation, the cached first word component is extracted and associated with the next sentence, and the following content of the first word component is determined according to the degree of association;

[0010] Inputting the first word components one by one into a semantic analysis model, and using the context of the first word components as a reference input of the semantic analysis model to obtain the word meaning corresponding to the first word components;

[0011] The semantic analysis model includes: semantic analysis of context content of a reference input to obtain a corresponding context word meaning, predicting a second word meaning corresponding to the context word meaning, comparing the second word meaning with the word meaning directly obtained from the semantics of the first word component, calculating a variance value, and using the variance value as a coefficient to correct the word meaning corresponding to the first word component;

[0012] Output the word meaning corresponding to the first word component to the server for keyword detection.

[0013] In combination with the first aspect, in a first possible implementation of the first aspect, the extracted sentence includes: setting extraction windows of different widths according to each word type, including updating the word type, and establishing a correspondence between the new word type and the extraction window width.

[0014] In combination with the first aspect, in a second possible implementation of the first aspect, the semantic analysis model performs semantic analysis according to sentence grammatical requirements.

[0015] In combination with the first aspect, in a third possible implementation of the first aspect, the kernels of the semantic analysis model and the syntactic model both use a neural network model.

[0016] In a second aspect, the present application provides a system for processing local information context-related word components, the system comprising a processor and a memory:

[0017] The memory is used to store program code and transmit the program code to the processor;

[0018] The processor is configured to execute any one of the four possible methods of the first aspect according to instructions in the program code.

[0019] In a third aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store program code, and the program code is used to execute any one of the four possible methods in the first aspect.

[0020] The present invention provides a method and system for processing local information context-related word components. The method inputs a sentence into a syntactic model, sets extraction windows of different widths for sentence segmentation, obtains a first word component, determines the length of the context content according to the extraction window, performs association calculations with the previous and next sentences respectively, determines the context content, performs semantic analysis on the first word component, and uses its context content as a reference input for semantic analysis to derive the word meaning corresponding to the first word component. By predicting the word meaning of the context content, the word meaning corresponding to the first word component can be corrected. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0022] Figure 1 Flowchart of the method of the present invention. DETAILED DESCRIPTION

[0023] The preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby making a clearer and more precise definition of the protection scope of the present invention.

[0024] Figure 1 The flowchart of the method for processing local information context-related word components provided in this application includes:

[0025] Acquire a network data stream, extract sentences from it, input the sentences into a syntactic model, and perform sentence segmentation, wherein the syntactic model sets extraction windows of different widths according to each word type, uses the extraction windows as a basis for sentence segmentation, and the words within the window width constitute the first word component, and outputs and caches the first word component;

[0026] Querying the length of the corresponding context content according to the width of the extraction window;

[0027] Performing an association calculation between the first word component and the previous sentence, and determining the previous content of the first word component based on the degree of association;

[0028] During the next sentence segmentation, the cached first word component is extracted and associated with the next sentence, and the following content of the first word component is determined according to the degree of association;

[0029] Inputting the first word components one by one into a semantic analysis model, and using the context of the first word components as a reference input of the semantic analysis model to obtain the word meaning corresponding to the first word components;

[0030] The semantic analysis model includes: semantic analysis of context content of a reference input to obtain a corresponding context word meaning, predicting a second word meaning corresponding to the context word meaning, comparing the second word meaning with the word meaning directly obtained from the semantics of the first word component, calculating a variance value, and using the variance value as a coefficient to correct the word meaning corresponding to the first word component;

[0031] Output the word meaning corresponding to the first word component to the server for keyword detection.

[0032] In some preferred embodiments, extracting the sentence includes setting extraction windows of different widths according to each word type, including updating the word type and establishing a correspondence between the new word type and the extraction window width.

[0033] In some preferred embodiments, the semantic analysis model performs semantic analysis according to sentence grammar requirements.

[0034] In some preferred embodiments, the kernels of the semantic analysis model and the syntactic model both use neural network models.

[0035] The present application provides a system for processing local information context-related word components, the system comprising: the system comprising a processor and a memory:

[0036] The memory is used to store program code and transmit the program code to the processor;

[0037] The processor is configured to execute the method described in any one of all embodiments of the first aspect according to instructions in the program code.

[0038] The present application provides a computer-readable storage medium, which is used to store program code, and the program code is used to execute the method described in any one of the embodiments of the first aspect.

[0039] In a specific implementation, the present invention further provides a computer storage medium, wherein the computer storage medium may store a program that, when executed, may include some or all of the steps of various embodiments of the present invention. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0040] Those skilled in the art will clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus the necessary general-purpose hardware platform. Based on this understanding, the technical solutions in the embodiments of the present invention, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments of the present invention or certain portions of the embodiments.

[0041] In particular, for the embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description in the method embodiments.

[0042] The above-described embodiments of the present invention do not limit the protection scope of the present invention.

Claims

1. A method for processing local information context-related word components, characterized in that: The method comprises: Acquire a network data stream, extract sentences from it, input the sentences into a syntactic model, and perform sentence segmentation, wherein the syntactic model sets extraction windows of different widths according to each word type, uses the extraction windows as a basis for sentence segmentation, and the words within the window width constitute a first word component, and outputs and caches the first word component; Querying the length of the corresponding context content according to the width of the extraction window; Performing an association calculation between the first word component and the previous sentence, and determining the previous content of the first word component based on the degree of association; During the next sentence segmentation, the first word component of the cache is extracted and associated with the next sentence, and the following content of the first word component is determined according to the degree of association; Inputting the first word components one by one into a semantic analysis model, and using the context of the first word components as a reference input of the semantic analysis model to obtain the word meaning corresponding to the first word components; The semantic analysis model includes: semantic analysis of context content of a reference input to obtain a corresponding context word meaning, predicting a second word meaning corresponding to the context word meaning, comparing the second word meaning with the word meaning directly obtained from the semantics of the first word component, calculating a variance value, and using the variance value as a coefficient to correct the word meaning corresponding to the first word component; Output the word meaning corresponding to the first word component to the server for keyword detection.

2. The method according to claim 1, wherein: The extracting of sentences includes setting extraction windows of different widths according to each word type, updating the word type, and establishing a corresponding relationship between the new word type and the extraction window width.

3. The method according to any one of claims 1-2, characterized in that: The semantic analysis model performs semantic analysis according to sentence grammar requirements.

4. The method according to any one of claims 1 to 3, characterized in that: The kernels of the semantic analysis model and the syntactic model both use neural network models.

5. A system for processing local information context-related word components, characterized in that: The system includes a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is configured to implement the method according to any one of claims 1 to 4 according to the instructions in the program code.

6. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store program code, and the program code is used to execute the method according to any one of claims 1 to 4.

Images