A standardized semantic mapping method and system based on artificial intelligence and big data

By building a semantic mapping library based on historical data, using the angle division of semantic eigenvectors and the central vector replacement vocabulary methods, the problem of low semantic mapping efficiency in the existing technology is solved, and more efficient semantic mapping rate and accuracy are achieved .

CN118940725BActive Publication Date: 2025-02-25HEFEI HUOYIN TECHNOLOGY CO LTD

Patent Information

Application Number
CN202410870562.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-01
Publication Date
2025-02-25
Estimated Expiration
2044-07-01

AI Technical Summary

Technical Problem

In the prior art, semantic mapping methods have low mapping efficiency on massive data, which increases the operating load of semantic mapping systems.

Method used

By obtaining the historical data to be mapped and the historical mapping result data, the semantic eigenvectors of the statement segments are extracted, and divided into several categories according to the angle sizes of these vectors, the center vectors and replacement vocabulary of high-frequency classes are selected, and the mapping library is built to improve the The rate of semantic mapping.

Benefits of technology

By rationally utilizing historical data, this method reduces repeated calculations, improves the rate of semantic mapping, reduces the operating load of semantic mapping system, and ensures the accuracy of semantic mapping.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118940725B_ABST
    Figure CN118940725B_ABST
Patent Text Reader

Abstract

The present invention relates to the technical field of semantic mapping, and specifically discloses a standardized semantic mapping method and system based on artificial intelligence and big data, comprising: step S1: dividing historical data to be mapped into various sentence segments, and dividing all semantic feature vectors into several classes; recording a class whose total number of semantic feature vectors exceeds the number threshold as a high-frequency class, and selecting a central vector; step S2: comparing the replaceable words of the mapping result and the central mapping result one by one, and obtaining the replacement words of the central mapping result; step S3: storing the central vector of the high-frequency class, all semantic feature vectors, the central mapping result and all replacement words to obtain a mapping library, and obtaining several quasi-mapping results; step S4: selecting the high-frequency class with the highest semantic similarity between the central vector and the semantic vector, obtaining the quasi-mapping result corresponding to the semantic feature vector with the highest similarity to the semantic vector in the high-frequency class, and using it as the mapping result of the sentence segment to be mapped.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of semantic mapping technology, and in particular to a standardized semantic mapping method and system based on artificial intelligence and big data. Background Art

[0002] Standardized semantic mapping is the process of mapping data from different sources and in different forms into a standard semantic representation; this process includes understanding data from different sources and converting it into a unified semantic expression form, making it easier for different systems to interact and understand each other.

[0003] The rate of semantic mapping refers to the speed at which semantic information is processed and converted within a specific time; the semantic mapping system usually includes information source, semantic knowledge base, semantic encoder, grammatical encoder and other parts; the rate of semantic mapping depends on multiple factors, including the efficiency of semantic encoding and decoding.

[0004] With the rapid development of technology today, the amount of data is growing explosively. In scenarios where standardized semantic mapping is required, it is particularly important to improve the rate of semantic mapping in the face of massive amounts of data to be semantically mapped. However, the semantic mapping methods in the prior art still map a large amount of data in sequence according to the order of sentences, resulting in low mapping efficiency and increasing the operating load of the semantic mapping system. Summary of the invention

[0005] The purpose of the present invention is to provide a standardized semantic mapping method and system based on artificial intelligence and big data to solve the above technical problems.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] A standardized semantic mapping method based on artificial intelligence and big data, comprising the following steps:

[0008] Step S1: obtaining historical data to be mapped and historical mapping result data; dividing the historical data to be mapped into individual sentence segments; obtaining semantic feature vectors of the sentence segments, and dividing all semantic feature vectors into several categories according to the angles between the semantic feature vectors;

[0009] A number threshold is set, a class whose total number of semantic feature vectors exceeds the number threshold is recorded as a high-frequency class, and a central vector is selected from the semantic feature vectors of the high-frequency class;

[0010] Step S2: in the historical mapping result data, obtain the mapping result of the semantic feature vector, record the mapping result of the center vector as the center mapping result; perform word segmentation processing on the mapping result and the center mapping result to obtain a plurality of words; select replaceable words according to the part of speech of each word, and the replaceable words include words of noun and adjective parts of speech; compare the replaceable words of the mapping result and the center mapping result one by one, and record the replaceable words in the mapping result that are different from the same position of the center mapping result as replacement words;

[0011] Step S3: storing the central vector of the high-frequency class, all semantic feature vectors, the central mapping results and all replacement words to obtain a mapping library; in the mapping library, sequentially replacing the replacement words in the central mapping result to obtain a number of quasi-mapping results; each semantic feature vector is associated with a quasi-mapping result;

[0012] Step S4: Obtain the semantic feature vector of the sentence segment to be mapped, recorded as the semantic vector; obtain the semantic similarity between the semantic vector and the central vector of each high-frequency class, and when the semantic similarity exceeds a preset similarity threshold, select the high-frequency class whose central vector has the highest semantic similarity with the semantic vector; according to the mapping library, obtain the quasi-mapping result corresponding to the semantic feature vector with the highest similarity to the semantic vector in the high-frequency class as the mapping result of the sentence segment to be mapped.

[0013] As a further solution of the present invention: in step S1, the process of dividing the sentence segments includes:

[0014] Acquire sentence symbols in the historical data to be mapped, where the sentence symbols include a period and a question mark;

[0015] The first glyph of the historical data to be mapped is recorded as the starting point, and the glyph is a basic unit of text in different languages; each punctuation mark in the historical data to be mapped is detected step by step from the starting point, and when the punctuation mark detected is a sentence mark, the previous glyph of the sentence mark is recorded as the end point, and all glyphs between the starting point and the end point are recorded as a sentence segment;

[0016] The next glyph of the end point is taken as a new starting point, and the punctuation mark behind is continuously detected from the new starting point. When the punctuation mark is detected as a sentence mark again, the next glyph of the sentence mark at this time is taken as the new end point, and all the glyphs before the new starting point and the new end point are recorded as a new sentence segment; and so on, each sentence segment is obtained in the historical data to be mapped.

[0017] As a further solution of the present invention: in step S1, the process of dividing the sentence segments into several categories includes:

[0018] Let the two semantic feature vectors be A and B respectively, then we get the angle between the two semantic feature vectors ; Semantic feature vectors whose angles between each other are lower than a preset threshold are recorded as the same class, and all semantic feature vectors are divided into several classes.

[0019] As a further solution of the present invention: in step S1, the process of setting the number threshold includes:

[0020] Get the total number of semantic feature vectors in each category {N 1 , N 2 , ..., N i}, where N 1 represents the total number of semantic feature vectors in the first category, i is the total number of categories; then the average value of the number is obtained , where s∈[1,i] and s is a positive integer; the standard deviation σ is obtained according to the mean value of the number:

[0021] ;

[0022] According to the number mean and standard deviation, the number threshold Nt=N is obtained. ave +σ.

[0023] As a further solution of the present invention: in step S1, the process of selecting the center vector includes:

[0024] The angles between each semantic feature vector and the rest of the semantic feature vectors are obtained in turn, and the average value of all angles is obtained, which is recorded as the angle average value; the semantic feature vector with the smallest angle average value is selected and recorded as the central vector of the high-frequency class.

[0025] As a further solution of the present invention: in step S1, when the average values ​​of the angles of two semantic feature vectors are equal, the coordinates of the two semantic feature vectors are recorded as (a x , a y , a z ) and (b x , b y , b z ), then we get the median vector M of the two semantic feature vectors:

[0026] ,

[0027] The median vector of the two semantic feature vectors is used as the central vector of the high-frequency class; at this time, one of the two semantic feature vectors is arbitrarily selected, and its mapping result is obtained as the central mapping result of the central vector.

[0028] As a further solution of the present invention: in step S2, the process of performing word segmentation processing on the mapping result and the central mapping result is to input the mapping result or the central mapping result into a natural language processing tool, perform word segmentation processing on it using the natural language processing tool, and obtain a word list; the word list includes several words, and the natural language processing tools include NLTK, Spacy, and Stanford CoreNLP.

[0029] As a further solution of the present invention: in step S4, the semantic similarity between the semantic vector and the central vector is the cosine value of the angle between the two.

[0030] As a further solution of the present invention: a standardized semantic mapping system based on artificial intelligence and big data, comprising the following modules:

[0031] Classification module: obtain historical data to be mapped and historical mapping result data; divide the historical data to be mapped into individual sentence segments; obtain semantic feature vectors of the sentence segments, and divide all semantic feature vectors into several categories according to the angles between the semantic feature vectors;

[0032] A number threshold is set, a class whose total number of semantic feature vectors exceeds the number threshold is recorded as a high-frequency class, and a central vector is selected from the semantic feature vectors of the high-frequency class;

[0033] High-frequency category processing module: in the historical mapping result data, obtain the mapping result of the semantic feature vector, record the mapping result of the center vector as the center mapping result; perform word segmentation processing on the mapping result and the center mapping result to obtain a number of words; select replaceable words according to the part of speech of each word, and the replaceable words include words of noun and adjective parts of speech; compare the replaceable words of the mapping result and the center mapping result one by one, and record the replaceable words in the mapping result that are different from the same position of the center mapping result as replacement words;

[0034] Storage module: storing the central vector of the high-frequency class, all semantic feature vectors, central mapping results and all replacement words to obtain a mapping library; in the mapping library, replacing the replacement words in the central mapping result in turn to obtain a number of quasi-mapping results; each semantic feature vector is associated with a quasi-mapping result;

[0035] Semantic mapping module: obtain the semantic feature vector of the sentence segment to be mapped, recorded as the semantic vector; obtain the semantic similarity between the semantic vector and the central vector of each high-frequency class, and when the semantic similarity exceeds a preset similarity threshold, select the high-frequency class with the highest semantic similarity between the central vector and the semantic vector; according to the mapping library, obtain the quasi-mapping result corresponding to the semantic feature vector with the highest similarity to the semantic vector in the high-frequency class as the mapping result of the sentence segment to be mapped.

[0036] Beneficial effects of the present invention:

[0037] With the rapid development of technology today, the amount of data is growing explosively. In scenarios where standardized semantic mapping is required, it is particularly important to improve the rate of semantic mapping in the face of massive amounts of data to be semantically mapped. However, the semantic mapping method in the prior art still maps a large amount of data in sequence according to the order of sentences, resulting in low mapping efficiency and increasing the operating load of the semantic mapping system.

[0038] Compared with the prior art, in the present invention, historical mapping data is obtained, several sentence segments are extracted from the historical mapping data, and semantic feature vectors of the sentence segments are obtained, all semantic feature vectors are divided into several categories according to the angles between the semantic feature vectors, and a category in which the total number of semantic feature vectors exceeds the number threshold is recorded as a high-frequency category; this method saves a category of semantic feature vectors with a higher number of occurrences according to the frequency of occurrence of semantic feature vectors in historical data; and a central vector is selected from the high-frequency category, the central vector represents the main characteristics of the category to which it belongs, and in subsequent analysis or comparison, the central vector can be used to simplify calculations and improve efficiency; in the historical mapping result data, the mapping results of the semantic feature vectors and the central mapping results of the central vector are obtained, the mapping results and the central mapping results of the replaceable words are compared one by one, and the replaceable words in the mapping results that are different from those in the same position as the central mapping results are recorded as replacement words ; Store the central vector of the high-frequency class, all semantic feature vectors, central mapping results and all replacement words to obtain a mapping library; in the mapping library, the replacement words in the central mapping result can be replaced in turn to obtain several quasi-mapping results; this method reduces the memory required to back up all mapping results, and at the same time indirectly saves all mapping results of the high-frequency class, ensuring the accuracy of semantic mapping; for a new sentence segment to be mapped, obtain its semantic vector, compare the semantic vector with the central vector of all high-frequency classes, determine the high-frequency class with the highest semantic similarity between the central vector and the semantic vector, and obtain the quasi-mapping result corresponding to the semantic feature vector with the highest similarity to the semantic vector in the high-frequency class in the mapping library as the mapping result of the sentence segment to be mapped; this method makes rational use of historical data and caches frequently accessed data results to reduce repeated calculations, thereby improving the mapping rate and helping to reduce the operating load of the semantic mapping system. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The present invention will be further described below in conjunction with the accompanying drawings.

[0040] Figure 1 It is a structural schematic diagram of a standardized semantic mapping method and system based on artificial intelligence and big data in the present invention. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] See also Figure 1 As shown, the present invention is a standardized semantic mapping method based on artificial intelligence and big data, comprising the following steps:

[0043] Step S1: obtaining historical data to be mapped and historical mapping result data; dividing the historical data to be mapped into individual sentence segments; obtaining semantic feature vectors of the sentence segments, and dividing all semantic feature vectors into several categories according to the angles between the semantic feature vectors;

[0044] A number threshold is set, a class whose total number of semantic feature vectors exceeds the number threshold is recorded as a high-frequency class, and a central vector is selected from the semantic feature vectors of the high-frequency class;

[0045] It is understandable that obtaining historical data to be mapped and historical mapping result data involves collecting previous data sets, which may contain original text (data to be mapped) and processed or classified data related thereto (mapping result data); dividing the historical data to be mapped into individual sentence segments, which usually means dividing long text into smaller units, such as sentences or paragraphs, for subsequent processing; obtaining semantic feature vectors of sentence segments, each of which will be converted into a semantic feature vector; this is usually achieved by using word embedding (such as Word2Vec, GloVe, FastText) or more complex models (such as BERT, XLNet), the purpose of which is to capture the semantic information of the sentence; dividing all semantic feature vectors into several classes according to the angle between the semantic feature vectors, which involves calculating the similarity between the semantic feature vectors (usually cosine similarity) and classifying them according to this similarity; the smaller the angle between the vectors, the higher their similarity; selecting a central vector from the semantic feature vectors of the high-frequency class, and selecting a representative vector, i.e., a central vector, from each high-frequency class; the central vector can be used as a representative of the class for subsequent analysis or comparison;

[0046] The process of dividing the sentence segments includes:

[0047] Acquire sentence symbols in the historical data to be mapped, where the sentence symbols include a period and a question mark;

[0048] The first glyph of the historical data to be mapped is recorded as the starting point, and the glyph is a basic unit of text in different languages; each punctuation mark in the historical data to be mapped is detected step by step from the starting point, and when the punctuation mark detected is a sentence mark, the previous glyph of the sentence mark is recorded as the end point, and all glyphs between the starting point and the end point are recorded as a sentence segment;

[0049] The next glyph of the end point is taken as a new starting point, and the punctuation mark behind is continuously detected from the new starting point. When the punctuation mark is detected as a sentence mark again, the next glyph of the sentence mark at this time is taken as a new end point, and all glyphs before the new starting point and the new end point are recorded as a new sentence segment; and so on, each sentence segment is obtained in the historical data to be mapped;

[0050] It can be understood that the segmentation process is a punctuation-based text segmentation method for segmenting continuous text data into separate sentence segments; identifying punctuation marks in the text used to indicate the end of a sentence, i.e., sentence marks, which are symbols marking the end of a sentence;

[0051] The process of dividing the sentence segments into several categories includes:

[0052] Let the two semantic feature vectors be A and B respectively, then we get the angle between the two semantic feature vectors ; Semantic feature vectors whose angles between each other are lower than a preset threshold are recorded as the same category, and all semantic feature vectors are divided into several categories;

[0053] The process of setting the number threshold includes:

[0054] Get the total number of semantic feature vectors in each category {N 1 , N 2 , ..., N i}, where N 1 represents the total number of semantic feature vectors in the first category, i is the total number of categories; then the average value of the number is obtained , where s∈[1,i] and s is a positive integer; the standard deviation σ is obtained according to the mean value of the number:

[0055] ;

[0056] According to the number mean and standard deviation, the number threshold Nt=N is obtained. ave +σ;

[0057] The number threshold is a dynamic threshold based on data distribution, which can better distinguish which categories are high-frequency categories containing more semantic feature vectors;

[0058] The process of selecting the center vector includes:

[0059] The angles between each semantic feature vector and the rest of the semantic feature vectors are obtained in turn, and the average value of all angles is obtained, which is recorded as the angle average value; the semantic feature vector with the smallest angle average value is selected and recorded as the central vector of the high-frequency class; from each semantic feature vector in the high-frequency class, the vector with the smallest angle average value is selected as the central vector of the class; this means that the selected central vector has the smallest average difference within the class, and can be regarded as the most representative or typical semantic feature in the class;

[0060] It is worth noting that the angle between each semantic feature vector and all other semantic feature vectors in the class is calculated; here, cosine similarity is usually used to calculate the angle between two vectors, because cosine similarity can effectively measure the similarity of two vectors in direction; the average value of these angles is obtained, and this average value reflects the average similarity level between the semantic feature vector and other vectors;

[0061] When the average values ​​of the angles between two semantic feature vectors are equal, the coordinates of the two semantic feature vectors are (a x , a y , a z ) and (b x , b y , b z ), then we get the median vector M of the two semantic feature vectors:

[0062] ,

[0063] The median vector of the two semantic feature vectors is used as the central vector of the high-frequency class; at this time, one of the two semantic feature vectors is arbitrarily selected, and its mapping result is obtained as the central mapping result of the central vector;

[0064] Step S2: in the historical mapping result data, obtain the mapping result of the semantic feature vector, record the mapping result of the center vector as the center mapping result; perform word segmentation processing on the mapping result and the center mapping result to obtain a plurality of words; select replaceable words according to the part of speech of each word, and the replaceable words include words of noun and adjective parts of speech; compare the replaceable words of the mapping result and the center mapping result one by one, and record the replaceable words in the mapping result that are different from the same position of the center mapping result as replacement words;

[0065] It is understandable that the mapping results and the center mapping results are segmented, that is, the text data is cut into individual words; this process is the basic step of most NLP tasks, which aims to split continuous text into manageable language units for further analysis; for example: for the sentence segment "the cat is basking in the sun on the roof" and the other sentence segment "the cat is basking in the sun on the windowsill", after the segmentation, "roof" and "windowsill" are different replaceable words;

[0066] The process of performing word segmentation processing on the mapping result and the central mapping result is as follows: inputting the mapping result or the central mapping result into a natural language processing tool, performing word segmentation processing on the mapping result or the central mapping result by using the natural language processing tool to obtain a word list; the word list includes a plurality of words, and the natural language processing tool includes NLTK, Spacy, and Stanford CoreNLP;

[0067] Step S3: storing the central vector of the high-frequency class, all semantic feature vectors, the central mapping results and all replacement words to obtain a mapping library; in the mapping library, sequentially replacing the replacement words in the central mapping result to obtain a number of quasi-mapping results; each semantic feature vector is associated with a quasi-mapping result;

[0068] It is worth noting that after the mapping library is built, multiple pseudo-mapping results can be obtained by replacing the replacement words in the central mapping result in sequence; this process is actually a kind of data enhancement, which generates new text instances by replacing key words. These instances are semantically related to the original text but different; each semantic feature vector is associated with a pseudo-mapping result. This step is critical because it establishes a direct connection from a specific text instance to its semantic representation (i.e., semantic feature vector);

[0069] Step S4: obtaining a semantic feature vector of the sentence segment to be mapped, recorded as a semantic vector; obtaining the semantic similarity between the semantic vector and the central vector of each high-frequency class, and when the semantic similarity exceeds a preset similarity threshold, selecting the high-frequency class with the highest semantic similarity between the central vector and the semantic vector; obtaining a quasi-mapping result corresponding to the semantic feature vector with the highest similarity to the semantic vector in the high-frequency class according to the mapping library, as the mapping result of the sentence segment to be mapped;

[0070] It can be understood that the preset similarity threshold indicates that the semantic vector can be regarded as a standard for frequently accessed data; obtaining the semantic similarity between the semantic vector and each central vector, determining whether each semantic similarity exceeds the preset similarity threshold, and selecting several semantic similarities exceeding the preset similarity threshold, and selecting the central vector corresponding to the highest semantic similarity among these several semantic similarities; obtaining the high-frequency class with the highest semantic similarity to the semantic vector, and determining the semantic feature vector with the highest similarity to the semantic vector in this high-frequency class;

[0071] The semantic similarity between the semantic vector and the central vector is the cosine value of the angle between them;

[0072] It can be understood that if the two semantic feature vectors are A and B, the semantic similarity of the two semantic feature vectors is .

[0073] A standardized semantic mapping system based on artificial intelligence and big data, including the following modules:

[0074] Classification module: obtain historical data to be mapped and historical mapping result data; divide the historical data to be mapped into individual sentence segments; obtain semantic feature vectors of the sentence segments, and divide all semantic feature vectors into several categories according to the angles between the semantic feature vectors;

[0075] A number threshold is set, a class whose total number of semantic feature vectors exceeds the number threshold is recorded as a high-frequency class, and a central vector is selected from the semantic feature vectors of the high-frequency class;

[0076] High-frequency category processing module: in the historical mapping result data, obtain the mapping result of the semantic feature vector, record the mapping result of the center vector as the center mapping result; perform word segmentation processing on the mapping result and the center mapping result to obtain a number of words; select replaceable words according to the part of speech of each word, and the replaceable words include words of noun and adjective parts of speech; compare the replaceable words of the mapping result and the center mapping result one by one, and record the replaceable words in the mapping result that are different from the same position of the center mapping result as replacement words;

[0077] Storage module: storing the central vector of the high-frequency class, all semantic feature vectors, central mapping results and all replacement words to obtain a mapping library; in the mapping library, replacing the replacement words in the central mapping result in turn to obtain a number of quasi-mapping results; each semantic feature vector is associated with a quasi-mapping result;

[0078] Semantic mapping module: obtain the semantic feature vector of the sentence segment to be mapped, recorded as the semantic vector; obtain the semantic similarity between the semantic vector and the central vector of each high-frequency class, and when the semantic similarity exceeds a preset similarity threshold, select the high-frequency class with the highest semantic similarity between the central vector and the semantic vector; according to the mapping library, obtain the quasi-mapping result corresponding to the semantic feature vector with the highest similarity to the semantic vector in the high-frequency class as the mapping result of the sentence segment to be mapped.

[0079] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A standardized semantic mapping method based on artificial intelligence and big data, characterized in that: The following steps are involved: Step S1: Obtain historical data to be mapped and historical mapping result data; divide the historical data to be mapped into various sentence segments; Acquire semantic feature vectors of the sentence segments, and divide all semantic feature vectors into several categories according to the angles between the semantic feature vectors; A number threshold is set, a class whose total number of semantic feature vectors exceeds the number threshold is recorded as a high-frequency class, and a central vector is selected from the semantic feature vectors of the high-frequency class; Step S2: in the historical mapping result data, obtain the mapping result of the semantic feature vector, record the mapping result of the center vector as the center mapping result; perform word segmentation processing on the mapping result and the center mapping result to obtain a plurality of words; select replaceable words according to the part of speech of each word, and the replaceable words include words of noun and adjective parts of speech; compare the replaceable words of the mapping result and the center mapping result one by one, and record the replaceable words in the mapping result that are different from the same position of the center mapping result as replacement words; Step S3: storing the central vector of the high-frequency class, all semantic feature vectors, the central mapping results and all replacement words to obtain a mapping library; in the mapping library, sequentially replacing the replacement words in the central mapping result to obtain a number of quasi-mapping results; each semantic feature vector is associated with a quasi-mapping result; Step S4: Obtain the semantic feature vector of the sentence segment to be mapped, recorded as the semantic vector; obtain the semantic similarity between the semantic vector and the central vector of each high-frequency class, and when the semantic similarity exceeds a preset similarity threshold, select the high-frequency class whose central vector has the highest semantic similarity with the semantic vector; according to the mapping library, obtain the quasi-mapping result corresponding to the semantic feature vector with the highest similarity to the semantic vector in the high-frequency class as the mapping result of the sentence segment to be mapped.

2. The standardized semantic mapping method based on artificial intelligence and big data according to claim 1, characterized in that: In step S1, the process of dividing the sentence segments includes: Acquire sentence symbols in the historical data to be mapped, where the sentence symbols include a period and a question mark; The first glyph of the historical data to be mapped is recorded as the starting point, and the glyph is a basic unit of text in different languages; each punctuation mark in the historical data to be mapped is detected step by step from the starting point, and when the punctuation mark detected is a sentence mark, the previous glyph of the sentence mark is recorded as the end point, and all glyphs between the starting point and the end point are recorded as a sentence segment; The next glyph of the end point is taken as a new starting point, and the punctuation mark behind is continuously detected from the new starting point. When the punctuation mark is detected as a sentence mark again, the next glyph of the sentence mark at this time is taken as the new end point, and all the glyphs before the new starting point and the new end point are recorded as a new sentence segment; and so on, each sentence segment is obtained in the historical data to be mapped.

3. The standardized semantic mapping method based on artificial intelligence and big data according to claim 1, characterized in that: In step S1, the process of dividing the sentence segments into several categories includes: Let the two semantic feature vectors be A and B respectively, then we get the angle between the two semantic feature vectors ; Semantic feature vectors whose angles between each other are lower than a preset threshold are recorded as the same class, and all semantic feature vectors are divided into several classes.

4. The standardized semantic mapping method based on artificial intelligence and big data according to claim 1, characterized in that: In step S1, the process of setting the number threshold includes: Get the total number of semantic feature vectors in each category {N1, N2, ..., N i }, where N1 represents the total number of semantic feature vectors in the first category, and i is the total number of categories; then the average value of the number is obtained , where s∈[1,i] and s is a positive integer; the standard deviation σ is obtained according to the mean value of the number: ; According to the number mean and standard deviation, the number threshold Nt=N is obtained. ave +σ.

5. The standardized semantic mapping method based on artificial intelligence and big data according to claim 1, characterized in that: In step S1, the process of selecting the center vector includes: The angles between each semantic feature vector and the rest of the semantic feature vectors are obtained in turn, and the average value of all angles is obtained, which is recorded as the angle average value; the semantic feature vector with the smallest angle average value is selected and recorded as the central vector of the high-frequency class.

6. The standardized semantic mapping method based on artificial intelligence and big data according to claim 5, characterized in that: In step S1, when the average values ​​of the angles of two semantic feature vectors are equal, the coordinates of the two semantic feature vectors are recorded as (a x , a y , a z ) and (b x , b y , b z ), then we get the median vector M of the two semantic feature vectors: , The median vector of the two semantic feature vectors is used as the central vector of the high-frequency class; at this time, one of the two semantic feature vectors is arbitrarily selected, and its mapping result is obtained as the central mapping result of the central vector.

7. The standardized semantic mapping method based on artificial intelligence and big data according to claim 1, characterized in that: In step S2, the process of performing word segmentation processing on the mapping result and the central mapping result is to input the mapping result or the central mapping result into a natural language processing tool, perform word segmentation processing on it using the natural language processing tool, and obtain a word list; the word list includes several words, and the natural language processing tools include NLTK, Spacy, and StanfordCoreNLP.

8. The standardized semantic mapping method based on artificial intelligence and big data according to claim 1, characterized in that: In step S4, the semantic similarity between the semantic vector and the central vector is the cosine value of the angle between the two.

9. A standardized semantic mapping system based on artificial intelligence and big data, including the following modules: Classification module: obtains historical data to be mapped and historical mapping result data; divides the historical data to be mapped into various sentence segments; Acquire semantic feature vectors of the sentence segments, and divide all semantic feature vectors into several categories according to the angles between the semantic feature vectors; A number threshold is set, a class whose total number of semantic feature vectors exceeds the number threshold is recorded as a high-frequency class, and a central vector is selected from the semantic feature vectors of the high-frequency class; High-frequency category processing module: in the historical mapping result data, obtain the mapping result of the semantic feature vector, record the mapping result of the center vector as the center mapping result; perform word segmentation processing on the mapping result and the center mapping result to obtain a number of words; select replaceable words according to the part of speech of each word, and the replaceable words include words of noun and adjective parts of speech; compare the replaceable words of the mapping result and the center mapping result one by one, and record the replaceable words in the mapping result that are different from the same position of the center mapping result as replacement words; Storage module: storing the central vector of the high-frequency class, all semantic feature vectors, central mapping results and all replacement words to obtain a mapping library; in the mapping library, replacing the replacement words in the central mapping result in turn to obtain a number of quasi-mapping results; each semantic feature vector is associated with a quasi-mapping result; Semantic mapping module: obtain the semantic feature vector of the sentence segment to be mapped, recorded as the semantic vector; obtain the semantic similarity between the semantic vector and the central vector of each high-frequency class, and when the semantic similarity exceeds a preset similarity threshold, select the high-frequency class with the highest semantic similarity between the central vector and the semantic vector; according to the mapping library, obtain the quasi-mapping result corresponding to the semantic feature vector with the highest similarity to the semantic vector in the high-frequency class as the mapping result of the sentence segment to be mapped.

Citation Information

Patent Citations

  • Data mapping and error correction method based on big data and artificial intelligence

    CN114462421A

  • Domain knowledge question-answering method, system and device and medium

    CN116775846A

Cited By

  • Intelligent mapping method for heterogeneous data based on cognitive reasoning and self-supervised contrastive learning

    CN122692885A