An AI-based data classification retrieval method and system

Abstract

The application discloses an AI-based data classification retrieval method and system, which comprises the following steps: obtaining a data segment to be retrieved and extracting a keyword group; classifying and identifying the keywords and establishing a search index; searching in a database based on the search index to obtain a search tree; extracting a first feature vector of the search index and a second feature vector of the search tree, calculating the similarity; and optimizing based on the similarity arrangement result. The application splits the data segment to be retrieved, extracts the keyword group in the data segment to be retrieved according to the word frequency and weight, classifies and pools the keyword group, reduces the complexity of the data and improves the accuracy of the retrieval process, calculates the similarity of the first feature vector and the second feature vector, improves the quality of the retrieval result, overcomes the problem of semantic deviation in the traditional text matching mode, and improves the retrieval efficiency.

Description

Technical Field

[0001] This invention relates to the field of big data information technology, specifically to an AI-based data classification and retrieval method and system. Background Technology

[0002] With the rapid development of technologies such as the Internet and the Internet of Things, the amount of data is exploding. In the field of artificial intelligence, massive amounts of data are collected for model training, testing, and optimization. For example, the field of image recognition has massive amounts of image data from various monitoring devices, satellite remote sensing, medical imaging equipment, etc.; the field of natural language processing has a large amount of text data such as social media text, news articles, and academic papers.

[0003] Currently, users select keywords for their search and input them. The system then uses simple text matching to search the database for documents or records containing these keywords, filters the search results, and presents the results to the user.

[0004] However, the above text matching method is poor at handling synonyms and near-synonyms, which leads to semantic deviation from the search content and fails to present the content that meets the user's needs. Summary of the Invention

[0005] Based on this, the purpose of this invention is to provide an AI-based data classification and retrieval method and system, which aims to solve the problem that current retrieval methods have poor processing capabilities for synonyms and near-synonyms, resulting in semantic deviation from the retrieved content and failure to present the content that meets the user's needs.

[0006] To achieve the above objectives, this invention proposes an AI-based data classification and retrieval method, which includes:

[0007] Obtain the data segment to be searched and extract the keyword group;

[0008] The keywords are categorized and labeled, and a search index is established. The search tree is obtained by retrieving data from the database based on the search index.

[0009] Extract the first feature vector of the search index and the second feature vector of the search tree, and calculate the similarity.

[0010] Optimization is performed based on the similarity ranking results.

[0011] In summary, the AI-based data classification and retrieval method proposed in this invention calculates the word frequency and weight of each word in the acquired data segment to be retrieved, extracting keyword groups; classifies and performs pooling analysis on the keyword groups, selecting keywords with certain weight values, establishing a search index; based on the established search index, several search trees close to the data segment to be retrieved are built in the database according to semantic analysis; the first feature vector of the search index and the second feature vector of each search tree are extracted, and the similarity is calculated using the first and second feature vectors to obtain the correlation between each second feature vector and the first feature vector. The loss of the currently established search index is calculated using a loss function to dynamically adjust the preset weight values ​​for optimization. This invention reduces data complexity and improves the accuracy of the retrieval process by splitting the data segment to be retrieved, extracting keyword groups based on word frequency and weight, and classifying and pooling the keyword groups. Simultaneously, the similarity calculation of the first and second feature vectors improves the quality of the retrieval results, overcomes the semantic deviation problem in traditional text matching methods, and improves retrieval efficiency.

[0012] According to one aspect of the above technical solution, the step of obtaining the data segment to be retrieved and extracting the keyword group specifically includes:

[0013] Based on the acquired data segment to be retrieved, identification markers are added to both ends of the data segment to be retrieved, and keywords are extracted from the data segment to be retrieved through semantic model recognition.

[0014] Calculate the word frequency in the data segment to be retrieved:

[0015]

[0016] Among them, T i n represents the word frequency of word i in the data segment to be retrieved. i ∑ is the number of times word i appears in the data segment to be retrieved. k n k It is the total number of words in the data segment to be retrieved.

[0017] According to one aspect of the above technical solution, after obtaining the word frequency, the inverse document frequency F of word i is calculated. i :

[0018]

[0019] Where N is the total number of documents, df i It is the number of documents containing the word i;

[0020] Based on the data segment to be retrieved, the weight W of word i is calculated. P :

[0021] W P =∑ i T i ×F i

[0022] Based on the weights and the semantic model recognition results, keyword groups are extracted.

[0023] According to one aspect of the above technical solution, the step of classifying and identifying the keywords, establishing a search index, and retrieving a search tree from the database based on the search index specifically includes: performing pooling analysis on the classified word groups, classifying and identifying the keywords of the keyword group, so as to obtain word groups with at least noun, adjective, and verb classification labels respectively;

[0024] Keywords with weight values ​​not less than a preset weight value are selected. The word with the highest weight value is used as the root node, and nodes are extended sequentially according to a weight value descending order to construct a search index. Based on the search index, synonyms and / or near-synonyms of word groups are used as root nodes in the database. A semantic model and matching mechanism are used to semantically encode the search index. Codes with a similarity greater than a preset similarity value are searched, and a search tree is built in the database based on these codes. According to one aspect of the above technical solution, the step of extracting the first feature vector of the search index and the second feature vector of the search tree, and calculating the similarity, specifically includes:

[0025] Construct the first feature vector of the search index: Arrange the weight values ​​in the search index from largest to smallest to form the first feature vector; and extract the second feature vector of the search tree, and calculate the similarity between the first feature vector and the second feature vector:

[0026] d(u,0)=u

[0027] d(0,v)=v

[0028] d(u,v)=min{d(u-1,v)+1,d(u,v-1)+1,d(u-1,v-1)+[a[u]≠b[v]]} where a is the first feature vector of the search index, b is the second feature vector of the search tree, d(u,v) is the edit distance required to convert the first u characters in the first feature vector into the first v characters in the second feature vector, a[u] is the u-th character in the first feature vector, b[v] is the v-th character in the second feature vector, and [a[u]≠b[v]] is an indicator function; if [a[u]≠b[v]], its value is 1; if [a[u]=b[v]], its value is 0. Based on the edit distance, the similarity between the first feature vector and the second feature vector is obtained, and the results are arranged based on the similarity to present the search results.

[0029] According to one aspect of the above technical solution, the step of optimizing based on the similarity ranking results specifically includes:

[0030] During keyword pooling classification, the loss function is:

[0031]

[0032] Where c is the number of words, d is the category of word group, and y eg Let y be the one-hot encoding of the true class label for the e-th sample. If sample e belongs to class g, then y eg =1, if sample e does not belong to category g, then y eg =0, Represented as probability vectors for each category;

[0033] Based on the loss function, preset weight values ​​are dynamically adjusted to reduce the loss during the pooling process.

[0034] This invention also proposes an AI-based data classification and retrieval system, which is used to implement the aforementioned AI-based data classification and retrieval method. The AI-based data classification and retrieval system includes:

[0035] The extraction module is used to obtain the data segment to be retrieved and extract keyword groups;

[0036] A module is established to classify and identify the keywords, create a search index, and retrieve a search tree from the database based on the search index.

[0037] The calculation module is used to extract the first feature vector of the search index and the second feature vector of the search tree, and calculate the similarity.

[0038] The optimization module is used to optimize the ranking results based on similarity.

[0039] The present invention also proposes a computer-readable storage medium storing a computer program thereon, characterized in that the program, when executed by a processor, implements the above-mentioned AI-based data classification and retrieval method.

[0040] The present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the aforementioned AI-based data classification and retrieval method.

[0041] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0042] Figure 1 This is a flowchart of the AI-based data classification and retrieval method in Embodiment 1 of the present invention;

[0043] Figure 2 This is a schematic diagram of the structure of the AI-based data classification and retrieval system in Embodiment 2 of the present invention;

[0044] Figure 3 This is a structural block diagram of the electronic device in Embodiment 4 of the present invention. Detailed Implementation

[0045] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Several embodiments of the invention are illustrated in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.

[0046] It should be noted that when a component is said to be "fixed to" another component, it can be directly on the other component or there may be an intervening component. When a component is said to be "connected to" another component, it can be directly connected to the other component or there may be an intervening component. The terms "vertical," "horizontal," "left," "right," and similar expressions used in this document are for illustrative purposes only.

[0047] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0048] Example 1

[0049] like Figure 1 The flowchart shown is a data classification and retrieval method based on AI according to Embodiment 1 of the present invention. The data classification and retrieval method based on AI includes the following steps S01-S04, wherein:

[0050] S01. Obtain the data segment to be searched and extract the keyword group.

[0051] Input the data segment to be retrieved, and add identification tags to both ends of the data segment for semantic model recognition, thereby extracting keywords from the data segment.

[0052] Keyword extraction is based on the frequency of the word in the data segment to be retrieved, as well as the semantic position of the keyword in semantic analysis.

[0053] The formula for calculating the word frequency of keywords in the data segment to be retrieved is:

[0054]

[0055] Among them, T i n represents the word frequency of word i in the data segment to be retrieved. i ∑ is the number of times word i appears in the data segment to be retrieved. k n k It is the total number of words in the data segment to be retrieved.

[0056] The calculation formula above shows that the frequency of a keyword appearing in the data segment to be searched reflects the importance of the keyword in that data segment.

[0057] Furthermore, the inverse document frequency F of word i i :

[0058]

[0059] Where N is the total number of documents, df i This represents the number of documents containing word i. Inverse document frequency (IVF) can be used to represent weighting techniques in information retrieval and text mining, primarily to quantify the importance of word i within the set of data segments to be retrieved. It's worth noting that if a word appears in many documents, it is likely a common word with low importance in distinguishing document content; conversely, if a word appears in only a few documents, it may have high specificity and be significant in distinguishing document content.

[0060] The weight W of word i is calculated based on the term frequency and inverse document frequency of word i. P :

[0061] W P =∑i T i ×F i

[0062] According to weight W P And the semantic model's recognition results, and then extract keyword groups.

[0063] S02. Classify and label the keywords, establish a search index, and retrieve the search tree from the database based on the search index.

[0064] Based on the keyword group obtained in step S01, pooling analysis is performed on the classified keyword group to reduce data dimensionality and retain the most important feature information, selecting marginal features as the most representative, thereby reducing data complexity. Simultaneously, pooling analysis helps reduce computational load and prevent overfitting, making it easier for subsequent classifiers to process these features.

[0065] The keyword group is classified to obtain word groups that have at least the classification labels of nouns, pronouns, verbs, adjectives, numerals, classifiers, adverbs, prepositions, etc.

[0066] The pooled and categorized word groups are filtered to select keywords with weights no less than a preset weight value. The word with the highest weight value is used as the root node, and extension nodes are split and extended according to a weight value descending order to construct a search index. This search index is input into the database to directly retrieve the user's desired search content.

[0067] Furthermore, based on the constructed search index, synonyms and / or near-synonyms of word groups are searched in the database using a semantic model matching mechanism, with the root node being the synonyms and / or near-synonyms of the word group. Based on semantic encoding, synonyms and / or near-synonyms with semantic encoding similarity greater than a preset similarity value are found. These synonyms and / or near-synonyms are then semantically edited, and several search trees are established in the database. These search trees represent the content retrieved by the user.

[0068] S03. Extract the first feature vector of the search index and the second feature vector of the search tree, and calculate the similarity.

[0069] By calculating the similarity between the first feature vector of the search index and the second feature vector of the search tree in the database, the search results are presented to the user according to the similarity ranking rule from high to low, thereby improving the accuracy of the search.

[0070] The output process of the first feature vector is as follows: The weight values ​​in the search index are arranged from largest to smallest to form the first feature vector; similarly, each search tree outputs its own second feature vector in the same way. The similarity between the search tree and the search index is obtained by calculating the first and second feature vectors.

[0071] d(u,0)=u

[0072] d(0,v)=v

[0073] d(u,v)=min{d(u-1,v)+1,d(u,v-1)+1,d(u-1,v-1)+[a[u]≠b[v]]}

[0074] Where a is the first feature vector of the search index, b is the second feature vector of the search tree, d(u,v) is the edit distance required to convert the first u characters in the first feature vector into the first v characters in the second feature vector, a[u] is the u-th character in the first feature vector, b[v] is the v-th character in the second feature vector, and [a[u]≠b[v]] is an indicator function. If [a[u]≠b[v]], its value is 1; if [a[u]=b[v]], its value is 0.

[0075] By calculating the minimum number of edits required to transform a search index into a search tree, the smaller the edit distance, the higher the similarity.

[0076] S04. Optimize based on similarity ranking results.

[0077] Since there are losses during keyword pooling and keyword extraction in step S02, these losses need to be optimized. The loss function is:

[0078]

[0079] Where c is the number of words, d is the category of word group, and y eg Let y be the one-hot encoding of the true class label for the e-th sample. If sample e belongs to class g, then y eg =1, if sample e does not belong to category g, then y eg =0, It is represented as a probability vector for each category.

[0080] Based on the aforementioned loss function, preset weight values ​​are dynamically adjusted to reduce the loss during the pooling process.

[0081] If the loss function value is higher than the preset loss value, the preset weight value is lowered to reduce pooling losses; if the loss function value is lower than the preset loss value, the preset weight value is raised to improve the accuracy of retrieval.

[0082] In summary, the AI-based data classification and retrieval method proposed in this invention calculates the word frequency and weight of each word in the acquired data segment to be retrieved, extracting keyword groups; classifies and performs pooling analysis on the keyword groups, selecting keywords with certain weight values, establishing a search index; based on the established search index, several search trees close to the data segment to be retrieved are built in the database according to semantic analysis; the first feature vector of the search index and the second feature vector of each search tree are extracted, and the similarity is calculated using the first and second feature vectors to obtain the correlation between each second feature vector and the first feature vector. The loss of the currently established search index is calculated using a loss function to dynamically adjust the preset weight values ​​for optimization. This invention reduces data complexity and improves the accuracy of the retrieval process by splitting the data segment to be retrieved, extracting keyword groups based on word frequency and weight, and classifying and pooling the keyword groups. Simultaneously, the similarity calculation of the first and second feature vectors improves the quality of the retrieval results, overcomes the semantic deviation problem in traditional text matching methods, and improves retrieval efficiency.

[0083] Example 2

[0084] In another aspect, the present invention also provides an AI-based data classification and retrieval system, which is used to implement the AI-based data classification and retrieval method in Embodiment 1. Please refer to [link / reference needed]. Figure 2 The diagram shown is a schematic representation of the AI-based data classification and retrieval system in Embodiment 2 of the present invention. The AI-based data classification and retrieval system includes:

[0085] Extraction module 11 is used to obtain the data segment to be retrieved and extract keyword groups;

[0086] Module 12 is used to classify and identify the keywords, and to create a search index, and to retrieve a search tree from the database based on the search index;

[0087] Calculation module 13 is used to extract the first feature vector of the search index and the second feature vector of the search tree, and calculate the similarity.

[0088] Optimization module 14 is used to optimize the ranking results based on similarity.

[0089] Input the data segment to be retrieved, and add identification tags to both ends of the data segment for semantic model recognition, thereby extracting keywords from the data segment.

[0090] Keyword extraction is based on the frequency of the word in the data segment to be retrieved, as well as the semantic position of the keyword in semantic analysis.

[0091] The formula for calculating the word frequency of keywords in the data segment to be retrieved is:

[0092]

[0093] Among them, T i n represents the word frequency of word i in the data segment to be retrieved. i ∑ is the number of times word i appears in the data segment to be retrieved. k n k It is the total number of words in the data segment to be retrieved.

[0094] The calculation formula above shows that the frequency of a keyword appearing in the data segment to be searched reflects the importance of the keyword in that data segment.

[0095] Furthermore, the inverse document frequency F of word i i :

[0096]

[0097] Where N is the total number of documents, df i This represents the number of documents containing word i. Inverse document frequency (IVF) can be used to represent weighting techniques in information retrieval and text mining, primarily to quantify the importance of word i within the set of data segments to be retrieved. It's worth noting that if a word appears in many documents, it is likely a common word with low importance in distinguishing document content; conversely, if a word appears in only a few documents, it may have high specificity and be significant in distinguishing document content.

[0098] The weight W of word i is calculated based on the term frequency and inverse document frequency of word i. P :

[0099] W P =∑ i T i ×F i

[0100] According to weight W P And the semantic model's recognition results, and then extract keyword groups.

[0101] Based on the acquired keyword groups, pooling analysis is performed on the categorized word groups to reduce data dimensionality and retain the most important feature information, selecting marginal features as the most representative, thereby reducing data complexity. Simultaneously, pooling analysis helps reduce computational load and prevent overfitting, making it easier for subsequent classifiers to handle these features.

[0102] The keyword group is classified to obtain word groups that have at least the classification labels of nouns, pronouns, verbs, adjectives, numerals, classifiers, adverbs, prepositions, etc.

[0103] The pooled and categorized word groups are filtered to select keywords with weights no less than a preset weight value. The word with the highest weight value is used as the root node, and extension nodes are split and extended according to a weight value descending order to construct a search index. This search index is input into the database to directly retrieve the user's desired search content.

[0104] Furthermore, based on the constructed search index, synonyms and / or near-synonyms of word groups are searched in the database using a semantic model matching mechanism, with the root node being the synonyms and / or near-synonyms of the word group. Based on semantic encoding, synonyms and / or near-synonyms with semantic encoding similarity greater than a preset similarity value are found. These synonyms and / or near-synonyms are then semantically edited, and several search trees are established in the database. These search trees represent the content retrieved by the user.

[0105] By calculating the similarity between the first feature vector of the search index and the second feature vector of the search tree in the database, the search results are presented to the user according to the similarity ranking rule from high to low, thereby improving the accuracy of the search.

[0106] The output process of the first feature vector is as follows: The weight values ​​in the search index are arranged from largest to smallest to form the first feature vector; similarly, each search tree outputs its own second feature vector in the same way. The similarity between the search tree and the search index is obtained by calculating the first and second feature vectors.

[0107] d(u,0)=u

[0108] d(0,v)=v

[0109] d(u,v)=min{d(u-1,v)+1,d(u,v-1)+1,d(u-1,v-1)+[a[u]≠b[v]]}

[0110] Where a is the first feature vector of the search index, b is the second feature vector of the search tree, d(u,v) is the edit distance required to convert the first u characters in the first feature vector into the first v characters in the second feature vector, a[u] is the u-th character in the first feature vector, v[v] is the v-th character in the second feature vector, and [a[u]≠b[v]] is an indicator function. If [a[u]≠b[b]], its value is 1; if [a[u]=b[v]], its value is 0.

[0111] By calculating the minimum number of edits required to transform a search index into a search tree, the smaller the edit distance, the higher the similarity.

[0112] Since there are losses during keyword pooling and keyword extraction, these losses need to be optimized. The loss function is as follows:

[0113]

[0114] Where c is the number of words, d is the category of word group, and y eg Let y be the one-hot encoding of the true class label for the e-th sample. If sample e belongs to class g, then y eg =1, if sample e does not belong to category g, then y eg =0, It is represented as a probability vector for each category.

[0115] Based on the aforementioned loss function, preset weight values ​​are dynamically adjusted to reduce the loss during the pooling process.

[0116] If the loss function value is higher than the preset loss value, the preset weight value is lowered to reduce pooling losses; if the loss function value is lower than the preset loss value, the preset weight value is raised to improve the accuracy of retrieval.

[0117] In summary, the AI-based data classification and retrieval system proposed in this invention calculates the word frequency and weight of each word in the acquired data segment to be retrieved, extracting keyword groups. It then performs classification and pooling analysis on these keyword groups, selecting keywords with certain weight values ​​and establishing a search index. Based on the established search index, it constructs several search trees in the database that are close to the data segment to be retrieved, using semantic analysis. It extracts the first feature vector of the search index and the second feature vectors of each search tree, and calculates the similarity using the first and second feature vectors to obtain the correlation between each second feature vector and the first feature vector. Furthermore, it calculates the loss of the currently established search index using a loss function to dynamically adjust preset weight values ​​for optimization. This invention reduces data complexity and improves the accuracy of the retrieval process by splitting the data segment to be retrieved, extracting keyword groups based on word frequency and weight, and classifying and pooling these keyword groups. Simultaneously, it improves the quality of retrieval results through the similarity calculation of the first and second feature vectors, overcoming the semantic deviation problem in traditional text matching methods and improving retrieval efficiency.

[0118] Example 3

[0119] In another aspect, the present invention also proposes a computer-readable storage medium having stored thereon one or more computer programs that, when executed by a processor, implement the aforementioned AI-based data classification and retrieval method.

[0120] Those skilled in the art will understand that the logic or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable storage medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable storage medium" can mean any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0121] More specific examples (a non-exhaustive list) of computer-readable storage media include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable storage media can even be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0122] Example 4

[0123] Figure 3 This is a structural block diagram of an electronic device provided in Embodiment 4. The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the AI-based data classification and retrieval method described in the above embodiments. Figure 3 The electronic device 30 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0124] like Figure 3 As shown, the electronic device 30 can be manifested as a general-purpose computing device, such as a server device. The components of the electronic device 30 may include, but are not limited to: at least one processor 31, at least one memory 32, and a bus 33 connecting different system components (including memory 32 and processor 31).

[0125] Bus 33 includes a data bus, an address bus, and a control bus.

[0126] The memory 32 may include volatile memory, such as RAM 321 (random access memory), and / or cache memory 322, and may further include ROM 323 (read-only memory).

[0127] The memory 32 may also include a program tool 325 having a set (at least one) of program modules 324, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0128] The processor 31 executes various functional applications and data processing by running computer programs stored in the memory 32, such as the AI-based data classification and retrieval method of the present invention as described above.

[0129] Electronic device 30 can also communicate with one or more external devices 34 (e.g., keyboard, pointing device, etc.). This communication can be performed via I / O interface 35 (input / output interface). Furthermore, the model-generated electronic device 30 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 36. Figure 3 As shown, network adapter 36 communicates with other modules of the model-generated electronic device 30 via bus 33. It should be understood that, although... Figure 3 As not shown, the electronic device 30 generated in conjunction with the model may use other hardware and / or software modules, including but not limited to: microcode, device drivers, redundant processors, disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems.

[0130] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided and embodied by multiple units / modules. In the description of this specification, references to the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples.

[0131] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. An AI-based data classification and retrieval method, characterized in that, The AI-based data classification and retrieval method includes: Obtain the data segment to be searched and extract the keyword group; Pooling analysis is performed on the classified word groups, and the keywords of the keyword groups are classified and labeled to obtain word groups with at least noun, adjective, and verb classification labels. Keywords with weight values ​​not less than preset weight values ​​are selected. The word with the highest weight value is used as the root node, and the nodes are split and extended sequentially according to the weight value from largest to smallest to construct a search index. Based on the search index, the synonyms and / or near-synonyms of the word groups are used as root nodes in the database. Through a semantic model, the search index is semantically encoded using a matching mechanism. By finding codes whose similarity to the semantic codes is greater than a preset similarity value, a search tree is built in the database based on the codes. Extract the first feature vector of the search index and the second feature vector of the search tree, and calculate the similarity. Optimization is performed based on the similarity ranking results.

2. The AI-based data classification and retrieval method according to claim 1, characterized in that, The steps of obtaining the data segment to be retrieved and extracting keyword groups specifically include: Based on the acquired data segment to be retrieved, identification markers are added to both ends of the data segment to be retrieved, and keywords are extracted from the data segment to be retrieved through semantic model recognition. Calculate the word frequency in the data segment to be retrieved: in, Indicator Word frequency in the data segment to be retrieved It is a word The number of times it appears in the data segment to be retrieved. It is the total number of words in the data segment to be retrieved.

3. The AI-based data classification and retrieval method according to claim 2, characterized in that, After obtaining word frequencies, calculate word frequencies. Inverse document frequency : in, It is the total number of documents. It is a word containing The number of documents; Based on the data segment to be retrieved, calculate the words. weight : × Based on the weights and the semantic model recognition results, keyword groups are extracted.

4. The AI-based data classification and retrieval method according to claim 1, characterized in that, The step of extracting the first feature vector of the search index and the second feature vector of the search tree, and calculating the similarity, specifically includes: Construct the first feature vector of the search index: Arrange the weight values ​​in the search index from largest to smallest to form the first feature vector; and extract the second feature vector of the search tree, and calculate the similarity between the first feature vector and the second feature vector: in, To find the first feature vector of the index, To find the second feature vector of the tree, The first feature vector is the first one. The characters are converted into the first part of the second feature vector. The edit distance required for each character, The first eigenvector is the first eigenvector. One character, The second eigenvector is the first one. One character, For indicator functions, if Its value is 1, if Its value is 0; Based on the edit distance, the similarity between the first feature vector and the second feature vector is obtained, and the results are arranged according to the similarity to present the search results.

5. The AI-based data classification and retrieval method according to claim 1, characterized in that, The optimization step based on the similarity ranking results specifically includes: During keyword pooling classification, the loss function is: Among them, the The number of words. As a category of word groups, Represented as for the first The one-hot encoding of the true class label of each sample, if the sample Category ,but =1, if the sample Not belonging to category ,but =0, Represented as probability vectors for each category; Based on the loss function, preset weight values ​​are dynamically adjusted to reduce the loss during the pooling process.

6. An AI-based data classification and retrieval system, wherein the AI-based data classification and retrieval system is used to implement the AI-based data classification and retrieval method according to any one of claims 1-5, and the AI-based data classification and retrieval system comprises: The extraction module is used to obtain the data segment to be retrieved and extract keyword groups; A module is established to perform pooling analysis on the classified word groups, classify and label the keywords of the keyword groups to obtain word groups with at least noun, adjective, and verb classification labels, filter out keywords with weight values ​​not less than preset weight values, use the word with the highest weight value as the root node, and sequentially split and extend the nodes according to the weight value from largest to smallest to construct a search index. Based on the search index, in the database, using synonyms and / or near-synonyms of the word groups as root nodes, semantically encode the search index using a semantic model and matching mechanism, and build a search tree in the database based on the code by finding codes with a similarity greater than a preset similarity value. The calculation module is used to extract the first feature vector of the search index and the second feature vector of the search tree, and calculate the similarity. The optimization module is used to optimize the ranking results based on similarity.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the AI-based data classification and retrieval method as described in any one of claims 1-5.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes a computer program, it implements the AI-based data classification and retrieval method as described in any one of claims 1-5.

Images