Data retrieval method and system based on multi-dimensional word vector large model and storage medium

Abstract

The application relates to the technical field of data processing, and discloses a data obtaining method and system based on a multi-dimensional word vector large model and a storage medium. The method comprises the following steps: obtaining data triggering an obtaining requirement corresponding to the data obtaining requirement; obtaining explanation question data corresponding to the data obtaining requirement; extracting keywords in the explanation question data by using a preset word segmentation algorithm; calculating keyword vectors corresponding to the keywords by using a conversion vector algorithm; matching multi-dimensional word vectors corresponding to the similarity from a vector database according to the keyword vectors; querying feature data corresponding to the keywords from the vector database according to the matched multi-dimensional word vectors; outputting question content and background content corresponding to the question content by using a preset question template; calling an AI model, inputting the question content and the background content into the AI model; obtaining data results output by the AI model, and completing data obtaining; and reducing the workload of user input to the AI model.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of data processing, and in particular to a data obtaining method and system based on a multi-dimensional word vector large model and a storage medium. BACKGROUND

[0002] AI models are developing rapidly. In existing technical scenarios, some product ecosystems integrate AI intelligent models in order to use the intelligent convenience of AI models, such as assisting in querying data capabilities, so that business personnel can query data by themselves or technical personnel can query data with the assistance of AI models. When a user proposes a data obtaining requirement to an AI model, the user first provides a question and a question background description, and then calls the capability of the AI model to analyze the question and obtain an answer, i.e., a SQL statement of the data obtaining requirement.

[0003] For example, the user needs to query how many students are currently in the sixth grade. Based on this question, the user provides the background of the question, such as the table structure of a grade table and a student table and the association relationship between the tables. After integrating the question and the question background, the capability of the AI model is called to analyze and obtain a SQL statement of the current data obtaining requirement, and the data result that the user wants is obtained by executing the SQL.

[0004] The existing technical defect is that the user needs to know the background context of the current data obtaining requirement when using the AI model, and the AI model may not understand the question and give a correct answer in the case that different people have different understandings of the data obtaining background and different semantic expressions. This is a high requirement and a heavy burden for business and technical personnel. Even if the user has understood the background of the question, the user needs to express more content when asking the question in order to make the AI model understand more accurately, which is time-consuming and laborious and has a heavy burden. SUMMARY

[0005] In order to reduce the workload of the user inputting to the AI model, the present application provides a data obtaining method and system based on a multi-dimensional word vector large model and a storage medium.

[0006] In a first aspect, the present application provides a data obtaining method based on a multi-dimensional word vector large model, which adopts the following technical solution:

[0007] A data obtaining method based on a multi-dimensional word vector large model includes the following steps:

[0008] S1: Based on a data obtaining requirement, obtaining data triggering the data obtaining requirement corresponding to the data obtaining requirement;

[0009] S2: According to the data triggering the data obtaining requirement, obtaining description question data corresponding to the data obtaining requirement;

[0010] S3: Using a preset word segmentation algorithm to extract keywords in the description question data;

[0011] S4: using a conversion vector algorithm in the AI model, calculating a keyword vector corresponding to the keyword, and matching a multidimensional word vector corresponding to the similarity from the vector database according to the keyword vector;

[0012] S5: querying feature data corresponding to the keyword from the vector database according to the matched multidimensional word vector, the feature data including technical description, data structure, and table relationship;

[0013] S6: outputting a question content and background content corresponding to the question content through a preset question template from the feature data;

[0014] S7: calling an AI model, and inputting the question content and the background content into the AI model;

[0015] S8: obtaining a data result output by the AI model, and completing data acquisition.

[0016] By using the above technical solution, through keyword extraction, conversion vector, and vector matching of user questions, the keywords in the questions are precisely associated with the vector database through word vector mapping and matching, sufficient related context is provided for the AI model, and the AI model generates results that meet the question intention; the user only needs to simply ask questions to obtain the required business data, greatly reducing the user's burden, improving the data acquisition efficiency and quality, and solving the problem that the user needs to bear the heavy responsibility of describing the problem background in the business data acquisition scenario, and different cognition cannot accurately express the problem.

[0017] Optionally, the conversion vector algorithm includes the following steps:

[0018] S41: calculating a plurality of key features related to the keyword based on the AI model;

[0019] S42: placing a conversion number of the key features into a preset empty vector template in the order of the calculation results to form the keyword vector;

[0020] S43: the value of the conversion number is related to the value of the number of occurrences of the keyword in the description question data; the greater the value of the number of occurrences, the smaller the value of the conversion number; the smaller the value of the number of occurrences, the greater the value of the conversion number.

[0021] By using the above technical solution, the accurate semantic similarity between words can be calculated after the keyword is converted by the conversion vector algorithm, which enables the related business and technical information in the vector database to be fully recalled without additional description.

[0022] Optionally, the step S4 includes the following steps:

[0023] S44: The vector database has a plurality of feature data, and the feature data is mapped with a corresponding multi-dimensional word vector, and the multi-dimensional word vector belongs to the vector database;

[0024] S45: Calculate the Euclidean distance between the keyword vector and the multi-dimensional word vector;

[0025] S46: Output the similarity between the keyword and the feature data according to the Euclidean distance.

[0026] By adopting the above technical solution, in the vector database, each feature data, that is, the vocabulary, is mapped to a multi-dimensional word vector, and the Euclidean distance between the multi-dimensional word vectors reflects the semantic similarity between the vocabularies.

[0027] Optionally, the step S7 further includes the following steps:

[0028] S71: Obtain a question template, the question template is based on a preset syntax structure, the syntax structure includes a plurality of content containers, each content container corresponds to an adjustment parameter, and the enabling state of the content container is changed according to the adjustment parameter;

[0029] S72: Put the question content and the background content into the corresponding content container in the question template;

[0030] S73: Input the question template into the AI model.

[0031] By adopting the above technical solution, a simple and unified syntax structure is used, supplemented by necessary adjustment parameters, to form a unified and easy-to-understand question template for the AI model, which is beneficial to cover the data extraction requirements of different businesses.

[0032] Optionally, the step S8 further includes the following steps:

[0033] S81: Obtain the answer content corresponding to the question content output by the AI model according to the preset answer template, and execute the data extraction SQL statement contained in the answer content;

[0034] S82: Automatically execute the data extraction SQL statement to obtain a data result, and convert the data result into a graphic result through a preset data image conversion algorithm;

[0035] S83: Output the data result and the graphic result.

[0036] By adopting the technical scheme, the answer template defines a standardized template, which can contain an SQL statement accurately matching a business requirement and a template answer mode of an AI model, greatly simplifying the complexity of subsequent automatic recognition and extraction of the data retrieval SQL statement and improving the accuracy and recall rate of data retrieval automation.

[0037] Optionally, the vector database is a text database, the content of the keyword vector is a first string, and the content of the multi-dimensional word vector is a second string.

[0038] By adopting the technical scheme, the vector database can be replaced by a common text database, and the user's question does not need to be converted into a vector. However, this scheme is poor in matching accuracy and breadth of keyword-related information, cannot model semantic relationships between words, and depends on the matching of word surface similarities, so the accuracy and richness are low.

[0039] Optionally, the step S46 further includes the following steps:

[0040] S461: comparing the Euclidean distance with a preset reference distance;

[0041] S462: if the Euclidean distance is less than the reference distance, outputting the keyword, otherwise calculating the next Euclidean distance;

[0042] S463: calculating an average number of the feature data corresponding to the processed keyword and a current number of the feature data corresponding to the current keyword, and calculating a number difference between the current number and the average number;

[0043] S464: adjusting the reference distance according to the number difference; the greater the number difference, the smaller the value of the reference distance, and the smaller the number difference, the greater the value of the reference distance.

[0044] By adopting the technical scheme, the reference distance is adjusted in real time according to the actual calculation process, so that the calculation result is more suitable for actual requirements, and the quality of data retrieval is improved.

[0045] Optionally, the step S46 further includes the following steps:

[0046] S461: comparing the Euclidean distance with a preset distance range;

[0047] S462: if the Euclidean distance is within the distance range, outputting the keyword, otherwise calculating the next Euclidean distance;

[0048] S463: Calculate the overall average distance value of the feature data corresponding to the processed keyword, and the current average distance value of the feature data corresponding to the current keyword, and calculate the distance difference value between the current average distance value and the overall average distance value;

[0049] S464: Adjust the distance range according to the distance difference value; the greater the distance difference value, the greater the distance range; the smaller the distance difference value, the smaller the distance range.

[0050] By adopting the above technical solution, the distance range is adjusted in real time according to the actual calculation process, so that the calculation result is more suitable for actual needs, and the quality of data acquisition is improved.

[0051] In a second aspect, the present application provides a data acquisition system based on a multi-dimensional word vector large model, which adopts the following technical solution:

[0052] A data acquisition system based on a multi-dimensional word vector large model, comprising a processor, wherein the processor runs a program of the data acquisition method based on the multi-dimensional word vector large model according to any one of the above.

[0053] In a third aspect, the present application provides a storage medium, which adopts the following technical solution:

[0054] A storage medium storing a program of the data acquisition method based on the multi-dimensional word vector large model according to any one of the above.

[0055] In summary, the present application includes at least one of the following beneficial technical effects: through the process of keyword extraction, vector conversion and vector matching of user questions, the keywords in the questions are precisely associated with the vector database through word vector mapping and matching, sufficient related context is provided for the AI model, and the AI model generates results that meet the intention of the questions; users only need to ask simple questions to obtain the required business data, greatly reducing the burden on users, improving the efficiency and quality of data acquisition, and solving the problem that users need to bear the heavy responsibility of describing the problem background in the business data acquisition scene, and different cognition cannot accurately express the problem. BRIEF DESCRIPTION OF DRAWINGS

[0056] Figure 1 is the overall method flow diagram of the data acquisition method based on the multi-dimensional word vector large model in the embodiments of the present application.

[0057] Figure 2 is the method flow diagram of the vector conversion algorithm in the embodiments of the present application.

[0058] Figure 3 is the method flow diagram of the similarity calculation method in the embodiments of the present application.

[0059] Figure 4 is a method flowchart for adjusting a reference distance in an embodiment of the present application.

[0060] Figure 5 is a method flowchart for adjusting a distance range in an embodiment of the present application.

[0061] Figure 6 is a method flowchart for a sub-step of step S7 in an embodiment of the present application.

[0062] Figure 7 is a method flowchart for a sub-step of step S8 in an embodiment of the present application. DETAILED DESCRIPTION

[0063] Embodiments of the present application are described in detail below, examples of which are shown in the accompanying drawings. Figures 1-7

[0064] In the description of the present specification, the description of the terms "certain embodiments", "one embodiment", "some embodiments", "illustrative embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in connection with the described embodiments or examples are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily mean the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0065] An embodiment of the present application discloses a data taking method based on a multi-dimensional word vector large model, referring to Figure 1 , comprising the following steps:

[0066] S1: Based on the data taking demand, the data taking trigger data corresponding to the data taking demand is obtained. The user can enter the corresponding question and explanation through the set input window, or can realize the problem and explanation into a certain format file, and then drag the file into the input window or read or open the file through the setting program to obtain the content in the file.

[0067] S2: According to the data taking trigger data, the explanation question data corresponding to the data taking demand is obtained. After the question and explanation are obtained, the question and explanation are integrated into the explanation question data, that is, the content of the question and the content of the explanation are maintained as new data using an embedded document template, wherein the document template can be a txt template, a word template, etc.

[0068] ​S3: Extract the keywords in the instruction question data using a preset segmentation algorithm. The segmentation algorithm includes extracting keywords using a segmentation tool, including NLPIR, HanLP, THULAC, jieba segmentation, or FudanNLP, etc. NLPIR segmentation is a text analysis method based on natural language processing and information retrieval, which can help to divide natural language text into meaningful words or phrases. It has a wide range of applications in information retrieval, text mining and natural language processing. Using NLPIR segmentation requires installing the segmentation library and performing corresponding import, loading, processing and other operations. Through NLPIR segmentation, the keywords and key phrases in the text can be extracted, which can improve the accuracy and efficiency of text analysis and provide basic support for subsequent semantic analysis. HanLP is an open source toolkit for Chinese natural language processing, which provides multiple segmentation methods, including standard segmentation, NLP segmentation, index segmentation, N-shortest path segmentation, CRF segmentation and ultra-fast dictionary segmentation, etc. THULAC is a high-efficiency Chinese lexical analysis toolkit with Chinese segmentation and part-of-speech tagging functions. The main function of jieba segmentation is to do Chinese segmentation, which can perform simple segmentation, parallel segmentation, command line segmentation, and of course it also supports keyword extraction, part-of-speech tagging, word position query and other functions. FudanNLP is an open source Chinese natural language processing (NLP) toolkit, which contains Chinese segmentation, keyword extraction, named entity recognition, part-of-speech tagging, time word extraction, syntax analysis and other functions, which are used in search engines, text analysis and other scenarios.

[0069] S4: using the conversion vector algorithm in the AI model, the keyword vector corresponding to the keyword is calculated, and the multidimensional word vector corresponding to the similarity is matched from the vector database according to the keyword vector. The vector database can also be called a vector knowledge base. The vector knowledge base can be pre-set with professional vocabulary in the customer business scenario, which can more accurately understand the business semantics of the user's question, accurately match the relevant technical description, and improve the effect of question answering and information retrieval. The role of the conversion vector algorithm is vectorization, which is a very common way to accelerate calculation, especially suitable for deep learning and other fields that require training of large data. In machine learning, data is usually represented in the form of vectors. Feature vectors are vectors that represent data features. They can contain information in multiple dimensions, such as image pixel values, text word frequencies, etc. Through processing and analysis of feature vectors, data classification, clustering, and other tasks can be achieved. The vector space model VSM is a commonly used text representation method that converts text into vector form to calculate text similarity, information retrieval, and other tasks. In VSM, each word in the text can be considered as a dimension, and the word vector of the text represents the feature value of the text in different dimensions. Word vectors are a vector representation method used to represent word semantics. By mapping words to vector space, word similarity calculation, text classification, sentiment analysis, and other tasks can be achieved. Common word vector models include Word2Vec, GloVe, etc.

[0070] For example, as shown in Figure 2 The conversion vector algorithm includes the following steps:

[0071] S41: based on the AI model, calculate a plurality of key features related to the keyword.

[0072] S42: place the converted number of key features in the pre-set empty vector template in the order of the calculation results to form a keyword vector.

[0073] S43: the value of the conversion number is related to the value of the number of times the keyword appears in the question data; the greater the value of the number of times, the smaller the value of the conversion number; the smaller the value of the number of times, the greater the value of the conversion number.

[0074] After the keyword is converted by the conversion vector algorithm, the precise semantic similarity between words can be calculated, which enables the relevant business and technical information in the vector time library to be fully recalled without additional description.

[0075] As shown in Figure 3 The similarity calculation method in step S4 includes the following steps:

[0076] S44: The vector database has a plurality of feature data, and the feature data is mapped with a corresponding multi-dimensional word vector, and the multi-dimensional word vector belongs to the vector database.

[0077] S45: Calculate the Euclidean distance between the keyword vector and the multi-dimensional word vector.

[0078] S46: Output the similarity between the keyword and the feature data according to the Euclidean distance.

[0079] In the vector database, each feature data, i.e. vocabulary, is mapped to a multi-dimensional word vector, and the Euclidean distance between the multi-dimensional word vectors reflects the semantic similarity between the vocabularies. Among them, the vector data is a vector composed of a plurality of sub-data, and the sub-data can be numerical, character or string content, etc. Each vector represents a point or feature in a multi-dimensional space. Euclidean distance is a common concept in mathematics, which refers to the real distance between two points in m-dimensional space. In two-dimensional and three-dimensional space, the Euclidean distance can be intuitively understood as the straight-line distance between two points. In two-dimensional space, the formula of Euclidean distance is:

[0080] d=sqrt((x1-x2)^2+(y1-y2)^2);

[0081] In three-dimensional space, the formula is: d=sqrt((x1-x2)^2+(y1-y2)^2+(z1-z2)^2).

[0082] In n-dimensional space, the formula of Euclidean distance is d=sqrt(∑(xi1-xi2)^2), where i=1,2,...,n, xi1 represents the i-th dimensional coordinate of the first point, and xi2 represents the i-th dimensional coordinate of the second point. Euclidean distance can also be regarded as the similarity between signals. The closer the distance, the higher the similarity; the farther the distance, the lower the similarity. In computer science, data analysis and machine learning, Euclidean distance is often used to calculate the similarity or difference between data points, so as to perform clustering, classification or regression analysis, etc. The idea of Euclidean distance is to standardize the data in each dimension first, and then calculate the standardized value=(value before standardization-mean of component) / standard deviation of component, and then calculate the Euclidean distance. The formula can use the formula in the prior art, which will not be described in detail herein.

[0083] As shown in Figure 4 , in the present embodiment, step S46 further includes the following steps:

[0084] S461: Compare the Euclidean distance with the preset reference distance.

[0085] S462: If the Euclidean distance is less than the reference distance, output the keyword, otherwise calculate the next Euclidean distance.

[0086] S463: Calculate the average number of feature data corresponding to the processed keywords, and the current number of feature data corresponding to the current keyword, and calculate the number difference between the current number and the average number.

[0087] S464: Adjust the reference distance according to the number difference; the greater the number difference, the smaller the value of the reference distance, and the smaller the number difference, the greater the value of the reference distance.

[0088] According to the actual calculation process, the reference distance is adjusted in real time, so that the calculation result is more suitable for the actual demand, and the quality of data acquisition is improved. For example, the value of the reference distance is 5, the keyword a corresponds to 3 feature data, the keyword b corresponds to 5 feature data, and the keyword c corresponds to 4 feature data, and the average number of feature data corresponding to the processed keywords is 4. If the current number of feature data corresponding to the current keyword is 6, the number difference between the current number and the average number is 2, and the value of the reference distance can be reduced to 4.5 or 4. If the current number of feature data corresponding to the current keyword is 3, the number difference between the current number and the average number is -1, and the value of the reference distance can be increased to 5.5 or 6.

[0089] As shown in FIG. 4, in another embodiment, step S46 further includes the following steps: Figure 5

[0090] S461: Compare the Euclidean distance with the preset distance range.

[0091] S462: If the Euclidean distance is within the distance range, output the keyword, otherwise calculate the next Euclidean distance.

[0092] S463: Calculate the overall average distance value of the feature data corresponding to the processed keywords, and the current average distance value of the feature data corresponding to the current keyword, and calculate the distance difference between the current average distance value and the overall average distance value.

[0093] S464: Adjust the distance range according to the distance difference; the greater the distance difference, the greater the distance range; the smaller the distance difference, the smaller the distance range.

[0094] ​According to the actual calculation process, the distance range is adjusted in real time, so that the calculation result is more in line with the actual demand, and the quality of data acquisition is improved. For example, the reference distance range value is (4, 6), the keyword a corresponds to 3 feature data, the keyword b corresponds to 5 feature data, and the keyword c corresponds to 4 feature data, and the average number of feature data corresponding to the keyword after calculation processing is 4. If the current number of feature data corresponding to the current keyword is 6, the number difference between the current number and the average number is 2, and the reference distance value can be reduced to (4.5, 5.5). If the current number of feature data corresponding to the current keyword is 3, the number difference between the current number and the average number is -1, and the reference distance value can be increased to (3.5, 6.5).

[0095] S5: According to the matched multi-dimensional word vector, the feature data corresponding to the keyword is queried from the vector database, and the feature data includes technical description, data structure and table relationship. In other embodiments, the vector database can also be a text database, and the content of the keyword vector can also be a first string, and the content of the multi-dimensional word vector can also be a second string. The vector database can be replaced by a general text database, and the user's question does not need to be converted into a vector, but this scheme is poor in matching accuracy and breadth of obtaining keyword related information, cannot model the semantic relationship between words, and depends on the matching of word surface similarity, and the accuracy and richness are low.

[0096] S6: The feature data is output through the preset question template to output the question content and the background content corresponding to the question content.

[0097] S7: The AI model is called, and the question content and the background content are input into the AI model.

[0098] As shown in Figure 6 , step S7 further includes the following steps:

[0099] S71: Obtain the question template, the question template is based on a preset syntax structure, the syntax structure includes a plurality of content containers, each content container corresponds to an adjustment parameter, and the enable state of the content container is changed according to the adjustment parameter. For example, the syntax structure can include content containers such as words, environment, action, etc. Based on the total number of times of action words appearing in the last N question data in the history record of the question, if the total number of times is less than a preset number of times reference value, the adjustment parameter of the content container corresponding to the action is set to 0, at this time the content in the content container is empty or 0. If the total number of times is greater than the preset number of times reference value in the subsequent calculation process, the adjustment parameter of the content container corresponding to the action is set to 1, at this time the content container normally puts in the content of the action.

[0100] S72: Put the question content and the background content into the corresponding content container in the question template.

[0101] S73: input the question template into the AI model.

[0102] Using a simple and unified syntax structure, supplemented by necessary adjustment parameters, a unified and easy-to-understand question template for AI models is formed, which is beneficial to cover the data retrieval needs of different businesses.

[0103] S8: obtain the data result output by the AI model, and complete the data retrieval.

[0104] As shown in Figure 7 , step S8 further includes the following steps:

[0105] S81: obtain the answer content corresponding to the question content output by the AI model according to the preset answer template, and execute the data retrieval SQL statement contained in the answer content. The answer template can also be based on another preset syntax structure, and the other syntax structure also includes a plurality of content containers, each of which corresponds to an adjustment parameter, and the enable state of the content container is changed according to the adjustment parameter. For example, the syntax structure can include content containers such as words, environment, action, data retrieval SQL statement, etc. Based on the total number of times of action words appearing in the last N question data in the history record of the question, if the total number of times is less than a preset number of times reference value, the adjustment parameter of the content container corresponding to the action is set to 0, at which time the content in the content container is empty or 0. If the total number of times is greater than the preset number of times reference value in the subsequent calculation process, the adjustment parameter of the content container corresponding to the action is set to 1, at which time the content container normally puts in the content of the action.

[0106] S82: automatically execute the data retrieval SQL statement to obtain the data result, and convert the data result into a graphical result through a preset data image conversion algorithm. Read the content container of the data retrieval SQL statement in the answer template, obtain the data retrieval SQL statement and execute it to obtain the corresponding data result from the database. Alternatively, read the content container of the data retrieval SQL statement in the answer template, obtain the data retrieval SQL statement and execute it to obtain the corresponding data from the database, and combine the obtained data into a data result. Before executing the data retrieval SQL statement, the data retrieval SQL statement will be identified, and if the data retrieval SQL statement does not meet the preset statement content requirements or statement format requirements, the data retrieval SQL statement will be abandoned. The data image conversion algorithm can present the obtained data result in the form of a column chart table to form a graphical result. The data image conversion algorithm can also display the data result in a preset queue, and match different contents with different display colors and other display features. The data image conversion algorithm can call the API of existing text-to-image or text-to-video software, input the data result into the software, obtain the picture or video output by the software, and display the picture or video beside the data result.

[0107] S83: output the data result and the graphical result.

[0108] The answer template defines a standardized template, which can include an SQL statement accurately matching the business requirement, template the answer mode of the AI model, and greatly simplify the complexity of subsequent automatic identification and extraction of the data retrieval SQL statement, and improve the accuracy and recall rate of the data retrieval automation. Through the bidirectional standardization of the question template and the answer template, the understanding ability of the AI model is fully utilized, so that the AI can still provide high-quality automated business data retrieval service in a complex question scenario, greatly improving the user experience.

[0109] To sum up, the application extracts keywords from user questions, converts vectors, and matches vectors. Through word vector mapping and matching, the word vector technology is used to split the question content, extract keywords, and find the corresponding word vectors when the user asks a question. By calculating the similarity between word vectors, it can be determined which business terms or technical terms in the vector database are most relevant to the keywords, providing a basis for subsequent positioning of related business descriptions or technical descriptions, data structures, and table relationships. Thus, the keywords in the question are precisely associated with the vector database, providing sufficient relevant context for the AI model to support the AI model to generate results that meet the question intent. Considering that the understanding and recognition ability of the AI model varies under different question modes.

[0110] To enable users to obtain the most accurate results in the business data retrieval scenario, a unified and easy-to-understand question template for the AI model is added, which covers different business data retrieval needs and uses a simple and unified syntax structure with necessary parameters. At the same time, a standardized answer template for the AI model is defined, which contains SQL statements that accurately match business requirements, and template and unify the answer mode of the AI model, which can greatly simplify the complexity of subsequent automatic identification and extraction of the data retrieval SQL statement, and improve the accuracy and recall rate of the data retrieval automation. Users only need to ask simple questions to obtain the required business data, greatly reducing the user's burden, improving the efficiency and quality of data retrieval, and solving the problem of users needing to describe the problem background in the business data retrieval scenario, and different cognitive difficulties in accurately expressing the problem.

[0111] The application also discloses a data retrieval system based on a multi-dimensional word vector large model, comprising a processor, and the processor runs a program of the data retrieval method based on the multi-dimensional word vector large model according to any one of the above.

[0112] The application also discloses a storage medium storing the program of the data retrieval method based on the multi-dimensional word vector large model according to any one of the above.

[0113] Although the embodiments of the present application have been shown and described above, it is understood that the above-described embodiments are exemplary and are not to be construed as limiting the present application, and that changes, modifications, substitutions and variations can be made by those skilled in the art without departing from the scope of the present application.

Claims

1. A data acquisition method based on a multi-dimensional word vector large model, characterized in that, Includes the following steps: S1: Based on the data retrieval requirements, obtain the data retrieval trigger data corresponding to the data retrieval requirements; S2: Based on the data retrieval trigger data, obtain the explanatory question data corresponding to the data retrieval requirement; S3: Use a preset word segmentation algorithm to extract keywords from the description question data; S4: Using the conversion vector algorithm in the AI ​​model, calculate the keyword vector corresponding to the keyword, and match the multi-dimensional word vector corresponding to the similarity from the vector database based on the keyword vector; S5: Based on the matched multidimensional word vectors, retrieve the feature data corresponding to the keywords from the vector database. The feature data includes technical descriptions, data structures, and table relationships. S6: Output the question content and the corresponding background content through the preset question template using the feature data; S7: Invoke the AI ​​model and input the question and background information into the AI ​​model; S8: Obtain the data output from the AI ​​model and complete the data retrieval; The transformation vector algorithm includes the following steps: S41: Calculate multiple key features related to the keyword based on the AI ​​model; S42: Place the key features of the conversion quantity into a preset empty vector template according to the order of the calculated results to form the keyword vector; S43: The conversion count is related to the number of times the keyword appears in the description question data; the larger the number of occurrences, the smaller the conversion count; the smaller the number of occurrences, the larger the conversion count.

2. The data acquisition method based on a large multidimensional word vector model according to claim 1, characterized in that, The similarity calculation method in step S4 includes the following steps: S44: The vector database contains multiple feature data, and the feature data is mapped to corresponding multi-dimensional word vectors, which belong to the vector database. S45: Calculate the Euclidean distance between the keyword vector and the multidimensional word vector; S46: Output the similarity between the keyword and the feature data based on the Euclidean distance.

3. The data acquisition method based on a large multidimensional word vector model according to claim 1, characterized in that, Step S7 also includes the following steps: S71: Obtain a question template, the question template is based on a preset syntax structure, the syntax structure includes multiple content containers, each content container has an adjustment parameter, and the activation state of the content container is changed according to the adjustment parameter; S72: Place the question content and the background content into the corresponding content container in the question template; S73: Input the question template into the AI ​​model.

4. The data acquisition method based on a large multidimensional word vector model according to claim 1, characterized in that, Step S8 also includes the following steps: S81: Obtain the AI ​​model's output of the answer content corresponding to the question content according to the preset answer template, and execute the data retrieval SQL statement contained in the answer content; S82: Automatically execute the data retrieval SQL statement to obtain the data result, and convert the data result into a graphic result through a preset data image conversion algorithm; S83: Output the data results and the graphical results.

5. The data acquisition method based on a large multidimensional word vector model according to claim 1, characterized in that, The vector database is a text database, the keyword vector contains a first string, and the multidimensional word vector contains a second string.

6. The data acquisition method based on a large multidimensional word vector model according to claim 2, characterized in that, Step S46 further includes the following steps: S461: Compare the Euclidean distance with a preset reference distance; S462: If the Euclidean distance is less than the reference distance, output the keyword; otherwise, calculate the next Euclidean distance. S463: Calculate the average number of feature data corresponding to the processed keyword and the current number of feature data corresponding to the current keyword, and calculate the difference between the current number and the average number; S464: Adjust the reference distance according to the quantity difference; The larger the difference in quantity, the smaller the value of the reference distance; the smaller the difference in quantity, the larger the value of the reference distance.

7. The data acquisition method based on a large multidimensional word vector model according to claim 2, characterized in that, Step S46 further includes the following steps: S461: Compare the Euclidean distance with a preset distance range; S462: If the Euclidean distance is within the distance range, output the keyword; otherwise, calculate the next Euclidean distance. S463: Calculate the overall average distance value of the feature data corresponding to the processed keyword, and the current average distance value of the feature data corresponding to the current keyword, and calculate the distance difference between the current average distance value and the overall average distance value; S464: Adjust the distance range according to the distance difference; The larger the distance difference, the larger the distance range; The smaller the distance difference, the smaller the distance range.

8. A data retrieval system based on a large multidimensional word vector model, characterized in that, Includes a processor, wherein the processor runs a program for the data retrieval method based on a large multidimensional word vector model as described in any one of claims 1-7.

9. A storage medium, characterized in that, The program stores the data retrieval method based on a large multidimensional word vector model as described in any one of claims 1-7.

Images