Semantic retrieval method and system for structured data

Abstract

The invention discloses a semantic retrieval method and system for structured data, and relates to the technical field of data analysis, and the method comprises the following steps: setting a label of field information in a relational database, expanding a field initial code according to input data, generating a field expansion code, and querying a jump order, and updating the Markdown document in combination with the field extension code and the query jump order, obtaining jump field information, and constructing a corresponding relationship between the query jump order and the second timestamp. According to the method, the retrieval range is dynamically adjusted by querying the jump order, full-table scanning is avoided, redundant calculation is remarkably reduced, field codes are automatically expanded according to input data, service rapid iteration is adapted, document changes are marked through the first timestamp, the latest effective fields are positioned in combination with the jump order, and historical data are prevented from being repeatedly processed; full-link automation from label setting to document generation, such as Markdown document automatic update field description.

Description

Technical Field

[0001] This invention relates to the field of data analysis technology, and in particular to a semantic retrieval method and system for structured data. Background Technology

[0002] In recent years, with the development of large models and vector databases, unstructured data, such as text and images, has been transformed into structured semantic features, thereby achieving unified semantic retrieval. Semantic retrieval of structured data is no longer limited to traditional keywords or SQL queries, but integrates vector retrieval, full-text retrieval and structured data query to achieve multi-path recall of scalar, vector and full-text data.

[0003] Currently, Chinese invention patent CN120407752A discloses an intelligent question-answering system and method for a large-scale traffic engineering model based on knowledge graphs. This system receives and parses user-input natural language queries, extracts structured features, semantic ambiguity features, and contextual features to generate a search intent vector. It then dynamically selects a search path based on an intent classification model. According to the search path, it constructs either a structured query or a semantic vector and searches in both the knowledge graph and a vector database to obtain a first and a second search result. The system performs bidirectional verification using entity consistency, semantic similarity, and relational connectivity indicators to filter candidate result sets and construct a reasoning chain. If the reasoning chain breaks, a large-scale model is used to resolve the issue. The inference gap-filling mechanism generates relay nodes to form a complete inference chain and generates inferential answer output based on the complete chain. Although this method improves the retrieval accuracy and inference coherence of the question-answering system, related technologies only use the numerical value of cosine similarity to represent the similarity between the query vector and the candidate vector, without dynamically updating the similarity according to different data types. This can easily lead to the similarity evaluation results deviating from the true semantic association. At the same time, this method expands the inference chain one by one until it is connected before outputting the inference result, resulting in low retrieval efficiency and high resource consumption. Especially in complex multi-hop query scenarios, the response latency increases significantly. This method is also prone to overfitting on small sample data, etc., and has certain limitations. Summary of the Invention

[0004] The technical problem solved by this invention is that related technologies only use the numerical value of cosine similarity to represent the similarity between query vectors and candidate vectors, without dynamically updating the similarity based on different data types. This can easily lead to the similarity evaluation results deviating from the true semantic association. At the same time, this method expands the reasoning chain one by one until it is connected before outputting the reasoning result, resulting in low retrieval efficiency and high resource consumption. In particular, the response delay increases significantly in complex multi-hop query scenarios. This method is also prone to overfitting on small sample data, etc., and has certain limitations.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution: First aspect, a semantic retrieval method for structured data, comprising the following steps: Step S1, setting labels for field information in a relational database according to a preset processing method;

[0006] Step S2: Construct the initial field code based on the label, expand the initial field code based on the input data, and generate the expanded field code and query jump order;

[0007] Step S3: Update the markdown document by combining field extension encoding and query jump order, generate the first document and extract the first timestamp;

[0008] Step S4: Obtain the jump field information based on the query jump order and the first timestamp, and verify the jump field information based on the label;

[0009] Step S5: Construct the correspondence between the query jump order and the second timestamp. The second timestamp is obtained by reconstructing the field extended encoding based on the first timestamp, updating the markdown document based on the reconstructed field extended encoding, and extracting the timestamp of the markdown document, which is recorded as the second timestamp.

[0010] As a preferred embodiment of the semantic retrieval method for structured data described in this invention, step S1 includes the following sub-steps: S11, obtaining field information from a relational database, wherein the field information is represented as a standardized description name, a standardized description word, and a standardized description sentence;

[0011] Step S12: Select the main table and the subordinate table corresponding to any data subject, and number the main table and the subordinate table. The number of the main table is represented as X. i The table number is represented as Y. ij , and , where i represents the i-th data subject and j represents the j-th sub-table corresponding to the subject;

[0012] Step S13: Number the corresponding character information according to the main table number and the slave table number. After the main table number and slave table number of the current data subject are set, proceed to step S14.

[0013] Step S14: Jump to the next data subject and repeat steps S12 to S13 until the main table number and slave table code of all data subjects are set, then jump to step S15.

[0014] Step S15: After all the main table numbers and slave table numbers of all data subjects have been set, proceed to step S16.

[0015] Step S16: Identify the data type of the field information, classify the data body according to the data type, and set the matching rules for the classified data body;

[0016] Step S17: According to the matching rules, set the labels for the field information of the data subject. The labels include main table labels and sub-table labels. The main table labels include explicit main table labels and implicit main table labels. The sub-table labels include explicit sub-table labels and implicit sub-table labels.

[0017] As a preferred embodiment of the semantic retrieval method for structured data described in this invention, step S16 includes setting explicit labels and setting implicit labels.

[0018] As a preferred embodiment of the semantic retrieval method for structured data described in this invention, step S1611 involves obtaining field information, judging the content in the field information, and when a descriptive sentence exists in the field information, performing word segmentation on the descriptive sentence to obtain the word segmentation result, and then proceeding to step S1612.

[0019] If the field information does not contain a descriptive sentence, proceed to step S1613;

[0020] Step S1612: For the word segmentation results, set the sliding step size, which is an integer. Slide the word segmentation results according to the sliding step size to obtain each first word. According to the TextRank formula, obtain the weight of the first word. Set the first keyword according to the weight of the first word. Jump to step S1614.

[0021] When the sliding step size decreases to 1, and all the first step sizes are less than the first value, the corresponding description sentence is deleted, and the corresponding position is set to the first character;

[0022] Step S1613: Identify the descriptive words. When the descriptive word is text, set the descriptive word as the second keyword and proceed to step S1614. When the descriptive word is a number, proceed to step S1615.

[0023] Step S1614: Set the first keyword or the second keyword as a candidate word. Based on the one-to-many correspondence between the main table and the subordinate table in the Markdown document, when the table attribute is a subordinate table, the corresponding subordinate description sentence or subordinate description word is obtained. Return to step S1612 to obtain the first subordinate keyword or the second subordinate keyword corresponding to the subordinate description sentence or subordinate description word. Calculate the similarity between the first keyword and the first subordinate keyword according to the cosine similarity formula, or calculate the similarity between the second keyword and the second subordinate keyword. Based on the similarity, select to set discrete and irrelevant fields as explicit labels for the field information where the candidate word is located, or jump to step S1616.

[0024] When the table attribute is a subordinate table, jump to the next candidate word until the table attribute is a main table. Based on the one-to-many correspondence between the main table and subordinate tables in the markdown document, obtain the corresponding subordinate description sentence or subordinate description word, and return to step S1612.

[0025] Step S1615: When the data type is date, set consecutive and parallel terms as explicit labels for the field information where the candidate words are located. When the data type is numeric, traverse the field information from top to bottom, calculate the first difference between the next descriptor and the previous descriptor, and obtain the difference curve by using the traversal sequence number as the horizontal axis and the first difference as the vertical axis according to the traversal order. Judge the trend of the difference curve according to the MK test method, obtain the judgment result, and set explicit labels according to the judgment result.

[0026] Step S1616: Add a conjunction to the adjacent first keyword or the adjacent second keyword. The conjunction includes "greater than" and "less than". The method of adding the conjunction includes adding it after the previous keyword, calling the language model, judging the semantic rationality of the connected keyword combination, and if the semantics are not reasonable, jumping to the next conjunction and repeating the addition and rationality judgment process. If the semantics are reasonable, setting discrete and increasing or discrete and decreasing as explicit labels.

[0027] When the added conjunction is greater than, the explicit label is discrete and decreasing; when the added conjunction is greater than, the explicit label is discrete and increasing.

[0028] As a preferred embodiment of the semantic retrieval method for structured data described in this invention, the method of setting explicit labels includes the following sub-steps: Step S1621, obtaining the field information of the main table and obtaining the description name in the field information;

[0029] Step S1622: According to the preset set matching rules, the description names are categorized into various sets.

[0030] Step S1623: Based on the relationship between sets, obtain the relationship between description names, and set the relationship between description names as implicit labels for the field information corresponding to the description names.

[0031] As a preferred embodiment of the semantic retrieval method for structured data described in this invention, the judgment result includes an upward trend, a downward trend, or no significant trend. When there is no significant trend, discrete and irrelevant information is set as the explicit label of the field information where the candidate word is located. When the judgment result is an upward trend or a downward trend, continuous and increasing or continuous and decreasing information is set as the explicit label of the field information where the candidate word is located.

[0032] As a preferred embodiment of the semantic retrieval method for structured data described in this invention, step S1623 further includes determining the relationship between the first set and the second set as an intersection relationship when the first number and the second number have the same part but are not completely the same.

[0033] When there are no common parts in the first number and the second number, the relationship between the first set and the second set is determined to be an exclusion relationship;

[0034] When the first number and the second number are exactly the same, the relationship between the first set and the second set is determined to be an overlap relationship;

[0035] When either the first or second number is entirely contained within either the second or first number, and the first and second numbers are not completely identical, the relationship between the first set and the second set is determined to be a subordinate relationship.

[0036] As a preferred embodiment of the semantic retrieval method for structured data described in this invention, any field information is selected, the set corresponding to the field information is obtained, and other sets that are related to the set are obtained, denoted as the associated set;

[0037] When the relationship between the associated set and the set is a cross relationship, set the content of the element in the associated set that is the same as the set to the first character, and get the description name number of the element in the associated set that is different from the set, which is recorded as the cross number.

[0038] When the relationship between the associated set and the set is a subordinate relationship, determine whether the set belongs to the associated set. If the set belongs to the associated set, set the content of all elements in the set to the first character. If the set does not belong to the associated set, set the content of the elements in the associated set to the first character and jump to the associated set for re-analysis.

[0039] When the relationships between the associated sets are overlapping, the content of the elements in the associated set is set to the first character, and the process of analyzing the relationship between the sets is repeated.

[0040] When the relationship corresponding to the association set is an exclusion relationship, obtain the number of all description names in the field information corresponding to the exclusion relationship, and record it as the exclusion number;

[0041] The implicit label for the current field information is generated based on the exclusion number, cross number, and first character, including setting the exclusion number and first character as the opposing sequence, setting the cross number as the supplementary sequence, and setting the supplementary sequence and opposing sequence as the implicit label for the current field information.

[0042] As a preferred embodiment of the semantic retrieval method for structured data described in this invention, any field information is selected, the set corresponding to the field information is obtained, and other sets that are related to the set are obtained, denoted as the associated set;

[0043] When the relationship between the associated set and the set is a cross relationship, set the content of the element in the associated set that is the same as the set to the first character, and get the description name number of the element in the associated set that is different from the set, which is recorded as the cross number.

[0044] When the relationship between the associated set and the set is a subordinate relationship, determine whether the set belongs to the associated set. If the set belongs to the associated set, set the content of all elements in the set to the first character. If the set does not belong to the associated set, set the content of the elements in the associated set to the first character and jump to the associated set for re-analysis.

[0045] When the relationships between the associated sets are overlapping, the content of the elements in the associated set is set to the first character, and the process of analyzing the relationship between the sets is repeated.

[0046] When the relationship corresponding to the association set is an exclusion relationship, obtain the number of all description names in the field information corresponding to the exclusion relationship, and record it as the exclusion number;

[0047] The implicit label for the current field information is generated based on the exclusion number, cross number, and first character, including setting the exclusion number and first character as the opposing sequence, setting the cross number as the supplementary sequence, and setting the supplementary sequence and opposing sequence as the implicit label for the current field information.

[0048] Secondly, a semantic retrieval system for structured data includes a construction module, an update module, and a verification module;

[0049] The construction module sets labels for field information in the relational database according to a preset processing method, constructs initial field codes based on the labels, expands the initial field codes based on the input data, and generates expanded field codes and query jump order.

[0050] The update module updates the markdown document by combining field extension encoding and query jump order, generates the first document and extracts the first timestamp;

[0051] The verification module obtains the jump field information based on the query jump order and the first timestamp, verifies the jump field information based on the label, and constructs the correspondence between the query jump order and the second timestamp.

[0052] The beneficial effects of this invention are as follows: By dynamically adjusting the retrieval range through query jump order, full table scans are avoided, significantly reducing redundant calculations. It is particularly suitable for large-scale structured data. Database fields are bound to business semantic tags, allowing retrieval to be based directly on business terms rather than technical field names, reducing the semantic gap. Field encoding is automatically expanded based on input data without the need for manual modification of the database structure, adapting to rapid business iterations. Document changes are marked by the first timestamp, and the latest valid fields are located by combining the jump order, avoiding repeated processing of historical data. The entire chain from tag setting to document generation is automated, such as automatically updating field descriptions in Markdown documents, reducing manual intervention. It is especially suitable for data platforms or API documentation scenarios that change frequently. Attached Figure Description

[0053] Figure 1 This is a schematic diagram of the basic process of a semantic retrieval method for structured data provided in one embodiment of the present invention. Detailed Implementation

[0054] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0055] It should be understood that the step numbers used herein are for ease of description only and are not intended to limit the order in which the steps are performed. It should also be understood that the terminology used in this specification is for the purpose of describing specific embodiments only and is not intended to limit the invention.

[0056] As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0057] The terms "comprising" and "including" indicate the presence of the described feature, whole, step, operation, element, and / or component, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or collections thereof.

[0058] The term “and / or” refers to any combination of one or more of the associated listed items, as well as all possible combinations, and includes these combinations.

[0059] With the development of large models and vector databases, unstructured data, such as text and images, are being transformed into structured semantic features, thereby enabling unified semantic retrieval. Semantic retrieval of structured data is no longer limited to traditional keywords or SQL queries, but integrates vector retrieval, full-text retrieval, and structured data queries to achieve multi-path recall of scalar, vector, and full-text data.

[0060] Based on this, the embodiments of this application provide a semantic retrieval method and system for structured data, which can dynamically update relevant data according to different data types, improve the reliability of the correlation between evaluation results and true semantics, configure a step-by-step reasoning chain, and output reasoning results discretely, thereby improving retrieval efficiency and making it suitable for complex multi-hop query scenarios. Moreover, this application does not rely on deep learning models, which can avoid problems such as lack of interpretability, difficulty in understanding the decision-making process of the model, and easy overfitting on small sample data, and has significant beneficial effects.

[0061] This application provides a semantic retrieval method and system for structured data, which will be specifically described through the following embodiments. First, the semantic retrieval method for structured data in this application is described.

[0062] This application provides a semantic retrieval method for structured data, relating to the field of data analysis. The semantic retrieval method for structured data provided in this application can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an implementation of a semantic retrieval method for structured data, but is not limited to the above forms.

[0063] This application can also be used in numerous general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0064] Example, refer to Figure 1 As an embodiment of the present invention, a semantic retrieval method for structured data is provided, comprising the following steps:

[0065] Step S1: According to the preset processing method, set the labels for the field information in the relational database;

[0066] Step S2: Construct the initial field code based on the label, expand the initial field code based on the input data, and generate the expanded field code and query jump order;

[0067] Step S3: Update the markdown document by combining field extension encoding and query jump order, generate the first document and extract the first timestamp;

[0068] Step S4: Obtain the jump field information based on the query jump order and the first timestamp, and verify the jump field information based on the label;

[0069] Step S5: Construct the correspondence between the query jump order and the second timestamp. The second timestamp is obtained by reconstructing the field extended encoding based on the first timestamp, updating the markdown document based on the reconstructed field extended encoding, and extracting the timestamp of the markdown document, which is recorded as the second timestamp.

[0070] More preferably, this invention dynamically adjusts the retrieval range by query jump order, avoiding full table scans and significantly reducing redundant calculations. It is particularly suitable for large-scale structured data. Database fields are bound to business semantic tags, allowing retrieval to be based directly on business terms rather than technical field names, reducing the semantic gap. Field codes are automatically expanded based on input data without requiring manual modification of the database structure, adapting to rapid business iterations. Document changes are marked by the first timestamp, and the latest valid fields are located by combining the jump order, avoiding the repeated processing of historical data. The entire chain from tag setting to document generation is automated, such as automatically updating field descriptions in Markdown documents, reducing manual intervention. It is especially suitable for data platforms or API documentation scenarios that change frequently.

[0071] Step S1 includes the following sub-steps: S11, obtaining field information from the relational database, where the field information is represented as standardized description name, standardized description words, and standardized description sentences;

[0072] Step S12: Select the main table and the subordinate table corresponding to any data subject, and number the main table and the subordinate table. The number of the main table is represented as X. i The table number is represented as Y. ij , and , where i represents the i-th data subject and j represents the j-th sub-table corresponding to the subject;

[0073] Step S13: Number the corresponding character information according to the main table number and the slave table number. After the main table number and slave table number of the current data subject are set, proceed to step S14.

[0074] Step S14: Jump to the next data subject and repeat steps S12 to S13 until the main table number and slave table code of all data subjects are set, then jump to step S15.

[0075] Step S15: After all the main table numbers and slave table numbers of all data subjects have been set, proceed to step S16.

[0076] Step S16: Identify the data type of the field information, classify the data body according to the data type, and set the matching rules for the classified data body;

[0077] Step S17: According to the matching rules, set the labels for the field information of the data subject. The labels include main table labels and sub-table labels. The main table labels include explicit main table labels and implicit main table labels. The sub-table labels include explicit sub-table labels and implicit sub-table labels.

[0078] Step S16 includes setting explicit tags and setting implicit tags. The method of setting explicit tags includes the following sub-steps: Step S1611: Obtain field information, judge the content in the field information, and when there is a descriptive sentence in the field information, perform word segmentation on the descriptive sentence to obtain the word segmentation result, and jump to step S1612.

[0079] If the field information does not contain a descriptive sentence, proceed to step S1613;

[0080] Step S1612: For the word segmentation results, set the sliding step size, which is an integer. Slide the word segmentation results according to the sliding step size to obtain each first word. Construct a weighted undirected graph of the first words according to the sliding order. Iteratively calculate the weight of the first words according to the TextRank formula. Set the first value as the weight threshold. When the weight of the first word is greater than or equal to the first value, select the first word with the largest value and set it as the first keyword. Proceed to step S1614.

[0081] When the weight of the first word is less than the first value, the first step length is set as the change amount of the sliding step length, where the first step length is an integer. Based on the first step length, the sliding step length is reduced, and the weight of the corresponding first word is continuously obtained until there is a first word whose weight after adjustment is greater than or equal to the first value. Then, the first word with the largest value is selected and set as the first keyword, and the process jumps to step S1614.

[0082] When the sliding step size decreases to 1, and all the first step sizes are less than the first value, the corresponding description sentence is deleted, and the corresponding position is set to the first character.

[0083] Step S1613: Identify the descriptive words. When the descriptive word is text, set the descriptive word as the second keyword and proceed to step S1614. When the descriptive word is a number, proceed to step S1615.

[0084] Step S1614: Set the first keyword or the second keyword as a candidate word, obtain the table attribute where the candidate word is located. When the table attribute is the main table, according to the one-to-many correspondence between the main table and the subordinate table of the Markdown document, obtain the corresponding subordinate description sentence or subordinate description word. Return to step S1612 to obtain the first keyword or the second keyword corresponding to the subordinate description sentence or subordinate description word, and record it as the first subordinate keyword or the second subordinate keyword. Calculate the similarity between the first keyword and the first subordinate keyword according to the cosine similarity formula, or calculate the similarity between the second keyword and the second subordinate keyword. Set the second value as the similarity threshold. Compare the similarity with the second value. When the similarity is greater than or equal to the second value, set discrete and irrelevant as the explicit label of the field information where the candidate word is located. When the similarity is less than the second value, jump to step S1616.

[0085] When the table attribute is a subordinate table, jump to the next candidate word until the table attribute is a main table. Based on the one-to-many correspondence between the main table and subordinate tables in the markdown document, obtain the corresponding subordinate description sentence or subordinate description word, and return to step S1612.

[0086] Step S1615: When the descriptor is a number, obtain the data type corresponding to the descriptor. When the data type is a date, set consecutive and level labels as explicit labels for the field information where the candidate word is located. When the data type is a numeric type, traverse the field information from top to bottom, calculate the first difference between the next descriptor and the previous descriptor, and obtain the difference curve according to the traversal order, with the traversal sequence number as the horizontal axis and the first difference as the vertical axis. According to the MK test method, judge the trend of the difference curve and obtain the judgment result. The judgment result includes an upward trend, a downward trend, or no significant trend. When there is no significant trend, set discrete and irrelevant labels as explicit labels for the field information where the candidate word is located. When the judgment result is an upward trend or a downward trend, set consecutive and increasing or consecutive and decreasing labels as explicit labels for the field information where the candidate word is located.

[0087] Step S1616: Add a conjunction to the adjacent first keyword or the adjacent second keyword. The conjunction includes "greater than" and "less than". The method of adding the conjunction includes adding it after the previous keyword, calling the language model, judging the semantic rationality of the connected keyword combination, and if the semantics are not reasonable, jumping to the next conjunction and repeating the addition and rationality judgment process. If the semantics are reasonable, setting discrete and increasing or discrete and decreasing as explicit labels.

[0088] When the added conjunction is greater than, the explicit label is discrete and decreasing; when the added conjunction is greater than, the explicit label is discrete and increasing.

[0089] The method for setting explicit tags includes the following sub-steps: Step S1621, obtain the field information of the main table and obtain the description name in the field information;

[0090] Step S1622: According to the preset set matching rules, the description names are categorized into various sets.

[0091] Step S1623: Based on the relationship between sets, obtain the relationship between description names, and set the relationship between description names as implicit labels for the field information corresponding to the description names.

[0092] More preferably, the set matching rules include: obtaining the major category to which the data subject belongs, which includes natural, social, cultural, abstract and other categories; setting the detailed categories of the major category as subcategories; the subcategories of the natural category include astronomy, geography, meteorology, seasons, time, animals and plants; the subcategories of the social category include people, organizations, occupations, events and economy; the subcategories of the cultural category include language, literature, art, education, science and technology and religion; the subcategories of the abstract category include emotions, qualities, concepts, knowledge, states and processes; and the subcategories of the other category include life, transportation, sports, entertainment, food and place names.

[0093] Based on the NPL recognition model, the description name is compared with each subclass, and the subclass with the highest similarity is taken as the subclass to which the description name belongs. The corresponding major class is obtained, and then the description name is classified into the major class corresponding to the subclass.

[0094] More preferably, the relationships between sets include exclusion, intersection, overlap, and subordination.

[0095] More preferably, the major category is set as a set, and each description name in the major category is set as an element in the set. Any two sets are selected and denoted as the first set and the second set. The numbers of the elements corresponding to the first set and the elements corresponding to the second set are obtained respectively and denoted as the first number and the second number. The first number and the second number are compared.

[0096] Step S1623 further includes determining the relationship between the first set and the second set as an intersection relationship when the first number and the second number have the same part but are not completely the same.

[0097] When there are no common parts in the first number and the second number, the relationship between the first set and the second set is determined to be an exclusion relationship;

[0098] When the first number and the second number are exactly the same, the relationship between the first set and the second set is determined to be an overlap relationship;

[0099] When either the first or second number is entirely contained within either the second or first number, and the first and second numbers are not completely identical, the relationship between the first set and the second set is determined to be a subordinate relationship.

[0100] Select any field information, retrieve the set corresponding to the field information, and retrieve other sets that are related to the set, denoted as the associated set;

[0101] When the relationship between the associated set and the set is a cross relationship, set the content of the element in the associated set that is the same as the set to the first character, and get the description name number of the element in the associated set that is different from the set, which is recorded as the cross number.

[0102] When the relationship between the associated set and the set is a subordinate relationship, determine whether the set belongs to the associated set. If the set belongs to the associated set, set the content of all elements in the set to the first character. If the set does not belong to the associated set, set the content of the elements in the associated set to the first character and jump to the associated set for re-analysis.

[0103] When the relationships between the associated sets are overlapping, the content of the elements in the associated set is set to the first character, and the process of analyzing the relationship between the sets is repeated.

[0104] When the relationship corresponding to the association set is an exclusion relationship, obtain the number of all description names in the field information corresponding to the exclusion relationship, and record it as the exclusion number;

[0105] The implicit label for the current field information is generated based on the exclusion number, cross number, and first character, including setting the exclusion number and first character as the opposing sequence, setting the cross number as the supplementary sequence, and setting the supplementary sequence and opposing sequence as the implicit label for the current field information.

[0106] More preferably, explicit labels are used to indicate the discrete or continuous state of field information, and explicit labels are also used to indicate the correlation between field information in the main table and any subsidiary table;

[0107] Implicit labels are used to indicate related data for fields, including exclusionary associations, cross associations, overlapping associations, and subordinate associations.

[0108] More preferably, the TextRank formula is: ;

[0109] Among them, W(V) l ) represents the weight of the l-th first word, d is the damping coefficient, and its value ranges from (0, 1). ul Let be the similarity between the l-th and u-th first words, obtained through cosine similarity, in(Vl ) represents the set of all first words pointing to the l-th first word, out(V l Let V be the set of all first words starting from the l-th first word. g Let g be the first word in the set that starts from the l-th first word and points to other first words.

[0110] More preferably, the relational database consists of tables representing various content to be retrieved. Each table belongs to only one data subject. Each data subject includes one master table and N slave tables. The tables include row information and column information. Row information represents records, and column information represents fields. The column headers of the column information are configured with descriptive names. According to the attributes of the table data, the data types of the same column are the same. Data types include character type, numeric type, and date type.

[0111] More preferably, for the master table and the slave table, an identification method is configured, including setting constraints, wherein the constraints are expressed as follows: modifying one record will simultaneously modify the corresponding records in the remaining tables; the constraints ensure data consistency through a database transaction mechanism.

[0112] If any table satisfies the constraints, the corresponding table is set as the parent table; if any table does not satisfy the constraints, the corresponding table is set as the child table.

[0113] More preferably, when the field information is a description name or a descriptive word, a first standardization method is configured for the description name or descriptive word. The first standardization method includes setting a standard naming for the field information, whereby the standard naming of the field information is represented as the original field information after removing spaces and symbols.

[0114] When the field information is a descriptive sentence, a second standardization method is configured for the descriptive sentence. The second standardization method includes segmenting the descriptive sentence into words using a word segmentation model, removing punctuation marks from the descriptive sentence to obtain individual descriptive words, and adjusting each descriptive word according to semantics based on the language model so that the adjusted descriptive words represent similar semantics.

[0115] More preferably, when the field information is a descriptive name or descriptive term, a first standardization method is adopted, which specifically includes removing all spaces and special symbols such as @, #, $, %, tabs, newlines, etc. from the original field information, retaining only letters, numbers, underscores, and semantic core characters to obtain a standardized name. For example, the original field name customer#-id is converted into the standardized name customer_id using the first standard method. The standardized name is uniformly converted to lowercase or uppercase to avoid synonyms and heterogeneity. Based on a pre-set business terminology dictionary, non-standard words are mapped to standard words. For example, cust_no is mapped to customer_id. If the number of characters in the standardized name is greater than 64 characters, the standardized name is divided into 4 parts with the same number of characters, and each part is appended with a 4-bit hash.

[0116] More preferably, when the field information is a descriptive sentence, a second standardization method is adopted, specifically including the following steps: deleting Chinese and English full-width and half-width punctuation marks, and removing meaningless stop words; then using a pre-trained language model to perform semantic vector encoding on the remaining words; calling a word segmentation model, such as jieba, HanLP, THULAC, or a self-developed domain word segmenter, to segment the descriptive sentence to obtain an initial descriptive word sequence. For example, the original descriptive sentence is "This field is used to record the unique identifier number of customers." After word segmentation, we get "this," "field," "used for," "record," "customer," "of," "unique," "identifier," and "number." High-frequency meaningless stop words are removed, such as "this," "used for," "of," etc., and "can." After filtering, we get "field," "record," "customer," "unique," "identifier," and "number." The remaining descriptive words are then input into a language model, such as B... ERT, RoBERTa, ERNIE, ChatGLM, or domain-specific fine-tuning models use NLP text similarity calculation methods to adjust semantically similar words into a standard word. A similarity threshold of 0.85 is set, and the most frequently occurring word is selected as the anchor. Similar words are replaced with anchor words, and the correspondence is recorded. For example, "unique," "identifier," and "number" are semantically similar and correspond to "unique identifier number." The adjusted descriptive words are then sorted by weights obtained from TextRank, and the sorting is in descending order, forming a standard descriptive word sequence. The final sequence is: customer, unique identifier number, field, record. The sorted words are then reassembled into a coherent phrase according to subject-verb-object conventions for subsequent display or review. For example, the standard descriptive sentence is: customer unique identifier number field record.

[0117] More preferably, the description name represents the data in the column header of any column of information, and the description words and description sentences represent the other data in any column of information besides the data in the column header, with the description words and description sentences corresponding to the description name.

[0118] More preferably, each candidate word is converted into a 768 / 1024-dimensional vector using a domain-fine-tuned language model. The cosine similarity of the remaining words in any two identical descriptive sentences is calculated. Using 0.85 as the truncation threshold, the matrix is ​​divided into clusters. The word with the highest frequency in each cluster is selected as the anchor point, and the remaining words are mapped to the anchor point. When there are more than or equal to 500 domain-gold label data, a grid search is used to select the similarity threshold from {0.80, 0.85, 0.90} to maximize the value of F1=2PR / (P+R), where P is the precision and R is the recall.

[0119] More preferably, the character information corresponding to the main table is numbered including X. i1 X i2k and , where X i1 X represents the index of the description name of the main table of the i-th data subject. i2k This represents the index of the k-th descriptor in the main table of the i-th data subject. Let N be the number of the kth descriptive sentence in the main table of the i-th data subject, N be the number of descriptive words obtained after splitting and deleting the descriptive sentence, and k be the number of the kth split descriptive word. The split descriptive words are numbered according to the order of appearance of the descriptive words in the standard descriptive sentence. k is a natural number. The number of the kth descriptive word in the main table of the i-th data subject is determined by its position in the standard descriptive sentence.

[0120] More preferably, the number of the character information corresponding to the table includes Y. ij1 Y ij2m and , where Y ij1 Y represents the index of the description name of the j-th sub-table of the i-th data body. ij2m This represents the index of the j-th descriptor in the m-th sub-table of the i-th data subject. Let Q represent the number of the m-th descriptive sentence in the j-th sub-table of the i-th data subject, Q be the number of descriptive words obtained after splitting and deleting the descriptive sentence, m be the number of the m-th split descriptive word, and the numbers of the split descriptive words are numbered according to the order of appearance of the descriptive words in the standard descriptive sentence. m is a natural number. The number of the m-th descriptive word in the j-th sub-table of the i-th data subject is determined by its position in the standard descriptive sentence.

[0121] More preferably, since the descriptive sentence is of the same type as the descriptive word before being combined after being split and deleted, the numbering of the standardized descriptive sentence overlaps with the representation symbol of the standardized descriptive word.

[0122] The initial field code is constructed based on explicit and implicit labels. Based on the initial field code, input data, and large language model, the initial field code is expanded to generate the expanded field code and the query jump order.

[0123] More preferably, the method for constructing the initial field encoding includes setting an explicit label identifier and combining the explicit label identifier with an implicit label.

[0124] More preferably, the method for setting explicit labels includes setting discrete and irrelevant as a1, discrete and increasing as a2, discrete and decreasing as a3, continuous and irrelevant as a4, continuous and increasing as a5, and continuous and decreasing as a6.

[0125] More preferably, the method for combining explicit labels with implicit labels includes combining implicit labels in sequence, concatenating explicit labels with the sequentially combined implicit labels (the concatenation is also referred to as sequential combination), and using the concatenated result as the initial encoding of the field.

[0126] Based on the initial field encoding and input data, the initial field encoding is expanded to generate the expanded field encoding and the query jump order;

[0127] The method for expanding the initial field code includes: obtaining input data, processing the input data according to the method of standardizing field information to obtain standardized input field information, and matching the standardized input field information with the initial field code;

[0128] If all correspond, the original number is retained, and the initial field code is not extended;

[0129] If none of them match, do not expand the initial field encoding, and output the field information corresponding to the exclusion encoding;

[0130] If there is a partial correspondence, the initial encoding of the field is expanded. For the non-corresponding parts, the process of classifying elements into a set is repeated. The non-corresponding parts are classified into the set and a number is set for the non-corresponding parts. The number of the non-corresponding parts has the same format as the original elements, but the subscripts representing the quantity are different.

[0131] After the mismatched parts are numbered, repeat the explicit label setting process and implicit label setting process for the field information, and repeat the matching process between the standardized input field information and the initial field code.

[0132] When no expansion is needed, Set as the query value, where, The value is a constant. When expansion is required, the number of expanded codes is set to the query value, the query waveform is plotted, the mathematical expression corresponding to the query waveform is obtained, the mathematical expression is subjected to Fourier transform to obtain the spectral representation, the frequency corresponding to the spectral representation is obtained, and the frequency is set as the query jump order.

[0133] More preferably, the frequency acquisition method includes smoothing the spectrum, obtaining the maximum value in the smoothed spectrum, and taking the frequency corresponding to the maximum value as the frequency.

[0134] More preferably, the smoothing method includes performing a moving average on the spectrum or performing Gaussian smoothing on the spectrum, where a larger query jump order indicates more augmented coding.

[0135] The markdown document fragments are updated by combining field extension encoding and query jump order to generate the first document, and the first timestamp of the first document is extracted.

[0136] More preferably, the method for updating the Markdown document includes: within a first time period, sorting each query jump order in descending order, selecting the query jump order with the largest value, using the extended code corresponding to the query jump order with the largest value as the reference code, and obtaining the cross-code, exclusion code, overlapping code, and one-way subordinate code between other extended codes and the reference code according to the set relationship. The subordinate relationship between other extended codes and the reference code only exists when other extended codes belong to the reference code, so the subordinate code is recorded as a one-way subordinate code, and the overlapping code and one-way subordinate code are deleted.

[0137] The cross-coding, exclusion coding, and reference coding are converted into Markdown document code, inserted into the Markdown document, and the Markdown document after the code is inserted is saved. The first correspondence between the query jump order with the largest value and the cross-coding, exclusion coding, and reference coding is constructed.

[0138] The markdown document is updated in a cycle, using the markdown document update method.

[0139] Based on the queried hop order and the first timestamp, the hop field information is obtained. The hop field information is verified based on the tags. Based on the verification results, the field extended encoding is reconstructed. The Markdown document is updated based on the reconstructed field extended encoding, and the second timestamp of the Markdown document is extracted.

[0140] More preferably, the method for obtaining jump field information includes: obtaining current query information, matching the query jump order of the current query information, inputting the query jump order into the first correspondence, and performing matching;

[0141] When the query jump order has a corresponding relationship with the first one, the output is the cross-code, exclusion code, and reference code;

[0142] When the query jump order does not have a corresponding relationship with the first one, retrieve the query jump order that is closest to the query jump order and its corresponding cross code, exclusion code, and reference code. Set the cross code, exclusion code, and reference code as the jump field information. After the jump field information is set, jump to the verification step.

[0143] When jumping to the verification step, the first timestamp of the jump field information is obtained, and the field extended code of the first timestamp is reconstructed. Specifically, the major categories of the field extended code are reclassified to obtain new field extended codes. When the new field extended code is the same as the original field extended code, the fourth value is set as the similarity change amount. The similarity is continuously increased or decreased according to the similarity change amount until the new field extended code is different from the original field extended code. At this point, the similarity adjustment is stopped, the markdown document code is updated according to the new field extended code, and the second timestamp of the markdown document is extracted.

[0144] More preferably, a second correspondence is constructed for the second timestamp of the query jump order.

[0145] Select the second timestamp based on the current input data, obtain the current jump field information based on the first trace link, obtain the current associated data based on the implicit label of the current input data, and convert the current associated data and the current jump field information into natural language.

[0146] More preferably, the current input data is obtained, the current query jump order of the current input data is matched, the second timestamp of the current query jump order is obtained according to the second correspondence, and the corresponding cross-coding, exclusion coding, and reference coding are obtained and output.

[0147] More preferably, this invention dynamically adjusts the retrieval range by query jump order, avoiding full table scans and significantly reducing redundant calculations. It is particularly suitable for large-scale structured data. Database fields are bound to business semantic tags, allowing retrieval to be based directly on business terms rather than technical field names, reducing the semantic gap. Field codes are automatically expanded based on input data without requiring manual modification of the database structure, adapting to rapid business iterations. Document changes are marked by the first timestamp, and the latest valid fields are located by combining the jump order, avoiding the repeated processing of historical data. The entire chain from tag setting to document generation is automated, such as automatically updating field descriptions in Markdown documents, reducing manual intervention. It is especially suitable for data platforms or API documentation scenarios that change frequently.

[0148] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0149] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A semantic retrieval method for structured data, characterized in that, The process includes the following steps: Step S1, setting labels for field information in the relational database according to a preset processing method; Step S2: Construct the initial field code based on the label, expand the initial field code based on the input data, and generate the expanded field code and query jump order; Step S3: Update the markdown document by combining field extension encoding and query jump order, generate the first document and extract the first timestamp; Step S4: Obtain the jump field information based on the query jump order and the first timestamp, and verify the jump field information based on the label; Step S5: Construct the correspondence between the query jump order and the second timestamp. The second timestamp is obtained by reconstructing the field extended encoding based on the first timestamp, updating the markdown document based on the reconstructed field extended encoding, and extracting the timestamp of the markdown document, which is recorded as the second timestamp.

2. The semantic retrieval method for structured data as described in claim 1, characterized in that, Step S1 includes the following sub-steps: S11, obtaining field information from the relational database, where the field information is represented as standardized description name, standardized description words, and standardized description sentences; Step S12: Select the main table and the subordinate table corresponding to any data subject, and number the main table and the subordinate table. The number of the main table is represented as X. i The table number is represented as Y. ij , and , where i represents the i-th data subject and j represents the j-th sub-table corresponding to the subject; Step S13: Number the corresponding character information according to the main table number and the slave table number. After the main table number and slave table number of the current data subject are set, proceed to step S14. Step S14: Jump to the next data subject and repeat steps S12 to S13 until the main table number and slave table code of all data subjects are set, then jump to step S15. Step S15: After all the main table numbers and slave table numbers of all data subjects have been set, proceed to step S16. Step S16: Identify the data type of the field information, classify the data body according to the data type, and set the matching rules for the classified data body; Step S17: According to the matching rules, set the labels for the field information of the data subject. The labels include main table labels and sub-table labels. The main table labels include explicit main table labels and implicit main table labels. The sub-table labels include explicit sub-table labels and implicit sub-table labels.

3. The semantic retrieval method for structured data as described in claim 2, characterized in that, Step S16 includes setting explicit labels and setting implicit labels.

4. The semantic retrieval method for structured data as described in claim 2, characterized in that, Step S1611: Obtain field information, judge the content in the field information, and when there is a descriptive sentence in the field information, perform word segmentation on the descriptive sentence to obtain the word segmentation result, and jump to step S1612; If the field information does not contain a descriptive sentence, proceed to step S1613; Step S1612: For the word segmentation results, set the sliding step size, which is an integer. Slide the word segmentation results according to the sliding step size to obtain each first word. According to the TextRank formula, obtain the weight of the first word. Set the first keyword according to the weight of the first word. Jump to step S1614. When the sliding step size decreases to 1, and all the first step sizes are less than the first value, the corresponding description sentence is deleted, and the corresponding position is set to the first character; Step S1613: Identify the descriptive words. When the descriptive word is text, set the descriptive word as the second keyword and proceed to step S1614. When the descriptive word is a number, proceed to step S1615. Step S1614: Set the first keyword or the second keyword as a candidate word. Based on the one-to-many correspondence between the main table and the subordinate table in the Markdown document, when the table attribute is a subordinate table, the corresponding subordinate description sentence or subordinate description word is obtained. Return to step S1612 to obtain the first subordinate keyword or the second subordinate keyword corresponding to the subordinate description sentence or subordinate description word. Calculate the similarity between the first keyword and the first subordinate keyword according to the cosine similarity formula, or calculate the similarity between the second keyword and the second subordinate keyword. Based on the similarity, select to set discrete and irrelevant fields as explicit labels for the field information where the candidate word is located, or jump to step S1616. When the table attribute is a subordinate table, jump to the next candidate word until the table attribute is a main table. Based on the one-to-many correspondence between the main table and subordinate tables in the markdown document, obtain the corresponding subordinate description sentence or subordinate description word, and return to step S1612. Step S1615: When the data type is date, set consecutive and parallel terms as explicit labels for the field information where the candidate words are located. When the data type is numeric, traverse the field information from top to bottom, calculate the first difference between the next descriptor and the previous descriptor, and obtain the difference curve by using the traversal sequence number as the horizontal axis and the first difference as the vertical axis according to the traversal order. Judge the trend of the difference curve according to the MK test method, obtain the judgment result, and set explicit labels according to the judgment result. Step S1616: Add a conjunction to the adjacent first keyword or the adjacent second keyword. The conjunction includes "greater than" and "less than". The method of adding the conjunction includes adding it after the previous keyword, calling the language model, judging the semantic rationality of the connected keyword combination, and if the semantics are not reasonable, jumping to the next conjunction and repeating the addition and rationality judgment process. If the semantics are reasonable, setting discrete and increasing or discrete and decreasing as explicit labels. When the added conjunction is greater than, the explicit label is discrete and decreasing; when the added conjunction is greater than, the explicit label is discrete and increasing.

5. The semantic retrieval method for structured data as described in claim 2, characterized in that, The method for setting explicit tags includes the following sub-steps: Step S1621, obtain the field information of the main table and obtain the description name in the field information; Step S1622: According to the preset set matching rules, the description names are categorized into various sets. Step S1623: Based on the relationship between sets, obtain the relationship between description names, and set the relationship between description names as implicit labels for the field information corresponding to the description names.

6. The semantic retrieval method for structured data as described in claim 4, characterized in that, The judgment result includes an upward trend, a downward trend, or no significant trend. When there is no significant trend, discrete and irrelevant labels are set as explicit labels for the field information where the candidate word is located. When the judgment result is an upward trend or a downward trend, continuous and increasing or continuous and decreasing labels are set as explicit labels for the field information where the candidate word is located.

7. The semantic retrieval method for structured data as described in claim 5, characterized in that, Step S1623 further includes determining the relationship between the first set and the second set as an intersection relationship when the first number and the second number have the same part but are not completely the same. When there are no common parts in the first number and the second number, the relationship between the first set and the second set is determined to be an exclusion relationship; When the first number and the second number are exactly the same, the relationship between the first set and the second set is determined to be an overlap relationship; When either the first or second number is entirely contained within either the second or first number, and the first and second numbers are not completely identical, the relationship between the first set and the second set is determined to be a subordinate relationship.

8. The semantic retrieval method for structured data as described in claim 7, characterized in that, Select any field information, retrieve the set corresponding to the field information, and retrieve other sets that are related to the set, denoted as the associated set; When the relationship between the associated set and the set is a cross relationship, set the content of the element in the associated set that is the same as the set to the first character, and get the description name number of the element in the associated set that is different from the set, which is recorded as the cross number. When the relationship between the associated set and the set is a subordinate relationship, determine whether the set belongs to the associated set. If the set belongs to the associated set, set the content of all elements in the set to the first character. If the set does not belong to the associated set, set the content of the elements in the associated set to the first character and jump to the associated set for re-analysis. When the relationships between the associated sets are overlapping, the content of the elements in the associated set is set to the first character, and the process of analyzing the relationship between the sets is repeated. When the relationship corresponding to the association set is an exclusion relationship, obtain the number of all description names in the field information corresponding to the exclusion relationship, and record it as the exclusion number; The implicit label for the current field information is generated based on the exclusion number, cross number, and first character, including setting the exclusion number and first character as the opposing sequence, setting the cross number as the supplementary sequence, and setting the supplementary sequence and opposing sequence as the implicit label for the current field information.

9. A semantic retrieval method for structured data as described in claim 8, characterized in that, Select any field information, retrieve the set corresponding to the field information, and retrieve other sets that are related to the set, denoted as the associated set; When the relationship between the associated set and the set is a cross relationship, set the content of the element in the associated set that is the same as the set to the first character, and get the description name number of the element in the associated set that is different from the set, which is recorded as the cross number. When the relationship between the associated set and the set is a subordinate relationship, determine whether the set belongs to the associated set. If the set belongs to the associated set, set the content of all elements in the set to the first character. If the set does not belong to the associated set, set the content of the elements in the associated set to the first character and jump to the associated set for re-analysis. When the relationships between the associated sets are overlapping, the content of the elements in the associated set is set to the first character, and the process of analyzing the relationship between the sets is repeated. When the relationship corresponding to the association set is an exclusion relationship, obtain the number of all description names in the field information corresponding to the exclusion relationship, and record it as the exclusion number; The implicit label for the current field information is generated based on the exclusion number, cross number, and first character, including setting the exclusion number and first character as the opposing sequence, setting the cross number as the supplementary sequence, and setting the supplementary sequence and opposing sequence as the implicit label for the current field information.

10. A semantic retrieval system for structured data, the system being used to execute the semantic retrieval method for structured data as described in claim 1, characterized in that, This includes a build module, an update module, and a verification module; The construction module sets labels for field information in the relational database according to a preset processing method, constructs initial field codes based on the labels, expands the initial field codes based on the input data, and generates expanded field codes and query jump order. The update module updates the markdown document by combining field extension encoding and query jump order, generates the first document and extracts the first timestamp; The verification module obtains the jump field information based on the query jump order and the first timestamp, verifies the jump field information based on the label, and constructs the correspondence between the query jump order and the second timestamp.

Images