Text unit relational reasoning method and device, storage medium and terminal
By acquiring and filtering text unit relationship pairs, the text unit inference range is expanded based on the semantic distance range, the problem of small inference range in the existing technology is solved, and the inference accuracy is improved.
Patent Information
- Application Number
- CN202411103704.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-13
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-08-13
AI Technical Summary
The existing text unit reasoning methods are too single and have a small scope of reasoning, and cannot make extensive reasoning based on the relationship between text units.
By obtaining inference relationship pairs, the semantic distance range is set based on the semantic distances of the inference subject and the inference auxiliary body, the set of close text units and the set of close text units is obtained, the text relationship pair is formed, and the inference accuracy is improved through filtering conditions.
It realizes similarity search based on text unit relationships based on unknown target text units, expands the scope of reasoning, and improves the accuracy of reasoning.
Smart Images

Figure CN118643156B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of text unit reasoning, and in particular to a text unit relational reasoning method and device, a storage medium and a terminal. Background Art
[0002] Text reasoning is the process of analyzing, understanding, and extracting text. It can help people understand text, perform cluster analysis, generate abstracts, extract information, and so on. Therefore, it is applied in various fields. With the development of Internet technology, text reasoning is increasingly dependent on the Internet.
[0003] The current text unit reasoning method can only realize reasoning on the basis of clear text units. For example, the acquisition of unknown text units requires the known text units to be inferred and the corresponding semantic distance to realize text unit reasoning. This text unit reasoning method targets a single text unit and has a small reasoning scope, and is not applicable to extensive reasoning. At the same time, this text unit reasoning method is also not applicable to fixed relationship reasoning. For example, if it is known that the capital of South Korea is Seoul, if you want to obtain the text unit pairs of all countries and corresponding capitals, it cannot be achieved based on the existing reasoning method.
[0004] Therefore, there is an urgent need for a text unit reasoning method that can reason based on the relationship between text units. Summary of the invention
[0005] The purpose of this application is to provide a text unit relational reasoning method and device, storage medium and terminal, which are used to solve the problem that the current text unit reasoning method is too single and the reasoning scope is small, and it is impossible to perform extensive reasoning based on the relationship between text units.
[0006] In a first aspect, the present application provides a text unit relational reasoning method, comprising:
[0007] Acquire a reasoning relationship pair, wherein the reasoning relationship pair includes a reasoning subject and a reasoning auxiliary body, and the reasoning subject and the reasoning auxiliary body belong to a reasoning database;
[0008] Based on the characteristics of the inference subject or the characteristics of the similar text units, obtaining a set of similar text units of the inference subject from the inference database;
[0009] A first semantic distance range is set based on the semantic distance between the inference subject and the inference auxiliary subject, and for each similar text unit in the similar text unit set, a set of all text units within the first semantic distance range is obtained as a subset of adjacent text units corresponding to the similar text unit;
[0010] Each of the similar text units is respectively formed into a text relationship pair with each adjacent text unit in the corresponding adjacent text unit subset, and all of the text relationship pairs are grouped as a target relationship pair set of the inference relationship pairs.
[0011] In one embodiment of the present application, obtaining the inference relationship pair includes:
[0012] Obtaining a given relationship pair, wherein the given relationship pair includes a given subject and a given auxiliary subject;
[0013] Based on the equivalent body mapping method, the reasoning subject corresponding to the given subject and the reasoning auxiliary body corresponding to the given auxiliary body are obtained from the reasoning database, and the given subject and the given auxiliary body constitute the reasoning relationship pair.
[0014] In one embodiment of the present application, based on the characteristics of the reasoning subject, obtaining a set of similar text units of the reasoning subject from the reasoning database includes:
[0015] A preset semantic distance is obtained, a second semantic distance range is set based on the preset semantic distance, and all text units in the inference database whose semantic distances to the inference subject are within the second semantic distance range are grouped as a similar text unit set of the inference subject.
[0016] In one embodiment of the present application, based on the characteristics of the reasoning subject, obtaining a set of similar text units of the reasoning subject from the reasoning database includes:
[0017] The set classification type of the inference subject is obtained, and all text units under the set classification type are searched from the inference database to form a set of similar text units of the inference subject, wherein the set of similar text units does not include the inference subject.
[0018] In one embodiment of the present application, based on the characteristics of the reasoning subject, obtaining a set of similar text units of the reasoning subject from the reasoning database includes:
[0019] Acquire the set classification type and preset classification semantic range of the inference subject, and group all classification types in the inference database whose semantic distance from the set classification type is within the preset classification semantic range as a target classification type set;
[0020] The set classification type and all text unit sets under the target classification type in the target classification type set are searched from the inference database as a similar text unit set of the inference subject, and the similar text unit set does not include the inference subject.
[0021] In one embodiment of the present application, based on the characteristics of the reasoning subject, obtaining a set of similar text units of the reasoning subject from the reasoning database includes:
[0022] Obtaining the semantic distance between all arbitrary two text units in the inference database, and forming a semantic distance network with all text units in the inference database using the obtained semantic distance as an edge and the text unit as a point;
[0023] The edges whose semantic distance in the semantic distance network is less than a preset threshold are removed to obtain multiple semantic distance sub-networks, and all text units in the semantic distance sub-network to which the reasoning subject belongs are grouped as a set of similar text units of the reasoning subject, and the set of similar text units does not include the reasoning subject.
[0024] In one embodiment of the present application, based on the characteristics of similar text units, obtaining a set of similar text units of the inference subject from the inference database includes:
[0025] Acquire similar classification types, and search the inference database for all text units under the similar classification types, taking the set as the inference subject, and a set of similar text units.
[0026] In one embodiment of the present application, before all the text relationship pairs are grouped as the target relationship pair set of the inference relationship pairs, all the formed text relationship pairs are filtered by a filtering condition;
[0027] The filtering condition includes at least one of the following conditions:
[0028] Filter out text relationship pairs in which the classification of the adjacent text units in the text relationship pairs and the inference subject in the inference database is inconsistent, or the classification of the adjacent text units in the text relationship pairs and the inference auxiliary body in the inference database is inconsistent;
[0029] Filter out text relationship pairs in which the knowledge structure of the adjacent text units in the inference database does not intersect with the knowledge structure of the inference subject in the inference database, or the knowledge structure of the adjacent text units in the text relationship pairs does not intersect with the knowledge structure of the inference auxiliary body in the inference database;
[0030] If the inference subject and the inference auxiliary have a containment or subordinate relationship, then filter out the text relationship pairs where the close text unit and the corresponding adjacent text unit do not have a containment or subordinate relationship;
[0031] If the inference subject has the same text property as the adjacent text unit, then filter out the text unit pairs whose text properties of the adjacent text unit are not the same as the text properties of the inference auxiliary body, where the text properties are characters, words, phrases, sentences, plurals, nouns, verbs, adjectives or adverbs.
[0032] In a second aspect, the present application further provides a text unit relational reasoning device, including a reasoning relation pair acquisition module, a similar text unit acquisition module, an adjacent text unit acquisition module, and a target relation pair acquisition module;
[0033] The inference relationship pair acquisition module is used to acquire the inference relationship pair, wherein the inference relationship pair includes an inference subject and an inference auxiliary body, and the inference subject and the inference auxiliary body belong to an inference database;
[0034] The similar text unit acquisition module is used to acquire a set of similar text units of the reasoning subject from the reasoning database based on the reasoning subject characteristics or similar text unit characteristics;
[0035] The adjacent text unit acquisition module is used to set a first semantic distance range based on the semantic distance between the inference subject and the inference auxiliary body, and for each adjacent text unit in the adjacent text unit set, obtain a set of all text units within the first semantic distance range as an adjacent text unit subset corresponding to the adjacent text unit;
[0036] The target relationship pair acquisition module forms a text relationship pair with each of the similar text units and each adjacent text unit in the corresponding adjacent text unit subset, and gathers all of the text relationship pairs into a target relationship pair set of the inference relationship pairs.
[0037] In a third aspect, the present application also provides a storage medium on which a computer program is stored, characterized in that when the program is executed by a processor, the text unit relational reasoning method is implemented.
[0038] In a fourth aspect, the present application further provides a terminal, comprising: a processor and a memory, wherein the memory is communicatively connected to the processor;
[0039] The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the terminal executes the text unit relationship reasoning method as described in the implementation.
[0040] Compared with the prior art, one or more embodiments of the above scheme may have the following advantages or beneficial effects:
[0041] By applying the text unit relational reasoning method provided in the embodiment of the present invention, a set of similar text units of the reasoning subject is obtained as needed, and at the same time, a screening semantic distance range is determined based on the semantic distance between the reasoning subject and the reasoning auxiliary body, and then the screening semantic distance range is used as the reasoning direction to obtain the adjacent text units of each similar text unit of the reasoning subject, that is, to realize the reasoning of all similar text units from the reasoning subject to the reasoning auxiliary body, and then obtain all target relationship pairs with similar reasoning relationships with the reasoning subject and the reasoning auxiliary body. The method of the present invention can perform similarity search based on the relationship between text unit relationship pairs on the basis of unknown target text units, and obtain all desired text unit pairs with a given relationship in the similar results, thereby expanding the text unit reasoning range. The screened text unit pairs are filtered by filtering conditions, which further improves the reasoning accuracy in the text unit pair relational reasoning process.
[0042] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:
[0044] Figure 1 Shown is a flow chart of the text unit relationship reasoning method described in an embodiment of the present application.
[0045] Figure 2 Shown is a structural schematic diagram of the semantic distance network and the semantic distance sub-network in the text unit relationship reasoning method described in an embodiment of the present application.
[0046] Figure 3 Shown is a schematic diagram of the structure of the text unit relationship reasoning device described in an embodiment of the present application.
[0047] Figure 4 Shown is a schematic diagram of the structure of the terminal described in an embodiment of the present application. DETAILED DESCRIPTION
[0048] The following will describe the implementation methods of the present invention in detail with reference to the accompanying drawings and embodiments, so that the implementation process of how the present invention applies technical means to solve technical problems and achieve technical effects can be fully understood and implemented accordingly. It should be noted that as long as there is no conflict, the various embodiments of the present invention and the various features in the embodiments can be combined with each other, and the technical solutions formed are all within the protection scope of the present invention.
[0049] It should be noted that the illustrations provided in the following embodiments only schematically illustrate the basic concept of the present application. Therefore, only the components related to the present application are shown in the drawings, rather than being drawn according to the number, shape, and size of the components in actual implementation. The form, quantity, and proportion of each component in actual implementation can be arbitrarily changed, and the component layout form may also be more complex.
[0050] Data Curation refers to the activity of organizing and integrating data collected from different sources to establish a trustworthy data repository. It includes a series of full-life cycle activities for effective data management, such as collection, screening, evaluation, preservation, maintenance, and utilization. Data Curation can be completed manually or processed by machines. The data quality after Data Curation is usually relatively high.
[0051] A knowledge base is a tool for storing and managing knowledge, which can be used to support applications such as knowledge systems, knowledge discovery, and knowledge sharing. A knowledge base usually contains relevant information and data in a certain domain or topic, as well as knowledge representation forms such as rules, methods, and concepts. The knowledge base can be classified, organized, and retrieved according to a certain structure and semantics, so as to improve the comprehensibility, reusability, and interoperability of knowledge. Text units (i.e., tokens) include words, phrases, and sentences. In a knowledge base, they refer to objects and categories, that is, the words, phrases, and sentences that make them up. They can be specific token objects, such as "Cogito, ergo sum", "Homo homini lupus", "Knowledge is power", "NBA Finals Most Valuable Player Award", etc., or token categories, such as "NBA Finals", "Chinese actors", etc.
[0052] Entities are the basic elements in a knowledge base, which represent objects, events, situations, or concepts in the real world. The entities here are a subset of tokens, which are some things in tokens that represent the real world.
[0053] Semantic distance refers to the concept used to measure the semantic similarity or difference between two words, phrases, or sentences in a semantic space. It can be used to complete tasks such as word sense disambiguation, text classification, and information retrieval.
[0054] Equivalents refer to a group of tokens that represent the same meaning, including a group of tokens with the same meaning in different languages. For example, "北大" and "北京大学" are equivalent, and the Chinese "北京大学" and the English "Peking University" are equivalent. Equivalents not only include synonyms, but also other words that can achieve the same function or effect.
[0055] The text unit relationship reasoning method of this embodiment can be applied to the field of artificial intelligence, and further applied to machine translation, machine reading comprehension, and question-answering systems for natural language processing in the field of artificial intelligence. The text unit relationship reasoning method of this embodiment can also be applied to other suitable application fields, which are not specifically limited here.
[0056] The following embodiments of the present application provide a text unit relational reasoning method and device, a storage medium and a terminal, which are used to solve the problem that the current text unit reasoning method is too single and has a small reasoning scope, and cannot perform extensive reasoning based on the relationship between text units.
[0057] The following will describe in detail the principles and implementation methods of a text unit relational reasoning method and device, storage medium and terminal of this embodiment in conjunction with the accompanying drawings, so that those skilled in the art can understand the text unit relational reasoning method and device, storage medium and terminal of this embodiment without creative work.
[0058] like Figure 1 As shown, this embodiment provides a text unit relationship reasoning method, including the following steps.
[0059] Step S101, obtaining a reasoning relationship pair, the reasoning relationship pair includes a reasoning subject and a reasoning auxiliary, and the reasoning subject and the reasoning auxiliary belong to a reasoning database.
[0060] Specifically, the text unit pair to be used as the basis for text unit relationship reasoning is taken as a given relationship pair, the main reasoning text unit in the given relationship pair is taken as a given subject, and the auxiliary text unit in the given relationship pair is taken as a given auxiliary body. The relationship between the given subject and the given auxiliary body in the given relationship pair is the relationship reasoning direction, and the database used in the reasoning process is set as the reasoning database, which is a semantic distance database calculated using a data-curated knowledge base.
[0061] In this embodiment, the given subject and the given auxiliary body can be a single text unit or a set of text units, and the process of relational reasoning is performed one relationship to one relationship pair. Therefore, we split the process of relational reasoning between the given subject and the given auxiliary body into: when the given subject and the given auxiliary body are single text units, relational reasoning is directly implemented based on the given subject and the given auxiliary body; and when the given subject and the given auxiliary body are a set of text units, the text units in the given subject and the given auxiliary body need to form relational pairs respectively and perform relational reasoning respectively, and merge the reasoning results of all relational pairs to obtain the final reasoning results of the given subject and the given auxiliary body. It should be noted that when the given subject and the given auxiliary body are a set of text units, the given subject and the given auxiliary body text units should be in a one-to-one correspondence, so that when reasoning is performed based on the given subject and the given auxiliary body, it is actually based on the reasoning performed on multiple pairs of relational pairs formed by the given subject and the given auxiliary body. That is, assuming that a given subject M = {M1, M2, M3...Mn}, a given auxiliary body must have N = {N1, N2, N3...Nn}, where M1 corresponds to N1, M2 corresponds to N2, M3 corresponds to N3...Mn corresponds to Nn. When implementing relational reasoning, M1 and N1 need to be used as the subject and object of reasoning to implement the corresponding relational reasoning, and then M1 and N1 need to be used as the subject and object of reasoning to implement the corresponding relational reasoning, and so on. Finally, all the reasoning results are merged to obtain the relational reasoning result of the given subject M and the given auxiliary body N. Therefore, it can be seen that the relational reasoning process of a given subject and a given auxiliary body as a set of text units is actually a repetition of the relational reasoning process of a given subject and a given auxiliary body as a single text unit. Therefore, this embodiment mainly describes the reasoning process of a given subject and a given auxiliary body as a single text unit.
[0062] Furthermore, if the given subject or the given auxiliary body is a text unit in the reasoning database, the given subject can be directly used as the reasoning subject or the given auxiliary body can be used as the reasoning auxiliary body. If the given subject or the given auxiliary body is not an original text unit in the reasoning database, it is necessary to convert the given subject into the text unit form in the corresponding reasoning database or convert the given auxiliary body into the text unit form in the corresponding reasoning database based on the equivalent body mapping relationship of the text unit in the reasoning database.
[0063] Before further realizing the conversion of text units based on the equivalent body mapping relationship of text units in the reasoning database, it is possible that the language type of the given subject or the given auxiliary body is different from the language type of the reasoning database. At this time, in order to implement the subsequent reasoning process, the given subject or the given auxiliary body must be translated into the language type of the reasoning database. Since the translation process for the given subject and the given auxiliary body is the same, the translation process is explained here by taking the given subject as an example. After obtaining the given subject, determine whether the language type of the given subject and the reasoning database is the same. If they are not the same, translate the given subject into the language type of the reasoning database, and use the translated given subject as the mapping subject. If they are the same, directly use the given subject as the mapping subject.
[0064] This setting can realize the reasoning of subjects of different languages.
[0065] At the same time, there may be polysemous words in the given subject and the given auxiliary body. For example, the text unit "apple" may refer to "apple" or "Apple Company". Therefore, when converting text units based on the equivalent body mapping relationship of the text units in the reasoning database, it is also necessary to first determine which word the text unit "apple" specifically refers to in the reasoning database. Since the process of converting the given subject into the reasoning subject and the process of converting the given auxiliary body into the reasoning auxiliary body based on the equivalent body mapping relationship of the text units in the reasoning database are the same, this implementation also takes the given subject as an example to illustrate the conversion process. Specifically, the process of converting the given subject into the reasoning subject based on the equivalent body mapping relationship of the text units in the reasoning database is: after the given subject is converted into the mapping subject through the translation process, it is determined whether the mapping subject corresponds to a single text unit in the reasoning database. If the mapping subject corresponds to a single text unit in the reasoning database, the mapping subject is used as the reasoning subject. If the mapping subject corresponds to multiple text units in the reasoning database, it is necessary to select the text unit with the correct meaning from the multiple text units corresponding to the mapping subject in the reasoning database based on the limiting features of the given subject to serve as the reasoning subject of the preset text unit. The definition of the text unit corresponding to the mapping subject is: the text unit identical to the mapping subject in the inference database; if the text unit identical to the mapping subject does not exist in the inference database, it can be determined whether there is a text unit in the inference database that has the mapping subject in the equivalent set corresponding to the mapping subject. If so, the text unit corresponding to the equivalent set is used as the text unit corresponding to the mapping subject.
[0066] For a mapping subject with polysemous words, when a corresponding given subject is given, the limiting feature corresponding to the given subject will be given, so that the text unit corresponding to the mapping subject can be clarified based on the limiting feature in the future. The limiting feature can be an attribute of the given subject, or a type of the given subject, etc. The method of selecting the text unit with the correct meaning from the multiple text units corresponding to the mapping subject based on the limiting feature of the given subject can be set as follows: obtaining all the text units corresponding to the mapping subject in the inference database, and then obtaining the knowledge structure of each corresponding text unit in the inference database, and each of them is used as a comparison knowledge structure. Then, it is determined which comparison knowledge structure among all the comparison knowledge structures contains the limiting feature of the given subject, and the text unit containing the limiting feature of the given subject in the comparison knowledge structure is selected as the text unit with the correct meaning corresponding to the mapping subject. For example, for the text unit "apple", if the limiting feature is "company", and the knowledge structure of "Apple Company" contains the category "company", then the text unit "apple" refers to "Apple Company"; and if the limiting feature is "fruit", and the knowledge structure of "apple" contains the category "fruit", then the text unit "apple" refers to the fruit "apple".
[0067] The semantic distance between the text unit and the limiting feature can also be used to determine the text unit with the correct meaning referred to by the polysemous word. Specifically, based on the limiting feature of the given subject, the text unit with the correct meaning can be selected from the multiple text units corresponding to the mapping subject. It can also be set as: obtaining all the text units corresponding to the mapping subject in the inference database, and then calculating the semantic distance between each text unit and the limiting feature of the given subject, and selecting the text unit with the closest semantic distance to the limiting feature of the given subject from all the text units corresponding to the mapping subject as the text unit with the correct meaning corresponding to the mapping subject. For example, determine whether "apple" is closer to the semantic distance of the company "google" or the semantic distance of the fruit "banana". If the semantic distance to "google" is close, then the text unit "apple" refers to "Apple Company"; otherwise, it refers to the fruit "apple".
[0068] Through the above method, the reasoning subject corresponding to the given subject can be obtained, and the reasoning auxiliary body corresponding to the given auxiliary body can be obtained. The reasoning subject and the reasoning auxiliary body constitute a reasoning relationship pair.
[0069] Step S102, based on the characteristics of the inference subject or the characteristics of the similar text units, a set of similar text units of the inference subject is obtained from the inference database.
[0070] The specific inferable subject characteristics can be the preset semantic distance of the distance inference subject, a certain set classification type corresponding to the inference subject, a similar classification type of the set classification type of the inference subject, or clustering the inference subject by a certain clustering method, etc. The similar text unit characteristics can be the type of the similar text unit of the pre-set inference subject, etc. The selection of the filtering condition based on the specific inference subject's similar text unit set can be based on actual needs.
[0071] When the inference subject characteristic is set to a preset semantic distance from the inference subject, the process of obtaining a set of similar text units of the inference subject from the inference database is as follows: the preset preset semantic distance must be obtained first. Since the set of similar text units of the inference subject is obtained directly based on the preset semantic distance, there may be a situation where the set of similar text units is empty. Therefore, it is also necessary to set a second error range parameter to assist the preset semantic distance, so as to determine a suitable second semantic distance range based on the specific situation of the set of similar text units. Further, firstly, the preset semantic distance is used as the second distance range, and all text units in the inference database whose semantic distance to the inference subject is within the second distance range are set as a temporary text unit set; then, when judging whether the number of text units in the temporary text unit set is less than the second preset number threshold, if so, the current second distance range is increased by the step distance to form a new second distance range, and the corresponding temporary text unit set is obtained again based on the new second distance range, until the formed second distance range is greater than or equal to the sum of the preset semantic distance and the second error range parameter, or until the number of text units in the temporary text unit set is judged to be greater than or equal to the second preset number threshold, at this time, the current second distance range is directly used as the second semantic distance range, and the current temporary text unit set is used as the similar text unit set of the inference subject. This method of obtaining a similar text unit set can be used when the preset semantic distance is known, and can also be implemented on the basis of having corresponding restrictions on similar text units, and the corresponding restrictions can be reflected in the form of the corresponding semantic distance of the distance inference subject.
[0072] When the characteristic of the inference subject is a set classification type corresponding to the inference subject, the process of obtaining the set of similar text units of the inference subject from the inference database is as follows: Since text units often belong to multiple classification types, if you want to realize the directional reasoning of the text unit relationship, you can directly set the classification type of the inference text to be used, and set it as the set classification type; it should be noted that the set classification type is pre-set by the reasoning personnel; after determining the set classification type, you can search all text units under the set classification type from the inference database, and collect all the searched text units as the set of similar text units of the inference subject. It should be noted that before forming the set of similar text units, the inference subject in the searched text unit needs to be eliminated, even if the set of similar text units does not contain the inference subject. This method of obtaining the set of similar text units can be implemented under the condition that the similar text units and the inference subject have the same classification type, so that the reasoning direction of the characteristic reasoning can be realized and the reasoning accuracy can be improved.
[0073] When the inference subject characteristic is a similar classification type to the classification type set by the inference subject, the process of obtaining the similar text unit set of the inference subject from the inference database is as follows: Based on the above principle, it can be seen that if the directional reasoning of the inference text is to be realized, the set classification type of the inference text needs to be set. At the same time, in order to expand the scope of reasoning, a preset classification semantic distance can also be set, that is, based on the preset classification semantic distance, all classification types similar to the set classification type can be obtained; further, all classification types in the inference database whose semantic distance to the set classification type is within the preset classification semantic range are collected as the target classification type collection. Then, all text units under the set classification type and the target classification type in the target classification type collection are searched from the inference database, and all the searched text units are collected as the similar text unit set of the inference subject. It should be noted that before forming the similar text unit set, the inference subject in the searched text unit needs to be eliminated, even if the similar text unit set does not contain the inference subject. This method of obtaining the similar text unit set can be implemented under the condition that the similar text unit and the inference subject have the same or similar classification type, so that the reasoning in a similar direction can be realized and the scope of reasoning can be expanded.
[0074] When the reasoning subject characteristic is to cluster the reasoning subject by a certain clustering method, the process of obtaining the set of similar text units of the reasoning subject from the reasoning database is as follows: calculate the semantic distance between any two text units whose semantic distance can be calculated among all the text units in the reasoning database, and convert the obtained semantic distance into an edge according to the numerical ratio, take the text unit as a point, and form a semantic distance network for all the text units in the reasoning database. It should be noted that no edge connection is used between two text units that do not have a semantic distance in the reasoning database, and only the edge corresponding to the semantic distance between the two text units is used to connect the two text units that have a semantic distance, so as to obtain the semantic distance network formed by all the text units in the reasoning database. Then, a preset threshold is obtained, which is the maximum allowable length of the edge corresponding to the semantic distance, and then the edges with semantic distances less than the preset threshold in the semantic distance network can be eliminated, so that the semantic distance network will be split into multiple semantic distance sub-networks; obtain all the text units in the semantic distance sub-network to which the reasoning subject belongs, and collect all the text units into the set of similar text units of the reasoning subject. It should be noted that before forming a set of similar text units, the inference subject in the searched text unit needs to be eliminated, even if the set of similar text units does not contain the inference subject. Figure 2 This is a structural diagram of a semantic distance network and a semantic distance subnetwork. This method of obtaining a set of similar text units realizes indirect search of text units. It not only obtains similar text units related to the reasoning subject, but also obtains similar text units of text units that belong to the same clustering type as the reasoning subject in semantic distance, which can expand the scope of reasoning as much as possible and is suitable for use in large-scale searches.
[0075] When the characteristic of the near text unit is the type of the near text unit of the pre-set reasoning subject, the process of obtaining the near text unit set of the reasoning subject from the reasoning database is as follows: if the relational reasoning direction is to be set, it can also be achieved by setting the classification type to which the near text unit of the reasoning subject belongs, specifically setting the classification type to which the near text unit belongs to be the near classification type; then search all the text units under the near classification type from the reasoning database, and collect all the found text units as the near text unit set of the reasoning subject. This method of obtaining the near text unit set realizes the method adopted when the classification type of the near text unit is known, and realizes the directional reasoning of the reasoning subject to the near text units, which is suitable for use in the case where there are corresponding restrictions on the classification type of the near text unit.
[0076] This embodiment may also obtain a set of similar text units of the inference subject from the inference database through other reasonable methods, which are not exhaustively listed here.
[0077] Step S103, setting a first semantic distance range based on the semantic distance between the inference subject and the inference auxiliary subject, and for each similar text unit in the similar text unit set, obtaining a subset of adjacent text units corresponding to the similar text unit whose distance to all text units within the first semantic distance range is obtained.
[0078] Specifically, the semantic distance between the inference subject and the inference auxiliary is calculated as the first semantic distance, and the first semantic distance range is set based on the first semantic distance. Then, each similar text unit in the similar text unit set is acted on based on the first semantic distance range to obtain a subset of adjacent text units corresponding to each similar text unit in the similar text unit set. It should be noted that in order to avoid the situation where the obtained adjacent text unit subset is empty or the text unit data rarely affects the reasoning, a first error range parameter can be set to assist in determining the first semantic distance range. Specifically, for each similar text unit, there is the following first semantic distance range and adjacent text unit subset acquisition process: first, the first semantic distance is used as the first distance range, and all text units in the inference database whose semantic distance to the similar text unit is within the first distance range are grouped as the inference text unit set; then, it is determined whether the number of text units in the inference text unit set is less than the first preset number threshold, and if so, the current first distance range is increased by the step distance to form a new first distance range, and the corresponding inference text unit set is acquired again based on the new first distance range, until the formed first distance range is greater than or equal to the sum of the preset semantic distance and the first error range parameter, or until it is determined that the number of text units in the inference text unit set is greater than or equal to the first preset number threshold, at which time the current first distance range is directly used as the first semantic distance range, and the current inference text unit set is used as the adjacent text unit subset of the similar text unit.
[0079] By the above method, the adjacent text unit subset corresponding to each adjacent text unit in the adjacent text unit set can be obtained. And the above setting can make the first semantic distance range value for each adjacent text unit adjustable based on the number of text units in the corresponding adjacent text unit subset, so as to obtain a reasonable reasoning range as much as possible.
[0080] Step S104 , respectively forming text relationship pairs with the similar text units and each adjacent text unit in the corresponding adjacent text unit subset, and grouping all text relationship pairs into a target relationship pair set of inference relationship pairs.
[0081] Specifically, for each similar text unit in the similar text unit set, a text relationship pair is formed between the similar text unit and each adjacent text unit in the corresponding adjacent text unit subset, and then all the obtained text unit pairs are set as a target relationship pair set of inference relationship pairs.
[0082] The target relationship pair set obtained through the above process may sometimes have hallucinations. In order to avoid unwanted results in the target relationship pair set, the target relationship pair set may be filtered by a filtering condition to obtain a filtered target relationship pair set.
[0083] The specific filtering conditions can include at least one of the following conditions:
[0084] It is judged whether the classification of the adjacent text units in the text relationship pair and the reasoning subject in the reasoning database is consistent. At the same time, it is also judged whether the classification of the adjacent text units in the text relationship pair and the reasoning auxiliary in the reasoning database is consistent. If there is inconsistency in the above judgment process, the corresponding text unit pair is directly filtered out. This filtering method is usually not suitable for the case where the reasoning subject characteristic is the similar classification type of the reasoning subject setting classification type.
[0085] It is determined whether there is an intersection between the knowledge structure of the adjacent text units in the text relationship pair in the reasoning database and the knowledge structure of the reasoning subject in the reasoning database. At the same time, it is also determined whether there is an intersection between the knowledge structure of the adjacent text units in the text relationship pair in the reasoning database and the knowledge structure of the reasoning auxiliary in the reasoning database. If the above does not have an intersection, the corresponding text unit pair is directly filtered out.
[0086] Determine whether there is an inclusion or subordination relationship between the reasoning subject and the reasoning auxiliary. If so, filter out the text relationship pairs in which the similar text units and the corresponding adjacent text units do not have an inclusion or subordination relationship; if there is no inclusion or subordination relationship between the reasoning subject and the reasoning auxiliary, there is no need to filter the text relationship pairs based on this filtering condition.
[0087] Determine whether the inference subject and the adjacent text units have the same text properties. If so, filter out the text unit pairs whose adjacent text units do not have the same text properties as the inference auxiliary text. Specific text properties include characters, words, phrases, sentences, plurals, nouns, verbs, adjectives, and adverbs.
[0088] It should be noted that, in this embodiment, other reasonable filtering conditions may be added based on actual conditions, and the content of the filtering conditions is not limited to the above-mentioned content.
[0089] In order to explain the text unit relationship reasoning method of this embodiment in detail, a specific example is used for illustration below.
[0090] Assume that the given subject in a given relation pair is "China", the given auxiliary is "Beijing", and the inference database is set to an English database. Since the language types of the given subject and the given auxiliary are the same as those of the inference database, and the given subject and the given auxiliary are not polysemous, and further, the given subject and the given auxiliary are both text units in the inference database, the given subject is the inference subject, and the given auxiliary is the inference auxiliary.
[0091] This example uses the preset semantic distance of the distance reasoning subject to obtain a set of similar text units. Specifically, it is assumed that the preset semantic distance is D, and the second semantic distance range is set based on the preset semantic distance, and the set of similar text units of the reasoning subject obtained based on the second semantic distance range is X = ["Japan", "Korea", "Hanghzou"...]. The semantic distance between the reasoning subject and the reasoning auxiliary is set to d1, and the first semantic distance range of each similar text unit Xi is determined based on the semantic distance d1; then, each similar text unit Xi in the similar text unit set X is traversed based on the first semantic distance range to obtain the adjacent text unit subset Yj corresponding to each similar text unit Xi in the adjacent text unit set X. For example, the adjacent text unit subset Y corresponding to "Japan" = ["Tokyo", "Osaka",...]; the adjacent text unit subset Y corresponding to "Korea" = ["Seoul",...]; the adjacent text unit subset Y corresponding to "Hangzhou" = ["Zhejiang",...]. Finally, we get the target relationship pair set “Xi-Yj” = [“Japan-Tokyo”, “Japan-Osaka”, “Korea-Seoul”, “Hangzhou-Zhejiang”…].
[0092] In order to improve the reasoning accuracy of the target relationship pair set, the current target relationship pair set can be filtered by filtering conditions. In this example, the first item in the filtering condition can be used for filtering. Specifically, the classification of the inference subject is obtained as "country", and the classification of the inference auxiliary is obtained as "capital". Then, according to "country"-"capital", the text relationship pairs in the current target relationship pair set are filtered and extracted, and finally the final target relationship pair set "Xi-Yj" = ["Japan-Tokyo", "Korea-Seoul"...] is obtained; in this process, the classification of "Osaka" and "Zhejiang" does not have "capital", and the classification of "Hangzhou" is not "country", so "Japan-Osaka" and "Hangzhou-Zhejiang" are filtered out.
[0093] It should be noted that all semantic distances obtained in this embodiment are calculated and obtained using the same semantic distance calculation method.
[0094] The text unit relational reasoning method provided by the embodiment of the present invention obtains a set of similar text units of the reasoning subject on demand, and determines a screening semantic distance range based on the semantic distance between the reasoning subject and the reasoning auxiliary body, and then uses the screening semantic distance range as the reasoning direction to obtain the adjacent text units of each similar text unit of the reasoning subject, that is, to realize the reasoning of all similar text units from the reasoning subject to the reasoning auxiliary body, and then obtain all target relationship pairs with similar reasoning relationships with the reasoning subject and the reasoning auxiliary body. The method of the present invention can perform similarity search based on the relationship between text unit relationship pairs on the basis of unknown target text units, and obtain all desired text unit pairs with given relationships in the similar results, thereby expanding the text unit reasoning range. The screened text unit pairs are filtered by filtering conditions, which further improves the reasoning accuracy in the text unit pair relational reasoning process.
[0095] like Figure 3 As shown, this embodiment also provides a text unit relationship reasoning device, including a reasoning relationship pair acquisition module, a similar text unit acquisition module, an adjacent text unit acquisition module and a target relationship pair acquisition module.
[0096] The reasoning relationship pair acquisition module is used to acquire the reasoning relationship pair, the reasoning relationship pair includes a reasoning subject and a reasoning auxiliary, and the reasoning subject and the reasoning auxiliary belong to the reasoning database.
[0097] The similar text unit acquisition module is used to acquire a set of similar text units of the reasoning subject from the reasoning database based on the characteristics of the reasoning subject or the characteristics of the similar text units.
[0098] The adjacent text unit acquisition module is used to set a first semantic distance range based on the semantic distance between the reasoning subject and the reasoning auxiliary body. For each adjacent text unit in the adjacent text unit set, an adjacent text unit subset corresponding to the adjacent text unit is obtained, which is a set of all text units within the first semantic distance range.
[0099] The target relationship pair acquisition module is used to form text relationship pairs with the similar text units and each adjacent text unit in the corresponding adjacent text unit subset, and to group all text relationship pairs into a target relationship pair set of inference relationship pairs.
[0100] The text unit relational reasoning device provided by the embodiment of the present invention obtains a set of similar text units of the reasoning subject on demand, and determines a screening semantic distance range based on the semantic distance between the reasoning subject and the reasoning auxiliary body, and then uses the screening semantic distance range as the reasoning direction to obtain the adjacent text units of each similar text unit of the reasoning subject, that is, to realize the reasoning of all similar text units from the reasoning subject to the reasoning auxiliary body, and then obtain all target relationship pairs with similar reasoning relationships with the reasoning subject and the reasoning auxiliary body. The device of the present invention can perform similarity search based on the relationship between text unit relationship pairs on the basis of unknown target text units, and obtain all desired text unit pairs with given relationships in the similar results, thereby expanding the text unit reasoning range. The screened text unit pairs are filtered by filtering conditions, which further improves the reasoning accuracy in the text unit pair relational reasoning process.
[0101] The present application embodiment also provides a computer-readable storage medium. A person of ordinary skill in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the processor through a program, and the program can be stored in a computer-readable storage medium, and the storage medium is a non-transitory medium, such as a random access memory, a read-only memory, a flash memory, a hard disk, a solid-state hard disk, a magnetic tape, a floppy disk, an optical disc, and any combination thereof. The above-mentioned storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a digital video disc (DVD)), or a semiconductor medium (for example, a solid-state disk (SSD)), etc.
[0102] like Figure 4 As shown, an embodiment of the present application provides a terminal.
[0103] The terminal of this embodiment includes a processor and a memory connected to each other; the memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory, so that all or part of the steps in the method of the above embodiment can be implemented when the terminal executes.
[0104] The beneficial effects of all or part of the steps of the method in the above embodiment are the same as the beneficial effects obtained by applying the terminal provided by the embodiment of the present invention, and will not be described in detail here.
[0105] It should be noted that the memory may include random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage. Similarly, the processor may also be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.
[0106] Although the embodiments disclosed in the present invention are as above, the contents described are only embodiments adopted for facilitating the understanding of the present invention and are not intended to limit the present invention. Any technician in the technical field to which the present invention belongs can make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed in the present invention, but the protection scope of the present invention shall still be subject to the scope defined in the attached claims.
Claims
1. A text unit relational reasoning method, comprising: Acquire a reasoning relationship pair, wherein the reasoning relationship pair includes a reasoning subject and a reasoning auxiliary body, and the reasoning subject and the reasoning auxiliary body belong to a reasoning database; Based on the characteristics of the inference subject or the characteristics of the similar text units, obtaining a set of similar text units of the inference subject from the inference database; A first semantic distance range is set based on the semantic distance between the inference subject and the inference auxiliary subject, and for each similar text unit in the similar text unit set, a set of all text units within the first semantic distance range is obtained as a subset of adjacent text units corresponding to the similar text unit; Each of the similar text units is respectively formed into a text relationship pair with each adjacent text unit in the corresponding adjacent text unit subset, and all of the text relationship pairs are grouped as a target relationship pair set of the inference relationship pairs.
2. The inference method according to claim 1, characterized in that: Obtaining inference relationship pairs includes: Obtaining a given relationship pair, wherein the given relationship pair includes a given subject and a given auxiliary subject; Based on the equivalent body mapping method, the reasoning subject corresponding to the given subject and the reasoning auxiliary body corresponding to the given auxiliary body are obtained from the reasoning database, and the given subject and the given auxiliary body constitute the reasoning relationship pair.
3. The inference method according to claim 1, characterized in that: Based on the characteristics of the reasoning subject, obtaining a set of text units close to the reasoning subject from the reasoning database includes: A preset semantic distance is obtained, a second semantic distance range is set based on the preset semantic distance, and all text units in the inference database whose semantic distances to the inference subject are within the second semantic distance range are grouped as a similar text unit set of the inference subject.
4. The inference method according to claim 1, characterized in that: Based on the characteristics of the reasoning subject, obtaining a set of text units close to the reasoning subject from the reasoning database includes: The set classification type of the inference subject is obtained, and all text units under the set classification type are searched from the inference database to form a set of similar text units of the inference subject, wherein the set of similar text units does not include the inference subject.
5. The inference method according to claim 1, characterized in that: Based on the characteristics of the reasoning subject, obtaining a set of text units close to the reasoning subject from the reasoning database includes: Acquire the set classification type and preset classification semantic range of the inference subject, and group all classification types in the inference database whose semantic distance from the set classification type is within the preset classification semantic range as a target classification type set; The set classification type and all text unit sets under the target classification type in the target classification type set are searched from the inference database as a similar text unit set of the inference subject, and the similar text unit set does not include the inference subject.
6. The inference method according to claim 1, characterized in that: Based on the characteristics of the reasoning subject, obtaining a set of text units close to the reasoning subject from the reasoning database includes: Obtaining the semantic distance between all arbitrary two text units in the inference database, and forming a semantic distance network with all text units in the inference database using the obtained semantic distance as an edge and the text unit as a point; The edges whose semantic distance in the semantic distance network is less than a preset threshold are removed to obtain multiple semantic distance sub-networks, and all text units in the semantic distance sub-network to which the reasoning subject belongs are grouped as a set of similar text units of the reasoning subject, and the set of similar text units does not include the reasoning subject.
7. The inference method according to claim 1, characterized in that: Based on the characteristics of similar text units, obtaining a set of similar text units of the inference subject from the inference database includes: Acquire similar classification types, and search the inference database for all text units under the similar classification types, taking the set as the inference subject, and a set of similar text units.
8. The inference method according to claim 1, characterized in that: Before all the text relationship pairs are grouped into the target relationship pair set of the inference relationship pairs, the method further includes filtering all the formed text relationship pairs by a filtering condition; The filtering condition includes at least one of the following conditions: Filter out text relationship pairs in which the classification of the adjacent text units in the text relationship pairs and the inference subject in the inference database is inconsistent, or the classification of the adjacent text units in the text relationship pairs and the inference auxiliary body in the inference database is inconsistent; Filter out text relationship pairs in which the knowledge structure of the adjacent text units in the inference database does not intersect with the knowledge structure of the inference subject in the inference database, or the knowledge structure of the adjacent text units in the text relationship pairs does not intersect with the knowledge structure of the inference auxiliary body in the inference database; If the inference subject and the inference auxiliary have a containment or subordinate relationship, then filter out the text relationship pairs where the close text unit and the corresponding adjacent text unit do not have a containment or subordinate relationship; If the inference subject has the same text property as the adjacent text unit, then filter out the text unit pairs whose text properties of the adjacent text unit are not the same as the text properties of the inference auxiliary body, where the text properties are characters, words, phrases, sentences, plurals, nouns, verbs, adjectives or adverbs.
9. A text unit relational reasoning device, characterized in that: It includes a reasoning relationship pair acquisition module, a similar text unit acquisition module, a neighboring text unit acquisition module and a target relationship pair acquisition module; The inference relationship pair acquisition module is used to acquire the inference relationship pair, wherein the inference relationship pair includes an inference subject and an inference auxiliary body, and the inference subject and the inference auxiliary body belong to an inference database; The similar text unit acquisition module is used to acquire a set of similar text units of the reasoning subject from the reasoning database based on the reasoning subject characteristics or similar text unit characteristics; The adjacent text unit acquisition module is used to set a first semantic distance range based on the semantic distance between the inference subject and the inference auxiliary body, and for each adjacent text unit in the adjacent text unit set, obtain a set of all text units within the first semantic distance range as an adjacent text unit subset corresponding to the adjacent text unit; The target relationship pair acquisition module is used to form a text relationship pair with each of the similar text units and each adjacent text unit in the corresponding adjacent text unit subset, and to group all of the text relationship pairs into a target relationship pair set of the inference relationship pairs.
10. A storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, it implements the text unit relational reasoning method described in any one of claims 1 to 8.
11. A terminal, characterized in that: include: A processor and a memory, wherein the memory is communicatively connected to the processor; The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the terminal executes the text unit relationship reasoning method as described in any one of claims 1 to 8.
Citation Information
Patent Citations
Semantic search system and search method based on domain ontology
CN110659350A
Knowledge graph semantic mapping method and system
CN111914534A
Text-enhanced knowledge graph joint representation learning method and device
CN112347268A
Text processing method and device, computer equipment and storage medium
CN113011172A
Question reasoning method and device based on semantic understanding
CN116341564A
