Entry processing method and device
By obtaining and analyzing the knowledge density of the initial entries in the database field and applying the corresponding entry generation strategy, the problem of mismatch in the knowledge density of entry in the existing technology is solved. The generated entries are more in line with user needs and quality and consistency are guaranteed.
Patent Information
- Application Number
- CN202510220440.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-26
- Publication Date
- 2025-06-13
AI Technical Summary
When the prior art generates explanations for database fields, it is difficult to meet user needs, and the knowledge density of generated entries may not match, resulting in the generation of simple fields being too complex or the generation of complex fields being too simple.
By obtaining the initial entry of the target field, determining its knowledge density, and generating new entry according to the matching entry generation strategy, ensuring that the generated entry content matches the knowledge density of the initial entry.
The generated entries are more in line with user needs, avoiding the problems of high knowledge density but simple content or low density but complex content, and ensuring the quality and consistency of entries.
Smart Images
Figure CN120144733A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing, and particularly to a method and apparatus for processing entries. Background Art
[0002] Currently, explanations can be generated for fields to facilitate users to obtain relevant information about the fields through the explanations. For example, explanations can be generated for fields in data tables stored in a database. Among them, the explanations generated for fields can also be referred to as entries.
[0003] The currently common solution for generating explanations for fields is: generating explanations for fields based on predefined templates. Specifically, corresponding templates can be selected according to the field type and relevant information can be filled in. For example, fields of the identifier (id) type are uniformly explained as "fields used to uniquely identify a certain entity". However, in this way, the entries generated for fields may not meet the user's needs. For example, in this way, overly complex entries may be generated for simple fields, while very simple entries may be generated for some relatively complex fields.
[0004] Therefore, there is an urgent need for a solution that can solve the above problems. Summary of the Invention
[0005] To solve or at least partially solve the above technical problems, this application provides a method and apparatus for processing entries.
[0006] In a first aspect, this application provides a method for processing entries, the method including:
[0007] Obtaining a first entry of a target field;
[0008] Determining the knowledge density of the first entry according to the first entry;
[0009] Generating a second entry for the target field according to an entry generation strategy matching the knowledge density.
[0010] Optionally, the determining the knowledge density of the first entry according to the first entry includes:
[0011] Determining the knowledge density of the first entry according to one or more of the entity density of the first entry, the syntactic complexity of the first entry, and the entity distribution complexity of the first entry.
[0012] Optionally, the entity density of the first entry is determined according to the weights of the respective entities in the multiple entities included in the first entry, the number of times each entity appears in the first entry, and the length of the first entry.
[0013] Optionally, the first entry includes a first entity, and the weight of the first entity is determined according to one or more of the context relevance of the first entity in the first entry, the timeliness of the first entity, and the reference value of the first entity.
[0014] Optionally, the context relevance of the first entity in the first entry is determined according to the word embedding of the first entity and the word embedding of the first entry.
[0015] Optionally, the timeliness of the first entity is determined according to the time interval between the time when the first entity was last used and the current time.
[0016] Optionally, the reference value of the first entity is determined according to the citation frequency of the first entity and the sum of the citation frequencies of all entities in the business to which the first entity belongs.
[0017] Optionally, the syntactic complexity of the first entry is determined according to one or more of the average depth of the syntax tree of the first entry, the average number of branches included in the syntax tree of the first entry, and the number of sentences included in the first entry.
[0018] Optionally, the entity distribution complexity of the first entry is determined according to the probability of each entity among the multiple entities included in the first entry appearing in the first entry and the number of entities included in the first entry.
[0019] Optionally, before determining the knowledge density of the first entry according to the first entry, the method further includes:
[0020] Using two retrieval methods, keyword retrieval and semantic retrieval, to determine at least one first knowledge fragment related to the target field;
[0021] Inputting the at least one first knowledge fragment and the first entry into a large model to obtain the entities included in the first entry.
[0022] Optionally, the entry generation strategy includes a field retrieval depth and / or a field retrieval width. Before generating a second entry for the target field according to the entry generation strategy matching the knowledge density, the method further includes:
[0023] According to the range to which the knowledge density belongs, determine the field retrieval depth and / or the field retrieval width, where the field retrieval depth is used to indicate the number of levels of upstream fields required to generate a second entry for the target field, and the field retrieval width is used to indicate the number of fields at the same level as the target field required to generate a second entry for the target field.
[0024] Optionally, the field retrieval width also indicates the proportion of fields of different types among the fields at the same level as the target field.
[0025] Optionally, generating a second entry for the target field according to the entry generation strategy matching the knowledge density includes:
[0026] Retrieving based on the fields to be retrieved according to the field retrieval depth and / or field retrieval width to obtain at least one second knowledge fragment;
[0027] Processing the at least one second knowledge fragment to obtain at least one third knowledge fragment, and the knowledge density of the at least one third knowledge fragment is within a preset knowledge density range;
[0028] Inputting the at least one third knowledge fragment into a large model to obtain the second entry.
[0029] Optionally, the at least one third knowledge fragment is obtained by adjusting the at least one second knowledge fragment N times according to multiple adjustment parameters, where:
[0030] The process of the first adjustment is as follows:
[0031] Determining the adjustment amplitude of the multiple adjustment parameters when performing the first adjustment according to the at least one second knowledge fragment;
[0032] Adjusting the at least one second knowledge fragment according to the adjustment amplitude of the multiple adjustment parameters when performing the first adjustment;
[0033] When i is an integer greater than or equal to 2 and less than or equal to N, the process of the i-th adjustment is as follows:
[0034] Determining the adjustment amplitude of the multiple adjustment parameters when performing the i-th adjustment according to the at least one knowledge fragment obtained from the (i - 1)-th adjustment;
[0035] Adjusting the at least one knowledge fragment obtained from the (i - 1)-th adjustment according to the adjustment amplitude of the multiple adjustment parameters when performing the i-th adjustment.
[0036] Optionally, the multiple adjustment parameters include at least two of the following:
[0037] Knowledge fragment size, overlap rate between knowledge fragments, associated field range of the target field, and historical entry quantity of the target field, where the associated field range is used to indicate the number of fields at the same level as the target field and / or the number of levels of the upstream fields of the target field.
[0038] Optionally, determining the adjustment range of the multiple adjustment parameters when performing the first adjustment according to the at least one second knowledge fragment includes:
[0039] Determining the knowledge density of each second knowledge fragment in the at least one second knowledge fragment;
[0040] Determining the adjustment range of the multiple adjustment parameters when performing the first adjustment according to the knowledge density of each second knowledge fragment.
[0041] In a second aspect, the present application provides a term processing device, and the device includes:
[0042] An acquisition unit, configured to acquire a first term of a target field;
[0043] A first determination unit, configured to determine the knowledge density of the first term according to the first term;
[0044] A generation unit, configured to generate a second term for the target field according to a term generation strategy matching the knowledge density.
[0045] Optionally, the first determination unit is configured to:
[0046] Determine the knowledge density of the first term according to one or more of the entity density of the first term, the syntactic complexity of the first term, and the entity distribution complexity of the first term.
[0047] Optionally, the entity density of the first term is determined according to the weights of the various entities in the multiple entities included in the first term, the number of times each entity appears in the first term, and the length of the first term.
[0048] Optionally, the first term includes a first entity, and the weight of the first entity is determined according to one or more of the context relevance of the first entity in the first term, the timeliness of the first entity, and the reference value of the first entity.
[0049] Optionally, the context relevance of the first entity in the first term is determined according to the word embedding of the first entity and the word embedding of the first term.
[0050] Optionally, the timeliness of the first entity is determined according to the time interval between the time when the first entity was last used and the current time.
[0051] Optionally, the reference value of the first entity is determined according to the citation frequency of the first entity and the sum of the citation frequencies of all entities in the business to which the first entity belongs.
[0052] Optionally, the syntactic complexity of the first entry is determined according to one or more of the average depth of the syntax tree of the first entry, the average number of branches included in the syntax tree of the first entry, and the number of sentences included in the first entry.
[0053] Optionally, the entity distribution complexity of the first entry is determined according to the probability of each entity among the multiple entities included in the first entry appearing in the first entry and the number of entities included in the first entry.
[0054] Optionally, the apparatus further includes:
[0055] A second determination unit, configured to, before determining the knowledge density of the first entry according to the first entry, use two retrieval methods, keyword retrieval and semantic retrieval, to determine at least one first knowledge fragment related to the target field;
[0056] A third determination unit, configured to input the at least one first knowledge fragment and the first entry into a large model to obtain the entities included in the first entry.
[0057] Optionally, the entry generation strategy includes a field retrieval depth and / or a field retrieval width, and the apparatus further includes:
[0058] A fourth determination unit, configured to, before generating a second entry for the target field according to the entry generation strategy matching the knowledge density, determine the field retrieval depth and / or the field retrieval width according to the range to which the knowledge density belongs, where the field retrieval depth is used to indicate the number of levels of upstream fields to be retrieved for generating the second entry for the target field, and the field retrieval width is used to indicate the number of fields at the same level as the target field to be retrieved for generating the second entry for the target field.
[0059] Optionally, the field retrieval width also indicates the proportion of different types of fields among the fields at the same level as the target field.
[0060] Optionally, the generation unit is configured to:
[0061] Retrieve based on the fields to be retrieved according to the field retrieval depth and / or the field retrieval width to obtain at least one second knowledge fragment;
[0062] Process the at least one second knowledge fragment to obtain at least one third knowledge fragment, where the knowledge density of the at least one third knowledge fragment is within a preset knowledge density interval;
[0063] Input the at least one third knowledge fragment into a large model to obtain the second entry.
[0064] Optionally, the at least one third knowledge fragment is obtained by performing N adjustments on the at least one second knowledge fragment according to a plurality of adjustment parameters, where:
[0065] The process of the first adjustment is as follows:
[0066] Based on the at least one second knowledge fragment, determine the adjustment amplitude of the plurality of adjustment parameters when performing the first adjustment;
[0067] Based on the adjustment amplitude of the plurality of adjustment parameters when performing the first adjustment, adjust the at least one second knowledge fragment;
[0068] When i is an integer greater than or equal to 2 and less than or equal to N, the process of the i-th adjustment is as follows:
[0069] Based on the at least one knowledge fragment obtained from the (i - 1)-th adjustment, determine the adjustment amplitude of the plurality of adjustment parameters when performing the i-th adjustment;
[0070] Based on the adjustment amplitude of the plurality of adjustment parameters when performing the i-th adjustment, adjust the at least one knowledge fragment obtained from the (i - 1)-th adjustment.
[0071] Optionally, the plurality of adjustment parameters include at least two of the following:
[0072] The size of the knowledge fragment, the overlap rate between knowledge fragments, the associated field range of the target field, and the number of historical entries of the target field, where the associated field range is used to indicate the number of fields at the same level as the target field and / or the number of levels of the upstream fields of the target field.
[0073] Optionally, based on the at least one second knowledge fragment, determining the adjustment amplitude of the plurality of adjustment parameters when performing the first adjustment includes:
[0074] Determine the knowledge density of each second knowledge fragment in the at least one second knowledge fragment;
[0075] Based on the knowledge density of each second knowledge fragment, determine the adjustment amplitude of the plurality of adjustment parameters when performing the first adjustment.
[0076] In a third aspect, an embodiment of the present application provides an electronic device, and the electronic device includes a processor and a memory;
[0077] The processor is configured to execute instructions stored in the memory, so that the electronic device executes the method according to any one of the above first aspects.
[0078] Fourthly, an embodiment of the present application provides a computer-readable storage medium, including instructions that direct a device to execute the method according to any one of the above first aspects.
[0079] Fifthly, an embodiment of the present application provides a computer program product, which when running on a computer, causes the computer to execute the method according to any one of the above first aspects.
[0080] Compared with the prior art, the embodiments of the present application have the following advantages:
[0081] The present application provides a method for processing entries. The method includes: obtaining a first entry of a target field, where the first entry may be the entry currently adopted by the target field. After obtaining the first entry, according to the first entry, determine the knowledge density of the first entry, where the knowledge density of the first entry is used to characterize the knowledge content included in the first entry. Then, generate a second entry for the target field according to an entry generation strategy that matches the knowledge density of the first entry. Since when generating the second entry, an entry generation strategy that matches the knowledge density of the first entry is adopted. Therefore, the content included in the generated second entry matches the knowledge density of the first entry, thereby being able to avoid the problem that the knowledge density of the first entry is high, but the content included in the first entry is very simple, or being able to avoid the problem that the knowledge density of the first entry is low, but the content included in the first entry is relatively complex. Therefore, the second entry generated by using this solution better meets the user's needs. BRIEF DESCRIPTION OF THE DRAWINGS
[0082] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments recorded in the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.
[0083] Figure 1 A flowchart of a method for processing entries provided by an embodiment of the present application;
[0084] Figure 2 A flowchart of a method for generating entries provided by an embodiment of the present application;
[0085] Figure 3 A flowchart of a method for processing entries provided by an embodiment of the present application;
[0086] Figure 4 A structural diagram of an entry processing device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0087] In order to enable those skilled in the art to better understand the solution of this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.
[0088] The following will describe in detail various non-limiting embodiments of this application in conjunction with the accompanying drawings.
[0089] Exemplary method
[0090] See Figure 1 , which is a schematic flowchart of a method for processing entries provided in an embodiment of this application.
[0091] The method provided in the embodiments of this application can be executed by a server. In some scenarios, this method can also be executed by a client, and the embodiments of this application do not make specific limitations.
[0092] In the following examples, the method provided in the embodiments of this application is described by taking the execution by the server as an example.
[0093] In this embodiment, the method may include the following steps: S101 - S103.
[0094] S101: Obtain the first entry of the target field.
[0095] In this application, the target field may be a certain field involved in the first service. For example, the first service may correspond to multiple data tables, and the target field may be a field in a certain data table corresponding to the first service. The embodiments of this application do not specifically limit the first service, and the first service can be determined according to the actual business scenario.
[0096] The first entry may be the entry currently adopted by the target field. It may be a historical entry generated by the server for the target field. For example, the first entry may be the entry generated by the server for the target field last time.
[0097] The embodiments of the present application do not specifically limit the manner in which the server generates the first entry. For example, the server may adopt a traditional method of generating an explanation for a field to generate the first entry for the target field. Alternatively, the server may obtain a third entry of the target field, where the third entry is an entry generated by the server for the target field before generating the first entry, and generate the first entry for the target field based on the third entry. Herein, the manner in which the server generates the first entry for the target field based on the third entry is the same as the manner in which the server generates the second entry for the target field based on the first entry, and will not be described in detail herein.
[0098] In the present application, the first entry may include at least one sentence. For example, the first entry may include at least one knowledge fragment, and each knowledge fragment includes at least one sentence.
[0099] S102: Determine the knowledge density of the first entry according to the first entry.
[0100] In the present application, the knowledge density of the first entry is used to characterize the knowledge content in the unit content of the first entry. The knowledge content in the first entry represents the complexity of the target field to a certain extent. The unit content mentioned herein may be several texts, and the embodiments of the present application do not make specific limitations.
[0101] In the present application, the knowledge density of the first entry is calculated according to the content included in the first entry. As a specific example, the knowledge density of the first entry is determined based on one or more of the entity density of the first entry, the syntactic complexity of the first entry, and the entity distribution complexity of the first entry.
[0102] In one example, the knowledge density of the first entry is determined according to the entity density of the first entry. For example, the knowledge density of the first entry is equal to the entity density of the first entry multiplied by a certain parameter.
[0103] In yet another example, the knowledge density of the first entry is determined according to the syntactic complexity of the first entry. For example, the knowledge density of the first entry is equal to the syntactic complexity of the first entry multiplied by a certain parameter.
[0104] In another example, the knowledge density of the first entry is determined according to the entity distribution complexity of the first entry. For example, the knowledge density of the first entry is equal to the entity distribution complexity of the first entry multiplied by a certain parameter.
[0105] In yet another example, the knowledge density of the first entry is obtained by weighted summation of at least two of the entity density of the aforementioned first entry, the syntactic complexity of the first entry, and the entity distribution complexity of the first entry. For example: The knowledge density of the first entry is calculated according to the following formula (1):
[0106] KD = w1 * ED + w2 * DC + w3 * SC Formula (1)
[0107] In Formula (1):
[0108] KD represents the knowledge density;
[0109] ED represents the entity density;
[0110] DC represents the syntactic complexity;
[0111] SC represents the entity distribution complexity;
[0112] w1 is the weight of ED;
[0113] w2 is the weight of DC;
[0114] w3 is the weight of SC;
[0115] w1 + w2 + w3 = 1.
[0116] Wherein:
[0117] The entity density of the first entry is used to characterize the number of entities included in the unit content of the first entry.
[0118] In one example, the entity density of the first entry is determined according to the number of entities included in the first entry and the length of the first entry. For example, the entity density of the first entry is equal to the number of entity occurrences in the first entry divided by the length of the first entry. Wherein, the length of the first entry is, for example, the number of texts included in the first entry. Wherein, the number of entity occurrences is greater than or equal to the number of entities included in the first entry, because an entity can appear at least once in the first entry.
[0119] In another example, it is considered that the amount of knowledge associated with different entities is different. Therefore, in order to make the knowledge density of the first entry calculated based on the entity density of the first entry more accurate, in one example, the entity density of the first entry is determined according to the weights of each entity among the multiple entities included in the first entry, the number of times each entity appears in the first entry respectively, and the length of the first entry. In this scenario, when calculating the entity density of the first entry, the weights of the entities included in the first entry are considered, so that the differences in the amount of knowledge associated with different entities can be reflected when calculating the entity density of the first entry. Correspondingly, the knowledge density of the first entry calculated based on the entity density of the first entry is more accurate. For example, the entity density of the first entry can be calculated according to the following formula (2).
[0120] ED = Σ(F * W) / L Formula (2)
[0121] In Formula (2):
[0122] ED is the entity density;
[0123] F is the number of times the entity appears in the first entry;
[0124] W is the weight of the entity;
[0125] L is the length of the first entry;
[0126] Σ means summing up (F * W) calculated for each entity included in the first entry respectively.
[0127] Regarding the weight of the entity, the first entity is taken as an example for illustration. Here, the first entity is any entity included in the first entry.
[0128] As described above, the weight of the first entity is related to the amount of knowledge associated with the first entity. Or rather, the weight of the first entity is related to the amount of knowledge associated with the first entity in the first entry. The more knowledge the first entity is associated with, the greater the weight of the first entity should be. The less knowledge the first entity is associated with, the smaller the weight of the first entity should be. In one example, considering the context relevance of the first entity in the first entry, the timeliness of the first entity, and the reference value of the first entity, all are related to the amount of knowledge associated with the first entity.
[0129] For example, the higher the context relevance of the first entity in the first entry, the higher the amount of knowledge associated with the first entity. The higher the timeliness of the first entity, the higher the amount of knowledge associated with the first entity. The higher the reference value of the first entity, the higher the amount of knowledge associated with the first entity.
[0130] Therefore, the weight of the first entity is determined according to one or more of the context relevance of the first entity in the first entry, the timeliness of the first entity, and the reference value of the first entity.
[0131] In a specific example, the weight of the first entity is calculated according to the following formula (3).
[0132] W = T * R * C Formula (3)
[0133] In Formula (3):
[0134] W is the weight of the first entity;
[0135] T is used to represent the timeliness of the first entity, which can be, for example, the timeliness weight (Time factor).
[0136] R is used to represent the reference value (Reference value) of the first entity.
[0137] C is used to represent the context relevance (Context relevance) of the first entity.
[0138] In this application, the context relevance of the first entity in the first entry is used to represent the degree of association between the first entity and other contents in the first entry. In an example, the context relevance of the first entity in the first entry is determined according to the word embedding of the first entity and the word embedding of the first entry. For example, T = cos (the word embedding of the first entity, the word embedding of the first entity), where the word embedding of the first entity can be obtained by inputting the first entity into the text-to-word embedding model, and the word embedding of the first entry can be obtained by inputting the first entry into the aforementioned text-to-word embedding model.
[0139] The timeliness of the first entity is used to represent the popularity of the first entity. Generally, the higher the popularity of the first entity, the more content related to the first entity can be displayed in the entry. Correspondingly, the lower the popularity of the first entity, the less content related to the first entity can be displayed in the entry. In an example, the timeliness of the first entity is determined according to the time interval between the time when the first entity was last used and the current time. For example, T = e^(-λt), where λ is the decay coefficient and t is the aforementioned time interval.
[0140] In one example, the first entity is used, for example, the first entity is queried, or the first entity is used in a certain business branch of the first business. In one example, entity records of the first entity can be saved in the server, and the entity records record the time when the first entity was last used. According to the entity records, the aforementioned time interval can be determined.
[0141] The reference value of the first entity can be understood as the importance level of the first entity. In one example, the reference value of the first entity is determined according to the citation frequency of the first entity. In another example, the reference value of the first entity is determined according to the citation frequency of the first entity and the sum of the citation frequencies of all entities in the business to which the first entity belongs, where the business to which the first entity belongs can be the first business mentioned above. For example, the reference value of the first entity is equal to the citation frequency of the first entity divided by the sum of the aforementioned citation frequencies.
[0142] In this application, the syntactic complexity of the first entry refers to the syntactic complexity of the sentences included in the first entry. It is not difficult to understand that generally, the higher the syntactic complexity, the more complex the content it can contain, and correspondingly, the relatively higher the knowledge content. In this application, the syntactic complexity of the first entry can be determined according to the syntax tree of the first entry. Specifically, the first entry can include multiple sentences, and each sentence can correspond to a syntax tree.
[0143] In one example, considering that both the depth of the syntax tree and the branches of the syntax tree can represent the syntactic complexity of the sentence. Therefore, in one example, the syntactic complexity of the first entry is determined according to the syntax tree depth of at least one sentence included in the first entry and the number of branches included in the at least one sentence.
[0144] In another example, the syntactic complexity of the first entry is determined according to one or more of the average syntax tree depth of the first entry, the average number of branches included in the syntax tree of the first entry, and the number of sentences included in the first entry. Among them, the average syntax tree depth is the average value of the syntax tree depths of at least one sentence included in the first entry, and the average number of branches is the average value of the number of branches included in the syntax trees of at least one sentence included in the first entry. For example, the syntactic complexity of the first entry is calculated according to the following formula (4).
[0145] SC = (D * B) / S Formula (4)
[0146] In Formula (4):
[0147] SC is the syntactic complexity;
[0148] D is the average depth of the syntax tree;
[0149] B is the average number of branches;
[0150] S is the number of branches included in the first entry.
[0151] In this application, the entity distribution complexity of the first entry represents the entity distribution in the first entry and is used to characterize whether the entity distribution in the first entry is uniform. In one example, the entity distribution complexity of the first entry is determined according to the probability of each entity among the multiple entities included in the first entry appearing in the first entry and the number of entities included in the first entry. For example, the distribution complexity of the first entry can be calculated according to the following formula (5).
[0152] DC = -Σ(P * log(P)) / log(N) Formula (5)
[0153] In Formula (5):
[0154] P is the occurrence probability of a certain entity in the first entry;
[0155] N is the number of entities included in the first entry;
[0156] Σ means summing after calculating P * log(P) for each entity included in the first entry respectively.
[0157] Based on the principle of information entropy, generally, it is hoped that the distribution complexity of the entry is small. For example, it is between 0.4 - 0.6. This is because for an entry, it is hoped that its content focuses on the core uses and key features of the field. An overly uniform interpretation will instead dilute the key information and reduce the retrieval efficiency.
[0158] S103: Generate a second entry for the target field according to the entry generation strategy matching the knowledge density of the first entry.
[0159] In this application, it is considered that the entry content of the first entry may not match the knowledge density of the first entry. For example, the knowledge density of the first entry is high, but the content included in the first entry is very simple, or the knowledge density of the first entry is low, but the content included in the first entry is relatively complex. To avoid this problem, after determining the knowledge density of the first entry, an entry generation strategy matching the knowledge density of the first entry can be obtained, and further, an entry can be regenerated for the target field based on this entry generation strategy.
[0160] In this application, different knowledge densities can correspond to different entry generation strategies. For example, the correspondence between the knowledge density and the entry generation strategy can be as shown in Table 1 below:
[0161] Table 1
[0162]
[0163] Therefore, after determining the knowledge density of the first entry, according to the knowledge density of the first entry, a term generation strategy matching the knowledge density corresponding to the first entry can be determined, so as to subsequently generate a second entry for the target field based on the term generation strategy matching the knowledge density corresponding to the first entry. For the convenience of description, the "term generation strategy matching the knowledge density corresponding to the first entry" is referred to as the "first entry generation strategy".
[0164] In the embodiments of the present application, the first entry generation strategy can be used to indicate the fields to be retrieved for generating the second entry for the target field. Specifically, when generating the second entry, the fields can be first used as retrieval conditions for retrieval to obtain at least one second knowledge fragment, and then the second entry can be obtained based on the at least one second knowledge fragment.
[0165] In the present application, different term generation strategies correspond to different field retrieval scopes. Or rather, for different term generation strategies, the fields that need to be retrieved are different. Among them, the fields that need to be retrieved include, in addition to the target field itself, the associated fields of the target field. The so-called associated fields of the target field refer to the fields that have an association relationship with the target field.
[0166] In one example, the foregoing first entry generation strategy includes a field retrieval depth and / or a field retrieval width. The field retrieval depth and / or the field retrieval width are used to determine the foregoing associated fields.
[0167] Wherein:
[0168] The field retrieval depth is used to indicate the number of levels of upstream fields to be retrieved for generating the second entry for the target field. Among them, there is a certain blood relationship between fields, and the blood relationship between fields can be represented by a tree structure. In the tree structure, a node can correspond to a field. The upstream field of the target field refers to the field corresponding to the parent node of the node corresponding to the target field. Among them, the number of levels of the upstream field indicates the number of levels of the blood relationship between the field to be retrieved and the target field. For example, when the field retrieval depth is 1, it means that the foregoing associated fields include the upstream fields of the target field; when the field retrieval depth is 2, it means that the foregoing associated fields include: the upstream fields of the target field, the upstream fields of the upstream fields of the target field, and so on. In one example, the field retrieval depth is determined according to the knowledge density of the first entry.
[0169] The field retrieval width is used to indicate the number of fields at the same level as the target field that need to be retrieved to generate the second entry for the target field. Among them, the fields at the same level as the target field can be the fields in the same data table as the target field. That is to say, the aforementioned associated fields also include the fields at the same level as the target field.
[0170] In this application, among the fields at the same level as the target field, there may be various types of fields. For example, it includes core fields, directly associated fields, and remotely associated fields. Among them, the core field refers to the core field in the aforementioned data table, and the core field is also an important field in the data table. The directly associated field refers to the field that has a direct association relationship with the target field. The remotely associated field refers to the field that has an indirect association relationship with the target field. In this application, the fields at the same level as the target field in the aforementioned associated fields can include multiple types of fields. Considering that the importance levels of different types of fields are different, therefore, in one example, the aforementioned field retrieval width can also be used to indicate the proportion of different types of fields among the fields at the same level as the target field, so that the server can determine the respective quantities of each type of field among the fields at the same level as the target field in the aforementioned associated fields. For example, the proportion of core fields is greater than the weight of related fields, and the proportion of directly associated fields is greater than the proportion of remotely associated fields. In one example, the proportion of different types of fields is determined according to the knowledge density of the first entry.
[0171] As described above, after determining the first entry generation strategy, the fields that need to be retrieved to generate the second entry for the target field can be determined. After determining the fields that need to be retrieved, the second entry for the target field can be generated through Figure 2 S201 - S203 shown below.
[0172] Figure 2 It is a schematic flowchart of a method for generating entries provided by an embodiment of this application.
[0173] S201: Retrieve based on the fields that need to be retrieved to obtain at least one second knowledge fragment.
[0174] In this application, the retrieved fields can be used as an index to retrieve from the knowledge base to obtain at least one second knowledge fragment. The knowledge base mentioned here can be a pre - constructed specific knowledge base (such as a knowledge base built based on business) or an Internet knowledge base. The embodiments of this application do not make specific limitations.
[0175] Regarding the method of retrieving with the retrieved fields as an index, it can adopt traditional retrieval methods, and the embodiments of this application do not make specific limitations.
[0176] In one example, after obtaining at least one second knowledge fragment, the at least one knowledge fragment can be processed such as deduplication, and the processing result is used as the second entry.
[0177] In another example, after obtaining at least one second knowledge fragment, S202 - S203 can be executed to obtain the second entry.
[0178] S202: Process the at least one second knowledge fragment to obtain at least one third knowledge fragment, and the knowledge density of the third knowledge fragment is within a preset knowledge density range.
[0179] S203: Input the at least one third knowledge fragment into a large model to obtain the second entry.
[0180] In the present application, a large model and second knowledge fragments can be used to generate a second entry for a target field. However, considering that the aforementioned at least one second knowledge fragment may form a complete document, if the complete document is directly input into the large model, the second entry generated by the large model may not meet the requirements. In view of this, in the present application:
[0181] After obtaining at least one second knowledge fragment, the at least one second knowledge fragment can be processed to obtain at least one third knowledge fragment, and the at least one third knowledge fragment is input into the large model, so that the large model generates a second entry based on the at least one third knowledge fragment.
[0182] In one example, considering that the knowledge density of the third knowledge fragment affects the effect of the second entry generated by the large model, therefore, the knowledge density of the third knowledge fragment obtained by processing the second knowledge fragment is within a preset knowledge density range. For example, the knowledge density of the third knowledge fragment is in the range of [0.4, 0.6].
[0183] In the present application, the at least one third knowledge fragment is obtained by adjusting the at least one second knowledge fragment N times according to multiple adjustment parameters. Wherein, N is an integer greater than or equal to 1.
[0184] The embodiments of the present application do not specifically limit the multiple adjustment parameters, and the multiple adjustment parameters can be determined according to the actual situation. As a specific example, the multiple parameters may include at least two of: the size of the knowledge fragment, the overlap rate between knowledge fragments, the range of associated fields of the target field, and the number of historical entries. Wherein:
[0185] The size of the knowledge fragment refers to the number of texts included in the knowledge fragment.
[0186] The overlap rate of knowledge fragments is used to represent the overlap situation between knowledge fragments, and it can be equal to the ratio of the number of overlapping texts between two knowledge fragments to the size of the knowledge fragments. Among them, there is a certain overlap rate between the knowledge fragments input into the large model, which can help the large model output high-quality second entries.
[0187] The associated field range of the target field is used to indicate the number of fields at the same level as the target field and / or the number of levels of the upstream fields of the target field.
[0188] The number of historical entries refers to the number of historical entries generated for the target field. When using the large model to generate a second entry for the target field, one or more historical entries of the target field can also be input into the large model, so that the large model can combine the relevant information of the historical entries of the target field when generating the second entry.
[0189] In this application, the process of the foregoing N adjustments is as follows:
[0190] At the first adjustment:
[0191] According to the at least one second knowledge fragment, determine the adjustment amplitude of the multiple adjustment parameters at the first adjustment. Among them, the adjustment amplitude of the multiple adjustment parameters at the first adjustment can include the adjustment amplitude corresponding to each adjustment parameter. For example, it includes the adjustment amplitude corresponding to the size of the knowledge fragment, the adjustment amplitude corresponding to the overlap rate between knowledge fragments, the adjustment amplitude corresponding to the associated field range of the target field, and the adjustment amplitude corresponding to the number of historical entries of the target field.
[0192] In one example, since the knowledge density of the adjusted third knowledge fragment needs to be within a preset knowledge density range, when determining the adjustment amplitude of the multiple adjustment parameters at the first adjustment, the knowledge density of the at least one second knowledge fragment can be referred to. The determination principle for the adjustment amplitude corresponding to each adjustment parameter is similar. As a specific example, for the adjustment amplitude corresponding to any adjustment parameter, the maximum adjustment amplitude of this adjustment parameter at the first adjustment can be determined first, and then, using the knowledge density of the at least one second knowledge fragment, smooth the maximum adjustment amplitude to obtain the adjustment amplitude of this adjustment parameter at the first adjustment. Among them:
[0193] The foregoing maximum adjustment amplitude can be determined according to the benchmark adjustment amplitude and other attenuation factors.
[0194] Smooth the maximum adjustment range. For example, the smoothing parameter can be determined according to the knowledge density of the at least one second knowledge fragment. For example, the smoothing parameter can be determined according to the variance of the knowledge density of the at least one second knowledge fragment. Then, use the smoothing parameter and the average knowledge density of the at least one second knowledge fragment to smooth the maximum adjustment range to obtain the adjustment range of the adjustment parameter at the first adjustment.
[0195] Regarding the knowledge density of each second knowledge fragment, it can refer to the calculation method of the knowledge density of the first entry, which will not be repeated here.
[0196] At the i-th adjustment:
[0197] Determine the adjustment range of the multiple adjustment parameters at the i-th adjustment according to the at least one knowledge fragment obtained from the (i - 1)-th adjustment; adjust the at least one knowledge fragment obtained from the (i - 1)-th adjustment according to the adjustment range of the multiple adjustment parameters at the i-th adjustment.
[0198] The principle of the i-th adjustment is the same as that of the first adjustment. The difference between the two is that at the first adjustment, the adjustment object is at least one second knowledge fragment, while at the i-th adjustment, the adjustment object is the knowledge fragment obtained from the (i - 1)-th adjustment.
[0199] Therefore, for the specific implementation of the i-th adjustment, reference can be made to the description part of the first adjustment in the previous text, which will not be repeated here. Wherein: i is an integer greater than or equal to 2 and less than or equal to N.
[0200] In one example, after executing S101 and before executing S102, entities in the first entry can also be recognized to facilitate further calculation of the knowledge density of the first entry.
[0201] In one example, traditional entity recognition methods can be used to recognize entities in the first entry.
[0202] In another example, a large model can be combined to recognize entities in the first entry. And, to improve the accuracy of the recognized entities. First, two retrieval methods, keyword retrieval and semantic retrieval, can be used to determine at least one first knowledge fragment related to the target field. For example, the keyword retrieval can adopt the BM25 algorithm, and the semantic retrieval can use the HNSW (Hierarchical Navigable Small World graphs) algorithm to determine at least one knowledge fragment with a high degree of relevance to the target field both semantically and literally.
[0203] After determining at least one first knowledge fragment, the at least one first knowledge fragment and the first entry can be input into the large model, so that when the large model identifies the entities in the first entry, it can refer to the foregoing at least one first knowledge fragment, thereby enabling the large model to more accurately identify the entities in the first entry.
[0204] The above has introduced the entry processing method provided by the embodiments of the present application. Next, in combination with Figure 3 , the entry processing method provided by the embodiments of the present application will be introduced. Figure 3 It is a schematic flowchart of an entry processing method provided by an embodiment of the present application.
[0205] As Figure 3 shown, the entry processing process provided by the embodiments of the present application is as follows:
[0206] First, obtain the first entry.
[0207] Then, determine the entity density, the syntactic complexity, and the entity distribution complexity of the first entry, and determine the knowledge density of the first entry.
[0208] Further, based on the knowledge density of the first entry, determine the matching entry retrieval strategy. This entry retrieval strategy includes the field retrieval depth and the field retrieval width.
[0209] Retrieve according to the entry retrieval strategy to obtain at least one second knowledge fragment, and process the at least one second knowledge fragment, that is, adjust based on the parameters of the four dimensions of the knowledge fragment size, the overlap rate between knowledge fragments, the associated field range, and the number of historical entries, to obtain at least one third knowledge fragment. Then, input the at least one third knowledge fragment into the large model to obtain the second entry generated for the target field.
[0210] As can be seen from the previous description, the first entry can also be generated in the same way as the second entry is generated. For this case, the large model can further evaluate the quality of the first entry and the second entry to determine which entry should be used for the current target field. For example, if the large model determines that the quality of the second entry is better than the quality of the first entry, the target field uses the second entry; if the large model determines that the quality of the first entry is better than the quality of the second entry, the target field continues to use the first entry.
[0211] Exemplary device
[0212] Based on the method provided in the above embodiments, an embodiment of the present application also provides a device. The following introduces this device in combination with the accompanying drawings.
[0213] See Figure 4 , Figure 4Schematic structural diagram of a term processing device provided by an embodiment of the present application.
[0214] Figure 4 The illustrated device 400 is used to execute the term processing method provided in the above embodiment.
[0215] As Figure 4 As shown, the device 400 includes: an acquisition unit 401, a first determination unit 402, and a generation unit 403.
[0216] The acquisition unit 401 is used to acquire a first term of a target field.
[0217] The first determination unit 402 is used to determine the knowledge density of the first term according to the first term.
[0218] The generation unit 403 is used to generate a second term for the target field according to a term generation strategy matching the knowledge density.
[0219] Optionally, the first determination unit 402 is used to:
[0220] Determine the knowledge density of the first term according to one or more of the entity density of the first term, the syntactic complexity of the first term, and the entity distribution complexity of the first term.
[0221] Optionally, the entity density of the first term is determined according to the weights of each entity among the multiple entities included in the first term, the number of times each entity appears in the first term, and the length of the first term.
[0222] Optionally, the first term includes a first entity, and the weight of the first entity is determined according to one or more of the context relevance of the first entity in the first term, the timeliness of the first entity, and the reference value of the first entity.
[0223] Optionally, the context relevance of the first entity in the first term is determined according to the word embedding of the first entity and the word embedding of the first term.
[0224] Optionally, the timeliness of the first entity is determined according to the time interval between the time when the first entity was last used and the current time.
[0225] Optionally, the reference value of the first entity is determined according to the citation frequency of the first entity and the sum of the citation frequencies of all entities in the business to which the first entity belongs.
[0226] Optionally, the syntactic complexity of the first entry is determined according to one or more of the average depth of the syntax tree of the first entry, the average number of branches included in the syntax tree of the first entry, and the number of sentences included in the first entry.
[0227] Optionally, the entity distribution complexity of the first entry is determined according to the probability of each entity among the multiple entities included in the first entry appearing in the first entry and the number of entities included in the first entry.
[0228] Optionally, the device further includes:
[0229] A second determination unit, configured to, before determining the knowledge density of the first entry according to the first entry, use two retrieval methods of keyword retrieval and semantic retrieval to determine at least one first knowledge fragment related to the target field;
[0230] A third determination unit, configured to input the at least one first knowledge fragment and the first entry into a large model to obtain the entities included in the first entry.
[0231] Optionally, the entry generation strategy includes a field retrieval depth and / or a field retrieval width, and the device further includes:
[0232] A fourth determination unit, configured to, before generating a second entry for the target field according to the entry generation strategy matching the knowledge density, determine the field retrieval depth and / or the field retrieval width according to the range to which the knowledge density belongs, where the field retrieval depth is used to indicate the number of levels of upstream fields to be retrieved for generating the second entry for the target field, and the field retrieval width is used to indicate the number of fields at the same level as the target field to be retrieved for generating the second entry for the target field.
[0233] Optionally, the field retrieval width also indicates the proportion of different types of fields among the fields at the same level as the target field.
[0234] Optionally, the generation unit 403 is configured to:
[0235] Retrieve based on the fields to be retrieved according to the field retrieval depth and / or the field retrieval width to obtain at least one second knowledge fragment;
[0236] Process the at least one second knowledge fragment to obtain at least one third knowledge fragment, where the knowledge density of the at least one third knowledge fragment is within a preset knowledge density interval;
[0237] Input the at least one third knowledge fragment into a large model to obtain the second entry.
[0238] Optionally, the at least one third knowledge segment is obtained by adjusting the at least one second knowledge segment N times according to a plurality of adjustment parameters, where:
[0239] The process of the first adjustment is as follows:
[0240] Based on the at least one second knowledge segment, determine the adjustment amplitude of the plurality of adjustment parameters when performing the first adjustment;
[0241] Adjust the at least one second knowledge segment according to the adjustment amplitude of the plurality of adjustment parameters when performing the first adjustment;
[0242] When i is an integer greater than or equal to 2 and less than or equal to N, the process of the i-th adjustment is as follows:
[0243] Based on the at least one knowledge segment obtained from the (i - 1)-th adjustment, determine the adjustment amplitude of the plurality of adjustment parameters when performing the i-th adjustment;
[0244] Adjust the at least one knowledge segment obtained from the (i - 1)-th adjustment according to the adjustment amplitude of the plurality of adjustment parameters when performing the i-th adjustment.
[0245] Optionally, the plurality of adjustment parameters include at least two of the following:
[0246] The size of the knowledge segment, the overlap rate between knowledge segments, the associated field range of the target field, and the number of historical entries of the target field, where the associated field range is used to indicate the number of fields at the same level as the target field and / or the number of levels of the upstream fields of the target field.
[0247] Optionally, determining the adjustment amplitude of the plurality of adjustment parameters when performing the first adjustment based on the at least one second knowledge segment includes:
[0248] Determine the knowledge density of each second knowledge segment in the at least one second knowledge segment;
[0249] Based on the knowledge density of each second knowledge segment, determine the adjustment amplitude of the plurality of adjustment parameters when performing the first adjustment.
[0250] Since the device 400 corresponds to the entry processing method provided in the above method embodiment, the specific implementation of each unit of the device 400 is based on the same concept as the above method embodiment. Therefore, for the specific implementation of each unit of the device 400, reference can be made to the relevant description part of the above method embodiment, which will not be elaborated here.
[0251] An embodiment of the present application further provides an electronic device, which includes a processor and a memory;
[0252] The processor is configured to execute the instructions stored in the memory, so that the electronic device executes the entry processing method provided in the above method embodiment.
[0253] An embodiment of the present application provides a computer-readable storage medium, including instructions, and the instructions direct the device to execute the entry processing method provided in the above method embodiment.
[0254] An embodiment of the present application further provides a computer program product, and when the computer program product runs on a computer, it causes the computer to execute the entry processing method provided in the above method embodiment.
[0255] Those skilled in the art will readily conceive of other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any variations, uses, or adaptations of the present application, which follow the general principles of the present application and include the common general knowledge or conventional technical means in the technical field not disclosed in the present disclosure. The specification and embodiments are only to be regarded as exemplary, and the true scope and spirit of the present application are pointed out by the following claims.
[0256] It should be understood that the present application is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present application is only limited by the appended claims.
[0257] The above are only the preferred embodiments of the present application, and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims
1. A method for processing a term, characterized in that: The method comprises: Get the first entry of the target field; Determining the knowledge density of the first term according to the first term; A second term is generated for the target field according to a term generation strategy that matches the knowledge density.
2. The method according to claim 1, characterized in that The step of determining the knowledge density of the first term according to the first term includes: The knowledge density of the first term is determined according to one or more of the entity density of the first term, the grammatical complexity of the first term, and the entity distribution complexity of the first term.
3. The method according to claim 2, characterized in that The entity density of the first term is determined according to the weight of each entity among the multiple entities included in the first term, the number of times each entity appears in the first term, and the length of the first term.
4. The method according to claim 3, characterized in that The first term includes a first entity, and a weight of the first entity is determined according to one or more of the contextual relevance of the first entity in the first term, the timeliness of the first entity, and the reference value of the first entity.
5. The method according to claim 4, characterized in that The context relevance of the first entity in the first term is determined based on the word embedding of the first entity and the word embedding of the first term.
6. The method according to claim 4, characterized in that The timeliness of the first entity is determined according to the time interval between the last time the first entity was used and the current time.
7. The method according to claim 4, characterized in that The reference value of the first entity is determined according to the sum of the citation frequency of the first entity and the citation frequencies of all entities in the business to which the first entity belongs.
8. The method according to claim 2, characterized in that: The grammatical complexity of the first term is determined according to one or more of an average depth of a grammatical tree of the first term, an average number of branches included in a grammar tree of the first term, and a number of sentences included in the first term.
9. The method according to claim 2, characterized in that: The entity distribution complexity of the first term is determined according to the probability of each entity among the multiple entities included in the first term appearing in the first term and the number of entities included in the first term.
10. The method according to claim 1, characterized in that Before determining the knowledge density of the first term according to the first term, the method further includes: Using keyword search and semantic search to determine at least one first knowledge fragment related to the target field; The at least one first knowledge fragment and the first term are input into a large model to obtain entities included in the first term.
11. The method according to claim 1, characterized in that: The term generation strategy includes field search depth and / or field search width. Before generating a second term for the target field according to the term generation strategy matching the knowledge density, the method further includes: According to the range to which the knowledge density belongs, the field search depth and / or field search width is determined, wherein the field search depth is used to indicate the number of levels of upstream fields required to be retrieved for generating the second entry for the target field, and the field search width is used to indicate the number of fields belonging to the same level as the target field required to be retrieved for generating the second entry for the target field.
12. The method according to claim 11, characterized in that The field search width also indicates the ratio of fields of different types among the fields belonging to the same level as the target field.
13. The method according to claim 11 or 12, characterized in that: Generating a second entry for the target field according to the entry generation strategy matching the knowledge density includes: According to the field search depth and / or field search width, searching is performed based on the field to be searched to obtain at least one second knowledge fragment; Processing the at least one second knowledge fragment to obtain at least one third knowledge fragment, wherein the knowledge density of the at least one third knowledge fragment is within a preset knowledge density range; The at least one third knowledge fragment is input into the big model to obtain the second term.
14. The method according to claim 13, characterized in that The at least one third knowledge fragment is obtained by adjusting the at least one second knowledge fragment N times according to multiple adjustment parameters, wherein: The first adjustment process is as follows: Determining, according to the at least one second knowledge segment, adjustment ranges of the plurality of adjustment parameters when performing a first adjustment; Adjusting the at least one second knowledge segment according to the adjustment ranges of the multiple adjustment parameters during the first adjustment; When i is an integer greater than or equal to 2 and less than or equal to N, the process of the i-th adjustment is as follows: Determining, according to at least one knowledge fragment obtained in the (i-1)th adjustment, adjustment ranges of the plurality of adjustment parameters when performing the i-th adjustment; According to the adjustment ranges of the multiple adjustment parameters during the i-th adjustment, the at least one knowledge segment obtained during the i-1-th adjustment is adjusted.
15. The method according to claim 14, characterized in that The multiple adjustment parameters include at least two of the following: The size of knowledge fragments, the overlap rate between knowledge fragments, the associated field range of the target field, and the number of historical entries of the target field, wherein the associated field range is used to indicate the number of fields belonging to the same level as the target field and / or the number of levels of upstream fields of the target field.
16. The method according to claim 14 or 15, characterized in that Determining, according to the at least one second knowledge fragment, adjustment ranges of the plurality of adjustment parameters when performing a first adjustment, includes: determining a knowledge density of each second knowledge segment in the at least one second knowledge segment; According to the knowledge density of each second knowledge segment, the adjustment range of the plurality of adjustment parameters when performing the first adjustment is determined.
17. A term processing device, characterized in that: The device comprises: An acquisition unit, used for acquiring the first term of a target field; A first determining unit, configured to determine the knowledge density of the first term according to the first term; A generating unit is used to generate a second term for the target field according to a term generation strategy matching the knowledge density.
18. An electronic device, characterized in that: The electronic device comprises a processor and a memory; The processor is used to execute instructions stored in the memory so that the electronic device executes the method according to any one of claims 1 to 16.
19. A computer-readable storage medium, characterized in that: The method comprises instructions, wherein the instructions instruct a device to execute the method according to any one of claims 1 to 16.