Text semantic similarity judgment method, electronic equipment, storage medium and product

Abstract

The embodiment of the invention provides a text semantic similarity judgment method, electronic equipment, a storage medium and a product. The method comprises the steps of obtaining a target text and a standard text similar to the target text, calculating the similarity between the target text and the standard text, and if the similarity exceeds a preset similarity threshold value, performing consistency judgment on digits included in the target text and digits included in the standard text to obtain a judgment result of the digits, and if the judgment result of the number indicates to be consistent, performing consistency judgment on the naming body included in the target text and the naming body included in the standard text to obtain a judgment result of the naming body, and if the judgment result of the naming body indicates to be consistent, outputting prompt information for indicating that the semantics of the target text is similar to the semantics of the standard text. According to the method, the accuracy of judging the similarity between the semantics of the target text and the semantics of the standard text is improved.

Description

Technical Field

[0001] The present application relates to the technical field of text processing, and in particular to a method for determining text semantic similarity, an electronic device, a storage medium and a product. Background Art

[0002] With the rapid development of text processing technology, by comparing the semantic similarity between two texts, efficient and accurate information processing and decision making can be achieved in various application scenarios.

[0003] At present, when judging the semantic similarity of two texts, a preset model is usually used to extract the sentence vector features of the two texts respectively, and then the semantic similarity of the sentence vector features of the two texts is calculated according to a preset similarity function algorithm.

[0004] However, the semantic similarity of two texts judged by the above method is not accurate enough, which affects the user experience. Summary of the invention

[0005] The text semantic similarity determination method, electronic device, storage medium and product provided in the embodiments of the present application are used to improve the accuracy of the semantic similarity determination results of two text segments.

[0006] In a first aspect, an embodiment of the present application provides a method for determining text semantic similarity, comprising:

[0007] Obtaining a target text and a standard text similar to the target text;

[0008] Calculating the similarity between the target text and the standard text;

[0009] If the similarity exceeds a preset similarity threshold, a consistency judgment is performed on the numbers included in the target text and the numbers included in the standard text to obtain a judgment result of the numbers;

[0010] If the digital judgment result indicates consistency, then the named body included in the target text and the named body included in the standard text are judged to be consistent, and a judgment result of the named body is obtained;

[0011] If the judgment result of the named body indicates consistency, prompt information is outputted to indicate that the semantics of the target text is similar to the semantics of the standard text.

[0012] In a possible implementation, the performing consistency judgment on the numbers included in the target text and the numbers included in the standard text to obtain a judgment result of the numbers includes:

[0013] Based on a preset regular matching algorithm, numbers included in the target text are sequentially extracted according to the order of numbers appearing in the target text, and numbers included in the standard text are sequentially extracted according to the order of numbers appearing in the standard text;

[0014] According to the extraction order, the numbers included in the target text are sequentially judged for consistency with the numbers included in the standard text to obtain the judgment result of the numbers;

[0015] The result of the number determination is used to indicate whether the number included in the target text is the same as the number included in the standard text.

[0016] In a possible implementation, the step of performing consistency judgment on the named body included in the target text and the named body included in the standard text to obtain a judgment result of the named body includes:

[0017] Based on a preset first language model, setting labels of entity objects included in the named body;

[0018] Based on the tags of the entity objects, sequentially identifying the named entities included in the target text according to the order in which the tags appear in the target text, and sequentially identifying the named entities included in the standard text according to the order in which the tags appear in the standard text;

[0019] According to the recognition order, the named entities included in the target text are sequentially judged for consistency with the named entities included in the standard text to obtain a judgment result of the named entities;

[0020] The determination result of the named body is used to indicate whether the named body included in the target text is the same as the named body included in the standard text.

[0021] In a possible implementation, calculating the similarity between the target text and the standard text includes:

[0022] Based on a preset second language model, acquiring sentence vector features of the target text and sentence vector features of the standard text;

[0023] Based on a preset cosine function algorithm, similarity is calculated for the sentence vector features of the target text and the sentence vector features of the standard text to obtain the similarity.

[0024] In a possible implementation manner, after acquiring the target text and the standard text similar to the target text, the method further includes:

[0025] Based on a preset comparison algorithm, the target text and the standard text are judged for text content similarity to obtain a judgment result, wherein the judgment result cannot indicate whether the numbers and names included in the target text are consistent with the numbers and names included in the standard text;

[0026] If the judgment result indicates that the text content of the target text is similar to that of the standard text, the step of calculating the similarity between the target text and the standard text is performed.

[0027] In a possible implementation, the method further includes:

[0028] If the determination result of the named body indicates inconsistency, a prompt message is outputted to indicate that the semantics of the target text is not similar to the semantics of the standard text.

[0029] In a second aspect, an embodiment of the present application provides a device for determining text semantic similarity, comprising:

[0030] An acquisition module, used for acquiring a target text and a standard text similar to the target text;

[0031] A calculation module, used for calculating the similarity between the target text and the standard text;

[0032] A judgment module, configured to judge the consistency of the numbers included in the target text and the numbers included in the standard text if the similarity exceeds a preset similarity threshold, and obtain a judgment result of the numbers;

[0033] The judgment module is further configured to, if the judgment result of the number indicates consistency, perform consistency judgment on the named body included in the target text and the named body included in the standard text to obtain a judgment result of the named body;

[0034] The output module is used to output prompt information indicating that the semantics of the target text is similar to the semantics of the standard text if the judgment result of the named body indicates consistency.

[0035] In a third aspect, an embodiment of the present application provides an electronic device, including: a memory, a processor;

[0036] The memory stores computer-executable instructions;

[0037] The processor executes the computer-executable instructions stored in the memory, so that the processor executes various possible implementations of the first aspect as described above.

[0038] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement various possible implementation methods of the first aspect above.

[0039] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements various possible implementation methods of the first aspect above.

[0040] The method for judging the semantic similarity of texts, the electronic device, the storage medium and the product provided by the embodiments of the present application obtain a target text and a standard text similar to the target text. If the similarity between the target text and the standard text exceeds a preset similarity threshold, it indicates that the text content of the target text and the standard text is similar, and then the numbers included in the target text and the numbers included in the standard text are judged for consistency. If they are consistent, it indicates that the numbers included in the target text and the standard text are the same. Then, the naming bodies included in the target text and the naming bodies included in the standard text are judged for consistency. If they are consistent, it indicates that the naming bodies in the target text and the standard text are the same, and finally a prompt information indicating that the semantics of the target text are similar to those of the standard text is output. The present application improves the accuracy of judging the semantic similarity between the semantics of the target text and the standard text by judging the text content, numbers and naming bodies in the target text and the standard text, and by performing multiple judgments. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0042] Figure 1 A flowchart of a method for determining text semantic similarity provided in an embodiment of the present application;

[0043] Figure 2 A flowchart of a method for obtaining a digital judgment result provided in an embodiment of the present application;

[0044] Figure 3 A flowchart of a method for obtaining a judgment result of a named entity provided in an embodiment of the present application;

[0045] Figure 4 A schematic diagram of setting a label of an entity object included in a named body provided in an embodiment of the present application;

[0046] Figure 5 A schematic diagram of the structure of a device for determining text semantic similarity provided in an embodiment of the present application;

[0047] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0048] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0049] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0050] In the description of the embodiments of the present application, terms such as "inside" and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the drawings. This is merely for the convenience of description and does not indicate or imply that the device or component must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it should not be understood as a limitation on the present application.

[0051] In the description of the embodiments of the present application, unless otherwise clearly specified and limited, the terms "connected" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For those skilled in the art, the specific meanings of the above terms in the embodiments of the present application can be understood according to the specific circumstances.

[0052] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0053] Semantic similarity refers to whether two texts are semantically similar. By calculating the semantic similarity between two texts, the degree of similarity between them can be measured.

[0054] In the related art, when judging whether two paragraphs of text are semantically similar, it is usually based on a preset pre-trained language model, such as Bidirectional Encoder Representations from Transformers (BERT), or a preset neural network model, such as Long Short-Term Memory (LSTM) model, to extract sentence vector features from the two paragraphs of text respectively, and then use a preset similarity function algorithm to calculate the similarity between the sentence vector features in the two paragraphs of text, and then judge whether the two paragraphs of text are semantically similar based on the similarity.

[0055] However, the above method is not accurate enough in judging the semantic similarity between two texts. It can roughly judge the similarity of the text content in the two texts, but it is not accurate enough in judging the detailed content such as numbers and names contained in the two texts.

[0056] Therefore, in view of the above technical problems in the prior art, the inventors found in the research process that by making a first comparison between the acquired target text and the standard text similar to the target text, it is possible to determine whether there is similarity in the text content between the target text and the standard text. If there is similarity, a second comparison is performed to determine whether the numbers contained in the target text are the same as the numbers contained in the standard text, so as to make a judgment on the digital content. Then, a third comparison is performed to determine whether the naming body contained in the target text is the same as the naming body contained in the standard text, so as to make a judgment on the naming body content. Thus, the semantic similarity between the target text and the standard text is judged from three dimensions: text content, numbers, and naming bodies. By making multiple judgments, the accuracy of the judgment result is effectively improved. Based on this, the present application proposes a method for judging text semantic similarity, an electronic device, a storage medium, and a product.

[0057] The present application can be applied to multiple scenarios where semantic similarity judgment is required, for example, in information retrieval scenarios, text mining scenarios, and insurance product compensation scenarios, etc. It is understandable that the above scenarios are only used for illustration and do not limit the present application.

[0058] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0059] Figure 1The flowchart of a method for determining text semantic similarity provided in an embodiment of the present application is shown in FIG. The execution subject of the method may be an intelligent terminal with a text semantic similarity determination function, for example, a personal computer, a laptop computer, a smart phone, a tablet computer, a portable wearable device, a server or a server cluster, etc. The method in this embodiment may be implemented by software, hardware, or a combination of software and hardware. Figure 1 As shown, the method may include the following steps:

[0060] S101, obtaining a target text and a standard text similar to the target text.

[0061] Optionally, the target text is taken as an example of content related to a certain product. In this embodiment, the salesperson pre-configures the corresponding standard text for each product based on the parameter information of each product, and configures the standard text corresponding to each product into a preset database. The parameter information includes but is not limited to: the name of the product, the identification information of the product, the company code information, and the purpose of the product.

[0062] A possible implementation method to obtain the target text and the standard text is:

[0063] In response to the user's description of the target product, the target text is obtained, and based on the target text, a standard text corresponding to the target product is matched from the above-mentioned preset database, and the standard text is a similar text to the target text.

[0064] Optionally, after obtaining the target text and the standard text, based on a preset comparison algorithm, for example, using the equals function in the JAVA tool, the text content similarity of the target text and the standard text is judged to obtain a judgment result. It should be noted that the judgment result obtained in this step can be used to characterize that there is a certain degree of similarity in text content between the target text and the standard text, but it cannot characterize whether the numbers and names included in the target text are consistent with the numbers and names included in the standard text. The judgment result can be output in the form of a prompt message.

[0065] For example, if the output prompt information is "The current product description is consistent with the standard", the judgment result indicates that there is a certain degree of similarity between the text content of the target text and the standard text, and the following step S102 is executed.

[0066] If the output prompt message is "The current product description is inconsistent with the standard", the judgment result indicates that the text content of the target text is not similar to that of the standard text, and the judgment ends.

[0067] S102: Calculate the similarity between the target text and the standard text.

[0068] One possible implementation is:

[0069] Based on the preset language model, the sentence vector features of the target text and the sentence vector features of the standard text are obtained, and based on the preset cosine function algorithm, the similarity of the sentence vector features of the target text and the sentence vector features of the standard text is calculated to obtain the similarity.

[0070] Optionally, the preset language model can be a pre-trained language model Sentence_BERT, which first performs text preprocessing on the target text and the standard text to segment the sentences and add special tags before and after the sentences. After encoding the segmented sentences, the encoding is input into the BERT model to obtain word embedding, and through pooling processing, the sentence vector features of the target text and the sentence vector features of the standard text are finally obtained. The cosine similarity of the sentence vector features of the target text and the sentence vector features of the standard text is calculated through a preset cosine function algorithm. Among them, the numerical range of cosine similarity belongs to (0,1), and the closer the similarity is to 1, the more similar the semantics of the target text and the standard text are.

[0071] Optionally, the preset language model includes multiple types, for example, it can be a language model used for Chinese short text, or it can be a language model used for Chinese long text, etc. The specific language model can be determined according to the type of language and the length of the text, and this application does not limit it.

[0072] In this embodiment, a similarity threshold is preset. If the calculated similarity exceeds the similarity threshold, it indicates that there is a certain degree of similarity between the semantics of the target text and the standard text.

[0073] Example 1,

[0074] The target text is: In the event of a collision, overturning, or collapse or falling of your vehicle due to an external object, or a collision with another vehicle, causing damage to your vehicle, your vehicle repair costs will be compensated in accordance with the terms.

[0075] The standard text is: The insured person provides value-added services in accordance with the contract and the special terms of the insurance. In the event of an accident, the loss of your own vehicle will be compensated. For example, if your vehicle collides, overturns, collapses, or is damaged by falling objects, or collides with other vehicles, causing damage to your vehicle, your vehicle repair costs will be compensated according to the terms.

[0076] Assume that the calculated similarity between the above two paragraphs of text is 0.85, and the similarity threshold is 0.9. The similarity does not exceed the preset similarity threshold, indicating that the semantics of the target text is not similar to the semantics of the standard text. Then, a prompt message is output to indicate that the semantics of the target text is not similar to the semantics of the standard text. For example, the prompt message is "The current product description is inconsistent with the standard", and the judgment process ends.

[0077] Example 2,

[0078] The target text is: The insured person provides value-added services in accordance with the contract and the special terms of the insurance. In the event of an accident, the loss of your own vehicle will be compensated. For example, if your vehicle collides, overturns, collapses, or is damaged by falling objects, or collides with other vehicles, causing damage to your vehicle, your vehicle repair costs will be compensated according to the terms.

[0079] The standard text is: The annual premiums for employees insured under this policy are as follows according to their ages: the annual premium for employees under the age of 60 is a yuan per person; the annual premium for employees aged 60-65 (inclusive) is b yuan per person.

[0080] Assume that the calculated similarity between the above two paragraphs of text is 0.4, and the similarity threshold is 0.9. The similarity does not exceed the preset similarity threshold, indicating that the semantics of the target text is not similar to the semantics of the standard text. Accordingly, a prompt message is output to indicate that the semantics of the target text is not similar to the semantics of the standard text. For example, the prompt message is "The current product description is inconsistent with the standard", and the judgment process ends.

[0081] Example 3,

[0082] The target text is: If an employee temporarily loses the ability to work for more than 3 days (not including 3 days), the insured person shall be subsidized for loss of work expenses at a yuan / person / day upon certification by a medical institution of level 2 or above (inclusive) or recognized by the insured person. The subsidy will be stopped after the medical period expires or the extent of the injury is determined, and the maximum period shall not exceed 365 days.

[0083] The standard text is: If an employee temporarily loses the ability to work for more than 5 days (not including 5 days), the insured person shall be subsidized for loss of work expenses at a yuan / person / day, with a certificate from a medical institution of level 2 or above (inclusive) or recognized by the insured person. The subsidy will be stopped after the medical period expires or the degree of disability is determined, and the maximum period shall not exceed 365 days.

[0084] Assume that the calculated similarity between the above two texts is 0.95, the similarity threshold is 0.9, and the similarity exceeds the preset similarity threshold, indicating that the semantics of the target text and the semantics of the standard text are similar to a certain extent, then execute the following steps.

[0085] S103: If the similarity exceeds a preset similarity threshold, a consistency judgment is performed on the numbers included in the target text and the numbers included in the standard text to obtain a judgment result of the numbers.

[0086] If the similarity exceeds the preset similarity threshold, it means that the semantics of the target text and the semantics of the standard text are similar to each other to a certain extent, and it is necessary to further determine whether the numbers included in the target text are the same as the numbers included in the standard text.

[0087] There are two types of number determination results, one indicating that the number included in the target text is the same as the number included in the standard text, and the other indicating that the number included in the target text is different from the number included in the standard text.

[0088] Optionally, based on a preset regular matching algorithm, the numbers included in the target text and the numbers included in the standard text are judged for consistency. For the specific steps for implementing the process, please refer to the following embodiment.

[0089] If the numbers included in the target text are inconsistent with the numbers included in the standard text, it means that the semantics of the target text are not similar to the semantics of the standard text, and a prompt message is output to indicate that the semantics of the target text are not similar to the semantics of the standard text. For example, the prompt message is "The current product description is inconsistent with the standard", and the judgment process ends.

[0090] S104: If the digital judgment result indicates consistency, a consistency judgment is performed on the named body included in the target text and the named body included in the standard text to obtain a judgment result of the named body.

[0091] The named entity is also called a named entity, which refers to an entity identified by a name, including but not limited to: personnel, organization, location, time, quantity, percentage, age, and cost.

[0092] If the result of the number determination indicates consistency, it means that the number included in the target text is the same as the number included in the standard text, and then it is necessary to further determine whether the named body included in the target text is the same as the named body included in the standard text.

[0093] There are two types of determination results of the named body, one indicating that the named body included in the target text is the same as the named body included in the standard text, and the other indicating that the named body included in the target text is different from the named body included in the standard text.

[0094] Optionally, based on a preset language model, the consistency between the named entities included in the target text and the named entities included in the standard text is judged. For the specific implementation steps, please refer to the following embodiment.

[0095] If the naming included in the target text is inconsistent with the naming included in the standard text, it means that the semantics of the target text is not similar to the semantics of the standard text, and a prompt message is output to indicate that the semantics of the target text is not similar to the semantics of the standard text. For example, the prompt message is "The current product description is inconsistent with the standard", and the judgment process ends.

[0096] S105. If the determination result of the named body indicates consistency, outputting prompt information indicating that the semantics of the target text is similar to the semantics of the standard text.

[0097] If the judgment result of the named body indicates consistency, it means that the named body included in the target text is the same as the named body included in the standard text, then a prompt message is output, for example, the prompt message is "the current product description is consistent with the standard", and the judgment process ends.

[0098] If the judgment result of the named body indicates inconsistency, a prompt message is output to indicate that the semantics of the target text is not similar to the semantics of the standard text, for example, the prompt message is "the current product description is inconsistent with the standard", and the judgment process ends.

[0099] It is understandable that the content of the prompt information output above is only used for illustration and does not limit the present application.

[0100] In the above-mentioned embodiment of the present application, by obtaining a target text and a standard text similar to the target text, and calculating the similarity between the target text and the standard text, if the similarity exceeds a preset similarity threshold, the numbers included in the target text and the numbers included in the standard text are judged for consistency, and a judgment result of the numbers is obtained. If the judgment result of the numbers indicates consistency, the naming body included in the target text and the naming body included in the standard text are judged for consistency, and a judgment result of the naming body is obtained. If the judgment result of the naming body indicates consistency, a prompt information indicating that the semantics of the target text and the semantics of the standard text are similar is output. The method of this embodiment improves the accuracy of the judgment of the semantic similarity between the semantics of the target text and the standard text by judging whether the text content in the target text and the standard text is similar, and whether the numbers and the naming body are the same, and by performing multiple judgments.

[0101] Furthermore, based on the above-mentioned embodiment, the following embodiment illustrates a process of performing consistency judgment on the numbers included in the target text and the numbers included in the standard text to obtain the judgment result of the numbers.

[0102] Figure 2 A flowchart of a method for obtaining a digital judgment result provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the method comprises the following steps:

[0103] S201. Based on a preset regular matching algorithm, numbers included in the target text are sequentially extracted according to the order in which numbers appear in the target text, and numbers included in the standard text are sequentially extracted according to the order in which numbers appear in the standard text.

[0104] In the process of similarity comparison between two texts, it is common to have errors in the correspondence of numbers. Therefore, in this embodiment, the comparison of numbers by a regular matching algorithm can improve the accuracy of the similarity judgment result. A regular expression is a text pattern, including common characters and special characters (metacharacters), used to describe and match a series of strings that conform to certain rules.

[0105] The numbers included in the target text and the standard text are extracted respectively through the preset regular expressions. It should be noted that when extracting numbers through regular expressions, they need to be extracted in the order in which the numbers appear, so as to improve the accuracy of the comparison of numbers between the target text and the standard text.

[0106] Take the exemplary embodiment 3 in the above embodiment as an example:

[0107] The target text is: If an employee temporarily loses the ability to work for more than 3 days (not including 3 days), the insured person shall be subsidized for loss of work expenses at a yuan / person / day upon certification by a medical institution of level 2 or above (inclusive) or recognized by the insured person. The subsidy will be stopped after the medical period expires or the extent of the injury is determined, and the maximum period shall not exceed 365 days.

[0108] The standard text is: If an employee temporarily loses the ability to work for more than 5 days (not including 5 days), the insured person shall be subsidized for loss of work expenses at a yuan / person / day, with a certificate from a medical institution of level 2 or above (inclusive) or recognized by the insured person. The subsidy will be stopped after the medical period expires or the degree of disability is determined, and the maximum period shall not exceed 365 days.

[0109] According to the order of the numbers appearing in the target text, the numbers extracted in the target text are 3, 3, a, and 365. According to the order of the numbers appearing in the standard text, the numbers included in the standard text are 5, 5, a, and 365.

[0110] S202: According to the extraction order, the numbers included in the target text are sequentially judged for consistency with the numbers included in the standard text to obtain a judgment result of the numbers.

[0111] The result of the number determination is used to indicate whether the number included in the target text is the same as the number included in the standard text.

[0112] If the judgment result of the number indicates consistency, it means that the number included in the target text is the same as the number included in the standard text, then a prompt message is output to indicate that the semantics of the target text is similar to the semantics of the standard text. For example, the prompt message is "The current product description is consistent with the standard", and the judgment process ends.

[0113] If the judgment result of the number indicates inconsistency, a prompt message is output to indicate that the semantics of the target text is not similar to the semantics of the standard text, for example, the prompt message is "the current product description is inconsistent with the standard", and the judgment process ends.

[0114] Taking the above example, the numbers included in the extracted target text, namely 3, 3, a, 365, are judged for consistency with the numbers included in the extracted standard text, namely 5, 5, a, 365. The judgment result indicates that the numbers included in the target text are different from the numbers included in the standard text, and a prompt message "The current product description is inconsistent with the standard" is output.

[0115] It should be understood that the numbers and prompt information in the above examples are only used for illustration and do not limit the present application.

[0116] In the above-mentioned embodiment of the present application, based on the preset regular matching algorithm, the numbers included in the target text are sequentially extracted according to the order of the numbers appearing in the target text, and the numbers included in the standard text are sequentially extracted according to the order of the numbers appearing in the standard text, and the numbers included in the target text are sequentially judged for consistency with the numbers included in the standard text according to the extraction order, to obtain the judgment result of the numbers. This embodiment improves the judgment efficiency and accuracy by making a judgment through the preset regular matching algorithm, and further improves the accuracy of the judgment of the semantic similarity between the target text and the standard text by making a consistency judgment on the numbers in the target text and the standard text.

[0117] Furthermore, based on the above-mentioned embodiment, the following embodiment illustrates a process of performing consistency judgment on the named body included in the target text and the named body included in the standard text to obtain the judgment result of the named body.

[0118] In the process of comparing the similarity of two texts, errors in naming are also easy to occur.

[0119] For example,

[0120] The target text is: The insured employees under this policy are logistics personnel. If the type of work they are doing at the time of the accident does not match the type of work in the policy, the insurer has the right to refuse compensation.

[0121] The standard text is: The insured employee's job type under this policy is a pile driver at a construction site. If the type of work performed at the time of the accident does not match the type specified in the policy, the insurer has the right to refuse compensation.

[0122] It can be seen from the above example that even if step S104 determines that the target text and the standard text have a high degree of semantic similarity, the problem of inconsistent naming is prone to occur. The naming "logistics personnel" in the above target text is different from the naming "construction site pile driver worker" in the standard text. Therefore, it is necessary to judge whether the naming in the two texts is consistent in order to improve the accuracy of the judgment of the semantic similarity between the two texts.

[0123] Figure 3 A flow chart of a method for obtaining a judgment result of a named entity provided in an embodiment of the present application is shown as follows: Figure 3 As shown, the method comprises the following steps:

[0124] S301: Based on a preset first language model, set labels for entity objects included in the named body.

[0125] Named Entity Recognition (NER) is a subtask of information extraction, which aims to locate and classify named entities in a text into predefined categories, such as occupations, people, organizations, locations, time expressions, quantities, percentages, ages, or costs, etc. Therefore, NER can detect the differences between named entities in two texts.

[0126] Based on the preset first language model and the preset model tool, the labels of the entity objects included in the named body are set, such as Figure 4 As shown, Figure 4 A schematic diagram of setting a label of an entity object included in a named body provided in an embodiment of the present application, in Figure 4 In the lower right corner of the , users can add labels for entity objects, such as "number of people", "product" or "unit type". Figure 4 The content in the text is "This policy covers employees of different ages, and the annual premiums are as follows: for employees under the age of 60, the annual premium is a yuan / person." According to the set entity object label, "60 years old" is identified as "age" and "a yuan" is identified as "expense." Therefore, according to the set entity object label, the named body included in the text can be identified.

[0127] S302. Based on the tags of the entity objects, the named entities included in the target text are sequentially identified according to the order in which the tags appear in the target text, and the named entities included in the standard text are sequentially identified according to the order in which the tags appear in the standard text.

[0128] For example, the target text is: The insured employee's job type under this policy is logistics personnel, and the unit type is Enterprise A. If the type of work and unit type engaged in at the time of the accident do not match the insurance policy, the insurance company has the right to refuse compensation.

[0129] The standard text is: The insured employee's job type is a construction site pile driver, and the unit type is Enterprise B. If the type of work and unit type at the time of the accident do not match the policy, the insurer has the right to refuse compensation.

[0130] According to the method in this step, the named entities extracted in the target text are: Occupation: {Logistics staff}, Unit type: {A enterprise}, and the named entities extracted in the standard text are: Occupation: {Construction site pile driver worker}, Unit type: {B enterprise}.

[0131] S303: According to the recognition order, the named entities included in the target text are sequentially judged to be consistent with the named entities included in the standard text to obtain a judgment result of the named entities.

[0132] The determination result of the named body is used to indicate whether the named body included in the target text is the same as the named body included in the standard text.

[0133] Taking the above example, according to the recognition order, the named entities included in the target text (occupation: {logistics staff}, unit type: {A enterprise}) are different from the named entities included in the standard text (occupation: {construction site pile driver}, unit type: {B enterprise}).

[0134] If the judgment result of the named body indicates inconsistency, a prompt message is output to indicate that the semantics of the target text is not similar to the semantics of the standard text, for example, the prompt message is "the current product description is inconsistent with the standard", and the judgment process ends.

[0135] If the judgment result of the naming body indicates consistency, it means that the naming body included in the target text is the same as the naming body included in the standard text, then a prompt message is output to indicate that the semantics of the target text is similar to the semantics of the standard text, for example, the prompt message is "The current product description is consistent with the standard", and the judgment process ends.

[0136] In this embodiment, if it is determined that the naming body of the target text is the same as the naming body of the standard text, the target text can also be stored in a preset database storing standard texts, and the target text can be stored as another new standard text to increase the standard description. When the user uses the target text next time, because the target text has been stored as the standard text, a quick judgment can be made, thereby reducing the number of online calculations of the language model and improving the judgment efficiency.

[0137] In this embodiment, the preset first language model may be fine-tuned according to actual use requirements to set labels for new entity objects included in the named body, and then the named body included in the two texts may be determined according to the new labels.

[0138] In the above-mentioned embodiment of the present application, the labels of the entity objects contained in the named bodies are set based on the preset first language model, and based on the labels of the entity objects, the named bodies included in the target text are sequentially identified according to the order of the labels appearing in the target text, and the named bodies included in the standard text are sequentially identified according to the order of the labels appearing in the standard text. Then, according to the recognition order, the named bodies included in the target text are sequentially judged for consistency with the named bodies included in the standard text to obtain the judgment result of the named bodies. The method of this embodiment further improves the accuracy of judging the semantic similarity between the target text and the standard text by judging the consistency between the named bodies included in the target text and the named bodies included in the standard text.

[0139] Figure 5 A structural diagram of a device for determining text semantic similarity provided in an embodiment of the present application is shown in FIG. Figure 5 As shown, the text semantic similarity determination device provided in this embodiment includes:

[0140] The acquisition module 501 is used to acquire the target text and the standard text similar to the target text.

[0141] The calculation module 502 is used to calculate the similarity between the target text and the standard text.

[0142] The judgment module 503 is used to judge the consistency of the numbers included in the target text and the numbers included in the standard text if the similarity exceeds a preset similarity threshold, and obtain a judgment result of the numbers.

[0143] The judgment module 503 is further configured to, if the judgment result of the number indicates consistency, perform consistency judgment on the named body included in the target text and the named body included in the standard text to obtain a judgment result of the named body.

[0144] The output module 504 is used to output prompt information indicating that the semantics of the target text is similar to the semantics of the standard text if the judgment result of the named body indicates consistency.

[0145] In a possible implementation, the determination module 503 is specifically configured to:

[0146] Based on a preset regular matching algorithm, the numbers included in the target text are extracted in sequence according to the order in which the numbers appear in the target text, and the numbers included in the standard text are extracted in sequence according to the order in which the numbers appear in the standard text.

[0147] According to the extraction order, the numbers included in the target text are sequentially judged for consistency with the numbers included in the standard text to obtain the judgment result of the numbers.

[0148] The result of the number determination is used to indicate whether the number included in the target text is the same as the number included in the standard text.

[0149] In a possible implementation, the determination module 503 is specifically configured to:

[0150] Based on the preset first language model, the labels of the entity objects contained in the named body are set.

[0151] Based on the tags of the entity objects, the named entities included in the target text are sequentially identified according to the order of the tags appearing in the target text, and the named entities included in the standard text are sequentially identified according to the order of the tags appearing in the standard text.

[0152] According to the recognition order, the named entities included in the target text are sequentially judged for consistency with the named entities included in the standard text to obtain a judgment result of the named entities.

[0153] The determination result of the named body is used to indicate whether the named body included in the target text is the same as the named body included in the standard text.

[0154] In a possible implementation, the calculation module 502 is specifically configured to:

[0155] Based on the preset second language model, the sentence vector features of the target text and the sentence vector features of the standard text are obtained.

[0156] Based on the preset cosine function algorithm, the similarity is calculated for the sentence vector features of the target text and the sentence vector features of the standard text to obtain the similarity.

[0157] In a possible implementation, the determination module 503 is further configured to:

[0158] Based on a preset comparison algorithm, the text content similarity of the target text and the standard text is judged to obtain a judgment result. The judgment result cannot indicate whether the numbers and names included in the target text are consistent with the numbers and names included in the standard text.

[0159] If the determination result indicates that the text contents of the target text and the standard text are similar, a step of calculating the similarity between the target text and the standard text is performed.

[0160] In a possible implementation, the output module 504 is further configured to:

[0161] If the determination result of the named body indicates inconsistency, a prompt message indicating that the semantics of the target text is not similar to the semantics of the standard text is output.

[0162] The device for determining text semantic similarity provided in this embodiment can execute the method provided in the above method embodiment, and its implementation principle and technical effect are similar, which will not be described in detail in this embodiment.

[0163] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 6 As shown, the electronic device provided in this embodiment includes: at least one processor 601 and a memory 602. Optionally, the device 60 further includes a communication component 603. The processor 601, the memory 602 and the communication component 603 are connected via a bus 604.

[0164] In a specific implementation process, at least one processor 601 executes the computer execution instructions stored in the memory 602, so that at least one processor 601 executes the above method.

[0165] The specific implementation process of the processor 601 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0166] In the above embodiments, it should be understood that the processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the invention can be directly implemented as a hardware processor, or can be implemented by a combination of hardware and software modules in the processor.

[0167] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (NVM), such as at least one disk storage.

[0168] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of this application is not limited to only one bus or one type of bus.

[0169] The present application also provides a computer program product, including a computer program, which implements the above-mentioned method for determining text semantic similarity when executed by a processor.

[0170] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above-mentioned method for determining text semantic similarity is implemented.

[0171] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0172] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0173] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0174] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0175] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0176] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0177] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0178] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A method for determining text semantic similarity, characterized in that: include: Obtaining a target text and a standard text similar to the target text; Calculating the similarity between the target text and the standard text; If the similarity exceeds a preset similarity threshold, a consistency judgment is performed on the numbers included in the target text and the numbers included in the standard text to obtain a judgment result of the numbers; If the digital judgment result indicates consistency, then the named body included in the target text and the named body included in the standard text are judged to be consistent, and a judgment result of the named body is obtained; If the judgment result of the named body indicates consistency, prompt information is outputted to indicate that the semantics of the target text is similar to the semantics of the standard text.

2. The method according to claim 1, characterized in that The consistency judgment of the numbers included in the target text and the numbers included in the standard text to obtain the judgment result of the numbers includes: Based on a preset regular matching algorithm, numbers included in the target text are sequentially extracted according to the order of numbers appearing in the target text, and numbers included in the standard text are sequentially extracted according to the order of numbers appearing in the standard text; According to the extraction order, the numbers included in the target text are sequentially judged for consistency with the numbers included in the standard text to obtain the judgment result of the numbers; The result of the number determination is used to indicate whether the number included in the target text is the same as the number included in the standard text.

3. The method according to claim 2, characterized in that The step of performing consistency judgment on the named body included in the target text and the named body included in the standard text to obtain a judgment result of the named body includes: Based on a preset first language model, setting labels of entity objects included in the named body; Based on the tags of the entity objects, sequentially identifying the named entities included in the target text according to the order in which the tags appear in the target text, and sequentially identifying the named entities included in the standard text according to the order in which the tags appear in the standard text; According to the recognition order, the named entities included in the target text are sequentially judged for consistency with the named entities included in the standard text to obtain a judgment result of the named entities; The determination result of the named body is used to indicate whether the named body included in the target text is the same as the named body included in the standard text.

4. The method according to claim 3, characterized in that The calculating the similarity between the target text and the standard text includes: Based on a preset second language model, acquiring sentence vector features of the target text and sentence vector features of the standard text; Based on a preset cosine function algorithm, similarity is calculated for the sentence vector features of the target text and the sentence vector features of the standard text to obtain the similarity.

5. The method according to claim 4, characterized in that After acquiring the target text and the standard text similar to the target text, the method further includes: Based on a preset comparison algorithm, the target text and the standard text are judged for text content similarity to obtain a judgment result, wherein the judgment result cannot indicate whether the numbers and names included in the target text are consistent with the numbers and names included in the standard text; If the judgment result indicates that the text content of the target text is similar to that of the standard text, the step of calculating the similarity between the target text and the standard text is performed.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: If the determination result of the named body indicates inconsistency, a prompt message is outputted to indicate that the semantics of the target text is not similar to the semantics of the standard text.

7. A device for determining text semantic similarity, characterized in that: include: An acquisition module, used for acquiring a target text and a standard text similar to the target text; A calculation module, used for calculating the similarity between the target text and the standard text; A judgment module, configured to judge the consistency of the numbers included in the target text and the numbers included in the standard text if the similarity exceeds a preset similarity threshold, and obtain a judgment result of the numbers; The judgment module is further configured to, if the judgment result of the number indicates consistency, perform consistency judgment on the named body included in the target text and the named body included in the standard text to obtain a judgment result of the named body; The output module is used to output prompt information indicating that the semantics of the target text is similar to the semantics of the standard text if the judgment result of the named body indicates consistency.

8. An electronic device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor executes the method for determining text semantic similarity as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method for determining text semantic similarity as described in any one of claims 1 to 6.

10. A computer program product, characterized in that The invention comprises a computer program, which, when executed by a processor, implements the method for determining text semantic similarity as described in any one of claims 1 to 6.

Images