Multi-segment text similarity calculation method and device and electronic equipment

The method and device enhance the accuracy of project similarity comparison by using text weight words and synonyms to score and segment text fields, ensuring precise identification of similar projects in unified resource management platforms.

CN119226812AActive Publication Date: 2024-12-31WUHAN BIG PULP IND DEV CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202411300118.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-18
Publication Date
2024-12-31
Estimated Expiration
2044-09-18

AI Technical Summary

Technical Problem

Existing methods for project similarity comparison in unified resource management platforms rely heavily on keyword matching, which is inaccurate due to high dependence on the searcher's ability and fuzzy matching, leading to inconsistent results.

Method used

A method and device for calculating multi-segment text similarity that involves obtaining text weight words and scores, using synonyms, and performing word segmentation to determine total scores, enabling accurate similarity comparison between new and existing projects.

Benefits of technology

Improves the accuracy of project similarity determination by scoring and segmenting text fields, allowing for precise identification of similar projects through automated comparison.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119226812A_ABST
    Figure CN119226812A_ABST
Patent Text Reader

Abstract

The invention relates to a multi-segment text similarity calculation method and device and electronic equipment, and belongs to the technical field of natural language process.The method comprises the steps that a text weight word and a text weight word score of each field in a target software table are obtained; synonyms corresponding to the text weight words are obtained; obtaining a line total score of each line of data in the target software table based on the score of the text weight word; performing word segmentation processing on fields in the to-be-newly added project to obtain new project fields after word segmentation; determining a total score of the new item field based on the text weight word, the synonym and the score of the text weight word; and based on the line total score, the total score of the new project field and a preset value, obtaining the similarity between the to-be-newly added project and each line of data in the target software table. According to the method, the similar item retrieval precision is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a method, device and electronic device for calculating the similarity of multi-segment texts. Background Art

[0002] The unified resource management platform is an information resource management platform. Its core management goal is to coordinate the rational use of information resources, optimize and eliminate those with low resource efficiency, and control project applications to avoid repeated investment. The main functions of this system include the approval of new project applications. This approval function requires the use of similar projects in the past as a basis for the approver's decision, which leads to the need for project similarity comparison.

[0003] The current technical solution is to input the query keywords, and then use SQL's fuzzy query to query and match multiple text segments of project information. The multiple text segments include project name, project type, construction goals, user roles, function introduction and keywords. As long as a match is found, it is considered to be a similar project.

[0004] The existing method is an active query method, in which the approver matches the maintained project information with the keywords summarized by himself during the approval process. This method not only places too high demands on the searcher's summary ability, but also has problems with the accuracy of fuzzy matching. It often leads to embarrassing situations where one more word does not match, and one less word matches a lot, resulting in low accuracy of the matched items. Summary of the invention

[0005] In view of this, it is necessary to provide a method, device and electronic device for calculating multi-segment text similarity to solve the problem of low accuracy caused by matching items through SQL keyword queries in the prior art.

[0006] In order to solve the above problem, the present invention provides a method for calculating the similarity of multi-segment texts, comprising: Obtain the text weight word and text weight word score of each field in the target software table; Obtaining synonyms corresponding to the text weight words; Obtaining a row total score for each row of data in the target software table based on the score of the text weight word; Perform word segmentation on the fields in the project to be added to obtain the new project fields after word segmentation; determining a total score of the new item field based on the text weight words, the synonyms, and the scores of the text weight words; Based on the total score of the row, the total score of the new item field and the preset value, the similarity between the item to be added and each row of data in the target software table is obtained. In a possible implementation, the text weight word and the text weight word score of each field in the target software table are obtained, including: Preprocessing the text of the field to obtain a preprocessed field; Calculating the scores of the segmented words of the preprocessed field, and sorting the segmented words according to the scores to obtain sorted segmented words; A preset number of segmented words are selected from the sorted segmented words as text weighted words of the corresponding field, and the scores of the segmented words corresponding to the text weighted words are the text weighted word scores.

[0007] In a possible implementation, preprocessing the text of the field to obtain the preprocessed field includes: Perform word segmentation processing and stop word processing on the field to obtain a preprocessed field.

[0008] In a possible implementation, calculating the score of the word segmentation of the preprocessed field includes: The word segmentation score of the preprocessed field is calculated based on the BM25 algorithm.

[0009] In a possible implementation, the acquiring the synonym corresponding to the text weight word includes: Adding the target software business logic-specific synonyms to the preset standard synonym library to obtain the target synonym library; Acquire synonyms corresponding to the text weight words based on the target synonym library.

[0010] In a possible implementation, obtaining the total score of each row of data in the table based on the score of the text weight word includes: Adding each text weight word of the field to obtain first data; Determine a weight coefficient for each field according to the business logic of the target software; The weight coefficient of each field is multiplied by the first data to obtain second data, where the second data is the total score of each row of data in the table.

[0011] In a possible implementation, determining the total score of the new item field based on the text weight word, the synonym and the score of the text weight word includes: Perform word segmentation on the fields in the project to be added to obtain the new project fields after word segmentation; Compare each word in the new project field with the text weight words and synonyms of each field in the table to determine whether they are hits; When determining a hit, the score of the new item field is obtained based on the sum of the segmentation scores of the hits.

[0012] In a possible implementation, obtaining the similarity between the to-be-added item and each row of data in the target software table based on the row total score, the total score of the new item field, and a preset value includes: Determine the difference between the absolute value of the row total score and the total score of the new item field; The items corresponding to the difference values ​​being smaller than the preset value are determined as items that match the similarity with the items to be added.

[0013] On the other hand, the present invention also provides a device for calculating the similarity of multi-segment texts, comprising: The first word segmentation acquisition module is used to obtain the text weight word and the text weight word score of each field in the target software table; A synonym acquisition module, used to acquire synonyms corresponding to the text weight words; A first total score acquisition module, used to obtain a row total score for each row of data in the target software table based on the score of the text weight word; The second word segmentation acquisition module is used to perform word segmentation on the fields in the to-be-added project to obtain the new project fields after word segmentation; A second total score acquisition module, configured to determine a total score of the new item field based on the text weight word, the synonym and the score of the text weight word; The similar project acquisition module is used to obtain the similarity between the project to be added and each row of data in the target software table based on the total score of the row, the total score of the new project field and a preset value.

[0014] On the other hand, the present invention also provides an electronic device, including a memory and a processor, wherein: The memory is used to store programs; The processor is coupled to the memory and is used to execute the program stored in the memory to implement the steps in the method for calculating the similarity of multiple paragraphs of text described in any one of the above implementations.

[0015] The beneficial effects of the present invention are as follows: the present invention provides a method, device and electronic device for calculating the similarity of multi-segment texts, which first obtains the text weight words and text weight word scores of each field in the field of the target software table, and then obtains the synonyms corresponding to the text weight words. Further based on the score of the text weight word, the total score of each row of data in the target software table is obtained. Through the above steps, each field in the target software is scored to obtain the total score of each data, so that the newly added project can quickly determine the similarity project by comparing the total score, and further the field in the to-be-added project is segmented to obtain the new project field after segmentation, and the total score of the new project field is determined based on the text weight word, the synonym and the score of the text weight word. Finally, the similarity between the to-be-added and each row of data in the target software table is obtained based on the row total score, the total score of the new project field and the preset value. The present invention obtains the total score of each row of data by obtaining the text weight word of each field in the target software and the score of the corresponding text weight word, and by calculating the total score of the field in the newly added project, the total score of the field in the newly added project is compared with the total score of the original data of the target software, so as to determine the project similar to the newly added project, thereby improving the accuracy of the retrieval. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A flowchart of an embodiment of a method for calculating multi-segment text similarity provided by the present invention; Figure 2 For the present invention Figure 1 A schematic flow chart of an embodiment of S101; Figure 3 For the present invention Figure 1 A schematic flow chart of an embodiment of S102; Figure 4 For the present invention Figure 1 A schematic flow chart of an embodiment of S103; Figure 5 For the present invention Figure 1 A schematic flow chart of an embodiment of S105; Figure 6 A schematic flow chart of an embodiment of a device for calculating the similarity of multi-segment texts provided by the present invention; Figure 7 A schematic structural diagram of an embodiment of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0017] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0018] Before presenting the embodiments, the following terms are explained.

[0019] BM25 algorithm: It is a ranking algorithm widely used in the field of information retrieval and is part of the probabilistic information retrieval model. The core idea of ​​BM25 is based on the term frequency and inverse document frequency (TF-IDF) model, which further optimizes the term frequency processing so that it can better reflect the importance of words under different document lengths and term frequency distributions, but it has certain limitations in semantic understanding.

[0020] Synonym Forest: It is a Chinese vocabulary resource, similar to the English WordNet, used for synonymous relations and semantic classification of vocabulary. The Synonym Forest contains a large number of synonyms and near-synonyms, and organizes the vocabulary hierarchically according to semantic categories to build a structured semantic network. The Synonym Forest can be used to expand and replace words, and improve the performance of the model in natural language processing (NLP) tasks such as information retrieval, text classification, and sentiment analysis.

[0021] Multi-paragraph text similarity: Each entity is described by multiple paragraphs of text in different dimensions to describe its attribute characteristics. The similarity of entities is determined by comparing the multi-paragraph text descriptions of different entities.

[0022] The present invention provides a method, device and electronic device for calculating the similarity of multi-segment texts, which are respectively described below.

[0023] Figure 1 A schematic diagram of an embodiment of a method for calculating the similarity of multi-segment texts provided by the present invention is shown in FIG. Figure 1 As shown, the method for calculating the similarity of multi-segment texts includes: S101, obtaining text weight words and text weight word scores for each field in the target software table; S102, obtaining a synonym corresponding to the text weight word; S103, obtaining a row total score for each row of data in the target software table based on the score of the text weight word; S104, performing word segmentation processing on the fields in the to-be-added project to obtain the new project fields after word segmentation; S105, determining a total score of the new item field based on the text weight word, the synonym and the score of the text weight word; S106: Obtain the similarity between the to-be-added item and each row of data in the target software table based on the total score of the row, the total score of the new item field and a preset value.

[0024] Compared with the prior art, the present embodiment provides a method, device and electronic device for calculating the similarity of multi-segment texts. The method first obtains the text weight word and the text weight word score of each field in the field in the target software table, and then obtains the synonyms corresponding to the text weight word. Further based on the score of the text weight word, the total score of each row of data in the target software table is obtained. Through the above steps, each field in the target software is scored to obtain the total score of each data, so that the newly added project can quickly determine the similarity project by comparing the total score, and further the field in the to-be-added project is segmented to obtain the new project field after segmentation, and the total score of the new project field is determined based on the text weight word, the synonym and the score of the text weight word. Finally, the similarity between the to-be-added and each row of data in the target software table is obtained based on the row total score, the total score of the new project field and the preset value. The present invention obtains the total score of each row of data by obtaining the text weight word of each field in the target software and the score of the corresponding text weight word, and by calculating the total score of the field in the newly added project, the total score of the field in the newly added project is compared with the total score of the original data of the target software, so as to determine the project similar to the newly added project, thereby improving the accuracy of the retrieval.

[0025] It should be noted that the application scenario of the present invention is software that needs to query tables. The target software in this embodiment is an information resource management platform. The project information includes project name, project type, construction target, user role, function introduction and keywords. It can be understood that the method of the present invention is applicable to all software that needs to be queried.

[0026] In a specific embodiment of the present invention, when maintaining existing projects in the unified resource management platform, keywords are extracted and similarity is calculated for the projects, and the results are stored in the database. When a new project is applied for, the stored data is automatically compared, and the projects whose similarity difference meets the set value are identified as similar projects and their relationship is stored, so that similar projects can be automatically brought out for decision-making during approval. Specifically, the corresponding relationship between a project to be applied for and a project that has been maintained is as shown in Table 1: Table 1: Correspondence table between a project to be newly built and a project already maintained

[0027] Specifically, business-specific synonyms are added to the standard synonym library, such as government services and government-enterprise services, to obtain a target synonym library. It should be noted that the standard synonym library is a synonym library existing in the prior art.

[0028] In some embodiments of the present invention, in step S101, Figure 2 As shown, the step of obtaining the text weight word and the text weight word score of each field in the field in the target software table includes: S201, preprocessing the text of the field to obtain a preprocessed field; S202. Calculate the scores of the word segments of the preprocessed field, and sort the word segments according to the scores to obtain the sorted word segments. S203. Select a preset number of word segments from the sorted word segments as the text weight words for the corresponding field, and the scores of the word segments corresponding to the text weight words are the text weight word scores.

[0029] In some embodiments of the present invention, the preprocessing of the text of the field to obtain the preprocessed field includes: Perform word segmentation processing and stop word processing on the field to obtain the preprocessed field.

[0030] In a specific embodiment of the present invention, word segmentation and stop word (referring to removing common redundant words or punctuation marks or subject-predicate-object or those added by the business itself. Redundant words such as "of" and "and".) processing are performed on the text of each field. For example, the word segmentation of the project name text field is: [xx City, Digital店小二].

[0031] In some embodiments of the present invention, the calculation of the scores of the word segments of the preprocessed field includes: Calculate the scores of the word segments of the preprocessed field based on the BM25 algorithm.

[0032] In a specific embodiment of the present invention, the word segments of each field are scored through the BM25 algorithm, and the top x (x is a custom coefficient, and 3 is taken this time) words with the highest scores are selected, sorted, and used as the text weight words for the field. Since there are only two words after word segmentation this time, the words and their scores are expressed as [{"word": "Digital店小二", "bm25_score": 0.41097438921682977}, {"word": "xx City", "bm25_score": 0.28768207245178}].

[0033] Perform synonym processing on the selected maximum weight words, obtain synonyms, and save them together with the Score of this text. [{"word": "Digital店小二|Government-Enterprise Service|Online Government Affairs", "bm25_score": 0.41097438921682977}, {"word": "xx City|Jiangcheng|City-Level", "bm25_score": 0.28768207245178}]. Sum(TopN(Score(Project Name Keyword Array, Project Name))) = 0.41097438921682977 + 0.28768207245178 Perform weighted calculation on each row of data in each field of the existing projects in the unified resource management platform to obtain the total score. The formula for the total score is: Total Score = Sum(TopN(Score(project name keyword array, project name))*0.2 weight coefficient + Sum(TopN(Score(construction target keyword array, construction target)))*0.4 weight coefficient + Sum(TopN(Score(user role keyword array, user role)))*0.2 weight coefficient + Sum (TopN (function introduction keyword array, function introduction))) * 0.2 weight coefficient The value of the weight coefficient of each field is set according to the business logic.

[0034] In some embodiments of the present invention, in step S102, as Figure 3 As shown, the obtaining of the synonyms corresponding to the text weight words includes: S301, adding the target software business logic exclusive synonyms to the preset standard synonym library to obtain a target synonym library; S303: Acquire synonyms corresponding to the text weight words based on the target synonym library.

[0035] In some embodiments of the present invention, in step S103, Figure 4 As shown, the total score of each row of data in the table is obtained based on the score of the text weight word, including: S401, adding each text weight word of the field to obtain first data; S402, determining a weight coefficient of each field according to the business logic of the target software; S403: Multiply the weight coefficient of each field by the first data to obtain second data, where the second data is the total score of each row of data in the table.

[0036] In some embodiments of the present invention, in step S105, Figure 5 As shown, the method of determining the total score of the new item field based on the text weight word, the synonym and the score of the text weight word includes: S501, performing word segmentation processing on the fields in the project to be added to obtain new project fields after word segmentation; S502, comparing each word in the new item field with the text weight words and synonyms of each field in the table to determine whether there is a hit; S503: When a hit is determined, the score of the new item field is obtained based on the sum of the hit segmentation scores.

[0037] In some embodiments of the present invention, obtaining the similarity between the to-be-added item and each row of data in the target software table based on the row total score, the total score of the new item field, and a preset value includes: Determine the difference between the absolute value of the row total score and the total score of the new item field; The items corresponding to the difference values ​​being smaller than the preset value are determined as items that match the similarity with the items to be added.

[0038] In a specific embodiment of the present invention, the steps of processing the project to be reported include: Segment the fields in the project to be applied for, score the segmentation of each field using the BM25 algorithm, and select the top x (x is a custom coefficient, 3 is taken this time) words with the highest score and sort them as the text weight words of the project field to be applied for. For example, the data after segmentation is [{"word": "xx"},{"word": "business-friendly services"},{"word": "platform"}].

[0039] According to the project type and its synonyms, all matching maintained projects are screened for comparison and calculation. The calculation with a certain maintained project is as shown in Table 2: Table 2: Synonym comparison table

[0040] According to the above table, the word segmentation of the proposed new project is compared with the word segmentation of a maintained project in turn. When any word of the proposed new project hits the word segmentation of a maintained project, the score is calculated only once globally regardless of how many times the word hits. Finally, the sum of all hit scores is the similarity score of the text field. The specific explanation is as follows: Assuming that the word segment of the proposed new project 1 hits the word segment 1 and word segment 2 of the maintained project, the score will be calculated twice, and the other subsequent words do not need to be calculated. The sum of the score of word segment 1 and the score of word segment 2 is taken as the score of the word segment of the proposed new project 1; Assuming that the proposed new project 1 segmentation hits the maintained project segmentation 1, and the proposed new project 2 segmentation hits the maintained project segmentation 1, only one score is counted; All text fields of the new project to be applied for will be calculated, and the total score will be obtained by weighted calculation based on the Score total formula.

[0041] The total scores of both sides are compared by absolute value calculation, and the range of the difference is adjusted according to the business accuracy to confirm the similarity. In this embodiment, the absolute value difference less than 0.25 is considered to be similar.

[0042] In order to better implement a method for calculating the similarity of a multi-paragraph text in an embodiment of the present invention, based on a method for calculating the similarity of a multi-paragraph text, correspondingly, Figure 6 As shown, an embodiment of the present invention further provides a device for calculating the similarity of multiple paragraphs of text. The device 600 for calculating the similarity of multiple paragraphs of text includes: The first word segmentation acquisition module 601 is used to acquire the text weight word and the text weight word score of each field in the target software table; A synonym acquisition module 602 is used to acquire synonyms corresponding to the text weight words; A first total score acquisition module 603, used to obtain a row total score for each row of data in the target software table based on the score of the text weight word; The second word segmentation acquisition module 604 is used to perform word segmentation processing on the fields in the to-be-added item to obtain the new item fields after word segmentation; A second total score acquisition module 605, configured to determine a total score of the new item field based on the text weight word, the synonym and the score of the text weight word; The similar project acquisition module 606 is used to obtain the similarity between the project to be added and each row of data in the target software table based on the total score of the row, the total score of the new project field and a preset value.

[0043] The device 600 for calculating the similarity of multiple paragraphs of text provided in the above embodiment can implement the technical solution described in the above embodiment of a method for calculating the similarity of multiple paragraphs of text. The specific implementation principles of the above modules or units can refer to the corresponding contents in the above embodiment of a method for calculating the similarity of multiple paragraphs of text, which will not be repeated here.

[0044] like Figure 7 As shown, the present invention also provides an electronic device 700. The electronic device 700 includes a processor 701, a memory 702 and a display 703. Figure 7 Only some components of the electronic device 700 are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0045] In some embodiments, the processor 701 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run program codes or process data stored in the memory 702, such as a method for calculating multi-segment text similarity in the present invention.

[0046] In some embodiments, the processor 701 may be a single server or a server group. The server group may be centralized or distributed. In some embodiments, the processor 701 may be local or remote. In some embodiments, the processor 701 may be implemented in a cloud platform. In some embodiments, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an internal cloud, a multi-cloud, etc., or any combination thereof.

[0047] In some embodiments, the memory 702 may be an internal storage unit of the electronic device 700, such as a hard disk or memory of the electronic device 700. In other embodiments, the memory 702 may also be an external storage device of the electronic device 700, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., equipped on the electronic device 700.

[0048] Furthermore, the memory 702 may include both an internal storage unit of the electronic device 700 and an external storage device. The memory 702 is used to store application software installed in the electronic device 700 and various data.

[0049] In some embodiments, the display 703 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 703 is used to display information on the electronic device 700 and to display a visual user interface. The components 701-703 of the electronic device 700 communicate with each other via a system bus.

[0050] In one embodiment, when the processor 701 executes a multi-segment text similarity calculation program in the memory 702, the following steps may be implemented: Obtain the text weight word and text weight word score of each field in the target software table; Obtaining synonyms corresponding to the text weight words; Obtaining a row total score for each row of data in the target software table based on the score of the text weight word; Perform word segmentation on the fields in the project to be added to obtain the new project fields after word segmentation; determining a total score of the new item field based on the text weight words, the synonyms, and the scores of the text weight words; The similarity between the to-be-added project and each row of data in the target software table is obtained based on the total score of the row, the total score of the new project field and the preset value.

[0051] It should be understood that: when the processor 701 executes a multi-segment text similarity calculation program in the memory 702, in addition to the above functions, other functions can also be implemented. For details, please refer to the description of the corresponding method embodiment above.

[0052] Furthermore, the embodiment of the present invention does not specifically limit the type of the electronic device 700 mentioned, and the electronic device 700 may be a portable electronic device such as a mobile phone, a tablet computer, a personal digital assistant (PDA), a wearable device, a laptop computer, etc. Exemplary embodiments of portable electronic devices include but are not limited to portable electronic devices equipped with IOS, Android, Microsoft or other operating systems. The above-mentioned portable electronic devices may also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the present invention, the electronic device 700 may not be a portable electronic device, but a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0053] Those skilled in the art will appreciate that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0054] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A method for calculating the similarity of multi-segment texts, characterized in that: include: Obtain the text weight word and text weight word score of each field in the target software table; Obtaining synonyms corresponding to the text weight words; Obtaining a row total score for each row of data in the target software table based on the score of the text weight word; Perform word segmentation on the fields in the project to be added to obtain the new project fields after word segmentation; determining a total score of the new item field based on the text weight words, the synonyms, and the scores of the text weight words; The similarity between the to-be-added project and each row of data in the target software table is obtained based on the total score of the row, the total score of the new project field and the preset value.

2. The method for calculating the similarity of multi-segment text according to claim 1, characterized in that: The step of obtaining the text weight word and the text weight word score of each field in the field in the target software table includes: Preprocessing the text of the field to obtain a preprocessed field; Calculating the scores of the segmented words of the preprocessed field, and sorting the segmented words according to the scores to obtain sorted segmented words; A preset number of segmented words are selected from the sorted segmented words as text weighted words of the corresponding field, and the scores of the segmented words corresponding to the text weighted words are the text weighted word scores.

3. The method for calculating the similarity of multi-segment text according to claim 2, characterized in that: The preprocessing of the text of the field to obtain the preprocessed field includes: Perform word segmentation processing and stop word processing on the field to obtain a preprocessed field.

4. The method for calculating the similarity of multi-segment text according to claim 2, characterized in that: The calculating the word segmentation score of the preprocessed field includes: The word segmentation score of the preprocessed field is calculated based on the BM25 algorithm.

5. The method for calculating the similarity of multi-segment text according to claim 1, characterized in that: The obtaining of synonyms corresponding to the text weight words includes: Adding the target software business logic-specific synonyms to the preset standard synonym library to obtain the target synonym library; Acquire synonyms corresponding to the text weight words based on the target synonym library.

6. The method for calculating the similarity of multi-segment text according to claim 1, characterized in that: The total score of each row of data in the table is obtained based on the score of the text weight word, including: Adding each text weight word of the field to obtain first data; Determine a weight coefficient for each field according to the business logic of the target software; The weight coefficient of each field is multiplied by the first data to obtain second data, where the second data is the total score of each row of data in the table.

7. The method for calculating the similarity of multi-segment text according to claim 1, characterized in that: The determining the total score of the new item field based on the text weight word, the synonym and the score of the text weight word comprises: Perform word segmentation on the fields in the project to be added to obtain the new project fields after word segmentation; Compare each word in the new project field with the text weight words and synonyms of each field in the table to determine whether they are hits; When determining a hit, the score of the new item field is obtained based on the sum of the segmentation scores of the hits.

8. The method for calculating the similarity of multi-segment text according to claim 1, characterized in that: The similarity between the to-be-added item and each row of data in the target software table is obtained based on the total score of the row, the total score of the new item field and a preset value, including: Determine the difference between the absolute value of the row total score and the total score of the new item field; The items corresponding to the difference values ​​being smaller than the preset value are determined as items that match the similarity with the items to be added.

9. A device for calculating the similarity of multi-segment texts, characterized in that: include: The first word segmentation acquisition module is used to obtain the text weight word and the text weight word score of each field in the target software table; A synonym acquisition module, used to acquire synonyms corresponding to the text weight words; A first total score acquisition module, used to obtain a row total score for each row of data in the target software table based on the score of the text weight word; The second word segmentation acquisition module is used to perform word segmentation on the fields in the to-be-added project to obtain the new project fields after word segmentation; A second total score acquisition module, configured to determine a total score of the new item field based on the text weight word, the synonym and the score of the text weight word; The similar project acquisition module is used to obtain the similarity between the project to be added and each row of data in the target software table based on the total score of the row, the total score of the new project field and a preset value.

10. An electronic device, characterized in that: comprising a memory and a processor, wherein: The memory is used to store programs; The processor is coupled to the memory and is used to execute the program stored in the memory to implement the steps in the method for calculating the similarity of multiple paragraphs of text as described in any one of claims 1 to 8 above.

Citation Information

Patent Citations

  • Synonym analysis-based science and technology project declaration similarity detection method

    CN107122340A

  • Text comparison method based on synonymous related words

    CN107908622A

  • Intelligent patent similarity searching method and device based on semantic retrieval

    CN113821646A

  • Text content scoring method and device, equipment and medium

    CN117408243A

  • Multi-dimensional duplicate checking method based on neural network

    CN117573868A