Similar text determination method, electronic equipment and storage medium

Abstract

The invention discloses a similar text determination method, electronic equipment and a storage medium, and the method comprises the steps: obtaining a plurality of electronic files, and extracting the text content of the electronic files; obtaining a first high-frequency word set of the text content of the first electronic file, and obtaining a second high-frequency word set of the text content of a second electronic file in other electronic files; obtaining a longest duplicate word in the first high-frequency word set and the second high-frequency word set based on a longest common subsequence algorithm; if the word number of the longest duplicate word meets a preset numerical value condition, determining that the second electronic file is an associated electronic file of the first electronic file; obtaining a first hash value of each first word in the first high-frequency word set and a second hash value of each second word in a second high-frequency word set of the second electronic file; and determining a similarity judgment result of the first electronic file and the second electronic file based on the first hash value and the second hash value. And the accuracy of similar text determination is improved.

Description

Technical Field

[0001] The present application relates to the technical field of natural language processing, and in particular to a method for determining similar texts, an electronic device, and a storage medium. Background Art

[0002] With the development of information technology, software products can be used to analyze the text of electronic assignments submitted by students to confirm whether there is plagiarism in the electronic assignments.

[0003] Traditional text similarity analysis algorithms mainly include text similarity analysis based on deep learning. They use an open source Chinese similarity database to train models and use the trained models to identify text similarity. However, there is a large difference between the similarity Chinese database used for training and the text data contained in the actual submitted electronic assignments, resulting in a low accuracy rate in determining text similarity during actual use. Summary of the invention

[0004] The embodiments of the present application provide a method for determining similar texts, an electronic device, and a storage medium to improve the accuracy of determining similar texts.

[0005] In a first aspect, an embodiment of the present application provides a method for determining similar texts, the method comprising:

[0006] Obtain multiple electronic files and extract text content of the electronic files;

[0007] Acquire a first high-frequency word set of text content of a first electronic file, and acquire a second high-frequency word set of text content of a second electronic file among other electronic files, wherein the first electronic file is any electronic file among multiple electronic files, the other electronic files are the remaining electronic files among the multiple electronic files except the first electronic file, and the second electronic file is any electronic file among the other electronic files;

[0008] Obtain the longest repeated word in the first high-frequency word set and the second high-frequency word set based on the longest common subsequence algorithm;

[0009] If the number of words of the longest repeated word meets a preset numerical condition, determining that the second electronic file is an associated electronic file of the first electronic file;

[0010] Obtaining a first Hash value of each first word in the first high-frequency word set and a second Hash value of each second word in the second high-frequency word set of the second electronic file;

[0011] Based on the first hash value and the second hash value, a similarity determination result between the first electronic file and the second electronic file is determined.

[0012] In a second aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the steps of the above method.

[0013] In a third aspect, an embodiment of the present application provides a computer storage medium, which stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the steps of the above method.

[0014] In an embodiment of the present application, when it is determined that the second electronic file is an associated electronic file of the first electronic file by using the first high-frequency word set and the second high-frequency word set, the hash value is further used to determine the similarity judgment result between the first electronic file and the second electronic file. When it is determined that the second electronic file is an associated electronic file of the first electronic file, it is further determined whether the first electronic file and the second electronic file are similar, which improves the accuracy of the judgment. Therefore, this solution can be applied to actual online usage scenarios, and improves the practicality of determining whether electronic files are similar. Moreover, in this solution, the second electronic file excluded from the associated electronic file does not need to use the hash value to determine the similarity judgment result between it and the first electronic file, which reduces the resource loss during calculation and improves the speed of determining the similarity judgment result between electronic files. Based on this, the electronic device can obtain the similarity judgment results between multiple electronic files in real time. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0016] Figure 1 A schematic diagram of a scenario of a method for determining similar texts provided in an embodiment of the present application;

[0017] Figure 2 A flowchart of a method for determining similar texts provided in an embodiment of the present application;

[0018] Figure 3 A flowchart of a method for determining similar texts provided in an embodiment of the present application;

[0019] Figure 4 A flowchart of a method for determining similar texts provided in an embodiment of the present application;

[0020] Figure 5 A flowchart of a method for determining similar texts provided in an embodiment of the present application;

[0021] Figure 6 An example schematic diagram of a keyword analysis set provided in an embodiment of the present application;

[0022] Figure 7 An example schematic diagram of a similar file set provided in an embodiment of the present application;

[0023] Figure 8 An example schematic diagram of a high-frequency word analysis set provided in an embodiment of the present application;

[0024] Fig. 9 A flowchart of a method for determining similar texts provided in an embodiment of the present application;

[0025] Fig.10 An example schematic diagram of a keyword analysis set provided in an embodiment of the present application;

[0026] Fig.11 An example schematic diagram of a similar file set provided in an embodiment of the present application;

[0027] Fig.12 An example schematic diagram of a high-frequency word analysis set provided in an embodiment of the present application;

[0028] Fig.13 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0030] In the description of the present application, it should be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In the description of the present application, it should be noted that, unless otherwise clearly specified and limited, "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood in specific circumstances. In addition, in the description of the present application, unless otherwise specified, "multiple" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B, which can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are an "or" relationship.

[0031] See also Figure 1 , which is a schematic diagram of an application scenario of a method for determining similar texts provided in an embodiment of the present application.

[0032] The similar text determination method of the present application can be applied to scenarios where it is necessary to confirm whether the content of electronic files is similar, such as academic research, news reporting, advertising and copyright protection, and search engine optimization. In the embodiments of the present application, a multimedia teaching application scenario is used as an example. In multimedia teaching, the similar text determination method of the present application can accurately determine whether the homework or paper submitted by the student is copied or plagiarized.

[0033] The execution subject of the present application may be an electronic device capable of executing the method for determining similar texts in the present application. Upon receiving electronic file A submitted by student A, electronic file B submitted by student B, electronic file C submitted by student C, and electronic file D submitted by student D, the electronic device extracts the text content in electronic files A, B, C, and D, and determines the first electronic file among the received multiple electronic files (for example, determines that electronic file A is the first electronic file), and uses other files other than the first electronic file as other electronic files (electronic files B, C, and D) among the multiple electronic files, and selects any electronic file from the other electronic files as the second electronic file (for example, selects electronic file B as the second electronic file), obtains the first high-frequency word set of the first electronic file (electronic file A), and the second high-frequency word set of the second electronic file (electronic file B), and determines the longest repeated word in the first high-frequency word set and the second high-frequency word set, and when determining that the second electronic file is the associated electronic file of the first electronic file based on the longest repeated word, it is determined that the second electronic file and the first electronic file need to be further similarly determined. Afterwards, the hash value of each first word in the first high-frequency word set and the second hash value of each second word in the second high-frequency word set are obtained, and the similarity judgment result of the first electronic file and the second electronic file is determined based on the first hash value and the second hash value. The similarity judgment result of the first electronic file (electronic file A) and the second electronic file (electronic file B) can be: determining that the first electronic file (electronic file A) and the second electronic file (electronic file B) are similar; or, determining that the first electronic file (electronic file A) and the second electronic file (electronic file B) are not similar.

[0034] After determining the similarity judgment results of the first electronic file (electronic file A) and the second electronic file (electronic file B), the second electronic file is replaced, for example, electronic file C is used as the second electronic file, and the similarity judgment results of the first electronic file (electronic file A) and the current second electronic file (electronic file C) are obtained. Similarly, there are two possibilities for the similarity judgment results of electronic files A and electronic files C. According to the above method, electronic file D is used as the second electronic file, and the similarity judgment results of electronic files A and electronic files D are obtained.

[0035] Thus, the similarity judgment result between electronic file A and other electronic files when electronic file A is used as the first electronic file is obtained. Then, the first electronic file is replaced, for example, electronic file B is used as the first electronic file. Optionally, since the similarity judgment result between electronic file B and electronic file A has been determined, in the embodiment of the present application, the similarity judgment result between electronic file B and electronic file A may no longer be repeatedly determined, and the similarity judgment results between electronic file B and electronic file C and electronic file D may be determined, so that the similarity judgment result between electronic file B and other electronic files when electronic file B is used as the first electronic file can be obtained. According to the above method, the similarity judgment results between multiple electronic files can be obtained by replacing the first electronic file. Exemplarily, in the embodiment of the present application, the similarity judgment results of multiple electronic files finally obtained may be: electronic file A is similar to electronic file B and electronic file C, and electronic file C is similar to electronic file D.

[0036] In the traditional solution, the model is trained using an open source Chinese similarity database through a deep learning text similarity analysis algorithm, and then the trained model is used to identify whether the text content of the electronic files submitted by the students is similar. There is a large difference between the similarity Chinese library used for training and the actual text content submitted by the students, resulting in poor accuracy in analyzing whether there are similarities between electronic files, and it cannot be applied to actual online usage scenarios. In this application, when the first high-frequency word set and the second high-frequency word set are used to determine that the second electronic file is an associated electronic file of the first electronic file, the hash value of the text content of the first electronic file and the hash value of the text content of the second electronic file are used to determine whether the first electronic file and the second electronic file are similar. Using two different similarity confirmation methods to confirm whether the first electronic file and the second electronic file are similar improves the accuracy of the confirmation result. Therefore, this solution can be applied to actual online usage scenarios, improving the practicality of determining whether electronic files are similar. Furthermore, the present application does not need to use a hash value to determine the similarity judgment result between the second electronic file excluded from the associated electronic file and the first electronic file, thereby reducing resource loss during calculation and improving the speed of determining the similarity judgment result between electronic files. Based on this, the electronic device can obtain the similarity judgment results between multiple electronic files in real time.

[0037] In an embodiment of the present application, when it is determined that the second electronic file is an associated electronic file of the first electronic file by using the first high-frequency word set and the second high-frequency word set, the hash value is further used to determine the similarity judgment result between the first electronic file and the second electronic file. When it is determined that the second electronic file is an associated electronic file of the first electronic file, it is further determined whether the first electronic file and the second electronic file are similar, which improves the accuracy of the judgment. Therefore, this solution can be applied to actual online usage scenarios, and improves the practicality of determining whether electronic files are similar. And in this solution, the second electronic file excluded from the associated electronic file does not need to use the hash value to determine the similarity judgment result between it and the first electronic file, which reduces the resource loss during calculation and improves the speed of determining the similarity judgment result between electronic files. Based on this, the electronic device can obtain the similarity judgment results between multiple electronic files in real time.

[0038] based on Figure 1 The scene diagram shown below will be combined with Figure 2-Figure 12 , the method for determining similar text provided in the embodiment of the present application is introduced in detail.

[0039] Please refer to Figure 2 , Figure 2 A flow chart of a method for determining similar texts of this application is shown in FIG. Figure 2 As shown, the method for determining similar texts includes steps S101 to S106.

[0040] S101, obtaining multiple electronic files and extracting text contents of the electronic files;

[0041] The electronic device obtains multiple electronic files that need to be similarly judged, where the electronic files can be electronic files in word, pdf, jpg, and text formats. The electronic device can obtain electronic files sent by other devices online, or obtain electronic files by reading the storage path set by the user, or obtain multiple electronic files by other means. This application does not limit the method of obtaining electronic files.

[0042] After obtaining the multiple electronic files, the electronic device extracts the text contents in the multiple electronic files in different ways according to the forms of the electronic files.

[0043] For example, for electronic files in word or text format, electronic devices can directly obtain the text content in the electronic files; for electronic files in jpg format, electronic devices can use OCR (optical character recognition) technology to recognize characters in the image and convert them into text format to obtain the text content.

[0044] S102, obtaining a first high-frequency word set of text content of a first electronic file, and obtaining a second high-frequency word set of text content of a second electronic file among other electronic files, wherein the first electronic file is any electronic file among a plurality of electronic files, the other electronic files are the remaining electronic files among the plurality of electronic files except the first electronic file, and the second electronic file is any electronic file among the other electronic files;

[0045] In the embodiment of the present application, if a certain electronic file is determined to be the first electronic file among multiple electronic files, then the electronic files other than the first electronic file among the multiple electronic files are determined to be other electronic files, and any electronic file among the other electronic files is determined to be the second electronic file.

[0046] The first high-frequency word set is a set consisting of first high-frequency words; the first high-frequency words are first words that appear more frequently in the first words included in the text content of the first electronic file; and the first words are words included in the text content of the first electronic file.

[0047] The second high-frequency word set is a set consisting of second high-frequency words; the second high-frequency words are second words that appear more frequently in the second words included in the text content of the second electronic file; and the second words are words included in the text content of the second electronic file.

[0048] After the first electronic file and the second electronic file are determined, a high-frequency word set in the first electronic file and a high-frequency word set in the second electronic file are obtained.

[0049] Reference Figure 3 , Figure 3 A schematic diagram of a process of an embodiment of the present application is shown in FIG. Figure 3 As shown, obtaining the high-frequency word set in the first electronic file and the high-frequency word set in the second electronic file includes steps S1021 to S1024.

[0050] S1021, performing word segmentation processing on the text content of the first electronic file to obtain a first word set of the first electronic file, and counting the number of each first word in the first word set based on the text content of the first electronic file;

[0051] The electronic device performs word segmentation processing on the text content of the first electronic file according to a certain word segmentation processing method.

[0052] Commonly used word segmentation processing methods include: dictionary-based word segmentation and statistics-based word segmentation. Among them, the dictionary-based word segmentation method uses a pre-built dictionary for word segmentation. Match the text content of the first electronic file with the words in the dictionary, find the longest matching words as the result of the word segmentation processing, and obtain the first word set of the text content of the first electronic file; the statistical-based word segmentation method uses statistical models and algorithms to obtain word segmentation rules by observing a large amount of text training. Then, the text content of the first electronic file is segmented according to the trained word segmentation rules to obtain the first word set of the first electronic file. Exemplarily, in this embodiment, an open source word segmentation tool library, such as jieba, SnowNLP (Python), ANSJ (Java), LTP (C++), etc., can be used to segment the text content of the first electronic file to obtain the first word set of the first electronic file. It can be understood that the first word set consists of the first word contained in the text content of the first electronic file.

[0053] In this embodiment, after obtaining the first word set of the first electronic file, the first words included in the text content are determined based on the text content of the first electronic file, the number of first words is calculated, and then the number of each first word in the first word set is obtained.

[0054] For example, based on the above application scenario, for the first electronic file (electronic file A), using jieeba, the first word contained in the text content of the electronic file can be: "technology", "technology", "artificial intelligence", "striving", "industrial reform", "language", "I", "life", "creation", "beauty", "learning", "effort". It is determined that the first word set can be {"technology", "technology", "artificial intelligence", "striving", "industrial reform", "language", "I", "life", "creation", "beauty", "learning", "effort"}.

[0055] Optionally, in this embodiment, in the process of performing word segmentation processing on the text content of the first electronic file to obtain the first word set, the set formed by the words obtained by directly analyzing the text content of the first electronic file can be used as the third word set, and after removing the words belonging to the target part of speech in the third word set, it is confirmed that the first word set is obtained.

[0056] The target part of speech may be a pronoun (eg, you, me, he) or a punctuation mark, which does not affect the result of determining the similarity between electronic files.

[0057] Referring to the above application scenario as an example, when the text content of the first electronic file is segmented, the words ""science and technology", "technology", "artificial intelligence", "endeavor", "industrial reform", "language", "I", "life", "creation", "beauty", "learning", and "effort" are obtained. The formed {"science and technology", "technology", "artificial intelligence", "endeavor", "industrial reform", "language", "I", "life", "creation", "beauty", "learning", and "effort"} are regarded as the third word set. After removing the pronouns, the first word set is {"science and technology", "technology", "artificial intelligence", "endeavor", "industrial reform", "language", "life", "creation", "beauty", "learning", and "effort"}.

[0058] In this embodiment, by removing words belonging to the target part of speech in the text content and then performing similarity confirmation between electronic files, the electronic device is prevented from confirming words that do not affect the similarity confirmation result, thereby improving the efficiency of the electronic device in confirming whether the electronic files are similar.

[0059] S1022, performing word segmentation processing on the text content of the second electronic file in the other electronic files to obtain a second word set of the second electronic file, and counting the number of each second word in the second word set based on the text content of the second electronic file;

[0060] In this embodiment, the way of performing word segmentation processing on the text content of the second electronic file is similar to the way of performing word segmentation processing on the text content of the first electronic file, and this application will not describe it in detail.

[0061] After obtaining the second word set of the second electronic file, the second words included in the text content are determined based on the text content of the second electronic file, the number of the second words is calculated, and then the number of each second word in the second word set is obtained.

[0062] Similarly, based on the above application scenario, for the second electronic file (electronic file B), using jieeba, the second words contained in the text content of the electronic file can be: "technology", "technology", "artificial intelligence", "artificial intelligence", "industry", "industry", "youth", "life", "creation", "beauty", "learning", "effort". It is determined that the first word set can be {"technology", "technology", "artificial intelligence", "artificial intelligence", "industry", "industry", "youth", "life", "creation", "beauty", "learning", "effort"}.

[0063] Optionally, in this embodiment, in the process of performing word segmentation processing on the text content of the second electronic file to obtain the second word set, the set formed by the words obtained by directly analyzing the text content of the second electronic file can be used as the fourth word set, and after removing the words belonging to the target part of speech in the fourth word set, it is confirmed that the second word set is obtained.

[0064] Referring to the above application scenario as an example, when the text content of the first electronic file is segmented, the words "technology", "technology", "artificial intelligence", "artificial intelligence", "industry", "industry", "industry", "youth", "life", "creation", "beauty", "learning", and "effort" are obtained. The set formed by {"technology", "technology", "artificial intelligence", "artificial intelligence", "industry", "industry", "youth", "life", "creation", "beauty", "learning", and "effort"} is regarded as the third word set. Since there are no words of the target part of speech, the first word set obtained is directly confirmed as {"technology", "technology", "artificial intelligence", "artificial intelligence", "industry", "industry", "youth", "life", "creation", "beauty", "learning", and "effort"}.

[0065] In this embodiment, by removing words belonging to the target part of speech in the text content and then performing similarity confirmation between electronic files, the electronic device is prevented from confirming words that do not affect the similarity confirmation result, thereby improving the efficiency of the electronic device in confirming whether the electronic files are similar.

[0066] S1023, based on the number of each first word, obtaining a preset number of first high-frequency words with the largest number in the first word set, and generating a first high-frequency word set based on the preset number of first high-frequency words;

[0067] In this embodiment, the first words in the first word set are sorted in descending order according to the number of each first word in the first word set. For example, referring to the above embodiment, the result of sorting the first words in descending order is: "technology", "artificial intelligence", "industrial reform", "life", "creation", "beauty", "learning", "effort", among which, when the preset number is 3, the preset number of first high-frequency words is determined to be: "technology", "artificial intelligence", "industrial reform", and the first high-frequency word set is further determined to be {"technology", "artificial intelligence", "industrial reform"}.

[0068] S1024, based on the number of each second word, obtain a preset number of second high-frequency words with the largest number in the second word set, and generate a second high-frequency word set based on the preset number of second high-frequency words; the preset number is determined by the median number of words contained in the text content of each electronic file in multiple electronic files.

[0069] In this embodiment, the second words in the second word set are sorted in descending order according to the number of each second word in the second word set. For example, referring to the above embodiment, the result of sorting the second person words in descending order is: "industry", "technology", "artificial intelligence", "youth", "life", "creation", "beauty", "learning", "effort", among which, when the preset number is 3, the preset number of second high-frequency words is determined to be: "industry", "technology", "artificial intelligence", and the second high-frequency word set is further determined to be {"industry", "technology", "artificial intelligence"}.

[0070] The preset number is determined by the median number of words contained in the text content of each electronic file among multiple electronic files.

[0071] For example, among multiple electronic files A, B, C, and D, it is determined that the number of words contained in the text content of electronic file A is 10, the number of words contained in the text content of electronic file B is 6, the number of words contained in the text content of electronic file C is 12, and the number of words contained in the text content of electronic file D is 12. Then, it is determined that there are two medians of the number of words contained in the text content of the electronic files, which are 10 and 12 respectively. Then, the average of the medians is obtained as the median of the multiple electronic files. It is calculated that the median of the words contained in the text content of each electronic file in the multiple electronic files A, B, C, and D is 11, and the preset number is determined to be 11.

[0072] It can be understood that in the embodiment of the present application, when determining the preset number, when the number of electronic files is an odd number, after obtaining the number of words contained in the text content of the electronic file, the number of words is sorted according to the value of the number of words, and the median of the number of words contained in the text content of each electronic file can be obtained; when the number of electronic files is an even number, after obtaining the number of words contained in the text content of the electronic file, the number of words is sorted according to the value of the number of words to obtain two medians, and the average of the two medians is used as the median of the number of words contained in the text content of each electronic file.

[0073] It can be understood that, in the process of obtaining a preset number of first high-frequency words in the first word set as the high-frequency word set and obtaining a preset number of second high-frequency words in the second word set as the sum of the high-frequency word set, when the number of first words in the first electronic file or the second electronic file is less than the median, the smaller value of the number of words in the first word set or the second word set is used as the median of the first electronic file and the second electronic file.

[0074] In this embodiment, after word segmentation processing is performed on the first electronic file and the second electronic file respectively, the first word of the text content of the first electronic file and the second word of the text content of the second electronic file are obtained, and a preset number of first high-frequency words with the largest number are obtained from the first words to generate a first high-frequency word set, and a preset number of second high-frequency words with the largest number are obtained from the second words to generate a second high-frequency word set, and the preset number of first high-frequency words and second high-frequency words with the largest number in the first electronic file and the second electronic file are obtained respectively, providing a basis for accurately determining whether the second electronic file is an associated electronic file of the first electronic file.

[0075] S103, obtaining the longest repeated word in the first high-frequency word set and the second high-frequency word set based on the longest common subsequence algorithm;

[0076] The longest common subsequence algorithm is to find the words with the largest number of co-existence in the first high-frequency word set and the second high-frequency word set.

[0077] Optionally, in an embodiment of the present application, firstly, the first words in the first high-frequency word set are sorted in lexicographic order to obtain a first-order high-frequency word set, and the second words in the second high-frequency word set are sorted in lexicographic order to obtain a second-order high-frequency word set. After obtaining the first-order high-frequency word set and the second-order high-frequency word set, the longest repeated word of the first-order high-frequency word set and the second-order high-frequency word set is obtained based on the longest common subsequence algorithm.

[0078] The dictionary order refers to arranging a set of strings according to the size of the ASCII code value of the first different character. If the first different characters are the same, the second different character is compared, and so on until a certain character is different, and the strings are arranged according to the size of the ASCII code value of the character. The first-order high-frequency word set is the set obtained by sorting the first high-frequency word set in dictionary order; the second-order high-frequency word set is the set obtained by sorting the second high-frequency word set in dictionary order.

[0079] Referring to the above description, when the first high-frequency word set is {"technology", "artificial intelligence", "industrial reform"} and the second high-frequency word set is {"industry", "technology", "artificial intelligence"}, after sorting the first high-frequency word set and the second high-frequency word set according to dictionary order, the first-order high-frequency word set is {"industrial reform", "artificial intelligence", "technology"}, and the second-order high-frequency word set is {"artificial intelligence", "industry", "technology"}.

[0080] It is found that the longest repeated words in the first high-frequency word set and the second high-frequency word set are "artificial intelligence" and technology.

[0081] S104, if the number of words of the longest repeated word meets a preset numerical condition, determining that the second electronic file is an associated electronic file of the first electronic file;

[0082] The preset numerical condition may be a condition for determining whether the second electronic file is an associated electronic file of the first electronic file, and the preset numerical condition may be further provided with a preset numerical value. If the number of words of the longest repeated word is greater than or equal to the preset numerical value, it is determined that the preset numerical condition is satisfied, and then the second electronic file is determined to be an associated electronic file of the first electronic file, wherein the preset numerical value is determined by the median of the number of words contained in the text content of each electronic file among multiple electronic files. Optionally, the preset numerical value may be half or one third of the median, and the specific value may be set by the user. For example, if the user requires strict screening, the preset numerical value is set to one fifth of the median, and if strict screening is not required, the preset numerical value may be set to one half of the median.

[0083] Referring to the above description, the median obtained in this application is 11, and the user sets the preset value to one fifth of the median, and the preset value obtained after rounding is 2. In this application, the longest repeated words are "artificial intelligence" and "technology", and the number of words is 2, which is equal to the preset value. It is determined that the number of words of the longest repeated word meets the preset value condition, and then the second electronic file is determined to be an associated electronic file of the first electronic file.

[0084] In this embodiment, the first words in the first high-frequency word set and the first words in the second high-frequency word set are sorted in lexicographic order to obtain a first-order high-frequency word set and a second-order high-frequency word set, respectively, and the number of words of the longest repeated words in the first-order high-frequency word set and the second-order high-frequency word set is obtained according to the longest common sequence algorithm, and whether the second electronic file is an associated file of the first electronic file is determined according to the number of words, thereby improving the accuracy of determining the associated files of the first electronic file.

[0085] S105, obtaining a first Hash value of each first word in the first high-frequency word set and a second Hash value of each second word in the second high-frequency word set of the second electronic file;

[0086] The first Hash value is a Hash value of a first word in a first high-frequency word set; and the second Hash value is a Hash value of a second word in a second high-frequency word set.

[0087] In this embodiment, the hash value of each word in the first high-frequency word set and the second high-frequency word set can be obtained by a common string encoding algorithm. Optionally, the string encoding algorithm can be a message digest algorithm (Message Digest Algorithm 5, MD 5) or a secure hash algorithm (Secure Hash Algorithm, SHA-1) to perform a hash operation to obtain the hash value of each word in the first high-frequency word set and the second high-frequency word set.

[0088] For example, referring to the above description, in this embodiment, the hash values ​​of the first words in the first high-frequency word set {"technology", "artificial intelligence", "industrial reform"} can be "10000", 11001", and 11011", respectively; the hash values ​​of the second words in the second high-frequency word set {"industry", "technology", and "artificial intelligence"} can be "10000", 11001", and 11011", respectively.

[0089] S106: Determine a similarity judgment result between the first electronic file and the second electronic file based on the first hash value and the second hash value.

[0090] The similarity judgment result includes two judgment results: the first electronic file and the second electronic file are similar, or the first electronic file and the second electronic file are not similar.

[0091] Optionally, in an embodiment of the present application, a first weight of each first word in the first high-frequency word set is obtained, and a second weight of each second word in the second high-frequency word set is obtained, a first product of the first weight of each first word and the first hash value of each first word is determined, and a second product of the second weight of each second word and the second hash value of each second word is determined, the first products of each first word are added up in a list to obtain a first simhash value of the first electronic file, and the second products of each second word are added up in a list to obtain a second simhash value of the second electronic file, and then according to the first simhash value and the second simhash value, when it is determined that the Hamming distance between the first electronic file and the second electronic file is less than a preset distance, the first electronic file is judged to be similar to the second electronic file; when it is determined that the Hamming distance between the first electronic file and the second electronic file is greater than a preset distance, the first electronic file is judged to be dissimilar to the second electronic file. The preset distance can be set by the user.

[0092] In this embodiment, when the Hamming distance between the first electronic file and the second electronic file is determined according to the first simhash value and the second simhash value, the number of different values ​​at the same position of the first simhash value and the second simhash value is obtained.

[0093] Referring to the above description, the hash values ​​of the first words in the first high-frequency word set {"technology", "artificial intelligence", "industrial reform"} can be "10000", "10001", and "11010" respectively; the hash values ​​of the second words in the second high-frequency word set {"industry", "technology", and "artificial intelligence"} are "10111", "10000", and "10001" respectively.

[0094] The first weights of the first words included in the first high-frequency word set are obtained to be 2 / 11, 1 / 11, and 1 / 11 respectively; the second weights of the first words included in the second high-frequency word set are obtained to be 3 / 13, 2 / 13, and 2 / 13 respectively.

[0095] Optionally, in this embodiment, since the first weight of the first word in the first high-frequency word set and the second weight of the second word in the second high-frequency word set are relatively small, in order to facilitate calculation, the obtained first weight and second weight can be uniformly amplified to determine the final first weight of the first word and the second weight of the second word. For example, after amplifying them 100 times, the obtained first weights of the first word are: 18, 9, 9; the second weights of the second word are: 23, 15, 15.

[0096] In this embodiment, the first products of the first weights of the first words and the first hash values ​​of the first words are (18)(-18)(-18)(-18), 9-9-9-99, 99-99-9, where (18) and (-18) represent a numerical value of the first product obtained after multiplication. Similarly, the second products of the second weights of the second words and the second hash values ​​of the second words are (23)(-23)(23)(23)(23), (15)(-15)(-15)(-15)(-15), (15)(-15)(-15)(-15)(15), where (23), (-23), (15), and (-15) represent a numerical value of the second product obtained after multiplication.

[0097] The first product is accumulated to obtain a first simhash value of the first electronic file. The first simhash value is (36)(-18)(-36)(-18)(-18), and the second simhash is (53)(-53)7(-7)(23). When the Hamming distance is obtained according to the first simhash value and the second simhash value, the positive numbers in the first simhash value and the second simhash value are set to 0 and the negative numbers are set to 1, so that the first simhash value is 10000 and the second simhash value is 10101. The number of different values ​​at the same position in the first simhash value and the second simhash value is 2, so it is determined that the Hamming distance between the first electronic file and the second electronic file is 2, which is less than the preset distance (3), so it is determined that the first electronic file and the second electronic file are similar.

[0098] In this embodiment, the Hamming distance between the first electronic file and the second electronic file is determined according to the first simhash value and the second simhash value, and then whether the two electronic files are similar is determined, thereby achieving accurate determination of the similarity between the electronic files.

[0099] Optionally, in this embodiment, obtaining the first weight of each first word in the first high-frequency word set can be: obtaining the first weight of each first word in the first word set based on a first ratio of the number of each first word in the first word set of the first electronic file to the total number of each first word, searching the first word set for the first word that exists in the first high-frequency word set, and determining that the weight of each first word corresponds to the first weight of each first word in the first high-frequency word set.

[0100] Optionally, after removing the first words belonging to the target part of speech in the third word set to obtain the first word set, the number of words of each first word in the first word set is obtained, and the first weight of each first word is obtained according to the ratio of the number of words of each first word in the first set to the total number of each first word, wherein the total number of each first word is obtained by accumulating the number of each first word.

[0101] For example, referring to the above description, after eliminating the first word belonging to the target part of speech in the third word set to obtain the first word set {"technology", "technology", "artificial intelligence", "artificial intelligence", "industrial reform", "industrial reform", "life", "creation", "beauty", "learning", "effort"}, there are a total of 11 words in the first word set. For the first word "technology", the number of words in the first word set is 2, so the first weight of the first word "technology" is 2 / 11, for the first word "artificial intelligence" the number of words in the first word set is 2, then the first weight of the first word "technology" is 2 / 11,..., for the first word "effort", the number of words in the first word set is 1, then the first weight of the first word "effort" is 1 / 11.

[0102] After obtaining the first high-frequency word, according to the words included in the first high-frequency word set {"technology", "artificial intelligence", "industrial reform"}, the first words existing in the first high-frequency word set are searched in the first word set, and the weight of each first word is determined to correspond to the first weight of each first word in the first high-frequency word set. For example, the first words found in the first word set that exist in the first high-frequency word set are "technology", "artificial intelligence", and "industrial reform", and their first weights are 2 / 11, 2 / 11, and 1 / 11 respectively, then the first weights of each first word in the first high-frequency word set are determined to be 2 / 11, 2 / 11, and 1 / 11 respectively.

[0103] Obtaining the second weight of each second word in the second high-frequency word set may be: obtaining the second weight of each second word in the second word set according to a second ratio of the number of each second word in the second word set of the second electronic file to the total number of each second word.

[0104] The second word set is searched for second words in the second high-frequency word set, and the weight of each second word is determined to correspond to the second weight of each second word in the first high-frequency word set.

[0105] It is understandable that the method of determining the second weight of each second word in the second high-frequency word set is similar to the method of determining the first weight of each first word in the first high-frequency word set described above, which will not be described in detail herein.

[0106] In an embodiment of the present application, the first weight of each first word in the first high-frequency word set is determined by the number of each first word in the first word set, and the second weight of each second word in the second high-frequency word set is determined by the number of each second word in the second word set. This achieves accurate determination of the first weight of each first word in the first high-frequency word set and the second weight of each second word in the second high-frequency word set, providing a basis for subsequent determination of the similarity between electronic files.

[0107] In an embodiment of the present application, when it is determined that the second electronic file is an associated electronic file of the first electronic file by using the first high-frequency word set and the second high-frequency word set, the hash value is further used to determine the similarity judgment result between the first electronic file and the second electronic file. When it is determined that the second electronic file is an associated electronic file of the first electronic file, it is further determined whether the first electronic file and the second electronic file are similar, which improves the accuracy of the judgment. Therefore, this solution can be applied to actual online usage scenarios, and improves the practicality of determining whether electronic files are similar. And in this solution, the second electronic file excluded from the associated electronic file does not need to use the hash value to determine the similarity judgment result between it and the first electronic file, which reduces the resource loss during calculation and improves the speed of determining the similarity judgment result between electronic files. Based on this, the electronic device can obtain the similarity judgment results between multiple electronic files in real time.

[0108] Reference Figure 4 , Figure 4 The method for determining similar texts of the present application is a flowchart diagram. The method further includes steps S107 to S111.

[0109] S107, obtaining a preset keyword set and a third weight of each third word in the keyword set;

[0110] S108, obtaining each first word in a first word set of the first electronic file and a first weight of each first word;

[0111] S109, determining, among the first words, a first word that matches the third word in the keyword set as a target word, and determining a first weight of the first word as a target weight of the target word;

[0112] S110, when there are multiple target words, determining the average of the products of the target weights of the target words and the corresponding third weights;

[0113] S111, determining a keyword analysis result of the first electronic file according to the mean value.

[0114] The keyword set is a set formed by words that score key points of the text content of the electronic document, and the additional words in the keyword set are the third words.

[0115] In the present application, the third word in the keyword set and the weight of the third word in the keyword set can be pre-set. For example, the keyword set is pre-set to {"industrial reform", "technology"}, where the weight of the third word "industrial reform" in the keyword set is set to 40% (third weight), and the weight of the third word "technology" in the keyword set is set to 60% (third weight).

[0116] When it is determined that "technology" and "industrial reform" are included in the first words contained in the high-frequency word set of the first electronic file, they are determined as target words, and the target weights corresponding to the target words are obtained as 2 / 11 and 1 / 11 respectively. If there are multiple target words, the average of the product of the target weight of the target word and the corresponding third weight is determined.

[0117] For example, when performing keyword analysis, the average value of the first electronic file is calculated to be 0.07 ((2 / 11*60%+1 / 11*40%) / 2).

[0118] In the present application, after obtaining the average of the product of the target weight of the target word and the corresponding third weight, the keyword analysis result of the first electronic file can be determined according to the average, thereby realizing keyword analysis of the first electronic file.

[0119] Reference Figure 5 , Figure 5 The method for determining similar texts of the present application is a flowchart diagram. The method further includes steps S112 to S115.

[0120] S112, obtaining words in the text contents of a plurality of electronic files;

[0121] S113, when it is determined that the word matches a third word in the preset keyword set, obtaining an identifier of the electronic file to which the word belongs, and adding the identifier to the keyword analysis set;

[0122] S114, when it is determined that the first electronic file and the second electronic file are similar, obtaining an identifier of the first electronic file and an identifier of the second electronic file, and adding the identifier of the first electronic file and the identifier of the second electronic file to a similar file set;

[0123] S115, obtaining the weight of the words, sorting the words in descending order according to the weight, obtaining the words in a preset sorting range, and adding the words to the high-frequency word analysis set.

[0124] In this embodiment, the text contents of multiple electronic files may be generally analyzed.

[0125] After obtaining words from the text contents of multiple electronic files, determine whether the obtained words are consistent with the third words in the preset keyword set. If there are consistent words, it means that there are words in the text contents of multiple electronic files that match the third words in the keyword set. Then, obtain the identifier of the electronic file to which the word belongs, and add the identifier of the electronic file to the keyword analysis set.

[0126] like Figure 6As shown, for example, multiple electronic files (electronic file A, electronic file B, electronic file C) are obtained. And the keyword set is {"technology", "reform", "life"}, after determining that the word "life" exists in electronic file A and electronic file B, which matches the third keyword "life", the identifiers of electronic file A and electronic file B, for example, the identifier "student 1 homework" of electronic file A and the identifier "student 2 homework" of electronic file B are added to the keyword analysis set.

[0127] like Figure 7 When it is determined that the first electronic file (electronic file A) and the second electronic file (electronic file B) are similar, the file identifiers of the first electronic file and the second electronic file, for example, "student 1 homework" and "student 2 homework", are added to the similar file set.

[0128] Reference Figure 8 ,After obtaining the words of the text content of the electronic file, the weight of each word is obtained, the weight is sorted in descending order, and the words sorted within the preset sorting range are added to the high frequency word analysis set. For example, in electronic file A, "technology" appears 5 times, "productivity" appears 3 times, "power" appears 1 time, and "life" appears 1 time. According to the weights, the weights of the words in electronic file A are ranked from large to small as "technology" (1 / 2), "productivity" (3 / 10), "power" (1 / 10), and "life" (1 / 10); in electronic file B, "technology" appears 3 times, "technology" appears 2 times, and "power" appears 1 time. According to the weights, the weights of the words in electronic file B are ranked from large to small as "technology" (1 / 2), "technology" (1 / 3), and "power" (1 / 6). The ranking results of the words in all electronic files in descending order are: "technology" (1 / 2), "technology" (1 / 3), "productivity" (3 / 10), "power" (1 / 6), and "life" (1 / 10). If the preset sorting range is 2, the words "technology" and "technology" are added to the high-frequency word analysis set.

[0129] In this embodiment, by adding different analysis results to corresponding analysis sets, classification analysis of the analysis results is achieved.

[0130] Reference Fig. 9 , Fig. 9 The method for determining similar texts of the present application is a flowchart diagram. The method further includes steps S116 to S119.

[0131] S116, obtaining a first word of the text content of the first electronic file;

[0132] S117, when it is determined that the first word matches a third word in the preset keyword set, the first word is added to a keyword analysis set corresponding to the identifier of the first electronic file;

[0133] S118, when it is determined that the first electronic file and the second electronic file are similar, obtaining an identifier of the first electronic file and an identifier of the second electronic file, and adding the identifier of the second electronic file to a similar file set corresponding to the identifier of the first electronic file;

[0134] S119, obtaining the weight of the first word, sorting the words in descending order according to the weight, obtaining words in a preset sorting range, and adding the words to a high-frequency word analysis set corresponding to the identifier of the first electronic file.

[0135] In this embodiment, the text content of a single electronic file can be analyzed. After obtaining the first word of the text content of the first electronic file, when it is determined that the first word matches the third word in the preset keyword set, the first word is added to the keyword analysis set corresponding to the identifier of the first electronic file. Fig.10 As shown, for example, the keyword set is {"technology", "reform", "life"}, and when it is determined that the words "life" and "reform" exist in electronic file A, which match the third keywords "life" and "reform", the first words "life" and "reform" are added to the keyword analysis set corresponding to the identifier of the first electronic file (student A's homework).

[0136] When it is determined that the first electronic file is similar to the second electronic file, the identifier of the second electronic file is added to the similar file set corresponding to the identifier of the first electronic file. Fig.11 As shown, when it is determined that the first electronic file and the second electronic file are similar, the file identifier of the second electronic file ("student B's homework") is added to the similar file set corresponding to the identifier of the first electronic file ("student A's homework").

[0137] Obtain weights of first words in the first electronic file, sort the first words in descending order according to the weights, obtain first words in a preset sorting range, and add the first words to a high-frequency word analysis set corresponding to the identifier of the first electronic file.

[0138] For example, refer to Fig.12, after obtaining that "technology" appears 5 times, "productivity" appears 3 times, "power" appears 1 time, and "life" appears 1 time in electronic file A, the weights of each word in electronic file A are sorted from large to small according to the weights: "technology" (1 / 2), "productivity" (3 / 10), "power" (1 / 10), and "life" (1 / 10). The preset sorting range is 2, and the first words "technology" and "productivity" are added to the high-frequency word analysis set corresponding to the identifier of electronic file A (student A homework).

[0139] In this embodiment, a single electronic file is analyzed, and the analysis result is added to a corresponding analysis set, thereby achieving classification analysis of multiple single electronic files.

[0140] The present application also provides a computer storage medium that can store multiple program instructions, which are suitable for being loaded and executed by a processor as described above. Figure 1-Figure 12 The method steps of the embodiment shown in the figure can be found in the specific implementation process. Figure 1-Figure 12 The specific description of the illustrated embodiment will not be repeated here.

[0141] See also Fig.13 , is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Fig.13 As shown, the electronic device 1000 may include: at least one processor 1001, such as a CPU, at least one network interface 1004, a user interface 1003, a memory 1005, and at least one communication bus 1002. The communication bus 1002 is used to realize the connection and communication between these components. The network interface 1004 may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 1005 may optionally also be at least one storage device located away from the aforementioned processor 1001. As shown in FIG. Fig.13 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, an input and output interface module, and a similar text determination program.

[0142] exist Fig.13 In the electronic device 1000 shown, the user interface 1003 is mainly used to provide an input interface for the user and obtain data input by the user.

[0143] In one embodiment, the processor 1001 may be used to call a similar text determination application stored in the memory 1005, and specifically perform the following operations:

[0144] Obtain multiple electronic files and extract text content of the electronic files;

[0145] Acquire a first high-frequency word set of text content of a first electronic file, and acquire a second high-frequency word set of text content of a second electronic file among other electronic files, wherein the first electronic file is any electronic file among multiple electronic files, the other electronic files are the remaining electronic files among the multiple electronic files except the first electronic file, and the second electronic file is any electronic file among the other electronic files;

[0146] Obtain the longest repeated word in the first high-frequency word set and the second high-frequency word set based on the longest common subsequence algorithm;

[0147] If the number of words of the longest repeated word meets a preset numerical condition, determining that the second electronic file is an associated electronic file of the first electronic file;

[0148] Obtaining a first Hash value of each first word in the first high-frequency word set and a second Hash value of each second word in the second high-frequency word set of the second electronic file;

[0149] Based on the first hash value and the second hash value, a similarity determination result between the first electronic file and the second electronic file is determined.

[0150] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0151] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A method for determining similar texts, characterized in that: The method comprises: Acquire multiple electronic files and extract text contents of the electronic files; Acquire a first high-frequency word set of text content of a first electronic file, and acquire a second high-frequency word set of text content of a second electronic file among other electronic files, wherein the first electronic file is any electronic file among the multiple electronic files, the other electronic files are the remaining electronic files among the multiple electronic files except the first electronic file, and the second electronic file is any electronic file among the other electronic files; Obtaining the longest repeated word in the first high-frequency word set and the second high-frequency word set based on the longest common subsequence algorithm; If the number of words in the longest repeated word meets a preset numerical condition, determining that the second electronic file is an associated electronic file of the first electronic file; Obtaining a first Hash value of each first word in the first high-frequency word set and a second Hash value of each second word in the second high-frequency word set of the second electronic file; Based on the first hash value and the second hash value, a similarity determination result between the first electronic file and the second electronic file is determined.

2. The method according to claim 1, characterized in that The step of acquiring a first high-frequency word set of text content of a first electronic file, and acquiring a second high-frequency word set of text content of a second electronic file in other electronic files, includes: Performing word segmentation processing on the text content of the first electronic file to obtain a first word set of the first electronic file, and counting the number of each first word in the first word set based on the text content of the first electronic file; Performing word segmentation processing on the text content of the second electronic file in the other electronic files to obtain a second word set of the second electronic file, and counting the number of each second word in the second word set based on the text content of the second electronic file; Based on the number of each first word, obtaining a preset number of first high-frequency words with the largest number in the first word set, and generating a first high-frequency word set based on the preset number of first high-frequency words; Based on the number of each second word, obtaining a preset number of second high-frequency words with the largest number in the second word set, and generating a second high-frequency word set based on the preset number of second high-frequency words; The preset number is determined by the median of the number of words contained in the text content of each electronic file among the multiple electronic files.

3. The method according to claim 2, characterized in that The performing word segmentation processing on the text content of the first electronic file to obtain a first word set of the first electronic file includes: Performing word segmentation processing on the text content of the first electronic file to obtain a third word set; The words belonging to the target part of speech in the third word set are eliminated to obtain the first word set.

4. The method according to claim 2, characterized in that: The performing word segmentation processing on the text content of the second electronic file in the other electronic files to obtain a second word set of the second electronic file includes: Performing word segmentation processing on the text content of the second electronic file among the other electronic files to obtain a fourth word set; The words belonging to the target part of speech in the fourth word set are eliminated to obtain a second word set.

5. The method according to claim 1, characterized in that: The method of obtaining the longest repeated word in the first high-frequency word set and the second high-frequency word set based on the longest common subsequence algorithm includes: Sorting the first words in the first high-frequency word set in a lexicographic order to obtain a first-order high-frequency word set; Sorting the second words in the second high-frequency word set in a lexicographical order to obtain a second-order high-frequency word set; Based on the longest common subsequence algorithm, obtaining the longest repeated word of the first high-frequency word set and the second high-frequency word set; If the number of words of the longest repeated word meets a preset numerical condition, determining that the second electronic file is an associated electronic file of the first electronic file includes: If the number of words in the longest repeated word is greater than or equal to a preset value, the second electronic file is determined to be an associated electronic file of the first electronic file, wherein the preset value is determined by the median number of words contained in the text content of each electronic file among the multiple electronic files.

6. The method according to claim 1, characterized in that The determining, based on the first Hash value and the second Hash value, a similarity judgment result between the first electronic file and the second electronic file includes: Obtaining a first weight of each first word in the first high-frequency word set, and obtaining a second weight of each second word in the second high-frequency word set; Determine a first product of a first weight of each first word and a first hash value of each first word; Determine a second product of a second weight of each second word and a second hash value of each second word; Accumulating the first products of the first words in a list to obtain a first simhash value of the first electronic file, and accumulating the second products of the second words in a list to obtain a second simhash value of the second electronic file; determining a Hamming distance between the first electronic file and the second electronic file according to the first simhash value and the second simhash value; When it is determined that the Hamming distance is less than the preset distance, it is determined that the first electronic file is similar to the second electronic file.

7. The method according to claim 6, characterized in that The step of obtaining a first weight of each first word in the first high-frequency word set and obtaining a second weight of each second word in the second high-frequency word set includes: Obtaining a first weight of each first word in the first word set according to a first ratio of the number of each first word in the first word set of the first electronic file to the total number of each first word; Obtaining a second weight of each second word in the second word set according to a second ratio of the number of each second word in the second word set of the second electronic file to the total number of each second word; Searching the first word set for the first words that are in the first high-frequency word set, and determining that the weight of each of the first words corresponds to the first weight of each of the first words in the first high-frequency word set; The second word set is searched for second words in the second high-frequency word set, and the weight of each second word is determined to correspond to the second weight of each second word in the first high-frequency word set.

8. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: Obtaining a preset keyword set and a third weight of each third word in the keyword set; Acquire each first word in a first word set of a first electronic file, and a first weight of each first word; Among the first words, determining the first word that matches the third word in the keyword set as the target word, and determining the first weight of the first word as the target weight of the target word; When there are multiple target words, determine the average of the products of the target weights of the target words and the corresponding third weights; A keyword analysis result of the first electronic file is determined according to the mean value.

9. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: Acquire words of text contents of the plurality of electronic files; When it is determined that the word matches a third word in the preset keyword set, an identifier of the electronic file to which the word belongs is obtained, and the identifier is added to the keyword analysis set; When it is determined that the first electronic file and the second electronic file are similar, obtaining an identifier of the first electronic file and an identifier of the second electronic file, and adding the identifier of the first electronic file and the identifier of the second electronic file to a similar file set; Obtain the weight of the words, sort the words in descending order according to the weight, obtain words in a preset sorting range, and add the words to the high-frequency word analysis set.

10. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: Acquire a first word of the text content of the first electronic file; When it is determined that the first word matches a third word in the preset keyword set, the first word is added to a keyword analysis set corresponding to the identifier of the first electronic file; When it is determined that the first electronic file and the second electronic file are similar, obtaining an identifier of the first electronic file and an identifier of the second electronic file, and adding the identifier of the second electronic file to a similar file set corresponding to the identifier of the first electronic file; The weight of the first word is obtained, the first word is sorted in descending order according to the weight, the first word in a preset sorting range is obtained, and the first word is added to a high-frequency word analysis set corresponding to the identifier of the first electronic file.

11. An electronic device, characterized in that: include: A processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the steps of the method according to any one of claims 1 to 10.

12. A computer storage medium, characterized in that: The computer storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the steps of the method according to any one of claims 1 to 10.

Images