Sorting a set of text documents

Abstract

This disclosure relates to a computer-implemented method for sorting a set of text documents. The method includes providing a formulation of a feature. The method further includes determining at least one new formulation of the feature by applying a generative NLP neural network. The method further includes determining, for each document in the set and for each formulation of the feature among several formulations, a text similarity value between the document and the formulation. The method further includes sorting the documents in the set based on the similarity values. [Fig. 2]

Description

Title of the invention: Sorting a set of text documents Technical field

[0001] The present disclosure relates to the field of computer systems and computer programs, and more particularly to a method, a program and a system for sorting a set of text documents including text documents. Technical background

[0002] Solutions exist for processing text documents, and in particular for measuring the syntactic similarity between two documents or portions of documents, for example Cosine similarity, Pearson correlation coefficient, Euclidean distance, Jaccard coefficient, Levenshtein distance, Dice index. However, there is a need for an improved solution for sorting a set of technical text documents in order to make a syntactic similarity measurement more reliable. Abstract

[0003] To this end, a computer-implemented (i.e. computer-implemented) method is proposed for sorting a set of technical text documents. The method comprises providing a formulation of a characteristic. The method further comprises determining at least one new formulation of the characteristic by applying a generative NLP-type neural network. The method further comprises determining, for each document in the set and each formulation among several formulations of the characteristic, a text similarity value between the document and the formulation. The method further comprises sorting the documents in the set according to the similarity values.

[0004] The method may comprise one or any combination of the following features: - the similarity value is a syntax similarity value; - the similarity is a cosine similarity; - determining, for each document in the set and each formulation among several formulations of the characteristic, the text similarity value between the document and the formulation includes: — an application of a stemming algorithm to said each formulation and to an abstract of said each document, so as to extract the root of keywords from said each formulation and from the abstract of said each document; — filtering of the punctuation and / or words of each of said formulations and of the abstract of each of said documents other than the keywords; — a vectorization of the results of filtering and application of the ra- algorithm cinization so as to obtain two vectors respective to said each formulation and to the abstract of said each document; — a calculation of a cosine similarity between the two vectors; - the algorithm is a Porter Stemmer rooting algorithm; - the neural network was trained on training data that includes technical documents of the same type as the documents in said set; - training data documents include feature formulations; - the documents in the set include at least parts of technical and / or textual documents; - the neural network is adapted to generate, as a new formulation, a summary-type formulation; and / or - the formulation provided consists of one or two sentences or a list of keywords.

[0005] A computer program comprising program code instructions for carrying out the steps of the method is also provided.

[0006] A data storage medium on which the computer program is recorded is also provided.

[0007] A computer system is also proposed comprising a processor coupled to a memory on which the computer program is recorded. Brief description of the figures

[0008] Non-limiting examples will be described with reference to the following figures:

[0009] Figures [Fig.l], [Fig.2] and [Fig.3] illustrate the method.

[0010] Figure [Fig.4] illustrates the system. Detailed description

[0011] To this end, a computer-implemented (i.e. computer-implemented) method is proposed for sorting a set of text documents. The method comprises providing a formulation of a characteristic. The method further comprises determining at least one new formulation of the characteristic by applying a generative NLP-type neural network. The method further comprises determining, for each document in the set and each formulation among several formulations of the characteristic, a text similarity value between the document and the formulation. The method further comprises sorting the documents in the set according to the similarity values.

[0012] The proposed method is an improved solution for sorting a set of technical text documents.

[0013] Indeed, the method calculates text similarity values ​​between the documents textual documents from the set of textual documents and a given formulation of a characteristic (i.e. the provided formulation). This makes it possible to measure the text similarity between the given formulation and each document, so as to be able to determine the documents which are, in terms of text, the most technically similar to the formulation. The method also comprises a step of sorting documents according to the calculated similarity values, for example from the most similar to the least similar, or vice versa. The method may further comprise any additional step of filtering the documents, for example consisting of extracting the documents for which the similarity value is greater than a certain threshold (or less than a certain threshold).

[0014] Furthermore, the method does not determine, or not only, the similarity value between each document of the set and only the provided formulation, but determines, for each document and each formulation among several formulations of the characteristic, a similarity value. The several formulations may or may not include the provided formulation. In any event, the several formulations include at least one new formulation of the characteristic, the new formulation being obtained by applying a generative NLP type neural network. In other words, at least one reformulation of the provided formulation is used for calculating the similarity value with the text, and several formulations are used. This allows more robustness in calculating the similarity value.The provided formulation might indeed not allow the determination of a text similarity value with the document that would faithfully reflect the technical similarity of the content of the formulation and the document, for example due to differences in the syntactic order between the initial formulation and the document. The use of at least one reformulation and several formulations in total solves this problem, and thus makes it possible to avoid, or at least reduce, similarity values ​​that are falsely too high (false positives) or falsely too low (false negatives). In other words, the method combines a syntactic approach with a semantic approach so as to optimize the realism of the similarity value. A syntactic approach to document comparison consists of comparing the character strings of the respective documents.Since several words can be used to talk about the same object, a syntactic approach is inappropriate when one document uses different words to talk about the same object as another document. A semantic approach to document comparison involves assessing the similarity of the meanings or semantic contents of the respective documents.

[0015] Furthermore, the new formulations are generated by applying a generative NLP type neural network, therefore adapted to this reformulation task. The new formulations are therefore determined in an objective, simple and efficient manner.

[0016] The method will now be discussed in more detail.

[0017] The method is a method for sorting a set of text documents. The method therefore takes as input a set of text documents, and provides as output a sorting of this set of documents. In this case, this sorting is that which results from the step of sorting the documents of the set according to the similarity values. The method may comprise the provision of the set of documents, for example by a user.

[0018] The method includes providing a formulation of a feature. The feature may be a statement of keywords or a phrase based on keywords to be searched. The formulation is a set of words describing the feature. For example, the formulation may consist of one or two sentences or a list of keywords. Providing the formulation may be performed by a user, for example by typing the formulation on a keyboard or dictating it via a microphone.

[0019] The method further comprises determining at least one new formulation of the characteristic by applying a generative NLP type neural network. An NLP type neural network is a neural network intended for automatic language processing (TAL), in English "Natural Language Processing" (NLP). The network is generative, that is to say that it is capable (trained to) generate text in response to a prompt. The neural network may be a network based on the so-called "transformer" architecture, known in the field of neural networks and machine learning. The neural network may be of the GPT type (from the English "generative pre-trained transformer").

[0020] The neural network may have been trained on training data that includes documents of the same type as the documents in said set. The neural network is thus capable of generating feature formulations from the documents in the set in a particularly reliable manner. The documents in the training data may include feature formulations, for example summaries.

[0021] The determination of at least one new formulation is done by applying the neural network. That is to say that the neural network is applied, for example upon request or instruction from the user, or automatically following the provision of the formulation, to determine at least one new formulation of the characteristic. For example, the neural network can receive, from the user for example, a guide message (for example in the form of an instruction sentence) telling it to reformulate the provided formulation, which triggers the application of the neural network, which at output provides a new formulation of the characteristic. Each application results in a new formulation, and there are therefore as many new formulations as there are applications of the neural network. The number of new formulations resulting from the application of the neural network can be pre determined and / or perhaps greater than 1 and / or between 3 and 5, for example greater than or equal to 2, for example greater than or equal to 3, for example greater than or equal to 4, for example greater than or equal to 5. This number of new formulations depends on the acuity of the previous new formulations and the number of words in the characteristic to be searched for in order to arrive at a word space equivalent (for example in number of significant words) to that of the document or the summary of the document to be compared. A new formulation of the characteristic is for example a reformulation of the characteristic. The application(s) of the neural network to reformulate the provided formulation can be automated, for example executed within an automatic process repeating several times (for example a predetermined number of times) to the neural network the prompt message to obtain several new formulations.

[0022] The neural network, for example being trained on data containing documents similar to those in the set, may be adapted (by virtue of its training) to generate, as a new formulation, a summary-type formulation. For example, the neural network may generate a summary-type reformulation from the provided formulation based on a prompt telling it to do so.

[0023] The prompt may be in any language, for example English or French. The formulation and new formulations may also be in any language, for example the same language as the prompt. The documents in the set may be in any language. A portion, for example a significant portion, for example a majority, of these documents may be in the same language as the formulation. In examples, each formulation, the prompt, and a significant portion of the documents (for example at least 50%, for example at least 75%, for example close to 100%, for example 100%) may be in the same language, for example English or French. Some documents may in this case be translations of documents originally in another language (for example German, Chinese or Japanese).

[0024] The method further comprises determining for each document in the set and each formulation among several formulations of the characteristic, a text similarity value between the document and the formulation. The several formulations may comprise all new formulations determined by applying the neural network. The several formulations may further comprise the provided formulation. For each document and each formulation, determining the similarity value may comprise applying any method taking as input a meaningful formulation extracted from the document (i.e. a formulation of a characteristic to which the document relates, for example a summary) and said each formulation and calculating the similarity value between these two formulations. The similarity value measures / quantifies the text similarity between the two formulations, for example as a value, e.g., between 0 and 1, with 0 meaning no similarity and 1 meaning maximum similarity.

[0025] The similarity value may be a syntax similarity value, i.e. the similarity value quantifies how similar the syntax of two formulations is (e.g. in terms of identical and / or synonymous words and / or in terms of the position of these words in the formulations). The similarity may for example be a cosine similarity, which measures the cosine between two vectorizations of the two formulations (the method then comprising a suitable vectorization step before each application of the cosine similarity). In other words, the method uses a simple similarity, measuring syntax similarity and not a semantic similarity. Nevertheless, precise and reliable similarity values, in terms of semantic similarity, are obtained, thanks to the use of new formulations, as explained previously.

[0026] Determining, for each document in the set and each formulation among several formulations of the characteristic, the text similarity value between the document and the formulation may comprise: applying a stemming algorithm (e.g., a Porter Stemmer stemming algorithm) to said each formulation and to a summary of said each document, so as to extract the keyword root of said each formulation and of the summary of said each document; filter out punctuation and / or words from said each formulation and abstract of said each document other than keywords. This may include removing words, if any, other than nouns and verbs, for example, articles or conjunctions, to keep only nouns and verbs, and / or removing punctuation if any; vectorizing the results of the filtering and the application of the stemming algorithm so as to obtain two vectors, for example in TF-IDF (term frequency-inverse document frequency) format, respective to said each formulation and to the abstract of said each document. This may consist of the application of any known method for vectorizing text, for example any known method (for example the use of Tfidftransformer) for transformation to TF-IDF format; calculate a cosine similarity between the two vectors, i.e. calculate a cosine between the two vectors (modulo normalization for example). The words that can be removed can be those present in the reformulation instruction as detailed later, for example “summary”, “condensed”, “reformulation”, “utility”. Thus, these terms are not taken into account in the similarity index. Without these terms, the similarity index could take into account some of these terms present in the reformulation while they are absent from the unreformulated summary, the similarity index obtained would be weaker while the summary could be potentially relevant.

[0027] The method further comprises sorting the documents in the set according to the similarity values. This may consist of sorting, for example automatically, in ascending or descending order, the documents according to their similarity values ​​with each formulation, for example according to a similarity value averaged over each formulation, i.e. an average of the similarity values ​​between the document and each formulation.

[0028] The method may further comprise displaying, on a screen of the system executing the method or on the screen of another system (for example connected to said system executing the method), the results of the sorting. The display may for example consist of a table presenting, for example in the sorting order, the documents and their associated similarity values, and optionally other information, such as bibliographic data relating to the document (for example title, publication date). The method may additionally or alternatively comprise exporting the results of the sorting in the form of a computer file, for example an Excel file.

[0029] Implementations of the method will now be discussed.

[0030] In implementations, the method may be used to compare the provided formulation to text summaries (the documents in the set then being, or being at least essentially, texts, for example articles in scientific publications and / or texts in patents or patent applications). This use of the method may for example be carried out within the framework of a relevance study (i.e. the provided formulation is compared to the documents in the set to determine whether the characteristic exists or does not exist elsewhere).

[0031] [Fig.l] illustrates the use of cosine similarity in implementations of the method for comparing a feature to be searched 10 with a summary of a text document, in the context of sorting a set 100 of documents (corpus) comprising hundreds of summaries. For each document, the summary and the formulation are converted into two TF-IDF (from the English "terni frequency-inverse document frequency") vectors, by known NLP / TAL methods. The scalar product of the two vectors is then calculated, and the cosine of this scalar product gives a similarity value (similarity index) between 0 and 1.

[0032] [Fig.2] shows a flowchart illustrating implementations of the method. As illustrated by the figure, the method comprises in these implementations, the provision S10 of the feature to be searched 10, and the provision, to the neural network (of the GPT type), of the instruction (in English) “Request for reformulation of the following feature” followed by the provided feature. Another example of an instruction (in English) can be “Can you reformulate this feature to include the usefulness of it followed by the provided feature. This results S20 in the new formulations 20, which are compared S30 to summaries (documents in the set) via cosine similarity 30 as explained above.

[0033] [Fig.3] shows functional steps of a software code implementing the method. For example, a python code could implement the method.

[0034] The first step in implementing the method consists of importing libraries (for example, the “nltk” library, NLP toolkit).

[0035] The second step of the implementation consists of defining and loading the cosine similarity function and the functions necessary for its use (for example, transforming a character string into a list of words, enumerating non-significant words, the function returning the cosine similarity value between two sentences where each of the sentences (i.e. character string) is transformed into a list of words, then pruned of non-significant words).

[0036] The third step of the implementation consists of defining the query function in reformulation.

[0037] The fourth step of the implementation consists of reading all the documents.

[0038] The fifth step of the implementation consists of providing the feature to be searched for and implementing the reformulation query. The reformulation query may include a number of desired reformulations.

[0039] The sixth step of the implementation consists of calculating the cosine similarity values.

[0040] The seventh step of the implementation consists of sorting according to the similarity values.

[0041] The eighth and final step of the implementation is not shown in [Fig.3], and is optional. In this step, the sorted documents are exported to an Excel file.

[0042] The method is computer-implemented. When referring to a computer-implemented method, it is understood that all or substantially all of the steps of the method are performed by a computer or a set of computers. The steps may be performed fully automatically, or partially automatically. In some embodiments, the triggering of certain steps may be performed in response to interaction with a user. The intended degree of automation may be predefined and / or defined by the user.

[0043] Referring to [Fig.4], an example of a system that can be used to perform the method is provided. In this example, the system is a computer, for example a workstation.

[0044] The computer thus comprises a processor unit 1010 connected to a computer bus 1000, and a random access memory 1070 (RAM) also connected to the bus in computer 1000. The computer further includes a graphics processor unit 1110 that is associated with a video RAM 1100 connected to the computer bus. A mass storage device controller 1020 manages access to a mass storage device, such as a hard disk drive 1030. Mass storage devices suitable for tangibly representing computer program instructions and data include all forms of non-volatile memory, including, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices; magnetic disks such as internal hard disk drives and removable disks; magneto-optical disks, and CD-ROM disks. These may also be supplemented by or incorporated into specific ASICs (application-specific integrated circuits). A network adapter 1050 manages access to a network 1060.The computer may also include a haptic device 1090 such as a cursor control device, a keyboard, or the like. A cursor control device is used to allow the user to selectively position a cursor at any location on the display 1080. In addition, the cursor control device allows the user to select various commands and control signals as input. The cursor control device includes signal generating devices for inputting control signals to the system. Typically, this may be a mouse, with the mouse button being used to generate the signals. The computer system may also include a touchscreen and / or a touchpad.

[0045] The computer program may comprise instructions executable by a computer, the instructions comprising means for causing the above system to implement the method. The program may be recordable on any data carrier, including the memory of the system. The program may for example be implemented in digital electronic circuits, or in computer hardware, firmware or software, or combinations thereof. The program may be implemented as an apparatus, for example a product tangibly represented in a memory device that can be read by a machine for execution by a programmable processor. Method steps may be performed by a programmable processor executing a program of instructions to perform functions of the method by processing input data and generating outputs.The processor may thus be programmable and be coupled to receive data and instructions from, and to transmit data and instructions to, a memory device, at least one input device and at least one output device. The program may be implemented in a procedural or object-oriented high-level programming language, or in a machine or assembly language. The language may be compiled or interpreted. The program may be a program. full installation or update program. Application of the program to the system leads to instructions for performing the process. The program may alternatively be stored and executed in a cloud computing environment, with the server communicating over the network with one or more clients. In this case, a processing unit / processor unit executes the instructions included in the program, thus causing the process to be executed in the cloud computing environment.

[0046] A simulation of the process made it possible to obtain the following numerical results in order to compare the number of documents actually deemed relevant among the classified documents.

[0047] A number of batches were defined based on similarity index values. The first batch groups together the documents potentially most relevant to the characteristic sought.

[0048] It is found that depending on the additional terms provided by the reformulations, the number of actually relevant documents in the batches with a higher similarity index increases. In a case where more than thirty words were provided by the reformulations, all the actually relevant documents were present in the first batch.

[0049] Ultimately, the terms resulting from the reformulations of a characteristic can be used to define and enrich a functional dictionary.

Claims

Claims

1. Computer-implemented method for sorting a set (100) of text documents, the method comprising: • providing (S 10) a formulation (10) of a characteristic; • determining (S20) at least one new formulation (20) of the characteristic by applying a generative NLP-type neural network; • determining (S30), for each document in the set and each formulation among several formulations of the characteristic, a text similarity value (30) between the document and the formulation; • sorting the documents in the set according to the similarity values ​​(30).

2. The method of claim 1, wherein the similarity value (30) is a syntax similarity value.

3. The method of claim 2, wherein the similarity (30) is a cosine similarity.

4. Method according to any one of claims 1 to 4, wherein determining (S30), for each document of the set (100) and each formulation among several formulations of the functionality, the text similarity value (30) between the document and the formulation, comprises: • applying a stemming algorithm, to said each formulation and to a summary of said each document, so as to extract the root of keywords from said each formulation and from the summary of said each document; • filtering the punctuation and / or the words from said each formulation and from the summary of said each document other than the keywords; • vectorizing the results of the filtering and of the application of the stemming algorithm so as to obtain two respective vectors for said each formulation and to the abstract of said each document; • calculating a cosine similarity between the two vectors.

5. The method of claim 4, wherein the algorithm is a Porter Stemmer stemming algorithm.

6. A method according to any one of claims 1 to 5, wherein the neural network has been trained on training data which comprises documents of the same type as the documents in said set (100).

7. The method of claim 6, wherein the training data documents include feature formulations.

8. The method of claim 7, wherein the neural network is adapted to generate, as a new formulation, a summary-type formulation.

9. A method according to any one of claims 1 to 8, wherein the provided formulation (10) consists of one or two sentences or a list of keywords.

10. A computer program comprising program code instructions for carrying out the steps of the method according to any one of claims 1 to 9 when said program is executed on a computer.

11. A data storage medium on which a computer program according to claim 10 is recorded.

12. A computer system comprising a processor coupled to a memory on which is recorded a computer program according to claim 10.

Images