Method for reducing imbalance between categories in a document corpus, application to training an AI model
By grouping and summarizing vectors of the majority category, the method addresses class imbalance in AI training data, ensuring balanced and informative training data for improved model performance.
Patent Information
- Authority / Receiving Office
- FR · FR
- Patent Type
- Applications
- Current Assignee / Owner
- TECKNOWMETRIX SAS
- Filing Date
- 2024-12-12
- Publication Date
- 2026-06-19
Smart Images
Figure 00000000_0000_ABST
Abstract
Description
Title of the invention: Method for reducing imbalance between categories in a corpus of documents, application to training an AI model. TECHNICAL FIELD OF THE INVENTION
[0001] The invention relates to the field of data mining, also called data exploration, which requires the preparation of large quantities of documents in various formats, followed by an analysis of their contents using automated methods. These automated methods may be based on principles of applying an artificial intelligence (AI) model to a classification task. TECHNOLOGICAL BACKGROUND
[0002] Data mining refers to the analysis of raw data using various complementary methods, with the aim of transforming this raw data into useful information by establishing relationships between the data, for example by sorting the data. Such analysis relies on algorithms that can be complex and implement different analytical tools, for example, processing methods based on AI models.
[0003] An AI model can be trained to make predictions from new data. Training the AI model can involve providing it with known data to perform what is often referred to as a deep learning, or machine learning, step. This step requires large volumes of data, but the data must also be of high quality. In particular, it must represent all the situations that the AI model will likely encounter when used after training. Obtaining a training data set of sufficient quality for training the AI model is crucial.
[0004] A recurring problem in training an AI model for a classification task concerns the balance between the situations represented by the training data. Consider, for example, a corpus of training data consisting of texts classified into two categories. If one of the two categories is significantly larger than the other (for example, if the larger category represents 75% of the corpus, compared to 25% for the smaller category), this imbalance will have a very strong impact on the quality of the training and the predictions that the model will subsequently make.
[0005] One solution to overcome this problem is to reduce the size of the majority category by discarding some of its data. This approach is known as The term is downsampling. Traditional approaches, while imperfect, generally propose identifying data whose removal has little impact on the overall corpus, in order to exclude them from training. This can lead to the omission of a large portion of the information contained in the removed data.
[0006] In this context, one object of the present application is to address this need for improved preparation of training data for the AI model. Description of the invention
[0007] An object of the invention is a method for overcoming class imbalance within a training data corpus for an artificial intelligence model.
[0008] With a view to achieving these objectives, a first aspect of the invention is a method for reducing size imbalances of categories in a corpus intended for training an artificial intelligence model, said method being implemented by computer and comprising the steps of: receiving training data in the form of texts, the texts being classified into categories, the training data forming a corpus intended for training the artificial intelligence model; determining a size for each of the categories, the sizes each being representative of a number of texts classified respectively in these categories; determining that a ratio between a size of a first of the categories, called the majority category, and a size of a second of the categories is greater than a predetermined size ratio threshold; representing each of the texts of the majority category by vectors to form a set of vectors;From the vectors representing the texts in the majority category, determine at least one set of vectors composed of vectors that are located from each other at distances less than a predetermined distance threshold; generate a set summary of the texts represented by the vectors in the determined set; represent the set summary by a set vector; and in the set of vectors, replace the vectors in the determined set with the set vector.
[0009] One advantage of this method is that it addresses class imbalances within a training data set for an artificial intelligence model. In particular, the invention makes it possible to selectively reduce the size of a data category while limiting the loss of potentially relevant information.
[0010] Furthermore, a conventional approach to downsampling (reducing the amount of data in a dataset) consists of discarding some of the vectors that are close to each other. In this case, all the information that these vectors carry is lost (except for potentially redundant information). with selected vectors (although the practitioner has little control over this aspect). In the solution according to the invention, the most salient information from the omitted texts is always at least partially present in the generated summaries, thus advantageously limiting information loss. According to additional, non-limiting features of the first aspect of the invention, considered individually or in any technically feasible combination:
[0011] - the step of generating an overall summary of the texts represented by the vectors of the determined set may include the generation of a concatenated text by concatenating the texts represented by the vectors of the set, then the generation of the set summary from the concatenated text;
[0012] - the at least one set of vectors may comprise at least two sets of vectors, the process may include the steps of: determining, during a test step, whether at least one of the vectors belongs to two of the at least two sets of vectors; in response to the test step, determining a vector called the starting vector whose average distance to its nearest neighbors (Nb) is the smallest among the vectors representing the texts of the majority category, Nb being a predetermined integer; generating a summary of the texts represented by the starting vector and its nearest neighbors (Nb); representing the summary by a new vector; and replacing the starting vector and its nearest neighbors (Nb) with the new vector; and
[0013] - following the step of replacing the starting vectors and its nearest Nb neighbors by the new vector, among the vectors representing the texts of the majority category, the process can return to the step of determining at least two sets of vectors.
[0014] Additional aspects of the invention relate to:
[0015] - a method for preparing the corpus of the first aspect of the invention, in which the corpus can be represented by vectors representing the received training data, the set vectors being able to be replaced by the set vector;
[0016] - a method for learning an artificial intelligence model, the method including a step of providing the training corpus as input to the artificial intelligence model during its learning;
[0017] - a data processing system comprising means for implementing steps of a process according to the invention;
[0018] - a computer program comprising instructions which, when the program is executed by a computer, leading the computer to implement the steps of a process according to the invention; and
[0019] - a computer-readable medium comprising instructions which, when they executed by a computer, lead the computer to implement the steps of a process according to the invention. BRIEF DESCRIPTION OF THE FIGURES
[0020] Other features and advantages of the invention will become apparent from the detailed description of the invention which follows with reference to the accompanying figures in which:
[0021] [Fig.1] Fig.1 illustrates a generic method for reducing the size of a category of documents and its application to training an AI model;
[0022] [Fig.2] Fig.2 illustrates a vector representation of text documents;
[0023] [Fig.3] The [Fig.3] illustrates a reduction in size of a data category;
[0024] [Fig.4] Fig.4 illustrates conflict management in the membership of a vector in a set of vectors; and
[0025] [Fig. 5] Figure 5 illustrates the generation of a summary of a set of documents textual. DETAILED DESCRIPTION OF THE INVENTION
[0026] The invention is detailed in relation to Figures 1 to 5. The descriptions applying to elements identified by a certain identifier in a given figure also apply to elements identified by the same identifier in the other figures.
[0027] Fig. 1 illustrates by block diagram a data processing method according to the invention, this method being able to be implemented by computer and having the purpose of preparing a corpus of data intended for training an AI model.
[0028] In a first step 110, training data classified into categories is received. This training data consists of data elements, each of which can take the form of a text, whether they are newspaper articles or general or scientific journals, patent documents, or even study or learning manuals.
[0029] The training data can be categorized in advance. The classification method and the categories used are not restricted to a particular method and categories. They will depend on the user's objective, i.e., the task to be performed by the AI model to be trained, regardless of the type of data received.
[0030] If we were to consider, for example, the use of colorants in different industries, we could classify a first text in a "food" category if this text concerns colorants in the food industry and a second text in a "cosmetic" category if this text concerns colorants in the cosmetic industry.
[0031] At a step 120, a size is determined for each of the categories of training data received at step 110. This may simply involve counting the number of texts belonging to a given category, for each of the categories.
[0032] At a step 130, it is determined that a first of the categories has a significantly larger size than a second of the categories and constitutes a category called the majority category.
[0033] The aim here is to determine whether one or more categories are larger than one or more of the other categories. It is possible to introduce one or more thresholds applicable to the ratios between the sizes of the categories considered. The objective is indeed to identify a potential imbalance between the sizes of the categories, which could negatively influence the learning of an AI model. Thus, it can be determined that a first category, called the majority category, is larger by more than a certain threshold than a second category, called the minority category. This threshold could correspond to a level beyond which the size difference between the categories compromises the training of the AI model or reduces its quality.This size difference can be assessed using a ratio R between the sizes of the categories, and it can be estimated that if R exceeds a predetermined size ratio threshold S, then the data corpus, and more specifically the data belonging to the first category, should be processed in such a way as to limit or eliminate the influence of the imbalance between category sizes on the training of the AI model. The size ratio threshold S is greater than 1.
[0034] Thus, step 130 can consist of a test determining whether a ratio R between the size of a first category and a size of a second category is greater than a predetermined size ratio threshold S. If so, indicated by "Y" in [Fig.1], the first category is then considered to be the majority category and the process of reducing imbalance between categories in a corpus of documents proceeds to a step 140.
[0035] The value of the predetermined threshold S depends on the specific tasks to be handled by this AI model and the training data. It is difficult to specify a given value for S, as this value must be determined on a case-by-case basis. In a situation where two categories are present, for example, if the majority category represents more than 60% of the total data—that is, if its size is more than 50% greater than the size of the minority category, or if the R-squared ratio exceeds a threshold S of 1.5—then it is necessary to process the data included in the majority category to reduce its weight in the training.
[0036] As mentioned in the "Technological Background" section, it is possible to simply reduce the size of the majority class by discarding some of its data. This operation mechanically reduces the differences between class sizes, but carries the risk of removing relevant data from the learning process.
[0037] The present method proposes an alternative for reducing the size of the majority category while mitigating the risk of removing data relevant to the AI model's training. This alternative can be implemented by means of the following steps.
[0038] In step 140, the Doc texts belonging to the majority category Maj defined in response to step 130 are each represented as a vector V, as illustrated in [Fig. 2]. This step can be performed in advance, for example, even before receiving the training data. The training data can comprise a dual set of data: the texts on the one hand, and the vectors representing the texts on the other.
[0039] This is a conventional operation in the field of AI model training methods. It can be performed using methods known as embedding or one-hot encoding. Embedding can be defined as a way of representing objects such as text, images, and audio as points in a vector space, where the locations of these points in space, defined by vectors, are semantically significant for machine learning algorithms. Thus, vectors of texts that are semantically close, that is, of similar or overlapping content, will be close to each other.
[0040] In an example where the vectors V would be two-dimensional, they can be represented by points in a space Esp with two axes Axl and Ax2 as illustrated by [Fig.3](A), where the vectors V are each represented by a circle, as shown in the cartouche accompanying the frame.
[0041] In this example, we observe that (i) the size of the majority category, defined as the number of texts composing it (or of vectors representing each of these texts respectively), is 20 and (ii) the vectors V are not uniformly distributed in the space Esp but form, in this example, two clusters of vectors close to each other, therefore representative of texts presenting themselves with contents close to each other.
[0042] In a step 150, sets of vectors are determined, composed of vectors V that are located at distances less than a predetermined distance threshold SDist. This step includes calculating the distances separating the vectors from each other. The distances between each of the vectors can be evaluated, for example, by calculating cosine similarity, but any evaluation method a similarity between two vectors can be used (Euclidean distance, dot product...).
[0043] In the example of [Fig.3](A), this step leads to the determination of the two sets Ens 1 and Ens2, comprising respectively 6 and 5 vectors V, therefore formed respectively from 6 and 5 texts Doc.
[0044] This proximity between the vectors of the same set can be used to reduce the size of the majority category while minimizing the loss of data content.
[0045] Figure 3 illustrates a situation in which there is no conflict in forming the sets: the vectors V are either clearly separated from each other by distances greater than the SDist distance threshold, or grouped into sets of vectors separated from each other by distances less than the SDist distance threshold and separated from other vectors by distances greater than the SDist distance threshold. However, it may happen that two sets of vectors are separated from each other by a distance less than the SDist distance threshold. There is then a conflict regarding the membership of one or more of the vectors V in one of the two sets, as illustrated by Figure 4, with the membership of vectors V at the intersection of two sets of vectors Ens1' and Ens2' composed of vectors that are located from each other at distances less than the predetermined SDist distance threshold.
[0046] To address the situation illustrated in [Fig. 4], a test step T determines whether at least one of the vectors belongs to two or more sets of vectors defined in step 150. If not, as indicated by "N", the process continues in step 160 described below. Conversely, if at least one vector belongs to two sets, as indicated by "Y" in the figure, then the process continues in successive steps 152, 154, 156, and 158 described below.
[0047] In step 152 following test step T, a vector V0 whose average distance to its nearest neighbors Nb is the smallest is determined as a starting vector. Nb is an integer defined by the user, based on the training data received and their proximity to each other.
[0048] In step 154A, the text corresponding to the starting vector V0 and the set of texts whose vectors are located at a distance less than a distance threshold SDist are concatenated. The concatenation product serves as the basis for generating a summary 154B. The distance threshold SDist' can be equal to or less than the distance threshold SDist, and allows for more or less precise management of the vector groups, depending on its value. Steps 154A and 154B collectively form a step 154. In the example of [Fig. 4], the SDist' threshold is less than SDist.
[0049] At a step 156, this summary is represented by a new vector V'.
[0050] At a step 158, the new vector V' replaces the vectors of the texts summarized in step 154, as illustrated by [Fig.4], in which the vector V' replaces three of the vectors V.
[0051] Following step 158, the process returns to step 150. By repeating steps 152 to 158, the conflicts in the membership of the vectors V in a set will disappear. At this point, the process will continue with step 160 after the test step T.
[0052] In step 160, and separately for each of the determined sets (Ensl and Ens2 in this example), a set summary of the texts represented by the vectors of the set under consideration is generated. Figure 5 illustrates the case of the Ensl set, whose Doc texts are processed in such a way as to generate a ResEnsl set summary of these texts. The set summaries are therefore each representative of the informational content of the texts whose vectors form the different determined sets, respectively. In other words, each of the sets is associated with a set summary.
[0053] Since these sets are formed of vectors close to each other, the overall summary can be of good quality, neglecting less potentially relevant information than if the summary concerned texts from the majority category chosen arbitrarily.
[0054] Step 160 can include two operations: a first operation 160A of generating a concatenated text Concat.Doc by concatenating the texts represented by the vectors of the set considered, Ensl or Ens2 in the present example, then an operation 160B of generating the set summary ResEnsi from the concatenated text Concat.Doc, as illustrated by [Fig.5] for the case of the set Ensl.
[0055] At a step 170, for each of the set summaries, the set summary under consideration is represented by a set vector, VEnsi or VEns2 in the present example, for example in the same way that the Doc texts were represented by the vectors V in step 140. [Fig.5] illustrates this step applied to the set summary ResEnsl, represented by the set vector VEnsi.
[0056] At a step 180, for each of the determined sets, the vectors V of a considered set, Ensl or Ens2 in this example, are replaced by the corresponding set vector, VEnsi or Vpn^, as illustrated by [Fig.3](B).
[0057] Thus, the 6 vectors V of the set Ensl are replaced by the single set vector VEnsi, and the 5 vectors V of the set Ens2 are replaced by the single set vector VEns2. In this way, the size of the majority category Maj has been reduced from 20 to 11.
[0058] Steps 130 to 180 can be repeated until a situation is reached that complies with the fixed threshold parameter S, i.e., until the ratio R between the sizes of The categories must become less than or equal to the predetermined size ratio threshold S. Adherence to this criterion ensures a degree of homogeneity in category sizes desired by the user (for example, all sizes must fall within a predetermined range) and / or keeps the size of the categories above a minimum threshold (it is important to avoid reducing category sizes below a certain threshold). Of course, the majority category may differ at each iteration of steps 130 to 180.
[0059] At each iteration of steps 130 to 180, the majority category may change, depending on the size reduction that was applied to the majority category in the previous iteration.
[0060] Finally, the process can be stopped when, at step 150, it is not possible to identify a set of vectors composed of vectors located at distances less than the predetermined distance threshold SDist. The vectors are then considered too far apart to perform summarization without a high risk of information loss.
[0061] Following the adjustment of the sizes of the training data categories, the data corpus they represent is provided as input to an AI model during its training in step 200. It should be noted that if the test in step 130 does not identify any imbalance requiring rebalancing in steps 140 to 180, as indicated by "N", the process proceeds directly from step 130 to step 200, as illustrated in [Fig. 1]. In the corpus provided as input to the AI model, the corpus is represented by vectors representing the received training data, with the vectors V of a set, for example Ensl, being replaced by the vector VEnsi of that set. Thus, the training data exhibits an improved balance between the different categories that compose it, and the quality of the learning is thereby improved.
[0062] Of course the invention is not limited to the modes of implementation described and alternative embodiments can be made without departing from the scope of the invention as defined by the claims.
Claims
Demands
1. A method (100) for reducing size imbalances in categories of a corpus intended for training an artificial intelligence model, said method being implemented by computer and comprising the steps of: - receiving (110) training data in the form of texts, the texts being classified into categories, the training data forming a corpus intended for training the artificial intelligence model; - determining (120) a size for each of the categories, the sizes each being representative of a number of texts classified respectively in these categories; - determining (130) that a ratio (R) between a size of a first of the categories, called the majority category (Maj), and a size of a second of the categories is greater than a predetermined size ratio threshold; - representing (140) each of the texts of the majority category by vectors (V) to form a set (J) of vectors (V);- among the vectors (V) representing the texts of the majority category, determine (150) at least one set (Ensl, Ens2, Ensl', Ens2') of vectors composed of vectors which are located from each other at distances less than a predetermined distance threshold; - generate (160) a set summary (ResEnsi) of the texts represented by the vectors of the determined set (Ensl, Ens2); - represent (170) the set summary (ResEnsi) by a set vector (VEnsi, VEns2); and - in the set (J) of vectors, replace (180) the vectors (V) of the determined set (Ensl, Ens2) by the set vector (VEnsi, VEns2).
2. A method according to claim 1, wherein the step of generating (160) an overall summary (ResEnsi) of the texts represented by the vectors of the determined set (Ensl, Ens2) comprises the generation (160A) of a concatenated text (Concat.Doc) by concatenating the texts represented by the vectors of the set (Ensl, Ens2), and then the generation (160B) of the overall summary (ResEnsi) from the concatenated text (Concat.Doc).
3. A method according to claim 1 or 2, wherein the at least one set of vectors comprises at least two sets (Ens1, Ens2, Ens1', Ens2') of vectors (V), the method comprising the steps of: - determining, during a test step (T), whether at least one of the vectors (V) belongs to two of the at least two sets of vectors; - in response to the test step (T), determining (152) a vector called the starting vector (VO) whose average distance to its nearest neighbors (Nb) is the smallest among the vectors (V) representing the texts of the majority category, Nb being a predetermined integer; - generating (154) a summary of the texts represented by the starting vector and its nearest neighbors (Nb); - representing (156) the summary by a new vector (V'); and - replacing (158) the starting vector (VO) and its nearest neighbors (Nb) with the new vector (V').
4. A method according to claim 3, wherein, following the step of replacing (158) the starting vector (VO) and its nearest neighbors Nb with the new vector (V'), from among the vectors (V) representing the texts of the majority category, the method returns to the step of determining (150) at least two sets (Ensl, Ens2, Ensl', Ens2') of vectors.
5. Method (100) of corpus preparation according to any one of claims 1 to 4, wherein the corpus is represented by vectors representing the received training data, the vectors (V) of the set (Ensl, Ens2) being replaced by the set vector (VEnsi, VEns2).
6. Method (100) of learning an artificial intelligence model, comprising the step of providing (200) the corpus obtained according to the method of any one of claims 1 to 4 as input to the artificial intelligence model during its learning.
7. Data processing system comprising means for implementing the steps of the process according to any one of claims 1 to 6.
8. A computer program comprising instructions which, when the program is executed by a computer, cause the computer to
9. implement the steps of the process according to any one of claims 1 to 6. Computer-readable support comprising instructions which, when executed by a computer, cause the computer to carry out the steps of the process according to any one of claims 1 to 6.