Dimension reduction method

Abstract

The invention relates to a dimensionality reduction method for training a multi-dimensional feature space of a machine learning model (50) by machine learning, comprising the following steps: providing (101) at least one data pair (30) in which raw data elements (31) and modified data elements (32) each have a feature difference ([Delta] f) between one another, which feature difference is specific to a respective prescribed task for machine learning; determining (102) at least one task-specific feature space, which is specific to the at least one feature difference ([delta] f), based on a comparison of the respective data pair (30); the dimensionality reduction is performed (103) based on the determined task-specific feature space.

Description

Technical Field

[0001] The present invention relates to a method for reducing the dimension of a feature space for training a machine learning model. The present invention also relates to a machine learning model, a computer program, a device and a storage medium all for this purpose. Background Art

[0002] Diversity sampling is a common aspect of active learning (e.g., see Yang, Yi et al., “Multi-class active learning by uncertainty sampling with diversity maximization”, International Journal of Computer Vision, 2015). In order to select different groups of images, a distribution in a high-dimensional feature space is often used. This feature space can be obtained from the basic framework of a multi-task model or from a pre-trained multi-task model such as “CLIP” (Ramesh, Aditya et al., “Hierarchical Text-Conditional Image Generation with CLIP Latents”, arXiv:2204.06125[cs.CV.CV]). Regardless of the origin, these feature spaces are typically very high-dimensional. In order to select relevant directions, techniques such as principal component analysis (PCA) are often used to reduce the dimensionality of this space.

[0003] PCA efficiently determines and orders vectors in feature space through which the data has the largest variance. What produces the largest variance in a set of images may be of some interest, but is not necessarily associated with the directions that are most relevant to the underlying task of the head of a multi-task model (hereinafter also referred to as the task head). Using previous layers of a specific task head as feature layers can provide some task focus, but the variance in the relevant directions may be low, especially in the case of objects that appear less frequently in the data set. In an extremely simplified representation of traffic sign recognition, feature directions associated with the presence of a yield sign may have lower variance than other feature directions, such as the presence of snow, night, buildings, or vehicles, but in order to select appropriately diverse images, it may be desirable to focus on this feature space.

[0004] Ideally, the relevant feature space directions can be explicitly specified and pre-selected before applying dimensionality reduction techniques such as PCA. Using an early layer from the task head as a feature layer may somewhat restrict the directions towards the target task. Summary of the invention

[0005] The object of the present invention is to provide a method, a machine learning model, a computer program, a device and a computer-readable storage medium. Other features and details of the present invention are apparent from the description and the drawings. The features and details described in conjunction with the method of the present invention are naturally also applicable to the machine learning model of the present invention, the computer program of the present invention, the device of the present invention and the computer-readable storage medium of the present invention, and vice versa, so the disclosures are always cross-referenced or cross-referenced to individual aspects of the present invention.

[0006] The object of the present invention is in particular a dimensionality reduction method for a multidimensional feature space, preferably a high-dimensional feature space, in particular for training a machine learning model by machine learning. The method may comprise providing at least one data pair, in which both the original data element and the modified data element (as a data pair) are provided and / or have characteristic differences with each other, the characteristic differences being specific to at least one respective prescribed task for machine learning. The at least one or more different prescribed tasks may be provided by a task header of the machine learning model.

[0007] The at least one data pair may also be specific to sensor data originating from the acquisition of a sensor, such as an image sensor. This means, for example, that the at least one data pair includes sensor data from the acquisition of a sensor and / or an image record based at least in part on an image sensor and may have corresponding image information. For example, the image information may include an image of the vehicle environment that has been acquired by an image sensor. The sensor data may therefore include, for example, a pixel or pixels representing the acquired environment. It is also conceivable that the at least one data pair includes sensor data in the form of measurement data, which has been determined by the metrological acquisition of the sensor. In addition, it may be provided that the at least one data pair is specific to the sensor data because it includes at least partially constructed training data, which allows the machine learning model to be trained for an application on the sensor data. Various different methods may also be combined here, such as enhancing or at least partially simulating sensor data. This may have the advantage that the machine learning model is prepared for a variety of different situations and environments and therefore has a higher accuracy for the application.

[0008] Furthermore, the method may include the following steps, which are preferably performed sequentially and / or repeatedly, preferably iteratively for various prescribed tasks:

[0009] - based on the comparison of the respective data pairs, determining at least one task-specific feature space specific to at least one feature difference, wherein the task-specific feature space is preferably implemented as a subspace of a multidimensional feature space, preferably a subspace of a high-dimensional feature space and / or comprises feature space vectors relevant to the respective task,

[0010] -Perform dimensionality reduction based on the determined task-specific feature space.

[0011] An advantage of this method is that the relevant feature space vectors can be determined in a simple way. This subspace can then be projected, for example by deconvolution and preferably rotation, so that PCA can be used to (reduce) the subspace for dimensionality reduction. In particular, the dimensionality reduction based on the determined task-specific feature space is used to predetermine and preselect the relevant feature space directions before adopting another dimensionality reduction technique such as PCA. Alternatively, the implementation of dimensionality reduction can already include the implementation of PCA. PCA can also provide at least one parameter such as a transformation result, which can also be taken into account in subsequent applications and in particular in the reasoning of the trained machine learning model.

[0012] It is also conceivable within the scope of the present invention to carry out the following steps:

[0013] - training the machine learning model for the at least one prescribed task based on the performed dimensionality reduction, wherein the at least one prescribed task comprises identifying and / or detecting the at least one feature difference, in particular for classification

[0014] and / or form of object detection,

[0015] - providing a trained machine learning model for application and preferably inference, in which at least one prescribed task is applied to the sensor data and / or further sensor data, preferably image data, which originate from the

[0016] The sensor and / or another sensor, preferably an image sensor.

[0017] It can be provided that the values ​​of the sensor data and preferably the values ​​of the image data are used to represent the vehicle surroundings and preferably the traffic scene. At least one of the tasks may also include a classification based on these values ​​and preferably an image classification, for example in order to detect objects in the traffic scene. Classification and image classification may also be provided in the form of semantic segmentation (i.e. pixel or area-based classification) and / or object detection. The application may also include traffic sign recognition and / or recognition of traffic signals from a traffic light system. Thus, the output of the machine learning model, such as a classification result, may be used to navigate and / or control an at least partially autonomous robot and / or an at least partially autonomous vehicle taking into account the traffic scene.

[0018] Furthermore, it is optionally feasible within the scope of the invention that the provided data elements are each specific to image data, wherein the at least one characteristic difference is provided as a difference in an image feature of the image data. Preferably, the identification of the at least one characteristic difference (e.g. object detection) can be performed based on pixel values ​​of the image data. The image data can be, for example, an image from a radar sensor, an ultrasonic sensor, a lidar sensor and / or a thermal imaging camera. The image can therefore also be a radar image and / or an ultrasonic image and / or a thermal image and / or a lidar image.

[0019] It is also conceivable in the context of the present invention to perform at least partially autonomous navigation of a robot and / or vehicle based on the recognition and preferably classification and / or object detection, wherein the image data may represent a traffic scene during navigation, wherein the at least one characteristic difference may be provided as a difference in an image feature of the image data, which indicates a navigation-relevant difference in the traffic scene, preferably in the form of different signals of a traffic light system and / or different traffic signs. In this way, the reliability of such navigation can be improved by dimensionality reduction and, if necessary, selection of training data based thereon.

[0020] The invention may preferably provide that the transformation result is obtained based on the dimensionality reduction performed, preferably by using a principal component analysis (PCA for short) that reduces the feature space. The transformation result may preferably be specific to the weighting or loading of the PCA. The transformation result can be used for dimensionality reduction of the sensor data when applying a trained machine learning model. To this end, the transformation result can be used to form an additional layer in the machine learning model, in particular on the trained neural network of the machine learning model, to perform dimensionality reduction.

[0021] Furthermore, it can be provided that at least one specified task comprises several different tasks for which the dimensionality reduction is performed. For this purpose, a specific feature space can be determined in each case. Preferably, the different tasks are provided by different task heads of the machine learning model to allow reliable application, for example in autonomous vehicles.

[0022] It is further conceivable that providing at least one data pair comprises at least one of the following steps:

[0023] - one of the data elements of the mask data pair,

[0024] - replace part of one data element with part of another data element,

[0025] -Perform inline drawing to modify one of the data elements of a data pair.

[0026] In this way, a simple method can be used to determine the task-relevant feature space vectors in which the data element is masked, replaced by a cutout of another data element, or whitewashed. These measures are used to generate two data elements whose difference in feature space should be related to the feature difference in the two data elements.

[0027] Another object of the present invention is a machine learning model trained by the method according to the present invention. The machine learning model can be implemented, for example, as a multi-task model with multiple task heads to provide various prescribed tasks.

[0028] Another object of the present invention is a computer program, in particular a computer program product, comprising instructions which, when executed on a computer, cause the computer to perform the method of the present invention. Thus, the computer program according to the present invention has the same advantages as those described in detail with reference to the method of the present invention.

[0029] Another object of the invention is a data processing device configured to perform the method according to the invention. The device may be, for example, a computer that executes the computer program according to the invention. The computer may have at least one processor for executing the computer program. A non-transient data memory may also be provided in which the computer program is stored and from which the processor can read the computer program for execution.

[0030] Another object of the present invention may be a computer-readable storage medium, which contains a computer program according to the present invention and / or includes instructions, which when executed by a computer cause the computer to perform the method according to the present invention. The storage medium is, for example, configured as a data storage such as a hard disk and / or a non-volatile memory and / or a memory card. For example, the storage medium can be integrated into a computer.

[0031] Furthermore, the method according to the present invention may also be realized as a computer-implemented method. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Further advantages, features and details of the invention are shown in the following description of embodiments of the invention in detail with reference to the accompanying drawings. The features mentioned in the claims and the description may be essential to the invention individually or in any combination. The drawings show:

[0033] Figure 1 is a schematic representation of a method, a machine learning model, a device, a storage medium, and a computer program according to an embodiment of the present invention, and

[0034] Figure 2 is another diagram showing a method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0035] Figure 1 A method 100, an apparatus 10, a storage medium 15, a machine learning model 50 and a computer program 20 according to an embodiment of the present invention are schematically shown.

[0036] A common problem with machine learning is that the feature space of diversity selection and the task of the machine learning model 50 are difficult to level out. Embodiments of the present invention ensure that the task of diversity selection is not neglected. The advantages of the proposed method are that it is simple and intuitive. It can also be applied to multiple tasks and can be easily automated in a pipeline once the target image has been created.

[0037] Embodiments of the present invention may be used as an upstream part of a "machine learning toolchain". Here, a machine learning toolchain may be a collection of software tools used to develop, train, and implement machine learning models in a coordinated process. In addition, embodiments of the present invention may also be used for navigation of at least partially autonomous robots and / or vehicles.

[0038] According to an embodiment of the present invention, Figure 1 A method 100 for reducing the dimensionality of a multi-dimensional or high-dimensional feature space for training a machine learning model 50 is shown. The training can be carried out by machine learning, in which one or more specified tasks are learned in particular. The dimensionality reduction can be carried out upstream of the training, in particular in order to prepare training data for the training. According to a first method step 101, at least one data pair 30 can be provided for this purpose, in which the original data element 31 and the modified data element 32 can each have a characteristic difference Δf with respect to each other, which characteristic difference is specific to the at least one specified task for machine learning. For this purpose, the modified data element 32 can be generated based on the original data element 31, for example by modifying the content of the original data element 31. Then, according to a second method step 102, at least one task-specific feature space can be determined, which (in each case) can be a subspace of the original multi-dimensional feature space of the data pair. A separate task-specific feature space can be determined for each task that is different from each other. In this way, the respective task-specific feature space can be specific to the characteristic difference Δf. In order to determine the feature space, it can be provided to compare the respective data pairs 30. According to a third method step 104, the dimensionality reduction can be performed based on the (respectively) determined task-specific feature space. Furthermore, the machine learning model 50 can be trained for a specified task based on the performed dimensionality reduction 400. This allows providing a trained machine learning model 50 for an application in which the at least one specified task is applied to sensor data and / or another sensor data, preferably image data, derived, for example, from the acquisition of a sensor 40 and / or another sensor 40, preferably an image sensor 40.

[0039] In particular, it is an innovative idea to determine the (task) relevant feature space vector in a simple way, project this subspace by rotation and then apply PCA to the reduced space. For example, as a simple method for determining the relevant feature space vector, an image can be masked, replaced with a part of another image or painted to create two images. The two images include an original image and a modified image, and the difference in their feature space should be related to the difference in information in the two images. Performing a forward pass through the grid and extracting the relevant feature space of the two images can provide feature space vectors for the original image and the modified image,

[0040]

[0041] By repeating this process for a small set of several sample images, the normalized mean of this separation can be obtained:

[0042]

[0043] This should correspond to a direction in the feature space that is closely associated with this feature, where the angle brackets here denote normalization to the unit vector. We can then verify whether the change is meaningfully related, i.e.,

[0044]

[0045] It should not be much smaller than 1, otherwise the correlation is not meaningfully related to the desired feature direction. In these cases, the choice of feature space may be insensitive to the target, or the sensitivity to the target is not well determined. In some cases, it may make sense to extract more than one feature direction for a target space (e.g., a feature hypersurface).

[0046] For example, in a picture with a green traffic light, the green traffic light can be replaced by a picture with a yellow traffic light. The difference between the two in feature space, i.e., the feature difference Δf i It should be highly correlated with switching from green to yellow light. If you repeat this with a small number of different example images, the average of this separation should give you a rough indication of the direction of the difference between green and yellow light.

[0047] Figure 2 An example of such a method according to an embodiment of the present invention is shown. Point 201 represents the location of a difference vector between an image pair in the n-dimensional feature space, for example, in an image portion with and without a corresponding object. Arrow 202 is a fitted line, here in the direction f i , f j The dominant low position is in the upper direction, with a small component in the other direction (labeled as f n ). The direction of this arrow will be the direction of the highlighted feature for the image pair, such as the yellow traffic light.

[0048] After completing this task for a single task target, other small image sets can be used to find other desired features specific to the task head (or desired features from other task heads). An orthogonal basis can be iteratively constructed by removing the projection on the previous direction, i.e.,

[0049]

[0050] When trying to form task-specific directions, the orthogonal basis may suffer from some vector redundancy. can be determined here. In this case, the direction can simply be omitted since it is already taken into account by the chosen direction. Once a set of vectors is defined and an orthogonal basis is extracted, the feature space can be rotated. If there are p orthogonal task-specific vectors and q desired basis vectors, the feature space can be rotated as follows:

[0051] f′=R1f,

[0052] Thus the first task-specific direction is now aligned with the first direction in n-dimensional space. This can be repeated iteratively for all p directions, creating a single matrix,

[0053]

[0054] The first p dimensions of the n>>p dimensional space are adapted to the target subspace. This matrix can be applied to the feature vectors of all images to perform dimensionality reduction based on the determined task-specific feature space. PCA can then be applied to extract the qp dimension with the highest variance from the np dimensional subspace. The p orthogonal task-specific directions and the direction containing the qp direction with the highest variance from PCA within the orthogonal subspace together form the diversity-reduced feature space in which the task-specific directions are now encoded. In the rotated n-dimensional basis, the PCA matrix is ​​of K(qp)×(np) dimensions, but can be easily extended by the p-dimensional identity and formed using the rotation matrix to form a combined q×n single matrix, which brings the original feature vector into the task-specific PCA direction.

[0055] Finally, to ensure that task-specific feature directions are fully considered in diversity selection, post-selection scaling can be used to obtain unit variance in all directions.

[0056] This method can be used together with the selection of pre-processed images to achieve certain task-specific goals. For example, in an embodiment of the present invention, the following steps can be specified:

[0057] First, the feature directions of interest can be extracted from each set of preselected image pairs. Then, the resulting vector orthogonal basis can be constructed. The p vector can be constructed. The rotation matrix R can be determined and applied to all images in the data set. PCA can then be performed, and the first p dimensions are taken out to obtain q-pPCA directions. The identity can then be expanded to obtain Z and extraction of the reduced dimensional space can be performed. After scaling to the unit normal, a selection diversity sampling algorithm can be applied.

[0058] This setting can also be done online. Once the model is trained and the above steps have been performed, the modified PCA can be set as the weights of the linear neural network layer associated with the model. In this way, the transformed results can be used when applying the machine learning model. During inference, it is very cheap to extract these feature vectors. High positive values ​​in the corresponding feature directions can be used for online triggering.

[0059] The above description of the embodiments describes the invention only within the scope of examples. Of course, the individual features of the embodiments can be combined freely with one another, without exceeding the scope of the invention, as long as this is technically advantageous.

Claims

1. A method (100) for reducing the dimensionality of a multidimensional feature space for training a machine learning model (50) by machine learning, the method (100) comprising the following steps: - providing (101) at least one data pair (30) in which the original data element (31) and the modified data element (32) each have a characteristic difference (Δf) with respect to one another, said characteristic difference being specific to the respective prescribed task for machine learning, said at least one data pair (30) being specific to sensor data acquired from a sensor (40), - determining (102) at least one task-specific feature space based on a comparison of the corresponding data pairs (30), the at least one task-specific feature space being specific to at least one of the feature differences (Δf), - performing (103) said dimensionality reduction based on the determined task-specific feature space.

2. The method (100) according to claim 1, characterized in that: Perform the following steps: - training the machine learning model (50) for the at least one prescribed task based on the performed dimensionality reduction, wherein the at least one prescribed task comprises identifying the at least one feature difference (Δf), the at least one feature difference (Δf) preferably being in the form of classification and / or object detection, -Providing the trained machine learning model (50) for an application in which the at least one specified task is applied to the sensor data and / or another sensor data, preferably image data, acquired from the sensor (40) and / or another sensor (40), preferably an image sensor (40).

3. The method (100) according to claim 2, characterized in that: Each data element is specific to the image data, wherein the at least one characteristic difference (Δf) is provided as a difference of an image characteristic of the image data, and the identification of the at least one characteristic difference (Δf) is performed based on pixel values ​​of the image data.

4. The method (100) according to claim 2 or 3, characterized in that: Navigation of an at least partially autonomous robot and / or vehicle is performed based on the recognition and preferably classification and / or target detection, wherein the image data represent a traffic scene during navigation, wherein the at least one characteristic difference (Δf) is provided as a difference in image features of the image data, which difference indicates a navigation-relevant difference in the traffic scene, preferably in the form of different signals of a traffic light system and / or different traffic signs.

5. The method (100) according to any one of claims 2 to 4, characterized in that Based on the performed dimensionality reduction, a transformation result is preferably obtained by applying a principal component analysis (PCA) of a reduced feature space, wherein the transformation result is preferably specific to the weighting or loading of the principal component analysis, and wherein the transformation result is used in the application of the trained machine learning model (50) to reduce the dimensionality of the sensor data.

6. The method (100) according to any one of the preceding claims, characterized in that The at least one predetermined task comprises a plurality of different tasks for which a dimensionality reduction is performed, wherein a feature space specific to the task is determined in each case for this purpose, wherein the different tasks are preferably provided by different task heads of the machine learning model.

7. The method (100) according to any one of the preceding claims, characterized in that The providing (101) of the at least one data pair (30) comprises at least one of the following steps: - masking one of the data elements of said data pair (30), - replace part of one data element with part of another data element, - Performing inlining to modify one of the data elements of the data pair.

8. A machine learning model (50) trained by a method (100) according to any one of the preceding claims.

9. A computer program (20) comprising instructions which, when the computer program (20) is executed by a computer (10), cause the computer (10) to perform the method (100) according to any one of the preceding claims.

10. A data processing device (10) configured to perform the method (100) according to any one of claims 1 to 7.

11. A computer-readable storage medium (15) comprising instructions which, when executed by a computer (10), cause the computer to perform the method (100) according to any one of claims 1 to 7.

Images