Method and apparatus for improving the training of a classifier

Abstract

The invention relates to a method and a device (100) for improving the training of a classifier (113). The device (100) for identifying an object (200) has a first image detection device (120) which is arranged in a first position for detecting the object (200) from a first viewing angle (121). The device also has a maintained first classifier which is arranged to identify the object based on the detected first viewing angle. According to the invention, at least one second image detection device is arranged in a second position for detecting the object from a second viewing angle different from the first viewing angle. The data processing device is also arranged to identify the object detected by means of the first image detection device based on the first classifier and to assign a class predetermined by the first classifier to the second viewing angle. As a result, the existing classifier can be expanded with respect to identifying objects from different additional viewing angles (131).

Description

Technical Field

[0001] The present invention generally relates to the field of artificial intelligence. In particular, the present invention relates to a method for training an artificial intelligence module (KI module), and the present invention also relates to a device for identifying objects and / or situations. Background Art

[0002] There are many technical applications in which objects and / or situations are to be recognized by machines, that is to say in particular automatically by data processing devices. This is usually done by feature recognition or pattern recognition, which is particularly important in the field of artificial intelligence. Exemplary applications are found in the search for images using keywords in Internet search engines, in the use of robots or in automotive technology, where computer-assisted driver assistance systems or autonomous vehicles require reliable object recognition in order to move the vehicle safely through traffic.

[0003] In order to recognize objects, different classification methods can be used, which are also called classifiers. Such classifiers are usually trained with a large amount of training data, so as to gradually develop recognition performance for the object to be recognized. However, the recognition performance of such a classifier is usually limited to the image information of the previous training data, which leads to a significant limitation of the recognition performance, especially when the viewing angle of the detection image is changed. This means that the classifiers, which are usually trained and guaranteed at high cost, can only be used for specific purposes. Therefore, new training data must be generated for new purposes and the classifier must be trained again. Summary of the invention

[0004] It is therefore an object of the present invention to provide a method for adapting existing classifiers to new purposes of use.

[0005] This object is achieved by a method for training an artificial intelligence module (KI module for short) and a device for recognizing an object. Advantageous embodiments of the invention are apparent from the description and the drawings.

[0006] The proposed method is suitable for training a KI module, which is, for example, a software module for a data processing device, a computer-supported control device of a motor vehicle, etc. The KI module can be configured to generate an output, for example, for a control device of a motor vehicle and to provide the output to the control device, which can, for example, determine an evasive maneuver and / or a braking maneuver based on the output. The control device can also cause the motor vehicle to perform an evasive maneuver and / or a braking maneuver by controlling an actuator, etc. For this purpose, the KI module can include a program code, in particular a multi-layer and / or folded artificial neural network (KNN).

[0007] The method according to the invention can be implemented, for example, in a computer-assisted manner in a data processing device having at least one memory unit and a computing unit, the method comprising the following steps:

[0008] First, an object to be recognized by the KI module is detected from a first viewing angle by a first image capture device, which is arranged in a first position. The object may be, for example, a traffic sign, a person, a vehicle, a static object (e.g., a house, a tree, a road marking, etc.) in road traffic. The image capture device may be a camera (of a driver assistance system, if necessary). The first position may be in or on a vehicle, wherein the first viewing angle may correspond to the viewing direction of the first image capture device from the first position to the object to be recognized.

[0009] Then, a first classifier is provided, which is configured to recognize an object based on the detected first viewing angle. A classifier can be understood as a function (e.g., an algorithm implemented in a program code) that assigns an object to a category based on its feature vector. Accordingly, classification or categorization can be understood as a process or method by which an object can be divided into types or categories. In order to set up a first classifier for recognition, the first classifier can be trained in advance by a large amount of training data, which, for example, consists of image label pairs detected from the first viewing angle.

[0010] According to the invention, the object to be recognized is then recognized based on the detected first perspective by a first classifier which is provided for this purpose or which has been trained to recognize objects.

[0011] The object is detected in advance, simultaneously or subsequently by a second image detection device from at least one second viewing angle different from the first viewing angle, the second image detection device being arranged in a second position. The second image detection device can also be a camera that detects the object to be identified from another viewing angle or viewing direction. In this case, one or more images can be recorded and, if necessary, stored. In addition, n image detection devices can be provided, which are used to detect the object from n different viewing angles.

[0012] Then, the label obtained by the recognition of the object by the first classifier is assigned to the second perspective (e.g., the image taken from this perspective). In this case, the label can be understood as the category attribution used for training data in machine learning. A classification identifier or machine-readable label.

[0013] By means of this method, a number of advantageous effects can be achieved simultaneously. Thus, for example, the recognition performance of an existing first classifier can be expanded with relatively little computational overhead in such a way that the first classifier can recognize one and the same object not only from a previously trained first perspective but also from a perspective different from the first perspective (i.e., a second perspective). Thus, objects can be recognized more reliably or more robustly in batch or production use (i.e., when running a search engine or in an autonomously driven vehicle). This can also significantly expand the application field of the classifier. In vehicle technology, the classifier expanded in this way can thus be used to detect objects starting from different vehicle types, so that it is no longer necessary to pay attention to the uniform first position of the first detection device. For example, in this way, a classifier trained for a passenger car (PKW) can be simply expanded for agricultural machines (e.g., tractors) that are significantly different in structure, in that, for example, the first image detection device is arranged in a first position, which corresponds to the first position of the passenger car, at least during the execution of the method described herein. Then, in batch or production use of the agricultural machine, the (e.g., also unique) image detection device can be arranged in a position corresponding to the second position, wherein objects can also be detected from this position by expanding the recognition performance of the classifier.

[0014] An advantageous further development of the invention provides that the label includes the class of the object known in advance by the first classifier. The class may be, for example: person, animal, plant, obstacle, road marking, traffic sign, etc. As a result, a persuasive training data set can be provided to the classifier, which, in addition to the image of the object, also contains the corresponding class.

[0015] In order to be able to extend the classifier by machine learning, a training data set for the KI module can be generated from images detected from the second viewing angle with assigned labels. Thus, the training data set can contain at least one image-label pair, which can be provided, for example, to the input layer of the KNN of the KI module as an input parameter. Thus, the recognition performance of the classifier can be extended with relatively little overhead.

[0016] In order to avoid the need to record each image-label pair or each training data set, for example, by means of a vehicle driving in or in the form of an image, it may be advantageous to generate or reconstruct a 3D model of the object by means of a plurality of detected viewing angles in the image acquisition device, and then generate or reconstruct further images of the object from different viewing angles from this 3D model. In this way, the object can be digitized by means of a plurality of image acquisition devices in a single recording situation in such a way that (similar to the universal derivation of a 2D drawing from a CAD-3D model) any viewing angle separated by the second image acquisition device in a computer-supported manner can be generated, which is different from the first viewing angle that has been trained. In other words, such an offline expansion of the classifier can be realized based on the 3D model of the object to be recognized. In this case, the viewing angle on the object can be changed in a very short time with a high degree of variability (for example also in the angle-minute range). As a result, the training expenditure can be significantly reduced again, but the recognition performance can still be improved due to the good quality of the training data.

[0017] Furthermore, it may be advantageous to assign a label to each additional image of the object generated from the 3D model, ie, for example, a class that is predetermined and assigned to the object, and thereby generate a training data set for the KI module. Thus, a high-quality training data set for the KI module can be generated quickly and easily.

[0018] As an alternative or in addition to offline training, the assignment of labels can also be performed during the continuous operation of the KI module and / or the image acquisition device (i.e., online). In this case, as clearly described, the first image acquisition device can mark or identify objects for the second image acquisition device, thereby enabling online recognition of objects with low computational overhead.

[0019] Another embodiment of the invention provides that one or more optically perceptible features of an object obtained from a first viewing angle are additionally assigned to the image captured from a second viewing angle. The features of an object can be general - for example, color, geometric shape or contour, rough texture, etc. With this additional information, a second or even further image capture device or a general KI module can recognize a potential unknown object from its viewing angle.

[0020] It has proven to be particularly advantageous to obtain one or more features of the object from the overlapping region of the first viewing angle and the second viewing angle. In this case, the overlapping region can be understood as a section detected on the image of the object that is detected not only by the first image capture device but also by the second image capture device, that is, an image of one image capture device can also be found again in sections in the other image capture device.

[0021] However, it is not necessary to compulsorily expand the maintained first classifier. Instead, a second classifier can be generated for identifying objects from a second perspective. Thus, for example, the data size of the first classifier can be kept very low. In addition, a specific classifier for a specific application can be maintained.

[0022] The invention also relates to a device for identifying an object or for training a KI module, wherein the device is particularly suitable for carrying out the above method and can be extended accordingly as described for the method. Conversely, the method can also be extended according to the following description. The device has:

[0023] A first image capture device (eg a camera) is arranged in a first position in order to detect an object from a first viewing angle.

[0024] A first classifier is maintained, the first classifier being configured to identify an object based on the detected first perspective.

[0025] According to the invention, the device further comprises at least one second image capture device which is arranged in a second position in order to capture the object from a second viewing angle which is different from the first viewing angle.

[0026] A data processing device, which has, for example, a processor, a memory, etc., is configured to perform training, for example, in the following aspects: recognizing objects captured by means of the first image capture device according to a first classifier and assigning a class predetermined by the first classifier to a second perspective.

[0027] With this configuration of the device, the advantages described above for the method can in principle be achieved. Thus, in particular, the recognition performance of existing classifiers can be expanded in a simple manner for new purposes of use.

[0028] One embodiment of the invention provides that the first position and the second position are geometrically matched to each other, and the first image capture device and the second image capture device are connected to each other for data exchange. In other words, the two image capture devices can be calibrated to each other.

[0029] In addition, the first position and the second position of the image capture device can also differ from each other in particular by the height and / or angle relative to the object. In other words, the image capture device can be arranged in another inner position or outer position that is higher or lower relative to the vehicle vertical axis and / or the vehicle transverse axis.

[0030] In order to be able to provide the second viewing angle with further optically perceptible features of the object, the first viewing angle and the second viewing angle may overlap in a common overlapping region.

[0031] Other measures to improve the present invention will be further illustrated below together with the description of the preferred embodiments of the present invention based on the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Embodiments of the present invention will be described in detail below with reference to the accompanying drawings. In the accompanying drawings:

[0033] Figure 1 According to one embodiment of the present invention, a device for identifying an object is shown;

[0034] Figure 2 A block diagram of a device for identifying an object is shown according to an embodiment of the present invention;

[0035] Figure 3 A block diagram of a device for identifying an object is shown according to another embodiment of the present invention;

[0036] Figure 4 According to another embodiment of the present invention, a device for identifying an object is shown;

[0037] Figure 5 According to one embodiment of the present invention, a flow chart of a method for training an artificial intelligence module according to the present invention is shown.

[0038] The figures are only schematic and not to scale. In the figures, identical, identical or similar elements generally have the same reference numerals. DETAILED DESCRIPTION

[0039] Figure 1 According to one exemplary embodiment, a device 100 is shown for detecting an object 200, wherein device 100 is integrated here by way of example into a vehicle 300. Vehicle 300 is designed here by way of example as a motor vehicle in the form of a passenger car.

[0040] The device 100 has a data processing device 110 in the form of an artificial intelligence module, which has a processor 111 and a memory 112 for storing program code to be processed by the processor 111, wherein the program code includes at least one first classifier 113. The device 100 also includes a first image acquisition device 120 in the form of a camera, which is arranged in a first position and coupled to the data processing device 110 for detecting an object 200 from a first viewing angle 121. The first position is here, for example, approximately at the height of the headlights of the motor vehicle 300, so that the first viewing angle 121 detects the object 200 frontally relative to the motor vehicle 300. It should be noted that the first classifier 112 has been trained from the first viewing angle 121, i.e., it has been set up to recognize the object 200 from the first viewing angle 121. Therefore, the classifier 113 is able to classify the object 200, i.e., to assign a certain class to the object.

[0041] Furthermore, the device 100 also includes a second image detection device 130, which is arranged in a second position and coupled to the data processing device 110 in order to detect the same object 200 from a second viewing angle 131 that is different from the first viewing angle 121. In the present embodiment, the second position is in the roof area of ​​the motor vehicle 300, that is, above the first position of the first image detection device 120 in the direction of the vertical axis of the vehicle. Therefore, the second image detection device 130 detects the object 200 from another height and another angle, which in the present embodiment corresponds to detection from obliquely above. The difference is that only a relatively small section of the object 200 is detected on the front side, so that the second image detection device cannot provide an image of the object 200 as an example: the image is sufficient for reliable recognition of the object 200. However, as Figure 1 As shown, first viewing angle 121 and second viewing angle 131 have a common overlapping region 140 , which is arranged here, by way of example, at the front of object 200 relative to motor vehicle 300 .

[0042] according to Figure 2 , which shows the device 100 in a block diagram, the functional interaction of the first image acquisition device 120 and the second image acquisition device 130 will now be explained functionally using the first classifier 113 .

[0043] Thus, the first classifier 113 is a functional part of the data processing device 110 and, as described above, is capable of recognizing the object 200 based on a first viewing angle 121 of the object 200 detected by the first image capture device 120 in the image 122 by means of training with the aid of a certain number of training data sets and assigning a relevant class in the form of a label 114 suitable for machine learning to the object, i.e., for example, identifying the object 200 as another motor vehicle. Furthermore, the data processing device 110 is configured to assign a label 114 to a second viewing angle 131 detected by the second image capture device 130 in the image 132, which is Figure 2 13 is illustrated by way of example by means of a data link 115. In this exemplary embodiment, the data link 115 starts from one or more images 132 of the second viewing angle 131 and leads to the data processing device 110 starting from the label 114 containing the category. Therefore, the data link 115 as an image-label pair constitutes a training data set for the data processing device 110 configured as a KI module.

[0044] Figure 3The device 100 is shown in a further embodiment in which, in order to further improve the recognition performance of the first classifier 113, characteristic features 141 of the object 200 from the overlapping area 140 of the two perspectives 121, 131 are additionally linked in the data link 115. In this case, the features 141 are, for example, the color of the object 200, its geometry and / or its roughness or shape.

[0045] exist Figure 4 , in which vehicle 300 is now implemented as an agricultural machine. Based on this embodiment, an advantageous application of device 100 is now explained. Device 100 of this embodiment is essentially the same as device 100 in the above-described embodiment, with the difference that additional n second image acquisition devices 130 are arranged on agricultural machine 300 with additional n different second viewing angles 131.

[0046] Agricultural machine 300 differs significantly from a passenger car in its design, in particular in its size, so that without further measures, classifier 113 trained for the above-mentioned motor vehicle cannot be used for agricultural machine 300. This is because due to the different designs, the detectable viewing angles of image capture devices 120, 130 differ significantly from the above-mentioned viewing angles for the passenger car, so that object 200 cannot be detected without further measures.

[0047] Therefore, the device 100 of this embodiment is again configured to use the first image detection device 120 to identify the object 200, for which purpose the first image detection device 120 (for example, via an auxiliary holding device, etc.) is approximately arranged in the following first position: the first position approximately corresponds to the first position of the vehicle 300 implemented as a sedan in the above-mentioned embodiment.

[0048] In order to expand the recognition performance of the classifier 113, as shown in FIG. Figure 2 and Figure 3 As explained, data processing device 110 generates a data link 115 for the (n+1)th second image capture device 130 from the nth image 122 of the nth second image capture device 130. As in the cascade, this process is carried out as described above for all other (n+1)th second image capture devices 130 in order to expand first classifier 113 in such a way that it ensures object 200 recognition for all n other second image capture devices 130.

[0049] Figure 5 A flow chart of a method according to the invention of a data processing device 110 embodied as a KI module is shown, wherein the method is carried out using the apparatus 100 .

[0050] In a first step S1, an object 200 is detected from a first viewing angle 121 by a first image detection device 120 arranged in a first position. In step S2, a first classifier 113 is provided, wherein the classifier is set or trained to recognize the object 200 and assign the object to a predetermined category. Then, in step S3, the object 200 is recognized by the first classifier 113 according to the detected first viewing angle 121. Then, in step S4, the object 200 is detected from a second viewing angle 131 different from the first viewing angle 121, for example, by a second image detection device 130 arranged in a second position. Then, in step S5, a label 114 obtained by the recognition of the object 200 by the first classifier 113 is assigned to the second viewing angle 131. In an optional step S6, a training data set for the KI module 110 is generated from the label 114 and the second image 132 of the second viewing angle 131, and the training data set is provided to the KI module. The method can be performed according to a reference of the data processing device 110. Figure 2 and 3 The described working method is expanded accordingly.

[0051] Starting from the embodiment shown, the device 100 according to the invention and the method according to the invention can be modified in many aspects. For example, instead of expanding the first classifier 113, a second or more further classifiers can also be generated, which can then be used specifically for a second or more further viewing angles.

Claims

1. A method for training an artificial intelligence module - a KI module - (110), the method comprising the following steps: A first image of an object (200) is detected (S1) from a first viewing angle (121) by a first image detection device (120), the first image detection device being arranged in a first position, and a first classifier (113) being provided, the first classifier being configured to identify the object (200) based on the first image detected from the first viewing angle (121), characterized in that the object (200) is identified (S3) by the first classifier (113) based on the first image detected from the first viewing angle (121), and the object (200) is identified (S4) by a second image detection device (130) from at least one first viewing angle different from the first viewing angle (121). The second image of the object (200) is detected (S4) from the second viewing angle (131), the second image detection device being arranged in a second position, and the label (114) obtained by identifying the object (200) by the first classifier (113) based on the first image detected from the first viewing angle (121) being assigned (S5) to the second image detected from the second viewing angle (131), thereby generating a training data set for the KI module (110), wherein the training data set comprises an image label pair, the image label pair comprising the second image detected from the second viewing angle (131) and the assigned label (114).

2. The method according to claim 1, characterized in that The label (114) includes the class of the object (200) known in advance by the first classifier (113).

3. The method according to any one of the preceding claims, characterized in that A 3D model of the object (200) is generated by means of a plurality of perspectives (121, 131) captured by the image capture device (120, 130), and further images (132) of the object (200) are then generated from the 3D model from different perspectives.

4. The method according to claim 3, characterized in that Each further generated image (132) of the object (200) is assigned the label (114), and a training data set for the KI module (110) is generated (S6) from this.

5. The method according to claim 1 or 2, characterized in that: The assignment (S5) of the label (114) is performed during the continuous operation of the KI module (110) and / or the image acquisition device (120, 130).

6. The method according to claim 1 or 2, characterized in that: At least one optically perceptible feature (141) of the object (200) obtained from the first perspective (121) is additionally assigned to the image (132) detected from the second perspective (131).

7. The method according to claim 6, characterized in that At least one feature (141) of the object (200) is obtained from an overlapping area (140) between the first viewing angle and the second viewing angle (131, 141).

8. The method according to claim 1 or 2, characterized in that: In order to perform recognition (S3) from the second viewing angle (131), a second classifier is generated.

9. A device (100) for identifying an object (200), the device comprising a first image detection device (120), the first image detection device being arranged in a first position in order to detect a first image of the object (200) from a first viewing angle (121), a first classifier (113) being maintained, the first classifier being configured to identify the object (200) based on the first image detected from the first viewing angle (121), characterized in that At least one second image detection device (130), arranged in a second position for detecting a second image of the object (200) from a second viewing angle (131) different from the first viewing angle (121), a data processing device (110), configured to identify the object (200) detected by means of the first image detection device (120, 130) according to the first classifier (113), and to assign a category predetermined by the first classifier (113) according to the first image detected from the first viewing angle (121) to the second image detected from the second viewing angle (131), the data processing device (110) also configured to generate a training data set for a KI module (110), the training data set comprising an image label pair, the image label pair comprising a second image detected from the second viewing angle (131) and an assigned label (114).

10. The device (100) according to claim 9, characterized in that The first position and the second position are geometrically matched to each other, and the first image detection device (120) and the second image detection device (130) are connected to each other for data exchange.

11. The device (100) according to claim 9 or 10, characterized in that The first position and the second position of the image detection device (120, 130) differ from each other by a height and / or an angle relative to the object (200).

12. The device (100) according to claim 9 or 10, characterized in that The first viewing angle and the second viewing angle (121, 131) overlap in an overlapping area (140).

Images