Method and device for ascertaining object detection of image

Abstract

To relate to a computer-implemented method (100) for ascertaining an output signal, which characterizes an object detection of an object of an image.SOLUTION: A method includes: a. a step (101) of ascertaining a plurality of object detections with respect to an image; b. a step (102) of ascertaining a graph based on the plurality of object detections, the object detections of the plurality of object detections being characterized by nodes of the graph and overlaps between two object detections each being characterized by edges of the graph; c. a step (103) of ascertaining a cluster of the graph based on the nodes and on the edges of the graph with the aid of a density-based clustering method; d. and a step (104) ascertaining an object detection based on the cluster and providing the object detection in an output signal.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] Conventional technology Radosavovic et al.'s paper, "Data Distillation: Towards Omni-Supervised Learning," 2018, Conference on Computer Vision and Pattern Recognition (CVPR), discloses a method for identifying object detection parts in images. [Background technology]

[0002] Background of the Invention Machine learning systems are increasingly being used to detect objects in images. The most high-performing of these systems—that is, those with the best detection capabilities—are typically supervised learning systems, trained on annotated images. Training such machine learning systems requires a large number of annotated images for training, because typically, the performance of a machine learning system increases with the number of annotated images during training.

[0003] Annotating objects in images is a time-consuming and costly undertaking, making semi-supervised learning training methods for machine learning systems increasingly important. In this type of training, both annotated and unannotated images are fed to the machine learning system. In this case, it is desirable to form the annotations as accurately as possible. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Radosavovic et al., “Data Distillation: Towards Omni-Supervised Learning”, 2018, Conference on Computer Vision and Pattern Recognition (CVPR) [Overview of the project] [Means for solving the problem]

[0005] Disclosure of the invention Given this background, the present invention provides a computer-implemented method for identifying an output signal that characterizes an object detection unit of an object in an image, a. A step of identifying multiple object detection units with respect to the image, b. A step of identifying a graph based on multiple object detection units, wherein the object detection units in the multiple object detection units are characterized by nodes in the graph, and the overlap between each of two object detection units is characterized by edges in the graph. c. A step of identifying clusters in the graph based on the nodes and edges of the graph using a density-based cluster analysis method, d. The steps of identifying one object detection unit based on the cluster and providing the said object detection unit in the output signal, Regarding methods including

[0006] The identified output signal can be understood as an annotation characterizing one or more object detection areas in the image. The image in this case may be recorded by an optical sensor, for example, a camera sensor, LiDAR sensor, radar sensor, ultrasonic sensor, or thermal camera.

[0007] In particular, steps c. and d. of this method can be performed multiple times in order to provide multiple object detection units in the output signal.

[0008] The object detection unit can be characterized, in particular, by a bounding box (English: Bounding Box). Therefore, the output signal can particularly include one or more bounding boxes. However, it is also possible to characterize the object detection unit generally by a continuous region, for example, by a polygon.

[0009] To identify multiple object detection units for an image, in particular, a pre-trained object detector can be used. For example, as part of a semi-supervised learning method (English: semi-supervised learning), annotate the image, and it is possible to identify multiple object detection units by a pre-trained object detector for the already annotated image. Generally, the object detector can particularly include a neural network, and based on the output of the neural network, the object detection unit can be identified. It is also possible to use the neural network as the object detector.

[0010] The object detector can be configured, in particular, to process an image and predict multiple object detection units. The object detector is preferably configured to identify, for each bounding box, a confidence value indicating that the bounding box is the bounding box of an object in the image. This confidence value can be, for example, a percentage number.

[0011] In an advantageous embodiment of the method, it is possible to identify multiple object detection units based on multiple different pre-processings of the image.

[0012] Image preprocessing can be configured, for example, so that the image is scaled according to various scales, and / or so that the image is horizontally flipped, and / or so that the edges of the image are cropped. One or more of these preprocessing methods can be used to identify the preprocessed image. In this case, multiple object detection units can be combined, particularly from an object detection unit for the image and an object detection unit for the preprocessed image. Preferably, multiple preprocessed images can be identified by various applications or parameterizations of the preprocessing method, and the object detection units for these multiple preprocessed images can be used as part of multiple object detection units.

[0013] In particular, to enable filtering of object detection areas relating to the same object when using various pre-processed image variations, this method advantageously employs density-based cluster analysis. The objective of this method is understood to be to group together object detection areas that characterize sufficiently similar locations and areas of objects in the image. Advantageously, in the proposed method, density-based cluster analysis is used to identify clusters of object detection areas, rather than non-maximum suppression or bounding box voting. The inventors have found that the multiple object detection areas identified in this way are more precise than those using one of the other methods.

[0014] To perform cluster analysis on object detection units, a graph can be identified from the multiple object detection units identified. For this purpose, each object detection unit in the multiple object detection units can be understood as a node in the graph, where nodes are connected via edges, and the weights of the edges characterize the overlap between nodes connected by edges (and consequently, the overlap between object detection units). As an indicator of overlap, a metric such as the ratio of intersection over union (IoU) between nodes connected by edges, and consequently between object detection units, can be used.

[0015] In the following, the terms "node" and "object detection unit characterized by the node" can be understood as synonymous. In particular, the confidence level identified by the object detector with respect to the object detection unit can be understood as the confidence level of the node.

[0016] In this method, if the overlap between two object detection units is below a predetermined threshold, it can be assumed that there is no edge between the two object detection units in the graph. It is also possible to include only object detection units whose confidence level, as identified by the object detector, exceeds a predetermined threshold in the graph.

[0017] Preferably, this method makes it possible to annotate objects of various classes on an image. In this case, one separate graph is created for each class, and a density-based cluster analysis method can be performed for each of the graphs thus identified. This can be understood as considering only the object detection units belonging to the same class among multiple object detection units in the graph.

[0018] Preferably, the cluster analysis method is performed based on a cluster density measure and a cluster connectivity measure.

[0019] The advantage of this approach is that the number of clusters, and therefore the number of objects in the image, does not need to be known, as this method can automatically identify them. This further improves the accuracy of the object detection section of the output signal.

[0020] The results of applying cluster analysis can be one or more clusters. Each cluster preferably includes multiple object detection units (e.g., multiple bounding boxes). A cluster may include a cluster center in particular. Preferably, a node can characterize the cluster center. Preferably, the nodes of a cluster characterize the center with the highest confidence among the confidence levels of the cluster nodes. When clusters are identified by cluster analysis, the center can be provided as an object detection unit in the output signal. It is also possible to merge the nodes of a cluster in a weighted manner according to the confidence level of that node. For example, the object detection units of a cluster can be characterized by the center point, height, and width of the bounding box, respectively. The center point, height, and width can be represented in particular as vectors, and the merging of object detection units can be performed by weighted addition of the vectors. In particular, to identify object detection units based on clusters, the result of the addition can be divided by the sum of the confidence levels of the cluster nodes.

[0021] In a further embodiment of this method, instead of the center point, another point on the bounding box, such as a corner of the bounding box, can be selected.

[0022] However, the inventors have discovered that it may be perfectly appropriate not to provide an object detection unit in the output signal that contains only one object detection unit in a cluster. By excluding such a center, the accuracy of the object detection unit in the output signal can be improved.

[0023] In a preferred embodiment of the method for identifying the output signal, a density-based cluster analysis method is used. e. The steps include identifying the starting node based on the sum of the edge weights of each node in the graph and providing the starting node as a node in the cluster, f. The step of identifying the graph node that is not included in the cluster and has the most connections to the cluster, g. The step of adding a node to a cluster if the cluster density measure expanded by the addition of the node does not fall below a predetermined first threshold, and the cluster connectivity measure expanded by the addition of the node does not fall below a predetermined second threshold. h. Otherwise, the step of providing a cluster, It may include.

[0024] To identify the starting node, we can first determine the sum of the weights of the edges connected to each node in the graph. This allows us to identify one exponent for each node, and the node with the highest exponent is used as the starting node. Furthermore, the starting node can be understood as the center of a cluster, and this cluster can be expanded in subsequent steps of the cluster analysis method by adding more nodes.

[0025] Next, the graph node that has the largest numerical connection to the cluster and is not yet part of the cluster is identified. The connection between a node and the cluster can be understood as identifying the edge with the largest weight among all the edges connecting that node to the cluster nodes. Subsequently, the weight of this edge can be used as the connection between the node and the cluster. Then, among the nodes that do not belong to the cluster, the node with the largest connection, i.e., the node with the largest numerical connection to the cluster, can be identified.

[0026] A node can be added to a cluster if the density measure of the cluster expanded by the addition of that node does not fall below a predetermined first threshold, and the connectivity measure of the cluster expanded by the addition of that node does not fall below a predetermined second threshold. This step can be understood as first examining how much impact the inclusion of a node in the cluster may have on its effects. The density measure and the connectivity measure can be used as indices to determine this effect.

[0027] The density measure can be understood as a characteristic that guarantees that, after incorporating a node, all nodes in the cluster remain sufficiently connected to one another. In this case, whether they are sufficiently connected is checked using a first threshold. Preferably, the density measure is given by the following formula d Z =2·(W Z +W Zk ) / n·(n+1) It can be characterized by, where W Z This characterizes the sum of all weights of the edges between nodes in cluster Z, and W Zk n is the sum of all weights of the edges from node to node in the cluster, and n is the number of nodes in the cluster, i.e., the number of nodes in the cluster before any nodes were potentially incorporated into the cluster.

[0028] The connectivity metric can be understood as characterizing whether, upon inclusion of a node, it is guaranteed that the node itself will have sufficient connectivity with the nodes in the cluster. In this case, whether it is sufficiently connected is checked using a second threshold. Preferably, the connectivity metric is given by the following formula cp Z =W Zk / d Z (n+1) It can be characterized by the following.

[0029] Preferably, in this method, nodes can be iteratively identified in step f. and added to a cluster until at least one of the conditions in step g. is no longer met. If at least one of the conditions is no longer met, a cluster is provided and nodes of the cluster are removed from the graph.

[0030] This can be understood as a greedy algorithm gradually identifying clusters. In each iteration, a new starting node is selected first, and then nodes are iteratively added based on this starting node to form clusters. In particular, steps e through h can be repeated iteratively until no more nodes exist in the graph. The inventors have found that, unlike other methods, such as those based on non-maximum suppression methods, the object detection provided in the clusters and thus the output signal is more accurate.

[0031] Furthermore, object detection units identified based on clusters can be excluded from the output signal if the cluster consists only of the start node. The inventors were able to confirm that clusters consisting of only one object detection unit are typically due to false positives in the object detector. Therefore, further improvement in results can be achieved by excluding clusters that contain only one node, i.e., only the start node.

[0032] Generally, a graph can include all classes of object detection units that can be recognized by an object detector. For example, an object detector can be configured to recognize vehicles, pedestrians, cyclists, and traffic lights in an image of the area surrounding a vehicle. However, it is also preferable to specify the graph in terms of a class of object detection units. In other words, when specifying a graph, only object detection units belonging to the same class can be considered. Therefore, in the above example, a graph relating to an object detection unit of the vehicle class, a graph relating to an object detection unit of the pedestrian class, a graph relating to an object detection unit of the cyclist class, and a graph relating to an object detection unit of the traffic light class can be specified, respectively.

[0033] In particular, for each class of object that can be classified by the object detector, it is advantageous that the method described above in one of the features described above can be implemented for each of the classes of objects that can be classified by the object detector. Subsequently, the object detection unit for the various different classes thus identified can be provided in the output signal.

[0034] In a further embodiment, the present invention relates to a computer-implemented method for training a machine learning system, i. One embodiment of a method for identifying an output signal includes the steps of identifying an output signal relating to an image, j. A step of training a machine learning system, wherein an image is used as the input signal to the machine learning system, and an identified output signal is used as the desired output signal. Regarding methods including

[0035] The training method can be used, in particular, as part of a semi-supervised training method. Specifically, the machine learning system may be an object detector that identifies multiple object detection units. In this case, the machine learning system can be further trained based on the identified output signals. Preferably, this step is performed multiple times for each different image. This makes it possible to annotate the image dataset, i.e., to enrich the images in the dataset with the desired output signals that the machine learning system should predict for those images. In this case, the machine learning system can be trained on the images in the dataset and on the images that were originally annotated for training purposes, in combination with the identified output signals. Self-supervised training improves the prediction accuracy of the machine learning system.

[0036] Furthermore, it is possible to identify a first output signal using a method for identifying output signals for at least one image, identify a second output signal using a second machine learning system, and then check whether the first output signal matches the second output signal. In other words, the second machine learning system can be tested using the method for identifying output signals. Conventional measures can be used to check for a match, for example, mean average precision.

[0037] Embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. [Brief explanation of the drawing]

[0038] [Figure 1] This diagram provides a schematic flow chart of the method for identifying annotations. [Figure 2] This diagram schematically shows a training system for training a classifier. [Figure 3] This diagram schematically shows the structure of a control system for driving an actuator. [Figure 4] This diagram schematically illustrates an embodiment for controlling at least a semi-autonomous robot. [Figure 5] This diagram schematically illustrates an example of controlling a manufacturing system. [Figure 6] This diagram schematically illustrates an example of controlling an access system. [Figure 7] This diagram schematically illustrates an example of controlling a monitoring system. [Figure 8] This diagram schematically illustrates an example of controlling a personal assistant. [Figure 9] This figure schematically illustrates an example of controlling a medical imaging system. [Modes for carrying out the invention]

[0039] Description of the Examples Figure 1 schematically shows the flow of a method (100) for identifying annotations, in which annotations for multiple images are iteratively identified, and these annotations characterize the object detection units of the objects in the images. Preferably, the images and identified annotations are provided as a training dataset (T) at the end of the method, for example, when all images or a desired number of images have been annotated.

[0040] In the first step (101), multiple object detection units are identified with respect to an image. For this purpose, an object detector can be used in particular, which is configured to identify object detection units of objects imaged in the image. Advantageously, the object detector is further configured to identify, for each object detection unit, the probability that each object detection unit is characterizing an object in the image. The image is preferably processed by a preprocessing method, and thus a preprocessed image is identified. The object detector is applied to the preprocessed image, and the object detection units thus identified are incorporated together into the multiple object detection units. If the preprocessing characterizes a mirror image of the image, the object detection units identified with respect to the preprocessed image can preferably be mirrored back before they are incorporated into the multiple object detection units. Preferably, multiple preprocessed images can be identified using different preprocessing methods and / or parameterization of different preprocessing methods, and these preprocessed images are each processed by an object detector, and the resulting object detection units are incorporated into the multiple object detection units.

[0041] In the second step (102) of method (100), a graph is identified based on a plurality of object detection units, where the object detection units in the plurality of object detection units are characterized by nodes in the graph, and the overlap between each pair of object detection units is characterized by edges in the graph. Thus, there is a one-to-one relationship between each node in the graph and each of the plurality of object detection units. Such a one-to-one relationship can also be understood as each node corresponding to each object detection unit. Preferably, each node in the graph can be connected to each other in the graph by edges. In particular, the measure of the overlap between object detection units to which the corresponding nodes are connected by edges can be used as the weight of the edges. Preferably, edges in the graph can also be inserted only if the weight of the edge reaches or exceeds a predetermined threshold. Furthermore, the graph can be identified with respect only to the class of object detection units.

[0042] In the third step (103), clusters are identified based on the nodes and edges of the graph using a density-based clustering method. In a preferred embodiment, the third step (103) includes a first substep (103a), in which the starting node of a cluster is first identified. To identify the starting node, the following methods can be preferably selected: First, for each node in the graph, the sum of the weights of the edges connected to that node is identified. This identifies one exponent for each node. The node with the numerically largest exponent is then selected as the starting node.

[0043] Next, in the second sub-step (103b) of the third step (103), it is possible to identify a node of the graph that is not yet included in the cluster and has the maximum connection to the cluster. The connection between a node and a cluster can be understood as, in particular, for a node, among all the edges connecting that node to the nodes of the cluster, the edge with the maximum weight is identified. Subsequently, the weight of this edge can be used as the connection between the node and the cluster. Then, among the nodes not belonging to the cluster, it is possible to identify the node with the maximum connection, that is, the node with the numerically maximum connection to the cluster.

[0044] In the third sub-step (103c), for the node identified in the second sub-step, it is possible to check whether the density measure of the cluster extended by that node due to the addition of that node is less than a predetermined first threshold. Further, it is possible to check whether the connection measure of the cluster extended by that node due to the addition is less than a predetermined second threshold. If an affirmative answer can be given to both checks, then that node can be added to the cluster. The third sub-step (103c) can be understood as first examining how much impact the incorporation of a node into the cluster may have on the action on that cluster. The density measure and the connection measure can be used as indices for determining this action.

[0045] The density measure can be understood as characterizing whether, by incorporating a node, it is ensured that all the nodes of the cluster are still sufficiently connected to each other after the incorporation. In this case, whether they are sufficiently connected is checked using the first threshold. Preferably, the density measure can be characterized by the following formula d Z =2·(W Z +W Zk ) / n·(n + 1) where W ZThis characterizes the sum of all weights of the edges between nodes in cluster Z, and W Zk n is the sum of all weights of the edges from node to node in the cluster, and n is the number of nodes in the cluster, i.e., the number of nodes in the cluster before any nodes were potentially incorporated into the cluster.

[0046] The connectivity metric can be understood as characterizing whether, upon inclusion of a node, it is guaranteed that the node itself will have sufficient connectivity with the nodes in the cluster. In this case, whether it is sufficiently connected is checked using a second threshold. Preferably, the connectivity metric is given by the following formula cp Z =W Zk / d Z (n+1) It can be characterized by the following.

[0047] If a node is incorporated into the cluster, this method preferably allows us to return to the second substep (103b), identify the new node in the graph, and check whether this new node should also be incorporated into the cluster, as described above.

[0048] The loop formed by the return in method (100) can be carried out until any nodes that are not to be included in the cluster are identified, or until there are no further nodes in the graph that are not yet included in the cluster. In this case, in the fourth substep (103d), all nodes that are included in the cluster can be removed from the graph to provide the cluster. If there are still nodes in the graph, preferably the process can be returned to the first substep to identify a new cluster. Once the graph is empty, the third step (103) can be terminated.

[0049] Next, in the fourth step (104), based on the clusters identified in the third step (103), one object detection unit can be identified for each cluster. To this end, object detection units corresponding to the nodes of the clusters can be merged. Advantageously, this merging can be performed by a weighted sum of the representations of the object detection units. The representation may be, for example, a vector representation of the object detection unit. For example, the center point, height, and width can be represented via a vector, in which case the vector is a representation of a box which is the object detection unit. In this case, this merging can be performed using a weighted sum of the vectors. For example, the vectors can be weighted according to the confidence level of the object detection unit corresponding to each vector. Preferably, to identify the object detection unit for a cluster, the result of this sum can be divided by the sum of the confidence levels.

[0050] In this way, one or more object detection units can be identified for the clusters identified in the third step (103). If images still need to be annotated, the process proceeds back to the first step (101) after the fourth step (104), using images that have not yet been annotated. Once all or any desired number of images have been annotated, these images and their corresponding annotations can be provided as a training dataset (T).

[0051] Figure 2 shows an example of a training system (140) for training a classifier (60) of a control system (40) using a training dataset (T). The training dataset (T) consists of multiple input signals (x) used to train the classifier (60). i The training dataset (T) includes, and each of the input signals (x) is further divided into two parts. i ) for one desired output signal (t i ) includes this output signal (t i ) is the input signal (x i It supports the input signal (x i This characterizes the classification of ).

[0052] For training, the training data unit (150) accesses a computer implementation database (St2), which provides a training dataset (T). The training data unit (150) selects, preferably randomly, at least one input signal (x) from the training dataset (T). i ) and this input signal (x i ) corresponds to the desired output signal (t i ) and identify the input signal (x i The input signal (x) is transmitted to the classifier (60). The classifier (60) processes the input signal (x). i Based on ) the output signal (y i Identify )

[0053] Desired output signal (t i ) and the identified output signal (y i This is transmitted to the change unit (180).

[0054] Next, the desired output signal (t i ) and the identified output signal (y i Based on this, the change unit (180) determines a new parameter (Φ') for the classifier (60). For this purpose, the change unit (180) uses a loss function to determine the desired output signal (t i ) and the identified output signal (y i ) is compared with the specified output signal (y i ) is the desired output signal (t i A first loss value is identified that characterizes the degree of deviation from ). In this embodiment, the negative log-likelihood function is selected as the loss function. Other loss functions may be considered in alternative embodiments.

[0055] Furthermore, the identified output signal (y i ) and the desired output signal (t i) can be assumed to include, for example, multiple sub-signals in the form of a tensor, and the desired output signal (t i Each of the sub-signals of ) is the identified output signal (y i ) corresponds to one of the sub-signals. For example, classifier (60) is configured for object detection, and the first sub-signal corresponds to each of the input signals (x i The identified output signal (y) characterizes the probability of an object occurring for a portion of it, and it can be assumed that the second sub-signal characterizes the precise location of the object. i ) and the desired output signal (t i If the first signal includes multiple corresponding sub-signals, preferably a second loss value is determined for each corresponding sub-signal using an appropriate loss function, and these determined second loss values ​​are appropriately merged for the first loss value, for example, via a weighted sum.

[0056] The change unit (180) identifies a new parameter (Φ') based on the first loss value. In this embodiment, this is done by gradient descent, preferably stochastic gradient descent, Adam, or AdamW. In further embodiments, training may be based on an evolutionary algorithm or second-order optimization.

[0057] The identified new parameter (Φ') is stored in the model parameter memory (St1). Preferably, the identified new parameter (Φ') is supplied to the classifier (60) as parameter (Φ).

[0058] In a further preferred embodiment, the training described above is repeated iteratively for a predetermined number of iteration steps, or until the first loss value falls below a predetermined threshold. Alternatively or additionally, training may be terminated when the average first loss value on the test dataset or validation dataset falls below a predetermined threshold. In at least one of the multiple iterations, the new parameter (Φ') determined in the previous iteration is used as the parameter (Φ) of the classifier (60).

[0059] Furthermore, the training system (140) may include at least one processor (145) and at least one machine-readable storage medium (146), the at least one machine-readable storage medium (146) containing instructions, which, when executed by the processor (145), cause the training system (140) to perform a training method according to one aspect of the present invention.

[0060] Figure 3 shows a control system (40) for controlling an actuator (10) in an environment (20) of the actuator (10) based on an output signal (y) of a classifier (60). The environment (20) is preferably detected at regular time intervals by a sensor (30), particularly an imaging sensor such as a camera sensor, and this sensor (30) can be provided by multiple sensors, for example, a stereo camera. The sensor signals (S) of the sensor (30)—or, if there are multiple sensors, one sensor signal (S) from each—are transmitted to the control system (40). Thus, the control system (40) receives a sequence of sensor signals (S). The control system (40) identifies a drive signal (A) from this sequence of sensor signals (S), and this drive signal (A) is transmitted to the actuator (10).

[0061] The control system (40) receives a sequence of sensor signals (S) from the sensor (30) in an optional receiving unit (50), and the receiving unit (50) converts the sequence of sensor signals (S) into a sequence of input signals (x) (alternatively, each sensor signal (S) can also be received directly as input signals (x)). The input signals (x) may be, for example, a part of or subsequent processing of the sensor signals (S). In other words, the input signals (x) are determined depending on the sensor signals (S). The sequence of input signals (x) is supplied to the classifier (60).

[0062] The classifier (60) is preferably parameterized by parameters (Φ), which are stored in a parameter memory (P) and supplied by the parameter memory (P).

[0063] The classifier (60) identifies an output signal (y) from an input signal (x). The output signal (y) is supplied to an optional deformation unit (80), which identifies a drive signal (A) from this output signal (y), and this drive signal (A) is supplied to the actuator (10) to drive the actuator (10) accordingly.

[0064] The actuator (10) receives a drive signal (A), is driven accordingly, and performs a corresponding action. In this case, the actuator (10) may include drive logic (which is not necessarily structurally integrated), the drive logic identifies a second drive signal from the drive signal (A), and the actuator (10) is then driven by this second drive signal.

[0065] In further embodiments, the control system (40) includes a sensor (30). In yet another embodiment, the control system (40) also includes an actuator (10) as an alternative or additional measure.

[0066] In a further preferred embodiment, the control system (40) includes at least one processor (45) and at least one machine-readable storage medium (46) on which instructions are stored, and these instructions cause the control system (40) to carry out the method according to the present invention when executed by at least one processor (45).

[0067] In an alternative embodiment, a display unit (10a) is provided instead of or in addition to the actuator (10).

[0068] Figure 4 shows how the control system (40) can be used to control at least a semi-autonomous robot, in this case at least a semi-autonomous automobile (100).

[0069] The sensor (30) may, for example, preferably be a video sensor located inside a vehicle (100). The input signal (x) can be understood as an input image in this case, and the classifier (60) can be understood as an image classifier.

[0070] The image classifier (60) is configured to identify recognizable objects on the input image (x).

[0071] Preferably, the actuator (10) located inside the automobile (100) may be, for example, the brake, drive unit, or steering unit of the automobile (100). In that case, a drive signal (A) can be specified to drive the actuator (10) so that the automobile (100) drives to prevent a collision with an object identified by, for example, an image classifier (60), particularly when the object is an object of a predetermined class, such as a pedestrian.

[0072] Alternatively or additionally, the display unit (10a) can be driven by a drive signal (A) to display, for example, an identified object. The display unit (10a) can also be driven by the drive signal (A) to output an optical or acoustic warning signal if it is determined that there is a risk of collision between one of the identified objects and the automobile (100). Warnings using warning signals can also be implemented using tactile warning signals, for example, by vibration of the automobile's (100) steering wheel.

[0073] Alternatively, the at least semi-autonomous robot may be another mobile robot (not shown), for example, a robot that moves forward by flying, swimming, diving, or walking. The mobile robot may be, for example, an at least semi-autonomous lawnmower or an at least semi-autonomous cleaning robot. In such cases as well, the drive signal (A) can be specified so that the drive unit and / or steering unit of the mobile robot are driven so that the at least semi-autonomous robot prevents collisions with objects identified, for example, by an image classifier (60).

[0074] Figure 5 shows an embodiment in which a control system (40) is used to drive the manufacturing machine (11) of the manufacturing system (200) by driving an actuator (10) that controls the manufacturing machine (11). The manufacturing machine (11) may be, for example, a machine for punching, sawing, drilling, and / or cutting. Furthermore, the manufacturing machine (11) may be configured to grip the manufactured products (12a, 12b) using a gripper.

[0075] In this case, the sensor (30) may be, for example, a video sensor that detects the transport surface of the conveyor belt (13), and manufactured products (12a, 12b) can be placed on the conveyor belt (13). In this case, the input signal (x) is an input image (x), and the classifier (60) is an image classifier. The image classifier (60) can be configured, for example, to identify the position of the manufactured products (12a, 12b) on the conveyor belt. The actuator (10) that controls the manufacturing machine (11) can then be driven depending on the position of the identified manufactured products (12a, 12b). For example, the actuator (10) can be driven to punch, saw, puncture, and / or cut the manufactured products (12a, 12b) at predetermined locations on the manufactured products (12a, 12b).

[0076] Furthermore, the image classifier (60) may be configured to identify additional characteristics of the manufactured products (12a, 12b) in place of or in addition to their position. In particular, the image classifier (60) may be configured to identify whether the manufactured products (12a, 12b) have defects and / or damage. In this case, the actuator (10) can be driven so that the manufacturing machine (11) sorts out the manufactured products (12a, 12b) that have defects and / or damage.

[0077] Figure 6 shows an embodiment in which a control system (40) is used to control an access system (300). The access system (300) may include physical access control, such as a door (401). The sensor (30) may be a video sensor or a thermographic sensor configured to detect an area in front of the door (401). Thus, the classifier (60) can be understood as an image classifier. The detected image can be interpreted using the image classifier (60). In particular, the image classifier (60) can detect people on an input image (x) transmitted to the image classifier (60). If multiple people are detected simultaneously, their identities can be particularly reliably identified by correlating these people (i.e., objects) with each other, for example by analyzing the actions of these people.

[0078] The actuator (10) may be a lock that releases or de-releases access control depending on a drive signal (A), for example, a lock that opens or closes a door (401). For this purpose, the drive signal (A) can be selected depending on an output signal (y) identified for an input image (x) using an image classifier (60). For example, the output signal (y) may include information characterizing the identity of a person detected by the image classifier (60), and the drive signal (A) may be selected based on the identity of that person.

[0079] Instead of physical access control, logical access control can also be implemented.

[0080] Figure 7 shows an embodiment in which a control system (40) is used to control a monitoring system (400). This embodiment differs from the embodiment shown in Figure 4 in that a display unit (10a) driven by the control system (40) is provided instead of an actuator (10). For example, a sensor (30) can record an input image (x) in which at least one person is to be recognized, and the position of at least one person can be detected using an image classifier (60). The input image (x) can then be displayed on the display unit (10a), in which case the detected person can be displayed with color emphasis.

[0081] Figure 8 shows an embodiment in which a control system (40) is used to control a personal assistant (250). The sensor (30) is preferably an optical sensor, such as a video sensor or a thermographic camera, that receives images of the user's (249) gestures. In this case, the classifier (60) is an image classifier.

[0082] The control system (40) identifies a drive signal (A) for the personal assistant (250) depending on the signal from the sensor (30), for example, by having the image classifier (60) perform gesture recognition. This identified drive signal (A) is then transmitted to the personal assistant (250), causing the personal assistant (250) to perform the appropriate drive. The identified drive signal (A) can be selected in particular to correspond to a presumed desired drive by the user (249). This presumed desired drive can be identified depending on the gesture recognized by the image classifier (60). The control system (40) can then select a drive signal (A) to transmit to the personal assistant (250) depending on the presumed desired drive, and / or select a drive signal (A) to transmit to the personal assistant (250) in accordance with the presumed desired drive.

[0083] Such corresponding actions may include, for example, a personal assistant (250) retrieving information from a database and playing this information back for the user (249) to receive.

[0084] Instead of a personal assistant (250), it is also possible to provide and operate a household appliance (not shown), particularly a washing machine, stove, oven, microwave oven, or dishwasher.

[0085] Figure 9 shows an embodiment in which a control system (40) is used to control a medical imaging system (500), such as an MRT device, an X-ray device, or an ultrasound device. A sensor (30) can be provided, for example, by an imaging sensor. Thus, the classifier (60) can be understood as an image classifier. The control system (40) drives the display unit (10a).

[0086] The sensor (30) is configured to identify images of a patient, such as X-ray images, MRT images, or ultrasound images. At least a portion of the images is transmitted to the image classifier (60) as an input image (x). The image classifier (60) can be configured, for example, to classify different types of tissue to be recognized on the input image (x) by semantic segmentation.

[0087] Next, a drive signal (A) can be selected so that the identified type of tissue is displayed with color emphasis on the display unit (10a).

[0088] In further embodiments (not shown), the imaging system (500) can also be used for non-medical purposes, for example, to identify the material properties of a workpiece. For this purpose, the imaging system (500) can record an image of the workpiece. The image classifier (60) can be configured to receive at least a portion of the image as an input image (x) and classify it with respect to the material properties of the workpiece. This can be done, for example, by semantic segmentation of the input image (x). The classification thus identified can be displayed, for example, on a display device (10a) together with the input image.

[0089] The term "computer" includes any device for processing a given set of computational rules. These computational rules may exist in the form of software, in the form of hardware, or in a combination of software and hardware.

[0090] Generally, a plural can be understood as having an index; that is, each element in the plural is assigned a unique index, preferably by associating consecutive integers with the elements contained within the plural. Preferably, if the plural contains N elements and N is the number of elements in the plural, these elements are assigned integers from 1 to N.

Claims

1. 1. A computer-implemented method (100) for identifying an output signal characterizing an object detector of an object in an image, comprising: a. identifying (101) a plurality of object detectors for the image; b. identifying (102) a graph based on the plurality of object detectors, wherein each object detector in the plurality of object detectors is characterized by a node of the graph, and an overlap between each two object detectors is characterized by an edge of the graph; c. identifying clusters of the graph based on the nodes and edges of the graph using a density-based cluster analysis method (103); d. identifying an object detector based on the cluster and providing the object detector in the output signal (104); A method (100) comprising:

2. The cluster analysis method is performed based on a density measure of the clusters and on a connectivity measure of the clusters. The method (100) of claim 1.

3. The cluster analysis method includes: e. identifying a starting node based on the sum of edge weights of each of said nodes in said graph and providing said starting node as a node of said cluster (103a); f. Identifying (103b) nodes of said graph that are not included in said cluster and that have the maximum connection to said cluster; g. adding the node to the cluster if the density measure of the cluster expanded by the addition of the node does not fall below a predetermined first threshold and the connectivity measure of the cluster expanded by the addition of the node does not fall below a predetermined second threshold; h. Otherwise, providing said cluster as the identified cluster (103d); The method (100) of claim 2, comprising:

4. In step f., nodes are iteratively identified and added to the cluster until at least one of the conditions in step g. is no longer satisfied; If the at least one condition is no longer satisfied, the cluster is provided and the node of the cluster is removed from the graph. The method (100) of claim 3.

5. Steps e. through h. are repeated iteratively until no more nodes are contained in the graph. The method (100) of claim 4.

6. the object detector identified based on the cluster is not provided in the output signal if the cluster consists of only the starting node; The method (100) of claim 4.

7. The plurality of object detectors are identified based on a plurality of different pre-processing steps of the image. The method (100) of claim 1.

8. The graph is specified with respect to a class of object detectors. The method (100) of claim 1.

9. 1. A computer-implemented method for training a machine learning system (60), comprising: i. The method of claim 1, ｉ ) output signal (t ｉ ) j. Training the machine learning system (60), ｉ ) is used as an input signal for the machine learning system, and the determined output signal (t ｉ ) is used as the desired output signal; A method comprising:

10. The method of claim 1 determines one output signal for each of a plurality of images, the machine learning system is trained based on the plurality of images and the identified output signals.

10. The method of claim 9.

11. The density measure is expressed by the following formula: d Ｚ =2・(W Ｚ +W Ｚｋ ) / n・(n+1) characterized by, The method of claim 1.

12. The connectivity measure is expressed by the following formula: cp Ｚ =W Ｚｋ / d Ｚ ・(n+1) characterized by, The method of claim 1.

13. A training device (140) configured to perform the method of claim 9.

14. A computer program configured to perform the method of claim 1 when executed by a processor (45, 145).

15. A machine-readable storage medium (46, 146) having stored thereon a computer program according to claim 14.

Images