METHOD FOR OPTIMIZING SENSOR PARAMETERS
Patent Information
- Application Number
- DE502020010782
- Authority / Receiving Office
- DE · DE
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2020-07-27
- Publication Date
- 2025-04-30
- Estimated Expiration
- 2040-07-27
AI Technical Summary
Existing object recognition systems using sensors like cameras and lidar struggle to achieve optimal results due to fixed or deterministic sensor parameters, limiting the learning ability and efficiency of deep learning algorithms, especially in applications requiring precise measurements like the automotive sector.
A procedure and system that dynamically adjust sensor parameters of two sensors (such as camera and lidar) using a Reinforcement Learning algorithm, iteratively refining the sensor settings to maximize object recognition accuracy within specific sensor parameter ranges.
This approach enables the full potential of object recognition algorithms to be exploited, improving recognition accuracy and efficiency by adaptively optimizing sensor parameters based on real-time data and performance feedback.
Description
[0001] The present invention relates to a method and a system for optimizing sensor parameters of two sensors, which are used in particular in object detection.
[0002] Sensors are usually provided with parameters or sensor parameters that are fixed or run according to a deterministic scheme. Currently, software for interpreting the data acquired by the respective sensors is not satisfactorily coupled with the 'behavior' or actions of the respective sensor, i.e., there is no feedback as to the extent to which the sensor parameters set on the respective sensors influence the data acquired by the respective sensors. This means that the learning capability of modern sensors is at least limited, but generally non-existent. As a result, object detection by sensors using methods known from the state of the art remains particularly limited.
[0003] DE 10 2018 126 112 A1 describes a training method for a neural network for automatically adjusting a camera, wherein the neural network outputs at least one camera adjustment parameter. First, an image is captured using the camera, and image data from the captured image is provided as input data for the neural network and analyzed with the aim of detecting at least one object in the image. Furthermore, at least part of the neural network is influenced depending on whether at least one object has been detected.
[0004] With fixed sensor parameters, as described above, the full potential of the algorithms used for object detection is not utilized. For example, a deep learning algorithm may achieve a certain 'score' or a result with a certain agreement value through training. However, especially during an operational phase, some objects may still not be optimally interpreted by the deep learning algorithm due to occlusion or unfavorable sensor parameters.
[0005] There are areas, such as the automotive sector, that require extremely precise measurement or object detection. Currently, deep learning algorithms rarely meet these requirements due to the aforementioned problems. The actions of a particular sensor are not efficiently coupled with the interpretation by the algorithms used for object detection.
[0006] It was therefore an object of the present invention to provide a way of adapting sensor parameters of a respective sensor used in object recognition in such a way that an object recognition algorithm used for object recognition delivers optimal results in a given situation.
[0007] To achieve the aforementioned object, a method and a system having the respective features of the independent claims are provided. Further embodiments can be found in the dependent claims, the description, and the figures.
[0008] According to claim 1, a method is provided for adapting sensor parameters to be set for two sensors when using the two sensors in operative connection with an object detection algorithm during object detection in a given scenario, wherein the sensors are selected from the group comprising: camera, LiDAR sensor, radar sensor, ultrasonic sensor, infrared sensor, sonar sensor. The method according to the invention is characterized in that a. the sensor parameters to be set are set on the two sensors, b. the two sensors are activated with the set sensor parameters, and the two sensors record data using sensors and provide it to the object recognition algorithm as sensory data, c. the object recognition algorithm determines a result comprising at least one recognized object with a match value assigned to the recognized object and provides it to a machine learning algorithm that is implemented as a reinforcement learning agent, d.the machine learning algorithm, to which the result (21) and the set sensor parameters (15) that are set for capturing the currently sensor-captured data (12) at the two sensors (11) are provided, varies the respective sensor parameters of the two sensors within a respective sensor parameter range and provides the respective varied sensor parameters as sensor parameters to be set for the two sensors, e. steps a. to d. are repeated iteratively until the respective match value assigned to the at least one detected object reaches a maximum within at least one of the respective sensor parameter ranges.
[0009] By adapting the respective sensor parameters of the sensors used in object detection to a specific situation or scenario, it can be made easier for the object detection algorithm that is functionally connected to the sensor(s) used, e.g. the deep learning algorithm, to exploit its full potential.
[0010] In the context of the present disclosure, a functional or communicative connection is understood to mean a connection that enables data to be transmitted or exchanged between the interconnected components, here the object recognition algorithm and the two sensors. Data acquired by the sensors, which is transmitted to the object recognition algorithm via the functional connection, enables the object recognition algorithm to recognize an object located in the scenario based on the data. The object recognition algorithm is also communicatively connected to the machine learning algorithm in order to provide the result to the machine learning algorithm.In one possible embodiment, the newly adjusted sensor parameters calculated by the machine learning algorithm based on the result are provided directly to the at least one sensor via a dedicated interface, and in another embodiment, are automatically adjusted to the at least one sensor. The terms "functional connection," "functional connection," and "communicative connection" are used synonymously in the present disclosure.
[0011] The two sensors, the object recognition algorithm, and the machine learning algorithm are interconnected via corresponding communicative connections, i.e., they communicate with each other. Each of the communicative connections between the various components can be a direct connection or an indirect connection. Each communicative connection can be a wired or wireless connection. Any suitable communication technology can be used. Each component—i.e., the two sensors, the object recognition algorithm, or the computer unit implementing and / or executing it, and the machine learning algorithm, or the computer unit implementing and / or executing it—comprises one or more interfaces for communicating with each other.Such communication may be performed using a wired data transmission protocol such as Fiber Distributed Data Interface (FDDI), Digital Subscriber Line (DSL), Ethernet, Asynchronous Transfer Mode (ATM), or another wired transmission protocol. Alternatively, communication may be performed wirelessly over wireless communication networks using any of a variety of protocols such as General Packet Radio Service (GPRS), Universal Mobile Telecommunications System (UMTS), Code Division Multiple Access (CDMA), Long Term Evolution (LTE), Wireless Universal Serial Bus (USB), and / or any other wireless protocol. The respective communication may be a combination of wireless and wired communication.
[0012] The respective computer or processing unit may include or communicate with one or more input devices, such as a touchscreen, an audio input, a motion input, a mouse, a keyboard input, and / or the like. Furthermore, the respective computer or processing unit may include or communicate with one or more output devices, such as an audio output, a video output, a screen / display output, and / or the like.
[0013] The respective sensor parameters are adjusted using reinforcement learning, i.e. using a reinforcement learning agent. This means that the machine learning algorithm is implemented by a reinforcement learning agent, whereby the agent independently learns a strategy in order to maximize the respective match value assigned to at least one detected object in the given situation or in the given scenario. One of the advantages of reinforcement learning is that no prior training data is required. Instead, this is generated and labeled, i.e. identified, in many runs using a trial-and-error process during a training phase. Causalities are observed and used to learn the strategy. The agent therefore learns through various trials which sensor parameters, such as the size, should be used by the object recognition algorithm.A deep learning algorithm can achieve good results by reporting the respective result, or the corresponding match value, from the object detection algorithm to the machine learning algorithm. This can increase the match value within given sensor parameter ranges.
[0014] Instead of a reinforcement learning agent for optimizing the sensor parameters to be set, another learning method can also be used.
[0015] According to the invention, the two sensors are selected from the group comprising at least: camera, LiDAR sensor, radar sensor, ultrasonic sensor, infrared sensor, sonar sensor. It is also conceivable to use multiple sensors in combination for object detection, i.e., the data acquired by each sensor is merged and processed by the object detection algorithm to determine the result. The sensors can be of the same or different types. It is also conceivable for multiple sensors of the same or different types, e.g., a LiDAR sensor and a camera, to be installed together in a housing to form a technical unit.
[0016] In a further embodiment, if one of the sensors is a camera, the sensor parameters to be set are selected for the camera from a group of variable camera parameters, in particular comprising camera position, aperture, illumination angle, focus, recording angle, number of pixels in a pixel cloud, recording frequency, and / or, if one of the sensors is a LiDAR sensor, are selected for the LiDAR sensor from a group of variable LiDAR parameters.
[0017] It is also conceivable, for example, that at least two sensors are used and the result determined by the object recognition algorithm further comprises an overlap value determined with respect to the at least two sensors.
[0018] In a further embodiment of the method according to the invention, the object recognition algorithm is functionally connected to a scoring function. In particular, the object recognition algorithm comprises the scoring function, the scoring function being designed to determine the respective match value assigned to the at least one recognized object. The match value corresponds to a quality measure that indicates how well the at least one recognized object is actually correctly recognized by the object recognition algorithm, i.e. to what extent an object predicted or recognized by the object recognition algorithm actually corresponds to the real object according to a ground truth or a reference. For example, a maximum achievable match value, expressed e.g.in % corresponding to 100%, a category "car", that the object detected from the point of view of the object detection algorithm is a car and according to a ground truth is actually a car.
[0019] The agreement value can be represented or described, for example, as an overlap and / or as a confidence value. Overlap is defined as an overlap between a reference value or ground truth and a prediction, i.e., an object detected by the object detection algorithm. Confidence is defined as a measure of the certainty of a categorization. For example, 100% confidence in the category "car" means that the object detected by the object detection algorithm is a car and, according to the ground truth, is indeed a car.
[0020] If the sensor is a camera, for example, it is conceivable that the camera is filming a car and the object recognition algorithm that processes the recorded camera images, i.e. the sensor data, displays a confidence whose value depends on the position of the camera. The machine learning algorithm that optimizes the sensor parameters then, if necessary, i.e. depending on the match value provided to it by the object recognition algorithm, provides adjusted sensor parameters within the respective sensor parameter ranges that improve the position of the camera, i.e. result in greater confidence. A respective sensor parameter range specifies a physical value range within which the respective sensor parameters that can be set on the sensor must lie. This means that a respective sensor parameter range includes all realizable values, i.e. values that can be set on the sensor, for a respective sensor parameter.
[0021] According to one embodiment, the method according to the invention or the steps a. to d. comprised therein are implemented by a number i of, in particular by a plurality of, nested functions f ( i )< , which has a corresponding number i of nesting shells, where i is a natural number, wherein the set sensor parameters of the at least one sensor K , R and the data recorded by the at least one sensor with the set sensor parameters from an innermost function f (1)< ( x, K, R ) of the nested functions, then by a function directly enclosing the innermost of the nested functions and executing the object detection algorithm f (2)< ( y, K, R) of the nested functions are further processed, then by a function that directly encloses the function that executes the object recognition algorithm and executes the evaluation function f (3)< ( z, K, R ) of the nested functions, then by a function that directly wraps the function executing the evaluation function and executes the machine learning algorithm f (4)< ( v, K, R ) of the nested functions are further processed, and the sensor parameters to be set are defined by an outermost function f (5)< ( w, K, R ) of the nested functions, where v, w, x, y, z represent the respective function variables. By nesting, the procedure or the steps a. to d. can be represented by a function f for example for two sensors K , R as follows: f K R = f 5 f 4 f 3 f 2 f 1 K R , K , R , K , R K , R K , R where f(K, R)ultimately specifies the adjusted sensor parameters to be set or values for the sensor parameters to be set for the respective sensors or at least the sensor parameters to be set from f ( K, R ) can be derived or arise. Using these sensor parameters, steps a. to d. are then repeated. Steps a. to d. are repeated or f ( K, R ) is applied until the respective match value assigned to the at least one detected object reaches a maximum within at least one, preferably all, of the respective sensor parameter ranges.
[0022] In a further embodiment of the method according to the invention, a deep learning algorithm is used as the object recognition algorithm.
[0023] The method according to the invention can be carried out in at least one of the following fields: automation technology, highly autonomous driving (HAF), robotics, mobile devices, in particular next-generation mobile devices.
[0024] Furthermore, the present invention relates to a system comprising at least one computer-readable medium and two sensors, wherein the sensors are selected from the group comprising at least: camera, LiDAR sensor, radar sensor, ultrasonic sensor, infrared sensor, sonar sensor, wherein the computer-readable medium is functionally connected to the two sensors and comprises a computer program with program codes which, when the computer program is loaded and executed on a computer unit which is functionally connected to the two sensors, are designed to A. set the sensor parameters to be set on the two sensors and activate the two sensors with the set sensor parameters, B. provide the data sensed by the two sensors to an object recognition algorithm, C. use the object recognition algorithm to determine a result, comprising at least one detected object with a match value assigned to the detected object, and provide it to a machine learning algorithm that is implemented as a reinforcement learning agent, D. use the machine learning algorithm, to which the result and the sensor parameters to be set, which are set for recording the currently sensed data on the two sensors, to vary the respective sensor parameters of the two sensors within a respective sensor parameter range and to provide the respective varied sensor parameters as sensor parameters to be set for the two sensors, E.to repeat steps A to D iteratively until the respective match value assigned to the at least one detected object reaches a maximum within at least one of the respective sensor parameter ranges.
[0025] The machine learning algorithm is implemented by a reinforcement learning agent, whereby the agent is trained to independently learn a strategy to maximize the respective match value assigned to at least one recognized object.
[0026] In a further embodiment of the computer-readable medium, the object recognition algorithm is functionally connected to an evaluation function, a scoring function, in particular the object recognition algorithm comprises the evaluation function, wherein the evaluation function is designed to determine the respective match value assigned to the at least one recognized object.
[0027] In a further embodiment of the computer-readable medium, the program codes are designed to execute a plurality of nested functions forming a corresponding number of nesting shells, wherein the set sensor parameters of the at least one sensor and the data respectively acquired by the at least one sensor with the set sensor parameters are provided by an innermost function of the nested functions, then further processed by a function of the nested functions that directly encloses the innermost of the nested functions and executes the object recognition algorithm, then further processed by a function of the nested functions that directly encloses the function that executes the object recognition algorithm and executes the evaluation function,then further processed by a function of the nested functions that directly encloses the function executing the evaluation function and executes the machine learning algorithm, and the sensor parameters to be adjusted are output by an outermost function of the nested functions.
[0028] In a further embodiment of the computer-readable medium, the object recognition algorithm is a deep learning algorithm.
[0029] Computer-readable media (whether or not transient) suitable for storing computer program instructions and code and data include all forms of non-volatile memory, media and storage devices, including, for example, semiconductor memory devices, e.g., erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto-optical disks; and CD-ROM, DVD+ / - R, DVD-RAM, and DVD-ROM disks.Memory may store various objects or data, including caches, classes, frameworks, applications, backup data, jobs, web pages, web page templates, database tables, storage units (repositories) that store business and / or dynamic information, and any other suitable information, including any parameters, variables, algorithms, instructions, rules, constraints, or references thereto. In addition, memory may also contain other suitable data, such as logs, policies, security or access data, report files, and others. The processor and memory may be supplemented by or integrated with dedicated logic circuitry.
[0030] A program software, software application, module, software module, script, or program code may be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages, and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. In one embodiment, the computer-executable instructions of the present disclosure are written in HTML, TS (TypeScript), and CSS (Cascading Style Sheets).
[0031] A computer program may, but need not, correspond to a file in a file system. A program may be stored in a part of a file that contains other programs or data, e.g., one or more scripts stored in a Markup Language document, in a single file dedicated to the program in question, or in multiple coordinated files, e.g., files storing one or more modules, subroutines, or pieces of code. A computer program may be implemented so that it can be executed on one computer or on multiple computers located at one location or distributed across multiple locations and interconnected by a communications network. For the purposes of the present disclosure, a computing entity means both one computer and multiple computers interconnected by a communications network.A computer program may contain a number of submodules, third-party services, components, libraries, and the like, as needed. Conversely, the features and functions of different components may be combined into individual components, if necessary.
[0032] One technical field of the invention is automation technology. Further applications can be found in the field of highly autonomous driving (HAD) and robotics. Another area of application concerns mobile devices, especially next-generation mobile devices.
[0033] Furthermore, the present invention relates to a system comprising the above-described computer-readable medium and two sensors, wherein the computer-readable medium is functionally connected to the two sensors or can be brought into functional connection in order to carry out a method described above for adapting sensor parameters to be set of two sensors when using the two sensors in an object detection.
[0034] Further advantages and embodiments of the invention will become apparent from the description and the accompanying drawings.
[0035] It is understood that the features mentioned above and those to be explained below can be used not only in the combination specified in each case, but also in other combinations or on their own, without departing from the scope of the present invention.
[0036] The invention is illustrated schematically in the drawing using an exemplary embodiment and is described in detail below with reference to the drawing. Figure 1 shows a block diagram for the schematic representation of an embodiment of the method according to the invention. Figure 2 illustrates in a schematic representation a further embodiment of the method according to the invention. Figure 3 shows a schematic representation of data acquired by sensors and their evaluation after processing by an object recognition algorithm according to another embodiment of the method according to the invention. Figure 4 shows a schematic representation of possible sensor parameters of sensors that can be used for object detection. Figure 5 shows a block diagram to illustrate a further embodiment of the method according to the invention.
[0037] The figures are described across all figures, with the same reference symbols being assigned to the same components.
[0038] Figure 1shows a block diagram for the schematic representation of an embodiment of the method according to the invention. An image or a video stream 12 is recorded using a sensor, e.g. an HD camera 11 shown here, wherein the image or the video stream 12 here shows a section of road and a motor vehicle located thereon, in this case a passenger car 13. The HD camera 11 is used here for object recognition in a given scenario, i.e. for recognizing one or more objects 13, here in particular the passenger car 13 on the road section 14, and is for this purpose in functional connection with an object recognition algorithm 20. At the beginning and in order to operate the sensor or the HD camera 11, the sensor parameters to be set are first set on the HD camera 11. Then, the sensor 11 or the HD camera 11 is activated with the set sensor parameters 15, whereby the HD camera 11 records data sensorically, for example in the form of the image or the video stream 12.Alternatively or in addition to the HD camera shown and described here, any other sensor suitable for object detection, such as a LiDAR sensor, a radar sensor, an ultrasonic sensor or an infrared sensor, can also be used. The sensor parameters to be set vary accordingly. The sensor 11, here the HD camera 11, therefore provides, on the one hand, as indicated by arrow 1, sensor parameters 15 set on the sensor 11, such as illumination angle, focus, exposure time, etc., and, on the other hand, as indicated by arrow 2, sensor-captured data 12. The sensor-captured data 12 are provided or transmitted to the object detection algorithm 20 for evaluation. The object detection algorithm 20 determines a result 21, comprising at least one detected object with at least one match value assigned to the detected object. The result 21 orAt least the at least one match value included therein, such as an overlap and / or a confidence, is provided to a machine learning algorithm 30, as indicated by arrow 3. Furthermore, the set sensor parameters 15, which are set for detecting the currently sensor-detected data 12 on the sensor 11, are transmitted to the machine learning algorithm 30. With knowledge of the at least one detected match value, the machine learning algorithm 30 now varies the respective sensor parameters of the sensor 11 within a respective sensor parameter range and, as indicated by arrow 4, provides the varied sensor parameters as sensor parameters 16 to be set for the sensor 11.Now, the sensor 11 is readjusted, namely, as indicated by arrow 5, with the sensor parameters varied by the machine learning algorithm 30, and the process is repeated iteratively until the respective match value assigned to the at least one detected object 13, here, for example, the passenger car 13, reaches a maximum within at least one of the respective sensor parameter ranges, preferably within each of the respective sensor parameter ranges of the sensor parameters. Since there are usually a plurality of sensor parameters, it would be best if each of the sensor parameters were adjusted from its respective sensor parameter range such that, when all of these sensor parameters are adjusted on the sensor, the match value is at its greatest and thus optimal within the sensor parameter ranges of all sensor parameters to be adjusted.
[0039] Figure 21 illustrates a schematic representation of a further embodiment of the method according to the invention. It shows which data, here in the form of a "Stream I" 22 and a "Stream II" 23, are provided to the machine learning algorithm 30 for varying or adapting sensor parameters to be set on a sensor to be used for object detection. In Stream I 22, as indicated by arrow 1, the data 12 acquired by the sensor 11 are provided as "raw data" 12 in a step 201. If the sensor is an image capture unit, in particular a camera, this can be an image or a sequence of images, e.g., a video stream, or if the sensor is a LiDAR sensor, it can be a LiDAR stream.Other types of sensors that can be used for object detection can also be used here, capturing corresponding sensor-specific data and providing it as raw data 12. In Stream II 23, the sensor parameters 15 set on the sensor 11 for capturing the data 12 are provided in step 201, as indicated by arrow 2.
[0040] As indicated by arrow 1-1, the sensor-acquired data 12 is processed as raw data 12 by the object recognition algorithm 20 in a step 202 in order to recognize and, if necessary, also localize at least one object in the acquired data 12. The object recognition algorithm 20 outputs a result 21 that includes at least one recognized object and at least one match value assigned to the recognized object, generally formulated as a probability value, e.g., as an overlap or confidence. In parallel, in step 202, as indicated by arrow 2-1, the set sensor parameters 15 that were set on the sensor 11 when the data 12 was acquired are provided. Depending on the sensor or sensors used, sensor-specific sensor parameters relevant to the acquisition of the data 12 are provided.The sensor parameters 15 and the result 21 thus provided are then passed to the machine learning algorithm.
[0041] Figure 3shows, in a schematic representation, data 12 acquired by sensors, for example a camera and a LiDAR sensor, and their evaluation after processing by the object recognition algorithm 20 according to another embodiment of the method according to the invention. The data 12 acquired by the respective sensors are processed jointly by the object recognition algorithm 20 to obtain the result 21. The result 21 comprises two agreement values, namely an overlap and a confidence. For the car recognized by the object recognition algorithm 20, a confidence of 0.99 or 99% and an overlap of 0.894 are determined or achieved. Based on the confidence, the overlap and the sensor parameters set for the acquisition of the sensor data of the respective sensors, here the camera and the LiDAR sensor, the machine learning algorithm subsequently adapts, modifies, i.e.changed and thus successively optimized.
[0042] Figure 4shows a schematic representation of possible sensor parameters of sensors 11 that can be used for object recognition. On the left-hand side, exemplary sensor parameters of a camera 11-1 that can be used for object recognition, in particular an HD camera, are shown. The sensor parameters include, for example, an illumination angle a, a lens position b, a coverage c, and a repetition rate R. Each of these sensor parameters is assigned a respective sensor parameter range within which the respective sensor parameter can be varied. On the right-hand side, exemplary sensor parameters of a LiDAR sensor 11-2 that can be used for object recognition are shown, which here include, for example, a rotation frequency f, an aperture angle o, and a number of scannable layers I. Each of these sensor parameters is also assigned a respective sensor parameter range within which the respective sensor parameter can be varied.The LiDAR sensor 11-2 has a laser emitter 11-2.1 and a laser receiver 11-2.2.
[0043] Figure 5 shows a block diagram to illustrate a further embodiment of the method according to the invention. According to one embodiment, the method according to the invention or the steps a. to d. comprised therein are implemented by a number of i of nested functions f ( i )< , which has a corresponding number i of nesting shells, where i a natural number, here 5. The nested functions form a function f ( K,R ) . The set sensor parameters 15 of at least one sensor K , R and the data 12 respectively recorded by the at least one sensor with the set sensor parameters from an innermost function f (1)< ( x, K, R) of the nested functions, shown in nesting level 501. These are then, as indicated by arrow a, provided by a function that directly encloses the innermost of the nested functions and executes the object recognition algorithm f (2)< ( y, K, R ) of the nested functions in a nesting level 502 and, as indicated by arrow b, by a function directly enclosing the function executing the object recognition algorithm and executing the evaluation function f (3)< ( z, K, R ) of the nested functions in a nesting level 503. In this case, a result is provided. The result is then, as indicated by arrow c, together with the set sensor parameters, processed by a function that directly encloses the function executing the evaluation function and executes the machine learning algorithm. f (4)< ( v, K, R) of the nested functions are further processed in a nesting level 504. The resulting sensor parameters to be set are ultimately, as indicated by arrow d, determined by an outermost function f (5)< ( w, K, R ) of the nested functions in a nesting level 505. Here, v, w, x, y, z are the respective function variables. Through the nesting, the method or the steps a. to d. included therein can be represented by the function f for example for two sensors K , R as follows: f K R = f 5 f 4 f 3 f 2 f 1 K R , K , R , K , R K , R K , R where f ( K, R) ultimately specifies the adjusted sensor parameters to be set or values for the sensor parameters to be set for the respective sensors or at least the sensor parameters to be set from f ( K, R) can be derived or arise. Using these sensor parameters, steps a. to d. are then repeated. Steps a. to d. are repeated or f ( K, R) is applied until the respective match value assigned to at least one detected object reaches a maximum within at least one of the respective sensor parameter ranges. List of reference symbols
[0044] 1Arrow 1-1Arrow 2-1Arrow 2Arrow 3Arrow 4Arrow 5Arrow 11Sensor 11-1Camera, HD camera 11-2LiDAR sensor 11-2.1Laser emitter 11-2.2Laser receiver 12Data 13Object, motor vehicle, passenger car 14Road section 15Sensor parameters 16Sensor parameters to be set 20Object detection algorithm 21Result 22Stream I 23Stream II 201Step 202Step 30Machine learning algorithm 501Nesting level 502Nesting level 503Nesting level 504Nesting level 505Nesting level aArrow bArrow cArrow dArrow
Claims
1. A method for adjusting sensor parameters to be set for two sensors (11) when using the two sensors (11) in functional connection with an object recognition algorithm (20) in an object recognition, in which the sensors are selected from the group comprising: camera, LiDAR sensor, radar sensor, ultrasonic sensor, infrared sensor, sonar sensor, in which a. the sensor parameters (16) to be set are set on the two sensors (11), b. the two sensors (11) are activated with the set sensor parameters (15), and the two sensors (11) capture data (12) sensorily and provide it to the object recognition algorithm (20), c. the object recognition algorithm (20) determines a result (21), comprising at least one recognized object (13), with a match value assigned to the recognized object, and provides it to a machine learning algorithm (30) realized as a reinforcement learning agent, d. the machine learning algorithm (30), to which are provided the result (21) and the set sensor parameters (15), which are set for capturing at the two sensors (11) the data (12) currently being sensorily captured, varies the respective sensor parameters of the two sensors (11) within a respective sensor parameter range and provides for the two sensors (11) the respective varied sensor parameters as sensor parameters (16) to be set, e. the steps a. to d. are iteratively repeated until the respective match value assigned to the at least one recognized object (13) reaches a maximum within at least one of the respective sensor parameter ranges.
2. The method according to claim 1, in which the reinforcement learning agent (30) independently learns a strategy as a reinforcement learning agent to maximize the respective match value assigned to the at least one recognized object (13).
3. The method according to claim 1 or 2, in which the two sensors (11) are selected from the group comprising at least: camera (11-1), LiDAR sensor (11-2).
4. The method according to claim 3, in which the sensor parameters (16) to be set, when one of the two sensors (11) is a camera (11-1), are selected for the camera (11-1) from a group of variable camera parameters, in particular camera position, aperture, illumination angle, focusing, recording angle, number of pixels in a pixel cloud, recording frequency and / or wherein the sensor parameters (16) to be set, when one of the two sensors (11) is a LiDAR sensor (11-2), are selected for the LiDAR sensor (11-2) from a group of variable LiDAR parameters.
5. The method according to any of the preceding claims, in which the object recognition algorithm (20) is functionally connected to an evaluation function, in particular comprises the evaluation function, wherein the evaluation function is designed to determine the respective match value assigned to the at least one recognized object (13).
6. The method according to any of claims 1 to 4, which is realized by a plurality of nested functions, wherein the set sensor parameters (15) of the two sensors (11) and the data (12) respectively sensorily captured by the two sensors (11) with the set sensor parameters (15) are provided by an innermost function of the nested functions, then are further processed by a function of the nested functions, which function directly envelops the innermost of the nested functions and executes the object recognition algorithm (20), then are further processed by a function of the nested functions, which function directly envelops the function executing the object recognition algorithm (20) and executes the evaluation function, then are further processed by a function of the nested functions, which function directly envelopes the function executing the evaluation function and executes the machine learning algorithm, and the sensor parameters to be set (16) are output by an outermost function of the nested functions.
7. The method according to any of the preceding claims, in which a deep learning algorithm is used as the object recognition algorithm (20).
8. A system comprising at least one computer-readable medium and two sensors (11), wherein the sensors are selected from the group comprising: camera, LiDAR sensor, radar sensor, ultrasonic sensor, infrared sensor, sonar sensor, wherein the computer-readable medium is in functional connection with the two sensors (11) and comprises a computer program with program codes which, when the computer program is loaded and executed on a computer unit which is in functional connection with the two sensors (11), are designed to A. set, at the two sensors (11), sensor parameters (16) to be set, and to activate the two sensors (11) with the set sensor parameters (15), B. provide the data (12) captured by the two sensors (11) to an object recognition algorithm (20), C. determine with the object recognition algorithm (20) a result (21) comprising at least one recognized object (13) with a match value assigned to the recognized object, and provide the result (21) to a machine learning algorithm (30) realized as a reinforcement learning agent, D. vary the respective sensor parameters of the two sensors (11) within a respective sensor parameter range and provide for the two sensors (11) the respective varied sensor parameters as sensor parameters (16) to be set, using the machine learning algorithm (30), to which are provided the result (21) and the set sensor parameters (15) which are set for capturing at the two sensors (11) the data (12) currently being sensorily captured. E. iteratively repeat steps A. to D. until the respective match value assigned to the at least one detected object (13) reaches a maximum within at least one of the respective sensor parameter ranges.
9. The system according to claim 8, in which the reinforcement learning agent (30) is designed as a reinforcement learning agent to independently learn a strategy to maximize the respective match value assigned to the at least one recognized object (13).
10. The system according claim 8 or 9, in which the object recognition algorithm (20) is functionally connected to an evaluation function, in particular comprises the evaluation function, wherein the evaluation function is designed to determine the respective match value assigned to the at least one recognized object (13).
11. The system according to any of claims 8 or 9, wherein the program codes are designed to perform a plurality of nested functions, wherein the set sensor parameters (15) of the two sensors (11) and the data (12) respectively sensorily captured by the two sensors (11) with the set sensor parameters (15) are provided by an innermost function of the nested functions, then are further processed by a function of the nested functions, which function directly envelops the innermost of the nested functions and executes the object recognition algorithm (20), then are further processed by a function of the nested functions, which function directly envelops the function executing the object recognition algorithm (20) and executes the evaluation function, then are further processed by a function of the nested functions, which function directly envelopes the function executing the evaluation function and executes the machine learning algorithm, and the sensor parameters to be set (16) are output by an outermost function of the nested functions.
12. The system according to one of claims 8 to 11, in which the object recognition algorithm (20) is a deep learning algorithm.