System and method for improved blindness detection

Abstract

The invention relates to a method for detecting the availability of an environmental sensor system, wherein the environmental sensor system comprises at least a first environmental sensor which is positioned on a mobile machine and which is used for a function for avoiding a collision of the mobile machine with an external object, wherein the method comprises the following steps: a. determining, by the first environmental sensor and a second environmental sensor positioned on the mobile machine, whether an external object is located within at least a portion of a field of view of the first environmental sensor; b. Determining whether the detection from the first and second environmental sensors from step a. match; c. Detecting a possible lack of availability of the environmental sensors if it is determined in step b. that the determinations of the first and the second environmental sensors do not match.

Description

Collisions between mobile work machines pose a serious risk both to the people they are serving and to the machines themselves. Particularly in situations with limited visibility, for example when a wheel loader with a loaded bucket or a dumper with a filled bucket is in the way, there is the risk of serious accidents. There are already known systems which aim to avoid such collisions or to minimize their effects. These systems include backup cameras, bird eye view cameras for enhanced surround view, and various warning and avoidance systems based on camera, ultrasound, radar, or lidar technology. They act as assistance systems to increase the safety during operation of these machines.If systems are concerned which are based on surroundings sensor systems such as radar, the sensors detect obstacles such as persons, other vehicles or walls. These known systems then calculate a time-to-collision and a critical distance, at which a warning should be issued if undershot--in this case there remains enough time for the driver to brake manually, on the basis of the transmitted data, such as the distance and relative speed of the detected object. Likewise, a distance is determined at which an automatic braking or deceleration intervention is to be triggered when the distance falls below.Similar systems are already established in the automotive sector and can also be used with adapted configurations in mobile work machines. The implementation of these systems generally includes one or more sensors, such as radar, inertial sensors for transmitting their own movements (yaw rates) to the corresponding radar sensors, and a control unit for evaluating the objects detected by the sensor and for calculating the relevant variables which are relevant for triggering warnings or interventions.These systems are an important step in improving safety in the operation of mobile work machines and help avoid accidents or at least reduce their severity.In collision avoidance systems, the constant availability of the sensor technology used is not always guaranteed. Situations where camera, radar, ultrasonic, or lidar based systems fail temporarily or permanently may occur, for example, when the sensor heads are affected by dirt such as dust, mud, or ice. In such cases, it is of decisive importance that the system immediately informs about the malfunction ("sensor system not available-sensor is blind"), so that functions which are based on these sensors and serve for collision warning or avoidance can be deactivated accordingly. Moreover, the driver must be informed of this situation.In established sensor systems, the detection of the "blindness" is carried out by the sensor system itself. For example, in known radar systems it is checked whether the radar recognizes objects in the field of view. As long as objects are detected, it is assumed that the sensor is not impaired. However, if objects are no longer detected over a longer period of time, this may basically reflect two situations: i. The sensor is impaired. Objects are present in the field of view of the sensor, but due to dirt or other factors (e.g., defects), these cannot be detected. ii.In case i), it is of decisive importance that the detection of the impairment of the sensor takes place quickly and corresponding messages are generated. In case ii), the sensor operates as expected and no notification of impairment should be made because the system is functional and therefore collision avoidance should still be available.The main object is thus to ensure an extremely fast and reliable detection of sensors which are in a "blind" state, so that the system is deactivated only when a genuine impairment actually exists. This is of critical importance to ensure the safety and functionality of the collision avoidance systems while avoiding unnecessary shut-downs.SUMMARYAccording to one specific embodiment of the present invention, a method for detecting the availability of a surroundings sensor system (this surroundings sensor system means at least one surroundings sensor that makes it possible to observe the surroundings), the surroundings sensor system including at least one first surroundings sensor that is positioned on a mobile machine and that is used for a function for avoiding a collision of the mobile machine with a foreign (or external) object, and a second surroundings sensor that is positioned on the mobile machine (1), the method including the following steps: a. The first surroundings sensor (S 1) and the second surroundings sensor (S 2) ascertaining whether an external object (99) is located within at least part of a field of view (21, 22) of the first (S 1) and the second (S 2) surroundings sensor by means of the first surroundings sensor (S 1), wherein the part of the field of view of the first environment sensor is equal to the part of the field of view of the second environment sensor; b. determining whether the ascertainments of the first and the second environment sensor (S 1, S 2) from step a. coincide; c. detecting a possible lack of availability of the environment sensor system if it is determined in step b. that the ascertainments of the first and the second environment sensor (S 1, S 2) do not coincideThe invention is based on the evaluation of at least one further environment sensor for environment detection (S 2) in order to realize the detection of the blindness of the first environment sensor (S 1) as reliably as possible. The addition of a further environment sensor makes it possible to plausibilize the blindness detection of the first environment sensor by evaluating the additional information of the second environment sensor.This solution enables improved blindness detection and thus increases the availability of the system, because only a genuine blindness leads to the system being shut down. In addition, this solution enables increased machine safety because the driver is reliably informed of the blindness of the sensor system and then knows that the system is no longer available for collision avoidance, since something is not available in the system (either the first or the second environment sensor).BRIEF DESCRIPTION OF THE FIGURESThe present invention will be described with reference to the accompanying figures, wherein like reference numerals refer to like parts and / or to like parts and / or to corresponding parts of the system. Regarding the figures: FIG. 1 shows a wheel loader as an example of a mobile machine. FIG. 2 shows a method for detecting an availability of a surroundings sensor system according to a first specific embodiment of the present invention; FIG. 3 shows a method for detecting an availability of a surroundings sensor system according to a second specific embodiment of the present invention; FIG. 4 shows a method for detecting an availability of a surroundings sensor system according to a third specific embodiment of the present invention; FIG. 5 shows, by way of example, a block diagram for a collision avoidance function of a mobile machine comprising a method for detecting availability of a surroundings sensor system according to one specific embodiment of the present invention.DETAILED DESCRIPTIONThe present invention will now be described with reference to specific embodiments as shown in the accompanying figures. Nevertheless, the present invention is not limited to the particular embodiments described in the following detailed description and shown in the figures, but the described embodiments are merely illustrative of some aspects of the present invention, the scope of which is defined by the claims.Other modifications and variations of the present invention will be apparent to those skilled in the art. The present description thus includes all modifications and / or variations of the present invention, the scope of which is defined by the claims.In the present invention, the phrase "detection of a possible lack of availability of the environment sensor system" means that the environment sensor system may not be available: either because an environment sensor is switched off / defective or because the view of the environment sensor is at least partially covered, e.g. due to dirt. In the present invention, very often speaking is "blindness" or "blindness" of a surroundings sensor. It should be noted that these words mean that the environment sensor does not perform its task because it may be contaminated or defective.FIG. 1 shows a wheel loader 1 as an example of a work machine. However, it is worth noting that the present invention can also be applied in other fields, such as in the automotive field, and therefore cannot be applied exclusively to a mobile work machine. Moreover, the wheel loader is only an example of a mobile work machine. It is not necessary for the mobile work machine to comprise a working kinematics, as in the wheel loader.The wheel loader 1 has a boom which in turn has a plurality of boom elements for receiving loads. The boom elements are here, by way of example, a lifting arm 2 (also known as a "boom arm") and a bucket 4, which are connected to one another or to a boom carrier 5 (e.g. chassis) of the wheel loader 1 by means of axles 6 in a rotatable or pivotable manner. The cantilever elements 2, 4 are movable by means of actuators 8, 10, i.e. the rotational or pivoting movement about the axes is effected by a movement of the actuators.A first actuator 8 is provided for the lifting arm 2, which brings about the movement or rotation of the lifting arm 2 relative to the boom support 5. Likewise, a second actuator 10 is provided for the blade 4, which moves or rotates (tilts) the blade 4. The actuators comprise in particular hydraulic cylinders 12, 14, i.e. a first hydraulic cylinder 12 of the first actuator 8 and a second hydraulic cylinder 14 of the second actuator 10.The actuators 8, 10 can be controlled by a controller 18, wherein in the case of hydraulic cylinders 12, 14 directional valves are provided which control the flow of hydraulic fluid to the hydraulic cylinders.The mobile work machine 1 includes a plurality of wheels that allow movement of the mobile work machine. The movement may result in collisions between the mobile work machine and external objects (such as another mobile work machine or a tree). Such collisions pose a serious risk both to the people they are serving and to the machines themselves. In order to avoid such a risk, collision avoidance functions are generally used.In such a function, environment sensors (radar sensors, ultrasonic sensors, cameras and lidar) will generally detect their environment and recognize obstacles, other vehicles or persons in the vicinity. The acquired data is then analyzed by a control system to detect potential collisions. Once potential collision hazards are detected, the machine may trigger warnings or alarms.These can be done visually via screens or LED displays, acoustically via warning tones or by vibrations in the interior of the vehicle in order to alert the driver or the operators. Advanced systems may activate automatic emergency brakes or reduce speed to avoid collisions. These systems may be capable of acting independently to stop or slow the machine when there is a risk of a collision. Some mobile machines are equipped with driver assistance systems that help the driver avoid collisions. These systems may include steering assist, lane keeping assist, adaptive cruise control, and other functions to ensure that the engine is operated safely and without collisions. By establishing zones or areas in which the machine can operate safely and implementing zone detection systems, the machine can be programmed to perform certain actions or adjust its speed in certain areas.The exact implementation of these measures depends on the type of machine, its intended uses and the safety standards. A combination of different technologies and systems typically provides the most reliable collision avoidance for mobile machines.Detailed description of a collision avoidance function will be omitted in this description because the method of the present invention can be applied to any collision avoidance function.For this reason, the mobile work machine (as shown in FIG. 2 ) comprises at least one first environment sensor S 1 and one second environment sensor S 2. The environment sensors are part of an environment sensor system of the mobile machine. The environment sensors need not necessarily be of the same type of environment sensors. Known types of environment sensors are, for example, camera-based, radar-based, ultrasonic- or lidar-based sensors. For example, the first environment sensor S 1 could be a camera and the second environment sensor could be a radar.Referring now to Figs. 2 to 4, three different embodiments of the present invention will be described. In the three specific embodiments, the availability of the surroundings sensor system is detected.As shown in FIG. 2, the first environment sensor S 1 has a first field of view 21 and the second environment sensor S 2 has a second field of view 22. For this reason, it is possible for the two surroundings sensors to ascertain an external object 99 which is located in the overlapping part of the fields of view 21 and 22.By means of this specific embodiment, first environment sensor S 1 is checked for plausibility by second environment sensor S 2. In particular, it is determined whether the ascertainment of first environment sensor S 1 and of second environment sensor S 2 coincide.If both environment sensors see the same object in the overlap region, it can be determined that the environment sensor system is working completely.If first environment sensor S 1 does not see object 99 in the overlap area and, on the other hand, second environment sensor S 2 sees object 99 in the overlap area, it is recognized that first environment sensor S 1 is blind.In the same way, in the event that first environment sensor S 1 sees an object 99 in the overlap region and, on the other hand, second environment sensor S 2 sees no object 99 in the overlap region, it is recognized that second environment sensor S 2 is blind (or, in general, that the second environment sensor is not functioning correctly).It is to be noted that in the case where both environment sensors are not viewing an object, it cannot be determined whether or not the environment sensor system is actually functioning, because there are naturally many cases where no object is present in the overlap region.In order to solve this drawback, a second embodiment shown in FIG. 3 has been added. In this second specific embodiment, a third surroundings sensor S 3 has been added, which has a third field of view 23. The three fields of view 21, 22, 23 overlap at least partially.By means of this second specific embodiment, first environment sensor S 1 and second environment sensor S 2 are checked for plausibility by third environment sensor S 3. In particular, it is determined whether the ascertainment of first S 1, second environment sensor S 2 and third environment sensor S 3 coincide.If all three environment sensors see the same object 99 in the overlap region, it can be determined that the environment sensor system is working completely.If both the first environment sensor S 1 and the second environment sensor S 2 do not see an object 99 in the overlap region and, on the other hand, the third environment sensor S 3 sees an object 99 in the overlap region, it is recognized that the first and the second environment sensor are blind (or, in general, that the two environment sensors are not functioning correctly).It is to be noted that even in this embodiment, in the case where all three environment sensors are not viewing an object in the overlap region, it is not 100% possible to determine whether or not the environment sensor system actually functions because there are naturally cases where no object is present in the overlap region (however, the risk has been significantly limited compared to the first embodiment).In addition, this specific embodiment may of course function in such a way that the plausibility check functions in pairs: first surroundings sensor S 1 is checked for plausibility by second surroundings sensor S 2 for the overlap region of fields of view 21 and 22, first surroundings sensor S 1 is checked for plausibility by third surroundings sensor S 3 for the overlap region of fields of view 21 and 23, second surroundings sensor S 2 is checked for plausibility by third surroundings sensor S 3 for the overlap region of fields of view 22 and 23.Referring now to Fig. 4, a third embodiment will be described. What the first and second embodiments have in common is that the fields of view at least partially overlap. However, this causes an increase in hardware cost because a plurality of sensors are needed to cover a predetermined field of view.FIG. 4, on the other hand, shows that the core idea of this earth finding can also function with two surroundings sensors, the fields of view of which do not overlap.As shown in FIG. 4, the environment sensor system includes a first and a second environment sensor S 1 and S 2. The two surroundings sensors are positioned in each case on the front and rear sides of mobile machine 1. It is pointed out that this illustrated positioning of the two surroundings sensors can also vary. It is only important for this third specific embodiment that the fields of view of the two surroundings sensors do not overlap.In order to execute the method for detecting the availability of the environment sensor system, the movement of the mobile machine must be taken into account in this third specific embodiment. A motion model of the mobile machine 1 is preferably used for these purposes.At a first point in time t 1, a specific field of view 21 is viewed by first environment sensor S 1. It is known by a measurable movement of the mobile machine 1 that, at a second point in time t 2, the second environment sensor S 2 has a field of view 22, which at least partially overlaps the region previously seen by the environment sensor S 1.In this embodiment, a comparison can then also be made. If the second environment sensor S 2 has seen an object 99 in the overlap region at the second point in time t 2, which the first environment sensor S 1 has not detected at the first point in time t 1, blindness may be inferred by the first environment sensor S 1 (or, in general, the first environment sensor is not functioning correctly).A prerequisite for this third specific embodiment is the assumption about a static environment, i.e., it must be ruled out that the objects in the overlap region have changed between points in time t 1 and t 2 (because, for example, vehicles have moved). It is thus either to be ensured that only static objects are viewed or that the distance between points in time t1 and t2 is very small, whereby a change in the viewed object situation becomes less likely.FIG. 5 illustrates a method for avoiding a collision of a mobile machine with an external object. In a first step 100 and 101, it is determined by the first environment sensor and the second environment sensor (and, if applicable, by the third environment sensor) whether an external object is located within at least part of a field of view of the first and the second environment sensor. The information from the environment sensors is used both for the detection of the availability of the environment sensor system 102 (according to one of the embodiments 1-3 of the present invention) and for the collision avoidance function 104.In the event that a possible lack of availability of the environment sensor system is detected in step 102, a logic 103 for shutting down the system for collision avoidance is activated and thus the collision avoidance function 104 is deactivated. In parallel, for this case, an output for sensor blindness is then generated (or detection of a defective environment sensor system), for example by a human machine interface.The described method is stored in a memory unit and is executed by the control unit 18. The present invention furthermore comprises a computer program which causes a computing unit to carry out the described method when it is executed on the computing unit. Moreover, a machine-readable storage medium having the computer program stored thereon is disclosed.While the present invention has been described with reference to the above-described embodiments, it will be apparent to those skilled in the art that it is possible to realize various modifications, variations and improvements of the present invention in the light of the above-described teaching and within the scope of the appended claims without departing from the scope of the invention.Moreover, the areas of skill in the art would not be described herein to unnecessarily obscure the described invention. Accordingly, the invention is not to be limited by the specific illustrative embodiments, but by the scope of the appended claims.

Claims

Method for detecting availability of a surroundings sensor system, wherein the surroundings sensor system comprises at least one first surroundings sensor (S1), which is positioned on a mobile machine (1) and which is applied for a function for avoiding a collision of the mobile machine with an external object (99), and a second surroundings sensor (S2) which is positioned on the mobile machine (1), wherein the method comprises the following steps: a. The first surroundings sensor (S1) and the second surroundings sensor (S2) determine whether an external object (99) is located within at least part of a field of view (21, 22) of the first (S1) and the second (S2) surroundings sensor; b. The method determines whether the determinations from the first and the second surroundings sensor (S1, S2) coincide from step a; c. Detection of a possible lack of availability of the environment sensor system if it is determined in step b. that the ascertainments of the first and of the second environment sensor (S 1, S 2) do not match.Method according to Claim 1, wherein after it is detected in step c. that there is a lack of availability of the environment sensor system, said function for avoiding the collision of the mobile machine (1) with an external object (99) is interrupted.Method according to either of Claims 1 and 2, wherein, after it is detected in step c. that there is a lack of availability of the environment sensor system, a signal is sent to the driver of the mobile machine (1) that the environment sensor system is not available.Method according to one of Claims 1 to 3, wherein the field of view (21) of the first environment sensor (S1) and the field of view (22) of the second environment sensor (S2) overlap at least partially.Method according to one of Claims 1 to 4, wherein the environment sensor system of the mobile machine (1) is provided with a third environment sensor (S3), wherein the field of view (23) of the third environment sensor (S3) overlaps at least partially with the field of view of the first and of the second environment sensor (21, 22), wherein in step a. it is furthermore determined with the third environment sensor (S3) whether an external object (99) is located within the part of the field of view (21, 22) of the first and of the second environment sensor (S1, S2), and wherein in step b. the determinations of the first, second and third environment sensors (S1, S2, S3) are checked during the determination.Method according to one of Claims 1 to 3, wherein the field of view (21) of the first environment sensor (S1) and the field of view (22) of the second environment sensor (S2) do not overlap.Method according to Claim 6, wherein the ascertainment of step a. takes place taking account of a movement of the mobile machine (1), and wherein, in step a., the two ascertainments by the two environment sensors (S1, S2) take place successively at two different points in time (t1, t2).Method according to one of Claims 1 to 7, wherein the first, the second and optionally the third environment sensor (S1, S2, S3) can each be a camera, radar, ultrasound or lidar sensor.Method for avoiding a collision of a mobile machine (1) with an external object (99), wherein the method comprises the following steps: d. obtaining environment data from an environment sensor system; e. detecting availability of the environment sensor system according to a method according to one of Claims 1 to 8; f. interrupting the function for avoiding a collision of the mobile machine with an external object if a possible lack of availability of the environment sensor system is detected in step e.Computing unit which is configured to carry out a method according to one of the preceding claims.Mobile working machine (1) configured to absorb loads, environment sensors being provided which are set up to observe an environment of the mobile working machine; having a computing unit according to Claim 9.A computer program that causes a computing unit to perform a method according to any one of claims 1 to 9 when executed on the computing unit.A machine readable storage medium having stored thereon a computer program according to claim 12.

Images