Device and system for detecting hazards in the surroundings of a vehicle

Abstract

The invention relates to a device (20) for detecting hazards in the surroundings (44) of a vehicle (50), in particular of a construction vehicle or land vehicle or a mobile work machine (54), comprising an input interface (22) for receiving an identification signal of an identification transmitter (40) and a sensor signal of a surroundings sensor (30) having surroundings information relating to the surroundings (44); a processing unit (24) for determining a position from the identification signal and for defining a surroundings region (42) which includes the position, wherein the surroundings region (42) is part of the surroundings (44), and for determining sensor data from the sensor signal, wherein the sensor data comprise the sensor signal in the defined surroundings region (42); an evaluation unit (26) for evaluating the sensor data in the defined surroundings region (42) and detecting a person (60) or an object (48) within the surroundings region (42) from the sensor data and ascertaining information relating to the person (60) or the object (48); and an output interface (28) for outputting the information relating to the person (60) or the object (48) or the surroundings information.

Description

[0001] Device and system for detecting hazards in the environment of a vehicle

[0002] The present invention relates to a device and a system for detecting hazards in the environment of a vehicle, in particular a construction or agricultural vehicle or a mobile working machine, and a corresponding method.

[0003] Hazards can occur in the vicinity of vehicles, and these should be detected as early as possible. Drivers are supported in this by assistance systems that monitor the vehicle's surroundings. Various sensors are used for this monitoring, such as radar, lidar, and / or optical sensors. The driver is typically provided with an image of the surroundings. Hazards, such as people on the road, can be detected with the help of these assistance systems. Such detection is very computationally intensive and requires high-performance computers, which must be installed in the vehicle.

[0004] For construction vehicles, such as cranes, or for agricultural vehicles, such as harvesters, or other mobile machinery, attachments or tools must be considered in addition to the vehicle's own chassis, thus increasing the area of ​​a potential hazard zone around the vehicle itself. To better monitor the surroundings of such vehicles and other vehicles, surround-view cameras, stereo cameras, or multiple cameras on all sides of the vehicle are frequently used to create a comprehensive image of the surroundings. Analyzing the 360° environment and detecting people or objects in the vicinity is computationally intensive. This either requires computers with high processing power, which are generally not available in construction vehicles, agricultural vehicles, or mobile machinery, or results in a slow and time-consuming process of mapping the vehicle's surroundings.To simply provide the driver with an image of the surroundings, for example from a bird's-eye view or in the form of a fisheye view, the computing power is usually sufficient. However, the detection of relevant objects or people near the vehicle, or of an increased hazard posed by short working distances to the vehicle or to the attached tools, is very computationally intensive.

[0005] DE 10 2021 107 703 A1 proposes an environmental monitoring system for a crane, which is carried out depending on the assembly status and the tools attached or changes to the crane boom. The monitoring area for the environmental monitoring can be changed depending on the assembly status of the crane.

[0006] US 2020 / 0048052 A1 proposes a driver assistance system, intended specifically for vehicles with a crane or boom, that provides a three-dimensional display of the vehicle's surroundings. To detect hazards near the vehicle, an image-based three-dimensional representation of the environment is generated. This process utilizes additional image information from external sources, such as GIS data or other available data, which may include representations of buildings or other immobile structures like bridges. Based on this information, warning messages can be issued to the driver.

[0007] WO 2023 / 026568 A1 discloses a remote control system for a construction machine in which multiple cameras capture images of the machine's surroundings and transmit them to a remote control unit for the operator of the construction machine and an excavator. Sensors from multiple machines can also be evaluated and displayed together. This is intended to create a better understanding of the environment and reduce the risk of collisions. The collected data is processed by an external computer unit, which can be correspondingly powerful.

[0008] Due to the limited computing capacities available in vehicles, there is a particular need to make optimal use of the existing sensors for detecting the vehicle's surroundings, especially in the vicinity of construction and agricultural machinery or mobile work machines, and to achieve optimal utilization of the available computing capacities in order to provide the best possible support for the driver in detecting hazards in the vehicle's surroundings.

[0009] According to a first aspect, the present invention relates to a device for detecting hazards in the environment of a vehicle, in particular a construction or agricultural vehicle or a mobile working machine, comprising an input interface for receiving an identification signal from an identification transmitter and a sensor signal from an environmental sensor containing environmental information relating to the environment; a processing unit for determining a position from the identification signal and for defining an environmental area that includes the position, wherein the environmental area is a part of the environment, and for determining sensor data from the sensor signal, wherein the sensor data includes the sensor signal in the defined environmental area;an evaluation unit for evaluating the sensor data in the defined environmental area and detecting a person or object within the environmental area from the sensor data and determining information about the person or object; and an output interface for outputting the information about the person or object or the environmental information.

[0010] According to a further aspect, the present invention relates to a system for detecting hazards in the environment of a vehicle, in particular a construction or land vehicle or a mobile working machine, comprising a marking transmitter for generating a marking signal with information on the position of the marking transmitter; an environment sensor for generating a sensor signal with environmental information relating to the environment; and a device as described above.

[0011] Further aspects of the present invention relate to a vehicle with a corresponding system, a method designed according to the device, and a computer program product with program code for executing the steps of the method when the program code is executed on a computer. In addition, one aspect of the invention relates to a storage medium on which a computer program is stored which, when executed on a computer, causes the execution of the method described herein.

[0012] Preferred embodiments of the invention are described in the dependent claims. It is understood that the features mentioned above and those to be explained below can be used not only in the combinations specified, but also in other combinations or individually, without departing from the scope of the present invention. In particular, the system, the method, and the computer program product can be implemented according to the embodiments described for the device in the dependent claims.

[0013] Within the scope of the invention, it was found that by focusing on relevant detection areas and processing the sensor data within these areas, the required computing power can be reduced, or faster processing can be achieved with the available computing power. In this way, hazards in identified danger zones can be detected more quickly. Based on this, the invention is based on the idea of ​​combining the environmental sensors of a vehicle with an external identification transmitter, which is worn by people, for example on a construction site, or attached to objects located near or within the vehicle's working area.

[0014] According to the invention, the device for detecting hazards evaluates not only one or more environmental sensors, but also a marker transmitter. At least the position of this transmitter is recorded. From the position data, an environmental area can be defined in which the marker transmitter is located and which must be subjected to more detailed analysis. The computing capacity is thus limited to the environmental area detected by the marker transmitter, so that only a smaller area needs to be evaluated with high accuracy, and the evaluation can therefore be performed more quickly. The computing time for the exact 3D reconstruction of the environmental area and for detecting objects within it is therefore significantly less than if the entire environment, for example in a surround view, were evaluated. This results in a prioritization of the environmental area.This area can also be prioritized in the driver's display. Another advantage is that data transmission from the sensors can be limited. Only those sensors that detect the area surrounding the identification transmitter need to be prioritized for evaluation.

[0015] The identification transmitter thus indicates a specific area of ​​the environment that should be prioritized during hazard analysis and searches. Detection of objects or people within the sensor's field of view becomes faster and more reliable. A key advantage is that the identification transmitter can be a mobile device, for example, worn by a person on a safety vest or construction helmet, or attached to a piece of equipment or tool within the construction site.

[0016] Since the identification transmitter is mobile, the parameterization and selection of the environmental area can be dynamically adjusted. Depending on the location of the identification transmitter, different environmental areas are selected and prioritized. For example, it is possible to perform an initial evaluation and then parameterize the image based on this evaluation, such as determining whether zooming into a specific area of ​​the image is necessary or desirable, or whether data compression in certain areas within or outside the environmental area is advantageous.

[0017] In a preferred embodiment, the device can include a warning unit for generating a warning signal. The warning signal is based on an evaluation of sensor data in the vehicle's surroundings. The output interface is preferably configured to output the warning signal. The warning signal can be visual or audible. For example, it can be displayed as a warning symbol on a screen in the vehicle to alert the driver to a hazardous situation. The warning signal can be a control signal or an actuator signal and is preferably forwarded to another module or device in the vehicle to restrict general vehicle capabilities, for example, reducing the boom slewing range of an excavator or crane, or decreasing or slowing down the vehicle speed.The warning signal could, for example, be a trigger signal for the brakes, allowing the vehicle to be brought to a standstill. It could also function as an emergency stop switch, halting all movement of the vehicle or mobile machinery.

[0018] In a preferred embodiment, the sensor signal is a signal from a radar sensor, a lidar sensor, or an optical sensor. The sensor signal can be the signal from a camera, a surround camera, and / or one or more stereo cameras. A combination of individual sensors and their signals can be included in the sensor signal.

[0019] In a preferred embodiment, the output interface is configured to output an image signal based on the sensor signal to generate an image or display. The environment and / or the surrounding area are output. The output can be displayed on a screen.

[0020] In a preferred embodiment, the identification signal includes parameters that can be used to reduce the sensor data. The evaluation unit is preferably configured to perform the detection of a person or object based on parameter recognition. Parameter recognition is particularly preferably carried out by means of triangulation, wherein the accuracy of the triangulation can be weighted for individual points in the surrounding area or environment.

[0021] In a preferred embodiment, the processing unit is configured to prioritize the determination of sensor data. This can be achieved, for example, by preferably allocating a higher processing capacity to the processing unit in order to prioritize the evaluation of some or specific sensor data, for example, within or relating to the environmental area. Particularly preferably, the prioritization is achieved by allocating a longer processing time for the evaluation of sensor data within the environmental area. The detection of objects or people in a specific sensor area can then be performed more quickly and / or accurately.

[0022] In a preferred embodiment of the system according to the invention, the environmental sensor is a radar sensor, a lidar sensor, an optical sensor, one or more cameras, a surround camera, and / or one or more stereo cameras. The environmental sensor can comprise several of the sensors mentioned here.

[0023] The invention is described and explained in more detail below with reference to some selected embodiments in conjunction with the accompanying drawings. These show:

[0024] Fig. 1 shows a schematic representation of a system with a device according to one aspect of the present invention;

[0025] Fig. 2 shows a representation of a vehicle with a device according to the invention and a marking transmitter;

[0026] Fig. 3 shows a representation of a vehicle in the form of a crane with the device and a marking transmitter;

[0027] Fig. 4 shows a representation of the crane from Figure 3 with a surrounding area;

[0028] Fig. 5 shows a representation of the crane from Figure 3 with stereo cameras and their

[0029] Images for determining the surrounding area;

[0030] Fig. 6 shows a crane with the device according to the invention and with a marking transmitter on a load; Fig. 7 shows a crane with the device according to the invention and a marking transmitter on a person;

[0031] Fig. 8 shows a mobile working machine with an environmental sensor on the tool of the working machine;

[0032] Fig. 9 shows a crane with an environmental sensor and a marking transmitter on another crane;

[0033] Fig. 10 shows a mobile work machine and a marking transmitter on another work machine;

[0034] Fig. 11 shows a schematic representation of the process according to the invention; and

[0035] Fig. 12 shows a further illustration of the process.

[0036] Fig. 1 shows a schematic drawing of a system 10 for detecting hazards in the vicinity of a vehicle, comprising a device 20, an environmental sensor 30, and an identification transmitter 40. While the environmental sensor 30 is directly connected to the device 20, for example via a wired connection such as a CAN bus or a similar bus system of the vehicle, the identification transmitter 40 is wirelessly connected to the device 20. The identification sensor 40 is a mobile transmitter that can, for example, be worn by a person or attached to a tool or piece of equipment.

[0037] The device 20 comprises an input interface 22 to which the environmental sensor 30 and the identification sensor 40 are connected. The device 20 further comprises a processing unit 24 for determining a position from the identification signal of the identification transmitter 40 and for defining an environmental area that includes the position of the identification transmitter 40. The processing unit 24 is also configured to determine sensor data from the sensor signal that encompasses the sensor signal within the defined environmental area.

[0038] An evaluation unit 26 of the device 20 is configured to evaluate the sensor data in the defined environmental area and to recognize a person or object within the environmental area from the sensor data and to determine further information about the person or object, such as outlines, movements, or other characteristics. The device 20 also includes an output interface 28, which is configured to output information about the person or object or environmental information, for example, to an assistance module 29 or another module or processing unit in the vehicle. The output can be wireless or, preferably, wired via a bus system.

[0039] Fig. 2 shows a vehicle 50 with a device 20 and an environmental sensor 30, which is a camera 32. A person 60 in the vicinity of the vehicle 50 wears an identification sensor 40, so that the person 60 can be detected by the device 20 and the environmental sensor 30, or the camera 32, can be aligned with the position of the identification transmitter 40 and thus with the position of the person. Only the sensor data from the sensor signal of the camera 32, which covers the area 42 defined by the identification transmitter 40, are analyzed.

[0040] Figures 3 to 10 show different versions and situations in which the invention is used.

[0041] Fig. 3 shows a vehicle 50 configured as a crane 52 with a monitoring system for the surroundings 44. The crane 52 is equipped with the device 20 (not shown here) and has four cameras 32 to provide a surround view. An image of the surroundings 44 is displayed to the operator of the crane 52. The device 20 comprises a processing unit and an evaluation unit, which are also configured to provide a surround view display on a screen for the operator. Object recognition in the surroundings 44 and a 3D reconstruction of the entire 360° environment 44 around the work vehicle 54 (crane 52) would be desirable; however, these are very computationally intensive. In addition, the operator only needs a "classic," less computationally intensive display of the surroundings 44 such that the images from the four cameras 32 are combined into a single image. This is easily achievable and requires little computation according to the prior art.

[0042] Due to the required computing power, it is not possible to enable object detection of the entire environment 44, which would include, for example, the location 46 of an object 48. The area of ​​primary interest is the region around location 48, allowing for prioritization.

[0043] Fig. 4 shows the work machine 54, designed as a crane 52, in which the object 48 is equipped with a tracking transmitter 40, so that the location 46 can be determined by the device 20 according to the invention in the crane 52. The crane 52 has four cameras 32, each of which can cover or capture an approximately 90-degree angle around the crane 52. By using the tracking transmitter 40, the approximate location 46 of the object 48 can now be recognized from the sensor signals, i.e., the data supplied by the camera, so that the sensor data for the surrounding area 42 can be determined from the sensor signal.

[0044] The display on one of the crane 52's screens for the operator can still be implemented as a 360° view according to the state of the art. The computing power of the crane 52's processing unit is sufficient for this. The 3D reconstruction, i.e., the search for common features between different images, the detection of the object 48, and the determination of the object 48's distance to the camera 32, only takes place in the area 42 where the identification transmitter 40 is located. This reduces the operator's distraction from irrelevant displays. The detection of the identification transmitter 40 also enables a pre-selection for the display of relevant data. Fig. 5 shows the work machine 54, designed as a crane 52, from Figs. 3 and 4, where the crane 52 has four stereo cameras 36, each capturing 90 degrees of the surroundings 44. The images 34 from the stereo cameras 36 are plotted along the angular axis from 0 to 360 degrees.The identification transmitter 40 is also indicated on the angular axis, which is positioned at approximately 90 to 110 degrees.

[0045] The lower representation of the angular axis also shows the four camera images 34 from the stereo cameras 36 and the surrounding area 42 around the marking transmitter 40. A tolerance can be calculated for the angle of the marking transmitter 40, which, for example, can be + / - 45 degrees and defines the surrounding area 42. The tolerance can depend on the accuracy of the position determination of the marking transmitter 40, the distance of the estimated position of the marking transmitter 40 from the machine 54, and / or the cameras 32 used. It is possible that, due to the tolerance, as shown here, areas from two stereo cameras 36 must be processed, but only partially in each case. Thus, only partial areas of the respective images 34 from the stereo cameras 36 need to be processed.

[0046] Since the position of the identification transmitter 40 can be given in world coordinates, just like the position of the machine 54, it may be necessary to convert the position of the identification transmitter 40 into the coordinates of the machine 54. This may involve reductions of the longitudinal and / or lateral components. Furthermore, it may be necessary to convert the angles of the identification transmitter 40 into the coordinates of the cameras 32, which may depend on the camera geometry and its intrinsic calibration.

[0047] Alternatively, a 3D reconstruction can be performed in the entire 360° environment 44. However, the triangulation is performed at a higher level and with higher quality only in the selected area (environmental area 42). For this purpose, for example, the positions of the stereo matching can be calculated with pixel-level to subpixel accuracy, or longer distances between left and right in the angular range around location 46 can be examined. It is also possible that, for example, only a lesser reduction of data is carried out in the environmental area 42.

[0048] A similar strategy is used for person detection. Either a search is performed only in the separate environment 42 within the image 34 of camera 32, or the search is performed in the entire image, i.e., in all images 34 from all cameras 32. However, a more precise and computationally intensive classification and evaluation of the images 34 is also performed here only within the environment 42.

[0049] The display on the crane 52's screen can also be optimized. To optimize data exchange between an assistance module, an ECU, and / or the device 20 and a display in the crane 52, a higher resolution is used for the display only in the area 42 of the crane 52's screen. This also saves processing time, which can be used for evaluation and position detection or detection of objects 48 or people 60 in the crane 52's area 44.

[0050] A similar strategy can be preferentially employed for person detection. For a 2D search for persons 60, the sensor signals of the entire environment 44 are evaluated for the crane 52. For example, 2D searches can be performed using the recordings of a surround view. For a more precise search and for determining the position of a person 60 in 3D, the stereo system consisting of the four cameras 32 is used. Additionally, another camera could be provided, for example, on the tool or at the end of the crane 52's boom. 3D evaluation preferably takes place only within the environmental area 42 defined by the identification transmitter 40.

[0051] The same applies here to the display, in order to optimize the data exchange and data rate between the ECU, an assistance module or the device and a display or screen in the crane 52. A higher resolution for the display is also only shown in the surrounding area 42 on the screen of the crane 52. Fig. 6 shows a situation in which a work machine 54 designed as a crane 52 is equipped with the device 20 according to the invention. The surrounding area 44 of the crane 52 is monitored by several cameras 32.

[0052] A load 56 is equipped with an identification transmitter 40, also called a position transmitter. The load 56 is detected by another work machine 54 and is to be transported. For collision monitoring, an area 42 around the location 46 of the load 56 is defined. While a 3D reconstruction of the entire 360° environment 44 around the crane 52 would be desirable, the necessary computing power is generally not available. Therefore, the area 42 encompassing the position of the identification transmitter 40 is determined, and a more precise and computationally intensive evaluation and object recognition are performed only within this area. This also saves computing time.

[0053] In Fig. 7, a construction machine 54, designed as an excavator 58, is equipped with a device 20 according to the invention. The area 44 around the excavator 58 is monitored. An employee or person 60 is equipped with an identification transmitter 40. In order to perform hazard detection at the location of person 60, a 3D reconstruction of the environment or the area 42 encompassing person 60 is carried out at the location 46 of person 60, and images from the cameras 32 of the excavator 58 are evaluated.

[0054] In the area 42 where person 60 is located at position 46, a computationally intensive 3D reconstruction and object recognition process is performed. For example, stereo cameras can be used to determine the distance towards the identification transmitter 40 based on left-right matching. With standard cameras 32, a 3D reconstruction using "Structure from Motion" can only be performed in the direction of the external identification transmitter 40 using vehicle odometry. Alternatively, a search for persons 60 in a 3D point cloud within the area 42, i.e., in the direction of the identification transmitter 40, can be performed using a standard camera and simultaneous "Location and Mapping". If a person 60 is detected in the first pass, person detection can be prioritized. If insufficient resources are available, for example, if there are multiple areas around the work machine 54, the process can be repeated.If the excavator 58 needs to be evaluated because several people 60 and / or loads are located in the vicinity 44 of the excavator 58, a warning or control signal can be issued. Using the warning or control signal, the working speed of the excavator 58, its travel speed, the speed of the grab arm, or the rotation of the machine or excavator 58 can be limited, for example. Reducing the speeds by half, for instance, allows the frame rate of the sensors or cameras 32 to be reduced sufficiently to ensure the same level of safety for all people and loads in the vicinity 44. For example, two or three areas around the identification transmitter 40 can be intensively monitored and resolved with very high environmental sensing resolution.

[0055] Fig. 8 shows two machine tools 54, each with an environmental sensor 30 mounted on its tool 62. Such an environmental sensor 30 can, for example, be a surround camera, i.e., a 360° camera. Alternatively, four cameras 32 or four stereo cameras 36 can be arranged. Depending on the design of the environmental sensor 30, a calculation of the surround view and, if necessary, stitching of several images from the stereo cameras 36 is performed in a processing unit and / or the device 20.

[0056] In the situation depicted here, a person 60 wears an identification transmitter 40, which is detected by the respective device 20. This creates an area 42 for both machines 54, within which the location 46 of person 60, and thus of the identification transmitter 40, is found. The position of the external identification transmitter 40 must be transferred to the respective coordinate system on the tool 62. For this purpose, the position of the respective machines 54 must be calculated in relation to the current position of the tool 62, including any rotation of the tool 62 if it is rotatable, as shown here. Subsequently, the individual images are precisely evaluated and processed with high computing power, only in the direction of the identification transmitter 40, to determine the position and identify the person. The overlaps of the two areas 42 can also be processed in this way.Overall, it is necessary for the respective systems to communicate with each other. For example, the identification transmitter 40 and the two devices 20 of the respective work machines 54 can communicate with each other via an external computing unit, a server, a cloud computer, or a network.

[0057] Figures 9 and 10 show further situations in which a machine 54 with an environmental sensor 30 is arranged on a tool 62 of the machine. A surround view is provided at the tool 62 by means of one or more cameras 32. The machine 54 includes a device 20 according to the invention for processing the sensor signals from the environmental sensors 30. Alternatively, other computing units can be used within the vehicle 50. Optionally, the computing unit and / or the device 20 can, for example, have access to further sensors via a CAN interface to determine the position of the tool 62 in space and / or relative to the vehicle 50.

[0058] Since it is generally not possible to equip all construction machines 54 with the device 20 according to the invention on construction sites, other vehicles 50 or construction machines 54 are equipped with identification transmitters 40. In Fig. 9, for example, an identification transmitter 40 is arranged on the hook 64 of the crane 52.

[0059] Fig. 10 shows an excavator 58 with an identification transmitter 40 on its attachment 62. The vulnerable construction machines 54 can be equipped with an identification transmitter 40 temporarily or permanently. This makes them, or their attachments 62 or hooks 64, more visible and easier to detect for the construction machines 54 equipped with the device 20 according to the invention. Collisions between the construction machines 54 can thus be avoided because the operator is specifically warned of the presence of the other construction machines 54 with the identification transmitters 40. A more detailed evaluation and higher resolution of the image data or sensor signals from the environmental sensors 30 or cameras 32, and optional evaluation of other vehicle data, can be easily performed without any limitations in computing capacity.For example, a 3D reconstruction can be performed from the vehicle movement and the data from the environmental sensor 30, as well as from the data from the identification transmitter 40, only in the direction of the identification transmitter 40 on the tool 62 of the other machine 54. Additionally, the position of the tool 62 can also be evaluated, so that a computationally intensive 3D reconstruction and person or object recognition from the respective tool 62 is performed only in the direction of the identification transmitter 40 of the other machine 54.

[0060] Fig. 11 shows the schematic sequence of the inventive method for detecting hazards in an environment 44 of a vehicle 50, in particular a construction or agricultural vehicle or a mobile working machine 54. The method comprises several steps. In step S10, an identification signal from an identification transmitter 40 and a sensor signal from an environmental sensor 30 containing environmental information relating to the environment 44 are received. Step S12 involves determining a position from the identification signal. In step S14, an environmental area 42 is defined that includes the position, in particular the position of the identification transmitter 40. The environmental area 42 is part of the environment 44.

[0061] One step S16 relates to determining sensor data from the sensor signal of the environmental sensor, wherein the sensor data comprise the sensor signal in the defined environmental area 42, preferably the sensor data represent the part of the sensor signal that corresponds to the defined environmental area 42.

[0062] In step S18, the sensor data within the defined environmental area 42 are evaluated. Step S20 involves detecting a person 60 or an object 48 within the environmental area 42 based on the sensor data. Step S22 is aimed at retrieving information about the person 60 or the object 48. This involves detecting persons located within the environmental area 42. Furthermore, parts of persons 60, such as the torso, head, a single arm and / or a single leg, or just the legs or individual limbs, can also be detected. The same applies to retrieving information about the object 48. Here, too, parts of the object 48 can be detected.

[0063] In step S24, the determined information about person 60 or object 48 is output, or environmental information about the environment 44 or the surrounding area 42 is output. Alternatively and additionally, in this step a warning signal or a control signal can also be output to another element or unit of a vehicle, for example, the speed of the vehicle 50, or the movement of a tool 62, such as the boom of a crane or an excavator, can be limited or stopped.

[0064] Fig. 12 shows a graphical representation of a scheme for detecting hazards in the environment 44 of a vehicle 50. In step BS10, sensor signals from an environmental sensor 30, such as a camera 32, are processed. In step BS12, identification signals from an identification transmitter 40 are determined and preferably evaluated to ascertain, for example, data such as the position, type, purpose, or similar information of the identification transmitter 40 and the object or person equipped with it. This identification signal is processed together with the sensor signal in step BS14. In this step, the sensor signal is reduced based on the identification signal, so that sensor data for an environmental area within the vehicle's surroundings are determined. This results in a data reduction to the environmental area in which the identification transmitter is located.

[0065] The identification signal is further processed, and in step BS16, additional parameters are determined from the information, for example, to perform zooming, panning, compression, or noise reduction. For instance, in step BS18, parameters for zooming and similar functions can be provided, which can then be further processed in step BS14. From a data set D1, which comprises all data or the sensor signal for the entire environment 44 of the vehicle, reduced data D2 results after execution of step BS14. This reduced data set comprises the sensor data, specifically the sensor data, limited to the environmental area. In step BS20, further parameters are determined, which are necessary, for example, for object detection. These could be, for instance, soft points that indicate the accuracy of triangulation.In step BS22, the parameters for recognition are provided to be used in step BS24 for object recognition or the detection of objects or people. The resulting data set D3 comprises the sensor data, i.e., the data reduced from the sensor signal, as well as the data from a surround view and data on the detected objects.

[0066] Step BS26 involves determining further parameters used for the display. This could include, for example, a varying image resolution or an image resolution that can be changed or modified for specific areas. In step BS28, these parameters required for the display are made available so that the data D3 can be processed in step BS30 to enable a display. Alternatively or additionally, warnings can be generated, which can be issued in visual or audible form. For example, the screen color can change, a text message or warning symbols can appear, or the screen can flash. The display data is shown on the driver's screen in step BS32.

[0067] In addition to warnings or indicators, vehicle responses can be triggered, step BS34. Vehicle responses can include braking, stopping, shutting down, limiting a value range, limiting a slew range, or an action or control signal, for example, to return a crane arm to a predetermined position.

[0068] The invention has been comprehensively described and explained with reference to the drawings and the description. The description and explanation are to be understood as examples and not as limiting. The invention is not limited to the disclosed embodiments. Other embodiments or variations will become apparent to a person skilled in the art when using the present invention and upon a detailed analysis of the drawings, the disclosure, and the subsequent claims.

[0069] In the claims, the words "comprise" and "with" do not preclude the presence of further elements or steps. The undefined article "a" or "an" does not preclude the presence of multiple elements. A single element or unit can perform the functions of several of the units mentioned in the claims. An element, unit, interface, device, and module can be implemented partially or completely in hardware and / or software. The mere mention of some measures in several different dependent claims is not to be understood as precluding the advantageous use of a combination of these measures. Reference numerals in the claims are not to be interpreted restrictively.

[0070] Reference mark

[0071] system

[0072] device

[0073] Input interface

[0074] Processing unit Evaluation unit

[0075] Output interface

[0076] Assistance module

[0077] environmental sensor

[0078] camera

[0079] Picture

[0080] Stereo camera

[0081] Identification transmitter

[0082] surrounding area

[0083] Vicinity

[0084] Location

[0085] object

[0086] vehicle

[0087] crane

[0088] work machine

[0089] charge

[0090] Excavator

[0091] person

[0092] Tool

[0093] Hook

Claims

Patent claims 1. Device (20) for detecting hazards in an environment (44) of a vehicle (50), in particular a construction or agricultural vehicle or a mobile working machine (54), with - an input interface (22) for receiving an identification signal from an identification transmitter (40) and a sensor signal from an environmental sensor (30) with environmental information concerning the environment (44); - a processing unit (24) for determining a position from the identification signal and for defining an ambient area (42) that includes the position, wherein the ambient area (42) is part of the environment (44), and for determining sensor data from the sensor signal, wherein the sensor data includes the sensor signal in the defined ambient area (42); - an evaluation unit (26) for evaluating the sensor data in the defined environmental area (42) and detecting a person (60) or an object (48) within the environmental area (42) from the sensor data and determining information about the person (60) or the object (48); and - an output interface (28) for outputting information about the person (60) or object (48) or the environment information.

2. Device (20) according to claim 1, wherein the device (20) comprises a warning unit for determining a warning signal and the warning signal is based on the evaluation of the sensor data in the ambient area (42), and the output interface (28) is configured to output the warning signal.

3. Device (20) according to claim 1 or 2, wherein the sensor signal is a signal from a radar sensor, a lidar sensor, an optical sensor, a camera (32), a surround camera and / or one or more stereo cameras (36).

4. Device (20) according to one of the preceding claims, wherein the output interface (28) is configured to output an image (34) of the environment (44) or the surrounding area (42) based on the sensor signal.

5. Device (20) according to one of the preceding claims, wherein the identification signal comprises parameters for data reduction and the evaluation unit (26) is configured to perform the recognition of a person (60) and / or an object (48) based on parameter recognition, preferably by means of triangulation.

6. Device (20) according to one of the preceding claims, wherein the processing unit (24) is configured to perform the determination of the sensor data in a prioritized manner, preferably by allocating a higher processing capacity, more preferably by allocating a longer processing time.

7. System (10) for detecting hazards in an environment (44) of a vehicle (50), in particular a construction or agricultural vehicle or a mobile working machine (54), comprising a marking transmitter (40) for generating a marking signal with information on the position of the marking transmitter (40); an environment sensor (30) for generating a sensor signal with environment information relating to the environment (44); and a device (20) according to any of the preceding claims.

8. System (10) according to the preceding claim, wherein the environment sensor (30) comprises a radar sensor, a lidar sensor, an optical sensor, a camera (32), a surround camera and / or one or more stereo cameras (36).

9. Vehicle (50), in particular a construction or agricultural vehicle or mobile working machine (54), with a system (10) according to claim 7 or 8.

10. Method for detecting hazards in the environment (44) of a vehicle (50), in particular a construction or agricultural vehicle or a mobile working machine (54), comprising the following steps: - Receiving a marking signal from a marking transmitter (40) and a sensor signal from an environmental sensor (30) with environmental information concerning the environment (44); - Determining a position from the identification signal; - Defining an environment area (42) that includes the position, wherein the environment area (42) is part of the environment (44), - Determining sensor data from the sensor signal, wherein the sensor data includes the sensor signal in the specified environmental area (42); - Evaluating the sensor data in the defined environmental area (42) and - Detecting a person (60) or an object (48) within the surrounding area (42) from the sensor data; - Obtaining information about the person (60) or the object (48); and - Outputting information about the person (60) or object (48) or the surrounding area.

11. Method according to the preceding claim, wherein the identification signal comprises parameters on which the selection of the environmental area (42) and / or the determination of the environmental area (42) and / or the determination of the sensor data is based, wherein the identification signal is preferably evaluated multiple times and / or continuously and / or discretely in time.

12. Computer program product comprising program code which, when executed by a computer, causes the computer to perform the steps of the method according to claim 10.

Images