Method for monitoring the operating state of a self-propelled work machine

Abstract

The invention describes a method for monitoring the operating state of a self-propelled work machine (100), comprising the steps of: determining the camber of at least one wheel (10) of the self-propelled work machine (100), checking whether a predetermined safety requirement for an operating state of the self-propelled work machine (100) is met, wherein the checking step is performed depending on the determined camber, and outputting a control signal to a device (20) of the self-propelled work machine (100) based on a test result resulting from the checking step. The invention also describes a control device (110) configured to perform the method and a monitoring system (120) comprising a camera (30) and such a control device (110).

Description

Technical area

[0001] The present invention relates to a method for monitoring an operating state of a self-propelled work machine. The present invention also relates to a control device for outputting a control signal to a device of a self-propelled work machine. The invention also relates to a monitoring system for monitoring an operating state of a self-propelled work machine. State of the art

[0002] It is known from the prior art to monitor the operation of a vehicle using sensors. DE 10 2020 120 103 A1 discloses determining the position of a wheel on a vehicle. For this purpose, a camera mounted on the vehicle detects a wheel to check for camber. In connection with such a sensory check, a wheel misalignment is output. Description of the invention

[0003] In one aspect, the present invention relates to a method for monitoring an operating state of a self-propelled work machine. The self-propelled work machine can be a vehicle that can be configured to perform a job. The self-propelled work machine can be a commercial vehicle that can be configured to transport people or goods. The self-propelled work machine can be a construction machine that can be configured to perform work on a construction site. The construction machine can be, for example, a wheel loader.

[0004] According to respective embodiments, the operating state can be a terrain state, a movement state, a loading state, or a working state. The terrain state can be a state of the terrain on which the self-propelled work machine is located. The movement state can be a longitudinal dynamic state of the self-propelled work machine. Alternatively or additionally, the movement state can be a transverse dynamic state of the self-propelled work machine. The loading state can be a loading state, which can exist during or after the self-propelled work machine picks up a load. Alternatively or additionally to the loading state, the loading state can be an unloading state, which can exist during or after the self-propelled work machine releases a load. The working state can be a state of a work tool of the self-propelled work machine.

[0005] The method includes, as one step, determining a wheel camber of at least one wheel of the self-propelled work machine. The wheel camber can comprise a camber angle between the wheel center plane of the wheel and a horizontal plane. The horizontal plane can be spanned in a vehicle coordinate system, wherein the horizontal plane can be spanned by the vehicle's longitudinal axis and the vehicle's transverse axis of the self-propelled work machine. Alternatively, the horizontal plane can be spanned in a terrain coordinate system in which the self-propelled work machine is located.

[0006] The method comprises a further step of checking whether a predetermined safety requirement for an operating state of the self-propelled work machine is met. The safety requirement can define a maximum value at which operational safety can be ensured by tipping or falling of the self-propelled work machine depending on the wheel camber. According to respective embodiments, the safety requirement can be a terrain requirement, a movement requirement, a loading requirement, or a work requirement. The terrain requirement can specify the condition of the terrain under which the self-propelled work machine can be operated safely on the terrain or a work can be carried out safely with the self-propelled work machine.The movement specification can specify the longitudinal dynamic state or the transverse dynamic state, in which case the self-propelled machine can be moved safely or the work with the self-propelled machine can be carried out safely. The loading specification can specify the loading state or the unloading state, in which case the self-propelled machine can be operated or moved safely or the work with the self-propelled machine can be carried out safely. The work specification can specify the state of a work device of the self-propelled machine, in which case the work device can be operated safely or the work with the work device can be carried out safely.The work equipment can be a component of a drive train of the self-propelled work equipment or a work tool of the self-propelled work machine. The work tool can, for example, be a tool for picking up a load or transporting a load with the self-propelled work machine.

[0007] The testing step is performed depending on the determined wheel camber. The testing step can be performed if the determined wheel camber exceeds a predetermined critical wheel camber. The critical wheel camber can be predetermined depending on a load-specific operating parameter of the self-propelled work machine. Alternatively or additionally, the critical wheel camber can be predetermined depending on a terrain-specific operating parameter of the self-propelled work machine. Further alternatively or additionally, the critical wheel camber can be predetermined depending on a movement-specific operating parameter of the self-propelled work machine.

[0008] The testing step can result in a test result that the predetermined safety specification for the operating state of the self-propelled work machine is met. The testing step can result in at least one of the terrain specification, the movement specification, the loading specification, and the work specification being met. The method has a further step of outputting a control signal to a device of the self-propelled work machine based on a test result resulting from the testing step. The control signal can be output if at least one of the test results is met. The device can be the work device or another work device. The device can also be a user interface.

[0009] The method can be used to check for a safe operating condition of the self-propelled work machine based on the wheel camber and in compliance with a predetermined safety requirement. Furthermore, the control signal can be output only if the predetermined safety requirement is not met. The predetermined safety requirement can already be met before the testing step, or an operator of the self-propelled work machine can ensure that the predetermined safety requirement is met after the testing step. The steps of the method can be performed repeatedly or iteratively in order to continuously check and ensure compliance with the safety requirement during operation of the self-propelled work machine.

[0010] According to one embodiment of the method, this comprises, as a further step, detecting the at least one wheel with a camera arranged on the self-propelled work machine. The camera can be a 360° camera. The camera can be arranged on the self-propelled work machine above the at least one wheel. The wheel camber can thus be inclined depending on the camber angle to a main recording direction of the camera. According to this embodiment, in the step of determining the wheel camber, the wheel camber can be determined image-based based on a camera image of the at least one wheel captured by the camera. The step of determining the wheel camber can therefore be carried out exclusively camera-based.

[0011] In the step of determining the wheel camber, the wheel camber can be determined from the camera image using image processing methods. The image processing methods can include, for example, template matching, pattern recognition, or contour detection of the wheel in the camera image. The wheel camber can therefore be estimated using an image-based approach based on the at least one detected wheel. The wheel camber can also be determined based on a parameterization of a geometric parameter of the wheel, whereby the geometric parameter can be estimated or derived from the camera image using an image-based approach using a classifier. The wheel camber can thus be efficiently determined using a camera-based approach.

[0012] According to a further embodiment of the method, an outer wheel surface of the wheel can be captured with the camera in the capturing step. The outer wheel surface of the wheel can be delimited by an outer contour of a tire of the wheel. The outer wheel surface can be at least a region of a tread of the tire. Alternatively or additionally to the tread, the outer wheel surface can be at least a region of a sidewall of the tire. Alternatively or additionally, the outer wheel surface of the wheel can be delimited by an outer contour of a rim of the wheel. Further alternatively or additionally, the outer wheel surface of the wheel can be delimited by an outer contour of a hub cap of the wheel. In the camera image captured with the camera, the outer wheel surface of the wheel can be delimited, at least in sections, in a conical section by at least one of the outer contours.According to this embodiment, in the determining step, the wheel camber can be determined image-based based on the detected outer wheel surface. The wheel camber can thus be reliably determined using a camera arranged above the wheel on the self-propelled work machine.

[0013] According to a further embodiment of the method, this can comprise, as a further step, outputting the predetermined safety specification for the operating state of the self-propelled work machine to a user interface of the self-propelled work machine. The step of outputting the predetermined safety specification can be performed before the checking step. The predetermined safety specification can be output to an operator of the self-propelled work machine via the user interface. The predetermined safety specification can be output via the user interface as a warning to maintain or establish safe operation of the self-propelled work machine, which may already be present or may only be present upon initiation of the safety specification.

[0014] The outputting step can be performed depending on the determined wheel camber. The outputting step can be performed when the determined wheel camber exceeds the predetermined critical wheel camber. In the checking step, it can thus be checked whether the outputted predetermined safety specification for the operating state of the self-propelled work machine has been brought about by the operator. The method can thus improve occupational safety for the operator or the occupational safety emanating from the self-propelled work machine, whereby the method can support the operator in bringing about and fulfilling the predetermined safety specification.

[0015] According to a further embodiment of the method, this can comprise, as a further step, recording the operating state of the self-propelled work machine with a camera arranged on the self-propelled work machine. The camera can be the camera arranged on the self-propelled work machine for recording the at least one wheel. The checking step can be carried out image-based on the recorded operating state. In the recording step, the predetermined safety specification for the operating state can be recorded with the camera or recognized based on a camera image recorded with the camera. In the checking step, the recorded safety specification can therefore be compared with the predetermined safety specification. The steps of the method can therefore be carried out autonomously by a camera system, which can have the camera.

[0016] According to a further embodiment of the method, the predetermined safety specification can comprise a predetermined load-specific safety specification that specifies a load-specific operating parameter of the self-propelled work machine. Alternatively or in addition to the predetermined load-specific safety specification, the predetermined safety specification can comprise a predetermined terrain-specific safety specification that specifies a terrain-specific operating parameter of the terrain on which the self-propelled work machine is located.

[0017] If the predetermined safety specification includes the predetermined load-specific safety specification, the predetermined safety specification may be the loading specification. The load-specific operating parameter of the self-propelled work machine may specify a loading volume, a loading location, or a loading weight of a cargo or load to be picked up or processed by the self-propelled work machine. The loading volume, the loading location, or the loading weight can be recorded with the camera. The predetermined load-specific safety specification may also be a predetermined load-specific safety specification. The predetermined load-specific safety specification may be predetermined depending on the wheel camber.

[0018] If the predetermined safety requirement includes the predetermined terrain-specific safety requirement, the predetermined safety requirement can be the terrain specification. The terrain-specific operating parameter can be a slope of the terrain in the transverse or longitudinal direction of the self-propelled work machine. The slope of the terrain can be recorded with the camera. The terrain-specific operating parameter can also be a surface condition of the terrain. The surface condition can be recorded with the camera. The predetermined terrain-specific safety requirement can be predetermined depending on the wheel camber.

[0019] At least one current parameter from the load-specific operating parameter and the terrain-specific operating parameter can be recorded using the camera mounted on the self-propelled work machine. In the checking step, an image-based check can thus be performed to determine whether the predetermined load-specific operating parameter is met by the recorded load-specific operating parameter. Alternatively or additionally, the checking step can also determine whether the predetermined terrain-specific operating parameter is met by the recorded terrain-specific operating parameter. A review of the safety requirement can thus be comprehensively monitored using images during operation of the self-propelled work machine.

[0020] According to a further embodiment of the method, the predetermined safety specification can comprise a predetermined movement-specific safety specification, which can specify a movement-specific operating parameter of the self-propelled work machine. If the predetermined safety specification comprises the predetermined movement-specific safety specification, the predetermined safety specification can be the movement specification. The movement-specific operating parameter can be a longitudinal dynamic operating parameter of the self-propelled work machine. Alternatively or additionally, the movement-specific operating parameter can be a transverse dynamic operating parameter of the self-propelled work machine.

[0021] The longitudinal dynamic working parameter can be a longitudinal speed or a longitudinal acceleration of the self-propelled machine. The lateral dynamic working parameter can be a steering angle, a steering angle profile, or a bend angle of the self-propelled machine. The steering angle, the steering angle profile, or the bend angle can be recorded with the camera. Egomotion methods, for example, can be used for this purpose. The predetermined movement-specific safety requirement can be predetermined depending on the wheel camber. In the testing step, it can be checked whether the predetermined movement-specific working parameter is met by the recorded movement-specific working parameter. A review of the safety requirement can thus also be monitored while taking into account the dynamics of the self-propelled machine.

[0022] According to a further embodiment of the method, in the step of outputting the control signal, the control signal can be output to a user interface of the self-propelled work machine. The user interface of the self-propelled work machine can be arranged at a workstation of the self-propelled work machine in which an operator of the self-propelled work machine can be located. In the step of outputting the control signal, the control signal can be output to communicate a safety warning to the operator of the self-propelled work machine if the resulting test result indicates that the predetermined safety requirement is not met.Thus, after the predetermined safety requirement has been issued, depending on whether the predetermined safety requirement is not present or has not been brought about by the operator, the safety warning can be issued to the operator in order to bring about the fulfillment of the safety requirement.

[0023] According to a further embodiment of the method, in the step of outputting the control signal, the control signal can be output to a drive train or the work tool to prevent the safety requirement from being exceeded. The drive train or the work tool can be controlled based on the control signal in such a way that maintaining or exceeding the predetermined safety requirement is prevented. For this purpose, the operation of the drive train or the work tool can be intervened in.

[0024] In a further aspect, the present invention relates to a control device for outputting a control signal to a device of a self-propelled work machine. The device can be the device described in the previous aspect. The control device is configured to carry out the method according to one of the preceding claims.

[0025] In a further aspect, the present invention relates to a monitoring system for monitoring an operating state of a self-propelled work machine. The monitoring system has a camera for detecting at least one wheel of the self-propelled work machine. The camera can be the camera described in the previous aspects. The monitoring system has the control device according to the previous aspect, which is additionally configured to carry out at least one step of the method image-based on the detected at least one wheel. According to one embodiment, the monitoring system is retrofitted to the self-propelled work machine. The monitoring system can be a self-sufficient safety system. Short description of the characters Fig. 1 shows a self-propelled work machine with a control device and a monitoring system for monitoring an operating state of the self-propelled work machine according to respective embodiments. Fig. 2 shows a flowchart with method steps for monitoring an operating state of the self-propelled work machine according to an embodiment. Detailed description of embodiments

[0026] Fig. 1 shows a self-propelled work machine 100. The self-propelled work machine 100 has a device 20, which according to one embodiment is a work tool 22 for picking up a load. The work tool 22 is a loading shovel for picking up and transporting the load. The self-propelled work machine 100 has wheels 10 with which the self-propelled work machine 100 is moved to transport the load picked up by the work tool 22. The wheels 10 are arranged in an exposed manner on the self-propelled work machine 100. The self-propelled work machine 100 has a monitoring system 120 for monitoring an operating state of the self-propelled work machine 100. According to one embodiment, the operating state is a loading state of the work tool 22, wherein the loading state indicates a loading volume of the load.

[0027] The monitoring system 120 has a camera 30 arranged above the wheels 10 on the self-propelled work machine 100. The camera 30 is configured to capture at least one camera image of the outer wheel surfaces 12 of the wheels 10. The outer wheel surfaces 12 of the wheels 10 can be captured as a function of a respective wheel camber of the wheels 10, wherein the geometry of the outer wheel surfaces 12 of the wheels 10 visible in a camera image captured by the camera 30 depends on the currently existing wheel cambers of the wheels 10. The monitoring system 120 also has a control device 110 connected to the camera 30 and a user interface 40. The user interface 40 is configured to output a safety warning to an operator of the self-propelled work machine 100 (not shown in the figures).

[0028] Fig.Figure 2 shows method steps performed by monitoring system 120 to monitor the operating state of self-propelled work machine 100. In a first step S1, at least one of wheels 10 is captured with camera 30. According to one embodiment, the outer wheel surface 12 is captured in a camera image of wheels 10 captured by camera 30.

[0029] In a further step S2, the camber of wheel 10 is determined. The camber is determined based on machine learning methods, wherein the visible geometry of the outer wheel surface 12 is used to determine whether the camber is critical for the safe operation of the self-propelled work machine 100. Critical camber exists when the detected camber exceeds a critical camber threshold. According to one embodiment, the camber of the wheel 10 associated with the outer wheel surface 12 is automatically derived from a detected outer contour of the outer wheel surface 12 using image processing and machine learning methods.

[0030] If the wheel camber is determined to be critical in step S2, a predetermined safety specification for the loading state of the work tool 22 is output to the operator of the self-propelled work machine 100 via the user interface 40 in a further step S3. The predetermined safety specification includes a predetermined loading state of the work tool 22. In a further step S4, the current loading state of the work tool 22 is recorded with the camera 30. The camera 30 quantitatively or qualitatively records that the work tool 22 is filled with the material in order to determine the current loading volume.

[0031] In step S3, the predetermined loading state of the work tool 22 can be output depending on the detected wheel camber, wherein the predetermined loading state is a permissible loading volume or permissible load weight depending on the wheel camber. In a further step S5, it is checked whether the predetermined loading state is met with the detected loading state. In step S5, it is checked whether the current loading volume exceeds the permissible loading volume.

[0032] If a test result P resulting from the checking step indicates that the current loading volume exceeds the permissible loading volume depending on the wheel camber, the control device 110 outputs a control signal to the user interface 40 in a further step S6. The control signal contains an output command to output a safety warning to the operator of the self-propelled work machine 100 via the user interface 40. If a test result P resulting from the checking step indicates that the current loading volume does not exceed the permissible loading volume depending on the wheel camber, steps S1 to S5 of the method are repeatedly performed to continuously monitor the operational safety of the self-propelled work machine 100. Reference symbol 10 wheels 12 Wheel outer surface 20 Facility 22 Work tools 30 Camera 40 User interface 100 self-propelled work machines 110 Control device 120 surveillance system P test result S1 camera-based detection of a wheel S2 Determining a wheel camber S3 Issue a security target S4 Recording operating status S5 Checking a predetermined safety requirement S6 Outputting a control signal QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] DE 10 2020 120 103 A1

[0002]

Claims

[1] Method for monitoring an operating state of a self-propelled work machine (100), comprising the step of determining (S2) a wheel camber of at least one wheel (10) of the self-propelled work machine (100), characterized by the further steps: checking (S5) whether a predetermined safety requirement for an operating state of the self-propelled work machine (100) is met, wherein the step of checking (S5) is carried out as a function of the determined wheel camber, and outputting (S6) a control signal to a device (20) of the self-propelled work machine (100) based on a test result (P) resulting from the step of checking (S5). [2] Method according to claim 1, comprising the further step of detecting (S1) the at least one wheel (10) with a camera (30) arranged on the self-propelled work machine (100), wherein in the step of determining (S2) the wheel camber, the wheel camber is determined on the basis of the detected at least one wheel (10) in an image-based manner. [3] Method according to claim 2, wherein in the step of detecting (S1) an outer wheel surface (12) of the wheel (10) is detected with the camera (30), and wherein in the step of determining (S2) the wheel camber is determined on the basis of the detected outer wheel surface (12) in an image-based manner. [4] Method according to one of the preceding claims, comprising the further step of outputting (S3) the predetermined safety specification for the operating state of the self-propelled work machine (100) to a user interface (40) of the self-propelled work machine (100), wherein the step of outputting (S3) is carried out as a function of the determined wheel camber. [5] Method according to one of the preceding claims, comprising the further step of detecting (S4) the operating state of the self-propelled work machine (100) with a camera (30) arranged on the self-propelled work machine (100), wherein the step of checking (S5) is carried out on the basis of the detected operating state in an image-based manner. [6] Method according to one of the preceding claims, wherein the predetermined safety specification comprises at least one of a predetermined load-specific safety specification which specifies a load-specific operating parameter of the self-propelled work machine (100), and a predetermined terrain-specific safety specification which specifies a terrain-specific operating parameter of the terrain on which the self-propelled work machine (100) is located. [7] Method according to one of the preceding claims, wherein the predetermined safety specification comprises a predetermined movement-specific safety specification which specifies a movement-specific operating parameter of the self-propelled work machine (100). [8] Method according to one of the preceding claims, wherein in the step of outputting (S6) the control signal, the control signal is output to a user interface (40) of the self-propelled work machine (100) for communicating a safety warning to an operator of the self-propelled work machine (100) if the resulting test result (P) is that the predetermined safety requirement is not met. [9] Control device (110) for outputting a control signal to a device (20) of a self-propelled work machine (100), characterized by that the control device (110) is configured to carry out the method according to one of the preceding claims. [10] Monitoring system (120) for monitoring an operating state of a self-propelled work machine (100), comprising a camera (30) for detecting at least one wheel (10) of the self-propelled work machine (100), and the control device (110) according to claim 9, which is configured to carry out at least one step of the method in an image-based manner based on the detected at least one wheel (10).

Images