Method for animating object recognition and vehicle assistance system driving intentions, assistance system, computer program, and computer-readable (storage) medium

Abstract

A solution is presented on how the vehicle driver's perception of assistance system information can be improved to facilitate system understanding. A method for animating the object recognition and driving intention of an assistance system for automated steering of a vehicle, comprising: -Representing object recognition through an assistance system; -Animating object recognition and wherein the vehicle is represented by an avatar 7 and the interaction of the avatar 7 with the detected objects is performed in an animated manner, - the avatar 7 mimics additional actions 9 in the animation of object recognition; - the driving intention of the assistance system, which indicates the presence of a control strategy, is expressed, and -The driving intention of the assistance system is expressed, and the driving intention is imitated by an avatar 7 in the animation.

Description

[Technical field]

[0001] The invention relates to a method for animating at least one object recognition and driving intention of an assistance system of a vehicle. Furthermore, the invention relates to an assistance system for a vehicle. Finally, the invention relates to a computer program and a computer readable (storage) medium. [Background technology]

[0002] Different driver assistance systems are known from the prior art, which support the driver when maneuvering a vehicle. For example, speed control systems (called Adaptive Cruise Control in English, ACC for short), lane change assistance systems and other assistance systems for at least partially automated maneuvering of a vehicle are known. Such assistance systems usually include a number of surrounding sensors capable of detecting objects in the vehicle's surroundings. The at least partially automated maneuver can be planned by means of the assistance system depending on the objects detected in the vehicle's surroundings. The objects detected in the surroundings can influence the control strategy of the assistance system during the at least partially automated maneuver.

[0003] Although the steering or control of the objects detected in the surroundings is at least partially automated, the driver of the vehicle is currently often responsible for the assistance system to operate correctly and in accordance with the rules. Therefore, it is essential that the driver of the vehicle always has a good understanding of the system, regardless of the current traffic situation. Therefore, it is necessary for the driver to be informed of the driving intention of the assistance system and its object recognition in appropriate traffic situations so that he or she can intervene to correct it if necessary.

[0004] For this purpose, current assistance systems known from the prior art inform the driver of the vehicle by displaying the relevant information on a combination meter, a head-up display and / or the like, but this does not guarantee that the driver's perception of the displayed information is in a purposeful and intuitive manner, and therefore an intuitive understanding of the system by the driver cannot be guaranteed in all traffic situations.

[0005] Patent document 1 relates to a method for operating a driver assist system of a vehicle comprising the steps of recognizing objects in the surroundings of the vehicle based on surrounding data, providing a display having the objects to be displayed and a displayed surroundings depicting the recognized objects and / or the surroundings of the vehicle, executing a response of the driver assist system depending on the shown objects and / or the displayed surroundings, providing a user interface for modifying the displayed objects and / or the displayed surroundings by operational input, and adapting the response of the driver assist system depending on the modification of the displayed objects and / or the displayed surroundings made in the operational input. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] German Patent Application No. 102020106380 Summary of the Invention [Problem to be solved by the invention]

[0007] The object of the invention is to show a solution in the case of an assistance system how the perception of the assistance system information by the driver of the vehicle can be improved in order to facilitate system understanding. [Means for solving the problem]

[0008] This problem is solved according to the invention by a method, by an assistance system, by a computer program and by a computer-readable (storage) medium. Advantageous developments of the invention are set forth in the respective dependent claims.

[0009] The method according to the invention is used for the animated representation (illustration) of at least one object perception and driving intention of an assist system for at least partially automated steering of a vehicle. The method can also be used for operating the assist system. The method includes representing at least one object perception by the assist system if an object detected in the at least one object perception affects the at least partially automated steering of the assist system in the vehicle's surroundings. In addition, the method animates the at least one object perception, the vehicle being represented by an avatar and the interaction of the avatar with the detected object being performed in the animation. The avatar then imitates at least one additional action in the animation of the at least one object perception if the object detected in the vehicle's surroundings affects the control strategy. In addition, the method includes representing a driving intention of the assist system representing the presence (intention) of a control strategy influenced by the detected object. Finally, a representation of the driving intention of the assist system is performed, the driving intention being imitated by the avatar in the animation in order to improve the system understanding of the vehicle driver.

[0010] In this regard, the concept of "at least partially automated" driving or maneuvering includes automated driving at any degree of automation, with exemplary degrees of automation being (in order of increasing degrees of automation) assisted driving, partially automated driving, highly automated driving, fully automated driving, and autonomous driving.

[0011] The concept of object recognition and object interpretation of the assist system as used herein refers to a step of object detection and is often called a sense step in a given control paradigm, for example in the control of a robot (based on English or technical terms, such a step is known in the Sense-Plan-Act-Paradimas). In this context, the concept (term) of driving intention of the assist system as used herein can be understood as the actual intention or actual action goal of a plan step.

[0012] Here, the actual intention or the actual operating target can be achieved in different ways. For example, a reduction in the actual driving speed of the vehicle can be achieved by a reduction in the drive torque and by a braking intervention. In particular, the realization of the driving intention may require several different interventions. The concept (term) of an affected control strategy as used in this specification can therefore be understood as a changing driving intention of the assistance system based on object recognition.

[0013] Thus, the method can be used to improve the driver's system understanding of an assistance system for at least partially automated steering of a vehicle. In the method, at least one object perception and driving intention of the assistance system can be represented by animation (animation). If the assistance system detects, for example by a camera, a traffic sign indicating a maximum permitted speed and requiring a reduction in the current driving speed of the vehicle, the detected traffic sign influences the control strategy of the assistance system. Relevant information that contributes to improving the system understanding of the assistance system of the vehicle can then be animatedly represented to the driver of the vehicle. The driver can thus be intuitively informed at all times of the object perception of the assistance system and its driving intention.

[0014] According to the above mentioned case, the assistance system can be, for example, a speed control system with knowledge of the currently permitted maximum speed. In this example, the detected object can be a traffic sign, with a traffic sign drawn on the traffic sign indicating the maximum permitted speed. However, further assistance systems or assistance functions are also conceivable. In addition, the detected object can be any object in the vehicle's surroundings, i.e., for example another traffic participant, an obstacle, a lane marking or the like.

[0015] In the animated display, the vehicle or the ego-vehicle can be represented by means of an avatar. The avatar can be, for example, a Chinese dragon, a cowboy riding a horse, a dog sled or the like. If an object, for example a traffic sign, is detected in the vehicle's surroundings in the at least one object recognition, the detected object can be illustrated with an avatar representing the vehicle. In addition, the illustration of the object or at least one object recognition can be animated, i.e. time-varying. By animation, i.e. by displaying a number of individual images that suggest a moving image to the driver, an improved understanding of the system can be achieved. Furthermore, the driver's perceivability of the illustrated information can be improved.

[0016] It should be emphasized that the avatar can interact with the detected objects in the animation. Such an interaction can be, for example, in the case of a Chinese dragon, the breathing of fire in the direction of the detected object. In the case of a cowboy on horseback, it can be illustrated in the form of a spinning lasso that catches the detected object.

[0017] In other words, the avatar's interaction with the detected objects in the animation therefore mimics additional actions, which are only mimicked by the avatar if the objects detected in the vehicle's surroundings affect the control strategy. Thus, in the above example with a traffic sign limiting the maximum permitted speed, additional actions can be mimicked if a reduction in the vehicle's current speed is required or if the current driving speed is greater than the maximum permitted speed.

[0018] Furthermore, in the animated representation (illustrated), the driving intention of the assist system can also be represented (illustrated). While the control strategy can be influenced in various ways, the driving intention of the assist system can show the driver of the vehicle the actual intention of the assist system. The display of the driving intention can then be done in an animated manner, which contributes to improving the driver's understanding of the system.

[0019] For example, when a traffic sign that allows a maximum permitted speed that is greater than the vehicle's current driving speed is detected by the assist system, the control strategy of the at least partially automated steering of the assist system may be affected. If the vehicle is represented in the animation as, for example, a cowboy on horseback, the object recognition of at least one of the traffic signs is animatedly represented as the cowboy on horseback catching the traffic sign with his lasso. This first indicates to the driver of the vehicle that an object, in this case a traffic sign, has been detected and this will affect the control strategy.

[0020] However, at this point, the result of the influence of the control strategy is not yet recognized by the driver of the vehicle. That this may lead to a higher driving speed can be notified to the driver by the animated representation of the driving intention of the assistance system. At this time, for example, a cowboy's horse can start galloping more or less quickly depending on the intended acceleration by clattering its hooves.

[0021] In summary, therefore, by using the method according to the invention for animated illustration of at least one object recognition and driving intention of the assistance system, it is possible to ensure that the driver perceives the relevant information of the assistance system better and thus obtains a better system understanding. The animated representation can improve the driver's perception compared to a static representation.

[0022] In addition, it is advantageous if at least one animation parameter is determined depending on the at least one object recognition and / or depending on the driving intention of the assistance system, and the at least one animation parameter influences the animation. For example, it is possible to change the avatar of the vehicle depending on the detected object and / or depending on the driving intention. In addition, it is conceivable that the avatar imitates in the animation of the at least one object recognition depending on the object recognition and / or the driving intention of the assistance system. It is also possible that the animation parameter influences the speed of the animated representation.

[0023] Therefore, the animation parameters further improve the driver's perception of the information of the assist system and enhance understanding of the system.

[0024] Here, it is particularly advantageous if the at least one animation parameter represents at least one urgency of the execution of the control strategy and the animation is performed depending on the urgency. For example, it may be necessary to strongly reduce the current driving speed of the vehicle due to another traffic participant entering in front of the vehicle. The detected object, i.e. in this example the entering traffic participant, can be detected in the object recognition by the assistance system. The at least one animation parameter representing at least one urgency of the execution of the control parameter can be determined by a time (in English, in technical terms, called Time-to-Collision) representing, for example, the time until the collision of the vehicle with the entering traffic participant.

[0025] Furthermore, depending on the at least one animation parameter or the urgency, it is conceivable, for example, to influence the animation speed. In addition, depending on the urgency, it is conceivable that the avatar mimics a different behavior in the animation of the at least one object perception, by which a driver of the vehicle can actually become aware of a critical situation and possibly intervene.

[0026] Finally, it is also advantageous that the animated representation is performed so that the driving intention of at least one object recognition or assistance system is recognized by the driver essentially by viewing in the peripheral field. Here, viewing in the peripheral field is a recognition in an area of ​​the driver's field of vision that is not depicted on the central retina. In peripheral (or indirect) viewing, the driver practically passes over (looks at) the animated representation. Viewing in the peripheral field is very effective for the recognition of movement. Thus, on the one hand, the driver of the vehicle is not disturbed by the animated representation of the object recognition and driving intention, and on the other hand, viewing in the peripheral field ensures an effective recognition by the animated representation of the avatar, i.e. the movement. Overall, therefore, a synergistic effect is obtained by the animated representation in an area that is recognized by viewing in the peripheral field of the driver of the vehicle and is therefore outside the driver's actual field of vision that is directed towards the road.

[0027] The assistance system for a vehicle according to the invention comprises a display device configured for at least one object recognition and driving intention of the assistance system by animation according to the method according to the invention. In addition, the assistance system may comprise at least one surroundings sensor and an electronic control or computing device. The assistance system may be configured in particular for at least partially automated maneuvering, i.e. for example for partially automated longitudinal and / or lateral driving.

[0028] A vehicle according to the invention, in particular a private car, comprises an assistance system according to the invention.

[0029] Another aspect of the invention relates to a computer program comprising commands which, when executed by a computing device, cause the computing device to animate the at least one object recognition and driving intention of the assistance system for the at least partially automated steering of a vehicle on a display device according to the inventive method of the assistance system.Furthermore, the invention relates to a computer-readable (storage) medium, characterized in that, when executed by a computing device, the program comprises commands which, when executed by a computing device, cause the computing device to animate the at least one object recognition and driving intention of the assistance system for the at least partially automated steering of a vehicle on a display device according to the inventive method of the assistance system.

[0030] In this case, the display device may be, for example, a monitor, a part of a combination meter, a display, a head-up display, or the like. Here, the computing device includes, in particular, at least one processor. In addition, the computing device may include a storage device (memory). The computing device may be formed by at least one electronic control device.

[0031] The preferred embodiments and advantages presented for the method according to the invention apply correspondingly to the assistance system according to the invention, to the vehicle according to the invention, to the computer program according to the invention and to the computer-readable (storage) medium according to the invention.

[0032] Further features of the invention are evident from the claims, the drawings and the description thereof. The features and combinations of features described hereinabove and those merely shown in the description and / or drawings below can be used not only in the respective combinations described, but also in other combinations or alone, without departing from the scope of the invention.

[0033] The present invention will now be described in detail based on preferred embodiments with reference to the accompanying drawings. [Brief description of the drawings]

[0034] [Figure 1] FIG. 2 is a schematic diagram of a vehicle display panel including a display device that suggests an animated illustration of object recognition. [Diagram 2] FIG. 1 shows a schematic diagram of a display panel of a vehicle, the display panel including a display device with suggested animated illustrations of object recognition and driving intentions. [Diagram 3] FIG. 3 shows a schematic diagram in enlarged form of the display device according to FIG. 2; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0035] In the various figures, identical or functionally identical elements are provided with the same reference numbers.

[0036] 1 shows a schematic representation of a display panel 2 of a vehicle 1, which includes a display device 3 suggesting an animated illustration of the object perception of an assistance system of the vehicle 1. Furthermore, the surroundings 4 of the vehicle 1 are shown from the point of view of the driver of the vehicle 1, said point of view being seen through the windshield of the vehicle 1.

[0037] The assistance system can for example be formed as a speed control system with traffic sign recognition, distance control and / or the like. The display device 3 diagrammatically illustrates the object recognition of the assistance system. In object recognition, the assistance system detects a stop sign 5. A stop sign 5 in the surroundings 4 of the vehicle 1 is illustrated by a virtual stop sign 5' in the display device 3. A stop line 6 in the surroundings 4 of the vehicle 1 is illustrated by a virtual stop line 6' in the display device 3. The vehicle 1 is illustrated with an avatar 7. The avatar 7 is illustrated as a Chinese dragon in the example of Fig. 1.

[0038] In the example of FIG. 1, the assistance system takes over the longitudinal driving (longitudinal guidance) of the vehicle 1. The vehicle 1 then moves in the perimeter 4 towards an intersection 8. At the intersection 8, the right of way is provided by a stop sign 5. The stop sign 5 is detected by the assistance system during object recognition. However, at the moment of detection of the stop sign 5 by the assistance system of the vehicle 1, i.e. at the moment of object recognition, the driver of the vehicle 1 is not necessarily aware that the stop sign 5 is accurately detected by the assistance system of the vehicle 1.

[0039] Thus, in some circumstances, the driver is even less aware that the control strategy of the at least partially automated steering of the assistance system, i.e. in particular the longitudinal control of the vehicle 1 by the assistance system, is influenced. Thus, in some circumstances, the driver of the vehicle 1 is not aware that a stop sign 5 has been detected and that the assistance system subsequently changes its control strategy. Thus, in other words, in some circumstances, the driver is not aware that the assistance system of the vehicle 1 intentionally stops at the stop line 6.

[0040] The detection by the assistance system of the stop sign 5 during object recognition can be made evident to the driver of the vehicle 1 in at least one object recognition animation by at least one additional action imitated by the avatar (here, breathing fire from the mouth 9). Alternatively, the additional action imitated by the avatar can be eating up the virtual stop sign 5'.

[0041] In summary, therefore, by using the method according to the present invention for animated illustration of at least one object recognition and driving intention of the assistance system, it is possible to ensure that the driver better perceives the relevant information of the assistance system and thus obtains a better system understanding.

[0042] 2 shows a schematic representation of a display panel 2 of a vehicle 1, which includes a display device 3 suggesting an animated illustration of the object recognition and driving intention of the assistance system of the vehicle 1. Furthermore, the surroundings 4 of the vehicle 1 are shown from the viewpoint of the driver of the vehicle 1, said viewpoint being seen through the windshield of the vehicle 1.

[0043] A vehicle 1 travels on a highway 10 having a roadway with three lanes in a perimeter 4. At this time, the vehicle 1 travels in a left lane 14 of the roadway. Furthermore, a lane reduction sign 11 indicates the end of the left lane 14.

[0044] There is another traffic participant 12 in the middle lane. In the case of an assistance system operating, which in this example controls at least one lateral and longitudinal steering of the vehicle 1, the driver of the vehicle 1 is not aware, in the example shown here, that the lane decrease sign 11 and the end of the left lane 14 have been correctly detected by the assistance system. This can be indicated to the driver of the vehicle 1 by means of the display device 3.

[0045] In the example according to Fig. 2, another traffic participant 12 can be shown on the display device 3 by means of an additional avatar 12' (shown in this example as a sheep). The lane decrease sign 11 can be shown by a virtual lane decrease sign 11' on the display device 3. The detection of the lane decrease sign 11 by the assistance system is signalled to the driver by imitating at least one additional action of the avatar 7 (here signalled as fire breathing 9).

[0046] However, depending on the circumstances, at this point in time, the driver of the vehicle 1 is not aware of the consequences of the lane reduction sign 11 for the control strategy or driving intention of the assistance system. In other words, it may be unclear to the driver what driving intention the assistance system has. The assistance system may therefore strongly slow down the vehicle 1 in the left lane 14 and move it in behind the two other traffic participants 12. Similarly, the assistance system may then have the driving intention to accelerate the vehicle 1 in order to move in between the two other traffic participants 12.

[0047] In the example of Fig. 2, the driving intention of the assistance system is first to accelerate the vehicle 1 and then to move it between two other traffic participants 12. This is suggested by the illustration by the trajectory 13. In an animated illustration of the driving intention of the assistance system, an avatar 7 (here exemplarily suggested as a Chinese dragon) can move along the trajectory 13 and clearly indicate the driving intention of the assistance system to the driver of the vehicle.

[0048] Depending on how strongly the vehicle 1 should accelerate in order to get between the other traffic participants 12, for example animation parameters can be determined. Such animation parameters can describe the urgency of the execution of the control strategy (i.e. for example acceleration) and can influence the animation speed, which can clearly indicate the urgency to the driver of the vehicle 1.

[0049] In FIG. 3 a schematic diagram of the traffic situation display device of FIG. 2 is shown in enlarged form.

Claims

1. 1. A method for animating at least one object perception and driving intention of an assistance system for at least partially automated steering of a vehicle (1), comprising: - representing said at least one object perception by said assistance system if in said at least one object perception detected objects in the surroundings (4) of said vehicle (1) influence the control strategy of said at least partially automated steering of said assistance system; - animating said at least one object recognition; wherein the vehicle (1) is represented by an avatar (7) and an interaction between detected objects and the avatar (7) is performed in the animation, - the avatar (7) imitating at least one additional action (9) in animating the at least one object perception when the detected object in the surroundings (4) of the vehicle (1) influences a control strategy; - the driving intention of the assistance system is expressed, which indicates the presence of the control strategy influenced by the detected object; and the expression of the driving intention of the assistance system is made in an animated form, and the driving intention is imitated by the avatar (7) in the animation in order to improve the system understanding of the driver of the vehicle (1); A method characterized by:

2. 2. The method according to claim 1, wherein at least one animation parameter is determined depending on the at least one object recognition and / or depending on a driving intention of the assistance system, and the at least one animation parameter influences the animation.

3. 3. The method of claim 2, wherein said at least one animation parameter represents at least one urgency of execution of said control strategy, and said animation is performed depending on said urgency.

4. 4. The method according to claim 1, wherein the animated representation is provided so that the at least one object recognition or driving intention of the assistance system is perceived by the driver essentially by sight in the driver's peripheral vision.

5. Assistance system for a vehicle (1), characterized in that it comprises a display device (3) configured to represent by animation at least one object recognition and driving intention of the assistance system according to the method of any one of claims 1 to 3.

6. 1. A computer program comprising commands that, when executed by a computing device, cause the computing device to animate a driving intention of at least one object recognition and assistance system for at least partially automated steering of a vehicle (1) on a display device (3) according to the method of any one of claims 1 to 3 of the assistance system.

7. A computer-readable (storage) medium comprising a program, which, when executed by a computing device, causes the computing device to animate a driving intention of at least one object recognition and assistance system for at least partially automated steering of a vehicle (1) on a display device (3) according to the method of any one of claims 1 to 3 of the assistance system.

Images