Method for supporting own agent, program, storage medium, supporting system, and vehicle

Abstract

To provide a computer-implemented method for supporting the own agent.SOLUTION: A method includes: at least one of: estimating a visible area of another agent that exists in an environment of the own agent; calculating a visible state of another agent with respect to the own agent using the estimated visible area of another agent; estimating a collision risk between the own agent and another agent; planning a behavior of the own agent by minimizing a total cost of the behavior of the agent, the total cost including the estimated collision risk; notifying the own agent of the estimated collision risk and / or the planned behavior of the own agent; outputting a warning that another agent does not recognize the own agent; and using the estimated collision risk and / or the planned behavior of the own agent to control the own agent.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to the field of assisting an own agent. In particular, a method for assisting an own agent, a corresponding program with a program code, a corresponding non-transitory computer readable storage medium, an assistance system for assisting an own agent, and a vehicle equipped with such an assistance system are proposed. [Background technology]

[0002] Document U.S. Patent No. 9,463,797 (B2) discloses a method for risk-based traffic scene analysis and a vehicle with an advanced driver assistance system. Document U.S. Patent No. 10,627,812 (B2) discloses risk-based driver assistance for approaching intersections with limited visibility. U.S. Patent Application Publication No. 2020 / 0231149 (A1) discloses a method for assisting a driver, a driver assistance system, and a vehicle equipped with such a driver assistance system. Document U.S. Patent Application Publication No. 18 / 190,932 discloses a driver assistance system based on determining a perceived situation from a plurality of possible situations and a vehicle equipped with the driver assistance system. Document U.S. Patent Application Publication No. 17 / 709,420 discloses an advanced driver assistance system for assisting a driver of a vehicle.

[0003] In particular, the document US Patent Application Publication No. 17 / 709,420 discloses a system for visualizing an individual's risk space. The document US Patent Application Publication No. 17 / 709,420 discloses an advanced driver assistance system for assisting a driver of a vehicle. The system comprises a sensor unit, a processing unit, and a display unit. The sensor unit is configured to sense the environment of the vehicle and provide a sensing output to the processing unit. The processing unit is configured to determine at least one characteristic of the environment based on the sensing output. The processing unit is configured to determine a risk zone for the characteristic for a current time by estimating a respective risk for the characteristic at each virtual location of two or more virtual locations of the vehicle based on at least one parameter of the vehicle at a current time, estimating two or more risks for the two or more virtual locations, and forming a risk zone based on the two or more risks. The display unit is configured to display the environment of the vehicle together with the characteristic and the risk zone for the characteristic.

[0004] In other words, the document US Patent Application Publication No. 17 / 709,420 discloses a processing unit configured to estimate, for a current time, a theoretical risk related to a detected feature in the environment of the vehicle based on at least one parameter of the vehicle at the current time, assuming that at the current time the vehicle is not at the actual location of the vehicle but at a virtual location. The virtual location corresponds to an assumed location of the vehicle that is different from the actual location and is therefore a theoretical location. In particular, the two or more virtual locations consist of or correspond to locations that are different from each other and different from the actual location of the vehicle at the current time.

[0005] The risk zone may represent a dangerous area where the vehicle should not be at the current time. The risk zone of a feature may represent the personal (i.e. peripersonal) space of the feature. This allows the driver to intuitively recognize the risk situation of the vehicle at the current time from the display unit based on the displayed vehicle environment, the displayed features, and the risk zone of the displayed features, since everyone has their own personal space that they do not want to be violated. Thus, by showing the personal space of the feature to the driver in the form of a risk zone, the driver does not simply receive information about two or more estimated risks regarding the feature from the display of the risk zone of the feature. At the same time, the driver recognizes the personal space of the feature, and he / she will intuitively pay attention to this personal space from his / her psychology. That is, the risk zone of the feature informs the driver of an area where the vehicle should not enter.

[0006] The personal risk space visualization system of US Patent Application Publication No. 17 / 709,420 is one example of how risk can be visualized. The present disclosure, particularly regarding visibility sharing, may be incorporated into the overall framework of US Patent Application Publication No. 17 / 709,420. US Patent Application Publication No. 17 / 709,420 then allows visualization of risk zones around the vehicle as well as the visibility of other drivers, which is the subject of this disclosure.

[0007] In the context of mobility, there are various agents involved in the environment. For example, with respect to a road, there may be various land vehicles, such as one or more cars, one or more bicycles, and one or more people (e.g., pedestrians) moving within the area of ​​the road. The term "path" may be used as a synonym for "road". An assistance system for assisting any of such agents may help to warn each assisted agent from the risk of collision with other agents. In this regard, the assisted agent may be referred to as an "ego-agent". For example, a blind spot system of a car driven by a driver on a road may warn the driver of other agents on the road (e.g., pedestrians, bicycles, or another car) that are present in one of the blind spots of the car. A blind spot of a car is an area of ​​the car where the driver driving the car has zero visibility. That is, any object present in the blind spot of the car cannot be visually perceived by the driver (e.g., directly or via a side mirror or rearview mirror). A blind spot of a car may represent a blind spot area of ​​the car.

[0008] However, such blind spot systems only inform the vehicle driver of the presence of agents in the vehicle's environment that are not visually perceptible by the driver, i.e., one or more agents present in the blind spot may not know that they are present in the vehicle's blind spot and therefore may not be aware that they are not visible to the vehicle driver.

[0009] Many accidents may occur due to human drivers overlooking critical traffic participants and due to a lack of communication or miscommunication between human drivers. Summary of the Invention [Problem to be solved by the invention]

[0010] It is therefore an object of the present disclosure to provide an improved computer-implemented method for supporting an own agent, and in particular, it may be an object to provide an improved computer-implemented method for supporting an own agent, which is capable of improving interactions between the own agent and other agents in an environment. [Means for solving the problem]

[0011] In a first aspect, a computer implemented method according to independent claim 1 solves the above mentioned problem. A program according to a second aspect, a non-transitory computer readable storage medium according to a third aspect, an assistance system according to a fourth aspect and a vehicle according to a fifth aspect provide further advantageous solutions to this problem.

[0012] In a first aspect, a computer-implemented method for assisting an own agent includes estimating a visibility area of ​​another agent present in an environment of the own agent. The method further includes calculating a visibility state of the other agent with respect to the own agent using the estimated visibility area of ​​the other agent. Furthermore, the method includes estimating a collision risk between the own agent and the other agent using the calculated visibility state of the other agent. The method further includes planning a behavior by minimizing a total cost of the own agent's behavior, the cost including the estimated collision risk. Furthermore, the method includes at least one of notifying the own agent of the estimated collision risk and / or the planned behavior of the own agent, outputting a warning that the other agent does not recognize the own agent in response to the estimated collision risk, and controlling the own agent using the estimated collision risk and / or the planned behavior of the own agent.

[0013] The method according to the first aspect provides an improved method for supporting an own agent, since it takes into account the ability of other agents present in the own agent's environment to perceive the own agent in order to estimate conflicts between the own agent and other agents, thus making it possible to improve the interaction between the own agent and other agents in the environment with regard to avoiding conflicts between the own agent and other agents.

[0014] The method according to the first aspect allows the self-agent to recognize that the other agent may have overlooked the self-agent. The method may provide improved assistance to the self-agent in both cases where the assistance system of the other agent is not functioning, the other agent does not have an assistance system, and the other agent does not react to a warning of the self-agent's assistance system regarding a collision between the other agent and the self-agent. In each of the above cases, the method of the first aspect allows the self-agent to know the other agent's limited visibility state with respect to the self-agent when the visibility state of the other agent is limited, so that a collision between the self-agent and the other agent can still be avoided by the self-driver.

[0015] The dependent claims define further advantageous embodiments of the present disclosure.

[0016] A program according to a second aspect comprises program code means for executing a method for assisting an own agent when the program is executed on a computer or digital signal processor. The method includes estimating a visibility area of ​​another agent present in an environment of the own agent. The method further includes calculating a visibility state of the other agent with respect to the own agent using the estimated visibility area of ​​the other agent. Furthermore, the method includes estimating a collision risk between the own agent and the other agent using the calculated visibility state of the other agent. The method further includes planning a behavior by minimizing a total cost of the own agent's behavior, the cost including the estimated collision risk. Furthermore, the method includes at least one of notifying the own agent of the estimated collision risk and / or the planned behavior of the own agent, outputting a warning that the other agent does not recognize the own agent in response to the estimated collision risk, and controlling the own agent using the estimated collision risk and / or the planned behavior of the own agent.

[0017] In other words, the program according to the second aspect comprises program code means for performing the method according to the first aspect, such as operations according to one of the embodiments of the method according to the first aspect, when the program is run on a computer or digital signal processor.

[0018] A non-transitory computer-readable storage medium according to a third aspect embodies a program of machine-readable instructions executable by a digital processing device, the instructions causing the digital processing device to do at least one of: estimate a visibility area of ​​another agent present in an environment of the own agent; calculate a visibility state of the other agent with respect to the own agent using the estimated visibility area of ​​the other agent; estimate a collision risk between the own agent and the other agent using the calculated visibility state of the other agent; plan a behavior by minimizing a total cost of the own agent's behavior, the total cost including the estimated collision risk; plan the behavior by informing the own agent of the estimated collision risk and / or the planned behavior of the own agent; output a warning that the other agent does not recognize the own agent in response to the estimated collision risk; and control the own agent using the estimated collision risk and / or the planned behavior of the own agent.

[0019] In other words, a non-transitory computer readable storage medium according to the third aspect embodies a program of machine readable instructions executable by a digital processing device, the instructions causing the digital processing device to perform a method according to the first aspect, such as an operation according to one of the embodiments of the method according to the first aspect.

[0020] A program according to the third aspect comprises program code means for performing the steps according to one of the embodiments of the method according to the first aspect when the program is run on a computer or digital signal processor.

[0021] According to a third aspect, a non-transitory computer-readable storage medium embodies a program of machine-readable instructions executable by a digital processing device, the instructions causing the digital processing device to perform operations according to one of the embodiments of the method according to the first aspect.

[0022] A support system according to a fourth aspect for supporting an own agent comprises a processor configured to estimate a visibility area of ​​another agent present in an environment of the own agent. The processor is configured to calculate a visibility state of the other agent with respect to the own agent using the estimated visibility area of ​​the other agent. The processor is configured to estimate a collision risk between the own agent and the other agent using the calculated visibility state of the other agent. The processor is configured to plan a behavior by minimizing a total cost of the own agent's behavior, the total cost including the estimated collision risk. The processor is configured to at least one of notify the own agent of the estimated collision risk and / or the planned behavior of the own agent, output a warning that the other agent does not recognize the own agent in response to the estimated collision risk, and control the own agent using the estimated collision risk and / or the planned behavior of the own agent.

[0023] In other words, the assistance system according to the fourth aspect comprises a processor configured to perform the method according to the first aspect, such as operations according to one of the embodiments of the method according to the first aspect.

[0024] A vehicle according to a fifth aspect includes an assistance system according to the fourth aspect.

[0025] The vehicle may be any vehicle known in the art that may travel on the surface, near the surface, in the air, underwater, and deep underwater. The vehicle may be, for example, a land vehicle (car, truck, bus, bicycle, motorcycle, fork truck, etc.), an air vehicle (plane, helicopter, drone, space vehicle, etc.), a surface / underwater vehicle (boat, submarine, etc.), etc. The vehicle may be a vehicle configured to be operated by an operator present in the vehicle. Optionally, the vehicle may be a vehicle remotely operated by an operator not present in the vehicle. The vehicle may be a vehicle configured to travel autonomously, such as an autonomous driving (AD) vehicle.

[0026] The description of the embodiments will refer to the accompanying drawings, in which: [Brief description of the drawings]

[0027] [Figure 1] 1 is a simplified flow chart of a method according to one embodiment. [Diagram 2] 4 is an example of an implementation of a method step according to one embodiment. [Diagram 3] 4 is an example of data that can be used in the implementation of a method step according to one embodiment. [Figure 4] 1A and 1B are each an example of an implementation of a step of a method according to one embodiment. [Diagram 5] FIG. 2 illustrates an optional step of an example implementation of a method according to one embodiment. [Figure 6] 4 is an example of visibility states of other agents that may be calculated by an implementation of method steps according to one embodiment. [Figure 7] 1A and 1B are each an example of an implementation of a step of a method according to one embodiment. [Figure 8] FIG. 2 illustrates an optional step of an example implementation of a method according to one embodiment. [Figure 9] FIG. 2 illustrates an optional step of an example implementation of a method according to one embodiment. [Figure 10] 1 is an example of an implementation of two steps of a method according to one embodiment. [Figure 11] FIG. 1 is a diagram of an assistance system according to one embodiment. [Figure 12] 1A and 1B each show a use case using an example method according to one embodiment. [Figure 13] 1A and 1B each show a use case using an example method according to one embodiment. [Figure 14] 1A, 1B, and 1C each show a use case using an example method according to one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0028] In the figures, corresponding elements have the same reference numbers. Descriptions of the same reference numbers in different figures are omitted where possible without negatively affecting comprehensibility.

[0029] The method according to the first aspect provides an advantageous solution to the above-mentioned problem. The program according to the second aspect, the non-transitory computer-readable storage medium according to the third aspect, the assistance system according to the fourth aspect and the vehicle according to the fifth aspect provide further advantageous solutions to this problem. The dependent claims define further advantageous embodiments of the present disclosure.

[0030] The method according to the first aspect may be performed by a processing unit such as a computer. The self-agent may also be referred to by the term "agent".

[0031] The term "field of view" may be used as a synonym for the term "visibility area". The visibility area of ​​an agent, such as other agents or the own agent, may be understood as the area that the agent is aware of and thus can perceive, e.g. visually perceive, other entities, such as further agents, obstacles, etc. In other words, the visibility area of ​​an agent is the area in which entities are visible to the agent. That is, entities located outside the visibility area of ​​an agent, e.g. other agents or the own agent, cannot be perceived, e.g. visually perceived, by the agent.

[0032] An agent's visibility state with respect to a second agent, such as the visibility state of another agent with respect to the own agent, may be understood as a state indicating whether the second agent is perceived, e.g. visually perceived, by the agent.

[0033] The other agent may be a vehicle. The vehicle may be any vehicle known in the art that may travel on the surface, near the surface, in the air, underwater, and deep underwater. The vehicle may be, for example, a land vehicle (car, truck, bus, bicycle, motorcycle, fork truck, etc.), an air vehicle (plane, helicopter, drone, space vehicle, etc.), a surface / underwater vehicle (boat, submarine, etc.), etc. The terms "land vehicle", "air vehicle", and "underwater vehicle" may be used synonymously with the terms "land vehicle", "air vehicle", and "surface / underwater vehicle", respectively.

[0034] Optionally, the other agent may be a vehicle operated by an operator present in the vehicle, i.e. a vehicle operated by a driver present in or on the vehicle. In this case, the viewing area of ​​the other agent is the viewing area of ​​the operator, i.e. the viewing area of ​​the driver of the vehicle. Optionally, the other agent may be a vehicle remotely operated by an operator not present in the vehicle. In this case, the viewing area of ​​the other agent is a viewing area provided by one or more sensors of the vehicle for providing the operator with a visual perception of the vehicle's environment. Optionally, the other agent may be a vehicle configured for autonomous movement, such as an autonomous driving (AD) vehicle. In this case, the viewing area of ​​the other agent is a viewing area provided by one or more sensors of the vehicle for providing a visual perception of the vehicle's environment to a control entity of the vehicle. The control entity may comprise one or more control devices on board the vehicle and / or one or more external control devices, e.g. an autonomous system. In this specification, the control device may include at least one of a processor, a microprocessor, a controller, a microcontroller, an application specific integrated circuit (ASIC), and a field programmable gate array (FPGA). Optionally, the other agent may be a person (e.g., a pedestrian). In such a case, the viewing area of ​​the other agent is the viewing area of ​​the person's eyes.

[0035] The vehicle may include a motor, such as a combustion motor, an electric motor, a hybrid motor, etc. If the vehicle is an airborne vehicle, such as an airplane, the road description is correspondingly valid when the airborne vehicle moves on or near the surface of the earth (e.g., a helicopter flying close to the surface of the earth). If the vehicle is an airborne vehicle, such as an airplane, the road description is correspondingly valid for an aerial corridor when the vehicle moves through the air. If the vehicle is a water / underwater vehicle, such as a boat, the road description is correspondingly valid for an underwater route, such as a waterway, channel, etc.

[0036] The above explanations regarding other agents are correspondingly valid for the own agent.

[0037] Estimating the collision risk between the own agent and the other agent using the calculated visibility state of the other agent may include using one or more risk models, such as one or more probabilistic risk models. The collision risk between the own agent and the other agent may be estimated using the current trajectories of the own agent and the other agent, the visibility state of the other agent, and optionally the visibility state of the own agent.

[0038] For example, estimating the collision risk between the self agent and another agent involves estimating the collision risk R collision and the collision risk R due to the visibility state of other agents. visibility The total collision risk R is equal to the sum of total (R total =R collision +R visibility ). For example, the above criteria may include the speed of the own agent, the speed of the other agent, the moving direction of the own agent, the moving direction of the other agent, environmental conditions (e.g., weather conditions, width and path of the road on which the own agent and the other agent may move), the planned behavior of the own agent, the planned behavior of the other agent, etc. For example, the collision risk R due to the visibility state of the other agent visibilitymay be equal to the parameter k divided by the sum of 1 and the distance of the agent's position to the other agent's visible area (R visibility =k / (1+dist(pos ego ,visibility area other )). The parameter k is a scaling parameter that specifies the significance of the risk arising from visibility. It can be seen as a weight on how much the risk arising from visibility is taken into account in the total collision risk cost. The risk coefficient can be calculated using other means, for example by varying the prediction of the other agent due to its own visibility and calculating the collision risk for the different predictions. Alternatively, the collision risk due to visibility of the other agent, R visibility Alternatively, a collision risk that depends on the visibility state of the other agent and the visibility state of the own agent may be used.

[0039] For example, the total collision risk R total is greater than the threshold (R total >R threshold,1 )) and the collision risk R due to the visibility state of other agents. visibility is greater than the threshold (R visibility >R threshold,2 )), a warning may be output to the agent that the other agent does not recognize the agent. The total collision risk R total is sometimes called the estimated collision risk, and is the collision risk R visibility is sometimes referred to as the risk factor calculated using the calculated visibility states of other agents.

[0040] Planning a behavior by minimizing a total cost of the self-agent's behavior, where the total cost includes the estimated collision risk, may include using a behavior planning algorithm. The higher the estimated collision risk, the higher the total cost may be, and vice versa. That is, an optimal behavior of the self-agent may be planned by minimizing a total cost of the behavior, where the total cost includes the estimated collision risk. Planning a behavior by minimizing a total cost of the self-agent's behavior may include planning a safe behavior using a planning model. For example, if the self-agent and the other agent are traveling on a road, a safe behavior may be braking to avoid a collision, following a lane, etc.

[0041] The steps of estimating the collision risk between the own agent and the other agent and planning the behavior of the own agent may be performed by one or more controllers of the own agent and / or one or more external controllers, e.g., autonomous systems. When one or more external controllers, e.g., autonomous systems, perform the above-mentioned steps, at least one of the estimated collision risk, the cost, and the planned behavior (e.g., optimal behavior) of the own agent may be communicated (e.g., wirelessly) to the own agent. Here, if there is communication between the two entities, it may be wireless and / or wired. Such communication may be performed using respective communication devices in each of the two entities. Such communication may be performed according to any known communication standard or protocol. For example, such communication may include vehicle-to-vehicle communication (V2V communication) technology and / or vehicle-to-infrastructure communication (V2I communication) technology.

[0042] Optionally, the step of estimating the visibility area of ​​the other agent and / or the step of calculating the visibility state of the other agent may be performed by one or more control devices of the own agent and / or one or more external control devices, e.g., an autonomous system.

[0043] Optionally, the step of estimating the visibility area of ​​the other agent and / or the step of calculating the visibility state of the other agent may be performed in the other agent. In this case, a communication device of the other agent may transmit the estimated visibility area and / or the calculated visibility state to a communication device of the own agent. The communication devices of the other agent and the own agent may be configured for wireless and / or wired communication. The communication may follow any known communication standard or protocol. For example, the communication may include vehicle-to-vehicle communication (V2V communication) and / or vehicle-to-infrastructure communication (V2I communication) technologies. Additionally or alternatively, the communication device of the other agent may transmit the estimated visibility area and / or the calculated visibility state to the autonomous system. The autonomous system may be configured to control the own agent, e.g. the movement of the own agent.

[0044] A warning that the other agent does not recognize the own agent may be output to the own agent. Information that may be output to the own agent may include information regarding safe behavior of the own agent to reduce collision risk, such as maintaining a distance from the other agent.

[0045] According to one embodiment of the method, estimating the visibility area of ​​the other agent includes estimating a direction of visual perception of the other agent, tracking a history of the direction of visual perception of the other agent, and generating the visibility area using the tracked history of the direction of visual perception of the other agent.

[0046] For example, estimating the direction of the other agent's visual perception may include applying computer vision, where a history of the direction of the other agent's visual perception may be tracked using a filter.

[0047] When the other agent is a vehicle operated by an operator present in the vehicle, the direction of visual perception of the other agent is the direction of visual perception of the operator (e.g., the eyes of the person operating the vehicle). For example, the direction of visual perception of the operator may be the direction of the operator's gaze. When the other agent is a vehicle remotely operated by an operator not present in the vehicle, the direction of visual perception of the other agent is the direction of visual perception provided by one or more sensors of the vehicle for providing the visual perception of the vehicle's environment to the operator. When the other agent is a vehicle configured to move autonomously, the direction of visual perception of the other agent is the direction of visual perception provided by one or more sensors of the vehicle for providing the visual perception of the vehicle's environment to a control entity of the vehicle. When the other agent is a person (e.g., a pedestrian), the direction of visual perception of the other agent is the direction of visual perception of the person's eyes. For example, the direction of visual perception of the person may be the direction of the person's gaze.

[0048] According to one embodiment of the method, estimating the visibility area of ​​the other agent includes using data from at least one of: one or more cameras of the other agent facing the face of the other agent's operator when the other agent is operated by an operator present at the other agent; one or more sensors of the other agent that sense whether the operator of the other agent is aware of or alerted to the own agent when the other agent is operated by an operator present at the other agent; one or more sensors of the other agent that sense the other agent's environment; one or more cameras located in the other agent's environment; and one or more cameras of the own agent facing the face of the other agent's operator when the other agent is operated by an operator present at the other agent.

[0049] Here, the operator may be a person or a humanoid robot. For example, if the other agent is a car driven by an operator (e.g., a person), the one or more cameras may be installed in the car such that the one or more cameras face the operator. For example, if the other agent is a motorcycle driven by a person (who is the operator), the one or more cameras may be installed in (e.g., inside) a helmet worn by the person such that the one or more cameras face the operator.

[0050] Here, the one or more sensors for sensing the environment or providing a visual perception of the environment may include at least one of one or more cameras, one or more radar sensors, one or more lidar sensors, one or more ultrasonic sensors, one or more infrared sensors, one or more occupancy sensors and / or motion sensors, and any other one or more sensors known in the art for sensing or visually perceiving the environment. The one or more sensors for sensing the environment or providing a visual perception of the environment may include sensors used for adaptive cruise control.

[0051] For example, if the self agent and the other agent are traveling on a road, one or more cameras may be installed on the road infrastructure, such as traffic lights, traffic signs, etc. That is, the environment of the other agent may be the road.

[0052] Optionally, estimating the viewing area of ​​the other agent includes using data of one or more cameras of the own agent pointed in the direction of the other agent. For example, if the own agent is a car and the other agent is another car driven by a driver, one or more cameras may be installed on the rear side of the car to perceive the driver of the other car, such as the driver's posture, face, and / or direction of visual perception (e.g., direction of gaze) through the front window of the other car. In this case, since the one or more cameras are located at the own agent, the data and the viewing area do not need to be transmitted to the own agent.

[0053] According to one embodiment of the method, estimating the visibility area of ​​the other agent includes applying ray projections from the position of the other agent and reducing the visibility area of ​​the other agent to correspond to an area obstructed by objects in the other agent's environment, which may include, for example, walls, other vehicles, trees, buildings, etc.

[0054] According to one embodiment of the method, estimating the visibility area of ​​the other agent includes setting the visibility area to zero square meters if the other agent is operated by or is a human and the human is looking at the mobile device or talking to at least one other person.

[0055] According to one embodiment of the method, the method includes estimating a visibility area of ​​a self-agent, calculating a visibility state of the self-agent with respect to the other agent using the estimated visibility area of ​​the self-agent, and estimating a collision risk between the self-agent and the other agent using the calculated visibility state of the other agent and the calculated visibility state of the self-agent.

[0056] In other words, the method may take into account the visibility state of the own agent in addition to the visibility state of the other agent to estimate the collision risk between the own agent and the other agent. That is, the collision risk may depend on a combination of the visibility state of the own agent and the visibility state of the other agent. This may be advantageous because even if the other agent is aware of the own agent, the collision risk may still be high if the own agent is not aware of the other agent itself. For example, if the own agent and the other agent are each vehicles driven by a driver on a road, the collision risk may depend on a combination of the visibility of the driver of the own vehicle and the visibility of the driver of the other vehicle. The collision risk may be greatest, for example, when both drivers are not aware of each other.

[0057] According to one embodiment of the method, the visibility state of the other agent is determined by a Boolean variable (e.g., pos ego ∈Visibility area other ), a distance variable indicating the distance of the agent's position to the other agent's viewing area (e.g., dist(pos ego ,visibility area other )), and a random variable that indicates whether the other agent is aware of the agent itself, which depends on the position of the agent itself and the visibility area of ​​the other agent (e.g., (pos ego ,visibility area other )) may include at least one of the following.

[0058] According to one embodiment of the method, calculating the visibility state of the other agent relative to the own agent using the estimated visibility area of ​​the other agent includes at least one of combining the calculated visibility state of the other agent with an estimated confidence value, and using at least one of a moving average, hysteresis, and outlier correction.

[0059] For example, the estimated confidence value may correspond to changes in viewing conditions over time. At least one of a moving average, hysteresis, and outlier correction may be used to make the calculated viewing conditions more robust.

[0060] According to one embodiment of the method, estimating a collision risk between the own agent and the other agent using the calculated visibility state of the other agent includes at least one of predicting a behavior of the other agent using the calculated visibility state of the other agent, and modifying the collision risk by a risk factor calculated using the calculated visibility state of the other agent.

[0061] For example, if the self-agent and the other agent are each vehicles driven by a driver on a road, the driver of the other agent is likely to change lanes when the driver of the other agent is not aware of the self-vehicle. Such a lane change may result in a collision between the other vehicle and the self-vehicle if the self-vehicle is traveling in the lane into which the other vehicle is changing. Thus, the behavior of the other agent has an effect on the collision risk between the self-agent and the other agent. This may be taken into account by modifying the collision risk by a risk coefficient calculated using the calculated visibility state of the other agent. The collision risk between the self-agent and the other agent may be estimated by considering the predicted behavior of the other agent. For example, the risk coefficient may be calculated using the predicted behavior of the other agent.

[0062] According to one embodiment of the method, the method includes calculating a risk factor using the calculated visibility state of the other agent, and if the estimated collision risk is higher than a collision risk threshold and the calculated risk factor is higher than a risk factor threshold, outputting a warning that the other agent does not recognize the own agent.

[0063] According to one embodiment of the method, the method includes performing at least one of: determining whether the other agent recognizes the own agent based on the other agent's estimated viewing area and the own agent's position; informing the own agent to use a visual notification to notify the other agent if the other agent recognizes the own agent, and to use an acoustic notification to notify the other agent otherwise; and controlling the own agent to use a visual notification to notify the other agent if the other agent recognizes the own agent, and to use an acoustic notification to notify the other agent otherwise.

[0064] According to one embodiment of the method, estimating the visibility area of ​​the other agent includes determining a blind spot area of ​​the other agent using a shape of the other agent, and planning the behavior of the own agent by minimizing a total cost of the behavior includes using a cost function that penalizes a position of the own agent within the blind spot area.

[0065] Determining the blind spot area of ​​the other agent using the shape of the other agent may include estimating the blind spot area from the geometric shape of the other agent. The blind spot area of ​​the other agent includes the blind spot of the other agent.

[0066] According to one embodiment of the method, notifying the own agent and / or outputting the alert is performed using a human machine interface (HMI).

[0067] According to an embodiment of the method, informing and / or outputting a warning to the own agent may be performed in at least one of a visual, an acoustic and a haptic manner. For example, an acoustic warning may include outputting a warning sound. For example, informing and / or outputting a warning to the own agent in a haptic manner may include vibrating a steering wheel if the own agent is a vehicle with a steering wheel, such as a car, an airplane, etc. For example, outputting a warning in a visual manner may include one or more warning icons. For example, informing the own agent in a visual manner may include highlighting the other agent if the other agent does not recognize the own agent. That is, entities present in the own agent's environment, such as the other agent or one or more further agents not recognizing the own agent, may be highlighted. For example, a visibility area of ​​the other agent or other entities present in the own agent's environment may be visualized on the display, for example by a bird's-eye view.

[0068] Outputting notifications and / or warnings to the self-agent may be performed by a human machine interface (HMI). The HMI may include a display for notifying and / or warning in a visual manner. The HMI may include one or more loudspeakers for notifying and / or warning in an acoustic manner. The HMI may include one or more actuators, such as one or more vibrators (e.g., located in a steering wheel for driving the self-agent), for notifying and / or warning in a haptic manner.

[0069] According to one embodiment of the method, the warning may be provided such that the modality of the warning and / or the intensity of the warning announcement depends on the estimated collision risk.

[0070] For example, the higher the estimated collision risk, the higher the intensity of the warning announcement, and vice versa.

[0071] Optionally, the other agent may be informed of the estimated collision risk and / or the planned behavior of the own agent (e.g., simultaneously with the own agent). Additionally or alternatively, a warning that the other agent does not recognize the own agent may be output to the other agent in response to the estimated collision risk (e.g., simultaneously with the own agent). Additionally or alternatively, the other agent may be controlled using the estimated collision risk and / or the planned behavior of the own agent (e.g., simultaneously with the own agent). The descriptions regarding informing and / or outputting a warning to the own agent are correspondingly valid with respect to the other agent.

[0072] Explanations about the agent itself may be valid for other agents, and explanations about other agents may be valid for the agent itself.

[0073] To realise the method according to the first aspect, some or all of the embodiment and optional features of the first aspect described above may be combined with each other.

[0074] According to one embodiment of the assistance system, the assistance system is configured to assist an operator of an own agent when the own agent is a vehicle operated by the operator.

[0075] Optionally, the assistance system may be included in a mobile device that includes a human machine interface (HMI) and is carried by the user. The assistance system may be configured to assist the user via the HMI.

[0076] The assistance system may be a Driver Assistance System (DAS). Optionally, the assistance system may be an Advanced Driver Assistance System (ADAS).

[0077] The processor of the assistance system may comprise one or more control devices in the own agent and / or one or more external control devices, for example autonomous systems. The one or more control devices in the own agent and the one or more external control devices may be configured to communicate with each other. The description of communication disclosed in this specification is correspondingly valid for this.

[0078] If the processor uses data acquired by one or more sensors, e.g., one or more cameras, of the other agent to estimate the visibility area of ​​the other agent, the assistance system may comprise a communication device for receiving the data. Optionally, the assistance system (e.g., the communication device) may transmit to the other agent or any other external entity (such as one or more further agents present in the environment of the own agent) at least one of the following: the estimated visibility area of ​​the other agent, the calculated visibility state of the other agent, the estimated visibility area of ​​the own agent, the calculated visibility state of the own agent, the estimated collision risk between the own agent and the other agent, the planned behavior of the own agent, and a warning that the other agent does not recognize the own agent.

[0079] The program according to the second aspect, the non-transitory computer-readable storage medium according to the third aspect, the assistance system according to the fourth aspect, and the vehicle according to the fifth aspect provide corresponding advantageous solutions as described with respect to the embodiments of the method according to the first aspect. The description of the method of the first aspect is correspondingly valid for the program according to the second aspect, the non-transitory computer-readable storage medium according to the third aspect, the assistance system according to the fourth aspect, and the vehicle according to the fifth aspect.

[0080] 1 shows a simplified flowchart of a method according to an embodiment. The method of FIG. 1 is an example of a computer-implemented method of the first aspect. The description of the method of the first aspect is correspondingly valid for the method of FIG. 1.

[0081] The method of FIG. 1 is a computer-implemented method for assisting an own agent. As shown in FIG. 1, the method includes, in step 100, estimating a visibility area of ​​another agent present in an environment of the own agent. Following step 100, in step 200, the method includes calculating a visibility state of the other agent with respect to the own agent using the estimated visibility area of ​​the other agent. Following step 200, the method includes, in step 300, estimating a collision risk between the own agent and the other agent using the calculated visibility state of the other agent. Following step 300, in step 400, the method includes planning a behavior by minimizing a total cost of the own agent's behavior, the total cost including the estimated collision risk. Following step 400, in step 500, the method includes at least one of notifying the own agent of the estimated collision risk and / or the planned behavior of the own agent, outputting a warning that the other agent does not recognize the own agent in response to the estimated collision risk, and controlling the own agent using the estimated collision risk and / or the planned behavior of the own agent.

[0082] The steps of the method may be performed in repeated time steps, i.e. the method may be performed iteratively. The repeated time steps may be cyclically repeated time steps.

[0083] Figure 2 shows an example of an implementation of a step of a method according to an embodiment. Figure 2 thus shows an example of an implementation of the method of Figure 1 and thus of the method of the first aspect. In particular, Figure 2 shows an example of an implementation of step 100 of the method of Figure 1.

[0084] As shown in FIG. 2, step 100 of estimating the viewing area of ​​another agent includes, in step 101, estimating the direction of visual perception of the other agent, in step 102 following step 101, tracking a history of the direction of visual perception of the other agent, and, in step 103 following step 102, generating a viewing area using the tracked history of the direction of visual perception of the other agent.

[0085] Figure 3 is an example of data that can be used in the implementation of one step of a method according to an embodiment. Figure 3 thus shows an example of data that can be used in the implementation of the method of Figure 1, and thus in the implementation of the method of the first aspect. In particular, Figure 3 shows an example of data that can be used in the implementation of step 100 of the method of Figure 1.

[0086] As shown in Fig. 3, the step 100 of estimating the visibility area of ​​the other agent may include using data 30. The data 30 may be data of at least one of one or more cameras 31 of the other agent facing the face of the operator of the other agent when the other agent is operated by the operator present at the other agent, one or more sensors 32 of the other agent sensing whether the operator of the other agent is aware of or alerted to the own agent when the other agent is operated by the operator present at the other agent, one or more sensors 33 of the other agent sensing the environment of the other agent, one or more cameras 34 located in the environment of the other agent, and one or more cameras 35 of the own agent facing the face of the operator of the other agent when the other agent is operated by the operator present at the other agent.

[0087] Figures 4A and 4B each show an example of an implementation of a step of a method according to an embodiment. Thus, Figures 4A and 4B each show an example of an implementation of the method of Figure 1, and thus of the method of the first aspect. In particular, Figures 4A and 4B each show an example of an implementation of step 100 of the method of Figure 1.

[0088] As shown in FIG. 4A, step 100 of estimating the visibility area of ​​the other agent may include, in step 104, applying ray projection from the position of the other agent, and, in step 105 following step 104, reducing the visibility area of ​​the other agent to correspond to the area obstructed by objects in the other agent's environment.

[0089] As shown in FIG. 4B, step 100 of estimating the visibility area of ​​the other agent may include step 106 of setting the visibility area to zero square meters if the other agent is operated by or is a human and the human is looking at the mobile device or talking to at least one other person.

[0090] Figure 5 shows an optional step of an example of a method implementation according to one embodiment. Figure 5 thus shows an example of an optional step in the implementation of the method of Figure 1, and therefore of the first aspect.

[0091] As shown in FIG. 5, the method of FIG. 1 may optionally include, in step 100a, estimating a visibility area of ​​the own agent. The method may optionally include, in step 200a following step 100a, calculating a visibility state of the own agent with respect to the other agent using the estimated visibility area of ​​the own agent. The method may optionally include, in step 300 following step 200a, estimating a collision risk between the own agent and the other agent using the calculated visibility state of the other agent and the calculated visibility state of the own agent. Step 100a may be performed before one of steps 100 and 200 of the method of FIG. 1, or simultaneously with at least one of steps 100 and 200 of the method of FIG. 1, or after step 200 of the method of FIG. 1. Step 200a may be performed before one of steps 100 and 200 of the method of FIG. 1, or simultaneously with at least one of steps 100 and 200 of the method of FIG. 1, or after step 200 of the method of FIG. 1. Step 300 of FIG. 5 may replace step 300 of the method of FIG. 1 if optional steps 100a and 200a are performed in addition to steps 100 and 200 of the method of FIG.

[0092] Fig. 6 shows an example of a visibility state of an other agent that may be calculated by an implementation of one step of the method according to an embodiment. Thus, Fig. 6 shows an example of a visibility state of an other agent that may be calculated by an implementation of the method of Fig. 1 and thus an example of a visibility state of an other agent that may be calculated by an implementation of the method of the first aspect. In particular, Fig. 6 shows an example of a visibility state of an other agent that may be calculated in an implementation of step 200 of the method of Fig. 1.

[0093] As shown in FIG. 6, the visibility state 60 of the other agent that may be calculated in step 200 of FIG. 1 may include at least one of a Boolean variable 61 indicating whether the position of the own agent is within the estimated visibility area of ​​the other agent, a distance variable 62 indicating the distance of the own agent's position to the visibility area of ​​the other agent, and a random variable 63 that the other agent is aware of the own agent, the random variable 63 depending on the position of the own agent and the visibility area of ​​the other agent.

[0094] Figures 7A and 7B each show an example of an implementation of a step of a method according to an embodiment. Thus, Figures 7A and 7B each show an example of an implementation of the method of Figure 1 and thus of the method of the first aspect. In particular, Figure 7A shows an example of an implementation of step 200 of the method of Figure 1 and Figure 7B shows an example of an implementation of step 300 of the method of Figure 1.

[0095] As shown in FIG. 7A, step 200 of calculating the visibility state of other agents relative to the own agent using the estimated visibility areas of the other agents may include at least one of step 201 of combining the calculated visibility state of the other agents with an estimated confidence value, and step 202 of using at least one of a moving average, hysteresis, and outlier correction.

[0096] As shown in FIG. 7B, step 300 of estimating a collision risk between a self agent and another agent using the calculated visibility state of the other agent includes at least one of step 301 of predicting the behavior of the other agent using the calculated visibility state of the other agent, and step 302 of modifying the collision risk by a risk coefficient calculated using the calculated visibility state of the other agent.

[0097] Figures 8 and 9 each show an optional step in an example implementation of a method according to one embodiment. Thus, Figures 8 and 9 each show an example of an optional step in the implementation of the method of Figure 1, and therefore of the first aspect.

[0098] As shown in Fig. 8, the method of Fig. 1 may optionally include, in step 600, calculating a risk factor using the calculated visibility state of the other agent. The method may optionally include, in step 700 following step 600, outputting a warning that the other agent does not see the own agent if the estimated collision risk is higher than a collision risk threshold and the calculated risk factor is higher than a risk factor threshold. Optional step 600 may be performed after step 200 of the method of Fig. 1. It may be performed simultaneously with at least one of steps 300-500. Optional step 700 may be performed simultaneously with at least one of steps 300-500.

[0099] As shown in FIG. 9, the method of FIG. 1 may optionally include, in step 800, determining whether the other agent recognizes the own agent based on the estimated viewing area of ​​the other agent and the position of the own agent. The method may optionally include, in step 900 following step 800, at least one of: informing the own agent to use a visual notification for notifying the other agent if the other agent recognizes the own agent, and to use an acoustic notification for notifying the other agent otherwise, and controlling the own agent to use a visual notification for notifying the other agent if the other agent recognizes the own agent, and to use an acoustic notification for notifying the other agent otherwise. The optional step 800 may be performed after step 100 of the method of FIG. 1. It may be performed simultaneously with at least one of steps 200-500. The optional step 900 may be performed simultaneously with at least one of steps 200-500.

[0100] Fig. 10 shows an example of implementation of two steps of a method according to an embodiment. Thus, Fig. 10 shows an example of implementation of the method of Fig. 1 and therefore of the first aspect. In particular, Fig. 10 shows an example of implementation of steps 100 and 400 of the method of Fig. 1.

[0101] 10, estimating 100 the visibility area of ​​the other agent may include determining 107 the blind spot area of ​​the other agent using the shape of the other agent. Planning 400 the behavior of the own agent by minimizing the total cost of the behavior of the own agent may include 401 using a cost function that penalizes the position of the own agent in the blind spot area.

[0102] The optional features shown in FIGS. 2-10 may be combined in any manner with each other and with the method steps of FIG.

[0103] 11 shows a support system according to an embodiment. The support system is an example of the support system according to the fourth aspect. The description of the support system according to the fourth aspect is valid for the support system of FIG.

[0104] As shown in FIG. 11, the assistance system 3 is an assistance system for assisting the own agent 1. The system 3 includes a processor 4. The processor 4 is configured to estimate a visual area of ​​another agent present in the environment of the own agent. The processor 4 is configured to calculate a visual state of the other agent with respect to the own agent using the estimated visual area of ​​the other agent. The processor 4 is configured to estimate a collision risk between the own agent and the other agent using the calculated visual state of the other agent. The processor 4 is configured to plan a behavior by minimizing a total cost of the own agent's behavior, the total cost including the estimated collision risk. The processor 4 is configured to at least one of notify the own agent of the estimated collision risk and / or the planned behavior of the own agent, output a warning that the other agent does not recognize the own agent in response to the estimated collision risk, and control the own agent using the estimated collision risk and / or the planned behavior of the own agent.

[0105] Figures 12(A) and 12(B) each show a use case using an example implementation of a method according to one embodiment. Thus, Figures 12(A) and 12(B) each show a use case using an example implementation of the method of Figure 1, and thus an example implementation of the method of the first aspect.

[0106] In the examples of Fig. 12(A) and Fig. 12(B), it is assumed that the own agent 1 and the other agent 2 existing in the environment of the own agent 1 are cars driven by a person. The following explanation is correspondingly valid when the own agent 1 and / or the other agent 2 are other kinds of vehicles. Fig. 12(A) and Fig. 12(B) show the viewing area 1a of the own agent 1 and the viewing area 2a of the other agent 2. Since the own agent 1 and the other agent 2 are assumed to be cars driven by a person, the viewing areas and viewing states of the own agent 1 and the other agent 2 are the viewing areas and viewing states of the person driving the own agent 1 or the other agent 2, respectively. According to the examples of Fig. 12(A) and Fig. 12(B), it is assumed that the own agent 1 and the other agent 2 drive in adjacent lanes of the road, and the other agent 2 runs ahead of the own agent 1 in the moving direction.

[0107] As shown in Figures 12(A) and 12(B), the other agent 2 may transmit its own viewing state (i.e., the viewing state of the other agent 2) to the own agent 1 (indicated by the dashed arrow). That is, the other agent 2 may be configured to estimate its own viewing area 2a (i.e., the viewing area 2a of the other agent 2) and use its own viewing area to calculate its own viewing state (i.e., the viewing state of the other agent 2) with respect to the own agent 1. For this purpose, one or more cameras may be installed on the other agent, facing towards the face of the person driving the other agent 2. The own agent 1 and the other agent 2 may each be equipped with a communication device for communicating with each other.

[0108] In the examples of Fig. 12(A) and Fig. 12(B), it is assumed that the person driving the own agent 1 looks in the direction of the other agent 2 and thus recognizes the other agent 2. This is indicated by the fact that the viewing area 1a of the own agent 1 includes the other agent 2. In the example of Fig. 12(A), it is assumed that the driver of the other agent 2 looks in the direction of the own agent 1 and thus recognizes the own agent 1, for example because he wants to change lanes and therefore looks behind. This is indicated by the fact that the viewing area 2a of the other agent 2 includes the own agent 1, as shown in Fig. 12(A). In the example of Fig. 12(B), it is assumed that the driver of the other agent 2 does not look in the direction of the own agent 1 and therefore does not recognize the own agent 1. This is indicated by the viewing area 2a of the other agent 2 not including the own agent 1, as shown in Fig. 12(B).

[0109] The self agent 1 may use the received visibility state of the other agent 2 to perform step 300 of the method of FIG. 1 and step 400 of the method of FIG. 1. Additionally or alternatively, the visibility state of the other agent 2 may be transmitted to an autonomous system that may perform steps 300 and 400 of the method of FIG. 1. In the example of FIG. 12(A), the calculated visibility state indicates that the driver of the other agent 2 recognizes the self agent 1, so the collision risk between the self agent and the other agent 2 is smaller compared to the example of FIG. 12(B), where the calculated visibility state indicates that the driver of the other agent 2 does not recognize the self agent 1. The estimated collision risk may depend on one or more further criteria. For example, if the self agent 1 drives at a higher speed and the other agent 2 plans to change into the self agent's lane, the collision risk between the self agent 1 and the other agent 2 is higher compared to when the self agent 1 drives at a lower speed.

[0110] As shown in Fig. 12(B), depending on the estimated collision risk, a warning that the other agent 2 does not recognize the own agent may be output to the driver of the own vehicle 1. In the example of Fig. 12(A), since the driver of the other agent 2 recognizes the own agent 1, such a warning does not need to be output.

[0111] The own agent 1 and the other agent 2 may be configured to perform any combination of the features of any of Figures 1 to 10. For further details regarding optional features, e.g., method steps that may be performed and implementation features of the own agent 1 and the other agent 2, please refer to the description of the method according to the first aspect.

[0112] Figures 13(A) and 13(B) each show a use case using an example implementation of one step of a method according to one embodiment. Thus, Figures 13(A) and 13(B) each show a use case using an example implementation of the method of Figure 1, and thus an example implementation of the method of the first aspect.

[0113] In the examples of Fig. 13(A) and Fig. 13(B), the self agent 1 is assumed to be an autonomous driving (AD) vehicle, and the other agent 2 present in the environment of the self agent 1 is assumed to be a vehicle driven by a person. The following description is correspondingly valid when the self agent 1 and / or the other agent 2 are other kinds of vehicles. Fig. 13(A) and Fig. 13(B) show the viewing area 1a of the self agent 1 and the viewing area 2a of the other agent 2. Since the self agent 1 is assumed to be an AD vehicle, the viewing area of ​​the self agent 1 is the viewing area provided by one or more sensors of the AD vehicle for providing a visual perception of the environment of this AD vehicle to a control entity of the AD vehicle. The control entity may comprise one or more control devices on board the AD vehicle and / or one or more external control devices, e.g., an autonomous system. Since the other agent 2 is assumed to be a vehicle driven by a person, the viewing area and viewing state of the other agent 2 are the viewing area and viewing state of the person driving the other agent 2.

[0114] In the examples of Figures 13(A) and 13(B), it is assumed that the own agent 1 and the other agent 2 are in adjacent lanes of the road, the other agent 2 is ahead of the own agent 1 in the direction of movement, and the other agent 2 cannot move forward due to a parked vehicle 5.

[0115] The other agent 2 may transmit its own visibility state (i.e., the visibility state of the other agent 2) to the own agent 1. That is, the other agent 2 may be configured to estimate its own visibility area 2a (i.e., the visibility area 2a of the other agent 2) and use its visibility area to calculate its own visibility state (i.e., the visibility state of the other agent 2) with respect to the own agent 1. For this purpose, one or more cameras may be installed on the other agent 2, facing towards the face of the person driving the other agent 2. The own agent 1 and the other agent 2 may each be equipped with a communication device for communicating with each other.

[0116] In the examples of Fig. 13(A) and Fig. 13(B), it is assumed that one or more sensors of the AD system of the own agent 1 perceive the other agent 2, and thus the AD system of the own agent 1 recognizes the other agent 2. This is indicated by the viewing area 1a of the own agent 1 at least partially encompassing the other agent 2. In the example of Fig. 13(A), it is assumed that the driver of the other agent 2 does not look in the direction of the own agent 1, and thus does not recognize the own agent 1. This is indicated by the viewing area 2a of the other agent 2 not encompassing the own agent 1, as shown in Fig. 13(A). In the example of Fig. 13(B), it is assumed that the driver of the other agent 2 looks in the direction of the own agent 1, and thus recognizes the own agent 1, because he wants to change lanes, for example, and therefore looks behind. This is indicated by the viewing area 2a of the other agent 2 at least partially encompassing the own agent 1, as shown in Fig. 13(B).

[0117] The own agent 1, i.e., its AD system, may use the received visibility state of the other agent 2 to perform step 300 of the method of Fig. 1 and step 400 of the method of Fig. 1. In the example of Fig. 13(B), the calculated visibility state indicates that the driver of the other agent 2 recognizes the own agent 1, so the collision risk between the own agent and the other agent 2 is smaller compared to the example of Fig. 13(A) where the calculated visibility state indicates that the driver of the other agent 2 does not recognize the own agent 1.

[0118] By sharing the visibility state of the other agent 2 with the own agent 1, the AD function of the own agent 1 can be improved. That is, the AD system of the own agent 1 can use this information, i.e., whether the other agent 2 recognizes the own agent 1, to inform the driver of the other agent 2 that the other agent 2 may go ahead and change into the own agent 1's lane to avoid the parked vehicle 5. For example, in the example of FIG. 13(A), the visibility state of the other agent 2 indicates that the driver of the other agent 2 does not recognize the own agent 1 because he is not looking in the direction of the own agent 1, so the AD system may use an acoustic notification, e.g., a horn, to inform the driver of the other agent 2 that he may go ahead and change his lane. In the example of FIG. 13(B), the visibility state of the other agent 2 indicates that the driver of the other agent 2 recognizes the own agent 1, so the AD system may use a visual signal, e.g., a car headlight, to inform the driver of the other agent 2 that he may go ahead and change lanes.

[0119] The above-mentioned notification behavior may be realized by controlling the behavior using a cost function and minimizing the cost function. Here, if the visibility state of the other agent 2 indicates that the driver of the other agent 2 recognizes the own agent 1, the behavior cost of using the headlight signal is small (cost headlight→0) When the visibility state of the other agent 2 indicates that the driver of the other agent 2 does not recognize the own agent 1, the behavior cost of using the horn signal is small (cost honk →0).

[0120] The own agent 1 and the other agent 2 may be configured to perform any combination of the features of any of Figures 1 to 10. For further details regarding optional features, e.g., method steps that may be performed and implementation features of the own agent 1 and the other agent 2, please refer to the description of the method according to the first aspect.

[0121] Figures 14(A), 14(B) and 14(C) each show a use case using an example of the implementation of the method according to one embodiment. Thus, Figures 14(A), 14(B) and 14(C) each show a use case using an example of the implementation of the method of Figure 1, and therefore an example of the implementation of the method of the first aspect. The self agent 1 and the other agent 2 may correspond to the self agent 1 and the other agent 2 of Figures 12(A) and 12(B). Thus, the description of Figures 12(A) and 12(B) may be valid for the self agent 1 and the other agent 2 of Figures 14(A), 14(B) and 14(C), and the following mainly describes optional features.

[0122] The own agent 1 may estimate the visibility area of ​​the other agent 2 by determining the blind spot area 6 of the other agent 2 using the shape of the other agent 2. The own agent 1 may plan its own behavior (i.e. the behavior of the own agent 1) by performing step 400 of the method of FIG. 1. To perform step 400 of FIG. 1, the own agent 1 may use a cost function that penalizes a position of the own agent 1 in the blind spot area 6 of the other agent 2. For example, the own agent 1 may use an additional zone risk in the cost function of the behavior planner of the own agent 1 that penalizes a position of the own agent 1 in the blind spot area 6 of the other agent 2. Thus, the behavior planner of the own vehicle 1 may be configured to plan the behavior of the own vehicle 1 such that a position of the own vehicle 1 in the blind spot area 6 of the other agent 2 is avoided. For example, the cost R of a particular trajectory along which the own vehicle 1 travels may be calculated as zone (trajectory) is the position pos ego It may be 1 when (trajectory) is within the blind area 6 and zero otherwise. This may be formulated as follows:

[0123]

number

[0124] In the use cases of Figures 14(A), 14(B), and 14(C), the visibility state of the other agent 2 is not necessarily sent from the other agent 2. The visibility state of the other agent 2 may be derived by the own agent 1 based on the shape of the other agent 2.

[0125] 14(A), since the own agent 1 is in the blind spot area 6 of the other agent 2, the driver of the own agent 1 may be notified of this and requested to accelerate in order to move the own agent 1 out of the blind spot area 6 of the other agent 2, or the own agent 1 may be controlled to autonomously accelerate in order to move out of the blind spot area 6 of the other agent 2. In the example of FIG. 14(B), since the own agent 1 is outside the blind spot area 6 of the other agent 2, the driver of the own agent 1 may be notified of this and requested to drive the own agent 1 at a constant speed in order to continue to be outside the blind spot area 6 of the other agent 2, or the own agent 1 may be controlled to autonomously move at a constant speed in order to continue to be outside the blind spot area 6 of the other agent 2. In the example of Figure 14(C), the own agent 1 is planning to move through the blind spot area 6 of the other agent 2 (indicated by the arrow), so the driver of the own agent 1 may be notified of this and asked to quickly overtake the other agent 2 so as to minimize the time spent in the blind spot area 6 of the other agent 2, or the own agent 1 may be controlled to quickly overtake the other agent 2 so as to minimize the time spent in the blind spot area 6 of the other agent 2.

[0126] The own agent 1 and the other agent 2 may be configured to perform any combination of the features of any of Figures 1 to 10. For further details regarding optional features, e.g., method steps that may be performed and implementation features of the own agent 1 and the other agent 2, please refer to the description of the method according to the first aspect.

[0127] For further details regarding the assistance system of FIG. 11 and the method used in any of the use cases of FIG. 12(A), FIG. 12(B), FIG. 13(A), FIG. 13(B), FIG. 14(A), FIG. 14(B), and FIG. 14(C), please refer to the description of the method of the first aspect and the description of the previous figure.

[0128] All steps performed by various entities described in this disclosure and functions described as being performed by various entities are intended to mean that the respective entities are adapted or configured to perform the respective steps and functions. In the claims as well as in the detailed description, the word "comprising" does not exclude the presence of other elements or steps.

[0129] The indefinite article "a" or "an" does not exclude a plurality. A single element or other unit may fulfill the functions of several entities or articles recited in the claims. The mere fact that certain means and features of a computer-implemented method for supporting an agent are recited in different dependent claims does not exclude that combinations of these means and features cannot be combined in an advantageous implementation.

Claims

1. 1. A computer-implemented method for assisting an own agent, comprising: estimating a visual recognition area of ​​another agent present in the environment of the own agent; calculating a visibility state of the other agent with respect to the own agent using the estimated visibility area of ​​the other agent; estimating a collision risk between the own agent and the other agent using the calculated visibility states of the other agents; planning a behavior of the own agent by minimizing a total cost of the behavior, the total cost including the estimated collision risk; informing the own agent of the estimated collision risk and / or the planned behavior of the own agent; outputting a warning that the other agent does not recognize the own agent in response to the estimated collision risk; and Controlling the ego-agent using the estimated collision risk and / or the planned behavior of the ego-agent. and performing at least one of the following:

2. estimating the visual recognition area of ​​the other agent, estimating a direction of visual perception of the other agent; tracking a history of the direction of the other agent's visual perception; and generating the visibility area using the tracked history of the direction of the visual perception of the other agent.

3. estimating the visual recognition area of ​​the other agent, one or more cameras of the other agent facing the face of the operator of the other agent when the other agent is operated by a operator present at the other agent; one or more sensors in the other agent that, when the other agent is operated by an operator present in the other agent, sense whether the operator of the other agent is aware of or alerted to the one-agent; one or more sensors of the other agent that sense the other agent's environment; one or more cameras located in the other agent's environment; and When the other agent is operated by an operator present at the other agent, one or more cameras of the own agent facing the face of the operator of the other agent The method of claim 1 , further comprising using at least one of the following data:

4. estimating the visual recognition area of ​​the other agent, applying a ray projection from the position of the other agent; and reducing the visibility area of ​​the other agent in response to an area obstructed by objects in the other agent's environment.

5. estimating the visual recognition area of ​​the other agent, 2. The method of claim 1, comprising setting the visibility area to zero square meters if the other agent is operated by or is a human and the human is looking at a mobile device or talking to at least one other person.

6. The method further comprising: estimating a visual recognition area of ​​said own agent; calculating a visibility state of the own agent with respect to the other agent using the estimated visibility area of ​​the own agent; and estimating the collision risk between the own agent and the other agent using the calculated visibility state of the other agent and the calculated visibility state of the own agent.

7. The visibility state of the other agent is a Boolean variable as to whether the own agent's position is within the estimated visibility area of ​​the other agent; a distance variable indicating the distance of the position of the own agent to the viewing area of ​​the other agent; and a random variable indicating that the other agent is aware of the own agent, the random variable depending on the position of the own agent and the visual recognition area of ​​the other agent; The method of claim 1 , which may include at least one of:

8. calculating a visibility state of the other agent with respect to the own agent using the estimated visibility area of ​​the other agent; combining the calculated visibility state of the other agent with a confidence value of the estimate; and using at least one of a moving average, hysteresis, and outlier correction; The method of claim 1 , comprising at least one of:

9. estimating a collision risk between the own agent and the other agent using the calculated visibility state of the other agent; predicting the behavior of the other agent using the calculated visibility state of the other agent; and modifying the collision risk by a risk factor calculated using the calculated visibility state of the other agent; The method of claim 1 , comprising at least one of:

10. The method further comprising: calculating a risk factor using the calculated visibility states of the other agents; and if the estimated collision risk is higher than the collision risk threshold and the calculated risk coefficient is higher than the risk coefficient threshold, outputting a warning that the other agent does not recognize the own agent.

11. The method further comprising: determining whether the other agent recognizes the own agent based on the estimated visual recognition area of ​​the other agent and a position of the own agent; notifying the own agent to use a visual notification to notify the other agent if the other agent is aware of the own agent, and to use an audio notification to notify the other agent otherwise; and controlling said own agent to use a visual notification to notify said other agent if said other agent is aware of said own agent, and to use an audio notification to notify said other agent otherwise; and performing at least one of the following:

12. estimating the visibility area of ​​the other agent includes determining a blind spot area of ​​the other agent using a shape of the other agent; The method of claim 1 , wherein planning the behavior by minimizing a total cost of the ego agent's behavior includes using a cost function that penalizes positions of the ego agent within the blind spot areas.

13. The method of claim 1 , wherein the informing the own agent and / or outputting the alert is performed using a human machine interface (HMI).

14. The method of claim 1 , wherein informing the own agent and / or outputting the alert may be performed in at least one of a visual, an audio, and a haptic manner.

15. The method of claim 1 , wherein the warning may be provided such that a modality of the warning and / or an intensity of the warning announcement depends on the estimated collision risk.

16. 1. A program comprising program code means for carrying out, when executed on a computer or digital signal processor, a method for supporting an own agent, said method comprising: estimating a visual recognition area of ​​another agent present in the environment of the own agent; calculating a visibility state of the other agent with respect to the own agent using the estimated visibility area of ​​the other agent; estimating a collision risk between the own agent and the other agent using the calculated visibility states of the other agents; planning a behavior of the own agent by minimizing a total cost of the behavior, the total cost including the estimated collision risk; informing the own agent of the estimated collision risk and / or the planned behavior of the own agent; outputting a warning that the other agent does not recognize the own agent in response to the estimated collision risk; and Controlling the ego-agent using the estimated collision risk and / or the planned behavior of the ego-agent. and performing at least one of the following:

17. A non-transitory computer readable storage medium embodying a program of machine readable instructions executable by a digital processing apparatus, the instructions causing the digital processing apparatus to: Estimating a visual recognition area of ​​another agent present in the environment of the own agent; calculating a visibility state of the other agent with respect to the own agent using the estimated visibility area of ​​the other agent; estimating a collision risk between the own agent and the other agent using the calculated visibility states of the other agents; planning a behavior of the own agent by minimizing a total cost of the behavior, the total cost including the estimated collision risk; informing the own agent of the estimated collision risk and / or the planned behavior of the own agent; outputting a warning that the other agent does not recognize the own agent in response to the estimated collision risk; and Controlling the ego-agent using the estimated collision risk and / or the planned behavior of the ego-agent. and performing at least one of the following: A non-transitory computer-readable storage medium that causes

18. A support system for supporting a self-agent, the system comprising: estimating a visual recognition area of ​​another agent present in the environment of the own agent; calculating a visibility state of the other agent with respect to the own agent using the estimated visibility area of ​​the other agent; estimating a collision risk between the own agent and the other agent using the calculated visibility states of the other agents; planning a behavior of the own agent by minimizing a total cost of the behavior, the total cost including the estimated collision risk; informing the own agent of the estimated collision risk and / or the planned behavior of the own agent; outputting a warning that the other agent does not recognize the own agent in response to the estimated collision risk; and Controlling the ego-agent using the estimated collision risk and / or the planned behavior of the ego-agent. and performing at least one of the following: An assistance system comprising a processor configured to:

19. The assistance system according to claim 18 , wherein the assistance system is configured to assist the operator of the own agent when the own agent is a vehicle operated by an operator.

20. A vehicle including an assistance system according to claim 18.

Images