Automated vehicle and method of operating an automated vehicle

Abstract

The present disclosure relates to a method of remotely operating a driving (TOD) AV (703) in a situation where an autonomous driving vehicle AV (703) cannot resolve a situation, for example, due to unclear traffic conditions. Because a TOD session is caused by a special, unclear, and relatively rare situation, and measures are applied that may violate normal traffic rules (e.g., a traffic controller directs a vehicle onto a bicycle lane because a road is blocked) or normal behavior expected from an AV, according to the solution, the vehicle notifies other AVs in the vicinity of the TOD session. In addition, according to another aspect of the present disclosure, other vehicles use information about the TOD session to help them recognize the situation and perform driving.

Description

Technical Field

[0001] The present invention relates to an automated vehicle and a method of operating an automated vehicle. In particular, the present invention relates to a method of operating an automated vehicle in a situation where an automated driving vehicle (AV) cannot resolve a traffic situation by automated driving. Background Art

[0002] Prior art defines an autonomous vehicle, also known as an AV, autonomous vehicle, driverless car, robotic car or robot car, as a vehicle that is able to sense its environment and move safely with little or no human input.

[0003] Self-driving cars use a combination of sensors to perceive their surroundings, such as radar, lidar, sonar, GPS, odometers, and inertial measurement units. Advanced control systems interpret the information obtained from the sensors to identify appropriate navigation paths, as well as obstacles and relevant landmarks.

[0004] Furthermore, automotive applications and mobile communications are becoming increasingly entangled, in particular due to the growing interest in autonomous driving which requires large amounts of data when compared to conventional driving. These amounts of data are partly provided by the vehicle itself, i.e. by its sensors, and partly via air interfaces. The communication is effected via air interfaces such as vehicle-to-vehicle (V2V) communication or vehicle-to-infrastructure (V2I) communication or vehicle-to-everything (V2X) communication, the latter including communication with roadside units RSU.

[0005] In the case where V2X communication is carried out via a cellular mobile network (such as, for example, an LTE or 5G communication network), it is referred to as Cellular-V2X, C-V2X. V2V and C-V2X communication can be carried out in an LTE or 5G network with a sidelink carrier at the PHY layer (PC5 sidelink), or based on WLAN communication according to the IEEE 802.11p standard.

[0006] Automotive standardization body SAE International published a six-level AV classification system in 2014, ranging from fully manual to fully automated systems, called J3016, Taxonomy and Definitions of Terms Relating to Automated Driving Systems for Road Motor Vehicles.

[0007] SAE Level 4 ("No Attention Required"): Based on SAE Level 3 (as each higher level builds on the lower ranking levels regarding automated driving capabilities), where driver attention is never required for safety, e.g. the driver can safely go to sleep or leave the driver's seat. Automated driving is only supported in limited spatial areas (geofences) or in special circumstances. Outside of these areas or circumstances, the vehicle must be able to safely abort the journey, e.g. to park, if the driver does not regain control.

[0008] SAE Level 5 (“Steering Wheel Optional”); Based on SAE Level 4, but without the need for human intervention at all. An example is a robotaxi. At SAE Level 5, the automated system will never need to request intervention.

[0009] It is known from the prior art to use teleoperated driving (TOD) maneuvers in situations that cannot be handled safely and automatically by the AV, in order to enable the AV to resolve traffic / road situations. However, during a TOD session, the AV is remotely controlled and may behave in an unconventional manner, for example, driving on sidewalks or otherwise violating normal traffic rules. Therefore, during a TOD session, there is an increased risk of collision with other vehicles.

[0010] It is therefore an object of the present disclosure to overcome or at least reduce the disadvantages of the prior art and to present an improved method for teleoperated driving of an automated vehicle which allows reducing the risk of collision with other vehicles. Summary of the invention

[0011] A first aspect of the present disclosure relates to a method for operating an automatic vehicle AV. Therein, the method includes the step of determining that operating the AV under current traffic conditions requires TOD operation of the AV. When the AV determines that TOD operation is required, the method of the present disclosure further includes the step of transmitting a TOD message to at least one other nearby receiver, wherein the TOD message includes information that the AV requires TOD operation. The receiver is preferably part of another vehicle, for example, a communication module of another vehicle. However, the receiver may also be part of a mobile device such as, for example, a smart phone. Advantageously, by receiving the TOD message, the receiver knows that the AV requires TOD operation. Therefore, based on the receipt of the TOD message, measures can be taken taking into account that the AV may behave in an unconventional manner and may even violate traffic rules (e.g., crossing a sidewalk, etc.).

[0012] In a preferred embodiment of the present disclosure, the method further comprises the following steps: determining, by the AV, a traffic condition near the AV based on at least one detected environmental signal representative of the AV's vicinity. In other words, the AV uses at least one sensor (such as, for example, LIDAR, camera, etc.) to detect a signal representative of its surroundings. In this way, the AV can obtain information about moving and stationary objects in its surroundings, and can determine traffic conditions based on this information. Wherein, the traffic condition can be determined, for example, by another car, a tree, or rubble blocking the road.

[0013] In this preferred embodiment, the method further includes the following steps: determining that the operation of the AV by the autonomous driving module of the AV is prohibited under the determined traffic conditions, and TOD operation is required. Exemplarily, the AV can use the determined traffic conditions as input to the autonomous driving module, and the autonomous driving module can determine that it is wrong or that it cannot operate the AV under such traffic conditions. Such a determination can be based on boundary conditions reflecting traffic rules and / or preferred operations of the AV, such as, for example, not passing through a sidewalk. In other words, when the autonomous driving module determines that the automatic operation of the AV will require crossing a sidewalk, the AV determines that the operation of the AV by the autonomous driving module of the AV is prohibited and TOD operation is required, and transmits a TOD message.

[0014] In other words, abnormal conditions may be referred to as boundary conditions or limitations such that when the AV reaches such boundary conditions, automatic / autonomous driving is not feasible (usually prohibited due to safety reasons), and thus a TOD session is required.

[0015] Preferably, the method further comprises the step of transmitting a TOD request message to a control center for requesting a TOD operation based on determining that a TOD operation is required, wherein the TOD request message comprises information about the current traffic conditions. Further preferably, the TOD request message comprises an identification of the reason for the required TOD session as analyzed by the AV itself. Thereby, the control center will know the current traffic conditions, which caused the TOD operation. In other words, such identification explains why the vehicle believes that the current situation requires a TOD session.

[0016] Preferably, the method further comprises the steps of: initiating a TOD session with a control center; allowing the control center to control the AV; receiving further data about the current traffic condition from the control center; and updating the TOD message transmitted by the AV to other receivers based on the further data. Advantageously, at this stage, the control center typically identifies itself back to the AV. In an embodiment, the further data about the current traffic condition may be a definition obtained from a human in the control center, who identifies the current traffic condition by examining the audio / video signals received from the AV and notifies the AV about the reason for the required support, such as a truck blocking the street. Thus, the further data provides more information (feedback) about the road condition initially identified by the AV. In some cases, the reason given by the possible further data may be different from the reason given by the AV itself.

[0017] Preferably, updating the TOD message includes adding information about at least one manoeuvre that allows the AV to reach a state in which the operation of the AV by the autonomous driving module of the AV is feasible. To this end, the TOD message may be updated with information related to at least one maneuver, which may be explicitly given by the control center or inferred from the driving of the AV under control from the control center. This will help other AVs to reuse possible remedies for the current traffic situation.

[0018] Exemplarily, information about at least one maneuver is inferred from the driving of the vehicle under control from the control center. In particular, it can be inferred from the final driving / movement path of the AV under control from the control center. As an example, such a maneuver may include not only a direction or path, but also other parameters such as speed and / or duration (e.g., so that the AV occupant can be informed).

[0019] Preferably, the information about at least one maneuver is derived from further data received from the control center. In particular, the further data may also include maneuver instructions allowing to solve the current traffic situation. Preferably, the information about at least one maneuver is a driving path. Even without intervention from the control center, other AVs in the current traffic situation can use such a driving path to solve the current traffic situation. As a benefit, when solutions (e.g. driving paths) can be shared between local AVs, initializing the vehicle's own TOD session can be avoided. Therefore, the number of TOD sessions can be reduced.

[0020] Preferably, the TOD message includes: a timestamp; an identifier of the AV; and a geographic location of the AV. Such data allows identification of the time and geographic location at which the identified AV encountered a situation requiring a TOD session. In some embodiments, the make / model of the vehicle may preferably be given with the vehicle type, size explicitly given, which may facilitate helping other vehicles determine whether they can easily apply the same at least one maneuver for the same road situation.

[0021] Preferably, the TOD message further includes: information indicating whether the AV is in TOD operation; an identifier of a command center obtained from further data; and / or an expected duration of the TOD session obtained from further data. The foregoing information allows other AVs in the vicinity to be informed that, for a given AV, a TOD operation is ongoing and that a particular control center is involved (which is also thereby identified as potentially knowing a way to resolve the current traffic situation). The expected duration increases awareness of other AVs. This information may help to let other vehicles know when a given TOD session may end, and therefore know when the outcome of such a TOD session can be expected.

[0022] Preferably, before transmitting the TOD request message, the method performs the following steps: receiving a TOD message from another AV; extracting information about the TOD session of the other AV from the TOD message; and utilizing the extracted information when establishing a TOD session or adjusting the driving of the AV based on the extracted information. In order to improve the awareness of other vehicles, they receive TOD messages. Such messages may be received by AVs or non-AVs. In the case of AVs, this allows avoiding a situation in which many AVs are calling (i.e. establishing TOD sessions) one or more control centers to solve the same problem (which is being or has been solved or otherwise solved). To this end, the AV can adjust the driving of the AV based on the extracted information, or at least utilize the extracted information when establishing a TOD session (so that such a session can, for example, last shorter). Preferably, the TOD message is also updated during the TOD session depending on changes in its content.

[0023] In another embodiment, the present invention relates to a method for operating an automatic vehicle AV, the method comprising the steps of: determining that a remote-controlled driving TOD operation of another AV is required by receiving TOD information from another AV; and adapting the autonomous driving operation of the AV and / or outputting a signal to the occupants of the AV in response to the determination. This is intended to notify the occupants about the TOD session and / or adapt the autonomous driving operation of the AV (i.e., without notifying any (one or more) occupants in silent mode). Preferably, the method further comprises the steps of: extracting information about the TOD session of another AV from the TOD message; utilizing the extracted information when establishing a TOD session and / or when adapting the autonomous driving operation of the AV. Further preferred implementations, for example, regarding the content of the received and / or updated TOD message correspond to the above-mentioned embodiments of the method for AV to transmit a TOD message in response to determining that a TOD operation is required.

[0024] The invention further relates to a computer program comprising program code means for performing all the steps of the computer-implemented method according to the method disclosed herein, when said program is run on a computer.

[0025] Furthermore, the object of the present disclosure is a computer-readable medium storing computer-executable instructions which, when executed on a computer, perform all the steps of the computer-implemented method according to the method disclosed herein.

[0026] Another aspect of the present invention is an automatic vehicle AV for remotely operated driving TOD, the AV comprising: a communication module configured for two-way communication with a control center; an automatic driving module configured to control the driving of the AV; a TOD session manager configured to determine whether operating the AV under current traffic conditions requires remotely operated driving TOD operation of the AV; and a controller configured to execute the method disclosed herein.

[0027] Preferably, the TOD session is established via Uu interface communication, such as, for example, a message transmitted via a 4G or 5G mobile communication network. Further preferably, the TOD session is established via a side link interface communication. Preferably, the TOD message is a broadcast message. However, in the context of the claimed method, a unicast message may also be transmitted as a TOD message. In such an embodiment, the method of the present disclosure preferably further includes the step of initializing communication between the AV and another AV in the vicinity of the AV.

[0028] Yet another aspect of the invention is a control center method for remote control driving, the method comprising the steps of: receiving a remote control driving request from a vehicle; establishing a communication session with the vehicle in order to perform a remote control driving session; accepting a remote control driving instruction from the control center and transmitting the instruction to the vehicle, the method being characterized by waiting for an identification of a cause of the remote control driving session from the control center and reporting the cause to the vehicle. In particular, the identification of the cause of the current traffic condition may be a definition obtained by a human in the control center by examining an audio / video signal received from the AV to identify the current traffic condition and notify the AV of the cause of the required support (e.g. a truck blocking the street). Thus, the further data provides more information (feedback) about the road condition initially identified by the AV. The cause given by the further data may be different from the cause given by the AV itself.

[0029] Preferably, reporting the cause to the vehicle further includes: an identifier of the control center; and / or an expected duration of the TOD session. The expected duration improves awareness of other AVs. This information may help other vehicles know when a given TOD session may end, and therefore know when the outcome of such a TOD session can be expected.

[0030] In a further aspect, the present invention relates to a control center system for remote control driving, comprising: a communication module configured to establish two-way communication with a vehicle; a remote control operation module configured to communicate with the vehicle to guide its behavior; a TOD session manager configured to establish a remote control driving TOD session with the vehicle; and a controller, the system being characterized in that the controller is further configured to execute all steps of the control center method.

[0031] Unless stated otherwise in individual cases, the various embodiments of the invention mentioned in this application can advantageously be combined with one another. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] These and other objects of the invention presented herein are achieved by providing an automated vehicle and a method of operating an automated vehicle. Further details and features of the invention, its nature and various advantages will become more apparent from the following detailed description of preferred embodiments illustrated in the accompanying drawings, in which:

[0033] Figure 1 A diagram of an automated vehicle system according to the present invention is presented;

[0034] Figure 2 A diagram of a method of a vehicle according to the invention is presented when the cause of an unclear situation is detected internally;

[0035] Figure 3 A diagram is presented of a method for a vehicle according to the invention when the cause of an unclear situation is detected externally;

[0036] Figure 4 An example of a TOD session message is presented;

[0037] Figure 5 A diagram of a control center system according to the present invention is presented;

[0038] Figure 6 A diagram of a control center method according to the present invention is presented;

[0039] Figure 7 Examples of traffic situations requiring a TOD session are shown. DETAILED DESCRIPTION

[0040] Some portions of the detailed description that follows are presented in terms of data processing procedures, steps, or other symbolic representations of operations on data bits that can be performed on a computer memory. Accordingly, the computer performs such logical steps which require physical manipulations of physical quantities.

[0041] Usually these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated in a computer system. For common reasons, these signals are referred to as bits, packets, messages, values, elements, symbols, characters, terms, numbers, or the like.

[0042] Furthermore, all of these and similar terms are associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Terms such as "process" or "create" or "transfer" or "execute" or "determine" or "detect" or "obtain" or "select" or "compute" or "generate" refer to the actions and processes of a computer system that manipulate and transform data represented as physical (electronic) quantities within the computer's registers and memories into other data similarly represented as physical quantities within the memories or registers or other such information storage devices.

[0043] Computer-readable (storage) media, such as referred to herein, may generally be and / or include non-transitory devices. In this context, non-transitory storage media may include devices that may be tangible, meaning that the device has a concrete physical form, although the device may change its physical state. Thus, for example, non-transitory refers to a device that remains tangible despite changes in state.

[0044] As utilized herein, the term "example" is meant to serve as a non-limiting example, instance, or illustration. As utilized herein, the terms "for example" and "such as" introduce a list of one or more non-limiting examples, instances, or illustrations.

[0045] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items. In addition, when describing embodiments of the present invention, the use of "may" refers to "one or more embodiments of the present invention."

[0046] In addition, in the following description of the embodiments of the present invention, terms in the singular form may include plural forms unless the presented context clearly indicates otherwise.

[0047] It should be understood that although the terms "first" and "second" are used to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, without departing from the scope of the present invention, the first element can be named as the second element, and similarly, the second element can be named as the first element. As used herein, the term "and / or" includes any and all combinations of one or more associated listed items and expressions, such as "at least one", when before an element list, modifies the entire element list.

[0048] Reference will now be made in detail to the embodiments illustrated in the accompanying drawings. The effects and features of the exemplary embodiments will be described with reference to the accompanying drawings. Wherein, the same reference numerals mark the same elements, and redundant descriptions are omitted. However, the present invention can be embodied in various different forms and should not be construed as being limited to the embodiments illustrated herein. On the contrary, these embodiments are provided only as examples to fully convey the various aspects and features of the present invention to those skilled in the art.

[0049] According to the present invention, in case the road is unclear, the AV needs external support to overcome such unclear / unexpected situation. The support may be given by a so-called command center (CC) or also referred to as a remote operator or control center. In such a case, the AV is remotely controlled by starting a TOD session with the CC. In most cases, the AV initiates the TOD session at its own request. During the TOD session, the AV is remotely controlled and may behave in an unconventional way, such as driving on sidewalks or otherwise violating normal traffic rules (such as those that may be directed by the police, etc.).

[0050] Because TOD sessions are caused by special, unclear, and relatively rare circumstances, and measures that may violate normal traffic rules are applied (e.g., traffic control personnel direct vehicles onto a bike lane because the road is blocked), it is advantageous to inform other AVs in the vicinity of the TOD session. Further, other vehicles may use, for example, information about the cause of the triggering TOD session (such as a truck blocking the street) to help them understand the situation and make driving-related decisions.

[0051] According to the present invention, AVs requesting or participating in a TOD session should share this information via direct communication through a TOD message (preferably broadcast, but other techniques such as multicast or unicast can also be applied). It will be a new broadcast message, and can also be included in the existing vehicle broadcast message system as, for example, a cooperative awareness message (CAM). Such cooperative awareness is the basis of a large number of intelligent transportation systems in which AVs share information.

[0052] Figure 1 A diagram of an AV system according to the invention is presented. The system is preferably present on the vehicle side, i.e. in the vehicle. The system can be embedded or portable, i.e. controlled by a user's device such as a tablet.

[0053] The system can be implemented using dedicated components or custom FPGA or ASIC circuits. The system includes a data bus 101 communicatively coupled to a memory 104. In addition, other components of the system are communicatively coupled to the data bus 101 so that they can be managed by a controller 105.

[0054] The memory 104 may store system configurations and / or computer programs (one or more) executed by the controller 105 in order to perform the steps of the method according to the present invention (see Figure 2-3 ). Different memory types may be organized under memory 104 modules such as RAM and / or non-volatile FLASH memory.

[0055] The communication module 107 allows two-way communication with the control center. For example, it can be 3G, 4G, 5G, Wi-Fi communication, etc. Thus, the AV can communicate with external resources, in particular with the control center and other vehicles in its vicinity. The communication can be achieved through a network such as the Internet and / or direct local radio communication such as wireless vehicle-to-vehicle communication (V2V) or cellular vehicle-to-everything (CV2X).

[0056] As will be specifically referenced later Figure 4 By definition, the communication module 107 also allows the transmission of TOD messages.

[0057] In some cases, the communication module 107 may support different communication technologies in parallel, for example, one communication link type for local communication with other vehicles (eg, transmission of TOD messages), and another communication link type for communication with a control center.

[0058] The autonomous driving module 102 is a general component that includes all logical and physical devices responsible for the control of the automated vehicle. It is clear to a person skilled in the art that such a module controls, for example, acceleration, steering, braking, and similar movement-related actions based on readings from corresponding sensors 103. One or more sensors 103 (also referred to as environmental sensors) may also detect traffic signals, for example, by capturing images of traffic lights, markings on the road surface (such as lane or turn signs, or traffic signs, etc.).

[0059] As should be apparent to those skilled in the art, the sensors 103 may include a geolocation sensor, a proximity sensor, one or more cameras, and any other sensors typically associated with autonomous and teleoperated driving. Non-limiting examples of the sensors 103 may be selected from the group consisting of an accelerometer, a gyroscope, a magnetometer, an optical sensor, an image capture sensor, and a radar sensor.

[0060] Optionally, the vehicle system may include a module for communicating with a user 108, which may allow the driver to communicate with the system and / or control center. Such communication components may be audio and / or video communication components. The driver will typically communicate with an operator who manages the TOD session at the control center.

[0061] Typically, there are three operating modes in modern AVs: manual mode, automatic mode, and remote control mode. However, sometimes (such as on SAE Level 5) the manual mode may be omitted.

[0062] The TOD session manager 106 is configured to determine unexpected or unclear traffic conditions (such conditions may also be referred to as abnormal or predefined conditions) that may be reported internally by the autonomous driving module 102 (see Figure 2 ), reaches a condition that is determined (based on predefined conditions) to require teleoperated driving. Examples of such predefined conditions may include hazardous conditions, such as heavy fog, a fallen tree blocking a road, or a portion of a road being flooded due to heavy rainfall, which makes automated driving not feasible (i.e., prohibited or not allowed under these conditions). In other words, such predefined or abnormal conditions may be referred to as boundary conditions or limitations, such that when the AV reaches such boundary conditions, automated / autonomous driving is not feasible (usually prohibited due to safety reasons), and therefore a TOD session is required.

[0063] In view of the above, under the determined traffic conditions, the operation of the AV by the autonomous driving module 102 of the AV is prohibited and TOD operation is required.

[0064] It should be apparent to one skilled in the art that determining that such a condition has occurred may involve a different analysis of the vehicle's environment and is beyond the scope of the present invention. Alternatively, information may be obtained from external sources such as other vehicles in the vicinity of the AV (see Figure 3 ) to report an unexpected situation.

[0065] When unexpected or unclear traffic conditions occur, the TOD session manager 106 may be further configured to establish a teleoperation session with the control center. This is accomplished using the communication module 107 .

[0066] Figure 2 A diagram of a vehicle method according to the invention is presented when the cause of an unclear situation is detected internally. It is clear that at different stages of the process the AV may update its TOD message, as will be apparent from the examples given below.

[0067] The method begins by determining an unexpected or unclear traffic situation at step 201. For example, a truck is blocking a one-way road, and the AV stops behind the truck while determining that the current situation is unclear.

[0068] Subsequently, the vehicle's system first initiates a TOD session by transmitting a TOD request message at step 202. The TOD request message preferably includes an identification of the reason for the desired TOD session as analyzed by the AV itself (ie, by the autonomous driving module 102).

[0069] For example, the vehicle calls the CC via a Uu link (Uu interface communication, typically long range communication linking the user equipment to the UMTS terrestrial radio access network) and also broadcasts via a side link (typically short range wireless communication) that it needs remote assistance.

[0070] Examples of sidelink communication standards are: PC5 based on cellular communication or Dedicated Short Range Communication (DSRC) based on the WiFi standard 802.11p.

[0071] According to an example, the Uu link is used for communication with the command center (and remote control: in uplink video data and downlink control data), and the side link is used for TOD messages in order to inform the environment about the TOD session.

[0072] Thus, the AV establishes communication with the control center at step 203. At this stage, the control center typically identifies itself back to the AV.

[0073] Next, at step 204 , the vehicle may have optionally updated its broadcast message (or more generally, TOD message) by identifying that it has established a TOD session or is waiting to establish a TOD session.

[0074] In an alternative embodiment, such a broadcast message may be updated (or generated for the first time) even after the AV has requested a TOD session and before the TOD session with the control center is actually initiated 202 or established 203. In this way, other AVs in the vicinity may already be aware of the request for TOD even before the TOD session is actually requested or started.

[0075] Additionally, at step 205, the process allows the control center to remotely control the automated vehicle, such as the control center operator directing the vehicle to a sidewalk or non-asphalted side of the road (e.g., a lawn). In another embodiment, this step is optional, as the TOD session may be established only to obtain information (or additional instructions or approval of a suggested automated behavior) without requiring actual control (in terms of real-time maneuvering) of the AV.

[0076] At step 206, the vehicle is configured to receive an enhanced identification of the unexpected condition (which enhanced identification may be otherwise referred to as further information regarding (the identification of the reason for the required TOD session)) from a control center, for example, a human operator at the control center identifies the condition by examining audio / video signals received from the AV and notifies the AV of the reason for the required support, such as a truck blocking the street.

[0077] The enhanced identification provides more information about the road condition initially identified by the AV (at steps 201-202), which gave rise to the TOD session. Such enhanced identification may be based on, for example, manual description or automatic classification of the photo(s) / video(s), or manual classification selecting one or more predefined road condition categories.

[0078] Subsequently, based on the information obtained at step 206 , the vehicle is updating 207 its broadcast message so that other vehicles can learn more about the unexpected condition.

[0079] Optionally, the update of the broadcast message may include a solution (or partial solution) to resolve the deadlock situation. The solution may be a proposed movement path (and other movement parameters) to achieve an AV state that allows driving without the need for teleoperational control.

[0080] More generally, a solution may be an identification of at least one action or maneuver that allows the AV to reach / be in a state that allows driving without teleoperational control, i.e., allows autonomous driving. The solution may be inferred, for example, from the driving of the AV under control from a control center (e.g., the final driving / movement path). Such a solution may include not only the path, but also other parameters such as speed and / or duration (e.g., so that the AV occupant may know it).

[0081] Figure 3 A diagram of a control center system according to the present invention is presented when the cause of an unclear situation is detected externally. The method starts at step 301 by determining an unexpected or unclear traffic situation by receiving a TOD message from another vehicle (or in general a broadcast message from another vehicle). Figure 4 Provide more details of such messages.

[0082] Subsequently, at step 302, the process verifies whether the current vehicle is an AV or a non-AV. In the case that the current vehicle is an AV, the process moves to step 303, where the vehicle records the expected support at a given location from the control center, preferably given by the message received at step 301. At this stage, the AV can already adjust its driving based on the information extracted from the received TOD message. Such adjustments may include, for example, increasing the alert level, reducing speed, or driving to a safe geographical location.

[0083] Next, at step 304, when or if the vehicle decides to establish its own session with the control center, it may share the received TOD message with its control center when initiating its own TOD session. Typically, such unexpected or unclear traffic conditions will have been resolved by a control center different from the control center of the AV's own TOD session. Such information may be shared within a given control center or between different control centers (even if not explicitly shared with the AV).

[0084] Optionally, at least a portion of the received message (301) may be shared because, for example, some data fields are empty or deemed irrelevant to the current situation of the particular AV.

[0085] To initiate its own TOD session, the process proceeds to step 202. Therefore, steps 302 and 303 are performed before step 202.

[0086] Alternatively, when at step 302, it is established that the current vehicle is a non-AV, then at step 306, the vehicle may inform the driver (or occupants in general) that the other vehicle is an AV and requires support from the control center. Optionally, the vehicle may display the geographic location of the reported TOD session of the other vehicle.

[0087] It is clear to those skilled in the art that the method according to the present invention, and in particular Figure 2-3 At least part of the method can be implemented by a computer. Therefore, the present invention can take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, microcode, etc.), or an embodiment combining software and hardware aspects, which are generally referred to herein as "circuits", "modules" or "systems".

[0088] Furthermore, the present invention may take the form of a computer program product embodied in any tangible medium of expression having computer usable program code embodied in the medium.

[0089] Those skilled in the art can easily recognize that the above method for operating an automatic vehicle can be implemented and / or controlled by one or more computer programs. Such computer programs are usually executed by utilizing computing resources in a computing device. The application is stored on a non-transitory medium. An example of a non-transitory medium is a non-volatile memory, such as a flash memory, and an example of a volatile memory is a RAM. The computer instructions are executed by a suitable processor. These memories are exemplary recording media for storing a computer program including computer executable instructions that implement all steps of the computer-implemented method according to the technical concepts presented herein.

[0090] Figure 4 An example of a TOD message is presented. Such a message 401 may be generated and preferably broadcast by each AV and updated periodically depending on changes in the situation of the autonomous vehicle involved.

[0091] It is clear to those skilled in the art that, depending on specific implementation requirements, the corresponding data fields of the TOD message 401 may have different names and different data formats, and the examples of data formats given below are merely non-limiting preferred options, while focusing on the meaning of the data rather than the specific names or formats of the variables.

[0092] The preferred TOD (broadcast) message 401 includes a TOD status sub-section 402 listing two variables.

[0093] The first variable 402A is preferably called "In TOD Session" and preferably identifies as a Boolean true / false whether the current vehicle is currently under remote control operation of the control center.

[0094] A second variable 402B is preferably called "Waiting for CC to start TOD session" and preferably identifies as a Boolean true / false whether the identified vehicle is currently waiting for a teleoperated session from a control center CC (i.e., a request for a TOD session has been sent, but such a session has not yet started, for example due to a queue of waiting TOD requests at a given control center).

[0095] The next optional field in the TOD message 401 is preferably a text field 403 that describes the reason for the required TOD session as analyzed by the vehicle itself, such as a truck is blocking the street. In other words, field 403 is information about the identification of the current traffic situation 403. In other embodiments, this field may identify one or more standard reasons (e.g., defined by a lookup table) by an identifier, such as an identifier "17" referring to an accident. In other words, this field identifies why the vehicle believes that the current situation requires a TOD session.

[0096] A further optional field 404 identifies a solution provided by the control center, such as a proposed movement path given from an operator at the control center. As already explained, in general, the above solution may be identification of at least one action or maneuver that allows reaching / being in an AV state that allows further driving without the need for teleoperated control (in other words, an AV state that allows autonomous driving).

[0097] The next field 405 identifies the reason for the TOD session as analyzed and established by the control center. This reason may be different from the reason given by the previous 403 field.

[0098] The following field 406 defines the identification (ID) number of the command center and / or provider. Since there may be different control centers, this field allows a given TOD session to be associated with a specific control center. Optionally, an identifier of a specific human operator may also be given.

[0099] Optionally, a further field 407 provides the expected duration of the TOD session. This information may be helpful in order to let other vehicles know when a given TOD session may end, and therefore when the outcome of such a TOD session may be expected, particularly the updated information under fields 404, 405. This duration will typically be given to the AV by the control center.

[0100] A further field is a timestamp 408 which helps other vehicles, upon receiving the AV TOD message 401 , determine whether the TOD message 401 is current or refers to present or past conditions.

[0101] Additionally, field 409 identifies the vehicle. In some embodiments, the make / model of the vehicle may preferably be given with the vehicle type, size clearly given, which may facilitate helping other vehicles determine whether they can easily apply the same solution 404 to the same road situation.

[0102] Finally, the message 401 includes the vehicle's geographic location (position) 410. This is of course critical information as it allows other vehicles to establish how far they are from the road conditions that have given rise to the TOD session.

[0103] It is clear that the AV may also update (also more than once) its TOD message 401 during a TOD session, depending on changes in its content. The control centre may make changes, for example, by specifying further information relevant to the ongoing TOD session.

[0104] The received TOD message 401 may be used by the other vehicle(s) to facilitate the initialization of their own TOD session(s) by sharing the information with their associated control center CC.

[0105] Figure 5 A diagram of a control center system according to the present invention is presented. The system can be implemented using dedicated components or customized FPGA or ASIC circuits. The system includes a data bus 501 communicatively coupled to a memory 504. In addition, other components of the system are communicatively coupled to the system bus 501 so that they can be managed by a controller 505.

[0106] The memory 504 may store (one or more) computer programs executed by the controller 505 in order to perform the steps of the method according to the present invention (see Figure 6). Different memory types, such as RAM and / or non-volatile FLASH memory, may be organized under the memory 504 module. The memory 504 also stores configuration data, such as an identifier of the corresponding control center 406.

[0107] The communication module 507 allows two-way communication with the autonomous vehicle. For example, it can be 3G, 4G, 5G communication, etc. (such as the Internet), and / or direct local radio communication, such as wireless vehicle-to-vehicle communication (V2V) or cellular vehicle-to-everything (CV2X).

[0108] The system may include a module 508 for communicating with the driver, which may allow the control center operator to communicate with the driver (sometimes the AV will not carry a driver, but rather an occupant such as a passenger). Such communication components may be audio and / or video communication components. The driver will typically communicate with a human operator who manages the TOD session at the control center.

[0109] The remote operation module 503 is configured to communicate with the AV to direct its behavior. In the remote operation mode, the AV can be remotely operated by a human operator present at the control center. In one embodiment, the driver (i.e., the occupant physically inside the vehicle) can be prompted to approve the remote operation mode. In another embodiment, the remote operation mode can be activated without driver confirmation and can even be executed without the occupant(s) realizing that the AV is not in an automatic mode.

[0110] When a human operator from a control center controls an AV, the operator can directly influence the AV system in real time and know the readings of the AV sensors in real time.

[0111] The TOD session manager 506 is configured to establish a TOD session with the AV and provide the AV with information to be used by the AV as data for fields 404 , 405 , 406 , and 407 of its TOD message 401 .

[0112] A fleet management module 502 may optionally be present and is configured to manage a group of AVs that are associated with each other and share data messages between them. In particular, such a module may enable management of platoons of vehicles. A platoon is a group of vehicles that can safely travel together in close proximity. Each vehicle communicates with the other vehicles in the platoon, with the lead vehicle controlling the speed and direction of movement, and all follower vehicles responding to the movement of the lead vehicle. The fleet management module may implement features related to jointly managing such a platoon or fleet, such as during a TOD session.

[0113] Figure 6A diagram of a control center method according to the present invention is presented. The method begins at step 601 by receiving a request from an AV to establish a TOD session. Next, at step 602, the process identifies the control center and accepts the request 603 for the TOD, thereby establishing a communication session with the AV in order to perform the TOD session.

[0114] Then, at step 604, the method waits for the human operator to identify the cause of the TOD so that it can be reported 605 to the AV. In other words, the system waits for identification of the cause of the teleoperated driving session from the control center. Similarly, at step 606, the expected TOD session duration can be determined and reported to the AV.

[0115] At step 607, the control center remotely controls the AV in order to avoid unexpected or unclear traffic conditions and reach a state where the AV can continue autonomous driving. As already defined, remote control may require explicit authorization from the driver to activate without the driver's confirmation. In another embodiment, this step is optional, as a TOD session may be established only to obtain information from the AV without actually controlling the AV.

[0116] Remote control operation is based on data captured by the AV's sensors 103 and requires specific commands to be sent to the AV. However, as will be appreciated by those skilled in the art, such commands are beyond the scope of the present invention.

[0117] During teleoperation, the expected TOD session duration may be determined again and reported to the AV as an update.

[0118] When the operator of the control center completes the remote control operation, at step 608, the TOD session may be ended.

[0119] Similarly, as in Figure 2-3 In the case of processing, those skilled in the art can easily recognize that the aforementioned method for remote control driving can be implemented and / or controlled by one or more computer programs.

[0120] Figure 7 An example of a traffic situation 700 requiring a TOD session is shown. A first vehicle 703 encounters a traffic situation where a truck 702 is blocking a one-way street. Using the communication infrastructure 701, the first vehicle may request 705 a TOD session from a command center 706.

[0121] Such a session may be established in order to solve the problem by teleoperating driving. During a TOD session, a second vehicle 704 may be informed that a TOD session is ongoing via a TOD (broadcast) message 401 of the first vehicle 703. The results of such a TOD session may also be broadcast 401 by the first vehicle 703 to other vehicles in the vicinity 704.

[0122] As an example, proximity or proximity may be defined depending on the road type or location. In urban areas it may be up to 500m or up to 1km, and in case of highways the vicinity may extend up to several kilometers (e.g. up to 2km or up to 3km).

[0123] In view of the foregoing detailed description, received TOD messages 401 may be used by other vehicles 704 (or other recipients, generally human or non-human, vehicular or non-vehicular, such as pedestrians' smartphones, smart watches, etc.) to facilitate advance awareness of problematic traffic conditions.

[0124] As more and more modern vehicles use V2V and V2X communications, the number of users of these communication protocols is increasing. Due to the number of users and increasingly complex applications, the amount of data transmitted will continue to increase. However, due to limited bandwidth and data rates, the increase in data can lead to bottlenecks.

[0125] However, particularly with regard to autonomous driving, the QoS of the radio link may be directly related to the safety of the automated process and, therefore, the safety of the driver. Therefore, limiting the data throughput via the radio link is one of the challenges of AV communications such as V2V and V2X.

[0126] Due to aspects of the present invention, a situation is avoided where many AVs are calling (ie establishing a TOD session) the control center to resolve the same problem (which is being or has been resolved or otherwise solved).

[0127] Another benefit of the present invention is to avoid initializing a vehicle's own TOD session when the solution can be shared between local AVs. Therefore, the number of TOD sessions can be reduced.

[0128] Reference Mark

[0129] 101 Data Bus

[0130] 102 Autonomous Driving Module

[0131] 103 Sensors

[0132] 104 Memory

[0133] 105 Controller

[0134] 106 TOD Session Manager

[0135] 107 Communication Module

[0136] 108 Module for communicating with the user

[0137] 201–207, 301–306 Methods and Steps

[0138] 401–410 message content

[0139] 501 Data Bus

[0140] 502 Fleet Management Module

[0141] 503 remote control module

[0142] 504 Memory

[0143] 505 Controller

[0144] 506 TOD Session Manager

[0145] 507 communication module

[0146] 508 module for communicating with the driver

[0147] 601–608 Method Steps

[0148] 700 Traffic Conditions

[0149] 701 Communications Infrastructure

[0150] 702 Truck

[0151] 703 First Vehicle

[0152] 704 Second Vehicle

[0153] 705 TOD session request

[0154] 706 Command Center.

Claims

1. A method for operating an automated vehicle (AV) (703), the method comprising the following steps: - determining (201): whether remote controlled driving of the AV (703), i.e., TOD operation, is required in order to operate the AV (703) under the current traffic conditions; and - in response to determining that remote control operation of the AV is required to operate the AV under the current traffic conditions, driving the TOD operation, - before actually establishing a TOD session with the control center, transmitting (204) a TOD message to at least one other receiver (704) in the vicinity, the at least one other receiver (704) not being the control center (706) configured to control the AV (703) requiring a TOD operation, the TOD message comprising: information (402B) that the AV (703) requires a TOD operation and information (403) about a current traffic condition as a reason for the required TOD session analyzed by the vehicle itself, wherein determining whether remote control operation of the AV is required in order to operate the AV under the current traffic condition. Driving a TOD operation comprises: - determining (201) traffic conditions in the vicinity of the AV (703) based on at least one detected signal representing the environment in the vicinity of the AV (703); - determining whether the operation of the AV (703) by the autonomous driving module (102) of the AV (703) is prohibited under the determined traffic condition, and Wherein TOD session information is extracted from a TOD message received by the AV, wherein the TOD session information is used to establish a TOD session for the AV, or to adjust driving of the AV based on the extracted information.

2. The method according to claim 1, further comprising the steps of: - based on determining (201) that a TOD operation is required, transmitting (202) a TOD request message to a control center (706) for requesting a TOD operation, - wherein the TOD request message includes information about current traffic conditions (403).

3. The method according to claim 2, further comprising the steps of: - Initiating (202) a TOD session (203) with a control center (706); - Allow (205) the control center (706) to control the AV (703); - receiving (206) further data on current traffic conditions from a control centre (706); and - Based on said further data, updating (207) the TOD message (401) transmitted by the AV (703) to other receivers.

4. The method of claim 3, wherein updating (207) the TOD message (401) comprises adding information (404) about at least one maneuver that allows the AV (703) to reach a state in which the operation of the AV (703) by the autonomous driving module (102) of the AV (703) is feasible.

5. The method according to claim 4, wherein the information (404) about at least one maneuver is inferred from driving of the vehicle under the control of the control center (706).

6. Method according to claim 4 or 5, wherein the information (404) about at least one maneuver is derived from further data received from a control center (706).

7. The method according to claim 4 or 5, wherein the information (404) about at least one maneuver is a driving path.

8. The method according to claim 1, wherein the TOD message (401) comprises: - Timestamp (408); - an identifier (409) of the AV (703); as well as - The geographic location (410) of the AV (703).

9. The method according to claim 3, wherein the TOD message (401) further comprises: - Information (402A) indicating whether the AV (703) is in TOD operation; - an identifier of a control centre (406) obtained from the further data; and / or - The expected duration of the TOD session obtained from further data (407).

10. The method according to claim 2, wherein before transmitting (202) the TOD request message, the method performs the following steps: - receiving (301) a TOD message (401) from another AV (704); - extracting (303) information about another AV's (704) TOD session from the TOD message (401); and - using the extracted information when establishing (304) a TOD session, or -Adjusting the driving of the AV (703) based on the extracted information.

11. A method for operating an automated vehicle (AV) (703), the method comprising the steps of: - determining whether a remote control operation of another AV (704) for driving a TOD operation is required by receiving (301) a TOD message (401) from another AV (704); as well as - in response to the determination, adapting autonomous driving operation of the AV (703) and / or outputting a signal to an occupant of the AV (703), wherein determining whether a TOD operation of the other AV is required is based on a TOD message from the other AV, wherein the TOD message indicates that a TOD operation is required in order to operate the other AV under current traffic conditions, Wherein, based on the traffic condition near the AV determined based on at least one detected environmental signal representing the vicinity of the AV, it is determined whether TOD operation of the other AV is required for the current traffic condition, and it is determined whether the operation of the AV by the AV's automatic driving module is prohibited under the determined traffic condition near the AV, and wherein TOD session information is extracted from the TOD message received by the AV, wherein the TOD session information is used to establish a TOD session for the AV, or to adjust the driving of the AV based on the extracted information.

12. The method according to claim 11, further comprising the steps of: - extracting (303) information about another AV's (704) TOD session from the TOD message (401); - Utilize the extracted information when establishing (304) a TOD session and / or when adapting the autonomous driving operation of the AV (703).

13. A computer readable medium storing computer executable instructions which, when executed on a computer, perform all the steps of the computer implemented method according to claim 1.

14. An automated vehicle (AV) (703) for remotely operating a TOD, the AV (703) comprising: - a communication module (107) configured for bidirectional communication with a control center (706); - an autonomous driving module (102) configured to control the driving of the AV (703); - a TOD session manager (106) configured to determine that operating the AV (703) under current traffic conditions requires (201) a remotely operated driving TOD operation of the AV (703); and - A controller (105) configured to perform the method according to claim 1.

Images