Teleoperation of a remote vehicle after an accident

Abstract

Disclosed is a computer-implemented method for teleoperation of a remote vehicle (100) post-accident via a vehicle teleoperation cloud (104). The process starts when the cloud (104) receives a request for teleoperation of the vehicle (100) involved in an accident. It uses a first real-time wireless connection with a radio access network (102). This connection allows the cloud (104) to receive a vehicle and occupant damage assessment (206). Based on this assessment, the cloud (104) determines the necessary level of teleoperation (208), specifying which technical systems of the vehicle (100) need control. These systems are then controlled in real-time via the connection (210).

Description

2022PF02806 1 VALEO.224.09EPTELEOPERATION OF A REMOTE VEHICLE AFTER AN ACCIDENTFIELD OF THE INVENTION

[0001] The invention relates to the field of teleoperation of a remote vehicle by a vehicle teleoperation cloud after an accident of the remote vehicle.BACKGROUND

[0002] The present invention relates to the field of teleoperation methods and systems for managing and controlling remote vehicles in the event of an accident using a vehicle teleoperation cloud. This involves real-time communication and control strategies to address immediate aftermath scenarios and ensure safety and functionality of the involved vehicle and its occupants.

[0003] The integration of teleoperation in vehicular technology has significantly advanced with the development of more reliable and faster communication technologies. Teleoperation refers to the remote control of a machine or vehicle where the operator is not physically present. Traditionally, these systems have been employed in hazardous environments or situations where human presence is impractical or dangerous.SUMMARY OF THE INVENTION

[0004] It is an objective to provide for a system and method for teleoperation of a remote vehicle by a vehicle teleoperation cloud after an accident of the remote vehicle. This objective is reached by the features of the independent claims.

[0005] In one aspect, a method for teleoperation of a remote vehicle by a vehicle teleoperation cloud after an accident of the remote vehicle is disclosed. A request for teleoperation of the remote vehicle involved in the accident is received by the vehicle teleoperation cloud via a first real-time wireless communication connection or a second network connection. In case the first real-time wireless communication connection is not established and the request for teleoperation is received via the second network connection, the first real-time wireless2022PF02806 2 VALEO.224.09EP communication connection between the vehicle teleoperation cloud and the remote vehicle is established. The first real-time wireless communication connection is used, in response to the request, in order to receive a vehicle and occupant damage assessment by the vehicle teleoperation cloud. A required level of teleoperation based on the vehicle and occupant damage assessment, the required level of teleoperation defining at least one technical system of a plurality of technical systems of the remote vehicle that is to be teleoperated, is determined by the vehicle teleoperation cloud. The at least one technical system of the remote vehicle is controlled in real-time by the vehicle teleoperation cloud via the first real-time wireless communication connection.

[0006] For example, teleoperation of a remote vehicle comprises control of a vehicle from a distance. For example, teleoperation comprises controlling the vehicle by an operator not physically present inside the remote vehicle. This control may be achieved through wireless communications and real-time data exchange between the remote vehicle and the vehicle teleoperation cloud. For example, the operator may be in communication with the vehicle teleoperation cloud in order to control the remote vehicle. The operator may be connected to the vehicle teleoperation cloud via a wireless or wired network connection. The operator may be located at a different or the same place as the vehicle teleoperation cloud.

[0007] For example, a vehicle teleoperation cloud comprises a centralized computer system that facilitates the remote control and management the remote vehicle through cloud computing technologies. For example, the centralized computer system comprises a single computer device or multiple computer devices. For example, the multiple computer devices are multiple data centers. For example, the multiple data centers are geographically separated and connected via a network connection. For example, the network connection is a wired and / or wireless network connection. This platform may integrate various teleoperation systems, data analytics, and communication technologies to enable efficient, real-time operation and monitoring of remote vehicles from a distance.

[0008] For example, the wireless communication connection comprises a link established and / or used between the vehicle teleoperation cloud and the remote vehicle.

[0009] For example, an accident comprises any event involving the remote vehicle that results in damage or harm to the remote vehicle itself, its occupants, other vehicles, pedestrians, and / or other objects or persons. For example, the remote vehicle comprises gasoline vehicles, diesel vehicles, electric vehicles, hybrid vehicles, hydrogen fuel cell vehicles. For example, the remote2022PF02806 3 VALEO.224.09EP vehicle is a car, a bicycle, a truck, a bus, a boat, a vessel, an airplane, a recreational vehicle, an emergency vehicle, a construction vehicle, an agricultural vehicle, an industrial vehicle, a specialpurpose vehicle, a marine vehicle, an aerial vehicle, or a motorized cycle, in particular a motorcycle, a scooter, a moped or a motorized tricycle. For example, the remote vehicle is a manually driven vehicle, an assisted driving vehicle, a semi-autonomous vehicle, or a fully autonomous vehicle.

[0010] For example, an occupant of a vehicle is any person, animal and / or living being who is inside or on a vehicle during its operation. This may include the driver and any passengers who are e.g., riding in the vehicle at the time. Occupants could be within any type of vehicle, such as cars, buses, trucks, motorcycles, bicycles, trains, boats, or aircraft. The term may encompass anyone or any being that is legitimately transported by the vehicle, regardless of whether they are e.g., seated, standing, or lying, and whether they are e.g., using safety devices like seat belts or helmets.

[0011] For example, real-time refers to the processing of a request at the time of occurrence. For example, the delay between the request and processing of the request is minimal. For example, the delay is lower than a threshold. For example, the threshold is 20 ms, in particular 10 ms, in particular 5 ms. For example, real-time operation means that a system handles data, performs computations, or controls processes as quickly as needed for the task to function effectively and securely within the specific context of teleoperation of a remote vehicle. For example, the computer-implemented method for teleoperation of a remote vehicle is automatically performed.

[0012] An advantage could be that the method could enable an immediate response to accidents by e.g., using a real-time wireless communication immediately after an accident. This rapid connectivity may ensure that any necessary actions to secure the vehicle and assist the occupants may be initiated without delay, potentially saving lives and preventing further damage to the remote vehicle, its occupants and other individuals and objects in or around the remote vehicle. By receiving a detailed assessment of vehicle and occupant damage, the system could determine the specific level and type of teleoperation required. This means that the response may be customized based on the actual conditions of the vehicle and the needs of its occupants, rather than a one-size-fits-all approach, leading to e.g., a more effective management of the situation.2022PF02806 4 VALEO.224.09EP

[0013] Furthermore, the ability to control critical technical systems of the vehicle remotely could allow for various safety and corrective measures to be applied as needed. For instance, if the vehicle poses a hazard in its current location, it could be moved; if its lights need to be activated for visibility, or if specific systems need to be shut down to prevent further damage, these actions could be taken remotely. In addition, the method may enhance the safety of both the occupants and the responders. By enabling remote assessment and control, it reduces the need for immediate physical presence, which could be dangerous in unstable or hazardous accident scenarios.

[0014] The use of a real-time wireless communication connection (e.g., employing technologies like 5G) may ensure a higher transfer speed and lower latency as compared to previous standards (such as 4G or 3G). This may be critical for transmitting large amounts of data for controlling the vehicle rapidly and with minimal delay, thereby potentially increasing the effectiveness and reliability of the teleoperation.

[0015] For example, transfer speed may be the bandwidth in terms of the capacity or data rate of a mobile connection and / or the maximum capacity or data rate of the mobile connection. This may refer to the highest amount of data that could be transmitted from one point to another in a given amount of time, for example measured in bits per second (bps). High bandwidth may indicate a greater capacity for data transfer, which may be essential for applications requiring real-time communication and high-speed data transmission.

[0016] The incorporation of a vehicle and occupant damage assessment into the decision process could allow for data-driven management. This not only enhances the appropriateness of responses but also helps in prioritizing actions based on severity and immediate needs, thereby optimizing resource utilization and response effectiveness.

[0017] For example, the second network connection is a network connection between the remote vehicle and the vehicle teleoperation cloud configured to allow data flow from the remote vehicle to the vehicle teleoperation cloud. For example, the first real-time wireless communication connection is a connection through a network. The network is any one of a) a radio access network, RAN, in accordance with a radio access technology, RAT, communication protocol, comprising at least network slicing or another radio access technology in particular with a transfer speed of at least 2 Gbps and / or in particular with a latency of less than 15 ms; or b) a satellite network; or c) a vehicle 2 infrastructure, V2I, network.2022PF02806 5 VALEO.224.09EP

[0018] The second network connection may or may not be a real-time connection. It might encompass a variety of wireless and / or wired technologies. For example, the second network connection is a 2G connection, a 3G connection, a 4G connection, a 5G connection, an ultra-wideband, UWB, connection, a vehicle 2 vehicle, V2V, connection, a vehicle 2 infrastructure, V2I, connection, a wi-fi connection, a Bluetooth connection , a Zigbee connection, a Long Range (LoRa) connection, a Narrowband loT (NB-loT) connection, a satellite communication link, a microwave link, a mesh network connection, a Li-Fi connection, a dedicated short-range communications (DSRC) connection, a private LTE connection, a WiMAX connection, a Z-Wave connection, a Sigfox connection, a cellular loT connection, an infrared (IR) communication link, a near-field communication (NFC) link, a radio-frequency identification (RFID) connection, a fiber optic link, a coaxial cable connection, an Ethernet connection, a Power Line Communication (PLC) link, a sub-1 GHz wireless connection, a wired connection.

[0019] For example, the radio access network, RAN, is part of a mobile telecommunication system. It may implement a radio access technology and facilitates wireless communication between mobile devices and core network components. The RAN may handle the communication between the vehicle teleoperation cloud and the remote vehicle.

[0020] For example, radio access technology, RAT, comprises underlying technology used for maintaining wireless communications. It may comprise New Radio, NR, used in 5G networks. For example, the first real-time wireless communication connection with the radio access network is in accordance with the 5G standard or any standard beyond 5G. For example, the RAT comprises any radio access technology beyond a New Radio, NR, technology that enables the real-time wireless connection.

[0021] For example, latency is a time it takes for a packet of data to travel from a source to a destination back again to the source. For example, the source is either the remote vehicle or the teleoperation cloud, and the destination is the respective other of the two.

[0022] For example, network slicing allows for the creation of virtualized and independent networks on the same physical infrastructure.

[0023] For example, a satellite network could involve a communication system where data is transmitted between the remote vehicle and the teleoperation cloud via satellites orbiting the earth. This type of network might use geostationary satellites, which could remain in a fixed position relative to the earth's surface, e.g., providing broad coverage areas but with higher latency. Alternatively or additionally, low earth orbit (LEO) satellites might be used, which may be2022PF02806 6 VALEO.224.09EP closer to the earth and may offer lower latency. They may require a constellation of satellites to e.g., provide continuous coverage due to their fast movement across the sky.

[0024] For example, a vehicle-to-infrastructure (V2I) network involves communication between the vehicle and road infrastructure elements. For example, road infrastructure elements comprise traffic lights, road signs, control systems, toll booths, parking infrastructure, emergency response systems, smart streetlights, pedestrian crossing signals, railway crossing gates, public transportation systems, and / or weather monitoring systems. V2I communication might be achieved using e.g., dedicated short-range communications (DSRC) or cellular vehicle-to-everything (C-V2X) technology. These networks may facilitate the exchange of data regarding traffic conditions, road hazards, and other relevant information, e.g., enhancing the vehicle's ability to operate safely and efficiently in various environments.

[0025] The second network connection may provide several advantages, especially considering it may not always be a real-time wireless connection. For example, utilizing a second network connection might allow for intermittent data transfer, which could significantly save on data usage and energy consumption. This may be particularly beneficial in scenarios where constant real-time communication may not be necessary. For example, the vehicle teleoperation cloud could rely on the second network connection for data exchanges that does not require the realtime wireless connection, such as receiving the request for teleoperation.

[0026] Additionally, by e.g., employing a non-real-time connection, this could enable the system to maintain a baseline level of connectivity without e.g., the high costs associated with continuous real-time communication. This might be crucial in preserving battery life, especially in remote or energy-constrained environments.

[0027] Moreover, the flexibility of e.g., switching between different types of network connections based on the situational needs could optimize overall system performance. For example, the vehicle teleoperation cloud might utilize the second network connection for receiving teleoperation requests during normal operation and switch to the real-time wireless connection only when an urgent teleoperation command is required. This dynamic approach could enhance the efficiency and longevity of the vehicle's communication systems.

[0028] For example, by not having to establish a real-time wireless connection at all times, the system could reduce operational costs and extend the operational life of both the vehicle's communication hardware and the vehicle teleoperation cloud's resources. This adaptability might also e.g., allow the system to function effectively in a wider range of environments, from e.g.,2022PF02806 7 VALEO.224.09EP urban areas with dense network infrastructure to e.g., remote rural locations where network availability may be sparse and intermittent.

[0029] An advantage could be that by using a system with a RAT protocol comprising at least network slicing or another radio access technology with a transfer speed of at least 2 Gbps, the system could handle data-intensive applications such as teleoperation of a remote-vehicle and every associated aspect. This may be crucial for remote vehicle operations, which may require the transmission of high-definition video, real-time sensor data, and other large data packets that e.g., facilitate detailed remote monitoring and real-time control. The high transfer speed may ensure that large amounts of data could be transmitted quickly, allowing for more complex commands and richer data feeds, which in turn may enable more precise and effective control of the remote vehicle.

[0030] An advantage could be that by using a system with a RAT protocol comprising at least network slicing or another radio access technology with a latency of less than 15 ms, real-time applications may be possible, as the level of responsiveness could be nearly instantaneous from a human perspective. This may be essential for applications where timing is critical, such as e.g., active vehicle control, emergency responses, and accident handling. In this context, low latency could mean that commands from the teleoperation cloud to the vehicle (and vice versa) could be executed with minimal delay, greatly reducing e.g., the risk of accidents caused by delayed reactions, especially in fast-moving environments.

[0031] An advantage could be that by utilizing network slicing for remote vehicle operations, a dedicated network slice could be configured with the necessary transfer speed and latency characteristics, optimized for safety-critical communications. By using a separate slice of the network, the data and control commands may be isolated from other non-critical data traffic, which enhances security and reduces the risk of interference or eavesdropping. The system may scale efficiently with the demand, e.g., as more vehicles require teleoperation, additional network slices may be created, or transfer speed allocations could be adjusted to accommodate this growth. Furthermore, the flexibility of modern RATs like 5G may allow for deployment in diverse environments, from densely populated urban areas to remote rural locations, ensuring e.g., consistent and reliable performance.

[0032] For example, using a satellite network as the first real-time wireless communication connection could offer significant advantages. For example, satellite networks provide extensive coverage, ensuring connectivity in remote or rural areas where terrestrial networks might not be2022PF02806 8 VALEO.224.09EP available. This wide coverage could be crucial for maintaining communication with the vehicle teleoperation cloud e.g., regardless of the vehicle's location. Additionally, satellite networks might be less prone to local interference, by e.g., providing a more stable and reliable connection for critical data transmission.

[0033] Similarly, employing a vehicle-to-infrastructure (V2I) network could offer distinct benefits. V2I networks could enhance urban connectivity by integrating vehicles with city infrastructure. For example, by facilitating direct communication between vehicles and infrastructure, V2I networks could support advanced teleoperation capabilities, ensuring vehicles remain connected and responsive to changing conditions in urban environments.

[0034] For example, the vehicle and occupant damage assessment comprises vital data of one or more occupants of the remote vehicle and vehicle data being descriptive of a physical status of the remote vehicle and its surrounding environment.

[0035] For example, a driver's and / or occupant's medical condition may be assessed based on the vital data of the one or more occupants of the remote vehicle. For example, teleoperation of the remote vehicle may be initiated based on the driver's and / or occupant's medical condition. For example, the driver's medical condition may render the driver unable to drive the remote vehicle. For example, the occupant's medical condition requires the driver and / or any other occupant of the remote vehicle to deliver first aid to the occupant with the medical condition.

[0036] For example, the medical condition comprises a heart attack, a stroke, an epileptic seizure, a severe allergic reaction (anaphylaxis), a diabetic shock, a hypoglycemia, a respiratory distress, an asthma attack, a severe bleeding, a hemorrhage, a traumatic brain injury, a loss of consciousness, a fainting, a severe dehydration, a heat stroke, a poisoning, a toxic exposure, a severe anxiety attack, a panic attack, a choking, a cardiac arrest, a broken bone, a severe fracture, a severe burn, an acute appendicitis, a high fever with convulsions, a severe infection, a sepsis, an internal bleeding, a complication from an existing medical condition, a kidney failure, an acute myocardial infarction, a hypertensive crisis, an acute pancreatitis, a gallbladder attack, a pulmonary embolism, a deep vein thrombosis, a meningitis, an encephalitis, an acute urinary retention, an acute glaucoma, a gastrointestinal bleeding, a ruptured aneurysm, a spinal cord injury, a severe hypothermia, a labor complication, a delivery complication, a mental health crisis, a psychotic episode, a drug overdose, an alcohol poisoning, an anaphylactic shock, a septic shock, an acute renal failure, a severe migraine, an acute laryngospasm, a severe frostbite, a hyperthermia, a severe bradycardia, a tachycardia, a severe anemia, an acute pericarditis, a2022PF02806 9 VALEO.224.09EP myocarditis, an acute respiratory distress syndrome (ARDS), an acute esophageal varices, an acute liver failure, a peritonitis, an acute intestinal obstruction, an acute mesenteric ischemia, an acute gout attack, an acute skin reaction, in particular Stevens-Johnson syndrome, a toxic epidermal necrolysis.

[0037] An advantage could be that by combining occupant vital data with vehicle status data may provide a holistic view of the accident's impact, allowing for e.g., a more comprehensive understanding of the impact of the accident on the remote vehicle and / or on the occupants. This integrated data approach may aid in assessing the severity and specific nature of the incident, and may facilitate more tailored and effective responses. With access to occupants' vital data, emergency responders could prioritize medical attention based on the severity of occupants' conditions. This may be life-saving in critical situations. Furthermore, information on the vehicle's structural integrity, operational systems, and environmental risks could enable operators to make informed decisions about how to move the vehicle safely or what emergency measures to activate, such as fuel cut-offs or unlocking doors.

[0038] An advantage in having real-time access to vital data could be the immediate availability of detailed vital data, which may ensure that the response is both timely and adjusted to the current needs, potentially reducing the risk of aggravating injuries or causing additional accidents. In case e.g., responders have to be called to the scene, knowing the exact scenario before arriving at the scene would prepare responders better, e.g., equipping them with the right tools and safety measures, thus safeguarding their own safety. Furthermore, the right tools and vehicles may be deployed to the scene.

[0039] For example, the vital data comprises information on at least any one of the following: heart rate, blood pressure, respiratory rate, skin temperature, physical body damage, mood, level of distress, oxygen saturation, level of fatigue, level of stress, pupil diameter, contactless electroencephalogram, contactless electrocardiogram, heart rate variability, body motion, facial expressions, voice analysis.

[0040] For example, the vital data is captured using a vital data sensor array of the remote vehicle, the vital data sensor array comprising sensors.

[0041] For example, heart rate comprises the number of times the heart beats per minute. For example, blood pressure comprises a measurement of the force of blood against artery walls. For example, respiratory rate comprises the number of breaths a person takes per minute. For example, skin temperature comprises the external body temperature. For example, physical body2022PF02806 10 VALEO.224.09EP damage comprises any physical injuries or trauma sustained by a person's body. For example, mood comprises an assessment of behavioral cues. For example, level of distress comprises an assessment of the degree of discomfort or anxiety a person is experiencing. For example, oxygen saturation comprises the percentage of oxygen in the blood. For example, level of fatigue comprises an assessment of how tired a person is.

[0042] For example, level of stress comprises an assessment of an emotional and physical response to pressures or threats. For example, pupil diameter comprises a measurement of a pupil diameter of a person. For example, contactless electroencephalogram comprises a measurement of brain waves through sensors placed in the vicinity of a head of a person. For example, contactless electrocardiogram comprises a measurement of a heart's electrical activity from a distance. For example, heart rate variability comprises a measurement of the variation in time between each heartbeat. For example, body motion comprises an assessment of movement patterns of a person. For example, facial expressions comprise an assessment of facial expressions of a person. For example, voice analysis comprises an assessment of vocal characteristics of a person.

[0043] For example, the vehicle data comprises information on at least any one of the following: vehicle orientation, fire detection, smoke detection, broken glass detection, airbag status, car body deformation detection, air quality, humidity levels, light exposure, ambient noise levels, environmental temperature, oil level, fuel level, battery status, engine temperature, engine status, exhaust emissions, suspension health, historical drive data, tire pressure.

[0044] For example, the vehicle data is captured using a vehicle data sensor array of the remote vehicle, the vehicle data sensor array comprising sensors.

[0045] For example, vehicle orientation comprises an indicator of the position of the remote vehicle, in particular whether the remote vehicle is upright, on a side of the remote vehicle or tilted. For example, fire detection comprises information of a presence or absence of heat and / or flames. For example, smoke detection comprises information of a presence or absence of smoke. For example, broken glass detection comprises information on glass breakage. For example, airbag status comprises information on whether airbags have deployed. For example, car body deformation detection comprises indicators of areas of the remote vehicle that have been damaged, dented, deformed and / or crushed. For example, air quality comprises information on air pollutants, gas concentrations, in particular concentrations of oxygen, carbon monoxide, carbon dioxide, hydrogen cyanide, nitrogen oxides, NOx, sulfur dioxide, formaldehyde and / or2022PF02806 11 VALEO.224.09EP acrolein, inside the remote vehicle. For example, humidity levels comprise information on humidity levels in the air inside and / or outside the vehicle. For example, light exposure comprises information on an intensity of light inside and / or outside of the remote vehicle.

[0046] For example, ambient noise levels comprise information on a sound environment inside and / or outside the vehicle. For example, environmental temperature sensors comprise information on temperature inside and / or outside the vehicle. For example, oil level comprises information on the oil level of the car. For example, fuel level comprises information on the fuel level of the car. For example, battery status comprises an indication of a charge level, a current draw, a voltage level, and a maximum voltage level of a battery, in particular of a battery of an electric vehicle. For example, engine temperature comprises information on current and past engine temperature of the remote vehicle. For example, engine status comprises information on the current operating status of an engine of the remote vehicle, in particular information on whether the engine is still running or not. For example, exhaust emissions comprise information on a presence of exhaust gases and their composition. For example, suspension health assessment comprises information on a deformation and / or damage to a suspension of the remote vehicle. For example, historical drive data comprises data on past vehicle usage, in particular the most recent velocity and usage of brakes, wherein most recent comprises a time span of 10 minutes, in particular 5 minutes. For example, the tire pressure comprises data on the current and past tire pressure.

[0047] For example, the vehicle teleoperation cloud further comprises at least one trained machine learning module. The at least one machine learning module is configured to determine the at least one technical system as output in response to receiving the vehicle and occupant damage assessment as input.

[0048] In an example, a machine learning module to be trained is provided. Training datasets for training the machine learning module to be trained are provided, each training dataset comprising a training vehicle and occupant damage assessment. The machine learning module to be trained is trained using the training datasets.

[0049] An advantage could be that the machine learning module may be comprised by the vehicle teleoperation cloud and thus has higher processing capabilities as if it would be comprised by the remote vehicle itself. In addition, when the machine learning module may be comprised by the vehicle teleoperation cloud, this may allow for the data e.g., generated by it to be directly available to the teleoperator controlling the teleoperation of the remote vehicle with2022PF02806 12 VALEO.224.09EP minimal delay. By outsourcing the machine learning module to the vehicle teleoperation cloud and connecting the remote vehicle via the first real-time wireless communication connection with the cloud, the enhanced processing capabilities may also be utilized in real-time within the remote vehicle. By hosting the machine learning module within the teleoperation cloud, which could have access to superior computational resources compared to those on the remote vehicle, more complex algorithms and data processing tasks may be executed. This might allow for more sophisticated analysis and faster decision-making.

[0050] The cloud-based architecture could enable scalability, allowing processing capabilities to be dynamically adjusted based on the current needs. This flexibility may be critical during peak times or when multiple vehicles require simultaneous assistance. Furthermore, offloading the machine learning processes to the cloud could reduce the computational burden on the vehicle's own systems, potentially leading to lower power consumption and less wear on the vehicle's hardware. This may also free up onboard resources for other critical functions.

[0051] Even though the processing is done remotely, the use of a real-time wireless communication connection may ensure that data analysis and resultant actions are relayed back to the vehicle swiftly in real-time. This setup could enable immediate responses to changing conditions or emergencies. In addition, with the machine learning module based in the cloud, it could continuously learn and improve from data gathered across a fleet of vehicles, rather than just a single vehicle. This may enhance the accuracy and effectiveness of the system over time, benefiting from a larger dataset for training and optimization.

[0052] Hosting the machine learning module in the cloud could offer better control over data security and management practices, as cloud services often provide robust security and privacy measures and regular updates. This arrangement may also facilitate compliance with data protection and privacy regulations more efficiently than individual vehicle-based systems.Eventually, by centralizing the machine learning capabilities in the cloud, costs related to hardware and maintenance on each vehicle could be minimized. This centralized approach may also facilitate more economical scaling of advanced capabilities across an entire fleet without the need for significant hardware upgrades in each vehicle.

[0053] For example, the plurality of technical systems comprises at least any one of the following systems: ignition system, steering system, acceleration system, braking system, gear selection system, lighting system, camera system, sensor system, emergency and safety system, infotainment system, diagnostic system, climate control system, power management system, in2022PF02806 13 VALEO.224.09EP particular a power management system of an electric vehicle, navigation system, Vehicle-to-Vehicle, V2V, communication system, Vehicle-to-lnfrastructure, V2I, communication system, autonomous driving system, exhaust management system, suspension system, security and antitheft system, active aerodynamics system, electric window system, door system, trunk lid and engine compartment system.

[0054] For example, the ignition system controls the remote vehicle's engine operation, wherein the remote vehicle's engine operation comprises process that starts the combustion necessary to power the vehicle. For example, the steering system controls the direction of travel of the remote vehicle through mechanical or electronic means. For example, the acceleration system controls the rate at which the vehicle increases its speed. For example, the braking system controls the rate at which the vehicle decreasing its speed, in particular by applying resistance mechanically or electronically to the wheels. For example, the gear selection system controls the change of gears in the vehicle. For example, the lighting system controls all the vehicle's interior and exterior lights. For example, the camera system comprises multiple cameras placed inside and / or outside the vehicle. For example, the sensor system comprises various sensors that detect and respond to physical inputs from the environment.

[0055] For example, the emergency and safety system controls airbags, other safety measures and / or crash sensors. For example, the infotainment system controls information delivery via audiovisual means, wherein the audiovisual means comprise speakers and / or displays. For example, the diagnostic system comprises means for onboard diagnostics. For example, the climate control system controls airflow, temperature and / or humidity inside the vehicle's cabin. For example, the power management system, particularly in an electric vehicle, controls distribution and usage of electrical power throughout the vehicle. For example, the navigation system controls GPS-based directions and route planning. For example, the Vehicle-to-Vehicle (V2V) communication system controls communication of vehicles with each other. For example, the Vehicle-to-lnfrastructure (V2I) communication system controls communication of the remote vehicle with road infrastructure, in particular with traffic lights and road signs.

[0056] For example, the autonomous driving system controls the vehicle without human input. For example, the exhaust management system controls the emissions released by the vehicle. For example, the suspension system controls the vehicle's ride height and damping. For example, the security and anti-theft system controls the remote vehicle's alarms, immobilizers, and / or tracking systems. For example, the active aerodynamics system controls aerodynamic components, in particular spoilers and vent grills. For example, the electric window system2022PF02806 14 VALEO.224.09EP controls the opening and closing of electric windows. For example, the door system controls electric opening and closing of doors. For example, the trunk lid and engine compartment system controls electric opening and closing of a trunk lid and / or an engine compartment of the remote vehicle.

[0057] An advantage could be that the extensive range of systems may allow for full-scale automation and control of almost all vehicle functions after an accident. Post-accident, the comprehensive range of control systems could allow a teleoperator to manage vehicle functions remotely, facilitating safe recovery or repositioning of the vehicle without needing immediate onsite human intervention. In the event of an accident, systems such as the emergency and safety systems may quickly activate critical safety features like airbags and hazard lights, enhancing occupant safety and alerting surrounding traffic, all controllable remotely by the teleoperator.

[0058] Another advantage could be that systems such as braking, suspension, and active aerodynamics could be remotely adjusted to stabilize the vehicle post-accident, potentially preventing further damage, or ensuring that the vehicle is safely immobilized until recovery teams arrive. The camera and sensor systems could provide real-time data to the teleoperator about the vehicle's surrounding environment and internal status, potentially allowing for an accurate assessment of the accident scene and enabling informed decision-making regarding necessary responses. With V2V and V2I communication systems, a teleoperator could transmit detailed accident information to nearby vehicles and road infrastructure, potentially alerting emergency services faster and managing traffic around the accident site more effectively.

[0059] The diagnostic system may be utilized by the teleoperator to assess the vehicle's operational state, identify critical issues, and decide whether the vehicle can be driven away or needs to be towed, thus speeding up the recovery process. If occupants are trapped or need to stay within the vehicle post-accident, the climate control system could be managed remotely to maintain a safe and comfortable environment, particularly under extreme weather conditions. The climate control system may also be used to control the air environment during the occurrence of smoke or fire. In addition, post-accident, the security and anti-theft system could be activated to protect the vehicle from looting or unauthorized access, especially if it is left unattended at the accident scene.

[0060] For example, the computer-implemented method further comprises a remote facility in communication with the vehicle teleoperation cloud. The controlling of the computer-implemented method comprises a determining based on the vehicle and occupant damage2022PF02806 15 VALEO.224.09EP assessment, by the vehicle teleoperation cloud, if the required level of teleoperation comprises a real-time presence of a teleoperator. In case it is determined that the real-time presence of the teleoperator is necessary, a real-time virtual-reality stream regarding the interior and exterior of the remote vehicle to the teleoperator is provided, or a request to establish a real-time virtual-reality stream regarding the interior and exterior of the remote vehicle is sent to the teleoperator.

[0061] An advantage could be that tailored response in case the real-time presence of a teleoperator is deemed necessary may be allowed, ensuring that the most suitable actions are taken based on the specific conditions assessed. By utilizing a virtual-reality stream, the teleoperator may receive an immersive, detailed view of both the interior and exterior of the remote vehicle, which could significantly enhance the accuracy of the remote operation and decision-making processes. If it is determined that real-time intervention is required, the prompt establishment of a virtual-reality stream may help in providing immediate assistance, potentially reducing the time to respond to critical situations and enhancing overall safety. Instead of potentially endangering a real person at the site of the accident, appropriate measures could be undertaken by a teleoperator from a distance.

[0062] The ability to either provide a direct real-time virtual-reality stream or send a request to establish such a stream e.g., offers flexibility, ensuring that the teleoperation e.g., adapts dynamically to the evolving situation. The decision to involve a teleoperator could be based on a thorough assessment of both vehicle and occupant conditions, which may lead to more precise and appropriate levels of intervention, tailored to the needs of the situation. The use of virtual reality may engage the teleoperator more effectively, providing them with a sense of presence at the scene, which could improve the handling and outcomes of the teleoperation tasks.

[0063] For example, the virtual reality stream comprises a live camera feed of the interior and exterior of the remote vehicle in multiple camera positions and multiple camera angles, wherein the teleoperator is taking over real-time control of the remote vehicle via means for controlling the plurality of technical systems of the remote vehicle, wherein the virtual reality stream is streamed to a display device, in particular to a virtual reality headset with means for headtracking.

[0064] For example, the live camera feed of the interior and exterior provides a comprehensive and immersive visual representation of the vehicle's interior space and exterior surroundings. The teleoperator could control functions for full control of the vehicle, in particular steering,2022PF02806 16 VALEO.224.09EP braking, acceleration. Means for headtracking may track the movement of the teleoperator's head to adjust the live camera feed of the interior and / or exterior of the remote vehicle accordingly.

[0065] One advantage could be that this setup allows the teleoperator to have a detailed and panoramic view, helping them better understand the current situation and conditions around the vehicle. This might allow for a better understanding of the environment around the vehicle and any situations that may require immediate attention, enhancing the teleoperator's ability to make informed decisions quickly. The means for controlling the plurality of technical systems of the remote vehicle may ensure that the teleoperator can intervene and manage the vehicle's operations as if they were physically present inside the vehicle. With real-time control over essential vehicle functions such as steering, braking, and acceleration, the teleoperator could manage the vehicle as effectively as if they were physically present. This level of control precision could be crucial, especially in emergency scenarios or complex driving conditions, and may lead to safer and more efficient vehicle management.

[0066] The means for headtracking may enhance the immersion and interactivity of the virtual reality experience, potentially making it easier for the teleoperator to navigate and manipulate the vehicle's environment effectively. The integration of headtracking technology in the VR headset could allow the system to adjust the live camera feed based on the teleoperator's head movements. This feature may make the teleoperation experience more intuitive and responsive, closely mimicking the natural interactions of controlling a vehicle directly, thus potentially reducing cognitive load and decreasing the likelihood of operator errors. By providing a VR environment that closely replicates a real vehicle's controls and visibility, the system could increase the teleoperator's engagement and comfort. This might lead to better performance, especially during extended periods of operation, as the teleoperator can interact with the vehicle's environment in a more natural and familiar way.

[0067] The ability to remotely control a vehicle in hazardous situations— such as environments that are unsafe or inaccessible to humans— using a VR system with comprehensive sensory inputs, could significantly reduce the risk to human operators. This could be particularly beneficial in e.g., an emergency response.

[0068] In another aspect, a computer program for teleoperation of a remote vehicle by a vehicle teleoperation cloud after an accident of the remote vehicle is disclosed. The computer program comprises program instructions, the program instructions being executable by a processor of a2022PF02806 17 VALEO.224.09EP computer device to cause the computer device to receive, by the vehicle teleoperation cloud via a first real-time wireless communication connection or a second network connection, a request for teleoperation of the remote vehicle involved in the accident. In case the first real-time wireless communication connection is not established and the request for teleoperation is received via the second network connection, the first real-time wireless communication connection between the vehicle teleoperation cloud and the remote vehicle is established. The first real-time wireless communication connection is used, in response to the request, in order to receive a vehicle and occupant damage assessment by the vehicle teleoperation cloud. A required level of teleoperation based on the vehicle and occupant damage assessment, the required level of teleoperation defining at least one technical system of a plurality of technical systems of the remote vehicle that is to be teleoperated, is determined by the vehicle teleoperation cloud. The at least one technical system of the remote vehicle is controlled in realtime by the vehicle teleoperation cloud via the first real-time wireless communication connection.

[0069] In another aspect, a computer device for teleoperation of a remote vehicle by a vehicle teleoperation cloud after an accident of the remote vehicle is disclosed. The computer device comprises a processor and a memory storing program instructions executable by the processor, execution of the program instructions by the processor causing the computer device to receive, by the vehicle teleoperation cloud via a first real-time wireless communication connection or a second network connection, a request for teleoperation of the remote vehicle involved in the accident. In case the first real-time wireless communication connection is not established and the request for teleoperation is received via the second network connection, the first real-time wireless communication connection between the vehicle teleoperation cloud and the remote vehicle is established. The first real-time wireless communication connection is used, in response to the request, in order to receive a vehicle and occupant damage assessment by the vehicle teleoperation cloud. A required level of teleoperation based on the vehicle and occupant damage assessment, the required level of teleoperation defining at least one technical system of a plurality of technical systems of the remote vehicle that is to be teleoperated, is determined by the vehicle teleoperation cloud. The at least one technical system of the remote vehicle is controlled in real-time by the vehicle teleoperation cloud via the first real-time wireless communication connection.

[0070] It is understood that examples of the aspect of the computer-implemented method for teleoperation of a remote vehicle by a vehicle teleoperation cloud after an accident of the2022PF02806 18 VALEO.224.09EP remote vehicle are also applicable to the other aspects, namely the computer program and / or the computer device for teleoperation of a remote vehicle by a vehicle teleoperation cloud after an accident of the remote vehicle.

[0071] It is understood that one or more of the aforementioned examples may be combined as long as the combined examples are not mutually exclusive.BRIEF DESCRIPTION OF THE DRAWINGS

[0072] In the following, examples are described in greater detail making reference to the drawings in which:

[0073] Fig. 1 is a schematic of an accident comprising a remote vehicle that is communicatively connected to a vehicle teleoperation cloud.

[0074] Fig. 2 is a block diagram of a computer-implemented method for teleoperation of a remote vehicle by a vehicle teleoperation cloud after an accident of the remote vehicle.

[0075] Fig. 3 shows an exemplary computer device for teleoperation of a remote vehicle by a vehicle teleoperation cloud after an accident of the remote vehicle.DETAILED DESCRIPTION

[0076] In the following, similar elements are denoted by the same reference numerals.

[0077] Figure 1 shows a schematic of an accident comprising a remote vehicle 100 that is communicatively connected to a vehicle teleoperation cloud 104 via a first real-time wireless communication connection with a radio access network 102 (RAN).

[0078] The remote vehicle 100 may be involved in an accident with another vehicle 106, which may be on fire. An example of the computer-implemented method will be outlined using the block diagram 200 depicted in figure 2.

[0079] In an example, systems and / or sensors of the remote vehicle 100 determine that the vehicle has been involved in an accident. The vehicle teleoperation cloud 104 receives in block 202, in response to the remote vehicle 100 determining that it has been involved in the accident, a request for teleoperation of the remote vehicle 100. The request for teleoperation may be received via a second network connection. The second network connection may be a wireless connection, which can be, but is not limited to, a wireless communication connection with a2022PF02806 19 VALEO.224.09EP radio access network. Any other means, such as other wireless connections that allow for communication between the remote vehicle 100 and the vehicle teleoperation cloud 104, may also be appropriate. The communication between the remote vehicle 100 and the vehicle teleoperation cloud 104 may also be routed through other stations and / or devices, such as other servers or computer devices, in order to be received as a request from the remote vehicle 100 by the vehicle teleoperation cloud 104.

[0080] Also in block 202, it is determined if a first real-time wireless communication connection is established and if the request has been received via the second network connection. For example, in block 202, the first real-time wireless communication connection is not established and the request for teleoperation is received via the second network connection, and thus, the first real-time wireless communication connection between the vehicle teleoperation cloud and the remote vehicle is established. In the example presented herein, the first real-time wireless communication connection is a first real-time wireless communication connection with a radio access network 102, RAN. It may also be a connection through a network, the network being any one of the radio access network, RAN, as presented before, or a satellite network, or a vehicle 2 infrastructure, V2I, network.

[0081] After the request is received by the vehicle teleoperation cloud 104, the first real-time wireless communication connection, which may be the first real-time wireless communication connection with the radio access network 102, RAN, is used in block 204. The first real-time wireless communication connection with the radio access network 102 may be a 5G mobile network connection, e.g., in accordance with the radio access technology, RAT, communication protocol comprising network slicing. For example, the first real-time wireless communication connection with the radio access network 102 in the example of figure 1 and 2 has a maximum transfer speed of peak data rates up to 20 gigabits per second (Gbps) under ideal conditions and a target latency as low as 1 millisecond (1ms) in ideal conditions. For example, the maximum transfer speed is a theoretical maximum, and actual data rates are influenced by factors such as network congestion, device capabilities, and signal quality. For example, the ultra-low latency is crucial for real-time applications like teleoperation of a remote vehicle. The first real-time wireless communication connection with the radio access network 102 could employ a communication standard advanced beyond 5G, where 'beyond' may indicate subsequent generations or iterations of the technology.

[0082] The first real-time wireless communication connection may be a network connection. The network may be any one of the radio access network, RAN, in accordance with a radio access2022PF02806 20 VALEO.224.09EP technology, RAT, communication protocol comprising at least network slicing or another radio access technology in particular with a transfer speed of at least 2 Gbps and / or in particular with a latency of less than 15 ms, or a satellite network, or a vehicle 2 infrastructure, V2I, network.

[0083] The first real-time wireless communication connection with the radio access network 102 may be used in order to execute the following: a vehicle and occupant damage assessment is received in block 206. The vehicle and occupant damage assessment may be received via the first real-time wireless communication connection with the radio access network 102 from the remote vehicle 100 by the vehicle teleoperation cloud 104. The vehicle and occupant damage assessment could comprise vital data of one or more occupants of the remote vehicle 100 and vehicle data being e.g., descriptive of a physical status of the remote vehicle 100 and its surrounding environment.

[0084] In the example of figure 1, there is only one occupant in the remote vehicle 100, which is the driver of the remote vehicle. The driver may be unconscious due to the accident. The vital data may be specific for the unconscious driver of the remote vehicle 100 and may comprises information on the following.

[0085] The following paragraph only outlines example values for example features: heart rate: An elevated heart rate around 120 beats per minute. Blood pressure: A low blood pressure around 90 / 60 mmHg. Respiratory rate: Irregular or shallow breathing, around 25 breaths per minute. Skin temperature: around 34°C (93.2°F). Physical body damage: Visible physical body damage such as lacerations or bruising. Mood: No observable mood.

[0086] The following paragraph only outlines example values for example features: level of distress: High level of distress, based on physiological readings (heart rate, blood pressure). Oxygen saturation: Low oxygen saturation, around 88%. Level of fatigue: No measurable level of fatigue. Level of stress: High level of stress, inferred from physiological data like elevated heart rate and decreased heart rate variability. Pupil diameter: Dilated pupils. Contactless Electroencephalogram (EEG): Abnormal contactless EEG patterns. Contactless Electrocardiogram (ECG): Irregular heart rhythms in the Contactless Electrocardiogram. Heart Rate Variability (HRV): Reduced heart rate variability. Body Motion: Absent body motion. Facial Expressions: Non-responsive facial expressions. Voice Analysis: No measurable voice.

[0087] In addition, in the vicinity of the remote vehicle 100, there may be another vehicle 106 involved in the accident, which may be on fire right next to the remote vehicle 100. Thus, the vehicle data of the remote vehicle 100 e.g. comprises information on the following:2022PF02806 21 VALEO.224.09EP

[0088] The following paragraph only outlines example values for example features: vehicle orientation: The remote vehicle 100 is skewed and off its normal driving alignment due to the impact at an irregular angle on the road but is still upright on all 4 tires. Fire detection: a fire in the vicinity of the remote vehicle 100 is detected. Smoke Detection: detection of smoke, sensors indicate elevated levels of smoke particles in the air. Broken glass detection: Multiple windows shattered, with glass break sensors activating. Airbag status: Airbags deployed in the front.

[0089] The following paragraph only outlines example values for example features: car body deformation detection: frontal deformation detected with compromise to the vehicle's structural integrity. Air quality: deteriorated air quality inside the vehicle. Humidity levels: elevated humidity inside the cabin, influenced by smoke. Light exposure: varying light exposure. Ambient noise levels: Slightly increased. Environmental temperature: elevated temperatures. Oil level: normal. Fuel level: half full. Battery status: status of 12V lead onboard battery: charged, 12.6 Volts. Engine Temperature: elevated. Engine status: running. Exhaust emissions: normal.Suspension health: normal. Historical drive data: indicate sudden force and abrupt stop. Tire pressure: 35 psi.

[0090] Then, the first real-time wireless communication connection with the radio access network 102 may be used in order to execute the following: the vehicle teleoperation cloud 104 determines in block 208 a required level of teleoperation based on the vehicle and occupant damage assessment, the required level of teleoperation defining at least one technical system of a plurality of technical systems of the remote vehicle 100 that is to be teleoperated. With the vehicle and occupant damage assessment comprising the information outlined above, the vehicle teleoperation cloud 104 may determine the following examples based on the data:

[0091] The following paragraph only outlines example values for example features: the heart rate is elevated due to stress and injury and the low blood pressure indicates a shock or internal injuries. The irregular respiratory rate indicates a potential respiratory distress. The lower-than-normal skin temperature suggests a shock. The visible physical body damage potentially indicates an internal injury not visible externally. No measurable mood indicates unconsciousness. A high level of distress and a high level of stress indicate a traumatic event. The low oxygen saturation indicates insufficient oxygen in the blood, possibly due to lung injury or impaired breathing. The level of fatigue is presumed high prior to unconsciousness. The pupil is dilated as a response to shock and / or brain injury. The eyes may be closed when unconscious.2022PF02806 22 VALEO.224.09EP

[0092] The following paragraph only outlines example values for example features: the abnormal electroencephalogram indicates a traumatic brain injury. The irregular electrocardiogram shows signs of cardiac stress. The reduced Heart Rate Variability suggests higher stress and cardiac complications. Absent body motion indicates unconsciousness.Nonresponsive facial expressions indicate unconsciousness. No measurable voice indicates unconsciousness.

[0093] It may be determined that the driver of the remote vehicle 100 as the single occupant of the remote vehicle 100 could be unconscious and could have received traumatic injuries, potentially inflicting damage onto their cardiovascular system and brain. Thus, the vehicle teleoperation cloud 104 may have determined that for example, immediate and full assistance in controlling all of the remote vehicle's technical systems is necessary.

[0094] During the analysis of the vehicle data, it may be determined that the remote vehicle 100 itself is not on fire, but a fire may be ongoing in the vicinity. Smoke may be detected entering the cabin of the remote vehicle 100. As the remote vehicle 100 may still be upright, the tire pressure may be normal, and the fuel and oil levels could also be normal, it may be determined that the remote vehicle 100 may be still in driving condition. In general, for example, the vehicle teleoperation cloud 104 has determined that the car body deformation is not bad enough to prevent driving of the remote vehicle 100.

[0095] For example, the vehicle teleoperation cloud 104 determines, using a trained machine learning module, that the remote vehicle 100 must be driven away from the scene of the accident immediately. This may be in order to prevent it catching fire. For that, the vehicle teleoperation cloud 104 may need to take complete control of all the remote vehicle's technical systems, which may comprise the ignition system, steering system, acceleration system, braking system, gear selection system, lighting system, camera system, sensor system, emergency and safety system, infotainment system, diagnostic system, climate control system, power management system, in particular a power management system of an electric vehicle, navigation system, Vehicle-to-Vehicle, V2V, communication system, Vehicle-to-lnfrastructure, V2I, communication system, autonomous driving system, exhaust management system, suspension system, security and anti-theft system, active aerodynamics system, electric window system, door system, trunk lid and engine compartment system.

[0096] After the determining 208, the first real-time wireless communication connection with the radio access network 102 is used in order to execute the following: the remote vehicle's2022PF02806 23 VALEO.224.09EP technical systems are controlled in block 210 in real-time by the vehicle teleoperation cloud 104 via the first real-time wireless communication connection. During the controlling in block 210 it may be determined that the real-time presence of a teleoperator 108 is necessary in order to e.g., control the remote vehicle's technical system. As the teleoperator 108 may be located in a remote facility in communication with the vehicle teleoperation cloud 104, they may be provided with a real-time virtual-reality stream regarding the interior and exterior of the remote vehicle 100.

[0097] The virtual reality stream may comprise a live camera feed of the interior and exterior of the remote vehicle (100) in e.g., multiple camera positions and multiple camera angles. The teleoperator may be taking over real-time control of the remote vehicle 100 via means for controlling all of the technical systems of the remote vehicle 100, wherein the virtual reality stream could be streamed to a display device, in particular to a virtual reality headset with means for headtracking. Other possible display devices may be a monitor, a beamer, an augmented-reality-environment comprising head-mounted displays, a holographic display, a heads-up display (HUD), a flexible OLED display, a 3D projector, a 3D monitor, a multi-screen setup, or a transparent display.

[0098] In the example of figure 1, the teleoperator 108 may have full control of the remote vehicle 100 and may be virtually present in real-time in the remote vehicle 100 via the virtual reality stream and the virtual reality headset. The teleoperator 108 could thus e.g., fully control the remote vehicle 100 to carry out one of the following examples, wherein the list of examples does not exclude other examples not listed here and shall not be understood as limiting. For example, the teleoperator 108 controls the remote vehicle 100 to carry out one or more of the following:

[0099] driving the unconscious operator / remote vehicle away from the fire hazard, driving the unconscious operator to a hospital, driving the unconscious operator towards an arriving ambulance, providing on-demand vehicle diagnostics to emergency responders, enabling remote vehicle shutdown in case of imminent danger, guiding traffic management systems to clear paths for emergency access via vehicle-to-vehicle and vehicle-to-infrastructure communication,

[0100] initiating vehicle-to-vehicle communication for accident scene coordination, broadcasting emergency alerts to nearby vehicles and infrastructure, enabling remote fuel cut-off to prevent further hazard, facilitating data transmission to accident investigation teams, managing on-board2022PF02806 24 VALEO.224.09EP first-aid protocols via automated systems, ensuring optimal route selection for emergency evacuation,

[0101] activating hazard lights and emergency signals automatically, deploying additional airbag systems if needed, interfacing with smart city infrastructure to alert relevant authorities, monitoring environmental conditions continuously for safety updates, adjusting internal cabin conditions to preserve occupant health until help arrives, e.g., via the air conditioning system and / or seat heating functions, relaying critical health data from the occupant to incoming medical teams,

[0102] managing electronic stability control to maintain vehicle steadiness despite damage, initiating crowd-sourced accident reporting through connected services, providing real-time updates to the family or contacts of the occupant, integrating with weather forecasting systems to adapt responses based on conditions,

[0103] securing personal data contained within the vehicle's systems during emergencies, leveraging Al-driven predictive analytics to anticipate additional risks, coordinating with other automated vehicles in the vicinity to ensure a coherent response, enabling full remote access for law enforcement for investigation and response planning, optimizing traffic light sequences to expedite emergency vehicle access,

[0104] activating thermal imaging systems to detect and monitor fire spread, integrating with urban monitoring systems to provide real-time crisis mapping, facilitating remote override of vehicle functions for rescue operations, using blockchain to securely log all incident-related data for transparency, activating autonomous drone support for overhead traffic management and surveillance, configuring vehicle communication systems to serve as emergency broadcast units,

[0105] harnessing vehicle sensors to provide geological or chemical sensor data in case of hazardous material involvement, engaging telematics to reconstruct the incident timeline for investigators, initiating an automatic SOS signal to satellite services for global positioning and alerting, coordinating with utility services to manage potential infrastructure impacts, enabling live stream capabilities for immediate media coverage and public awareness, and lastly, utilizing augmented reality interfaces to provide emergency workers with interactive assistance.

[0106] Figure 3 displays an exemplary computer device 300 for teleoperation of a remote vehicle 100 by a vehicle teleoperation cloud 104 after an accident of the remote vehicle 100.2022PF02806 25 VALEO.224.09EP

[0107] The computer device 300 is equipped with computing and hardware interfaces for receiving a request for teleoperation of a remote vehicle 100, establishing a first real-time wireless communication connection with radio access network 102, RAN, receiving a vehicle and / or occupant damage assessment and for controlling, in real-time, at least one technical system of the remote vehicle 100. The computer device 300 is intended to represent one or more computing units, which may be distributed. The computer device 300 is shown to comprise a computing system 304. The computing system 304 is intended to represent one or more computing systems. The computer device 300 is further shown to include an optional hardware interface 306. The hardware interface may enable the computer device 300 to send and receive data from external components. The computer device 300 is further shown to be in communication with an optional user interface 308. The computer device 300 may also comprise, for example, a display device. This could include, for example, a two-dimensional computer display, a touch screen, a virtual reality system, and an augmented reality system, or it may be in a form that it provides a virtual reality stream comprising a live camera view to a virtual reality headset. The hardware interface may also offer means to connect a headtracking device.

[0108] The computer device 300 is further shown to be in communication with a memory 310. The memory 310 is intended to represent various types of memory that the computing system 304 may have access to. In one example, the memory 310 is a non-volatile storage medium.

[0109] The memory 310 is configured to contain machine-executable instructions 320. The machine-executable instructions 320 may enable the computing system 304 to perform various numerical and computational tasks. The machine-executable instructions 320 may also enable the computing system 304 to send and receive data from external components via the hardware interface 306. Execution of the machine-executable instructions 320 by the computing system 304 cause the computing system 304 to execute a method similar or equal to the method 200.

[0110] While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive; the invention is not limited to the disclosed examples.

[0111] As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as an apparatus, method, computer program or computer program product.Accordingly, aspects of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, microcode, etc.) or an embodiment combining software and hardware aspects that may all generally2022PF02806 26 VALEO.224.09EP be referred to herein as a "circuit," "module" or "system." Furthermore, aspects of the present invention may take the form of a computer program product embodied in one or more computer readable medium(s) having computer executable code embodied thereon. A computer program comprises the computer executable code or "program instructions".

[0112] Any combination of one or more computer readable medium(s) may be utilized. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A 'computer-readable storage medium' as used herein encompasses any tangible storage medium that may store instructions which are executable by a processor of a computing device. The computer-readable storage medium may be referred to as a computer-readable non-transitory storage medium. The computer-readable storage medium may also be referred to as a tangible computer readable medium. In some embodiments, a computer-readable storage medium may also be able to store data that is able to be accessed by the processor of the computing device. Examples of computer-readable storage media include, but are not limited to: a floppy disk, a magnetic hard disk drive, a solid state hard disk, flash memory, a USB thumb drive, Random Access Memory (RAM), Read Only Memory (ROM), an optical disk, a magneto-optical disk, and the register file of the processor. Examples of optical disks include Compact Disks (CD) and Digital Versatile Disks (DVD), for example CD-ROM, CD-RW, CD-R, DVD-ROM, DVD-RW, or DVD-R disks. The term computer readable-storage medium also refers to various types of recording media capable of being accessed by the computer device via a network or communication link. For example, a data may be retrieved over a modem, over the internet, or over a local area network. Computer executable code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0113] A computer readable signal medium may include a propagated data signal with computer executable code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including, but not limited to, electromagnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

[0114] 'Computer memory' or 'memory' is an example of a computer-readable storage medium. Computer memory is any memory which is directly accessible to a processor. 'Computer storage' or 'storage' is a further example of a computer-readable storage medium. Computer storage is2022PF02806 T1 VALEO.224.09EP any non-volatile computer-readable storage medium. In some embodiments, computer storage may also be computer memory or vice versa.

[0115] A 'processor' as used herein encompasses an electronic component which is able to execute a program or machine executable instruction or computer executable code. References to the computing device comprising "a processor" should be interpreted as possibly containing more than one processor or processing core. The processor may for instance be a multi-core processor. A processor may also refer to a collection of processors within a single computer system or distributed amongst multiple computer systems. The term computing device should also be interpreted to possibly refer to a collection or network of computing devices each comprising a processor or processors. The computer executable code may be executed by multiple processors that may be within the same computing device or which may even be distributed across multiple computing devices.

[0116] Computer executable code may comprise machine executable instructions or a program which causes a processor to perform an aspect of the present invention. Computer executable code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages and compiled into machine executable instructions. In some instances the computer executable code may be in the form of a high level language or in a pre-compiled form and be used in conjunction with an interpreter which generates the machine executable instructions on the fly.

[0117] The computer executable code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0118] Generally, the program instructions can be executed on one processor or on several processors. In the case of multiple processors, they can be distributed over several different entities like clients, servers etc. Each processor could execute a portion of the instructions intended for that entity. Thus, when referring to a system or process involving multiple entities,2022PF02806 28 VALEO.224.09EP the computer program or program instructions are understood to be adapted to be executed by a processor associated or related to the respective entity.

[0119] Aspects of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block or a portion of the blocks of the flowchart, illustrations, and / or block diagrams, can be implemented by computer program instructions in form of computer executable code when applicable. It is further under stood that, when not mutually exclusive, combinations of blocks in different flowcharts, illustrations, and / or block diagrams may be combined. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0120] These computer program instructions may also be stored in a computer readable medium that can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer readable medium produce an article of manufacture including instructions which implement the function / act specified in the flowchart and / or block diagram block or blocks.

[0121] The computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.2022PF02806 29 VALEO.224.09EPREFERENCE SIGNS LIST100 remote vehicle102 first real-time wireless communication connection with a radio access network, RAN 104 vehicle teleoperation cloud106 other vehicle, on fire108 teleoperator200 method202 Receiving a request for teleoperation of a remote vehicle204 Using a first real-time wireless communication connection206 Receiving a vehicle and occupant damage assessment208 Determining a required level of teleoperation210 Controlling the at least one technical system of the remote vehicle300 computer device304 computing system306 hardware interface308 user interface310 memory320 machine-executable instructions

Claims

2022PF02806 30 VALEO.224.09EPCLAIMS1. A computer-implemented method for teleoperation of a remote vehicle (100) by a vehicle teleoperation cloud (104) after an accident of the remote vehicle (100), the method comprising:receiving (202), by the vehicle teleoperation cloud (104), a request for teleoperation of the remote vehicle (100) involved in the accident via a first real-time wireless communication connection or a second network connection;in case the first real-time wireless communication connection is not established and the request for teleoperation is received via the second network connection, establishing the first real-time wireless communication connection between the vehicle teleoperation cloud (104) and the remote vehicle (100);using (204), in response to the request, the first real-time wireless communication connection in order to:receive (206), by the vehicle teleoperation cloud (104), a vehicle and occupant damage assessment;determine (208), by the vehicle teleoperation cloud (104), a required level of teleoperation based on the vehicle and occupant damage assessment, the required level of teleoperation defining at least one technical system of a plurality of technical systems of the remote vehicle (100) that is to be teleoperated;control (210), in real-time, by the vehicle teleoperation cloud (104) via the first real-time wireless communication connection, the at least one technical system of the remote vehicle (100).

2. The computer-implemented method of claim 1, wherein the second network connection is a network connection between the remote vehicle (100) and the vehicle teleoperation cloud configured to allow data flow from the remote vehicle (100) to the vehicle teleoperation cloud (104) and / or wherein the first real-time wireless communication connection is a connection through a network, the network being any one of:a. a radio access network (102), RAN, in accordance with a radio access technology, RAT, communication protocol comprising at least network slicing or another radio access technology in particular with a transfer speed of at least 2 Gbps and / or in particular with a latency of less than 15 ms;b. a satellite network; orc. a vehicle 2 infrastructure, V2I, network.

3. The computer-implemented method of any of the previous claims, wherein the vehicle and occupant damage assessment comprises vital data of one or more occupants of the remote vehicle (100) and vehicle data being descriptive of a physical status of the remote vehicle (100) and its surrounding environment.2022PF02806 31 VALEO.224.09EP 4. The computer-implemented method of claim 3, wherein the vital data comprises information on at least any one of the following: heart rate, blood pressure, respiratory rate, skin temperature, physical body damage, mood, level of distress, oxygen saturation, level of fatigue, level of stress, pupil diameter, contactless electroencephalogram, contactless electrocardiogram, heart rate variability, body motion, facial expressions, voice analysis, and / or wherein the vehicle data comprises information on at least any one of the following: vehicle orientation, fire detection, smoke detection, broken glass detection, airbag status, car body deformation detection, air quality, humidity levels, light exposure, ambient noise levels, environmental temperature, oil level, fuel level, battery status, engine temperature, engine status, exhaust emissions, suspension health, historical drive data, tire pressure.

5. The computer-implemented method of any of the previous claims, wherein the vehicle teleoperation cloud (104) further comprises at least one trained machine learning module, the at least one machine learning module being configured to determine the at least one technical system as output in response to receiving the vehicle and occupant damage assessment as input.

6. The computer-implemented method of any of the previous claims, wherein the plurality of technical systems comprises at least any one of the following systems: ignition system, steering system, acceleration system, braking system, gear selection system, lighting system, camera system, sensor system, emergency and safety system, infotainment system, diagnostic system, climate control system, power management system, in particular a power management system of an electric vehicle, navigation system, Vehicle- to-Vehicle, V2V, communication system, Vehicle-to-lnfrastructure, V2I, communication system, autonomous driving system, exhaust management system, suspension system, security and anti-theft system, active aerodynamics system, electric window system, door system, trunk lid and engine compartment system.

7. The computer-implemented method of any of the previous claims, further comprising a remote facility in communication with the vehicle teleoperation cloud (104), the controlling comprising:determining based on the vehicle and occupant damage assessment, by the vehicle teleoperation cloud (104), if the required level of teleoperation comprises a real-time presence of a teleoperator (108);in case its determined that the real-time presence of the teleoperator (108) is necessary:providing a real-time virtual-reality stream regarding the interior and exterior of the remote vehicle (100) to the teleoperator (108); orsending a request, to the teleoperator (108), to establish a real-time virtual- reality stream regarding the interior and exterior of the remote vehicle (100).

8. The computer-implemented method of claim 7, wherein the virtual reality stream comprises a live camera feed of the interior and exterior of the remote vehicle (100) in2022PF02806 32 VALEO.224.09EP multiple camera positions and multiple camera angles, wherein the teleoperator (108) is taking over real-time control of the remote vehicle (100) via means for controlling the plurality of technical systems of the remote vehicle (100), wherein the virtual reality stream is streamed to a display device, in particular to a virtual reality headset with means for headtracking.

9. A computer program for teleoperation of a remote vehicle (100) by a vehicle teleoperation cloud (104) after an accident of the remote vehicle (100), the computer program comprising program instructions, the program instructions being executable by a processor of a computer device to cause the computer device to:receive (202), by the vehicle teleoperation cloud (104), a request for teleoperation of the remote vehicle (100) involved in the accident via a first real-time wireless communication connection or a second network connection;in case the first real-time wireless communication connection is not established and the request for teleoperation is received via the second network connection, establishing the first real-time wireless communication connection between the vehicle teleoperation cloud (104) and the remote vehicle (100);use (204), in response to the request, the first real-time wireless communication connection in order to:receive (206), by the vehicle teleoperation cloud (104), a vehicle and occupant damage assessment;determine (208), by the vehicle teleoperation cloud (104), a required level of teleoperation based on the vehicle and occupant damage assessment, the required level of teleoperation defining at least one technical system of a plurality of technical systems of the remote vehicle (100) that is to be teleoperated;control (210), in real-time, by the vehicle teleoperation cloud (104) via the first real-time wireless communication connection, the at least one technical system of the remote vehicle (100).

10. A computer device (300) for teleoperation of a remote vehicle (100) by a vehicle teleoperation cloud (104) after an accident of the remote vehicle (100), the computer device comprising a processor and a memory storing program instructions executable by the processor, execution of the program instructions by the processor causing the computer device to:receive (202), by the vehicle teleoperation cloud (104), a request for teleoperation of the remote vehicle (100) involved in the accident via a first real-time wireless communication connection or a second network connection;in case the first real-time wireless communication connection is not established and the request for teleoperation is received via the second network connection,2022PF02806 33 VALEO.224.09EP establishing the first real-time wireless communication connection between the vehicle teleoperation cloud (104) and the remote vehicle (100);use (204), in response to the request, the first real-time wireless communication connection in order to:receive (206), by the vehicle teleoperation cloud (104), a vehicle and occupant damage assessment;determine (208), by the vehicle teleoperation cloud (104), a required level of teleoperation based on the vehicle and occupant damage assessment, the required level of teleoperation defining at least one technical system of a plurality of technical systems of the remote vehicle (100) that is to be teleoperated;control (210), in real-time, by the vehicle teleoperation cloud (104) via the first real-time wireless communication connection, the at least one technical system of the remote vehicle (100).

Images